├── .gitattributes ├── .gitignore ├── LICENSE ├── README.md ├── data ├── caltech │ └── test.txt └── citypersons │ └── val.txt ├── eval_caltech ├── ResultsEval │ └── gt-new │ │ └── gt-Reasonable.mat ├── bbGt.m └── dbEval.m ├── eval_city ├── dt_txt2json.py ├── eval_script │ ├── coco.py │ ├── eval_MR_multisetup.py │ ├── eval_demo.py │ └── readme.txt └── val_gt.json ├── models ├── cspnet.py ├── l2norm.py └── samepad.py ├── shibuye.jpg ├── test_caltech.py ├── test_city.py ├── test_one_pic_face.py ├── test_one_pic_pedestrian.py └── utils ├── __init__.py ├── keras_weights_loader.py ├── py_cpu_nms.py └── utils.py /.gitattributes: -------------------------------------------------------------------------------- 1 | *.m linguist-language=python 2 | *.mat linguist-language=python 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | *.hdf5 3 | log* 4 | *.jpg 5 | eval_caltech/toolbox 6 | eval_caltech/ResultsEval 7 | eval_caltech/AS.mat 8 | output 9 | data 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Wang Xinyu 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CSP 2 | 3 | A pytorch implementation of CSP, `Wei Liu, etc. High-level Semantic Feature Detection: A New Perspective for Pedestrian Detection, CVPR 2019.` 4 | 5 | The authors' Keras implementation [liuwei16/CSP](https://github.com/liuwei16/CSP). 6 | 7 | ## Updates 8 | 9 | - 20 March 2020, Load keras weights into pytorch model, and got the same model outputs as `resnet50 nn_p3p4p5` in [liuwei16/CSP/resnet50.py](https://github.com/liuwei16/CSP/blob/785bc4c5f956860116d8d51754fd76202afe4bcb/keras_csp/resnet50.py#L264) 10 | - 23 March 2020, finish test_caltech, got 4.06 MR under Reasonable setting using authors trained weights. 11 | - 24 March 2020, finish test_city, got 11.02 MR under Reasonable setting using authors trained weights. 12 | - 27 March 2020, finish test_one_pic_face, load wider face trained model and test face detection on one pic. 13 | 14 | ## Environments 15 | 16 | - ubuntu16.04 17 | - cuda10.0 18 | - python3.6 19 | - pytorch1.4.0 20 | 21 | ## Test One Picture Pedestrian 22 | 23 | Download trained weights from Wei Liu's baidu netdisk [BaiduYun](https://pan.baidu.com/s/1SSPQnbDP6zf9xf8eCDi3Fw) (Code: jcgd). 24 | 25 | For this test, we are using ResNet-50 initialized from CityPersons: Height+Offset prediciton: model_CSP/caltech/fromcity/net_e82_l0.00850005054218.hdf5 26 | 27 | We'll use this keras hdf5 weights, and load it in pytorch, and run inference. 28 | 29 | ``` 30 | git clone https://github.com/wang-xinyu/csp.pytorch 31 | cd csp.pytorch 32 | // put net_e82_l0.00850005054218.hdf5 here 33 | python test_one_pic_pedestrian.py 34 | ``` 35 | 36 | The out.jpg as follows will be generated. 37 | 38 |

39 | 40 |

41 | 42 | ## Eval Caltech 43 | 44 | We still use net_e82_l0.00850005054218.hdf5 above. 45 | 46 | Put caltech test images(4024 images) in `data/caltech/images/`. The dir tree is as below. 47 | 48 | ``` 49 | data 50 | caltech 51 | test.txt 52 | images 53 | set06_V000_I00029.jpg 54 | set06_V000_I00059.jpg 55 | ... 56 | set10_V011_I01709.jpg 57 | ``` 58 | 59 | Get `AS.mat` from [liuwei16/CSP](https://github.com/liuwei16/CSP/blob/master/eval_caltech/AS.mat), put into `eval_caltech/` 60 | 61 | Then run test_caltech.py to generate results, and use octave to run eval and get MR. 62 | 63 | ``` 64 | python test_caltech.py 65 | cd eval_caltech/ 66 | git clone https://github.com/pdollar/toolbox.git 67 | sudo apt install octave 68 | octave-cli 69 | -> dbEval 70 | ``` 71 | 72 | A result file `eval_caltech/ResultsEval/eval-newReasonable.txt` will generated. 73 | 74 | ## Eval CityPersons 75 | 76 | For this test, we are using Height+Offset prediction: model_CSP/cityperson/withoffset/net_e121_l0.hdf5, download it from the baidu link above. 77 | 78 | Put citypersons images in `data/citypersons/`. The dir tree is as below. 79 | 80 | ``` 81 | data 82 | citypersons 83 | leftImg8bit 84 | train 85 | aachen 86 | bochum 87 | ... 88 | zurich 89 | val 90 | frankfurt 91 | lindau 92 | munster 93 | ``` 94 | 95 | Then run test_city.py to generate results, and run eval and get MR. 96 | 97 | ``` 98 | python test_city.py 99 | cd eval_city/ 100 | python dt_txt2json.py 101 | cd eval_script/ 102 | python eval_demo.py 103 | ``` 104 | 105 | ## Test One Picture Face 106 | 107 | Download trained weights from Wei Liu's baidu netdisk [BaiduYun](https://pan.baidu.com/s/1SSPQnbDP6zf9xf8eCDi3Fw) (Code: jcgd). 108 | 109 | For this test, we are using ResNet-50 Height+Width+Offset prediction: model_CSP/widerface/net_e382_l0.hdf5 110 | We'll use this keras hdf5 weights, and load it in pytorch, and run inference. 111 | 112 | ``` 113 | cd csp.pytorch 114 | // put net_e382_l0.hdf5 here 115 | wget https://github.com/TencentYoutuResearch/FaceDetection-DSFD/raw/master/data/worlds-largest-selfie.jpg 116 | python test_one_pic_face.py 117 | ``` 118 | 119 | The out.jpg as follows will be generated. 120 | 121 |

122 | 123 |

124 | 125 | 126 | 127 | 128 | -------------------------------------------------------------------------------- /data/citypersons/val.txt: -------------------------------------------------------------------------------- 1 | frankfurt/frankfurt_000000_000294_leftImg8bit.png 2 | frankfurt/frankfurt_000000_000576_leftImg8bit.png 3 | frankfurt/frankfurt_000000_001016_leftImg8bit.png 4 | frankfurt/frankfurt_000000_001236_leftImg8bit.png 5 | frankfurt/frankfurt_000000_001751_leftImg8bit.png 6 | frankfurt/frankfurt_000000_002196_leftImg8bit.png 7 | frankfurt/frankfurt_000000_002963_leftImg8bit.png 8 | frankfurt/frankfurt_000000_003025_leftImg8bit.png 9 | frankfurt/frankfurt_000000_003357_leftImg8bit.png 10 | frankfurt/frankfurt_000000_003920_leftImg8bit.png 11 | frankfurt/frankfurt_000000_004617_leftImg8bit.png 12 | frankfurt/frankfurt_000000_005543_leftImg8bit.png 13 | frankfurt/frankfurt_000000_005898_leftImg8bit.png 14 | frankfurt/frankfurt_000000_006589_leftImg8bit.png 15 | frankfurt/frankfurt_000000_007365_leftImg8bit.png 16 | frankfurt/frankfurt_000000_008206_leftImg8bit.png 17 | frankfurt/frankfurt_000000_008451_leftImg8bit.png 18 | frankfurt/frankfurt_000000_009291_leftImg8bit.png 19 | frankfurt/frankfurt_000000_009561_leftImg8bit.png 20 | frankfurt/frankfurt_000000_009688_leftImg8bit.png 21 | frankfurt/frankfurt_000000_009969_leftImg8bit.png 22 | frankfurt/frankfurt_000000_010351_leftImg8bit.png 23 | frankfurt/frankfurt_000000_010763_leftImg8bit.png 24 | frankfurt/frankfurt_000000_011007_leftImg8bit.png 25 | frankfurt/frankfurt_000000_011074_leftImg8bit.png 26 | frankfurt/frankfurt_000000_011461_leftImg8bit.png 27 | frankfurt/frankfurt_000000_011810_leftImg8bit.png 28 | frankfurt/frankfurt_000000_012009_leftImg8bit.png 29 | frankfurt/frankfurt_000000_012121_leftImg8bit.png 30 | frankfurt/frankfurt_000000_012868_leftImg8bit.png 31 | frankfurt/frankfurt_000000_013067_leftImg8bit.png 32 | frankfurt/frankfurt_000000_013240_leftImg8bit.png 33 | frankfurt/frankfurt_000000_013382_leftImg8bit.png 34 | frankfurt/frankfurt_000000_013942_leftImg8bit.png 35 | frankfurt/frankfurt_000000_014480_leftImg8bit.png 36 | frankfurt/frankfurt_000000_015389_leftImg8bit.png 37 | frankfurt/frankfurt_000000_015676_leftImg8bit.png 38 | frankfurt/frankfurt_000000_016005_leftImg8bit.png 39 | frankfurt/frankfurt_000000_016286_leftImg8bit.png 40 | frankfurt/frankfurt_000000_017228_leftImg8bit.png 41 | frankfurt/frankfurt_000000_017476_leftImg8bit.png 42 | frankfurt/frankfurt_000000_018797_leftImg8bit.png 43 | frankfurt/frankfurt_000000_019607_leftImg8bit.png 44 | frankfurt/frankfurt_000000_020215_leftImg8bit.png 45 | frankfurt/frankfurt_000000_020321_leftImg8bit.png 46 | frankfurt/frankfurt_000000_020880_leftImg8bit.png 47 | frankfurt/frankfurt_000000_021667_leftImg8bit.png 48 | frankfurt/frankfurt_000000_021879_leftImg8bit.png 49 | frankfurt/frankfurt_000000_022254_leftImg8bit.png 50 | frankfurt/frankfurt_000000_022797_leftImg8bit.png 51 | frankfurt/frankfurt_000001_000538_leftImg8bit.png 52 | frankfurt/frankfurt_000001_001464_leftImg8bit.png 53 | frankfurt/frankfurt_000001_002512_leftImg8bit.png 54 | frankfurt/frankfurt_000001_002646_leftImg8bit.png 55 | frankfurt/frankfurt_000001_002759_leftImg8bit.png 56 | frankfurt/frankfurt_000001_003056_leftImg8bit.png 57 | frankfurt/frankfurt_000001_003588_leftImg8bit.png 58 | frankfurt/frankfurt_000001_004327_leftImg8bit.png 59 | frankfurt/frankfurt_000001_004736_leftImg8bit.png 60 | frankfurt/frankfurt_000001_004859_leftImg8bit.png 61 | frankfurt/frankfurt_000001_005184_leftImg8bit.png 62 | frankfurt/frankfurt_000001_005410_leftImg8bit.png 63 | frankfurt/frankfurt_000001_005703_leftImg8bit.png 64 | frankfurt/frankfurt_000001_005898_leftImg8bit.png 65 | frankfurt/frankfurt_000001_007285_leftImg8bit.png 66 | frankfurt/frankfurt_000001_007407_leftImg8bit.png 67 | frankfurt/frankfurt_000001_007622_leftImg8bit.png 68 | frankfurt/frankfurt_000001_007857_leftImg8bit.png 69 | frankfurt/frankfurt_000001_007973_leftImg8bit.png 70 | frankfurt/frankfurt_000001_008200_leftImg8bit.png 71 | frankfurt/frankfurt_000001_008688_leftImg8bit.png 72 | frankfurt/frankfurt_000001_009058_leftImg8bit.png 73 | frankfurt/frankfurt_000001_009504_leftImg8bit.png 74 | frankfurt/frankfurt_000001_009854_leftImg8bit.png 75 | frankfurt/frankfurt_000001_010156_leftImg8bit.png 76 | frankfurt/frankfurt_000001_010444_leftImg8bit.png 77 | frankfurt/frankfurt_000001_010600_leftImg8bit.png 78 | frankfurt/frankfurt_000001_010830_leftImg8bit.png 79 | frankfurt/frankfurt_000001_011162_leftImg8bit.png 80 | frankfurt/frankfurt_000001_011715_leftImg8bit.png 81 | frankfurt/frankfurt_000001_011835_leftImg8bit.png 82 | frankfurt/frankfurt_000001_012038_leftImg8bit.png 83 | frankfurt/frankfurt_000001_012519_leftImg8bit.png 84 | frankfurt/frankfurt_000001_012699_leftImg8bit.png 85 | frankfurt/frankfurt_000001_012738_leftImg8bit.png 86 | frankfurt/frankfurt_000001_012870_leftImg8bit.png 87 | frankfurt/frankfurt_000001_013016_leftImg8bit.png 88 | frankfurt/frankfurt_000001_013496_leftImg8bit.png 89 | frankfurt/frankfurt_000001_013710_leftImg8bit.png 90 | frankfurt/frankfurt_000001_014221_leftImg8bit.png 91 | frankfurt/frankfurt_000001_014406_leftImg8bit.png 92 | frankfurt/frankfurt_000001_014565_leftImg8bit.png 93 | frankfurt/frankfurt_000001_014741_leftImg8bit.png 94 | frankfurt/frankfurt_000001_015091_leftImg8bit.png 95 | frankfurt/frankfurt_000001_015328_leftImg8bit.png 96 | frankfurt/frankfurt_000001_015768_leftImg8bit.png 97 | frankfurt/frankfurt_000001_016029_leftImg8bit.png 98 | frankfurt/frankfurt_000001_016273_leftImg8bit.png 99 | frankfurt/frankfurt_000001_016462_leftImg8bit.png 100 | frankfurt/frankfurt_000001_017101_leftImg8bit.png 101 | frankfurt/frankfurt_000001_017459_leftImg8bit.png 102 | frankfurt/frankfurt_000001_017842_leftImg8bit.png 103 | frankfurt/frankfurt_000001_018113_leftImg8bit.png 104 | frankfurt/frankfurt_000001_019698_leftImg8bit.png 105 | frankfurt/frankfurt_000001_019854_leftImg8bit.png 106 | frankfurt/frankfurt_000001_019969_leftImg8bit.png 107 | frankfurt/frankfurt_000001_020046_leftImg8bit.png 108 | frankfurt/frankfurt_000001_020287_leftImg8bit.png 109 | frankfurt/frankfurt_000001_020693_leftImg8bit.png 110 | frankfurt/frankfurt_000001_021406_leftImg8bit.png 111 | frankfurt/frankfurt_000001_021825_leftImg8bit.png 112 | frankfurt/frankfurt_000001_023235_leftImg8bit.png 113 | frankfurt/frankfurt_000001_023369_leftImg8bit.png 114 | frankfurt/frankfurt_000001_023769_leftImg8bit.png 115 | frankfurt/frankfurt_000001_024927_leftImg8bit.png 116 | frankfurt/frankfurt_000001_025512_leftImg8bit.png 117 | frankfurt/frankfurt_000001_025713_leftImg8bit.png 118 | frankfurt/frankfurt_000001_025921_leftImg8bit.png 119 | frankfurt/frankfurt_000001_027325_leftImg8bit.png 120 | frankfurt/frankfurt_000001_028232_leftImg8bit.png 121 | frankfurt/frankfurt_000001_028335_leftImg8bit.png 122 | frankfurt/frankfurt_000001_028590_leftImg8bit.png 123 | frankfurt/frankfurt_000001_028854_leftImg8bit.png 124 | frankfurt/frankfurt_000001_029086_leftImg8bit.png 125 | frankfurt/frankfurt_000001_029236_leftImg8bit.png 126 | frankfurt/frankfurt_000001_029600_leftImg8bit.png 127 | frankfurt/frankfurt_000001_030067_leftImg8bit.png 128 | frankfurt/frankfurt_000001_030310_leftImg8bit.png 129 | frankfurt/frankfurt_000001_030669_leftImg8bit.png 130 | frankfurt/frankfurt_000001_031266_leftImg8bit.png 131 | frankfurt/frankfurt_000001_031416_leftImg8bit.png 132 | frankfurt/frankfurt_000001_032018_leftImg8bit.png 133 | frankfurt/frankfurt_000001_032556_leftImg8bit.png 134 | frankfurt/frankfurt_000001_032711_leftImg8bit.png 135 | frankfurt/frankfurt_000001_032942_leftImg8bit.png 136 | frankfurt/frankfurt_000001_033655_leftImg8bit.png 137 | frankfurt/frankfurt_000001_034047_leftImg8bit.png 138 | frankfurt/frankfurt_000001_034816_leftImg8bit.png 139 | frankfurt/frankfurt_000001_035144_leftImg8bit.png 140 | frankfurt/frankfurt_000001_035864_leftImg8bit.png 141 | frankfurt/frankfurt_000001_037705_leftImg8bit.png 142 | frankfurt/frankfurt_000001_038245_leftImg8bit.png 143 | frankfurt/frankfurt_000001_038418_leftImg8bit.png 144 | frankfurt/frankfurt_000001_038645_leftImg8bit.png 145 | frankfurt/frankfurt_000001_038844_leftImg8bit.png 146 | frankfurt/frankfurt_000001_039895_leftImg8bit.png 147 | frankfurt/frankfurt_000001_040575_leftImg8bit.png 148 | frankfurt/frankfurt_000001_040732_leftImg8bit.png 149 | frankfurt/frankfurt_000001_041074_leftImg8bit.png 150 | frankfurt/frankfurt_000001_041354_leftImg8bit.png 151 | frankfurt/frankfurt_000001_041517_leftImg8bit.png 152 | frankfurt/frankfurt_000001_041664_leftImg8bit.png 153 | frankfurt/frankfurt_000001_042098_leftImg8bit.png 154 | frankfurt/frankfurt_000001_042384_leftImg8bit.png 155 | frankfurt/frankfurt_000001_042733_leftImg8bit.png 156 | frankfurt/frankfurt_000001_043395_leftImg8bit.png 157 | frankfurt/frankfurt_000001_043564_leftImg8bit.png 158 | frankfurt/frankfurt_000001_044227_leftImg8bit.png 159 | frankfurt/frankfurt_000001_044413_leftImg8bit.png 160 | frankfurt/frankfurt_000001_044525_leftImg8bit.png 161 | frankfurt/frankfurt_000001_044658_leftImg8bit.png 162 | frankfurt/frankfurt_000001_044787_leftImg8bit.png 163 | frankfurt/frankfurt_000001_046126_leftImg8bit.png 164 | frankfurt/frankfurt_000001_046272_leftImg8bit.png 165 | frankfurt/frankfurt_000001_046504_leftImg8bit.png 166 | frankfurt/frankfurt_000001_046779_leftImg8bit.png 167 | frankfurt/frankfurt_000001_047178_leftImg8bit.png 168 | frankfurt/frankfurt_000001_047552_leftImg8bit.png 169 | frankfurt/frankfurt_000001_048196_leftImg8bit.png 170 | frankfurt/frankfurt_000001_048355_leftImg8bit.png 171 | frankfurt/frankfurt_000001_048654_leftImg8bit.png 172 | frankfurt/frankfurt_000001_049078_leftImg8bit.png 173 | frankfurt/frankfurt_000001_049209_leftImg8bit.png 174 | frankfurt/frankfurt_000001_049298_leftImg8bit.png 175 | frankfurt/frankfurt_000001_049698_leftImg8bit.png 176 | frankfurt/frankfurt_000001_049770_leftImg8bit.png 177 | frankfurt/frankfurt_000001_050149_leftImg8bit.png 178 | frankfurt/frankfurt_000001_050686_leftImg8bit.png 179 | frankfurt/frankfurt_000001_051516_leftImg8bit.png 180 | frankfurt/frankfurt_000001_051737_leftImg8bit.png 181 | frankfurt/frankfurt_000001_051807_leftImg8bit.png 182 | frankfurt/frankfurt_000001_052120_leftImg8bit.png 183 | frankfurt/frankfurt_000001_052594_leftImg8bit.png 184 | frankfurt/frankfurt_000001_053102_leftImg8bit.png 185 | frankfurt/frankfurt_000001_054077_leftImg8bit.png 186 | frankfurt/frankfurt_000001_054219_leftImg8bit.png 187 | frankfurt/frankfurt_000001_054415_leftImg8bit.png 188 | frankfurt/frankfurt_000001_054640_leftImg8bit.png 189 | frankfurt/frankfurt_000001_054884_leftImg8bit.png 190 | frankfurt/frankfurt_000001_055062_leftImg8bit.png 191 | frankfurt/frankfurt_000001_055172_leftImg8bit.png 192 | frankfurt/frankfurt_000001_055306_leftImg8bit.png 193 | frankfurt/frankfurt_000001_055387_leftImg8bit.png 194 | frankfurt/frankfurt_000001_055538_leftImg8bit.png 195 | frankfurt/frankfurt_000001_055603_leftImg8bit.png 196 | frankfurt/frankfurt_000001_055709_leftImg8bit.png 197 | frankfurt/frankfurt_000001_056580_leftImg8bit.png 198 | frankfurt/frankfurt_000001_057181_leftImg8bit.png 199 | frankfurt/frankfurt_000001_057478_leftImg8bit.png 200 | frankfurt/frankfurt_000001_057954_leftImg8bit.png 201 | frankfurt/frankfurt_000001_058057_leftImg8bit.png 202 | frankfurt/frankfurt_000001_058176_leftImg8bit.png 203 | frankfurt/frankfurt_000001_058504_leftImg8bit.png 204 | frankfurt/frankfurt_000001_058914_leftImg8bit.png 205 | frankfurt/frankfurt_000001_059119_leftImg8bit.png 206 | frankfurt/frankfurt_000001_059642_leftImg8bit.png 207 | frankfurt/frankfurt_000001_059789_leftImg8bit.png 208 | frankfurt/frankfurt_000001_060135_leftImg8bit.png 209 | frankfurt/frankfurt_000001_060422_leftImg8bit.png 210 | frankfurt/frankfurt_000001_060545_leftImg8bit.png 211 | frankfurt/frankfurt_000001_060906_leftImg8bit.png 212 | frankfurt/frankfurt_000001_061682_leftImg8bit.png 213 | frankfurt/frankfurt_000001_061763_leftImg8bit.png 214 | frankfurt/frankfurt_000001_062016_leftImg8bit.png 215 | frankfurt/frankfurt_000001_062250_leftImg8bit.png 216 | frankfurt/frankfurt_000001_062396_leftImg8bit.png 217 | frankfurt/frankfurt_000001_062509_leftImg8bit.png 218 | frankfurt/frankfurt_000001_062653_leftImg8bit.png 219 | frankfurt/frankfurt_000001_062793_leftImg8bit.png 220 | frankfurt/frankfurt_000001_063045_leftImg8bit.png 221 | frankfurt/frankfurt_000001_064130_leftImg8bit.png 222 | frankfurt/frankfurt_000001_064305_leftImg8bit.png 223 | frankfurt/frankfurt_000001_064651_leftImg8bit.png 224 | frankfurt/frankfurt_000001_064798_leftImg8bit.png 225 | frankfurt/frankfurt_000001_064925_leftImg8bit.png 226 | frankfurt/frankfurt_000001_065160_leftImg8bit.png 227 | frankfurt/frankfurt_000001_065617_leftImg8bit.png 228 | frankfurt/frankfurt_000001_065850_leftImg8bit.png 229 | frankfurt/frankfurt_000001_066092_leftImg8bit.png 230 | frankfurt/frankfurt_000001_066438_leftImg8bit.png 231 | frankfurt/frankfurt_000001_066574_leftImg8bit.png 232 | frankfurt/frankfurt_000001_066832_leftImg8bit.png 233 | frankfurt/frankfurt_000001_067092_leftImg8bit.png 234 | frankfurt/frankfurt_000001_067178_leftImg8bit.png 235 | frankfurt/frankfurt_000001_067295_leftImg8bit.png 236 | frankfurt/frankfurt_000001_067474_leftImg8bit.png 237 | frankfurt/frankfurt_000001_067735_leftImg8bit.png 238 | frankfurt/frankfurt_000001_068063_leftImg8bit.png 239 | frankfurt/frankfurt_000001_068208_leftImg8bit.png 240 | frankfurt/frankfurt_000001_068682_leftImg8bit.png 241 | frankfurt/frankfurt_000001_068772_leftImg8bit.png 242 | frankfurt/frankfurt_000001_069633_leftImg8bit.png 243 | frankfurt/frankfurt_000001_070099_leftImg8bit.png 244 | frankfurt/frankfurt_000001_071288_leftImg8bit.png 245 | frankfurt/frankfurt_000001_071781_leftImg8bit.png 246 | frankfurt/frankfurt_000001_072155_leftImg8bit.png 247 | frankfurt/frankfurt_000001_072295_leftImg8bit.png 248 | frankfurt/frankfurt_000001_073088_leftImg8bit.png 249 | frankfurt/frankfurt_000001_073243_leftImg8bit.png 250 | frankfurt/frankfurt_000001_073464_leftImg8bit.png 251 | frankfurt/frankfurt_000001_073911_leftImg8bit.png 252 | frankfurt/frankfurt_000001_075296_leftImg8bit.png 253 | frankfurt/frankfurt_000001_075984_leftImg8bit.png 254 | frankfurt/frankfurt_000001_076502_leftImg8bit.png 255 | frankfurt/frankfurt_000001_077092_leftImg8bit.png 256 | frankfurt/frankfurt_000001_077233_leftImg8bit.png 257 | frankfurt/frankfurt_000001_077434_leftImg8bit.png 258 | frankfurt/frankfurt_000001_078803_leftImg8bit.png 259 | frankfurt/frankfurt_000001_079206_leftImg8bit.png 260 | frankfurt/frankfurt_000001_080091_leftImg8bit.png 261 | frankfurt/frankfurt_000001_080391_leftImg8bit.png 262 | frankfurt/frankfurt_000001_080830_leftImg8bit.png 263 | frankfurt/frankfurt_000001_082087_leftImg8bit.png 264 | frankfurt/frankfurt_000001_082466_leftImg8bit.png 265 | frankfurt/frankfurt_000001_083029_leftImg8bit.png 266 | frankfurt/frankfurt_000001_083199_leftImg8bit.png 267 | frankfurt/frankfurt_000001_083852_leftImg8bit.png 268 | lindau/lindau_000000_000019_leftImg8bit.png 269 | lindau/lindau_000001_000019_leftImg8bit.png 270 | lindau/lindau_000002_000019_leftImg8bit.png 271 | lindau/lindau_000003_000019_leftImg8bit.png 272 | lindau/lindau_000004_000019_leftImg8bit.png 273 | lindau/lindau_000005_000019_leftImg8bit.png 274 | lindau/lindau_000006_000019_leftImg8bit.png 275 | lindau/lindau_000007_000019_leftImg8bit.png 276 | lindau/lindau_000008_000019_leftImg8bit.png 277 | lindau/lindau_000009_000019_leftImg8bit.png 278 | lindau/lindau_000010_000019_leftImg8bit.png 279 | lindau/lindau_000011_000019_leftImg8bit.png 280 | lindau/lindau_000012_000019_leftImg8bit.png 281 | lindau/lindau_000013_000019_leftImg8bit.png 282 | lindau/lindau_000014_000019_leftImg8bit.png 283 | lindau/lindau_000015_000019_leftImg8bit.png 284 | lindau/lindau_000016_000019_leftImg8bit.png 285 | lindau/lindau_000017_000019_leftImg8bit.png 286 | lindau/lindau_000018_000019_leftImg8bit.png 287 | lindau/lindau_000019_000019_leftImg8bit.png 288 | lindau/lindau_000020_000019_leftImg8bit.png 289 | lindau/lindau_000021_000019_leftImg8bit.png 290 | lindau/lindau_000022_000019_leftImg8bit.png 291 | lindau/lindau_000023_000019_leftImg8bit.png 292 | lindau/lindau_000024_000019_leftImg8bit.png 293 | lindau/lindau_000025_000019_leftImg8bit.png 294 | lindau/lindau_000026_000019_leftImg8bit.png 295 | lindau/lindau_000027_000019_leftImg8bit.png 296 | lindau/lindau_000028_000019_leftImg8bit.png 297 | lindau/lindau_000029_000019_leftImg8bit.png 298 | lindau/lindau_000030_000019_leftImg8bit.png 299 | lindau/lindau_000031_000019_leftImg8bit.png 300 | lindau/lindau_000032_000019_leftImg8bit.png 301 | lindau/lindau_000033_000019_leftImg8bit.png 302 | lindau/lindau_000034_000019_leftImg8bit.png 303 | lindau/lindau_000035_000019_leftImg8bit.png 304 | lindau/lindau_000036_000019_leftImg8bit.png 305 | lindau/lindau_000037_000019_leftImg8bit.png 306 | lindau/lindau_000038_000019_leftImg8bit.png 307 | lindau/lindau_000039_000019_leftImg8bit.png 308 | lindau/lindau_000040_000019_leftImg8bit.png 309 | lindau/lindau_000041_000019_leftImg8bit.png 310 | lindau/lindau_000042_000019_leftImg8bit.png 311 | lindau/lindau_000043_000019_leftImg8bit.png 312 | lindau/lindau_000044_000019_leftImg8bit.png 313 | lindau/lindau_000045_000019_leftImg8bit.png 314 | lindau/lindau_000046_000019_leftImg8bit.png 315 | lindau/lindau_000047_000019_leftImg8bit.png 316 | lindau/lindau_000048_000019_leftImg8bit.png 317 | lindau/lindau_000049_000019_leftImg8bit.png 318 | lindau/lindau_000050_000019_leftImg8bit.png 319 | lindau/lindau_000051_000019_leftImg8bit.png 320 | lindau/lindau_000052_000019_leftImg8bit.png 321 | lindau/lindau_000053_000019_leftImg8bit.png 322 | lindau/lindau_000054_000019_leftImg8bit.png 323 | lindau/lindau_000055_000019_leftImg8bit.png 324 | lindau/lindau_000056_000019_leftImg8bit.png 325 | lindau/lindau_000057_000019_leftImg8bit.png 326 | lindau/lindau_000058_000019_leftImg8bit.png 327 | munster/munster_000000_000019_leftImg8bit.png 328 | munster/munster_000001_000019_leftImg8bit.png 329 | munster/munster_000002_000019_leftImg8bit.png 330 | munster/munster_000003_000019_leftImg8bit.png 331 | munster/munster_000004_000019_leftImg8bit.png 332 | munster/munster_000005_000019_leftImg8bit.png 333 | munster/munster_000006_000019_leftImg8bit.png 334 | munster/munster_000007_000019_leftImg8bit.png 335 | munster/munster_000008_000019_leftImg8bit.png 336 | munster/munster_000009_000019_leftImg8bit.png 337 | munster/munster_000010_000019_leftImg8bit.png 338 | munster/munster_000011_000019_leftImg8bit.png 339 | munster/munster_000012_000019_leftImg8bit.png 340 | munster/munster_000013_000019_leftImg8bit.png 341 | munster/munster_000014_000019_leftImg8bit.png 342 | munster/munster_000015_000019_leftImg8bit.png 343 | munster/munster_000016_000019_leftImg8bit.png 344 | munster/munster_000017_000019_leftImg8bit.png 345 | munster/munster_000018_000019_leftImg8bit.png 346 | munster/munster_000019_000019_leftImg8bit.png 347 | munster/munster_000020_000019_leftImg8bit.png 348 | munster/munster_000021_000019_leftImg8bit.png 349 | munster/munster_000022_000019_leftImg8bit.png 350 | munster/munster_000023_000019_leftImg8bit.png 351 | munster/munster_000024_000019_leftImg8bit.png 352 | munster/munster_000025_000019_leftImg8bit.png 353 | munster/munster_000026_000019_leftImg8bit.png 354 | munster/munster_000027_000019_leftImg8bit.png 355 | munster/munster_000028_000019_leftImg8bit.png 356 | munster/munster_000029_000019_leftImg8bit.png 357 | munster/munster_000030_000019_leftImg8bit.png 358 | munster/munster_000031_000019_leftImg8bit.png 359 | munster/munster_000032_000019_leftImg8bit.png 360 | munster/munster_000033_000019_leftImg8bit.png 361 | munster/munster_000034_000019_leftImg8bit.png 362 | munster/munster_000035_000019_leftImg8bit.png 363 | munster/munster_000036_000019_leftImg8bit.png 364 | munster/munster_000037_000019_leftImg8bit.png 365 | munster/munster_000038_000019_leftImg8bit.png 366 | munster/munster_000039_000019_leftImg8bit.png 367 | munster/munster_000040_000019_leftImg8bit.png 368 | munster/munster_000041_000019_leftImg8bit.png 369 | munster/munster_000042_000019_leftImg8bit.png 370 | munster/munster_000043_000019_leftImg8bit.png 371 | munster/munster_000044_000019_leftImg8bit.png 372 | munster/munster_000045_000019_leftImg8bit.png 373 | munster/munster_000046_000019_leftImg8bit.png 374 | munster/munster_000047_000019_leftImg8bit.png 375 | munster/munster_000048_000019_leftImg8bit.png 376 | munster/munster_000049_000019_leftImg8bit.png 377 | munster/munster_000050_000019_leftImg8bit.png 378 | munster/munster_000051_000019_leftImg8bit.png 379 | munster/munster_000052_000019_leftImg8bit.png 380 | munster/munster_000053_000019_leftImg8bit.png 381 | munster/munster_000054_000019_leftImg8bit.png 382 | munster/munster_000055_000019_leftImg8bit.png 383 | munster/munster_000056_000019_leftImg8bit.png 384 | munster/munster_000057_000019_leftImg8bit.png 385 | munster/munster_000058_000019_leftImg8bit.png 386 | munster/munster_000059_000019_leftImg8bit.png 387 | munster/munster_000060_000019_leftImg8bit.png 388 | munster/munster_000061_000019_leftImg8bit.png 389 | munster/munster_000062_000019_leftImg8bit.png 390 | munster/munster_000063_000019_leftImg8bit.png 391 | munster/munster_000064_000019_leftImg8bit.png 392 | munster/munster_000065_000019_leftImg8bit.png 393 | munster/munster_000066_000019_leftImg8bit.png 394 | munster/munster_000067_000019_leftImg8bit.png 395 | munster/munster_000068_000019_leftImg8bit.png 396 | munster/munster_000069_000019_leftImg8bit.png 397 | munster/munster_000070_000019_leftImg8bit.png 398 | munster/munster_000071_000019_leftImg8bit.png 399 | munster/munster_000072_000019_leftImg8bit.png 400 | munster/munster_000073_000019_leftImg8bit.png 401 | munster/munster_000074_000019_leftImg8bit.png 402 | munster/munster_000075_000019_leftImg8bit.png 403 | munster/munster_000076_000019_leftImg8bit.png 404 | munster/munster_000077_000019_leftImg8bit.png 405 | munster/munster_000078_000019_leftImg8bit.png 406 | munster/munster_000079_000019_leftImg8bit.png 407 | munster/munster_000080_000019_leftImg8bit.png 408 | munster/munster_000081_000019_leftImg8bit.png 409 | munster/munster_000082_000019_leftImg8bit.png 410 | munster/munster_000083_000019_leftImg8bit.png 411 | munster/munster_000084_000019_leftImg8bit.png 412 | munster/munster_000085_000019_leftImg8bit.png 413 | munster/munster_000086_000019_leftImg8bit.png 414 | munster/munster_000087_000019_leftImg8bit.png 415 | munster/munster_000088_000019_leftImg8bit.png 416 | munster/munster_000089_000019_leftImg8bit.png 417 | munster/munster_000090_000019_leftImg8bit.png 418 | munster/munster_000091_000019_leftImg8bit.png 419 | munster/munster_000092_000019_leftImg8bit.png 420 | munster/munster_000093_000019_leftImg8bit.png 421 | munster/munster_000094_000019_leftImg8bit.png 422 | munster/munster_000095_000019_leftImg8bit.png 423 | munster/munster_000096_000019_leftImg8bit.png 424 | munster/munster_000097_000019_leftImg8bit.png 425 | munster/munster_000098_000019_leftImg8bit.png 426 | munster/munster_000099_000019_leftImg8bit.png 427 | munster/munster_000100_000019_leftImg8bit.png 428 | munster/munster_000101_000019_leftImg8bit.png 429 | munster/munster_000102_000019_leftImg8bit.png 430 | munster/munster_000103_000019_leftImg8bit.png 431 | munster/munster_000104_000019_leftImg8bit.png 432 | munster/munster_000105_000019_leftImg8bit.png 433 | munster/munster_000106_000019_leftImg8bit.png 434 | munster/munster_000107_000019_leftImg8bit.png 435 | munster/munster_000108_000019_leftImg8bit.png 436 | munster/munster_000109_000019_leftImg8bit.png 437 | munster/munster_000110_000019_leftImg8bit.png 438 | munster/munster_000111_000019_leftImg8bit.png 439 | munster/munster_000112_000019_leftImg8bit.png 440 | munster/munster_000113_000019_leftImg8bit.png 441 | munster/munster_000114_000019_leftImg8bit.png 442 | munster/munster_000115_000019_leftImg8bit.png 443 | munster/munster_000116_000019_leftImg8bit.png 444 | munster/munster_000117_000019_leftImg8bit.png 445 | munster/munster_000118_000019_leftImg8bit.png 446 | munster/munster_000119_000019_leftImg8bit.png 447 | munster/munster_000120_000019_leftImg8bit.png 448 | munster/munster_000121_000019_leftImg8bit.png 449 | munster/munster_000122_000019_leftImg8bit.png 450 | munster/munster_000123_000019_leftImg8bit.png 451 | munster/munster_000124_000019_leftImg8bit.png 452 | munster/munster_000125_000019_leftImg8bit.png 453 | munster/munster_000126_000019_leftImg8bit.png 454 | munster/munster_000127_000019_leftImg8bit.png 455 | munster/munster_000128_000019_leftImg8bit.png 456 | munster/munster_000129_000019_leftImg8bit.png 457 | munster/munster_000130_000019_leftImg8bit.png 458 | munster/munster_000131_000019_leftImg8bit.png 459 | munster/munster_000132_000019_leftImg8bit.png 460 | munster/munster_000133_000019_leftImg8bit.png 461 | munster/munster_000134_000019_leftImg8bit.png 462 | munster/munster_000135_000019_leftImg8bit.png 463 | munster/munster_000136_000019_leftImg8bit.png 464 | munster/munster_000137_000019_leftImg8bit.png 465 | munster/munster_000138_000019_leftImg8bit.png 466 | munster/munster_000139_000019_leftImg8bit.png 467 | munster/munster_000140_000019_leftImg8bit.png 468 | munster/munster_000141_000019_leftImg8bit.png 469 | munster/munster_000142_000019_leftImg8bit.png 470 | munster/munster_000143_000019_leftImg8bit.png 471 | munster/munster_000144_000019_leftImg8bit.png 472 | munster/munster_000145_000019_leftImg8bit.png 473 | munster/munster_000146_000019_leftImg8bit.png 474 | munster/munster_000147_000019_leftImg8bit.png 475 | munster/munster_000148_000019_leftImg8bit.png 476 | munster/munster_000149_000019_leftImg8bit.png 477 | munster/munster_000150_000019_leftImg8bit.png 478 | munster/munster_000151_000019_leftImg8bit.png 479 | munster/munster_000152_000019_leftImg8bit.png 480 | munster/munster_000153_000019_leftImg8bit.png 481 | munster/munster_000154_000019_leftImg8bit.png 482 | munster/munster_000155_000019_leftImg8bit.png 483 | munster/munster_000156_000019_leftImg8bit.png 484 | munster/munster_000157_000019_leftImg8bit.png 485 | munster/munster_000158_000019_leftImg8bit.png 486 | munster/munster_000159_000019_leftImg8bit.png 487 | munster/munster_000160_000019_leftImg8bit.png 488 | munster/munster_000161_000019_leftImg8bit.png 489 | munster/munster_000162_000019_leftImg8bit.png 490 | munster/munster_000163_000019_leftImg8bit.png 491 | munster/munster_000164_000019_leftImg8bit.png 492 | munster/munster_000165_000019_leftImg8bit.png 493 | munster/munster_000166_000019_leftImg8bit.png 494 | munster/munster_000167_000019_leftImg8bit.png 495 | munster/munster_000168_000019_leftImg8bit.png 496 | munster/munster_000169_000019_leftImg8bit.png 497 | munster/munster_000170_000019_leftImg8bit.png 498 | munster/munster_000171_000019_leftImg8bit.png 499 | munster/munster_000172_000019_leftImg8bit.png 500 | munster/munster_000173_000019_leftImg8bit.png 501 | -------------------------------------------------------------------------------- /eval_caltech/ResultsEval/gt-new/gt-Reasonable.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wang-xinyu/csp.pytorch/8d5187358c1eb0fbf7ba5f184b9afed6c2518ad3/eval_caltech/ResultsEval/gt-new/gt-Reasonable.mat -------------------------------------------------------------------------------- /eval_caltech/bbGt.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wang-xinyu/csp.pytorch/8d5187358c1eb0fbf7ba5f184b9afed6c2518ad3/eval_caltech/bbGt.m -------------------------------------------------------------------------------- /eval_caltech/dbEval.m: -------------------------------------------------------------------------------- 1 | function dbEval 2 | % Evaluate and plot all pedestrian detection results. 3 | 4 | % ------------------------------------------------------------------------- 5 | % remaining parameters and constants 6 | dataName = 'UsaTest'; 7 | annVer='new'; % annotation version('ori' or 'new') 8 | fppiRn=[-2 0]; % fppi range for computing miss rate 9 | aspectRatio = .41; % default aspect ratio for all bbs 10 | bnds = [5 5 635 475]; % discard bbs outside this pixel range 11 | % if strcmp(annVer,'new'), bnds = [-inf -inf inf inf]; end 12 | plotRoc = 1; % if true plot ROC else PR curves 13 | plotAlg = 0; % if true one plot per alg else one plot per exp 14 | plotNum = 15; % only show best plotNum curves (and VJ and HOG) 15 | % samples = 10.^(fppiRn(1):.25:fppiRn(2)); % samples for computing area under the curve 16 | samples = 10.^(-2:.25:0); 17 | lims = [2e-4 50 .035 1]; % axis limits for ROC plots 18 | % lims = [2e-4 50 .025 1]; % axis limits for ROC plots 19 | bbsShow = 0; % if true displays sample bbs for each alg/exp 20 | bbsType = 'fp'; % type of bbs to display (fp/tp/fn/dt) 21 | 22 | % ------------------------------------------------------------------------- 23 | % paths 24 | pth.rootDir='./'; 25 | pth.resDir='../output/valresults/caltech/h/off'; 26 | pth.evalDir=[pth.rootDir 'ResultsEval/eval-' annVer]; 27 | pth.dtDir=[pth.rootDir 'ResultsEval/dt-']; 28 | pth.gtDir=[pth.rootDir 'ResultsEval/gt-' annVer]; 29 | % pth.videoDir=[pth.rootDir 'data-USA/']; 30 | % pth.annDir=[pth.rootDir 'anno_test_new']; 31 | 32 | exps = { 33 | 'Reasonable', [50 inf], [.65 inf], 0, .5, 1.25 34 | 'All', [20 inf], [.2 inf], 0, .5, 1.25 35 | 'Scale=large', [100 inf], [inf inf], 0, .5, 1.25 36 | 'Scale=near', [80 inf], [inf inf], 0, .5, 1.25 37 | 'Scale=medium', [30 80], [inf inf], 0, .5, 1.25 38 | 'Scale=far', [20 30], [inf inf], 0, .5, 1.25 39 | 'Occ=none', [50 inf], [inf inf], 0, .5, 1.25 40 | 'Occ=partial', [50 inf], [.65 1], 0, .5, 1.25 41 | 'Occ=heavy', [50 inf], [.2 .65], 0, .5, 1.25 42 | 'Ar=all', [50 inf], [inf inf], 0, .5, 1.25 43 | 'Ar=typical', [50 inf], [inf inf], .1, .5, 1.25 44 | 'Ar=atypical', [50 inf], [inf inf], -.1, .5, 1.25 45 | 'Overlap=25', [50 inf], [.65 inf], 0, .25, 1.25 46 | 'Overlap=50', [50 inf], [.65 inf], 0, .50, 1.25 47 | 'Overlap=75', [50 inf], [.65 inf], 0, .75, 1.25 48 | 'Expand=100', [50 inf], [.65 inf], 0, .5, 1.00 49 | 'Expand=125', [50 inf], [.65 inf], 0, .5, 1.25 50 | 'Expand=150', [50 inf], [.65 inf], 0, .5, 1.50 }; 51 | exps=cell2struct(exps',{'name','hr','vr','ar','overlap','filter'}); 52 | 53 | % ------------------------------------------------------------------------- 54 | % List of algorithms: { name, resize, color, style } 55 | % name - algorithm name (defines data location) 56 | % resize - if true rescale height of each box by 100/128 57 | n=300; clrs=zeros(n,3); 58 | for i=1:n, clrs(i,:)=max(.3,mod([78 121 42]*(i+1),255)/255); end 59 | alg_eval = dir(fullfile(pth.resDir)); 60 | algname = {alg_eval.name}; 61 | algsOt = cell(length(algname)-2,4); 62 | for i=1:length(algname)-2 63 | algsOt{i,1} = algname{i+2}; 64 | algsOt{i,2} = 0; 65 | algsOt{i,3} = clrs(i,:); 66 | if mod(i,2)==0 67 | algsOt{i,4} = '--'; 68 | else 69 | algsOt{i,4} = '-'; 70 | end 71 | end 72 | 73 | algsOt=cell2struct(algsOt',{'name','resize','color','style'}); 74 | for i=1:numel(algsOt), algsOt(i).type='other'; end 75 | exps = exps(1); 76 | algs =[algsOt(:)]; % select one or more algorithms for evaluation 77 | % directory path 78 | for i=1:numel(algs) 79 | algs(i).resDir = [pth.resDir '/' algs(i).name]; 80 | % algs(i).dtDir = [pth.dtDir algs(i).type '/']; 81 | end 82 | 83 | % select algorithms with results for current dataset 84 | algs 85 | %algs0=algs; names={algs0.name}; n=length(names); keep=false(1,n); 86 | %n 87 | %keep 88 | %for i=1:n, keep(i)=exist([algs(i).resDir '/set06'],'dir'); end 89 | %exist([algs(i).resDir '/set06'],'dir') 90 | %keep 91 | %algs=algs0(keep); 92 | %algs 93 | 94 | % name for all plots (and also temp directory for results) 95 | if(~exist(pth.evalDir,'dir')), mkdir(pth.evalDir); end 96 | 97 | % load vbb all 98 | % AS = loadAllVbb( pth.videoDir, dataName ); 99 | load('AS.mat'); 100 | % load detections and ground truth and evaluate 101 | dts = loadDt( algs, pth, aspectRatio, dataName, AS ); 102 | gts = loadGt( exps, pth, aspectRatio, bnds, dataName, AS ); 103 | res = evalAlgs( pth.evalDir , algs, exps, gts, dts ); 104 | % compute the scores 105 | [nGt,nDt]=size(res); xs=cell(nGt,nDt); ys=xs; scores=zeros(nGt,nDt); 106 | for g=1:nGt 107 | for d=1:nDt 108 | [xs{g,d},ys{g,d},~,score] = ... 109 | bbGt('compRoc',res(g,d).gtr,res(g,d).dtr,plotRoc,samples); 110 | if(plotRoc), ys{g,d}=1-ys{g,d}; score=1-score; end 111 | if(plotRoc), score=exp(mean(log(score))); else score=mean(score); end 112 | scores(g,d)=score; 113 | end 114 | end 115 | % fName=[pth.evalDir '/Roc/']; 116 | fName=pth.evalDir; 117 | if(~exist(fName,'dir')), mkdir(fName); end 118 | stra={res(1,:).stra}; stre={res(:,1).stre}; scores1=scores*100;%round(scores*10000)/100; 119 | fName1 = [fName stre{1}]; 120 | f=fopen([fName1 '.txt'],'w'); 121 | for d=1:nDt, fprintf(f,'%s %f\n',stra{d},scores1(1,d)); end; fclose(f); 122 | % plot curves and bbs 123 | plotExps( res, plotRoc, plotAlg, plotNum, pth.evalDir, ... 124 | samples, lims, reshape([algs.color]',3,[])', {algs.style}, {algs.type} ); 125 | % 126 | % set(gcf, 'PaperPositionMode', 'manual'); 127 | % set(gcf, 'PaperUnits', 'inches'); 128 | % set(gcf, 'PaperPosition', [2.5 2.5 8.5 4]) 129 | end 130 | 131 | 132 | 133 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 134 | 135 | function res = evalAlgs( plotName, algs, exps, gts, dts ) 136 | % Evaluate every algorithm on each experiment 137 | % 138 | % OUTPUTS 139 | % res - nGt x nDt cell of all evaluations, each with fields 140 | % .stra - string identifying algorithm 141 | % .stre - string identifying experiment 142 | % .gtr - [n x 1] gt result bbs for each frame [x y w h match] 143 | % .dtr - [n x 1] dt result bbs for each frame [x y w h score match] 144 | fprintf('Evaluating: %s\n',plotName); nGt=length(gts); nDt=length(dts); 145 | res=repmat(struct('stra',[],'stre',[],'gtr',[],'dtr',[]),nGt,nDt); 146 | for g=1:nGt 147 | for d=1:nDt 148 | gt=gts{g}; dt=dts{d}; n=length(gt); assert(length(dt)==n); 149 | stra=algs(d).name; stre=exps(g).name; 150 | fName = [plotName '/ev-' [stre '-' stra] '.mat']; 151 | if(exist(fName,'file')), R=load(fName); res(g,d)=R.R; continue; end 152 | fprintf('\tExp %i/%i, Alg %i/%i: %s/%s\n',g,nGt,d,nDt,stre,stra); 153 | hr = exps(g).hr.*[1/exps(g).filter exps(g).filter]; 154 | for f=1:n, bb=dt{f}; dt{f}=bb(bb(:,4)>=hr(1) & bb(:,4)=plotNum-2); 212 | kp=false(size(ord)); j=find(cumsum(~kp)>=plotNum-0); 213 | kp(1:j(1))=1; ord=fliplr(ord(kp)); 214 | xs1=xs1(ord); ys1=ys1(ord); lgd1=lgd1(ord); colors1=colors(ord,:); 215 | styles1=styles(ord); f=fopen([fName1 '.txt'],'w'); 216 | algtypes1=algtypes(ord); 217 | for d=1:nDt, fprintf(f,'%s %f\n',stra{d},scores(p,d)); end; fclose(f); 218 | end 219 | % plot curves and finalize display 220 | figure(1); clf; grid on; hold on; n=length(xs1); h=zeros(1,n); 221 | for i=1:n, h(i)=plot(xs1{i},ys1{i},'Color',colors1(i,:),... 222 | 'LineStyle',styles1{i},'LineWidth',2); end 223 | if( plotRoc ) 224 | yt=[.05 .1:.1:.5 .64 .8]; ytStr=int2str2(yt*100,2); 225 | for i=1:length(yt), ytStr{i}=['.' ytStr{i}]; end 226 | set(gca,'XScale','log','YScale','log',... 227 | 'YTick',[yt 1],'YTickLabel',[ytStr '1'],... 228 | 'XMinorGrid','off','XMinorTic','off',... 229 | 'YMinorGrid','off','YMinorTic','off'); 230 | xlabel('false positives per image','FontSize',14); 231 | ylabel('miss rate','FontSize',14); axis(lims); 232 | else 233 | x=1; for i=1:n, x=max(x,max(xs1{i})); end, x=min(x-mod(x,.1),1.0); 234 | y=.8; for i=1:n, y=min(y,min(ys1{i})); end, y=max(y-mod(y,.1),.01); 235 | xlim([0, x]); ylim([y, 1]); set(gca,'xtick',0:.1:1); 236 | xlabel('Recall','FontSize',14); ylabel('Precision','FontSize',14); 237 | end 238 | for i=1:numel(lgd1) 239 | if strcmp(algtypes1{i},'my') 240 | lgd1{i} = ['\bf{' lgd1{i} '}']; 241 | end 242 | end 243 | % h1=legend(h,lgd1,'Location','sw','FontSize',11); legend(h1,'boxoff'); 244 | % h1=legend(h,lgd1,'Location','SouthEastOutside','FontSize',11); legend(h1,'boxoff'); 245 | % legend(h,lgd1,'Location','ne','FontSize',10); 246 | 247 | % save figure to disk (uncomment pdfcrop commands to automatically crop) 248 | fName1 249 | savefig(fName1,1,'pdf','-r300','-fonts'); %close(1); 250 | if(0), setenv('PATH',[getenv('PATH') ':/usr/texbin/']); end 251 | if(0), system(['pdfcrop -margins ''-30 -20 -50 -10 '' ' ... 252 | fName1 '.pdf ' fName1 '.pdf']); end 253 | end 254 | 255 | end 256 | 257 | 258 | % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 259 | function AS = loadAllVbb( videoDir, dataName ) 260 | % Load given annotation (caches AS for speed). 261 | [setIds,vidIds]=getDBInfo(dataName); 262 | load() 263 | AS=cell(length(setIds),1e3); 264 | for s=1:length(setIds), s1=setIds(s); 265 | for v=1:length(vidIds{s}), v1=vidIds{s}(v); 266 | fName=sprintf('%s/annotations/set%02i/V%03i',videoDir,s1,v1); 267 | A=vbb('vbbLoad',fName); AS{s,v}=A; 268 | end 269 | end 270 | save('AS.mat','AS','-v6'); 271 | end 272 | % ------------------------------------------------------------------------- 273 | function gts = loadGt( exps, pth, aspectRatio, bnds, dataName, AS ) 274 | % Load ground truth of all experiments for all frames. 275 | fprintf('Loading ground truth: %s\n',pth.gtDir); 276 | nExp=length(exps); gts=cell(1,nExp); 277 | [setIds,vidIds,skip] = getDBInfo(dataName); 278 | if(~exist(pth.gtDir,'dir')), mkdir(pth.gtDir); end 279 | for i=1:nExp 280 | gName = [pth.gtDir '/gt-' exps(i).name '.mat']; 281 | if(exist(gName,'file')), gt=load(gName); gts{i}=gt.gt; continue; end 282 | fprintf('\tExperiment #%d: %s\n', i, exps(i).name); 283 | gt=cell(1,100000); k=0; 284 | lbls={'person','person?','people','ignore'}; 285 | % lbls={'person'}; 286 | ilbls={'ignore'}; 287 | pLoad={'lbls', lbls, 'ilbls', ilbls,... 288 | 'hRng',exps(i).hr,'vRng',exps(i).vr, ... 289 | 'xRng',bnds([1 3]),'yRng',bnds([2 4])}; 290 | for s=1:length(setIds), s1=setIds(s); 291 | for v=1:length(vidIds{s}), v1=vidIds{s}(v); 292 | A = AS{s,v}; 293 | for f=skip-1:skip:A.nFrame-1 294 | annName=sprintf('%s/set%02d_V%03d_I%05d.txt', pth.annDir, s1, v1, f); 295 | [~,bb]=bbGt('bbLoad',annName,pLoad); ids=bb(:,5)~=1; 296 | bb(ids,:)=bbApply('resize',bb(ids,:),1,0,aspectRatio); 297 | k=k+1; gt{k}=bb; 298 | end 299 | end 300 | end 301 | gt=gt(1:k); gts{i}=gt; save(gName,'gt','-v6'); 302 | end 303 | end 304 | 305 | % ------------------------------------------------------------------------- 306 | function dts = loadDt( algs, pth, aspectRatio, dataName, AS ) 307 | % Load detections of all algorithm for all frames. 308 | fprintf('Loading detections: %s\n',pth.dtDir); 309 | nAlg=length(algs); dts=cell(1,nAlg); 310 | [setIds,vidIds,skip] = getDBInfo(dataName); 311 | alltype=unique({algs(:).type}); 312 | for i=1:numel(alltype) 313 | dirName=[pth.dtDir alltype{i}]; 314 | if(~exist(dirName,'dir')), mkdir(dirName); end 315 | end 316 | for i=1:nAlg 317 | aName = [pth.dtDir algs(i).type '/dt-' algs(i).name '.mat']; 318 | if(exist(aName,'file')), dt=load(aName); dts{i}=dt.dt; continue; end 319 | fprintf('\tAlgorithm #%d: %s\n', i, algs(i).name); 320 | dt=cell(1,100000); k=0; aDir=algs(i).resDir; 321 | if(algs(i).resize), resize=100/128; else resize=1; end 322 | for s=1:length(setIds), s1=setIds(s); 323 | for v=1:length(vidIds{s}), v1=vidIds{s}(v); 324 | A=AS{s,v}; frames=skip-1:skip:A.nFrame-1; 325 | vName=sprintf('%s/set%02d/V%03d',aDir,s1,v1); 326 | if(~exist([vName '.txt'],'file')) 327 | % consolidate bbs for video into single text file 328 | bbs=cell(length(frames),1); 329 | for f=1:length(frames) 330 | fName = sprintf('%s/I%05d.txt',vName,frames(f)); 331 | if(~exist(fName,'file')), error(['file not found:' fName]); end 332 | bb=load(fName,'-ascii'); if(isempty(bb)), bb=zeros(0,5); end 333 | if(size(bb,2)~=5), error('incorrect dimensions'); end 334 | bbs{f}=[ones(size(bb,1),1)*(frames(f)+1) bb]; 335 | end 336 | for f=frames, delete(sprintf('%s/I%05d.txt',vName,f)); end 337 | bbs=cell2mat(bbs); dlmwrite([vName '.txt'],bbs); rmdir(vName,'s'); 338 | end 339 | bbs=load([vName '.txt'],'-ascii'); 340 | if isempty(bbs), bbs=zeros(1,6); end 341 | for f=frames, bb=bbs(bbs(:,1)==f+1,2:6); 342 | % bb=bbApply('resize',bb,resize,0,aspectRatio); 343 | k=k+1; dt{k}=bb; 344 | end 345 | end 346 | end 347 | dt=dt(1:k); dts{i}=dt; save(aName,'dt','-v6'); 348 | end 349 | end 350 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 351 | function [setIds,vidIds,skip,ext]=getDBInfo(name1) 352 | vidId=[]; setId=[]; 353 | switch lower(name1) 354 | case 'usatrain' % Caltech Pedestrian Datasets (training) 355 | setIds=0:5; skip=30; ext='jpg'; 356 | vidIds={0:14 0:5 0:11 0:12 0:11 0:12}; 357 | case 'usatest' % Caltech Pedestrian Datasets (testing) 358 | setIds=6:10; skip=30; ext='jpg'; 359 | vidIds={0:18 0:11 0:10 0:11 0:11}; 360 | otherwise, error('unknown data type: %s',name); 361 | end 362 | 363 | % optionally select only specific set/vid if name ended in ints 364 | if(~isempty(setId)), setIds=setIds(setId); vidIds=vidIds(setId); end 365 | if(~isempty(vidId)), vidIds={vidIds{1}(vidId)}; end 366 | 367 | end 368 | -------------------------------------------------------------------------------- /eval_city/dt_txt2json.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import os, sys 3 | import json 4 | import time 5 | 6 | """ 7 | Author: Antonin Vobecky 8 | Mail: a.vobecky@gmail.com, vobecant@fel.cvut.cz 9 | Github: https://github.com/vobecant 10 | LinkedIn: https://www.linkedin.com/in/antoninvobecky/ 11 | 12 | Usage: 13 | Python reimplementation of MATLAB code in dt_txt2json.m. 14 | The script takes as the first argument a path to the folder with directories that contain detection results (corresponds to main_path). 15 | If the path is not given, use default '../output/valresults/city/h/off'. 16 | The script processes each detections file and saves its JSON version to the same folder. 17 | """ 18 | 19 | def txt2jsonFile(res): 20 | out_arr = [] 21 | for det in res: 22 | det = det.rstrip("\n\r").split(' ') 23 | print('det {}'.format(det)) 24 | img_id = int(float(det[0])) 25 | bbox = [float(f) for f in det[1:5]] 26 | score = float(det[5]) 27 | det_dict = {'image_id': img_id, 28 | 'category_id': 1, 29 | 'bbox': bbox, 30 | 'score': score} 31 | out_arr.append(det_dict) 32 | return out_arr 33 | 34 | 35 | if __name__ == '__main__': 36 | if len(sys.argv) > 1: 37 | main_path = sys.argv[1] 38 | print('Look for models in "{}"'.format(main_path)) 39 | else: 40 | main_path = '../output/valresults/city/h/off' 41 | print('Use default path "{}"'.format(main_path)) 42 | 43 | start_t = time.time() 44 | 45 | dirs = [os.path.join(main_path, d) for d in os.listdir(main_path)] 46 | print('Found {} directories with detections.\n'.format(len(dirs))) 47 | 48 | for d in dirs: 49 | ndt = 0 50 | dt_coco = {} 51 | dt_path = os.path.join(d, 'val_det.txt') 52 | print('Processing detections from file {}'.format(dt_path)) 53 | if not os.path.exists(dt_path): 54 | print('File was not found! Skipping...') 55 | continue 56 | with open(dt_path, 'r') as f: 57 | res = f.readlines() 58 | out_path = os.path.join(d, 'val_dt.json') 59 | if os.path.exists(out_path): 60 | print('File was already processed. Skipping...') 61 | continue 62 | res_json = txt2jsonFile(res) 63 | 64 | with open(out_path, 'w') as f: 65 | json.dump(res_json, f) 66 | print('Saved detections to {}\n'.format(out_path)) 67 | 68 | elapsed_t = time.time() - start_t 69 | print('Conversion completed! Total time {:.1f}s'.format(elapsed_t)) 70 | -------------------------------------------------------------------------------- /eval_city/eval_script/coco.py: -------------------------------------------------------------------------------- 1 | __author__ = 'tylin' 2 | __version__ = '2.0' 3 | # Interface for accessing the Microsoft COCO dataset. 4 | 5 | # Microsoft COCO is a large image dataset designed for object detection, 6 | # segmentation, and caption generation. pycocotools is a Python API that 7 | # assists in loading, parsing and visualizing the annotations in COCO. 8 | # Please visit http://mscoco.org/ for more information on COCO, including 9 | # for the data, paper, and tutorials. The exact format of the annotations 10 | # is also described on the COCO website. For example usage of the pycocotools 11 | # please see pycocotools_demo.ipynb. In addition to this API, please download both 12 | # the COCO images and annotations in order to run the demo. 13 | 14 | # An alternative to using the API is to load the annotations directly 15 | # into Python dictionary 16 | # Using the API provides additional utility functions. Note that this API 17 | # supports both *instance* and *caption* annotations. In the case of 18 | # captions not all functions are defined (e.g. categories are undefined). 19 | 20 | # The following API functions are defined: 21 | # COCO - COCO api class that loads COCO annotation file and prepare data structures. 22 | # decodeMask - Decode binary mask M encoded via run-length encoding. 23 | # encodeMask - Encode binary mask M using run-length encoding. 24 | # getAnnIds - Get ann ids that satisfy given filter conditions. 25 | # getCatIds - Get cat ids that satisfy given filter conditions. 26 | # getImgIds - Get img ids that satisfy given filter conditions. 27 | # loadAnns - Load anns with the specified ids. 28 | # loadCats - Load cats with the specified ids. 29 | # loadImgs - Load imgs with the specified ids. 30 | # annToMask - Convert segmentation in an annotation to binary mask. 31 | # showAnns - Display the specified annotations. 32 | # loadRes - Load algorithm results and create API for accessing them. 33 | # download - Download COCO images from mscoco.org server. 34 | # Throughout the API "ann"=annotation, "cat"=category, and "img"=image. 35 | # Help on each functions can be accessed by: "help COCO>function". 36 | 37 | # See also COCO>decodeMask, 38 | # COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds, 39 | # COCO>getImgIds, COCO>loadAnns, COCO>loadCats, 40 | # COCO>loadImgs, COCO>annToMask, COCO>showAnns 41 | 42 | # Microsoft COCO Toolbox. version 2.0 43 | # Data, paper, and tutorials available at: http://mscoco.org/ 44 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2014. 45 | # Licensed under the Simplified BSD License [see bsd.txt] 46 | 47 | import json 48 | import time 49 | import matplotlib.pyplot as plt 50 | from matplotlib.collections import PatchCollection 51 | from matplotlib.patches import Polygon 52 | import numpy as np 53 | import copy 54 | import itertools 55 | # from . import mask as maskUtils 56 | import os 57 | from collections import defaultdict 58 | import sys 59 | PYTHON_VERSION = sys.version_info[0] 60 | if PYTHON_VERSION == 2: 61 | from urllib import urlretrieve 62 | elif PYTHON_VERSION == 3: 63 | from urllib.request import urlretrieve 64 | 65 | class COCO: 66 | def __init__(self, annotation_file=None): 67 | """ 68 | Constructor of Microsoft COCO helper class for reading and visualizing annotations. 69 | :param annotation_file (str): location of annotation file 70 | :param image_folder (str): location to the folder that hosts images. 71 | :return: 72 | """ 73 | # load dataset 74 | self.dataset,self.anns,self.cats,self.imgs = dict(),dict(),dict(),dict() 75 | self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list) 76 | if not annotation_file == None: 77 | # print('loading annotations into memory...') 78 | tic = time.time() 79 | dataset = json.load(open(annotation_file, 'r')) 80 | assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset)) 81 | # print('Done (t={:0.2f}s)'.format(time.time()- tic)) 82 | self.dataset = dataset 83 | self.createIndex() 84 | 85 | def createIndex(self): 86 | # create index 87 | # print('creating index...') 88 | anns, cats, imgs = {}, {}, {} 89 | imgToAnns,catToImgs = defaultdict(list),defaultdict(list) 90 | if 'annotations' in self.dataset: 91 | for ann in self.dataset['annotations']: 92 | imgToAnns[ann['image_id']].append(ann) 93 | anns[ann['id']] = ann 94 | 95 | if 'images' in self.dataset: 96 | for img in self.dataset['images']: 97 | imgs[img['id']] = img 98 | 99 | if 'categories' in self.dataset: 100 | for cat in self.dataset['categories']: 101 | cats[cat['id']] = cat 102 | 103 | if 'annotations' in self.dataset and 'categories' in self.dataset: 104 | for ann in self.dataset['annotations']: 105 | catToImgs[ann['category_id']].append(ann['image_id']) 106 | 107 | # print('index created!') 108 | 109 | # create class members 110 | self.anns = anns 111 | self.imgToAnns = imgToAnns 112 | self.catToImgs = catToImgs 113 | self.imgs = imgs 114 | self.cats = cats 115 | 116 | def info(self): 117 | """ 118 | Print information about the annotation file. 119 | :return: 120 | """ 121 | for key, value in self.dataset['info'].items(): 122 | print('{}: {}'.format(key, value)) 123 | 124 | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None): 125 | """ 126 | Get ann ids that satisfy given filter conditions. default skips that filter 127 | :param imgIds (int array) : get anns for given imgs 128 | catIds (int array) : get anns for given cats 129 | areaRng (float array) : get anns for given area range (e.g. [0 inf]) 130 | iscrowd (boolean) : get anns for given crowd label (False or True) 131 | :return: ids (int array) : integer array of ann ids 132 | """ 133 | imgIds = imgIds if type(imgIds) == list else [imgIds] 134 | catIds = catIds if type(catIds) == list else [catIds] 135 | 136 | if len(imgIds) == len(catIds) == len(areaRng) == 0: 137 | anns = self.dataset['annotations'] 138 | else: 139 | if not len(imgIds) == 0: 140 | lists = [self.imgToAnns[imgId] for imgId in imgIds if imgId in self.imgToAnns] 141 | anns = list(itertools.chain.from_iterable(lists)) 142 | else: 143 | anns = self.dataset['annotations'] 144 | anns = anns if len(catIds) == 0 else [ann for ann in anns if ann['category_id'] in catIds] 145 | anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]] 146 | if not iscrowd == None: 147 | ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd] 148 | else: 149 | ids = [ann['id'] for ann in anns] 150 | return ids 151 | 152 | def getCatIds(self, catNms=[], supNms=[], catIds=[]): 153 | """ 154 | filtering parameters. default skips that filter. 155 | :param catNms (str array) : get cats for given cat names 156 | :param supNms (str array) : get cats for given supercategory names 157 | :param catIds (int array) : get cats for given cat ids 158 | :return: ids (int array) : integer array of cat ids 159 | """ 160 | catNms = catNms if type(catNms) == list else [catNms] 161 | supNms = supNms if type(supNms) == list else [supNms] 162 | catIds = catIds if type(catIds) == list else [catIds] 163 | 164 | if len(catNms) == len(supNms) == len(catIds) == 0: 165 | cats = self.dataset['categories'] 166 | else: 167 | cats = self.dataset['categories'] 168 | cats = cats if len(catNms) == 0 else [cat for cat in cats if cat['name'] in catNms] 169 | cats = cats if len(supNms) == 0 else [cat for cat in cats if cat['supercategory'] in supNms] 170 | cats = cats if len(catIds) == 0 else [cat for cat in cats if cat['id'] in catIds] 171 | ids = [cat['id'] for cat in cats] 172 | return ids 173 | 174 | def getImgIds(self, imgIds=[], catIds=[]): 175 | ''' 176 | Get img ids that satisfy given filter conditions. 177 | :param imgIds (int array) : get imgs for given ids 178 | :param catIds (int array) : get imgs with all given cats 179 | :return: ids (int array) : integer array of img ids 180 | ''' 181 | imgIds = imgIds if type(imgIds) == list else [imgIds] 182 | catIds = catIds if type(catIds) == list else [catIds] 183 | 184 | if len(imgIds) == len(catIds) == 0: 185 | ids = self.imgs.keys() 186 | else: 187 | ids = set(imgIds) 188 | for i, catId in enumerate(catIds): 189 | if i == 0 and len(ids) == 0: 190 | ids = set(self.catToImgs[catId]) 191 | else: 192 | ids &= set(self.catToImgs[catId]) 193 | return list(ids) 194 | 195 | def loadAnns(self, ids=[]): 196 | """ 197 | Load anns with the specified ids. 198 | :param ids (int array) : integer ids specifying anns 199 | :return: anns (object array) : loaded ann objects 200 | """ 201 | if type(ids) == list: 202 | return [self.anns[id] for id in ids] 203 | elif type(ids) == int: 204 | return [self.anns[ids]] 205 | 206 | def loadCats(self, ids=[]): 207 | """ 208 | Load cats with the specified ids. 209 | :param ids (int array) : integer ids specifying cats 210 | :return: cats (object array) : loaded cat objects 211 | """ 212 | if type(ids) == list: 213 | return [self.cats[id] for id in ids] 214 | elif type(ids) == int: 215 | return [self.cats[ids]] 216 | 217 | def loadImgs(self, ids=[]): 218 | """ 219 | Load anns with the specified ids. 220 | :param ids (int array) : integer ids specifying img 221 | :return: imgs (object array) : loaded img objects 222 | """ 223 | if type(ids) == list: 224 | return [self.imgs[id] for id in ids] 225 | elif type(ids) == int: 226 | return [self.imgs[ids]] 227 | 228 | def showAnns(self, anns): 229 | """ 230 | Display the specified annotations. 231 | :param anns (array of object): annotations to display 232 | :return: None 233 | """ 234 | if len(anns) == 0: 235 | return 0 236 | if 'segmentation' in anns[0] or 'keypoints' in anns[0]: 237 | datasetType = 'instances' 238 | elif 'caption' in anns[0]: 239 | datasetType = 'captions' 240 | else: 241 | raise Exception('datasetType not supported') 242 | if datasetType == 'instances': 243 | ax = plt.gca() 244 | ax.set_autoscale_on(False) 245 | polygons = [] 246 | color = [] 247 | for ann in anns: 248 | c = (np.random.random((1, 3))*0.6+0.4).tolist()[0] 249 | if 'segmentation' in ann: 250 | if type(ann['segmentation']) == list: 251 | # polygon 252 | for seg in ann['segmentation']: 253 | poly = np.array(seg).reshape((int(len(seg)/2), 2)) 254 | polygons.append(Polygon(poly)) 255 | color.append(c) 256 | else: 257 | # mask 258 | t = self.imgs[ann['image_id']] 259 | if type(ann['segmentation']['counts']) == list: 260 | rle = maskUtils.frPyObjects([ann['segmentation']], t['height'], t['width']) 261 | else: 262 | rle = [ann['segmentation']] 263 | m = maskUtils.decode(rle) 264 | img = np.ones( (m.shape[0], m.shape[1], 3) ) 265 | if ann['iscrowd'] == 1: 266 | color_mask = np.array([2.0,166.0,101.0])/255 267 | if ann['iscrowd'] == 0: 268 | color_mask = np.random.random((1, 3)).tolist()[0] 269 | for i in range(3): 270 | img[:,:,i] = color_mask[i] 271 | ax.imshow(np.dstack( (img, m*0.5) )) 272 | if 'keypoints' in ann and type(ann['keypoints']) == list: 273 | # turn skeleton into zero-based index 274 | sks = np.array(self.loadCats(ann['category_id'])[0]['skeleton'])-1 275 | kp = np.array(ann['keypoints']) 276 | x = kp[0::3] 277 | y = kp[1::3] 278 | v = kp[2::3] 279 | for sk in sks: 280 | if np.all(v[sk]>0): 281 | plt.plot(x[sk],y[sk], linewidth=3, color=c) 282 | plt.plot(x[v>0], y[v>0],'o',markersize=8, markerfacecolor=c, markeredgecolor='k',markeredgewidth=2) 283 | plt.plot(x[v>1], y[v>1],'o',markersize=8, markerfacecolor=c, markeredgecolor=c, markeredgewidth=2) 284 | p = PatchCollection(polygons, facecolor=color, linewidths=0, alpha=0.4) 285 | ax.add_collection(p) 286 | p = PatchCollection(polygons, facecolor='none', edgecolors=color, linewidths=2) 287 | ax.add_collection(p) 288 | elif datasetType == 'captions': 289 | for ann in anns: 290 | print(ann['caption']) 291 | 292 | def loadRes(self, resFile): 293 | """ 294 | Load result file and return a result api object. 295 | :param resFile (str) : file name of result file 296 | :return: res (obj) : result api object 297 | """ 298 | res = COCO() 299 | res.dataset['images'] = [img for img in self.dataset['images']] 300 | 301 | # print('Loading and preparing results...') 302 | tic = time.time() 303 | if type(resFile) == str or type(resFile) == unicode: 304 | anns = json.load(open(resFile)) 305 | elif type(resFile) == np.ndarray: 306 | anns = self.loadNumpyAnnotations(resFile) 307 | else: 308 | anns = resFile 309 | assert type(anns) == list, 'results in not an array of objects' 310 | annsImgIds = [ann['image_id'] for ann in anns] 311 | assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \ 312 | 'Results do not correspond to current coco set' 313 | if 'caption' in anns[0]: 314 | imgIds = set([img['id'] for img in res.dataset['images']]) & set([ann['image_id'] for ann in anns]) 315 | res.dataset['images'] = [img for img in res.dataset['images'] if img['id'] in imgIds] 316 | for id, ann in enumerate(anns): 317 | ann['id'] = id+1 318 | elif 'bbox' in anns[0] and not anns[0]['bbox'] == []: 319 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories']) 320 | for id, ann in enumerate(anns): 321 | bb = ann['bbox'] 322 | x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]] 323 | if not 'segmentation' in ann: 324 | ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]] 325 | ann['area'] = bb[2]*bb[3] 326 | ann['height'] = bb[3] 327 | # if id>64181: 328 | # pass 329 | ann['id'] = id+1 330 | ann['iscrowd'] = 0 331 | elif 'segmentation' in anns[0]: 332 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories']) 333 | for id, ann in enumerate(anns): 334 | # now only support compressed RLE format as segmentation results 335 | ann['area'] = maskUtils.area(ann['segmentation']) 336 | if not 'bbox' in ann: 337 | ann['bbox'] = maskUtils.toBbox(ann['segmentation']) 338 | ann['id'] = id+1 339 | ann['iscrowd'] = 0 340 | elif 'keypoints' in anns[0]: 341 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories']) 342 | for id, ann in enumerate(anns): 343 | s = ann['keypoints'] 344 | x = s[0::3] 345 | y = s[1::3] 346 | x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y) 347 | ann['area'] = (x1-x0)*(y1-y0) 348 | ann['id'] = id + 1 349 | ann['bbox'] = [x0,y0,x1-x0,y1-y0] 350 | # print('DONE (t={:0.2f}s)'.format(time.time()- tic)) 351 | 352 | res.dataset['annotations'] = anns 353 | res.createIndex() 354 | return res 355 | 356 | def download(self, tarDir = None, imgIds = [] ): 357 | ''' 358 | Download COCO images from mscoco.org server. 359 | :param tarDir (str): COCO results directory name 360 | imgIds (list): images to be downloaded 361 | :return: 362 | ''' 363 | if tarDir is None: 364 | print('Please specify target directory') 365 | return -1 366 | if len(imgIds) == 0: 367 | imgs = self.imgs.values() 368 | else: 369 | imgs = self.loadImgs(imgIds) 370 | N = len(imgs) 371 | if not os.path.exists(tarDir): 372 | os.makedirs(tarDir) 373 | for i, img in enumerate(imgs): 374 | tic = time.time() 375 | fname = os.path.join(tarDir, img['file_name']) 376 | if not os.path.exists(fname): 377 | urlretrieve(img['coco_url'], fname) 378 | print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time()- tic)) 379 | 380 | def loadNumpyAnnotations(self, data): 381 | """ 382 | Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class} 383 | :param data (numpy.ndarray) 384 | :return: annotations (python nested list) 385 | """ 386 | print('Converting ndarray to lists...') 387 | assert(type(data) == np.ndarray) 388 | print(data.shape) 389 | assert(data.shape[1] == 7) 390 | N = data.shape[0] 391 | ann = [] 392 | for i in range(N): 393 | if i % 1000000 == 0: 394 | print('{}/{}'.format(i,N)) 395 | ann += [{ 396 | 'image_id' : int(data[i, 0]), 397 | 'bbox' : [ data[i, 1], data[i, 2], data[i, 3], data[i, 4] ], 398 | 'score' : data[i, 5], 399 | 'category_id': int(data[i, 6]), 400 | }] 401 | return ann 402 | 403 | def annToRLE(self, ann): 404 | """ 405 | Convert annotation which can be polygons, uncompressed RLE to RLE. 406 | :return: binary mask (numpy 2D array) 407 | """ 408 | t = self.imgs[ann['image_id']] 409 | h, w = t['height'], t['width'] 410 | segm = ann['segmentation'] 411 | if type(segm) == list: 412 | # polygon -- a single object might consist of multiple parts 413 | # we merge all parts into one mask rle code 414 | rles = maskUtils.frPyObjects(segm, h, w) 415 | rle = maskUtils.merge(rles) 416 | elif type(segm['counts']) == list: 417 | # uncompressed RLE 418 | rle = maskUtils.frPyObjects(segm, h, w) 419 | else: 420 | # rle 421 | rle = ann['segmentation'] 422 | return rle 423 | 424 | def annToMask(self, ann): 425 | """ 426 | Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask. 427 | :return: binary mask (numpy 2D array) 428 | """ 429 | rle = self.annToRLE(ann) 430 | m = maskUtils.decode(rle) 431 | return m -------------------------------------------------------------------------------- /eval_city/eval_script/eval_MR_multisetup.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import datetime 3 | import time 4 | from collections import defaultdict 5 | # from . import mask as maskUtils 6 | import copy 7 | import matplotlib.pyplot as plt 8 | import scipy.io as sio 9 | 10 | class COCOeval: 11 | # Interface for evaluating detection on the Microsoft COCO dataset. 12 | # 13 | # The usage for CocoEval is as follows: 14 | # cocoGt=..., cocoDt=... # load dataset and results 15 | # E = CocoEval(cocoGt,cocoDt); # initialize CocoEval object 16 | # E.params.recThrs = ...; # set parameters as desired 17 | # E.evaluate(); # run per image evaluation 18 | # E.accumulate(); # accumulate per image results 19 | # E.summarize(); # display summary metrics of results 20 | # For example usage see evalDemo.m and http://mscoco.org/. 21 | # 22 | # The evaluation parameters are as follows (defaults in brackets): 23 | # imgIds - [all] N img ids to use for evaluation 24 | # catIds - [all] K cat ids to use for evaluation 25 | # iouThrs - [.5:.05:.95] T=10 IoU thresholds for evaluation 26 | # recThrs - [0:.01:1] R=101 recall thresholds for evaluation 27 | # areaRng - [...] A=4 object area ranges for evaluation 28 | # maxDets - [1 10 100] M=3 thresholds on max detections per image 29 | # iouType - ['segm'] set iouType to 'segm', 'bbox' or 'keypoints' 30 | # iouType replaced the now DEPRECATED useSegm parameter. 31 | # useCats - [1] if true use category labels for evaluation 32 | # Note: if useCats=0 category labels are ignored as in proposal scoring. 33 | # Note: multiple areaRngs [Ax2] and maxDets [Mx1] can be specified. 34 | # 35 | # evaluate(): evaluates detections on every image and every category and 36 | # concats the results into the "evalImgs" with fields: 37 | # dtIds - [1xD] id for each of the D detections (dt) 38 | # gtIds - [1xG] id for each of the G ground truths (gt) 39 | # dtMatches - [TxD] matching gt id at each IoU or 0 40 | # gtMatches - [TxG] matching dt id at each IoU or 0 41 | # dtScores - [1xD] confidence of each dt 42 | # gtIgnore - [1xG] ignore flag for each gt 43 | # dtIgnore - [TxD] ignore flag for each dt at each IoU 44 | # 45 | # accumulate(): accumulates the per-image, per-category evaluation 46 | # results in "evalImgs" into the dictionary "eval" with fields: 47 | # params - parameters used for evaluation 48 | # date - date evaluation was performed 49 | # counts - [T,R,K,A,M] parameter dimensions (see above) 50 | # precision - [TxRxKxAxM] precision for every evaluation setting 51 | # recall - [TxKxAxM] max recall for every evaluation setting 52 | # Note: precision and recall==-1 for settings with no gt objects. 53 | # 54 | # See also coco, mask, pycocoDemo, pycocoEvalDemo 55 | # 56 | # Microsoft COCO Toolbox. version 2.0 57 | # Data, paper, and tutorials available at: http://mscoco.org/ 58 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015. 59 | # Licensed under the Simplified BSD License [see coco/license.txt] 60 | def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'): 61 | ''' 62 | Initialize CocoEval using coco APIs for gt and dt 63 | :param cocoGt: coco object with ground truth annotations 64 | :param cocoDt: coco object with detection results 65 | :return: None 66 | ''' 67 | if not iouType: 68 | print('iouType not specified. use default iouType segm') 69 | self.cocoGt = cocoGt # ground truth COCO API 70 | self.cocoDt = cocoDt # detections COCO API 71 | self.params = {} # evaluation parameters 72 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results [KxAxI] elements 73 | self.eval = {} # accumulated evaluation results 74 | self._gts = defaultdict(list) # gt for evaluation 75 | self._dts = defaultdict(list) # dt for evaluation 76 | self.params = Params(iouType=iouType) # parameters 77 | self._paramsEval = {} # parameters for evaluation 78 | self.stats = [] # result summarization 79 | self.ious = {} # ious between all gts and dts 80 | if not cocoGt is None: 81 | self.params.imgIds = sorted(cocoGt.getImgIds()) 82 | self.params.catIds = sorted(cocoGt.getCatIds()) 83 | 84 | 85 | def _prepare(self, id_setup): 86 | ''' 87 | Prepare ._gts and ._dts for evaluation based on params 88 | :return: None 89 | ''' 90 | p = self.params 91 | if p.useCats: 92 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds)) 93 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds)) 94 | else: 95 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds)) 96 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds)) 97 | 98 | 99 | # set ignore flag 100 | for gt in gts: 101 | gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0 102 | gt['ignore'] = 1 if (gt['height'] < self.params.HtRng[id_setup][0] or gt['height'] > self.params.HtRng[id_setup][1]) or \ 103 | ( gt['vis_ratio'] < self.params.VisRng[id_setup][0] or gt['vis_ratio'] > self.params.VisRng[id_setup][1]) else gt['ignore'] 104 | 105 | self._gts = defaultdict(list) # gt for evaluation 106 | self._dts = defaultdict(list) # dt for evaluation 107 | for gt in gts: 108 | self._gts[gt['image_id'], gt['category_id']].append(gt) 109 | for dt in dts: 110 | self._dts[dt['image_id'], dt['category_id']].append(dt) 111 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results 112 | self.eval = {} # accumulated evaluation results 113 | 114 | def evaluate(self, id_setup): 115 | ''' 116 | Run per image evaluation on given images and store results (a list of dict) in self.evalImgs 117 | :return: None 118 | ''' 119 | tic = time.time() 120 | # print('Running per image evaluation...') 121 | p = self.params 122 | # add backward compatibility if useSegm is specified in params 123 | if not p.useSegm is None: 124 | p.iouType = 'segm' if p.useSegm == 1 else 'bbox' 125 | print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType)) 126 | # print('Evaluate annotation type *{}*'.format(p.iouType)) 127 | p.imgIds = list(np.unique(p.imgIds)) 128 | if p.useCats: 129 | p.catIds = list(np.unique(p.catIds)) 130 | p.maxDets = sorted(p.maxDets) 131 | self.params=p 132 | 133 | self._prepare(id_setup) 134 | # loop through images, area range, max detection number 135 | catIds = p.catIds if p.useCats else [-1] 136 | 137 | computeIoU = self.computeIoU 138 | 139 | self.ious = {(imgId, catId): computeIoU(imgId, catId) \ 140 | for imgId in p.imgIds 141 | for catId in catIds} 142 | 143 | evaluateImg = self.evaluateImg 144 | maxDet = p.maxDets[-1] 145 | HtRng = self.params.HtRng[id_setup] 146 | VisRng = self.params.VisRng[id_setup] 147 | self.evalImgs = [evaluateImg(imgId, catId, HtRng, VisRng, maxDet) 148 | for catId in catIds 149 | for imgId in p.imgIds 150 | ] 151 | self._paramsEval = copy.deepcopy(self.params) 152 | toc = time.time() 153 | # print('DONE (t={:0.2f}s).'.format(toc-tic)) 154 | 155 | 156 | def computeIoU(self, imgId, catId): 157 | p = self.params 158 | if p.useCats: 159 | gt = self._gts[imgId,catId] 160 | dt = self._dts[imgId,catId] 161 | else: 162 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]] 163 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]] 164 | if len(gt) == 0 and len(dt) ==0: 165 | return [] 166 | inds = np.argsort([-d['score'] for d in dt], kind='mergesort') 167 | dt = [dt[i] for i in inds] 168 | if len(dt) > p.maxDets[-1]: 169 | dt=dt[0:p.maxDets[-1]] 170 | 171 | 172 | if p.iouType == 'segm': 173 | g = [g['segmentation'] for g in gt] 174 | d = [d['segmentation'] for d in dt] 175 | elif p.iouType == 'bbox': 176 | g = [g['bbox'] for g in gt] 177 | d = [d['bbox'] for d in dt] 178 | else: 179 | raise Exception('unknown iouType for iou computation') 180 | 181 | 182 | # compute iou between each dt and gt region 183 | iscrowd = [int(o['ignore']) for o in gt] 184 | ious = self.iou(d,g,iscrowd) 185 | return ious 186 | 187 | def iou( self, dts, gts, pyiscrowd ): 188 | dts = np.asarray(dts) 189 | gts = np.asarray(gts) 190 | pyiscrowd = np.asarray(pyiscrowd) 191 | ious = np.zeros((len(dts), len(gts))) 192 | for j, gt in enumerate(gts): 193 | gx1 = gt[0] 194 | gy1 = gt[1] 195 | gx2 = gt[0] + gt[2] 196 | gy2 = gt[1] + gt[3] 197 | garea = gt[2] * gt[3] 198 | for i, dt in enumerate(dts): 199 | dx1 = dt[0] 200 | dy1 = dt[1] 201 | dx2 = dt[0] + dt[2] 202 | dy2 = dt[1] + dt[3] 203 | darea = dt[2] * dt[3] 204 | 205 | unionw = min(dx2,gx2)-max(dx1,gx1) 206 | if unionw <= 0: 207 | continue 208 | unionh = min(dy2,gy2)-max(dy1,gy1) 209 | if unionh <= 0: 210 | continue 211 | t = unionw * unionh 212 | if pyiscrowd[j]: 213 | unionarea = darea 214 | else: 215 | unionarea = darea + garea - t 216 | 217 | ious[i, j] = float(t)/unionarea 218 | return ious 219 | 220 | def evaluateImg(self, imgId, catId, hRng, vRng, maxDet): 221 | ''' 222 | perform evaluation for single category and image 223 | :return: dict (single image results) 224 | ''' 225 | # if imgId==273: 226 | # pass 227 | p = self.params 228 | if p.useCats: 229 | gt = self._gts[imgId,catId] 230 | dt = self._dts[imgId,catId] 231 | else: 232 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]] 233 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]] 234 | if len(gt) == 0 and len(dt) ==0: 235 | return None 236 | 237 | for g in gt: 238 | if g['ignore']: 239 | g['_ignore'] = 1 240 | else: 241 | g['_ignore'] = 0 242 | # sort dt highest score first, sort gt ignore last 243 | gtind = np.argsort([g['_ignore'] for g in gt], kind='mergesort') 244 | gt = [gt[i] for i in gtind] 245 | dtind = np.argsort([-d['score'] for d in dt], kind='mergesort') 246 | # dtind = np.argsort([-d['score'] for d in dt]) 247 | dt = [dt[i] for i in dtind[0:maxDet]] 248 | # exclude dt out of height range 249 | # dt = [d for d in dt if d['height'] >= hRng[0] / self.params.expFilter and d['height'] < hRng[1] * self.params.expFilter] 250 | # dtind = np.array([int(d['id'] - dt[0]['id']) for d in dt]) 251 | dtind = [i for i in range(len(dt)) if dt[i]['height'] >= hRng[0] / self.params.expFilter and dt[i]['height'] < hRng[1] * self.params.expFilter] 252 | dt = [d for d in dt if d['height'] >= hRng[0] / self.params.expFilter and d['height'] < hRng[1] * self.params.expFilter] 253 | # if np.max(dtind)>1000: 254 | # pass 255 | 256 | # load computed ious 257 | if len(dtind) > 0: 258 | ious = self.ious[imgId, catId][dtind, :] if len(self.ious[imgId, catId]) > 0 else self.ious[imgId, catId] 259 | ious = ious[:, gtind] 260 | else: 261 | ious = [] 262 | 263 | T = len(p.iouThrs) 264 | G = len(gt) 265 | D = len(dt) 266 | gtm = np.zeros((T,G)) 267 | dtm = np.zeros((T,D)) 268 | gtIg = np.array([g['_ignore'] for g in gt]) 269 | dtIg = np.zeros((T,D)) 270 | if not len(ious)==0: 271 | for tind, t in enumerate(p.iouThrs): 272 | for dind, d in enumerate(dt): 273 | # information about best match so far (m=-1 -> unmatched) 274 | iou = min([t,1-1e-10]) 275 | bstOa = iou 276 | bstg = -2 277 | bstm = -2 278 | for gind, g in enumerate(gt): 279 | m = gtm[tind,gind] 280 | # if this gt already matched, and not a crowd, continue 281 | if m>0: 282 | continue 283 | # if dt matched to reg gt, and on ignore gt, stop 284 | if bstm!=-2 and gtIg[gind] == 1: 285 | break 286 | # continue to next gt unless better match made 287 | if ious[dind,gind] < bstOa: 288 | continue 289 | # if match successful and best so far, store appropriately 290 | bstOa=ious[dind,gind] 291 | bstg = gind 292 | if gtIg[gind] == 0: 293 | bstm = 1 294 | else: 295 | bstm = -1 296 | 297 | # if match made store id of match for both dt and gt 298 | if bstg ==-2: 299 | continue 300 | dtIg[tind,dind] = gtIg[bstg] 301 | dtm[tind,dind] = gt[bstg]['id'] 302 | if bstm == 1: 303 | gtm[tind,bstg] = d['id'] 304 | 305 | # store results for given image and category 306 | return { 307 | 'image_id': imgId, 308 | 'category_id': catId, 309 | 'hRng': hRng, 310 | 'vRng': vRng, 311 | 'maxDet': maxDet, 312 | 'dtIds': [d['id'] for d in dt], 313 | 'gtIds': [g['id'] for g in gt], 314 | 'dtMatches': dtm, 315 | 'gtMatches': gtm, 316 | 'dtScores': [d['score'] for d in dt], 317 | 'gtIgnore': gtIg, 318 | 'dtIgnore': dtIg, 319 | } 320 | 321 | def accumulate(self, p = None): 322 | ''' 323 | Accumulate per image evaluation results and store the result in self.eval 324 | :param p: input params for evaluation 325 | :return: None 326 | ''' 327 | # print('Accumulating evaluation results...') 328 | tic = time.time() 329 | if not self.evalImgs: 330 | print('Please run evaluate() first') 331 | # allows input customized parameters 332 | if p is None: 333 | p = self.params 334 | p.catIds = p.catIds if p.useCats == 1 else [-1] 335 | T = len(p.iouThrs) 336 | R = len(p.fppiThrs) 337 | K = len(p.catIds) if p.useCats else 1 338 | M = len(p.maxDets) 339 | ys = -np.ones((T,R,K,M)) # -1 for the precision of absent categories 340 | 341 | 342 | # create dictionary for future indexing 343 | _pe = self._paramsEval 344 | catIds = [1] #_pe.catIds if _pe.useCats else [-1] 345 | setK = set(catIds) 346 | setM = set(_pe.maxDets) 347 | setI = set(_pe.imgIds) 348 | # get inds to evaluate 349 | k_list = [n for n, k in enumerate(p.catIds) if k in setK] 350 | 351 | m_list = [m for n, m in enumerate(p.maxDets) if m in setM] 352 | i_list = [n for n, i in enumerate(p.imgIds) if i in setI] 353 | I0 = len(_pe.imgIds) 354 | 355 | # retrieve E at each category, area range, and max number of detections 356 | for k, k0 in enumerate(k_list): 357 | Nk = k0*I0 358 | for m, maxDet in enumerate(m_list): 359 | E = [self.evalImgs[Nk + i] for i in i_list] 360 | E = [e for e in E if not e is None] 361 | if len(E) == 0: 362 | continue 363 | 364 | dtScores = np.concatenate([e['dtScores'][0:maxDet] for e in E]) 365 | 366 | # different sorting method generates slightly different results. 367 | # mergesort is used to be consistent as Matlab implementation. 368 | 369 | inds = np.argsort(-dtScores, kind='mergesort') 370 | 371 | dtm = np.concatenate([e['dtMatches'][:, 0:maxDet] for e in E], axis=1)[:, inds] 372 | dtIg = np.concatenate([e['dtIgnore'][:, 0:maxDet] for e in E], axis=1)[:, inds] 373 | gtIg = np.concatenate([e['gtIgnore'] for e in E]) 374 | npig = np.count_nonzero(gtIg == 0) 375 | if npig == 0: 376 | continue 377 | tps = np.logical_and(dtm, np.logical_not(dtIg)) 378 | fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg)) 379 | inds = np.where(dtIg==0)[1] 380 | tps = tps[:,inds] 381 | fps = fps[:,inds] 382 | 383 | tp_sum = np.cumsum(tps, axis=1).astype(dtype=np.float) 384 | fp_sum = np.cumsum(fps, axis=1).astype(dtype=np.float) 385 | for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)): 386 | tp = np.array(tp) 387 | fppi = np.array(fp)/I0 388 | nd = len(tp) 389 | recall = tp / npig 390 | q = np.zeros((R,)) 391 | 392 | # numpy is slow without cython optimization for accessing elements 393 | # use python array gets significant speed improvement 394 | recall = recall.tolist() 395 | q = q.tolist() 396 | 397 | for i in range(nd - 1, 0, -1): 398 | if recall[i] < recall[i - 1]: 399 | recall[i - 1] = recall[i] 400 | 401 | inds = np.searchsorted(fppi, p.fppiThrs, side='right') - 1 402 | try: 403 | for ri, pi in enumerate(inds): 404 | q[ri] = recall[pi] 405 | except: 406 | pass 407 | ys[t,:,k,m] = np.array(q) 408 | self.eval = { 409 | 'params': p, 410 | 'counts': [T, R, K, M], 411 | 'date': datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), 412 | 'TP': ys, 413 | } 414 | toc = time.time() 415 | # print('DONE (t={:0.2f}s).'.format( toc-tic)) 416 | 417 | def summarize(self,id_setup, res_file): 418 | ''' 419 | Compute and display summary metrics for evaluation results. 420 | Note this functin can *only* be applied on the default parameter setting 421 | ''' 422 | def _summarize(iouThr=None, maxDets=100 ): 423 | p = self.params 424 | iStr = ' {:<18} {} @ {:<18} [ IoU={:<9} | height={:>6s} | visibility={:>6s} ] = {:0.2f}%' 425 | titleStr = 'Average Miss Rate' 426 | typeStr = '(MR)' 427 | setupStr = p.SetupLbl[id_setup] 428 | iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \ 429 | if iouThr is None else '{:0.2f}'.format(iouThr) 430 | heightStr = '[{:0.0f}:{:0.0f}]'.format(p.HtRng[id_setup][0], p.HtRng[id_setup][1]) 431 | occlStr = '[{:0.2f}:{:0.2f}]'.format(p.VisRng[id_setup][0], p.VisRng[id_setup][1]) 432 | 433 | mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets] 434 | 435 | # dimension of precision: [TxRxKxAxM] 436 | s = self.eval['TP'] 437 | # IoU 438 | if iouThr is not None: 439 | t = np.where(iouThr == p.iouThrs)[0] 440 | s = s[t] 441 | mrs = 1-s[:,:,:,mind] 442 | 443 | if len(mrs[mrs<2])==0: 444 | mean_s = -1 445 | else: 446 | mean_s = np.log(mrs[mrs<2]) 447 | mean_s = np.mean(mean_s) 448 | mean_s = np.exp(mean_s) 449 | print(iStr.format(titleStr, typeStr,setupStr, iouStr, heightStr, occlStr, mean_s*100)) 450 | # res_file.write(iStr.format(titleStr, typeStr,setupStr, iouStr, heightStr, occlStr, mean_s*100)) 451 | res_file.write(str(mean_s * 100)) 452 | res_file.write('\n') 453 | return mean_s 454 | 455 | if not self.eval: 456 | raise Exception('Please run accumulate() first') 457 | _summarize(iouThr=.5,maxDets=1000) 458 | 459 | def __str__(self): 460 | self.summarize() 461 | 462 | class Params: 463 | ''' 464 | Params for coco evaluation api 465 | ''' 466 | def setDetParams(self): 467 | self.imgIds = [] 468 | self.catIds = [] 469 | # np.arange causes trouble. the data point on arange is slightly larger than the true value 470 | 471 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True) 472 | self.fppiThrs = np.array([0.0100, 0.0178, 0.0316, 0.0562, 0.1000, 0.1778, 0.3162, 0.5623, 1.0000]) 473 | self.maxDets = [1000] 474 | self.expFilter = 1.25 475 | self.useCats = 1 476 | 477 | self.iouThrs = np.array([0.5]) # np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True) 478 | 479 | # self.HtRng = [[50, 1e5 ** 2], [50,75], [50, 1e5 ** 2], [20, 1e5 ** 2]] 480 | # self.VisRng = [[0.65, 1e5 ** 2], [0.65, 1e5 ** 2], [0.2,0.65], [0.2, 1e5 ** 2]] 481 | # self.SetupLbl = ['Reasonable', 'Reasonable_small','Reasonable_occ=heavy', 'All'] 482 | 483 | # self.HtRng = [[50, 1e5 ** 2], [50, 75], [75, 100], [100, 1e5 ** 2]] 484 | # self.VisRng = [[0.65, 1e5 ** 2], [0.65, 1e5 ** 2], [0.65, 1e5 ** 2], [0.65, 1e5 ** 2]] 485 | # self.SetupLbl = ['Reasonable', 'small', 'middle', 'large'] 486 | 487 | self.HtRng = [[50, 1e5 ** 2], [50, 1e5 ** 2], [50, 1e5 ** 2], [50, 1e5 ** 2]] 488 | self.VisRng = [[0.65, 1e5 ** 2], [0.9, 1e5 ** 2], [0.65, 0.9], [0, 0.65]] 489 | self.SetupLbl = ['Reasonable', 'bare', 'partial', 'heavy'] 490 | 491 | 492 | def __init__(self, iouType='segm'): 493 | if iouType == 'segm' or iouType == 'bbox': 494 | self.setDetParams() 495 | else: 496 | raise Exception('iouType not supported') 497 | self.iouType = iouType 498 | # useSegm is deprecated 499 | self.useSegm = None 500 | -------------------------------------------------------------------------------- /eval_city/eval_script/eval_demo.py: -------------------------------------------------------------------------------- 1 | import os 2 | from coco import COCO 3 | from eval_MR_multisetup import COCOeval 4 | 5 | annType = 'bbox' #specify type here 6 | 7 | #initialize COCO ground truth api 8 | annFile = '../val_gt.json' 9 | main_path = '../../output/valresults/city/h/off' 10 | for f in sorted(os.listdir(main_path)): 11 | #print f 12 | # initialize COCO detections api 13 | dt_path = os.path.join(main_path, f) 14 | resFile = os.path.join(dt_path,'val_dt.json') 15 | respath = os.path.join(dt_path,'results.txt') 16 | # if os.path.exists(respath): 17 | # continue 18 | ## running evaluation 19 | res_file = open(respath, "w") 20 | for id_setup in range(0,1): 21 | cocoGt = COCO(annFile) 22 | cocoDt = cocoGt.loadRes(resFile) 23 | imgIds = sorted(cocoGt.getImgIds()) 24 | cocoEval = COCOeval(cocoGt,cocoDt,annType) 25 | cocoEval.params.imgIds = imgIds 26 | cocoEval.evaluate(id_setup) 27 | cocoEval.accumulate() 28 | cocoEval.summarize(id_setup,res_file) 29 | 30 | res_file.close() 31 | -------------------------------------------------------------------------------- /eval_city/eval_script/readme.txt: -------------------------------------------------------------------------------- 1 | ################################ 2 | This script is created to produce miss rate numbers by making minor changes to the COCO python evaluation script [1]. It is a python re-implementation of the Caltech evaluation code, which is written in matlab. This python script produces exactly the same numbers as the matlab code. 3 | [1] https://github.com/pdollar/coco/tree/master/PythonAPI 4 | [2] http://www.vision.caltech.edu/Image_Datasets/CaltechPedestrians/code/code3.2.1.zip 5 | ################################# 6 | Usage 7 | 1. Prepare detection results in COCO format, and write them in a single .json file. 8 | 2. Run eval_demo.py. 9 | 3. Detailed evaluations will be written to results.txt. 10 | ################################# 11 | 12 | -------------------------------------------------------------------------------- /models/cspnet.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import math 3 | import torch.nn as nn 4 | from .l2norm import L2Norm 5 | from .samepad import SamePad2d 6 | 7 | class IdentityBlock(nn.Module): 8 | expansion = 4 9 | def __init__(self, inchannels, filters, dila=1): 10 | super(IdentityBlock, self).__init__() 11 | self.conv1 = nn.Conv2d(inchannels, filters, kernel_size=1) 12 | self.bn1 = nn.BatchNorm2d(filters, eps=1e-03, momentum=0.01) 13 | self.samepad = SamePad2d(3, 1, dilation=dila) 14 | self.conv2 = nn.Conv2d(filters, filters, kernel_size=3, dilation=dila) 15 | self.bn2 = nn.BatchNorm2d(filters, eps=1e-03, momentum=0.01) 16 | self.conv3 = nn.Conv2d(filters, filters * self.expansion, kernel_size=1) 17 | self.bn3 = nn.BatchNorm2d(filters * self.expansion, eps=1e-03, momentum=0.01) 18 | self.relu = nn.ReLU(inplace=True) 19 | 20 | def forward(self, x): 21 | out = self.conv1(x) 22 | out = self.bn1(out) 23 | out = self.relu(out) 24 | 25 | print('a shape --- ', out.shape) 26 | out = self.samepad(out) 27 | print('b shape --- ', out.shape) 28 | out = self.conv2(out) 29 | print('c shape --- ', out.shape) 30 | out = self.bn2(out) 31 | out = self.relu(out) 32 | 33 | out = self.conv3(out) 34 | out = self.bn3(out) 35 | 36 | out += x 37 | out = self.relu(out) 38 | 39 | return out 40 | 41 | class ConvBlock(nn.Module): 42 | expansion = 4 43 | def __init__(self, inchannels, filters, s=2, dila=1): 44 | super(ConvBlock, self).__init__() 45 | self.conv1 = nn.Conv2d(inchannels, filters, kernel_size=1, stride=s) 46 | self.bn1 = nn.BatchNorm2d(filters, eps=1e-03, momentum=0.01) 47 | self.samepad = SamePad2d(3, 1, dilation=dila) 48 | self.conv2 = nn.Conv2d(filters, filters, kernel_size=3, dilation=dila) 49 | self.bn2 = nn.BatchNorm2d(filters, eps=1e-03, momentum=0.01) 50 | self.conv3 = nn.Conv2d(filters, filters * self.expansion, kernel_size=1) 51 | self.bn3 = nn.BatchNorm2d(filters * self.expansion, eps=1e-03, momentum=0.01) 52 | self.conv4 = nn.Conv2d(inchannels, filters * self.expansion, kernel_size=1, stride=s) 53 | self.bn4 = nn.BatchNorm2d(filters * self.expansion, eps=1e-03, momentum=0.01) 54 | self.relu = nn.ReLU(inplace=True) 55 | 56 | def forward(self, x): 57 | out = self.conv1(x) 58 | out = self.bn1(out) 59 | out = self.relu(out) 60 | 61 | print('a shape --- ', out.shape) 62 | out = self.samepad(out) 63 | print('b shape --- ', out.shape) 64 | out = self.conv2(out) 65 | print('c shape --- ', out.shape) 66 | out = self.bn2(out) 67 | out = self.relu(out) 68 | 69 | out = self.conv3(out) 70 | out = self.bn3(out) 71 | 72 | shortcut = self.conv4(x) 73 | shortcut = self.bn4(shortcut) 74 | print('shortcut shape --- ', shortcut.shape) 75 | 76 | out += shortcut 77 | out = self.relu(out) 78 | 79 | return out 80 | 81 | 82 | class CSPNet_p3p4p5(nn.Module): 83 | def __init__(self, num_scale=1): 84 | super(CSPNet_p3p4p5, self).__init__() 85 | 86 | #resnet = resnet50(pretrained=True, receptive_keep=True) 87 | 88 | self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3) 89 | self.bn1 = nn.BatchNorm2d(64, eps=1e-03, momentum=0.01) 90 | self.relu = nn.ReLU(inplace=True) 91 | self.samepad1 = SamePad2d(3, 2) 92 | self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2) 93 | 94 | self.convblk2a = ConvBlock(64, 64, s=1) 95 | self.identityblk2b = IdentityBlock(256, 64) 96 | self.identityblk2c = IdentityBlock(256, 64) 97 | 98 | self.convblk3a = ConvBlock(256, 128) 99 | self.identityblk3b = IdentityBlock(512, 128) 100 | self.identityblk3c = IdentityBlock(512, 128) 101 | self.identityblk3d = IdentityBlock(512, 128) 102 | 103 | self.convblk4a = ConvBlock(512, 256) 104 | self.identityblk4b = IdentityBlock(1024, 256) 105 | self.identityblk4c = IdentityBlock(1024, 256) 106 | self.identityblk4d = IdentityBlock(1024, 256) 107 | self.identityblk4e = IdentityBlock(1024, 256) 108 | self.identityblk4f = IdentityBlock(1024, 256) 109 | 110 | self.convblk5a = ConvBlock(1024, 512, s=1, dila=2) 111 | self.identityblk5b = IdentityBlock(2048, 512, dila=2) 112 | self.identityblk5c = IdentityBlock(2048, 512, dila=2) 113 | 114 | self.p3 = nn.ConvTranspose2d(512, 256, kernel_size=4, stride=2, padding=1) 115 | self.p4 = nn.ConvTranspose2d(1024, 256, kernel_size=4, stride=4, padding=0) 116 | self.p5 = nn.ConvTranspose2d(2048, 256, kernel_size=4, stride=4, padding=0) 117 | 118 | nn.init.xavier_normal_(self.p3.weight) 119 | nn.init.xavier_normal_(self.p4.weight) 120 | nn.init.xavier_normal_(self.p5.weight) 121 | nn.init.constant_(self.p3.bias, 0) 122 | nn.init.constant_(self.p4.bias, 0) 123 | nn.init.constant_(self.p5.bias, 0) 124 | 125 | self.p3_l2 = L2Norm(256, 10) 126 | self.p4_l2 = L2Norm(256, 10) 127 | self.p5_l2 = L2Norm(256, 10) 128 | 129 | self.feat = nn.Conv2d(768, 256, kernel_size=3, stride=1, padding=1) 130 | self.feat_bn = nn.BatchNorm2d(256, eps=1e-03, momentum=0.01) 131 | 132 | self.center_conv = nn.Conv2d(256, 1, kernel_size=1) 133 | self.height_conv = nn.Conv2d(256, num_scale, kernel_size=1) 134 | self.offset_conv = nn.Conv2d(256, 2, kernel_size=1) 135 | 136 | nn.init.xavier_normal_(self.feat.weight) 137 | nn.init.xavier_normal_(self.center_conv.weight) 138 | nn.init.xavier_normal_(self.height_conv.weight) 139 | nn.init.xavier_normal_(self.offset_conv.weight) 140 | nn.init.constant_(self.center_conv.bias, -math.log(0.99/0.01)) 141 | nn.init.constant_(self.height_conv.bias, 0) 142 | nn.init.constant_(self.offset_conv.bias, 0) 143 | 144 | def forward(self, x): 145 | x = self.conv1(x) 146 | x = self.bn1(x) 147 | x = self.relu(x) 148 | x = self.samepad1(x) 149 | x = self.maxpool(x) 150 | 151 | x = self.convblk2a(x) 152 | x = self.identityblk2b(x) 153 | stage2 = self.identityblk2c(x) 154 | 155 | x = self.convblk3a(stage2) 156 | x = self.identityblk3b(x) 157 | x = self.identityblk3c(x) 158 | stage3 = self.identityblk3d(x) 159 | 160 | x = self.convblk4a(stage3) 161 | x = self.identityblk4b(x) 162 | x = self.identityblk4c(x) 163 | x = self.identityblk4d(x) 164 | x = self.identityblk4e(x) 165 | stage4 = self.identityblk4f(x) 166 | 167 | x = self.convblk5a(stage4) 168 | x = self.identityblk5b(x) 169 | stage5 = self.identityblk5c(x) 170 | 171 | p3up = self.p3(stage3) 172 | p4up = self.p4(stage4) 173 | p5up = self.p5(stage5) 174 | p3up = self.p3_l2(p3up) 175 | p4up = self.p4_l2(p4up) 176 | p5up = self.p5_l2(p5up) 177 | cat = torch.cat([p3up, p4up, p5up], dim=1) 178 | 179 | feat = self.feat(cat) 180 | feat = self.feat_bn(feat) 181 | feat = self.relu(feat) 182 | 183 | x_cls = self.center_conv(feat) 184 | x_cls = torch.sigmoid(x_cls) 185 | x_reg = self.height_conv(feat) 186 | x_off = self.offset_conv(feat) 187 | 188 | x_cls = x_cls.permute(0, 2, 3, 1) 189 | x_reg = x_reg.permute(0, 2, 3, 1) 190 | x_off = x_off.permute(0, 2, 3, 1) 191 | 192 | return x_cls, x_reg, x_off 193 | 194 | # def train(self, mode=True): 195 | # # Override train so that the training mode is set as we want 196 | # nn.Module.train(self, mode) 197 | # if mode: 198 | # # Set fixed blocks to be in eval mode 199 | # self.conv1.eval() 200 | # self.layer1.eval() 201 | # 202 | # # bn is trainable in CONV2 203 | # def set_bn_train(m): 204 | # class_name = m.__class__.__name__ 205 | # if class_name.find('BatchNorm') != -1: 206 | # m.train() 207 | # else: 208 | # m.eval() 209 | # self.layer1.apply(set_bn_train) 210 | 211 | -------------------------------------------------------------------------------- /models/l2norm.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | from torch.autograd import Function 4 | from torch.autograd import Variable 5 | import torch.nn.init as init 6 | 7 | 8 | class L2Norm(nn.Module): 9 | def __init__(self, n_channels, scale): 10 | super(L2Norm, self).__init__() 11 | self.n_channels = n_channels 12 | self.gamma = scale or None 13 | self.eps = 1e-10 14 | self.weight = nn.Parameter(torch.Tensor(self.n_channels)) 15 | self.reset_parameters() 16 | 17 | def reset_parameters(self): 18 | init.constant_(self.weight, self.gamma) 19 | 20 | def forward(self, x): 21 | norm = x.pow(2).sum(dim=1, keepdim=True).sqrt()+self.eps 22 | x = torch.div(x, norm) 23 | out = self.weight.unsqueeze(0).unsqueeze(2).unsqueeze(3).expand_as(x) * x 24 | return out 25 | -------------------------------------------------------------------------------- /models/samepad.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.functional as F 4 | import math 5 | 6 | class SamePad2d(nn.Module): 7 | """Mimics tensorflow's 'SAME' padding. 8 | """ 9 | 10 | def __init__(self, kernel_size, stride, dilation=1): 11 | super(SamePad2d, self).__init__() 12 | self.kernel_size = torch.nn.modules.utils._pair(kernel_size) 13 | self.stride = torch.nn.modules.utils._pair(stride) 14 | self.dilation = torch.nn.modules.utils._pair(dilation) 15 | 16 | def forward(self, input): 17 | in_width = input.size()[3] 18 | in_height = input.size()[2] 19 | out_width = math.ceil(float(in_width) / float(self.stride[0])) 20 | out_height = math.ceil(float(in_height) / float(self.stride[1])) 21 | 22 | effective_kernel_size_width = (self.kernel_size[0] - 1) * self.dilation[0] + 1 23 | effective_kernel_size_height = (self.kernel_size[1] - 1) * self.dilation[1] + 1 24 | 25 | pad_along_width = ((out_width - 1) * self.stride[0] + 26 | effective_kernel_size_width - in_width) 27 | pad_along_height = ((out_height - 1) * self.stride[1] + 28 | effective_kernel_size_height - in_height) 29 | pad_left = math.floor(pad_along_width / 2) 30 | pad_top = math.floor(pad_along_height / 2) 31 | pad_right = pad_along_width - pad_left 32 | pad_bottom = pad_along_height - pad_top 33 | return F.pad(input, (pad_left, pad_right, pad_top, pad_bottom), 'constant', 0) 34 | 35 | def __repr__(self): 36 | return self.__class__.__name__ 37 | -------------------------------------------------------------------------------- /shibuye.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wang-xinyu/csp.pytorch/8d5187358c1eb0fbf7ba5f184b9afed6c2518ad3/shibuye.jpg -------------------------------------------------------------------------------- /test_caltech.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import cv2 3 | import numpy as np 4 | import os 5 | from models.cspnet import CSPNet_p3p4p5 6 | from utils.keras_weights_loader import load_keras_weights 7 | from utils.utils import * 8 | 9 | if __name__ == '__main__': 10 | device = 'cuda:0' 11 | weights_path = 'net_e82_l0.00850005054218.hdf5' 12 | out_path = 'output/valresults/caltech/h/off/82' 13 | input_dim = [480, 640] 14 | 15 | if not os.path.exists(out_path): 16 | os.makedirs(out_path) 17 | 18 | for st in range(6, 11): 19 | set_path = os.path.join(out_path, 'set' + '%02d' % st) 20 | if not os.path.exists(set_path): 21 | os.mkdir(set_path) 22 | 23 | model = CSPNet_p3p4p5() 24 | load_keras_weights(model, weights_path) 25 | model.to(device).eval() 26 | 27 | f = open('data/caltech/test.txt', 'r') 28 | files = f.readlines(); 29 | num_imgs = len(files) 30 | 31 | for i in range(0, num_imgs): 32 | l = files[i] 33 | print(l) 34 | st = l.split('_')[0] 35 | video = l.split('_')[1] 36 | 37 | frame_number = int(l.split('_')[2][1:6]) + 1 38 | frame_number_next = int(files[i + 1].split('_')[2][1:6]) + 1 if i < num_imgs - 1 else -1 39 | print('next', frame_number_next) 40 | set_path = os.path.join(out_path, st) 41 | video_path = os.path.join(set_path, video + '.txt') 42 | 43 | if os.path.exists(video_path): 44 | continue 45 | if frame_number == 30: 46 | res_all = [] 47 | 48 | img = cv2.imread('data/caltech/images/' + l.strip()) 49 | x = format_img(img) 50 | x = torch.from_numpy(x).to(device) 51 | x = x.permute(0, 3, 1, 2) 52 | x_cls, x_reg, x_off = model(x) 53 | Y = [x_cls.detach().cpu().numpy(), x_reg.detach().cpu().numpy(), x_off.detach().cpu().numpy()] 54 | boxes = parse_det_offset(Y, input_dim, score=0.01, down=4) 55 | 56 | if len(boxes)>0: 57 | f_res = np.repeat(frame_number, len(boxes), axis=0).reshape((-1, 1)) 58 | boxes[:, [2, 3]] -= boxes[:, [0, 1]] 59 | res_all += np.concatenate((f_res, boxes), axis=-1).tolist() 60 | if frame_number_next == 30 or i == num_imgs - 1: 61 | np.savetxt(video_path, np.array(res_all), fmt='%6f') 62 | 63 | f.close() 64 | exit(0) 65 | 66 | -------------------------------------------------------------------------------- /test_city.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import cv2 3 | import numpy as np 4 | import os 5 | from models.cspnet import CSPNet_p3p4p5 6 | from utils.keras_weights_loader import load_keras_weights 7 | from utils.utils import * 8 | 9 | if __name__ == '__main__': 10 | device = 'cuda:0' 11 | weights_path = 'net_e121_l0.hdf5' 12 | out_path = 'output/valresults/city/h/off/121' 13 | input_dim = [1024, 2048] 14 | 15 | if not os.path.exists(out_path): 16 | os.makedirs(out_path) 17 | 18 | res_file = os.path.join(out_path, 'val_det.txt') 19 | 20 | model = CSPNet_p3p4p5() 21 | load_keras_weights(model, weights_path) 22 | model.to(device).eval() 23 | 24 | f = open('data/citypersons/val.txt', 'r') 25 | files = f.readlines(); 26 | num_imgs = len(files) 27 | 28 | res_all = [] 29 | for i in range(0, num_imgs): 30 | l = files[i] 31 | print(l) 32 | 33 | img = cv2.imread('data/citypersons/leftImg8bit/val/' + l.strip()) 34 | x = format_img(img) 35 | with torch.no_grad(): 36 | x = torch.from_numpy(x).to(device) 37 | x = x.permute(0, 3, 1, 2) 38 | x_cls, x_reg, x_off = model(x) 39 | Y = [x_cls.detach().cpu().numpy(), x_reg.detach().cpu().numpy(), x_off.detach().cpu().numpy()] 40 | boxes = parse_det_offset(Y, input_dim, score=0.1, down=4) 41 | 42 | if len(boxes)>0: 43 | f_res = np.repeat(i + 1, len(boxes), axis=0).reshape((-1, 1)) 44 | boxes[:, [2, 3]] -= boxes[:, [0, 1]] 45 | res_all += np.concatenate((f_res, boxes), axis=-1).tolist() 46 | np.savetxt(res_file, np.array(res_all), fmt='%6f') 47 | 48 | f.close() 49 | exit(0) 50 | 51 | -------------------------------------------------------------------------------- /test_one_pic_face.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import cv2 3 | import numpy as np 4 | from models.cspnet import CSPNet_p3p4p5 5 | from utils.keras_weights_loader import load_keras_weights 6 | from utils.utils import * 7 | 8 | device = 'cuda:1' 9 | 10 | def detect_face(model, img, scale=1, flip=False): 11 | img_h, img_w = img.shape[:2] 12 | img_h_new, img_w_new = int(np.ceil(scale * img_h / 16) * 16), int(np.ceil(scale * img_w / 16) * 16) 13 | scale_h, scale_w = img_h_new / img_h, img_w_new / img_w 14 | 15 | img_s = cv2.resize(img, None, None, fx=scale_w, fy=scale_h, interpolation=cv2.INTER_LINEAR) 16 | # img_h, img_w = img_s.shape[:2] 17 | # print frame_number 18 | input_dim = [img_h_new, img_w_new] 19 | 20 | if flip: 21 | img_sf = cv2.flip(img_s, 1) 22 | # x_rcnn = format_img_pad(img_sf, C) 23 | x_rcnn = format_img(img_sf) 24 | else: 25 | # x_rcnn = format_img_pad(img_s, C) 26 | x_rcnn = format_img(img_s) 27 | x = torch.from_numpy(x_rcnn).to(device) 28 | x = x.permute(0, 3, 1, 2) 29 | x_cls, x_reg, x_off = model(x) 30 | print('x reg shape ', x_reg.shape) 31 | Y = [x_cls.detach().cpu().numpy(), x_reg.detach().cpu().numpy(), x_off.detach().cpu().numpy()] 32 | boxes = parse_wider_offset(Y, input_dim, score=0.3, nmsthre=0.4) 33 | if len(boxes) > 0: 34 | keep_index = np.where(np.minimum(boxes[:, 2] - boxes[:, 0], boxes[:, 3] - boxes[:, 1]) >= 12)[0] 35 | boxes = boxes[keep_index, :] 36 | if len(boxes) > 0: 37 | if flip: 38 | boxes[:, [0, 2]] = img_s.shape[1] - boxes[:, [2, 0]] 39 | boxes[:, 0:4:2] = boxes[:, 0:4:2] / scale_w 40 | boxes[:, 1:4:2] = boxes[:, 1:4:2] / scale_h 41 | else: 42 | boxes = np.empty(shape=[0, 5], dtype=np.float32) 43 | return boxes 44 | 45 | 46 | 47 | if __name__ == '__main__': 48 | 49 | model = CSPNet_p3p4p5(num_scale=2) 50 | load_keras_weights(model, 'net_e382_l0.hdf5') 51 | model.to(device).eval() 52 | 53 | img = cv2.imread('worlds-largest-selfie.jpg') 54 | bboxes = detect_face(model, img) 55 | 56 | print('bbox----', bboxes.shape) 57 | for b in bboxes: 58 | cv2.rectangle(img, (int(b[0]), int(b[1])), (int(b[2]), int(b[3])), (0, 0, 255), 2) 59 | cv2.imwrite('out.jpg', img) 60 | -------------------------------------------------------------------------------- /test_one_pic_pedestrian.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import cv2 3 | import numpy as np 4 | from models.cspnet import CSPNet_p3p4p5 5 | from utils.keras_weights_loader import load_keras_weights 6 | from utils.utils import * 7 | 8 | if __name__ == '__main__': 9 | device = 'cuda:0' 10 | img = cv2.imread('shibuye.jpg') 11 | x = format_img(img) 12 | #x = np.load('/home/user/wangxinyu/CSP/filename.npy') 13 | input_dim = x.shape[1:3] 14 | print(x.shape) 15 | #print(x) 16 | #exit(0) 17 | #x = np.ones([1, 4, 4, 3], dtype='float32') 18 | x = torch.from_numpy(x).to(device) 19 | x = x.permute(0, 3, 1, 2) 20 | 21 | model = CSPNet_p3p4p5() 22 | load_keras_weights(model, 'net_e82_l0.00850005054218.hdf5') 23 | model.to(device).eval() 24 | 25 | x_cls, x_reg, x_off = model(x) 26 | Y = [x_cls.detach().cpu().numpy(), x_reg.detach().cpu().numpy(), x_off.detach().cpu().numpy()] 27 | bboxes = parse_det_offset(Y, input_dim, score=0.4, down=4) 28 | 29 | print('bbox----', bboxes.shape) 30 | for b in bboxes: 31 | cv2.rectangle(img, (b[0], b[1]), (b[2], b[3]), (0, 0, 255), 2) 32 | cv2.imwrite('out.jpg', img) 33 | -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wang-xinyu/csp.pytorch/8d5187358c1eb0fbf7ba5f184b9afed6c2518ad3/utils/__init__.py -------------------------------------------------------------------------------- /utils/keras_weights_loader.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import h5py 3 | import numpy as np 4 | #from cspnet import CSPNet_p3p4p5, ConvBlock 5 | 6 | def load_conv_weights(conv, f, layer_name): 7 | w = np.asarray(f[layer_name][layer_name + '_1/kernel:0'], dtype='float32') 8 | b = np.asarray(f[layer_name][layer_name + '_1/bias:0'], dtype='float32') 9 | conv.weight = torch.nn.Parameter(torch.from_numpy(w).permute(3, 2, 0, 1)) 10 | conv.bias = torch.nn.Parameter(torch.from_numpy(b)) 11 | 12 | def load_bn_weights(bn, f, layer_name): 13 | w = np.asarray(f[layer_name][layer_name + '_1/gamma:0'], dtype='float32') 14 | b = np.asarray(f[layer_name][layer_name + '_1/beta:0'], dtype='float32') 15 | m = np.asarray(f[layer_name][layer_name + '_1/moving_mean:0'], dtype='float32') 16 | v = np.asarray(f[layer_name][layer_name + '_1/moving_variance:0'], dtype='float32') 17 | bn.weight = torch.nn.Parameter(torch.from_numpy(w)) 18 | bn.bias = torch.nn.Parameter(torch.from_numpy(b)) 19 | bn.running_mean = torch.from_numpy(m) 20 | bn.running_var = torch.from_numpy(v) 21 | 22 | def load_conv_block_weights(conv_blk, f, blk_name): 23 | load_conv_weights(conv_blk.conv1, f, 'res' + blk_name + '_branch2a') 24 | load_bn_weights(conv_blk.bn1, f, 'bn' + blk_name + '_branch2a') 25 | load_conv_weights(conv_blk.conv2, f, 'res' + blk_name + '_branch2b') 26 | load_bn_weights(conv_blk.bn2, f, 'bn' + blk_name + '_branch2b') 27 | load_conv_weights(conv_blk.conv3, f, 'res' + blk_name + '_branch2c') 28 | load_bn_weights(conv_blk.bn3, f, 'bn' + blk_name + '_branch2c') 29 | load_conv_weights(conv_blk.conv4, f, 'res' + blk_name + '_branch1') 30 | load_bn_weights(conv_blk.bn4, f, 'bn' + blk_name + '_branch1') 31 | 32 | def load_identity_block_weights(identity_blk, f, blk_name): 33 | load_conv_weights(identity_blk.conv1, f, 'res' + blk_name + '_branch2a') 34 | load_bn_weights(identity_blk.bn1, f, 'bn' + blk_name + '_branch2a') 35 | load_conv_weights(identity_blk.conv2, f, 'res' + blk_name + '_branch2b') 36 | load_bn_weights(identity_blk.bn2, f, 'bn' + blk_name + '_branch2b') 37 | load_conv_weights(identity_blk.conv3, f, 'res' + blk_name + '_branch2c') 38 | load_bn_weights(identity_blk.bn3, f, 'bn' + blk_name + '_branch2c') 39 | 40 | def load_l2norm_weights(l2norm, f, layer_name): 41 | w = np.asarray(f[layer_name][layer_name + '_1/' + layer_name + '_gamma:0'], dtype='float32') 42 | l2norm.weight = torch.nn.Parameter(torch.from_numpy(w)) 43 | 44 | def load_keras_weights(model, weights_path): 45 | with h5py.File(weights_path, 'r') as f: 46 | print(f.attrs['layer_names']) 47 | 48 | load_conv_weights(model.conv1, f, 'conv1') 49 | load_bn_weights(model.bn1, f, 'bn_conv1') 50 | 51 | load_conv_block_weights(model.convblk2a, f, '2a') 52 | load_identity_block_weights(model.identityblk2b, f, '2b') 53 | load_identity_block_weights(model.identityblk2c, f, '2c') 54 | 55 | load_conv_block_weights(model.convblk3a, f, '3a') 56 | load_identity_block_weights(model.identityblk3b, f, '3b') 57 | load_identity_block_weights(model.identityblk3c, f, '3c') 58 | load_identity_block_weights(model.identityblk3d, f, '3d') 59 | 60 | load_conv_block_weights(model.convblk4a, f, '4a') 61 | load_identity_block_weights(model.identityblk4b, f, '4b') 62 | load_identity_block_weights(model.identityblk4c, f, '4c') 63 | load_identity_block_weights(model.identityblk4d, f, '4d') 64 | load_identity_block_weights(model.identityblk4e, f, '4e') 65 | load_identity_block_weights(model.identityblk4f, f, '4f') 66 | 67 | load_conv_block_weights(model.convblk5a, f, '5a') 68 | load_identity_block_weights(model.identityblk5b, f, '5b') 69 | load_identity_block_weights(model.identityblk5c, f, '5c') 70 | 71 | load_conv_weights(model.p3, f, 'P3up') 72 | load_conv_weights(model.p4, f, 'P4up') 73 | load_conv_weights(model.p5, f, 'P5up') 74 | 75 | load_l2norm_weights(model.p3_l2, f, 'P3norm') 76 | load_l2norm_weights(model.p4_l2, f, 'P4norm') 77 | load_l2norm_weights(model.p5_l2, f, 'P5norm') 78 | 79 | load_conv_weights(model.feat, f, 'feat') 80 | load_bn_weights(model.feat_bn, f, 'bn_feat') 81 | 82 | load_conv_weights(model.center_conv, f, 'center_cls') 83 | load_conv_weights(model.height_conv, f, 'height_regr') 84 | load_conv_weights(model.offset_conv, f, 'offset_regr') 85 | 86 | -------------------------------------------------------------------------------- /utils/py_cpu_nms.py: -------------------------------------------------------------------------------- 1 | # -------------------------------------------------------- 2 | # Fast R-CNN 3 | # Copyright (c) 2015 Microsoft 4 | # Licensed under The MIT License [see LICENSE for details] 5 | # Written by Ross Girshick 6 | # -------------------------------------------------------- 7 | 8 | import numpy as np 9 | 10 | def py_cpu_nms(dets, thresh): 11 | """Pure Python NMS baseline.""" 12 | x1 = dets[:, 0] 13 | y1 = dets[:, 1] 14 | x2 = dets[:, 2] 15 | y2 = dets[:, 3] 16 | scores = dets[:, 4] 17 | 18 | areas = (x2 - x1 + 1) * (y2 - y1 + 1) 19 | order = scores.argsort()[::-1] 20 | 21 | keep = [] 22 | while order.size > 0: 23 | i = order[0] 24 | keep.append(i) 25 | xx1 = np.maximum(x1[i], x1[order[1:]]) 26 | yy1 = np.maximum(y1[i], y1[order[1:]]) 27 | xx2 = np.minimum(x2[i], x2[order[1:]]) 28 | yy2 = np.minimum(y2[i], y2[order[1:]]) 29 | 30 | w = np.maximum(0.0, xx2 - xx1 + 1) 31 | h = np.maximum(0.0, yy2 - yy1 + 1) 32 | inter = w * h 33 | ovr = inter / (areas[i] + areas[order[1:]] - inter) 34 | 35 | inds = np.where(ovr <= thresh)[0] 36 | order = order[inds + 1] 37 | 38 | return keep 39 | -------------------------------------------------------------------------------- /utils/utils.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from .py_cpu_nms import py_cpu_nms 3 | 4 | 5 | img_channel_mean = [103.939, 116.779, 123.68] 6 | 7 | def format_img_channels(img): 8 | """ formats the image channels based on config """ 9 | # img = img[:, :, (2, 1, 0)] 10 | img = img.astype(np.float32) 11 | img[:, :, 0] -= img_channel_mean[0] 12 | img[:, :, 1] -= img_channel_mean[1] 13 | img[:, :, 2] -= img_channel_mean[2] 14 | # img /= C.img_scaling_factor 15 | # img = np.transpose(img, (2, 0, 1)) 16 | img = np.expand_dims(img, axis=0) 17 | return img 18 | 19 | def format_img(img): 20 | """ formats an image for model prediction based on config """ 21 | img = format_img_channels(img) 22 | return img #return img, ratio 23 | 24 | def parse_det_offset(Y, input_dim, score=0.1,down=4): 25 | seman = Y[0][0, :, :, 0] 26 | height = Y[1][0, :, :, 0] 27 | offset_y = Y[2][0, :, :, 0] 28 | offset_x = Y[2][0, :, :, 1] 29 | y_c, x_c = np.where(seman > score) 30 | print(input_dim) 31 | print(seman.shape) 32 | print(y_c.shape) 33 | print(x_c.shape) 34 | boxs = [] 35 | if len(y_c) > 0: 36 | for i in range(len(y_c)): 37 | h = np.exp(height[y_c[i], x_c[i]]) * down 38 | w = 0.41*h 39 | o_y = offset_y[y_c[i], x_c[i]] 40 | o_x = offset_x[y_c[i], x_c[i]] 41 | s = seman[y_c[i], x_c[i]] 42 | x1, y1 = max(0, (x_c[i] + o_x + 0.5) * down - w / 2), max(0, (y_c[i] + o_y + 0.5) * down - h / 2) 43 | boxs.append([x1, y1, min(x1 + w, input_dim[1]), min(y1 + h, input_dim[0]), s]) 44 | boxs = np.asarray(boxs, dtype=np.float32) 45 | keep = py_cpu_nms(boxs, 0.5) 46 | boxs = boxs[keep, :] 47 | return boxs 48 | 49 | def soft_bbox_vote(det,thre=0.35,score=0.05): 50 | if det.shape[0] <= 1: 51 | return det 52 | order = det[:, 4].ravel().argsort()[::-1] 53 | det = det[order, :] 54 | dets = [] 55 | while det.shape[0] > 0: 56 | # IOU 57 | area = (det[:, 2] - det[:, 0] + 1) * (det[:, 3] - det[:, 1] + 1) 58 | xx1 = np.maximum(det[0, 0], det[:, 0]) 59 | yy1 = np.maximum(det[0, 1], det[:, 1]) 60 | xx2 = np.minimum(det[0, 2], det[:, 2]) 61 | yy2 = np.minimum(det[0, 3], det[:, 3]) 62 | w = np.maximum(0.0, xx2 - xx1 + 1) 63 | h = np.maximum(0.0, yy2 - yy1 + 1) 64 | inter = w * h 65 | o = inter / (area[0] + area[:] - inter) 66 | 67 | # get needed merge det and delete these det 68 | merge_index = np.where(o >= thre)[0] 69 | det_accu = det[merge_index, :] 70 | det_accu_iou = o[merge_index] 71 | det = np.delete(det, merge_index, 0) 72 | 73 | if merge_index.shape[0] <= 1: 74 | try: 75 | dets = np.row_stack((dets, det_accu)) 76 | except: 77 | dets = det_accu 78 | continue 79 | else: 80 | soft_det_accu = det_accu.copy() 81 | soft_det_accu[:, 4] = soft_det_accu[:, 4] * (1 - det_accu_iou) 82 | soft_index = np.where(soft_det_accu[:, 4] >= score)[0] 83 | soft_det_accu = soft_det_accu[soft_index, :] 84 | 85 | det_accu[:, 0:4] = det_accu[:, 0:4] * np.tile(det_accu[:, -1:], (1, 4)) 86 | max_score = np.max(det_accu[:, 4]) 87 | det_accu_sum = np.zeros((1, 5)) 88 | det_accu_sum[:, 0:4] = np.sum(det_accu[:, 0:4], axis=0) / np.sum(det_accu[:, -1:]) 89 | det_accu_sum[:, 4] = max_score 90 | 91 | if soft_det_accu.shape[0] > 0: 92 | det_accu_sum = np.row_stack((soft_det_accu, det_accu_sum)) 93 | 94 | try: 95 | dets = np.row_stack((dets, det_accu_sum)) 96 | except: 97 | dets = det_accu_sum 98 | 99 | order = dets[:, 4].ravel().argsort()[::-1] 100 | dets = dets[order, :] 101 | return dets 102 | 103 | def parse_wider_offset(Y, input_dim, score=0.1,down=4,nmsthre=0.5): 104 | seman = Y[0][0, :, :, 0] 105 | height = Y[1][0, :, :, 0] 106 | width = Y[1][0, :, :, 1] 107 | offset_y = Y[2][0, :, :, 0] 108 | offset_x = Y[2][0, :, :, 1] 109 | y_c, x_c = np.where(seman > score) 110 | boxs = [] 111 | if len(y_c) > 0: 112 | for i in range(len(y_c)): 113 | h = np.exp(height[y_c[i], x_c[i]]) * down 114 | w = np.exp(width[y_c[i], x_c[i]]) * down 115 | o_y = offset_y[y_c[i], x_c[i]] 116 | o_x = offset_x[y_c[i], x_c[i]] 117 | s = seman[y_c[i], x_c[i]] 118 | x1, y1 = max(0, (x_c[i] + o_x + 0.5) * down - w / 2), max(0, (y_c[i] + o_y + 0.5) * down - h / 2) 119 | x1, y1 = min(x1, input_dim[1]), min(y1, input_dim[0]) 120 | boxs.append([x1, y1, min(x1 + w, input_dim[1]), min(y1 + h, input_dim[0]), s]) 121 | boxs = np.asarray(boxs, dtype=np.float32) 122 | #keep = nms(boxs, nmsthre, usegpu=False, gpu_id=0) 123 | #boxs = boxs[keep, :] 124 | boxs = soft_bbox_vote(boxs,thre=0.3, score=0.7) 125 | return boxs 126 | 127 | 128 | --------------------------------------------------------------------------------