├── LICENSE.txt ├── README.md ├── materials_for_face ├── ir235_release.prototxt └── solver_train.prototxt ├── mnist_data.mat ├── model ├── center+softmax_loss │ └── 10epoches.caffemodel ├── coco_loss │ └── 10epoches.caffemodel ├── net_test.prototxt ├── softmax_loss │ └── 10epoches.caffemodel ├── solver_test.prototxt └── train_prototxts │ ├── net_centerloss.prototxt │ ├── net_cocoloss.prototxt │ ├── net_softmax.prototxt │ ├── solver_centerloss.prototxt │ ├── solver_cocoloss.prototxt │ └── solver_softmax.prototxt ├── output_sample.jpg └── script_show_minist.m /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Hongyang Li and Yu Liu 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # COCO Loss 2 | 3 | **Update:** we have released the network structure for face recognition. And we answer some common questions in Q&A. 4 | 5 | *Update:* we have an updated version of this work, which we call **COCO_v2** [[arXiv]](https://arxiv.org/pdf/1710.00870.pdf). The **difference from v1 to v2** is that, in a short story, we (a) propose an optimal scale factor imposed on the input feature before COCO loss; (b) experiments are extended to large-scale face recognition datasets beyond PIPA. 6 | 7 | The codebase is **UPGRADED to COCO_v2.** :) 8 | 9 | 10 | Codebase for Learning Deep Features via Congenerous Cosine Loss for Person Recognition (COCO_v1), [[arXiv]](https://arxiv.org/abs/1702.06890). In the paper, we use the newly proposed [PIPA](https://people.eecs.berkeley.edu/~nzhang/piper.html) dataset for evaluation; it is a large-scale dataset for person recognition. 11 | 12 | We provide a sample code for generating distributions of COCO Loss, Softmax Loss and Center Loss as Figure 3 in our paper. Please follow the steps below: 13 | 14 | ## Runing sample code 15 | 16 | 1. Compile [CaffeMex_v2](https://github.com/sciencefans/CaffeMex_v2/) with matlab interface 17 | 18 | 2. Link (Linux) or copy (Windows) the path `CaffeMex_v2/matlab/+caffe` to the root directory in this project 19 | 20 | 3. Run `script_show_minist.m` 21 | 22 | 4. It takes just few seconds to generate three figures as: 23 | 24 | ![](output_sample.jpg) 25 | 26 | Notice that the distributions are generated based on caffemodels in `model/[LOSS_NAME]`. You can reproduce the result by using network definition files in `model/train_prototxts/`. 27 | 28 | 29 | The proposed COCO loss layer (known as `center_projection_layer.cu` [here](https://github.com/sciencefans/CaffeMex_v2/blob/master/src/caffe/layers/center_projection_layer.cu)) and the normalized layer ([here](https://github.com/sciencefans/CaffeMex_v2/blob/master/src/caffe/layers/normalize_layer.cu)) are two operations related wiht COCO. The gradient derivation in the initial release (v1) on arXiv is **wrong**; please resort to v2 (latest version): Eqn.(8) in v2 paper corresponds to the implementation around lines [here](https://github.com/sciencefans/CaffeMex_v2/blob/master/src/caffe/layers/normalize_layer.cpp#L55). 30 | 31 | ## Q&A 32 | 33 | Here we list some commonly asked questions we received from the public. Thanks for your engagement to make our work matter! 34 | 35 | - *Relationship of person recognition task with face recognition, person Re-ID.* 36 | 37 | Face recognition is the traditional vision task to recognize a person's identify based solely on face (often with some setting constraints, like frontal faces); whereas the context of person recognition carries more. The main difference between person recognition and re-id resides in the data logistics. The former is to identify the same person across places and time. In most cases, the identity varies a lot in appearance under different occasions. The latter is to detect person in a consecutive video, meaning that the appearance and background do not vary much in terms of time. 38 | 39 | - *Training curve looks weird. Any tricks to make COCO work?* 40 | 41 | Hah, good question. There are mainly two. First is we initialize the network with a pre-trained model for the re-id task in order for the loss to converge well in a large-scale dataset like PIPA; second is we do not normalize the feature to 1. There are some numerical problems for gradients of centroids if we normalize all features and weights to 1. In MNIST, we initialize the scaler to 1 for weights and 2 for features while in PAPA we set 30 and 100, respectively. 42 | 43 | - **[New]** *How could COCO achieves 99.86% on LFW with 10 incorrect annotations?* 44 | 45 | Some people represent the issue of 'upper limit accuracy' of LFW such as [this issue](https://github.com/sciencefans/coco_loss/issues/9). We have to indicate that there are 10 incorrect pairs on LFW but only the **6 matching pairs** are 'really incorrect'. The faces in the other **4 mismatching pairs** are labeled with wrong identities but they are still *mismatching*. So the upper limit accuracy is *(6000-6)/6000=99.90%*. For more details please refer to [Errata of LFW](http://vis-www.cs.umass.edu/lfw/index.html). 46 | 47 | - **[New]** *Any comments on the evalutation of LFW and MegaFace?* 48 | 49 | We believe that both LFW and MegaFace are not precise benchmark for face recognition/verification. For LFW, most of the recent CNN based algorithms get the saturation area of this benchmark. For MegaFace, the top-1 accuracy introduces an inverse correlation with the accuracy of face detector. That is, the worse of your face detector, the less real faces you will detect in distractors, so there will be more meaningless background in distractors and the top-1 accuracy will be higher. In our work, we use the commercial version of [RSA](https://github.com/walkoncross/RSA-for-object-detection) to be our detector and we ensure that all the faces we detected are real faces. For the images we didn't detect a face, none of them includes faces larger than 10x10 (in fact, most of them are entire background). Above all, we appeal to the researchers in area of face recognition to pay more attention on ablation study but not the absolute value of accuracy. 50 | 51 | - **[New]** *Will the models for face recognition be released in future?* 52 | 53 | The network structure is released but the trained model will not be released at least in recent days. Since the alignment algorithm and the cleaned MS1M are classified and copyright by Sensetime Group Limited. 54 | 55 | 56 | 59 | 60 | ## Still having questions? 61 | 62 | Feel free to drop us an email sharing your ideas. 63 | 64 | ## Related work 65 | 66 | Please refer to the paper for details. 67 | 68 | - Feature Incay for Representation Regularization, [[link]](https://arxiv.org/abs/1705.10284), *arXiv, 2017*. 69 | - Pose-Aware Person Recognition, [[link]](https://arxiv.org/pdf/1705.10120.pdf), *CVPR, 2017*. 70 | - Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks, [[link]](https://arxiv.org/abs/1702.05870), *arXiv, 2017*. 71 | - Large-Margin Softmax Loss for Convolutional Neural Networks, [[link]](http://jmlr.org/proceedings/papers/v48/liud16.pdf), *ICML, 2016*. 72 | 73 | ## Citation 74 | Please kindly cite our work in your publications if it helps your research: 75 | 76 | @article{liu_2017_coco_v2, 77 | Author = {Liu, Yu and Li, Hongyang and Wang, Xiaogang}, 78 | Title = {Rethinking Feature Discrimination and Polymerization for Large-scale Recognition}, 79 | journal={arXiv preprint arXiv:1710.00870}, 80 | Year = {2017} 81 | } 82 | 83 | @article{liu_2017_coco_v1, 84 | Author = {Liu, Yu and Li, Hongyang and Wang, Xiaogang}, 85 | Title = {Learning Deep Features via Congenerous Cosine Loss for Person Recognition}, 86 | Journal = {arXiv preprint: 1702.06890}, 87 | Year = {2017} 88 | } 89 | -------------------------------------------------------------------------------- /materials_for_face/solver_train.prototxt: -------------------------------------------------------------------------------- 1 | net: "net/ir235_ws_nobnsync_21475/train_val.prototxt" 2 | base_lr: 0.001 3 | lr_policy: "multistep" 4 | gamma: 0.316 5 | stepvalue: 156250 6 | stepvalue: 234375 7 | stepvalue: 312500 8 | stepvalue: 350000 9 | max_iter: 400000 10 | display: 0 11 | momentum: 0.9 12 | weight_decay: 0.0001 13 | snapshot: 0 14 | #debug_info: true 15 | 16 | -------------------------------------------------------------------------------- /mnist_data.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/liuyuisanai/coco_loss/4051c62817756b566911fc7f1154beda87b0ef72/mnist_data.mat -------------------------------------------------------------------------------- /model/center+softmax_loss/10epoches.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/liuyuisanai/coco_loss/4051c62817756b566911fc7f1154beda87b0ef72/model/center+softmax_loss/10epoches.caffemodel -------------------------------------------------------------------------------- /model/coco_loss/10epoches.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/liuyuisanai/coco_loss/4051c62817756b566911fc7f1154beda87b0ef72/model/coco_loss/10epoches.caffemodel -------------------------------------------------------------------------------- /model/net_test.prototxt: -------------------------------------------------------------------------------- 1 | name: "LeNet++" 2 | 3 | input: "data" 4 | input_dim: 1 5 | input_dim: 1 6 | input_dim: 28 7 | input_dim: 28 8 | 9 | layer { 10 | name: "conv1" 11 | type: "Convolution" 12 | bottom: "data" 13 | top: "conv1" 14 | param { 15 | lr_mult: 1 16 | decay_mult: 1 17 | } 18 | param { 19 | lr_mult: 2 20 | decay_mult: 0 21 | } 22 | convolution_param { 23 | num_output: 32 24 | kernel_size: 5 25 | stride: 1 26 | pad: 2 27 | weight_filler { 28 | type: "xavier" 29 | } 30 | bias_filler { 31 | type: "constant" 32 | value: 0 33 | } 34 | } 35 | } 36 | layer { 37 | name: "prelu1" 38 | type: "PReLU" 39 | bottom: "conv1" 40 | top: "conv1" 41 | } 42 | layer { 43 | name: "conv1+" 44 | type: "Convolution" 45 | bottom: "conv1" 46 | top: "conv1+" 47 | param { 48 | lr_mult: 1 49 | decay_mult: 1 50 | } 51 | param { 52 | lr_mult: 2 53 | decay_mult: 0 54 | } 55 | convolution_param { 56 | num_output: 32 57 | kernel_size: 5 58 | stride: 1 59 | pad: 2 60 | weight_filler { 61 | type: "xavier" 62 | } 63 | bias_filler { 64 | type: "constant" 65 | value: 0 66 | } 67 | } 68 | } 69 | layer { 70 | name: "prelu1+" 71 | type: "PReLU" 72 | bottom: "conv1+" 73 | top: "conv1+" 74 | } 75 | layer { 76 | name: "pool1" 77 | type: "Pooling" 78 | bottom: "conv1+" 79 | top: "pool1" 80 | pooling_param { 81 | pool: MAX 82 | kernel_size: 2 83 | stride: 2 84 | } 85 | } 86 | layer { 87 | name: "conv2" 88 | type: "Convolution" 89 | bottom: "pool1" 90 | top: "conv2" 91 | param { 92 | lr_mult: 1 93 | decay_mult: 1 94 | } 95 | param { 96 | lr_mult: 2 97 | decay_mult: 0 98 | } 99 | convolution_param { 100 | num_output: 64 101 | kernel_size: 5 102 | stride: 1 103 | pad: 2 104 | weight_filler { 105 | type: "xavier" 106 | } 107 | bias_filler { 108 | type: "constant" 109 | value: 0 110 | } 111 | } 112 | } 113 | layer { 114 | name: "prelu2" 115 | type: "PReLU" 116 | bottom: "conv2" 117 | top: "conv2" 118 | } 119 | layer { 120 | name: "conv2+" 121 | type: "Convolution" 122 | bottom: "conv2" 123 | top: "conv2+" 124 | param { 125 | lr_mult: 1 126 | decay_mult: 1 127 | } 128 | param { 129 | lr_mult: 2 130 | decay_mult: 0 131 | } 132 | convolution_param { 133 | num_output: 64 134 | kernel_size: 5 135 | stride: 1 136 | pad: 2 137 | weight_filler { 138 | type: "xavier" 139 | } 140 | bias_filler { 141 | type: "constant" 142 | value: 0 143 | } 144 | } 145 | } 146 | layer { 147 | name: "prelu2+" 148 | type: "PReLU" 149 | bottom: "conv2+" 150 | top: "conv2+" 151 | } 152 | layer { 153 | name: "pool2" 154 | type: "Pooling" 155 | bottom: "conv2+" 156 | top: "pool2" 157 | pooling_param { 158 | pool: MAX 159 | kernel_size: 2 160 | stride: 2 161 | } 162 | } 163 | layer { 164 | name: "conv3" 165 | type: "Convolution" 166 | bottom: "pool2" 167 | top: "conv3" 168 | param { 169 | lr_mult: 1 170 | decay_mult: 1 171 | } 172 | param { 173 | lr_mult: 2 174 | decay_mult: 0 175 | } 176 | convolution_param { 177 | num_output: 128 178 | kernel_size: 5 179 | stride: 1 180 | pad: 2 181 | weight_filler { 182 | type: "xavier" 183 | } 184 | bias_filler { 185 | type: "constant" 186 | value: 0 187 | } 188 | } 189 | } 190 | layer { 191 | name: "prelu3" 192 | type: "PReLU" 193 | bottom: "conv3" 194 | top: "conv3" 195 | } 196 | layer { 197 | name: "conv3+" 198 | type: "Convolution" 199 | bottom: "conv3" 200 | top: "conv3+" 201 | param { 202 | lr_mult: 1 203 | decay_mult: 1 204 | } 205 | param { 206 | lr_mult: 2 207 | decay_mult: 0 208 | } 209 | convolution_param { 210 | num_output: 128 211 | kernel_size: 5 212 | stride: 1 213 | pad: 2 214 | weight_filler { 215 | type: "xavier" 216 | } 217 | bias_filler { 218 | type: "constant" 219 | value: 0 220 | } 221 | } 222 | } 223 | layer { 224 | name: "prelu3+" 225 | type: "PReLU" 226 | bottom: "conv3+" 227 | top: "conv3+" 228 | } 229 | layer { 230 | name: "pool3" 231 | type: "Pooling" 232 | bottom: "conv3+" 233 | top: "pool3" 234 | pooling_param { 235 | pool: MAX 236 | kernel_size: 3 237 | stride: 2 238 | } 239 | } 240 | layer { 241 | name: "ip1" 242 | type: "InnerProduct" 243 | bottom: "pool3" 244 | top: "ip1" 245 | param { 246 | lr_mult: 1 247 | decay_mult: 1 248 | } 249 | param { 250 | lr_mult: 1 251 | decay_mult: 0 252 | } 253 | inner_product_param { 254 | num_output: 3 255 | weight_filler { 256 | type: "xavier" 257 | } 258 | bias_filler { 259 | type: "constant" 260 | value: 0 261 | } 262 | } 263 | } 264 | -------------------------------------------------------------------------------- /model/softmax_loss/10epoches.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/liuyuisanai/coco_loss/4051c62817756b566911fc7f1154beda87b0ef72/model/softmax_loss/10epoches.caffemodel -------------------------------------------------------------------------------- /model/solver_test.prototxt: -------------------------------------------------------------------------------- 1 | net: "model/net_test.prototxt" 2 | base_lr: 0.1 3 | momentum: 0.9 4 | weight_decay: 0.0005 5 | lr_policy: "multistep" 6 | gamma: 0.1 7 | stepvalue: 5000 8 | stepvalue: 23000 9 | stepvalue: 25000 10 | display: 100 11 | max_iter: 10000 12 | snapshot: 0 13 | # solver mode: CPU or GPU 14 | solver_mode: GPU -------------------------------------------------------------------------------- /model/train_prototxts/net_centerloss.prototxt: -------------------------------------------------------------------------------- 1 | name: "LeNet++" 2 | 3 | input: "data" 4 | input_dim: 1 5 | input_dim: 1 6 | input_dim: 28 7 | input_dim: 28 8 | 9 | input: "label" 10 | input_dim: 1 11 | input_dim: 1 12 | input_dim: 1 13 | input_dim: 1 14 | 15 | mem_param { 16 | optimize_train: false 17 | } 18 | 19 | layer { 20 | name: "conv1" 21 | type: "Convolution" 22 | bottom: "data" 23 | top: "conv1" 24 | param { 25 | lr_mult: 1 26 | decay_mult: 1 27 | } 28 | param { 29 | lr_mult: 2 30 | decay_mult: 0 31 | } 32 | convolution_param { 33 | num_output: 32 34 | kernel_size: 5 35 | stride: 1 36 | pad: 2 37 | weight_filler { 38 | type: "xavier" 39 | } 40 | bias_filler { 41 | type: "constant" 42 | value: 0 43 | } 44 | } 45 | } 46 | layer { 47 | name: "prelu1" 48 | type: "PReLU" 49 | bottom: "conv1" 50 | top: "conv1" 51 | } 52 | layer { 53 | name: "conv1+" 54 | type: "Convolution" 55 | bottom: "conv1" 56 | top: "conv1+" 57 | param { 58 | lr_mult: 1 59 | decay_mult: 1 60 | } 61 | param { 62 | lr_mult: 2 63 | decay_mult: 0 64 | } 65 | convolution_param { 66 | num_output: 32 67 | kernel_size: 5 68 | stride: 1 69 | pad: 2 70 | weight_filler { 71 | type: "xavier" 72 | } 73 | bias_filler { 74 | type: "constant" 75 | value: 0 76 | } 77 | } 78 | } 79 | layer { 80 | name: "prelu1+" 81 | type: "PReLU" 82 | bottom: "conv1+" 83 | top: "conv1+" 84 | } 85 | layer { 86 | name: "pool1" 87 | type: "Pooling" 88 | bottom: "conv1+" 89 | top: "pool1" 90 | pooling_param { 91 | pool: MAX 92 | kernel_size: 2 93 | stride: 2 94 | } 95 | } 96 | layer { 97 | name: "conv2" 98 | type: "Convolution" 99 | bottom: "pool1" 100 | top: "conv2" 101 | param { 102 | lr_mult: 1 103 | decay_mult: 1 104 | } 105 | param { 106 | lr_mult: 2 107 | decay_mult: 0 108 | } 109 | convolution_param { 110 | num_output: 64 111 | kernel_size: 5 112 | stride: 1 113 | pad: 2 114 | weight_filler { 115 | type: "xavier" 116 | } 117 | bias_filler { 118 | type: "constant" 119 | value: 0 120 | } 121 | } 122 | } 123 | layer { 124 | name: "prelu2" 125 | type: "PReLU" 126 | bottom: "conv2" 127 | top: "conv2" 128 | } 129 | layer { 130 | name: "conv2+" 131 | type: "Convolution" 132 | bottom: "conv2" 133 | top: "conv2+" 134 | param { 135 | lr_mult: 1 136 | decay_mult: 1 137 | } 138 | param { 139 | lr_mult: 2 140 | decay_mult: 0 141 | } 142 | convolution_param { 143 | num_output: 64 144 | kernel_size: 5 145 | stride: 1 146 | pad: 2 147 | weight_filler { 148 | type: "xavier" 149 | } 150 | bias_filler { 151 | type: "constant" 152 | value: 0 153 | } 154 | } 155 | } 156 | layer { 157 | name: "prelu2+" 158 | type: "PReLU" 159 | bottom: "conv2+" 160 | top: "conv2+" 161 | } 162 | layer { 163 | name: "pool2" 164 | type: "Pooling" 165 | bottom: "conv2+" 166 | top: "pool2" 167 | pooling_param { 168 | pool: MAX 169 | kernel_size: 2 170 | stride: 2 171 | } 172 | } 173 | layer { 174 | name: "conv3" 175 | type: "Convolution" 176 | bottom: "pool2" 177 | top: "conv3" 178 | param { 179 | lr_mult: 1 180 | decay_mult: 1 181 | } 182 | param { 183 | lr_mult: 2 184 | decay_mult: 0 185 | } 186 | convolution_param { 187 | num_output: 128 188 | kernel_size: 5 189 | stride: 1 190 | pad: 2 191 | weight_filler { 192 | type: "xavier" 193 | } 194 | bias_filler { 195 | type: "constant" 196 | value: 0 197 | } 198 | } 199 | } 200 | layer { 201 | name: "prelu3" 202 | type: "PReLU" 203 | bottom: "conv3" 204 | top: "conv3" 205 | } 206 | layer { 207 | name: "conv3+" 208 | type: "Convolution" 209 | bottom: "conv3" 210 | top: "conv3+" 211 | param { 212 | lr_mult: 1 213 | decay_mult: 1 214 | } 215 | param { 216 | lr_mult: 2 217 | decay_mult: 0 218 | } 219 | convolution_param { 220 | num_output: 128 221 | kernel_size: 5 222 | stride: 1 223 | pad: 2 224 | weight_filler { 225 | type: "xavier" 226 | } 227 | bias_filler { 228 | type: "constant" 229 | value: 0 230 | } 231 | } 232 | } 233 | layer { 234 | name: "prelu3+" 235 | type: "PReLU" 236 | bottom: "conv3+" 237 | top: "conv3+" 238 | } 239 | layer { 240 | name: "pool3" 241 | type: "Pooling" 242 | bottom: "conv3+" 243 | top: "pool3" 244 | pooling_param { 245 | pool: MAX 246 | kernel_size: 3 247 | stride: 2 248 | } 249 | } 250 | layer { 251 | name: "ip1" 252 | type: "InnerProduct" 253 | bottom: "pool3" 254 | top: "ip1" 255 | param { 256 | lr_mult: 1 257 | decay_mult: 1 258 | } 259 | param { 260 | lr_mult: 2 261 | decay_mult: 0 262 | } 263 | inner_product_param { 264 | num_output: 3 265 | weight_filler { 266 | type: "xavier" 267 | } 268 | bias_filler { 269 | type: "constant" 270 | value: 0 271 | } 272 | } 273 | } 274 | layer { 275 | name: "preluip1" 276 | type: "PReLU" 277 | bottom: "ip1" 278 | top: "ip1" 279 | } 280 | ################## train ################## 281 | layer { 282 | name: "ip2" 283 | type: "InnerProduct" 284 | bottom: "ip1" 285 | top: "ip2" 286 | param { 287 | lr_mult: 1 288 | decay_mult: 1 289 | } 290 | inner_product_param { 291 | num_output: 10 292 | weight_filler { 293 | type: "xavier" 294 | } 295 | bias_term: false 296 | } 297 | } 298 | ############# softmax loss ############### 299 | layer { 300 | name: "softmax_loss" 301 | type: "SoftmaxWithLoss" 302 | bottom: "ip2" 303 | bottom: "label" 304 | top: "softmax_loss" 305 | loss_weight: 0.000001 306 | } 307 | ############# center loss ############### 308 | layer { 309 | name: "center_loss" 310 | type: "CenterLoss" 311 | bottom: "ip1" 312 | bottom: "label" 313 | top: "center_loss" 314 | param { 315 | lr_mult: 1 316 | decay_mult: 0 317 | } 318 | center_loss_param { 319 | num_output: 10 320 | center_filler { 321 | type: "xavier" 322 | } 323 | } 324 | loss_weight: 1 325 | } -------------------------------------------------------------------------------- /model/train_prototxts/net_cocoloss.prototxt: -------------------------------------------------------------------------------- 1 | name: "LeNet++" 2 | 3 | input: "data" 4 | input_dim: 1 5 | input_dim: 1 6 | input_dim: 28 7 | input_dim: 28 8 | 9 | input: "label" 10 | input_dim: 1 11 | input_dim: 1 12 | input_dim: 1 13 | input_dim: 1 14 | 15 | layer { 16 | name: "conv1" 17 | type: "Convolution" 18 | bottom: "data" 19 | top: "conv1" 20 | param { 21 | lr_mult: 1 22 | decay_mult: 1 23 | } 24 | param { 25 | lr_mult: 2 26 | decay_mult: 0 27 | } 28 | convolution_param { 29 | num_output: 32 30 | kernel_size: 5 31 | stride: 1 32 | pad: 2 33 | weight_filler { 34 | type: "xavier" 35 | } 36 | bias_filler { 37 | type: "constant" 38 | value: 0 39 | } 40 | } 41 | } 42 | layer { 43 | name: "prelu1" 44 | type: "PReLU" 45 | bottom: "conv1" 46 | top: "conv1" 47 | } 48 | layer { 49 | name: "conv1+" 50 | type: "Convolution" 51 | bottom: "conv1" 52 | top: "conv1+" 53 | param { 54 | lr_mult: 1 55 | decay_mult: 1 56 | } 57 | param { 58 | lr_mult: 2 59 | decay_mult: 0 60 | } 61 | convolution_param { 62 | num_output: 32 63 | kernel_size: 5 64 | stride: 1 65 | pad: 2 66 | weight_filler { 67 | type: "xavier" 68 | } 69 | bias_filler { 70 | type: "constant" 71 | value: 0 72 | } 73 | } 74 | } 75 | layer { 76 | name: "prelu1+" 77 | type: "PReLU" 78 | bottom: "conv1+" 79 | top: "conv1+" 80 | } 81 | layer { 82 | name: "pool1" 83 | type: "Pooling" 84 | bottom: "conv1+" 85 | top: "pool1" 86 | pooling_param { 87 | pool: MAX 88 | kernel_size: 2 89 | stride: 2 90 | } 91 | } 92 | layer { 93 | name: "conv2" 94 | type: "Convolution" 95 | bottom: "pool1" 96 | top: "conv2" 97 | param { 98 | lr_mult: 1 99 | decay_mult: 1 100 | } 101 | param { 102 | lr_mult: 2 103 | decay_mult: 0 104 | } 105 | convolution_param { 106 | num_output: 64 107 | kernel_size: 5 108 | stride: 1 109 | pad: 2 110 | weight_filler { 111 | type: "xavier" 112 | } 113 | bias_filler { 114 | type: "constant" 115 | value: 0 116 | } 117 | } 118 | } 119 | layer { 120 | name: "prelu2" 121 | type: "PReLU" 122 | bottom: "conv2" 123 | top: "conv2" 124 | } 125 | layer { 126 | name: "conv2+" 127 | type: "Convolution" 128 | bottom: "conv2" 129 | top: "conv2+" 130 | param { 131 | lr_mult: 1 132 | decay_mult: 1 133 | } 134 | param { 135 | lr_mult: 2 136 | decay_mult: 0 137 | } 138 | convolution_param { 139 | num_output: 64 140 | kernel_size: 5 141 | stride: 1 142 | pad: 2 143 | weight_filler { 144 | type: "xavier" 145 | } 146 | bias_filler { 147 | type: "constant" 148 | value: 0 149 | } 150 | } 151 | } 152 | layer { 153 | name: "prelu2+" 154 | type: "PReLU" 155 | bottom: "conv2+" 156 | top: "conv2+" 157 | } 158 | layer { 159 | name: "pool2" 160 | type: "Pooling" 161 | bottom: "conv2+" 162 | top: "pool2" 163 | pooling_param { 164 | pool: MAX 165 | kernel_size: 2 166 | stride: 2 167 | } 168 | } 169 | layer { 170 | name: "conv3" 171 | type: "Convolution" 172 | bottom: "pool2" 173 | top: "conv3" 174 | param { 175 | lr_mult: 1 176 | decay_mult: 1 177 | } 178 | param { 179 | lr_mult: 2 180 | decay_mult: 0 181 | } 182 | convolution_param { 183 | num_output: 128 184 | kernel_size: 5 185 | stride: 1 186 | pad: 2 187 | weight_filler { 188 | type: "xavier" 189 | } 190 | bias_filler { 191 | type: "constant" 192 | value: 0 193 | } 194 | } 195 | } 196 | layer { 197 | name: "prelu3" 198 | type: "PReLU" 199 | bottom: "conv3" 200 | top: "conv3" 201 | } 202 | layer { 203 | name: "conv3+" 204 | type: "Convolution" 205 | bottom: "conv3" 206 | top: "conv3+" 207 | param { 208 | lr_mult: 1 209 | decay_mult: 1 210 | } 211 | param { 212 | lr_mult: 2 213 | decay_mult: 0 214 | } 215 | convolution_param { 216 | num_output: 128 217 | kernel_size: 5 218 | stride: 1 219 | pad: 2 220 | weight_filler { 221 | type: "xavier" 222 | } 223 | bias_filler { 224 | type: "constant" 225 | value: 0 226 | } 227 | } 228 | } 229 | layer { 230 | name: "prelu3+" 231 | type: "PReLU" 232 | bottom: "conv3+" 233 | top: "conv3+" 234 | } 235 | layer { 236 | name: "pool3" 237 | type: "Pooling" 238 | bottom: "conv3+" 239 | top: "pool3" 240 | pooling_param { 241 | pool: MAX 242 | kernel_size: 3 243 | stride: 2 244 | } 245 | } 246 | layer { 247 | name: "ip1" 248 | type: "InnerProduct" 249 | bottom: "pool3" 250 | top: "ip1" 251 | param { 252 | lr_mult: 1 253 | decay_mult: 1 254 | } 255 | param { 256 | lr_mult: 1 257 | decay_mult: 0 258 | } 259 | inner_product_param { 260 | num_output: 3 261 | weight_filler { 262 | type: "xavier" 263 | } 264 | bias_filler { 265 | type: "constant" 266 | value: 0 267 | } 268 | } 269 | } 270 | 271 | 272 | layer{ 273 | name: "l2norm" 274 | type: "Normalize" 275 | bottom: "ip1" 276 | top: "ip1_norm" 277 | normalize_param { 278 | coeff: 1.0 279 | } 280 | } 281 | ################## train ################## 282 | layer { 283 | name: "ip2" 284 | type: "CenterProjection" 285 | bottom: "ip1_norm" 286 | top: "ip2" 287 | param { 288 | lr_mult: 1 289 | decay_mult: 0 290 | } 291 | center_projection_param { 292 | num_output: 10 293 | weight_filler { 294 | type: "xavier" 295 | } 296 | bias_term: false 297 | scale: 1 298 | } 299 | } 300 | 301 | 302 | ############# softmax loss ############### 303 | layer { 304 | name: "my_loss" 305 | type: "SoftmaxWithLoss" 306 | bottom: "ip2" 307 | bottom: "label" 308 | top: "my_loss" 309 | loss_weight: 1 310 | } 311 | -------------------------------------------------------------------------------- /model/train_prototxts/net_softmax.prototxt: -------------------------------------------------------------------------------- 1 | name: "LeNet++" 2 | 3 | input: "data" 4 | input_dim: 1 5 | input_dim: 1 6 | input_dim: 28 7 | input_dim: 28 8 | 9 | input: "label" 10 | input_dim: 1 11 | input_dim: 1 12 | input_dim: 1 13 | input_dim: 1 14 | 15 | layer { 16 | name: "conv1" 17 | type: "Convolution" 18 | bottom: "data" 19 | top: "conv1" 20 | param { 21 | lr_mult: 1 22 | decay_mult: 1 23 | } 24 | param { 25 | lr_mult: 2 26 | decay_mult: 0 27 | } 28 | convolution_param { 29 | num_output: 32 30 | kernel_size: 5 31 | stride: 1 32 | pad: 2 33 | weight_filler { 34 | type: "xavier" 35 | } 36 | bias_filler { 37 | type: "constant" 38 | value: 0 39 | } 40 | } 41 | } 42 | layer { 43 | name: "prelu1" 44 | type: "PReLU" 45 | bottom: "conv1" 46 | top: "conv1" 47 | } 48 | layer { 49 | name: "conv1+" 50 | type: "Convolution" 51 | bottom: "conv1" 52 | top: "conv1+" 53 | param { 54 | lr_mult: 1 55 | decay_mult: 1 56 | } 57 | param { 58 | lr_mult: 2 59 | decay_mult: 0 60 | } 61 | convolution_param { 62 | num_output: 32 63 | kernel_size: 5 64 | stride: 1 65 | pad: 2 66 | weight_filler { 67 | type: "xavier" 68 | } 69 | bias_filler { 70 | type: "constant" 71 | value: 0 72 | } 73 | } 74 | } 75 | layer { 76 | name: "prelu1+" 77 | type: "PReLU" 78 | bottom: "conv1+" 79 | top: "conv1+" 80 | } 81 | layer { 82 | name: "pool1" 83 | type: "Pooling" 84 | bottom: "conv1+" 85 | top: "pool1" 86 | pooling_param { 87 | pool: MAX 88 | kernel_size: 2 89 | stride: 2 90 | } 91 | } 92 | layer { 93 | name: "conv2" 94 | type: "Convolution" 95 | bottom: "pool1" 96 | top: "conv2" 97 | param { 98 | lr_mult: 1 99 | decay_mult: 1 100 | } 101 | param { 102 | lr_mult: 2 103 | decay_mult: 0 104 | } 105 | convolution_param { 106 | num_output: 64 107 | kernel_size: 5 108 | stride: 1 109 | pad: 2 110 | weight_filler { 111 | type: "xavier" 112 | } 113 | bias_filler { 114 | type: "constant" 115 | value: 0 116 | } 117 | } 118 | } 119 | layer { 120 | name: "prelu2" 121 | type: "PReLU" 122 | bottom: "conv2" 123 | top: "conv2" 124 | } 125 | layer { 126 | name: "conv2+" 127 | type: "Convolution" 128 | bottom: "conv2" 129 | top: "conv2+" 130 | param { 131 | lr_mult: 1 132 | decay_mult: 1 133 | } 134 | param { 135 | lr_mult: 2 136 | decay_mult: 0 137 | } 138 | convolution_param { 139 | num_output: 64 140 | kernel_size: 5 141 | stride: 1 142 | pad: 2 143 | weight_filler { 144 | type: "xavier" 145 | } 146 | bias_filler { 147 | type: "constant" 148 | value: 0 149 | } 150 | } 151 | } 152 | layer { 153 | name: "prelu2+" 154 | type: "PReLU" 155 | bottom: "conv2+" 156 | top: "conv2+" 157 | } 158 | layer { 159 | name: "pool2" 160 | type: "Pooling" 161 | bottom: "conv2+" 162 | top: "pool2" 163 | pooling_param { 164 | pool: MAX 165 | kernel_size: 2 166 | stride: 2 167 | } 168 | } 169 | layer { 170 | name: "conv3" 171 | type: "Convolution" 172 | bottom: "pool2" 173 | top: "conv3" 174 | param { 175 | lr_mult: 1 176 | decay_mult: 1 177 | } 178 | param { 179 | lr_mult: 2 180 | decay_mult: 0 181 | } 182 | convolution_param { 183 | num_output: 128 184 | kernel_size: 5 185 | stride: 1 186 | pad: 2 187 | weight_filler { 188 | type: "xavier" 189 | } 190 | bias_filler { 191 | type: "constant" 192 | value: 0 193 | } 194 | } 195 | } 196 | layer { 197 | name: "prelu3" 198 | type: "PReLU" 199 | bottom: "conv3" 200 | top: "conv3" 201 | } 202 | layer { 203 | name: "conv3+" 204 | type: "Convolution" 205 | bottom: "conv3" 206 | top: "conv3+" 207 | param { 208 | lr_mult: 1 209 | decay_mult: 1 210 | } 211 | param { 212 | lr_mult: 2 213 | decay_mult: 0 214 | } 215 | convolution_param { 216 | num_output: 128 217 | kernel_size: 5 218 | stride: 1 219 | pad: 2 220 | weight_filler { 221 | type: "xavier" 222 | } 223 | bias_filler { 224 | type: "constant" 225 | value: 0 226 | } 227 | } 228 | } 229 | layer { 230 | name: "prelu3+" 231 | type: "PReLU" 232 | bottom: "conv3+" 233 | top: "conv3+" 234 | } 235 | layer { 236 | name: "pool3" 237 | type: "Pooling" 238 | bottom: "conv3+" 239 | top: "pool3" 240 | pooling_param { 241 | pool: MAX 242 | kernel_size: 3 243 | stride: 2 244 | } 245 | } 246 | layer { 247 | name: "ip1" 248 | type: "InnerProduct" 249 | bottom: "pool3" 250 | top: "ip1" 251 | param { 252 | lr_mult: 1 253 | decay_mult: 1 254 | } 255 | param { 256 | lr_mult: 2 257 | decay_mult: 0 258 | } 259 | inner_product_param { 260 | num_output: 3 261 | weight_filler { 262 | type: "xavier" 263 | } 264 | bias_filler { 265 | type: "constant" 266 | value: 0 267 | } 268 | } 269 | } 270 | layer { 271 | name: "preluip1" 272 | type: "PReLU" 273 | bottom: "ip1" 274 | top: "ip1" 275 | } 276 | ################## train ################## 277 | layer { 278 | name: "ip2" 279 | type: "InnerProduct" 280 | bottom: "ip1" 281 | top: "ip2" 282 | param { 283 | lr_mult: 1 284 | decay_mult: 1 285 | } 286 | inner_product_param { 287 | num_output: 10 288 | weight_filler { 289 | type: "xavier" 290 | } 291 | bias_term: false 292 | } 293 | } 294 | ############# softmax loss ############### 295 | layer { 296 | name: "softmax_loss" 297 | type: "SoftmaxWithLoss" 298 | bottom: "ip2" 299 | bottom: "label" 300 | top: "softmax_loss" 301 | loss_weight: 1 302 | } 303 | -------------------------------------------------------------------------------- /model/train_prototxts/solver_centerloss.prototxt: -------------------------------------------------------------------------------- 1 | net: "net_centerloss.prototxt" 2 | base_lr: 0.01 3 | momentum: 0.9 4 | weight_decay: 0.0005 5 | lr_policy: "multistep" 6 | gamma: 0.1 7 | stepvalue: 5000 8 | stepvalue: 23000 9 | stepvalue: 25000 10 | display: 100 11 | max_iter: 100000 12 | snapshot: 0 13 | # solver mode: CPU or GPU 14 | solver_mode: GPU -------------------------------------------------------------------------------- /model/train_prototxts/solver_cocoloss.prototxt: -------------------------------------------------------------------------------- 1 | net: "net_cocoloss.prototxt" 2 | base_lr: 0.1 3 | momentum: 0.9 4 | weight_decay: 0.0005 5 | lr_policy: "multistep" 6 | gamma: 0.1 7 | stepvalue: 5000 8 | stepvalue: 23000 9 | stepvalue: 25000 10 | display: 100 11 | max_iter: 10000 12 | snapshot: 0 13 | # solver mode: CPU or GPU 14 | solver_mode: GPU -------------------------------------------------------------------------------- /model/train_prototxts/solver_softmax.prototxt: -------------------------------------------------------------------------------- 1 | net: "net_softmax.prototxt" 2 | base_lr: 0.01 3 | momentum: 0.9 4 | weight_decay: 0.0005 5 | lr_policy: "multistep" 6 | gamma: 0.1 7 | stepvalue: 5000 8 | stepvalue: 23000 9 | stepvalue: 25000 10 | display: 100 11 | max_iter: 100000 12 | snapshot: 0 13 | # solver mode: CPU or GPU 14 | solver_mode: GPU -------------------------------------------------------------------------------- /output_sample.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/liuyuisanai/coco_loss/4051c62817756b566911fc7f1154beda87b0ef72/output_sample.jpg -------------------------------------------------------------------------------- /script_show_minist.m: -------------------------------------------------------------------------------- 1 | %% register caffe environment for windows matlab 2 | %% you may comment line 4 to 6 if you are using linux platform 3 | addpath(fullfile(fileparts(pwd))) 4 | cd +caffe/private; 5 | caffe.reset_all; 6 | cd ../..; 7 | 8 | %% configure && active caffe 9 | clear input feature; 10 | caffe.reset_all; 11 | caffe.init_log('log/'); 12 | %% configure 13 | param.bs = 128; 14 | load mnist_data; 15 | s = caffe.get_solver('model/solver_test.prototxt', 0); 16 | i=1; 17 | for i = 1 : 3 18 | switch i 19 | case 1 20 | model_name = 'coco_loss'; 21 | case 2 22 | model_name = 'softmax_loss'; 23 | case 3 24 | model_name = 'center+softmax_loss'; 25 | end 26 | s.use_caffemodel(fullfile('model', model_name, '10epoches.caffemodel')); 27 | s.set_phase('test'); 28 | feature{i} = zeros(length(label_te), 3); 29 | for iter = 1 : ceil(length(label_te)/param.bs) 30 | this_id = mod((iter-1)*param.bs:(iter*param.bs-1), length(label_te))+1; 31 | input{1}{1} = reshape(single(convert_img2caffe(img_te(:,:,this_id)))-127.5, [28 28 1 param.bs])/255; 32 | s.reshape_as_input(input); 33 | s.set_input_data(input); 34 | s.forward_prefilled(); 35 | t = s.nets{1}.blobs('ip1').get_data(); 36 | feature{i}(this_id,:) =squeeze(t)'; 37 | end 38 | end 39 | color = hsv(10); 40 | close all; 41 | for j = 1 : 3 42 | figure(j) 43 | switch j 44 | case 1 45 | model_name = 'coco\_loss'; 46 | case 2 47 | model_name = 'softmax\_loss'; 48 | case 3 49 | model_name = 'center+softmax_loss'; 50 | end 51 | for i = 0 : 9 52 | id = find(label_te==i); 53 | scatter3(feature{j}(id(1:5:end),1), feature{j}(id(1:5:end),2),feature{j}(id(1:5:end),3), 4, 'MarkerEdgeColor', color(i+1,:)); 54 | hold on 55 | end 56 | legend('0', '1', '2', '3', '4', '5', '6', '7', '8', '9'); 57 | title(model_name); 58 | end --------------------------------------------------------------------------------