├── .gitignore ├── EyeglassesV2 ├── data_dual.prototxt ├── data_gen.prototxt ├── data_test.prototxt ├── discriminator.prototxt ├── generate.py ├── generator.prototxt ├── generator_dual.prototxt ├── generator_original.prototxt ├── lfw_list.txt ├── pixloss.prototxt ├── solver_data.prototxt ├── solver_data_dual.prototxt ├── solver_data_gen.prototxt ├── solver_discriminator.prototxt ├── solver_generator.prototxt ├── solver_generator_dual.prototxt ├── solver_pixloss.prototxt ├── solver_template.prototxt └── train.py ├── README.md └── assets ├── gen.png └── remove.png /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | 27 | # PyInstaller 28 | # Usually these files are written by a python script from a template 29 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 30 | *.manifest 31 | *.spec 32 | 33 | # Installer logs 34 | pip-log.txt 35 | pip-delete-this-directory.txt 36 | 37 | # Unit test / coverage reports 38 | htmlcov/ 39 | .tox/ 40 | .coverage 41 | .coverage.* 42 | .cache 43 | nosetests.xml 44 | coverage.xml 45 | *,cover 46 | .hypothesis/ 47 | 48 | # Translations 49 | *.mo 50 | *.pot 51 | 52 | # Django stuff: 53 | *.log 54 | local_settings.py 55 | 56 | # Flask stuff: 57 | instance/ 58 | .webassets-cache 59 | 60 | # Scrapy stuff: 61 | .scrapy 62 | 63 | # Sphinx documentation 64 | docs/_build/ 65 | 66 | # PyBuilder 67 | target/ 68 | 69 | # IPython Notebook 70 | .ipynb_checkpoints 71 | 72 | # pyenv 73 | .python-version 74 | 75 | # celery beat schedule file 76 | celerybeat-schedule 77 | 78 | # dotenv 79 | .env 80 | 81 | # virtualenv 82 | venv/ 83 | ENV/ 84 | 85 | # Spyder project settings 86 | .spyderproject 87 | 88 | # Rope project settings 89 | .ropeproject 90 | 91 | 92 | Eyeglasses 93 | caffe-fr-chairs-deepsim 94 | EyeglassesV3 95 | EyeglassesV3t 96 | EyeglassesV3.1 97 | Smiling 98 | EyeglassesV2/snapshots* 99 | EyeglassesV2/test_result 100 | EyeglassesV2/output 101 | 102 | -------------------------------------------------------------------------------- /EyeglassesV2/data_dual.prototxt: -------------------------------------------------------------------------------- 1 | 2 | layer { 3 | name: "data" 4 | type: "ImageData" 5 | top: "data" 6 | top: "label" 7 | transform_param { 8 | mirror: true 9 | # crop_size: 227 10 | # mean_file: "/data/deploy/caffe/data/ilsvrc12/imagenet_mean.binaryproto" 11 | mean_value: 104 12 | mean_value: 117 13 | mean_value: 123 14 | } 15 | image_data_param { 16 | source: "/home/Disk1/wangzd/CelebA/Anno/Eyeglasses_Positive.txt" 17 | root_folder:"/home/Disk1/wangzd/CelebA/Img/img_celeba_bbox/" 18 | batch_size: 64 19 | new_height: 64 20 | new_width: 64 21 | shuffle:true 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /EyeglassesV2/data_gen.prototxt: -------------------------------------------------------------------------------- 1 | 2 | layer { 3 | name: "data" 4 | type: "ImageData" 5 | top: "data" 6 | top: "label" 7 | transform_param { 8 | mirror: true 9 | # crop_size: 227 10 | # mean_file: "/data/deploy/caffe/data/ilsvrc12/imagenet_mean.binaryproto" 11 | mean_value: 104 12 | mean_value: 117 13 | mean_value: 123 14 | } 15 | image_data_param { 16 | source: "/home/Disk1/wangzd/CelebA/Anno/Eyeglasses_Negative.txt" 17 | root_folder:"/home/Disk1/wangzd/CelebA/Img/img_celeba_bbox/" 18 | batch_size: 64 19 | new_height: 64 20 | new_width: 64 21 | shuffle:true 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /EyeglassesV2/data_test.prototxt: -------------------------------------------------------------------------------- 1 | 2 | layer { 3 | name: "data" 4 | type: "ImageData" 5 | top: "data" 6 | top: "label" 7 | transform_param { 8 | mirror: true 9 | mean_value: 104 10 | mean_value: 117 11 | mean_value: 123 12 | } 13 | image_data_param { 14 | source: "lfw_list.txt" 15 | batch_size: 64 16 | new_height: 64 17 | new_width: 64 18 | shuffle:true 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /EyeglassesV2/discriminator.prototxt: -------------------------------------------------------------------------------- 1 | force_backward: true 2 | input: "data" 3 | input_shape { 4 | dim: 64 5 | dim: 3 6 | dim: 64 7 | dim: 64 8 | } 9 | 10 | input: "label" 11 | input_shape { 12 | dim: 64 13 | dim: 1 14 | } 15 | 16 | 17 | # Discriminator net 18 | 19 | layer { 20 | name: "Dconv1" 21 | type: "Convolution" 22 | bottom: "data" 23 | top: "Dconv1" 24 | param { 25 | lr_mult: 1 26 | decay_mult: 1 27 | } 28 | param { 29 | lr_mult: 0 30 | decay_mult: 0 31 | } 32 | convolution_param { 33 | num_output: 64 34 | kernel_size: 4 35 | stride: 2 36 | pad: 1 37 | weight_filler { 38 | type: "gaussian" 39 | std: 0.02 40 | } 41 | bias_filler { 42 | type: "constant" 43 | value: 0 44 | } 45 | # engine: CUDNN 46 | } 47 | } 48 | layer { 49 | name: "Drelu1" 50 | type: "ReLU" 51 | bottom: "Dconv1" 52 | top: "Dconv1" 53 | relu_param { 54 | negative_slope: 0.2 55 | } 56 | 57 | } 58 | 59 | layer { 60 | name: "Dconv2" 61 | type: "Convolution" 62 | bottom: "Dconv1" 63 | top: "Dconv2" 64 | param { 65 | lr_mult: 1 66 | decay_mult: 1 67 | } 68 | param { 69 | lr_mult: 0 70 | decay_mult: 0 71 | } 72 | convolution_param { 73 | num_output: 128 74 | kernel_size: 4 75 | stride: 2 76 | pad: 1 77 | weight_filler { 78 | type: "gaussian" 79 | std: 0.02 80 | } 81 | bias_filler { 82 | type: "constant" 83 | value: 0 84 | } 85 | # engine: CUDNN 86 | } 87 | } 88 | layer { 89 | name: "Dconv2_BN" 90 | type: "BatchNorm" include { phase: TRAIN} 91 | bottom: "Dconv2" 92 | top: "Dconv2_BN" 93 | param { 94 | lr_mult: 0 95 | decay_mult: 0 96 | } 97 | param { 98 | lr_mult: 0 99 | decay_mult: 0 100 | } 101 | param { 102 | lr_mult: 0 103 | decay_mult: 0 104 | } 105 | batch_norm_param { 106 | use_global_stats: false 107 | moving_average_fraction: 0.95 108 | } 109 | } 110 | layer { 111 | name: "Dconv2_BN" 112 | type: "BatchNorm" include { phase: TEST} 113 | bottom: "Dconv2" 114 | top: "Dconv2_BN" 115 | param { 116 | lr_mult: 0 117 | decay_mult: 0 118 | } 119 | param { 120 | lr_mult: 0 121 | decay_mult: 0 122 | } 123 | param { 124 | lr_mult: 0 125 | decay_mult: 0 126 | } 127 | batch_norm_param { 128 | use_global_stats: true 129 | moving_average_fraction: 0.95 130 | } 131 | } 132 | 133 | layer { 134 | name: "Drelu2" 135 | type: "ReLU" 136 | bottom: "Dconv2_BN" 137 | top: "Dconv2_BN" 138 | relu_param { 139 | negative_slope: 0.2 140 | } 141 | 142 | } 143 | layer { 144 | name: "Dconv3" 145 | type: "Convolution" 146 | bottom: "Dconv2_BN" 147 | top: "Dconv3" 148 | param { 149 | lr_mult: 1 150 | decay_mult: 1 151 | } 152 | param { 153 | lr_mult: 0 154 | decay_mult: 0 155 | } 156 | convolution_param { 157 | num_output: 256 158 | kernel_size: 4 159 | stride: 2 160 | pad: 1 161 | weight_filler { 162 | type: "gaussian" 163 | std: 0.02 164 | } 165 | bias_filler { 166 | type: "constant" 167 | value: 0 168 | } 169 | # engine: CUDNN 170 | } 171 | } 172 | layer { 173 | name: "Dconv3_BN" 174 | type: "BatchNorm" include { phase: TRAIN} 175 | bottom: "Dconv3" 176 | top: "Dconv3_BN" 177 | param { 178 | lr_mult: 0 179 | decay_mult: 0 180 | } 181 | param { 182 | lr_mult: 0 183 | decay_mult: 0 184 | } 185 | param { 186 | lr_mult: 0 187 | decay_mult: 0 188 | } 189 | batch_norm_param { 190 | use_global_stats: false 191 | moving_average_fraction: 0.95 192 | } 193 | } 194 | layer { 195 | name: "Dconv3_BN" 196 | type: "BatchNorm" include { phase: TEST} 197 | bottom: "Dconv3" 198 | top: "Dconv3_BN" 199 | param { 200 | lr_mult: 0 201 | decay_mult: 0 202 | } 203 | param { 204 | lr_mult: 0 205 | decay_mult: 0 206 | } 207 | param { 208 | lr_mult: 0 209 | decay_mult: 0 210 | } 211 | batch_norm_param { 212 | use_global_stats: true 213 | moving_average_fraction: 0.95 214 | } 215 | } 216 | 217 | layer { 218 | name: "Drelu3" 219 | type: "ReLU" 220 | bottom: "Dconv3_BN" 221 | top: "Dconv3_BN" 222 | relu_param { 223 | negative_slope: 0.2 224 | } 225 | 226 | } 227 | layer { 228 | name: "Dconv4" 229 | type: "Convolution" 230 | bottom: "Dconv3_BN" 231 | top: "Dconv4" 232 | param { 233 | lr_mult: 1 234 | decay_mult: 1 235 | } 236 | param { 237 | lr_mult: 0 238 | decay_mult: 0 239 | } 240 | convolution_param { 241 | num_output: 512 242 | kernel_size: 4 243 | stride: 2 244 | pad: 1 245 | weight_filler { 246 | type: "gaussian" 247 | std: 0.01 248 | } 249 | bias_filler { 250 | type: "constant" 251 | value: 0 252 | } 253 | # engine: CUDNN 254 | } 255 | } 256 | layer { 257 | name: "Dconv4_BN" 258 | type: "BatchNorm" include { phase: TRAIN} 259 | bottom: "Dconv4" 260 | top: "Dconv4_BN" 261 | param { 262 | lr_mult: 0 263 | decay_mult: 0 264 | } 265 | param { 266 | lr_mult: 0 267 | decay_mult: 0 268 | } 269 | param { 270 | lr_mult: 0 271 | decay_mult: 0 272 | } 273 | batch_norm_param { 274 | use_global_stats: false 275 | moving_average_fraction: 0.95 276 | } 277 | } 278 | layer { 279 | name: "Dconv4_BN" 280 | type: "BatchNorm" include { phase: TEST} 281 | bottom: "Dconv4" 282 | top: "Dconv4_BN" 283 | param { 284 | lr_mult: 0 285 | decay_mult: 0 286 | } 287 | param { 288 | lr_mult: 0 289 | decay_mult: 0 290 | } 291 | param { 292 | lr_mult: 0 293 | decay_mult: 0 294 | } 295 | batch_norm_param { 296 | use_global_stats: true 297 | moving_average_fraction: 0.95 298 | } 299 | } 300 | 301 | layer { 302 | name: "Drelu4" 303 | type: "ReLU" 304 | bottom: "Dconv4_BN" 305 | top: "Dconv4_BN" 306 | relu_param { 307 | negative_slope: 0.2 308 | } 309 | 310 | } 311 | layer { 312 | name: "Dconv5" 313 | type: "Convolution" 314 | bottom: "Dconv4_BN" 315 | top: "Dconv5" 316 | param { 317 | lr_mult: 1 318 | decay_mult: 1 319 | } 320 | param { 321 | lr_mult: 0 322 | decay_mult: 0 323 | } 324 | convolution_param { 325 | num_output: 512 326 | kernel_size: 4 327 | stride: 1 328 | pad: 0 329 | weight_filler { 330 | type: "gaussian" 331 | std: 0.02 332 | } 333 | bias_filler { 334 | type: "constant" 335 | value: 0 336 | } 337 | # engine: CUDNN 338 | } 339 | } 340 | #layer { 341 | # name: "relu5" 342 | # type: "ReLU" 343 | # bottom: "Dconv5" 344 | # top: "Dconv5" 345 | #} 346 | # 347 | #layer { 348 | # name: "Dpool5" 349 | # type: "Pooling" 350 | # bottom: "Dconv5" 351 | # top: "Dpool5" 352 | # pooling_param { 353 | # pool: AVE 354 | # kernel_size: 11 355 | # stride: 11 356 | # } 357 | #} 358 | 359 | layer { 360 | name: "Dfc7" 361 | type: "InnerProduct" 362 | bottom: "Dconv5" 363 | top: "Dfc7" 364 | param { 365 | lr_mult: 1 366 | decay_mult: 1 367 | } 368 | param { 369 | lr_mult: 0 370 | decay_mult: 0 371 | } 372 | inner_product_param { 373 | num_output: 3 374 | weight_filler { 375 | type: "gaussian" 376 | std: 0.01 377 | } 378 | bias_filler { 379 | type: "constant" 380 | value: 0 381 | } 382 | } 383 | } 384 | layer { 385 | name: "discr_loss" 386 | type: "SoftmaxWithLoss" 387 | bottom: "Dfc7" 388 | bottom: "label" 389 | top: "discr_loss" 390 | loss_weight: 1 391 | } 392 | 393 | 394 | #layer { 395 | # name: "discr_loss" 396 | # type: "SigmoidCrossEntropyLoss" 397 | # bottom: "Dfc7" 398 | # bottom: "label" 399 | # top: "discr_loss" 400 | # loss_weight: 1 401 | ## loss_weight: 1 402 | #} 403 | # 404 | -------------------------------------------------------------------------------- /EyeglassesV2/generate.py: -------------------------------------------------------------------------------- 1 | import sys 2 | sys.path.insert(0,'../caffe-fr-chairs-deepsim/python') 3 | import caffe 4 | import numpy as np 5 | import cv2 6 | import scipy.io 7 | import scipy.misc 8 | 9 | nz = 100 10 | img_size = 64 11 | batch_size = 64 12 | 13 | caffe.set_mode_gpu() 14 | gen_net = caffe.Net(sys.argv[1], sys.argv[2], caffe.TEST) 15 | #data_loader=caffe.Net('data_gen.prototxt',sys.argv[2],caffe.TEST) 16 | #data_loader=caffe.Net('data_test.prototxt',sys.argv[2],caffe.TEST) 17 | data_loader=caffe.Net('data_dual.prototxt',sys.argv[2],caffe.TEST) 18 | # Fix the seed to debug 19 | #np.random.seed(0) 20 | # 21 | #img = cv2.imread(sys.argv[3]) 22 | #img = np.array(img,dtype=np.float32) 23 | #img = cv2.resize(img,(img_size,img_size)) 24 | #img -= np.array((104, 117, 123)) 25 | #img = img.transpose(2,0,1) 26 | #img_batch = np.zeros((batch_size,3,img_size,img_size)) 27 | #for i in xrange(batch_size): 28 | # img_batch[i,:] = img[i] 29 | #gen_net.blobs['feat'].data[...] = img_batch 30 | data_loader.forward_simple() 31 | img_batch = data_loader.blobs['data'].data 32 | 33 | gen_net.blobs['feat'].data[...] =img_batch 34 | gen_net.forward_simple() 35 | 36 | generated_img = gen_net.blobs['generated'].data 37 | 38 | print generated_img.shape 39 | 40 | max_val, min_val = np.max(generated_img[0]), np.min(generated_img[0]) 41 | max_val_, min_val_ = np.max(img_batch[0]), np.min(img_batch[0]) 42 | 43 | 44 | # Concat all images into a big 8*8 image 45 | flatten_img = ((generated_img.transpose((0,2,3,1)))[:] - min_val) / (max_val-min_val) 46 | flatten_img_batch = ((img_batch.transpose((0,2,3,1)))[:] - min_val_) / (max_val_-min_val_) 47 | print flatten_img[0,:] 48 | print flatten_img.shape 49 | for i in xrange(batch_size): 50 | cv2.imwrite('test_result/'+str(i)+'.jpg',255*flatten_img[i,:]) 51 | cv2.imwrite('test_result/'+str(i)+'_ori.jpg',255*flatten_img_batch[i,:]) 52 | 53 | 54 | #cv2.imwrite('test1.jpg', 255*flatten_img.reshape(8,8,img_size,img_size,3).swapaxes(1,2).reshape(8*img_size,8*img_size, 3)) 55 | #cv2.imwrite('test_ori.jpg', 255*flatten_img_batch.reshape(8,8,img_size,img_size,3).swapaxes(1,2).reshape(8*img_size,8*img_size, 3)) 56 | 57 | #cv2.imshow('test', ((generated_img.transpose((0,2,3,1)))[2] - min_val) / (max_val-min_val)) 58 | #cv2.waitKey() 59 | -------------------------------------------------------------------------------- /EyeglassesV2/generator.prototxt: -------------------------------------------------------------------------------- 1 | force_backward: true 2 | input: "feat" 3 | input_shape { 4 | dim: 64 5 | dim: 3 6 | dim:64 7 | dim:64 8 | } 9 | 10 | 11 | 12 | layer { 13 | name: "conv1" 14 | type: "Convolution" 15 | bottom: "feat" 16 | top: "conv1" 17 | param { 18 | lr_mult: 1 19 | decay_mult: 0 20 | } 21 | param { 22 | lr_mult: 0 23 | decay_mult: 0 24 | } 25 | convolution_param { 26 | num_output: 64 27 | pad: 1 28 | kernel_size: 4 29 | stride: 2 30 | weight_filler { 31 | type: "gaussian" 32 | std: 0.02 33 | } 34 | bias_filler { 35 | type: "constant" 36 | } 37 | } 38 | } 39 | 40 | 41 | layer { 42 | name: "conv1_BN" 43 | type: "BatchNorm" include { phase: TRAIN} 44 | bottom: "conv1" 45 | top: "conv1_BN" 46 | param { 47 | lr_mult: 0 48 | decay_mult: 0 49 | } 50 | param { 51 | lr_mult: 0 52 | decay_mult: 0 53 | } 54 | param { 55 | lr_mult: 0 56 | decay_mult: 0 57 | } 58 | batch_norm_param { 59 | use_global_stats: false 60 | moving_average_fraction: 0.95 61 | } 62 | } 63 | layer { 64 | name: "conv1_BN" 65 | type: "BatchNorm" include { phase: TEST} 66 | bottom: "conv1" 67 | top: "conv1_BN" 68 | param { 69 | lr_mult: 0 70 | decay_mult: 0 71 | } 72 | param { 73 | lr_mult: 0 74 | decay_mult: 0 75 | } 76 | param { 77 | lr_mult: 0 78 | decay_mult: 0 79 | } 80 | batch_norm_param { 81 | use_global_stats: true 82 | moving_average_fraction: 0.95 83 | } 84 | } 85 | 86 | layer { 87 | name: "relu_conv1" 88 | type: "ReLU" 89 | bottom: "conv1_BN" 90 | top: "relu_conv1" 91 | } 92 | 93 | 94 | layer { 95 | name: "conv2" 96 | type: "Convolution" 97 | bottom: "relu_conv1" 98 | top: "conv2" 99 | param { 100 | lr_mult: 1 101 | decay_mult: 0 102 | } 103 | param { 104 | lr_mult: 0 105 | decay_mult: 0 106 | } 107 | convolution_param { 108 | num_output: 128 109 | pad: 1 110 | kernel_size: 4 111 | stride: 2 112 | weight_filler { 113 | type: "gaussian" 114 | std: 0.02 115 | } 116 | bias_filler { 117 | type: "constant" 118 | } 119 | } 120 | } 121 | 122 | 123 | layer { 124 | name: "conv2_BN" 125 | type: "BatchNorm" include { phase: TRAIN} 126 | bottom: "conv2" 127 | top: "conv2_BN" 128 | param { 129 | lr_mult: 0 130 | decay_mult: 0 131 | } 132 | param { 133 | lr_mult: 0 134 | decay_mult: 0 135 | } 136 | param { 137 | lr_mult: 0 138 | decay_mult: 0 139 | } 140 | batch_norm_param { 141 | use_global_stats: false 142 | moving_average_fraction: 0.95 143 | } 144 | } 145 | layer { 146 | name: "conv2_BN" 147 | type: "BatchNorm" include { phase: TEST} 148 | bottom: "conv2" 149 | top: "conv2_BN" 150 | param { 151 | lr_mult: 0 152 | decay_mult: 0 153 | } 154 | param { 155 | lr_mult: 0 156 | decay_mult: 0 157 | } 158 | param { 159 | lr_mult: 0 160 | decay_mult: 0 161 | } 162 | batch_norm_param { 163 | use_global_stats: true 164 | moving_average_fraction: 0.95 165 | } 166 | } 167 | 168 | layer { 169 | name: "relu_conv2" 170 | type: "ReLU" 171 | bottom: "conv2_BN" 172 | top: "relu_conv2" 173 | } 174 | 175 | 176 | 177 | 178 | layer { 179 | name: "conv3" 180 | type: "Convolution" 181 | bottom: "relu_conv2" 182 | top: "conv3" 183 | param { 184 | lr_mult: 1 185 | decay_mult: 0 186 | } 187 | param { 188 | lr_mult: 0 189 | decay_mult: 0 190 | } 191 | convolution_param { 192 | num_output: 256 193 | pad: 1 194 | kernel_size: 4 195 | stride: 2 196 | weight_filler { 197 | type: "gaussian" 198 | std: 0.02 199 | } 200 | bias_filler { 201 | type: "constant" 202 | } 203 | } 204 | } 205 | 206 | 207 | layer { 208 | name: "conv3_BN" 209 | type: "BatchNorm" include { phase: TRAIN} 210 | bottom: "conv3" 211 | top: "conv3_BN" 212 | param { 213 | lr_mult: 0 214 | decay_mult: 0 215 | } 216 | param { 217 | lr_mult: 0 218 | decay_mult: 0 219 | } 220 | param { 221 | lr_mult: 0 222 | decay_mult: 0 223 | } 224 | batch_norm_param { 225 | use_global_stats: false 226 | moving_average_fraction: 0.95 227 | } 228 | } 229 | layer { 230 | name: "conv3_BN" 231 | type: "BatchNorm" include { phase: TEST} 232 | bottom: "conv3" 233 | top: "conv3_BN" 234 | param { 235 | lr_mult: 0 236 | decay_mult: 0 237 | } 238 | param { 239 | lr_mult: 0 240 | decay_mult: 0 241 | } 242 | param { 243 | lr_mult: 0 244 | decay_mult: 0 245 | } 246 | batch_norm_param { 247 | use_global_stats: true 248 | moving_average_fraction: 0.95 249 | } 250 | } 251 | 252 | layer { 253 | name: "relu_conv3" 254 | type: "ReLU" 255 | bottom: "conv3_BN" 256 | top: "relu_conv3" 257 | } 258 | 259 | 260 | 261 | 262 | 263 | 264 | layer { 265 | name: "conv4" 266 | type: "Convolution" 267 | bottom: "relu_conv3" 268 | top: "conv4" 269 | param { 270 | lr_mult: 1 271 | decay_mult: 0 272 | } 273 | param { 274 | lr_mult: 0 275 | decay_mult: 0 276 | } 277 | convolution_param { 278 | num_output: 512 279 | pad: 1 280 | kernel_size: 4 281 | stride: 2 282 | weight_filler { 283 | type: "gaussian" 284 | std: 0.02 285 | } 286 | bias_filler { 287 | type: "constant" 288 | } 289 | } 290 | } 291 | 292 | 293 | layer { 294 | name: "conv4_BN" 295 | type: "BatchNorm" include { phase: TRAIN} 296 | bottom: "conv4" 297 | top: "conv4_BN" 298 | param { 299 | lr_mult: 0 300 | decay_mult: 0 301 | } 302 | param { 303 | lr_mult: 0 304 | decay_mult: 0 305 | } 306 | param { 307 | lr_mult: 0 308 | decay_mult: 0 309 | } 310 | batch_norm_param { 311 | use_global_stats: false 312 | moving_average_fraction: 0.95 313 | } 314 | } 315 | layer { 316 | name: "conv4_BN" 317 | type: "BatchNorm" include { phase: TEST} 318 | bottom: "conv4" 319 | top: "conv4_BN" 320 | param { 321 | lr_mult: 0 322 | decay_mult: 0 323 | } 324 | param { 325 | lr_mult: 0 326 | decay_mult: 0 327 | } 328 | param { 329 | lr_mult: 0 330 | decay_mult: 0 331 | } 332 | batch_norm_param { 333 | use_global_stats: true 334 | moving_average_fraction: 0.95 335 | } 336 | } 337 | 338 | layer { 339 | name: "relu_conv4" 340 | type: "ReLU" 341 | bottom: "conv4_BN" 342 | top: "relu_conv4" 343 | } 344 | 345 | 346 | 347 | 348 | 349 | layer { 350 | name: "conv5" 351 | type: "Convolution" 352 | bottom: "relu_conv4" 353 | top: "conv5" 354 | param { 355 | lr_mult: 1 356 | decay_mult: 0 357 | } 358 | param { 359 | lr_mult: 0 360 | decay_mult: 0 361 | } 362 | convolution_param { 363 | num_output: 100 364 | pad: 0 365 | kernel_size: 4 366 | stride: 2 367 | weight_filler { 368 | type: "gaussian" 369 | std: 0.02 370 | } 371 | bias_filler { 372 | type: "constant" 373 | } 374 | } 375 | } 376 | 377 | 378 | layer { 379 | name: "conv5_BN" 380 | type: "BatchNorm" include { phase: TRAIN} 381 | bottom: "conv5" 382 | top: "conv5_BN" 383 | param { 384 | lr_mult: 0 385 | decay_mult: 0 386 | } 387 | param { 388 | lr_mult: 0 389 | decay_mult: 0 390 | } 391 | param { 392 | lr_mult: 0 393 | decay_mult: 0 394 | } 395 | batch_norm_param { 396 | use_global_stats: false 397 | moving_average_fraction: 0.95 398 | } 399 | } 400 | layer { 401 | name: "conv5_BN" 402 | type: "BatchNorm" include { phase: TEST} 403 | bottom: "conv5" 404 | top: "conv5_BN" 405 | param { 406 | lr_mult: 0 407 | decay_mult: 0 408 | } 409 | param { 410 | lr_mult: 0 411 | decay_mult: 0 412 | } 413 | param { 414 | lr_mult: 0 415 | decay_mult: 0 416 | } 417 | batch_norm_param { 418 | use_global_stats: true 419 | moving_average_fraction: 0.95 420 | } 421 | } 422 | 423 | layer { 424 | name: "relu_conv5" 425 | type: "ReLU" 426 | bottom: "conv5_BN" 427 | top: "relu_conv5" 428 | } 429 | 430 | 431 | 432 | 433 | layer { 434 | name: "deconv5" 435 | type: "Deconvolution" 436 | bottom: "relu_conv5" 437 | top: "deconv5" 438 | param { 439 | lr_mult: 1 440 | decay_mult: 0 441 | } 442 | param { 443 | lr_mult: 0 444 | decay_mult: 0 445 | } 446 | convolution_param { 447 | num_output: 512 448 | pad: 0 449 | kernel_size: 4 450 | stride: 1 451 | weight_filler { 452 | type: "gaussian" 453 | std: 0.02 454 | } 455 | bias_filler { 456 | type: "constant" 457 | } 458 | # engine: CUDNN 459 | } 460 | } 461 | 462 | layer { 463 | name: "deconv5_BN" 464 | type: "BatchNorm" include { phase: TRAIN} 465 | bottom: "deconv5" 466 | top: "deconv5_BN" 467 | param { 468 | lr_mult: 0 469 | decay_mult: 0 470 | } 471 | param { 472 | lr_mult: 0 473 | decay_mult: 0 474 | } 475 | param { 476 | lr_mult: 0 477 | decay_mult: 0 478 | } 479 | batch_norm_param { 480 | use_global_stats: false 481 | moving_average_fraction: 0.95 482 | } 483 | } 484 | layer { 485 | name: "deconv5_BN" 486 | type: "BatchNorm" include { phase: TEST} 487 | bottom: "deconv5" 488 | top: "deconv5_BN" 489 | param { 490 | lr_mult: 0 491 | decay_mult: 0 492 | } 493 | param { 494 | lr_mult: 0 495 | decay_mult: 0 496 | } 497 | param { 498 | lr_mult: 0 499 | decay_mult: 0 500 | } 501 | batch_norm_param { 502 | use_global_stats: true 503 | moving_average_fraction: 0.95 504 | } 505 | } 506 | 507 | layer { 508 | name: "relu_deconv5" 509 | type: "ReLU" 510 | bottom: "deconv5_BN" 511 | top: "deconv5_BN" 512 | } 513 | layer { 514 | name: "conv5_1" 515 | type: "Deconvolution" 516 | bottom: "deconv5_BN" 517 | top: "conv5_1" 518 | param { 519 | lr_mult: 1 520 | decay_mult: 0 521 | } 522 | param { 523 | lr_mult: 0 524 | decay_mult: 0 525 | } 526 | convolution_param { 527 | num_output: 256 528 | pad: 1 529 | kernel_size: 4 530 | stride: 2 531 | weight_filler { 532 | type: "gaussian" 533 | std: 0.02 534 | } 535 | bias_filler { 536 | type: "constant" 537 | } 538 | # engine: CUDNN 539 | } 540 | } 541 | layer { 542 | name: "deconv5_1_BN" 543 | type: "BatchNorm" include { phase: TRAIN} 544 | bottom: "conv5_1" 545 | top: "deconv5_1_BN" 546 | param { 547 | lr_mult: 0 548 | decay_mult: 0 549 | } 550 | param { 551 | lr_mult: 0 552 | decay_mult: 0 553 | } 554 | param { 555 | lr_mult: 0 556 | decay_mult: 0 557 | } 558 | batch_norm_param { 559 | use_global_stats: false 560 | moving_average_fraction: 0.95 561 | } 562 | } 563 | layer { 564 | name: "deconv5_1_BN" 565 | type: "BatchNorm" include { phase: TEST} 566 | bottom: "conv5_1" 567 | top: "deconv5_1_BN" 568 | param { 569 | lr_mult: 0 570 | decay_mult: 0 571 | } 572 | param { 573 | lr_mult: 0 574 | decay_mult: 0 575 | } 576 | param { 577 | lr_mult: 0 578 | decay_mult: 0 579 | } 580 | batch_norm_param { 581 | use_global_stats: true 582 | moving_average_fraction: 0.95 583 | } 584 | } 585 | 586 | layer { 587 | name: "relu_conv5_1" 588 | type: "ReLU" 589 | bottom: "deconv5_1_BN" 590 | top: "deconv5_1_BN" 591 | } 592 | layer { 593 | name: "deconv4" 594 | type: "Deconvolution" 595 | bottom: "deconv5_1_BN" 596 | top: "deconv4" 597 | param { 598 | lr_mult: 1 599 | decay_mult: 0 600 | } 601 | param { 602 | lr_mult: 0 603 | decay_mult: 0 604 | } 605 | convolution_param { 606 | num_output: 128 607 | pad: 1 608 | kernel_size: 4 609 | stride: 2 610 | weight_filler { 611 | type: "gaussian" 612 | std: 0.02 613 | } 614 | bias_filler { 615 | type: "constant" 616 | } 617 | # engine: CUDNN 618 | } 619 | } 620 | layer { 621 | name: "deconv4_BN" 622 | type: "BatchNorm" include { phase: TRAIN} 623 | bottom: "deconv4" 624 | top: "deconv4_BN" 625 | param { 626 | lr_mult: 0 627 | decay_mult: 0 628 | } 629 | param { 630 | lr_mult: 0 631 | decay_mult: 0 632 | } 633 | param { 634 | lr_mult: 0 635 | decay_mult: 0 636 | } 637 | batch_norm_param { 638 | use_global_stats: false 639 | moving_average_fraction: 0.95 640 | } 641 | } 642 | layer { 643 | name: "deconv4_BN" 644 | type: "BatchNorm" include { phase: TEST} 645 | bottom: "deconv4" 646 | top: "deconv4_BN" 647 | param { 648 | lr_mult: 0 649 | decay_mult: 0 650 | } 651 | param { 652 | lr_mult: 0 653 | decay_mult: 0 654 | } 655 | param { 656 | lr_mult: 0 657 | decay_mult: 0 658 | } 659 | batch_norm_param { 660 | use_global_stats: true 661 | moving_average_fraction: 0.95 662 | } 663 | } 664 | 665 | layer { 666 | name: "relu_deconv4" 667 | type: "ReLU" 668 | bottom: "deconv4_BN" 669 | top: "deconv4_BN" 670 | } 671 | layer { 672 | name: "conv4_1" 673 | type: "Deconvolution" 674 | bottom: "deconv4_BN" 675 | top: "conv4_1" 676 | param { 677 | lr_mult: 1 678 | decay_mult: 0 679 | } 680 | param { 681 | lr_mult: 0 682 | decay_mult: 0 683 | } 684 | convolution_param { 685 | num_output: 64 686 | pad: 1 687 | kernel_size: 4 688 | stride: 2 689 | weight_filler { 690 | type: "gaussian" 691 | std: 0.02 692 | } 693 | bias_filler { 694 | type: "constant" 695 | } 696 | # engine: CUDNN 697 | } 698 | } 699 | 700 | layer { 701 | name: "deconv4_1_BN" 702 | type: "BatchNorm" include { phase: TRAIN} 703 | bottom: "conv4_1" 704 | top: "deconv4_1_BN" 705 | param { 706 | lr_mult: 0 707 | decay_mult: 0 708 | } 709 | param { 710 | lr_mult: 0 711 | decay_mult: 0 712 | } 713 | param { 714 | lr_mult: 0 715 | decay_mult: 0 716 | } 717 | batch_norm_param { 718 | use_global_stats: false 719 | moving_average_fraction: 0.95 720 | } 721 | } 722 | layer { 723 | name: "deconv4_1_BN" 724 | type: "BatchNorm" include { phase: TEST} 725 | bottom: "conv4_1" 726 | top: "deconv4_1_BN" 727 | param { 728 | lr_mult: 0 729 | decay_mult: 0 730 | } 731 | param { 732 | lr_mult: 0 733 | decay_mult: 0 734 | } 735 | param { 736 | lr_mult: 0 737 | decay_mult: 0 738 | } 739 | batch_norm_param { 740 | use_global_stats: true 741 | moving_average_fraction: 0.95 742 | } 743 | } 744 | 745 | layer { 746 | name: "relu_conv4_1" 747 | type: "ReLU" 748 | bottom: "deconv4_1_BN" 749 | top: "deconv4_1_BN" 750 | } 751 | layer { 752 | name: "generated" 753 | type: "Deconvolution" 754 | bottom: "deconv4_1_BN" 755 | top: "residual" 756 | param { 757 | lr_mult: 1 758 | decay_mult: 0 759 | } 760 | param { 761 | lr_mult: 0 762 | decay_mult: 0 763 | } 764 | convolution_param { 765 | num_output: 3 766 | pad: 1 767 | kernel_size: 4 768 | stride: 2 769 | weight_filler { 770 | type: "gaussian" 771 | std: 0.02 772 | } 773 | bias_filler { 774 | type: "constant" 775 | } 776 | # engine: CUDNN 777 | } 778 | } 779 | 780 | layer { 781 | name:"sum" 782 | type:"Eltwise" 783 | bottom:"feat" 784 | bottom:"residual" 785 | top:"generated" 786 | eltwise_param{ 787 | operation:SUM 788 | } 789 | } 790 | -------------------------------------------------------------------------------- /EyeglassesV2/generator_dual.prototxt: -------------------------------------------------------------------------------- 1 | force_backward: true 2 | input: "feat" 3 | input_shape { 4 | dim: 64 5 | dim: 3 6 | dim:64 7 | dim:64 8 | } 9 | 10 | 11 | 12 | layer { 13 | name: "conv1" 14 | type: "Convolution" 15 | bottom: "feat" 16 | top: "conv1" 17 | param { 18 | lr_mult: 1 19 | decay_mult: 0 20 | } 21 | param { 22 | lr_mult: 0 23 | decay_mult: 0 24 | } 25 | convolution_param { 26 | num_output: 64 27 | pad: 1 28 | kernel_size: 4 29 | stride: 2 30 | weight_filler { 31 | type: "gaussian" 32 | std: 0.02 33 | } 34 | bias_filler { 35 | type: "constant" 36 | } 37 | } 38 | } 39 | 40 | 41 | layer { 42 | name: "conv1_BN" 43 | type: "BatchNorm" include { phase: TRAIN} 44 | bottom: "conv1" 45 | top: "conv1_BN" 46 | param { 47 | lr_mult: 0 48 | decay_mult: 0 49 | } 50 | param { 51 | lr_mult: 0 52 | decay_mult: 0 53 | } 54 | param { 55 | lr_mult: 0 56 | decay_mult: 0 57 | } 58 | batch_norm_param { 59 | use_global_stats: false 60 | moving_average_fraction: 0.95 61 | } 62 | } 63 | layer { 64 | name: "conv1_BN" 65 | type: "BatchNorm" include { phase: TEST} 66 | bottom: "conv1" 67 | top: "conv1_BN" 68 | param { 69 | lr_mult: 0 70 | decay_mult: 0 71 | } 72 | param { 73 | lr_mult: 0 74 | decay_mult: 0 75 | } 76 | param { 77 | lr_mult: 0 78 | decay_mult: 0 79 | } 80 | batch_norm_param { 81 | use_global_stats: true 82 | moving_average_fraction: 0.95 83 | } 84 | } 85 | 86 | layer { 87 | name: "relu_conv1" 88 | type: "ReLU" 89 | bottom: "conv1_BN" 90 | top: "relu_conv1" 91 | } 92 | 93 | 94 | layer { 95 | name: "conv2" 96 | type: "Convolution" 97 | bottom: "relu_conv1" 98 | top: "conv2" 99 | param { 100 | lr_mult: 1 101 | decay_mult: 0 102 | } 103 | param { 104 | lr_mult: 0 105 | decay_mult: 0 106 | } 107 | convolution_param { 108 | num_output: 128 109 | pad: 1 110 | kernel_size: 4 111 | stride: 2 112 | weight_filler { 113 | type: "gaussian" 114 | std: 0.02 115 | } 116 | bias_filler { 117 | type: "constant" 118 | } 119 | } 120 | } 121 | 122 | 123 | layer { 124 | name: "conv2_BN" 125 | type: "BatchNorm" include { phase: TRAIN} 126 | bottom: "conv2" 127 | top: "conv2_BN" 128 | param { 129 | lr_mult: 0 130 | decay_mult: 0 131 | } 132 | param { 133 | lr_mult: 0 134 | decay_mult: 0 135 | } 136 | param { 137 | lr_mult: 0 138 | decay_mult: 0 139 | } 140 | batch_norm_param { 141 | use_global_stats: false 142 | moving_average_fraction: 0.95 143 | } 144 | } 145 | layer { 146 | name: "conv2_BN" 147 | type: "BatchNorm" include { phase: TEST} 148 | bottom: "conv2" 149 | top: "conv2_BN" 150 | param { 151 | lr_mult: 0 152 | decay_mult: 0 153 | } 154 | param { 155 | lr_mult: 0 156 | decay_mult: 0 157 | } 158 | param { 159 | lr_mult: 0 160 | decay_mult: 0 161 | } 162 | batch_norm_param { 163 | use_global_stats: true 164 | moving_average_fraction: 0.95 165 | } 166 | } 167 | 168 | layer { 169 | name: "relu_conv2" 170 | type: "ReLU" 171 | bottom: "conv2_BN" 172 | top: "relu_conv2" 173 | } 174 | 175 | 176 | 177 | 178 | layer { 179 | name: "conv3" 180 | type: "Convolution" 181 | bottom: "relu_conv2" 182 | top: "conv3" 183 | param { 184 | lr_mult: 1 185 | decay_mult: 0 186 | } 187 | param { 188 | lr_mult: 0 189 | decay_mult: 0 190 | } 191 | convolution_param { 192 | num_output: 256 193 | pad: 1 194 | kernel_size: 4 195 | stride: 2 196 | weight_filler { 197 | type: "gaussian" 198 | std: 0.02 199 | } 200 | bias_filler { 201 | type: "constant" 202 | } 203 | } 204 | } 205 | 206 | 207 | layer { 208 | name: "conv3_BN" 209 | type: "BatchNorm" include { phase: TRAIN} 210 | bottom: "conv3" 211 | top: "conv3_BN" 212 | param { 213 | lr_mult: 0 214 | decay_mult: 0 215 | } 216 | param { 217 | lr_mult: 0 218 | decay_mult: 0 219 | } 220 | param { 221 | lr_mult: 0 222 | decay_mult: 0 223 | } 224 | batch_norm_param { 225 | use_global_stats: false 226 | moving_average_fraction: 0.95 227 | } 228 | } 229 | layer { 230 | name: "conv3_BN" 231 | type: "BatchNorm" include { phase: TEST} 232 | bottom: "conv3" 233 | top: "conv3_BN" 234 | param { 235 | lr_mult: 0 236 | decay_mult: 0 237 | } 238 | param { 239 | lr_mult: 0 240 | decay_mult: 0 241 | } 242 | param { 243 | lr_mult: 0 244 | decay_mult: 0 245 | } 246 | batch_norm_param { 247 | use_global_stats: true 248 | moving_average_fraction: 0.95 249 | } 250 | } 251 | 252 | layer { 253 | name: "relu_conv3" 254 | type: "ReLU" 255 | bottom: "conv3_BN" 256 | top: "relu_conv3" 257 | } 258 | 259 | 260 | 261 | 262 | 263 | 264 | layer { 265 | name: "conv4" 266 | type: "Convolution" 267 | bottom: "relu_conv3" 268 | top: "conv4" 269 | param { 270 | lr_mult: 1 271 | decay_mult: 0 272 | } 273 | param { 274 | lr_mult: 0 275 | decay_mult: 0 276 | } 277 | convolution_param { 278 | num_output: 512 279 | pad: 1 280 | kernel_size: 4 281 | stride: 2 282 | weight_filler { 283 | type: "gaussian" 284 | std: 0.02 285 | } 286 | bias_filler { 287 | type: "constant" 288 | } 289 | } 290 | } 291 | 292 | 293 | layer { 294 | name: "conv4_BN" 295 | type: "BatchNorm" include { phase: TRAIN} 296 | bottom: "conv4" 297 | top: "conv4_BN" 298 | param { 299 | lr_mult: 0 300 | decay_mult: 0 301 | } 302 | param { 303 | lr_mult: 0 304 | decay_mult: 0 305 | } 306 | param { 307 | lr_mult: 0 308 | decay_mult: 0 309 | } 310 | batch_norm_param { 311 | use_global_stats: false 312 | moving_average_fraction: 0.95 313 | } 314 | } 315 | layer { 316 | name: "conv4_BN" 317 | type: "BatchNorm" include { phase: TEST} 318 | bottom: "conv4" 319 | top: "conv4_BN" 320 | param { 321 | lr_mult: 0 322 | decay_mult: 0 323 | } 324 | param { 325 | lr_mult: 0 326 | decay_mult: 0 327 | } 328 | param { 329 | lr_mult: 0 330 | decay_mult: 0 331 | } 332 | batch_norm_param { 333 | use_global_stats: true 334 | moving_average_fraction: 0.95 335 | } 336 | } 337 | 338 | layer { 339 | name: "relu_conv4" 340 | type: "ReLU" 341 | bottom: "conv4_BN" 342 | top: "relu_conv4" 343 | } 344 | 345 | 346 | 347 | 348 | 349 | layer { 350 | name: "conv5" 351 | type: "Convolution" 352 | bottom: "relu_conv4" 353 | top: "conv5" 354 | param { 355 | lr_mult: 1 356 | decay_mult: 0 357 | } 358 | param { 359 | lr_mult: 0 360 | decay_mult: 0 361 | } 362 | convolution_param { 363 | num_output: 100 364 | pad: 0 365 | kernel_size: 4 366 | stride: 2 367 | weight_filler { 368 | type: "gaussian" 369 | std: 0.02 370 | } 371 | bias_filler { 372 | type: "constant" 373 | } 374 | } 375 | } 376 | 377 | 378 | layer { 379 | name: "conv5_BN" 380 | type: "BatchNorm" include { phase: TRAIN} 381 | bottom: "conv5" 382 | top: "conv5_BN" 383 | param { 384 | lr_mult: 0 385 | decay_mult: 0 386 | } 387 | param { 388 | lr_mult: 0 389 | decay_mult: 0 390 | } 391 | param { 392 | lr_mult: 0 393 | decay_mult: 0 394 | } 395 | batch_norm_param { 396 | use_global_stats: false 397 | moving_average_fraction: 0.95 398 | } 399 | } 400 | layer { 401 | name: "conv5_BN" 402 | type: "BatchNorm" include { phase: TEST} 403 | bottom: "conv5" 404 | top: "conv5_BN" 405 | param { 406 | lr_mult: 0 407 | decay_mult: 0 408 | } 409 | param { 410 | lr_mult: 0 411 | decay_mult: 0 412 | } 413 | param { 414 | lr_mult: 0 415 | decay_mult: 0 416 | } 417 | batch_norm_param { 418 | use_global_stats: true 419 | moving_average_fraction: 0.95 420 | } 421 | } 422 | 423 | layer { 424 | name: "relu_conv5" 425 | type: "ReLU" 426 | bottom: "conv5_BN" 427 | top: "relu_conv5" 428 | } 429 | 430 | 431 | 432 | 433 | layer { 434 | name: "deconv5" 435 | type: "Deconvolution" 436 | bottom: "relu_conv5" 437 | top: "deconv5" 438 | param { 439 | lr_mult: 1 440 | decay_mult: 0 441 | } 442 | param { 443 | lr_mult: 0 444 | decay_mult: 0 445 | } 446 | convolution_param { 447 | num_output: 512 448 | pad: 0 449 | kernel_size: 4 450 | stride: 1 451 | weight_filler { 452 | type: "gaussian" 453 | std: 0.02 454 | } 455 | bias_filler { 456 | type: "constant" 457 | } 458 | # engine: CUDNN 459 | } 460 | } 461 | 462 | layer { 463 | name: "deconv5_BN" 464 | type: "BatchNorm" include { phase: TRAIN} 465 | bottom: "deconv5" 466 | top: "deconv5_BN" 467 | param { 468 | lr_mult: 0 469 | decay_mult: 0 470 | } 471 | param { 472 | lr_mult: 0 473 | decay_mult: 0 474 | } 475 | param { 476 | lr_mult: 0 477 | decay_mult: 0 478 | } 479 | batch_norm_param { 480 | use_global_stats: false 481 | moving_average_fraction: 0.95 482 | } 483 | } 484 | layer { 485 | name: "deconv5_BN" 486 | type: "BatchNorm" include { phase: TEST} 487 | bottom: "deconv5" 488 | top: "deconv5_BN" 489 | param { 490 | lr_mult: 0 491 | decay_mult: 0 492 | } 493 | param { 494 | lr_mult: 0 495 | decay_mult: 0 496 | } 497 | param { 498 | lr_mult: 0 499 | decay_mult: 0 500 | } 501 | batch_norm_param { 502 | use_global_stats: true 503 | moving_average_fraction: 0.95 504 | } 505 | } 506 | 507 | layer { 508 | name: "relu_deconv5" 509 | type: "ReLU" 510 | bottom: "deconv5_BN" 511 | top: "deconv5_BN" 512 | } 513 | layer { 514 | name: "conv5_1" 515 | type: "Deconvolution" 516 | bottom: "deconv5_BN" 517 | top: "conv5_1" 518 | param { 519 | lr_mult: 1 520 | decay_mult: 0 521 | } 522 | param { 523 | lr_mult: 0 524 | decay_mult: 0 525 | } 526 | convolution_param { 527 | num_output: 256 528 | pad: 1 529 | kernel_size: 4 530 | stride: 2 531 | weight_filler { 532 | type: "gaussian" 533 | std: 0.02 534 | } 535 | bias_filler { 536 | type: "constant" 537 | } 538 | # engine: CUDNN 539 | } 540 | } 541 | layer { 542 | name: "deconv5_1_BN" 543 | type: "BatchNorm" include { phase: TRAIN} 544 | bottom: "conv5_1" 545 | top: "deconv5_1_BN" 546 | param { 547 | lr_mult: 0 548 | decay_mult: 0 549 | } 550 | param { 551 | lr_mult: 0 552 | decay_mult: 0 553 | } 554 | param { 555 | lr_mult: 0 556 | decay_mult: 0 557 | } 558 | batch_norm_param { 559 | use_global_stats: false 560 | moving_average_fraction: 0.95 561 | } 562 | } 563 | layer { 564 | name: "deconv5_1_BN" 565 | type: "BatchNorm" include { phase: TEST} 566 | bottom: "conv5_1" 567 | top: "deconv5_1_BN" 568 | param { 569 | lr_mult: 0 570 | decay_mult: 0 571 | } 572 | param { 573 | lr_mult: 0 574 | decay_mult: 0 575 | } 576 | param { 577 | lr_mult: 0 578 | decay_mult: 0 579 | } 580 | batch_norm_param { 581 | use_global_stats: true 582 | moving_average_fraction: 0.95 583 | } 584 | } 585 | 586 | layer { 587 | name: "relu_conv5_1" 588 | type: "ReLU" 589 | bottom: "deconv5_1_BN" 590 | top: "deconv5_1_BN" 591 | } 592 | layer { 593 | name: "deconv4" 594 | type: "Deconvolution" 595 | bottom: "deconv5_1_BN" 596 | top: "deconv4" 597 | param { 598 | lr_mult: 1 599 | decay_mult: 0 600 | } 601 | param { 602 | lr_mult: 0 603 | decay_mult: 0 604 | } 605 | convolution_param { 606 | num_output: 128 607 | pad: 1 608 | kernel_size: 4 609 | stride: 2 610 | weight_filler { 611 | type: "gaussian" 612 | std: 0.02 613 | } 614 | bias_filler { 615 | type: "constant" 616 | } 617 | # engine: CUDNN 618 | } 619 | } 620 | layer { 621 | name: "deconv4_BN" 622 | type: "BatchNorm" include { phase: TRAIN} 623 | bottom: "deconv4" 624 | top: "deconv4_BN" 625 | param { 626 | lr_mult: 0 627 | decay_mult: 0 628 | } 629 | param { 630 | lr_mult: 0 631 | decay_mult: 0 632 | } 633 | param { 634 | lr_mult: 0 635 | decay_mult: 0 636 | } 637 | batch_norm_param { 638 | use_global_stats: false 639 | moving_average_fraction: 0.95 640 | } 641 | } 642 | layer { 643 | name: "deconv4_BN" 644 | type: "BatchNorm" include { phase: TEST} 645 | bottom: "deconv4" 646 | top: "deconv4_BN" 647 | param { 648 | lr_mult: 0 649 | decay_mult: 0 650 | } 651 | param { 652 | lr_mult: 0 653 | decay_mult: 0 654 | } 655 | param { 656 | lr_mult: 0 657 | decay_mult: 0 658 | } 659 | batch_norm_param { 660 | use_global_stats: true 661 | moving_average_fraction: 0.95 662 | } 663 | } 664 | 665 | layer { 666 | name: "relu_deconv4" 667 | type: "ReLU" 668 | bottom: "deconv4_BN" 669 | top: "deconv4_BN" 670 | } 671 | layer { 672 | name: "conv4_1" 673 | type: "Deconvolution" 674 | bottom: "deconv4_BN" 675 | top: "conv4_1" 676 | param { 677 | lr_mult: 1 678 | decay_mult: 0 679 | } 680 | param { 681 | lr_mult: 0 682 | decay_mult: 0 683 | } 684 | convolution_param { 685 | num_output: 64 686 | pad: 1 687 | kernel_size: 4 688 | stride: 2 689 | weight_filler { 690 | type: "gaussian" 691 | std: 0.02 692 | } 693 | bias_filler { 694 | type: "constant" 695 | } 696 | # engine: CUDNN 697 | } 698 | } 699 | 700 | layer { 701 | name: "deconv4_1_BN" 702 | type: "BatchNorm" include { phase: TRAIN} 703 | bottom: "conv4_1" 704 | top: "deconv4_1_BN" 705 | param { 706 | lr_mult: 0 707 | decay_mult: 0 708 | } 709 | param { 710 | lr_mult: 0 711 | decay_mult: 0 712 | } 713 | param { 714 | lr_mult: 0 715 | decay_mult: 0 716 | } 717 | batch_norm_param { 718 | use_global_stats: false 719 | moving_average_fraction: 0.95 720 | } 721 | } 722 | layer { 723 | name: "deconv4_1_BN" 724 | type: "BatchNorm" include { phase: TEST} 725 | bottom: "conv4_1" 726 | top: "deconv4_1_BN" 727 | param { 728 | lr_mult: 0 729 | decay_mult: 0 730 | } 731 | param { 732 | lr_mult: 0 733 | decay_mult: 0 734 | } 735 | param { 736 | lr_mult: 0 737 | decay_mult: 0 738 | } 739 | batch_norm_param { 740 | use_global_stats: true 741 | moving_average_fraction: 0.95 742 | } 743 | } 744 | 745 | layer { 746 | name: "relu_conv4_1" 747 | type: "ReLU" 748 | bottom: "deconv4_1_BN" 749 | top: "deconv4_1_BN" 750 | } 751 | layer { 752 | name: "generated" 753 | type: "Deconvolution" 754 | bottom: "deconv4_1_BN" 755 | top: "residual" 756 | param { 757 | lr_mult: 1 758 | decay_mult: 0 759 | } 760 | param { 761 | lr_mult: 0 762 | decay_mult: 0 763 | } 764 | convolution_param { 765 | num_output: 3 766 | pad: 1 767 | kernel_size: 4 768 | stride: 2 769 | weight_filler { 770 | type: "gaussian" 771 | std: 0.02 772 | } 773 | bias_filler { 774 | type: "constant" 775 | } 776 | # engine: CUDNN 777 | } 778 | } 779 | 780 | layer { 781 | name:"sum" 782 | type:"Eltwise" 783 | bottom:"feat" 784 | bottom:"residual" 785 | top:"generated" 786 | eltwise_param{ 787 | operation:SUM 788 | } 789 | } 790 | -------------------------------------------------------------------------------- /EyeglassesV2/generator_original.prototxt: -------------------------------------------------------------------------------- 1 | force_backward: true 2 | input: "feat" 3 | input_shape { 4 | dim: 64 5 | dim: 100 6 | } 7 | layer { 8 | name: "reshape" 9 | type: "Reshape" 10 | bottom: "feat" 11 | top: "reshape_defc5" 12 | reshape_param { 13 | shape { 14 | dim: 64 15 | dim: 100 16 | dim: 1 17 | dim: 1 18 | } 19 | } 20 | } 21 | layer { 22 | name: "deconv5" 23 | type: "Deconvolution" 24 | bottom: "reshape_defc5" 25 | top: "deconv5" 26 | param { 27 | lr_mult: 1 28 | decay_mult: 0 29 | } 30 | param { 31 | lr_mult: 0 32 | decay_mult: 0 33 | } 34 | convolution_param { 35 | num_output: 512 36 | pad: 0 37 | kernel_size: 4 38 | stride: 1 39 | weight_filler { 40 | type: "gaussian" 41 | std: 0.02 42 | } 43 | bias_filler { 44 | type: "constant" 45 | } 46 | # engine: CUDNN 47 | } 48 | } 49 | 50 | layer { 51 | name: "deconv5_BN" 52 | type: "BatchNorm" include { phase: TRAIN} 53 | bottom: "deconv5" 54 | top: "deconv5_BN" 55 | param { 56 | lr_mult: 0 57 | decay_mult: 0 58 | } 59 | param { 60 | lr_mult: 0 61 | decay_mult: 0 62 | } 63 | param { 64 | lr_mult: 0 65 | decay_mult: 0 66 | } 67 | batch_norm_param { 68 | use_global_stats: false 69 | moving_average_fraction: 0.95 70 | } 71 | } 72 | layer { 73 | name: "deconv5_BN" 74 | type: "BatchNorm" include { phase: TEST} 75 | bottom: "deconv5" 76 | top: "deconv5_BN" 77 | param { 78 | lr_mult: 0 79 | decay_mult: 0 80 | } 81 | param { 82 | lr_mult: 0 83 | decay_mult: 0 84 | } 85 | param { 86 | lr_mult: 0 87 | decay_mult: 0 88 | } 89 | batch_norm_param { 90 | use_global_stats: true 91 | moving_average_fraction: 0.95 92 | } 93 | } 94 | 95 | layer { 96 | name: "relu_deconv5" 97 | type: "ReLU" 98 | bottom: "deconv5_BN" 99 | top: "deconv5_BN" 100 | } 101 | layer { 102 | name: "conv5_1" 103 | type: "Deconvolution" 104 | bottom: "deconv5_BN" 105 | top: "conv5_1" 106 | param { 107 | lr_mult: 1 108 | decay_mult: 0 109 | } 110 | param { 111 | lr_mult: 0 112 | decay_mult: 0 113 | } 114 | convolution_param { 115 | num_output: 256 116 | pad: 1 117 | kernel_size: 4 118 | stride: 2 119 | weight_filler { 120 | type: "gaussian" 121 | std: 0.02 122 | } 123 | bias_filler { 124 | type: "constant" 125 | } 126 | # engine: CUDNN 127 | } 128 | } 129 | layer { 130 | name: "deconv5_1_BN" 131 | type: "BatchNorm" include { phase: TRAIN} 132 | bottom: "conv5_1" 133 | top: "deconv5_1_BN" 134 | param { 135 | lr_mult: 0 136 | decay_mult: 0 137 | } 138 | param { 139 | lr_mult: 0 140 | decay_mult: 0 141 | } 142 | param { 143 | lr_mult: 0 144 | decay_mult: 0 145 | } 146 | batch_norm_param { 147 | use_global_stats: false 148 | moving_average_fraction: 0.95 149 | } 150 | } 151 | layer { 152 | name: "deconv5_1_BN" 153 | type: "BatchNorm" include { phase: TEST} 154 | bottom: "conv5_1" 155 | top: "deconv5_1_BN" 156 | param { 157 | lr_mult: 0 158 | decay_mult: 0 159 | } 160 | param { 161 | lr_mult: 0 162 | decay_mult: 0 163 | } 164 | param { 165 | lr_mult: 0 166 | decay_mult: 0 167 | } 168 | batch_norm_param { 169 | use_global_stats: true 170 | moving_average_fraction: 0.95 171 | } 172 | } 173 | 174 | layer { 175 | name: "relu_conv5_1" 176 | type: "ReLU" 177 | bottom: "deconv5_1_BN" 178 | top: "deconv5_1_BN" 179 | } 180 | layer { 181 | name: "deconv4" 182 | type: "Deconvolution" 183 | bottom: "deconv5_1_BN" 184 | top: "deconv4" 185 | param { 186 | lr_mult: 1 187 | decay_mult: 0 188 | } 189 | param { 190 | lr_mult: 0 191 | decay_mult: 0 192 | } 193 | convolution_param { 194 | num_output: 128 195 | pad: 1 196 | kernel_size: 4 197 | stride: 2 198 | weight_filler { 199 | type: "gaussian" 200 | std: 0.02 201 | } 202 | bias_filler { 203 | type: "constant" 204 | } 205 | # engine: CUDNN 206 | } 207 | } 208 | layer { 209 | name: "deconv4_BN" 210 | type: "BatchNorm" include { phase: TRAIN} 211 | bottom: "deconv4" 212 | top: "deconv4_BN" 213 | param { 214 | lr_mult: 0 215 | decay_mult: 0 216 | } 217 | param { 218 | lr_mult: 0 219 | decay_mult: 0 220 | } 221 | param { 222 | lr_mult: 0 223 | decay_mult: 0 224 | } 225 | batch_norm_param { 226 | use_global_stats: false 227 | moving_average_fraction: 0.95 228 | } 229 | } 230 | layer { 231 | name: "deconv4_BN" 232 | type: "BatchNorm" include { phase: TEST} 233 | bottom: "deconv4" 234 | top: "deconv4_BN" 235 | param { 236 | lr_mult: 0 237 | decay_mult: 0 238 | } 239 | param { 240 | lr_mult: 0 241 | decay_mult: 0 242 | } 243 | param { 244 | lr_mult: 0 245 | decay_mult: 0 246 | } 247 | batch_norm_param { 248 | use_global_stats: true 249 | moving_average_fraction: 0.95 250 | } 251 | } 252 | 253 | layer { 254 | name: "relu_deconv4" 255 | type: "ReLU" 256 | bottom: "deconv4_BN" 257 | top: "deconv4_BN" 258 | } 259 | layer { 260 | name: "conv4_1" 261 | type: "Deconvolution" 262 | bottom: "deconv4_BN" 263 | top: "conv4_1" 264 | param { 265 | lr_mult: 1 266 | decay_mult: 0 267 | } 268 | param { 269 | lr_mult: 0 270 | decay_mult: 0 271 | } 272 | convolution_param { 273 | num_output: 64 274 | pad: 1 275 | kernel_size: 4 276 | stride: 2 277 | weight_filler { 278 | type: "gaussian" 279 | std: 0.02 280 | } 281 | bias_filler { 282 | type: "constant" 283 | } 284 | # engine: CUDNN 285 | } 286 | } 287 | 288 | layer { 289 | name: "deconv4_1_BN" 290 | type: "BatchNorm" include { phase: TRAIN} 291 | bottom: "conv4_1" 292 | top: "deconv4_1_BN" 293 | param { 294 | lr_mult: 0 295 | decay_mult: 0 296 | } 297 | param { 298 | lr_mult: 0 299 | decay_mult: 0 300 | } 301 | param { 302 | lr_mult: 0 303 | decay_mult: 0 304 | } 305 | batch_norm_param { 306 | use_global_stats: false 307 | moving_average_fraction: 0.95 308 | } 309 | } 310 | layer { 311 | name: "deconv4_1_BN" 312 | type: "BatchNorm" include { phase: TEST} 313 | bottom: "conv4_1" 314 | top: "deconv4_1_BN" 315 | param { 316 | lr_mult: 0 317 | decay_mult: 0 318 | } 319 | param { 320 | lr_mult: 0 321 | decay_mult: 0 322 | } 323 | param { 324 | lr_mult: 0 325 | decay_mult: 0 326 | } 327 | batch_norm_param { 328 | use_global_stats: true 329 | moving_average_fraction: 0.95 330 | } 331 | } 332 | 333 | layer { 334 | name: "relu_conv4_1" 335 | type: "ReLU" 336 | bottom: "deconv4_1_BN" 337 | top: "deconv4_1_BN" 338 | } 339 | layer { 340 | name: "generated" 341 | type: "Deconvolution" 342 | bottom: "deconv4_1_BN" 343 | top: "generated" 344 | param { 345 | lr_mult: 1 346 | decay_mult: 0 347 | } 348 | param { 349 | lr_mult: 0 350 | decay_mult: 0 351 | } 352 | convolution_param { 353 | num_output: 3 354 | pad: 1 355 | kernel_size: 4 356 | stride: 2 357 | weight_filler { 358 | type: "gaussian" 359 | std: 0.02 360 | } 361 | bias_filler { 362 | type: "constant" 363 | } 364 | # engine: CUDNN 365 | } 366 | } 367 | 368 | #layer { 369 | # name: "deconv3_relu" 370 | # type: "TanH" 371 | # bottom: "deconv3" 372 | # top: "deconv3_relu" 373 | #} 374 | 375 | #layer { 376 | # name: "deconv0_crop" 377 | # type: "CropSimple" 378 | # bottom: "deconv3" 379 | # top: "deconv0_crop" 380 | # crop_param { 381 | # crop_height: 64 382 | # crop_width: 64 383 | # } 384 | #} 385 | #layer { 386 | # name: "generated" 387 | # type: "Eltwise" 388 | # bottom: "deconv3" 389 | # top: "generated" 390 | #} 391 | -------------------------------------------------------------------------------- /EyeglassesV2/pixloss.prototxt: -------------------------------------------------------------------------------- 1 | force_backward:true 2 | input:"data" 3 | input_shape{ 4 | dim:64 5 | dim:3 6 | dim:64 7 | dim:64 8 | } 9 | 10 | layer { 11 | name:"l1reg" 12 | type:"Reduction" 13 | bottom:"data" 14 | top:"l1reg" 15 | reduction_param{ 16 | operation:ASUM 17 | axis:0 18 | } 19 | loss_weight:1 20 | } 21 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_data.prototxt: -------------------------------------------------------------------------------- 1 | net: "data.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_data_dual.prototxt: -------------------------------------------------------------------------------- 1 | net: "data_dual.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_data_gen.prototxt: -------------------------------------------------------------------------------- 1 | net: "data_gen.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_discriminator.prototxt: -------------------------------------------------------------------------------- 1 | net: "discriminator.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_generator.prototxt: -------------------------------------------------------------------------------- 1 | net: "generator.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_generator_dual.prototxt: -------------------------------------------------------------------------------- 1 | net: "generator_dual.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_pixloss.prototxt: -------------------------------------------------------------------------------- 1 | net: "pixloss.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/solver_template.prototxt: -------------------------------------------------------------------------------- 1 | net: "@NET@.prototxt" 2 | display: 0 3 | # time_per_iter: 1 4 | base_lr: 0.0002 5 | momentum: 0.5 6 | momentum2: 0.999 7 | weight_decay: 0.0004 8 | lr_policy: "multistep" 9 | gamma: 0.5 10 | stepvalue: 6000 11 | stepvalue: 10000 12 | stepvalue: 140000 13 | stepvalue: 180000 14 | stepvalue: 220000 15 | max_iter: 15000 16 | solver_mode: GPU 17 | solver_type: ADAM 18 | device_id: 0 19 | 20 | -------------------------------------------------------------------------------- /EyeglassesV2/train.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import time 3 | import os 4 | import sys 5 | sys.path.insert(0,'../caffe-fr-chairs-deepsim/python/') 6 | print sys.path 7 | import caffe 8 | if len(sys.argv) == 1: 9 | start_snapshot = 0 10 | 11 | image_size = 64 # image size 12 | max_iter = int(1e6) # maximum number of iterations 13 | display_every = 20 # show losses every so many iterations 14 | snapshot_every = 1000 # snapshot every so many iterations 15 | snapshot_folder = 'snapshots_test' # where to save the snapshots (and load from) 16 | gpu_id = 3 17 | im_size = (3,image_size,image_size) 18 | batch_size = 64 19 | alpha = 1e-6 # The L1 regulazation(pix_loss) parameter 20 | snapshot_at_iter = -1 21 | snapshot_at_iter_file = 'snapshot_at_iter.txt' 22 | 23 | sub_nets = ('generator','generator_dual', 'discriminator', 24 | 'data_gen','data_dual','pixloss') 25 | 26 | if not os.path.exists(snapshot_folder): 27 | os.makedirs(snapshot_folder) 28 | 29 | #make solvers 30 | with open ("solver_template.prototxt", "r") as myfile: 31 | solver_template=myfile.read() 32 | 33 | for curr_net in sub_nets: 34 | with open("solver_%s.prototxt" % curr_net, "w") as myfile: 35 | myfile.write(solver_template.replace('@NET@', curr_net)) 36 | 37 | #initialize the nets 38 | caffe.set_device(gpu_id) 39 | caffe.set_mode_gpu() 40 | generator = caffe.AdamSolver('solver_generator.prototxt') 41 | generator_dual = caffe.AdamSolver('solver_generator_dual.prototxt') 42 | discriminator = caffe.AdamSolver('solver_discriminator.prototxt') 43 | data_reader_gen = caffe.AdamSolver('solver_data_gen.prototxt') 44 | data_reader_dual = caffe.AdamSolver('solver_data_dual.prototxt') 45 | pix_loss_com = caffe.AdamSolver('solver_pixloss.prototxt') 46 | 47 | ##load from snapshot 48 | #if start_snapshot: 49 | # curr_snapshot_folder = snapshot_folder +'/' + str(start_snapshot) 50 | # print >> sys.stderr, '\n === Starting from snapshot ' + curr_snapshot_folder + ' ===\n' 51 | # generator_caffemodel = curr_snapshot_folder +'/' + 'generator.caffemodel' 52 | # if os.path.isfile(generator_caffemodel): 53 | # generator.net.copy_from(generator_caffemodel) 54 | # else: 55 | # raise Exception('File %s does not exist' % generator_caffemodel) 56 | # discriminator_caffemodel = curr_snapshot_folder +'/' + 'discriminator.caffemodel' 57 | # if os.path.isfile(discriminator_caffemodel): 58 | # discriminator.net.copy_from(discriminator_caffemodel) 59 | # else: 60 | # raise Exception('File %s does not exist' % discriminator_caffemodel) 61 | 62 | #read weights of losses 63 | discr_loss_weight = discriminator.net._blob_loss_weights[discriminator.net._blob_names_index['discr_loss']] 64 | 65 | train_discr = True 66 | train_gen = True 67 | train_gen_dual = True 68 | 69 | #do training 70 | start = time.time() 71 | for it in range(start_snapshot,max_iter): 72 | 73 | # read the data 74 | data_reader_gen.net.forward_simple() 75 | data_reader_dual.net.forward_simple() 76 | 77 | # feed the negative data to the generator and run it 78 | generator.net.blobs['feat'].data[...] = data_reader_gen.net.blobs['data'].data 79 | generator.net.forward_simple() 80 | generated_img = generator.net.blobs['generated'].data 81 | 82 | # feed the positive data to the dual generator 83 | generator_dual.net.blobs['feat'].data[...] = data_reader_dual.net.blobs['data'].data 84 | generator_dual.net.forward_simple() 85 | generated_img_dual = generator_dual.net.blobs['generated'].data 86 | 87 | # run the discriminator on real positive data 88 | discriminator.net.blobs['data'].data[...] = data_reader_dual.net.blobs['data'].data 89 | discriminator.net.blobs['label'].data[...] = np.ones((batch_size,1), dtype='float32') 90 | discriminator.net.forward_simple() 91 | discr_pos_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 92 | if train_discr: 93 | discriminator.increment_iter() 94 | discriminator.net.clear_param_diffs() 95 | discriminator.net.backward_simple() 96 | 97 | # run the discriminator on real negative data 98 | discriminator.net.blobs['data'].data[...] = data_reader_gen.net.blobs['data'].data 99 | discriminator.net.blobs['label'].data[...] = 2 * np.ones((batch_size,1), dtype='float32') 100 | discriminator.net.forward_simple() 101 | discr_neg_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 102 | if train_discr: 103 | discriminator.net.backward_simple() 104 | 105 | # run the discriminator on generated data 106 | discriminator.net.blobs['data'].data[...] = generated_img 107 | discriminator.net.blobs['label'].data[...] = np.zeros((batch_size,1), dtype='float32') 108 | discriminator.net.forward_simple() 109 | discr_fake_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 110 | if train_discr: 111 | discriminator.net.backward_simple() 112 | 113 | # run the discriminator on dual generated data 114 | discriminator.net.blobs['data'].data[...] = generated_img_dual 115 | discriminator.net.blobs['label'].data[...] = np.zeros((batch_size,1), dtype='float32') 116 | discriminator.net.forward_simple() 117 | discr_fake_dual_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 118 | if train_discr: 119 | discriminator.net.backward_simple() 120 | discriminator.apply_update() 121 | 122 | # run the discriminator on generated data with opposite label 1, to get the gradient for the generator 123 | discriminator.net.blobs['data'].data[...] = generated_img 124 | discriminator.net.blobs['label'].data[...] = np.ones((batch_size,1), dtype='float32') 125 | discriminator.net.forward_simple() 126 | discr_fake_for_generator_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 127 | pix_loss_com.net.blobs['data'].data[...] = generator.net.blobs['residual'].data[...] 128 | pix_loss_com.net.forward_simple() 129 | pix_loss_gen = alpha * np.copy(pix_loss_com.net.blobs['l1reg'].data[...]) / batch_size 130 | if train_gen: 131 | generator.increment_iter() 132 | generator.net.clear_param_diffs() 133 | discriminator.net.backward_simple() 134 | pix_loss_com.net.backward_simple() 135 | generator.net.blobs['generated'].diff[...] = discriminator.net.blobs['data'].diff + \ 136 | alpha * pix_loss_com.net.blobs['data'].diff / batch_size 137 | generator.net.backward_simple() 138 | generator.apply_update() 139 | 140 | # run the discriminator on dual generated data with label 2, to get the gradient for the dual generator 141 | discriminator.net.blobs['data'].data[...] = generated_img_dual 142 | discriminator.net.blobs['label'].data[...] = 2*np.ones((batch_size,1), dtype='float32') 143 | discriminator.net.forward_simple() 144 | discr_fake_for_dual_generator_loss = np.copy(discriminator.net.blobs['discr_loss'].data) 145 | pix_loss_com.net.blobs['data'].data[...] = generator_dual.net.blobs['residual'].data[...] 146 | pix_loss_com.net.forward_simple() 147 | pix_loss_gen_dual = alpha * np.copy(pix_loss_com.net.blobs['l1reg'].data[...]) / batch_size 148 | 149 | if train_gen_dual: 150 | generator_dual.increment_iter() 151 | generator_dual.net.clear_param_diffs() 152 | discriminator.net.backward_simple() 153 | pix_loss_com.net.backward_simple() 154 | generator_dual.net.blobs['generated'].diff[...] = discriminator.net.blobs['data'].diff + \ 155 | alpha * pix_loss_com.net.blobs['data'].diff / batch_size 156 | generator_dual.net.backward_simple() 157 | generator_dual.apply_update() 158 | 159 | 160 | #display 161 | if it % display_every == 0: 162 | print >> sys.stderr, "[%s] Iteration %d: %f seconds" % (time.strftime("%c"), it, time.time()-start) 163 | print >> sys.stderr, " discr real positive loss: %e * %e = %f" % (discr_pos_loss, discr_loss_weight, discr_pos_loss*discr_loss_weight) 164 | print >> sys.stderr, " discr real negative loss: %e * %e = %f" % (discr_neg_loss, discr_loss_weight, discr_neg_loss*discr_loss_weight) 165 | #print >> sys.stderr, " discr fake loss: %e * %e = %f" % (discr_fake_loss, discr_loss_weight, discr_fake_loss*discr_loss_weight) 166 | print >> sys.stderr, " discr fake loss for generator: %e * %e = %f" % (discr_fake_for_generator_loss, discr_loss_weight, discr_fake_for_generator_loss*discr_loss_weight) 167 | print >> sys.stderr, " discr fake loss for dual generator: %e * %e = %f" % (discr_fake_for_dual_generator_loss, discr_loss_weight, discr_fake_for_dual_generator_loss*discr_loss_weight) 168 | 169 | start = time.time() 170 | if os.path.isfile(snapshot_at_iter_file): 171 | with open (snapshot_at_iter_file, "r") as myfile: 172 | snapshot_at_iter = int(myfile.read()) 173 | 174 | #snapshot 175 | if it % snapshot_every == 0 or it == snapshot_at_iter: 176 | curr_snapshot_folder = snapshot_folder +'/' + str(it) 177 | print >> sys.stderr, '\n === Saving snapshot to ' + curr_snapshot_folder + ' ===\n' 178 | if not os.path.exists(curr_snapshot_folder): 179 | os.makedirs(curr_snapshot_folder) 180 | generator_caffemodel = curr_snapshot_folder + '/' + 'generator.caffemodel' 181 | generator.net.save(generator_caffemodel) 182 | generator_dual_caffemodel = curr_snapshot_folder + '/' + 'generator_dual.caffemodel' 183 | generator_dual.net.save(generator_dual_caffemodel) 184 | discriminator_caffemodel = curr_snapshot_folder + '/' + 'discriminator.caffemodel' 185 | discriminator.net.save(discriminator_caffemodel) 186 | 187 | #switch optimizing discriminator and generator, so that neither of them overfits too much 188 | discr_loss_ratio = (discr_pos_loss + discr_neg_loss + discr_fake_loss + discr_fake_dual_loss) / discr_fake_for_generator_loss 189 | if discr_loss_ratio < 2e-1 and train_discr: 190 | train_discr = False 191 | train_gen = True 192 | if discr_loss_ratio > 5e-1 and not train_discr: 193 | train_discr = True 194 | train_gen = True 195 | if discr_loss_ratio > 1e1 and train_gen: 196 | train_gen = False 197 | train_discr = True 198 | print >> sys.stderr, "[real_pos_loss=%e, real_neg_loss=%e, fake_loss=%e, fake_loss_for_generator=%e, train_discr=%d, train_gen=%d ]" % (discr_pos_loss,discr_neg_loss, discr_fake_loss, discr_fake_for_generator_loss, train_discr, train_gen) 199 | 200 | 201 | #switch optimizing discriminator and dual generator, so that neither of them overfits too much 202 | discr_loss_ratio = (discr_pos_loss + discr_neg_loss + discr_fake_loss + discr_fake_dual_loss) / discr_fake_for_dual_generator_loss 203 | if discr_loss_ratio < 2e-1 and train_discr: 204 | train_discr = False 205 | train_gen_dual = True 206 | if discr_loss_ratio > 5e-1 and not train_discr: 207 | train_discr = True 208 | train_gen_dual = True 209 | if discr_loss_ratio > 1e1 and train_gen: 210 | train_gen_dual = False 211 | train_discr = True 212 | print >> sys.stderr, "[ DLoss=%e,GLoss=%e,GDLoss=%e]"%(discr_pos_loss+discr_neg_loss+discr_fake_loss+discr_fake_dual_loss,discr_fake_for_generator_loss+pix_loss_gen,discr_fake_for_dual_generator_loss+pix_loss_gen_dual) 213 | print >> sys.stderr, "[ DRLoss=%e, DFLoss=%e, DDLoss=%e, GLoss=%e ,GDLoss=%e, GPLoss=%e, GDPLoss=%e,train_discr=%d, train_gen=%d, train_gen_dual=%d ]" % (discr_pos_loss+discr_neg_loss, discr_fake_loss,discr_fake_dual_loss, discr_fake_for_generator_loss, discr_fake_for_dual_generator_loss, pix_loss_gen,pix_loss_gen_dual,train_discr,train_gen, train_gen_dual) 214 | 215 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## GAN Face Attribute Manipulation with Caffe Implementation 2 | 3 | This project uses Caffe python interface to implement [Face Attribute Manipulation](https://arxiv.org/abs/1612.05363). 4 | 5 | Thanks to the enhanced version of py-caffe in [deepsim-caffe](https://github.com/dosovits/caffe-fr-chairs/tree/deepsim), a few lines of python code can train GANs quickly without any hack in caffe core lib. At present this project is incomplete yet, with only GAN loss + pixel loss + residual learning are implemented. 6 | 7 | ## Demo 8 | - Generate Eyeglasses 9 | ![gen](assets/gen.png) 10 | - Remove Eyeglasses 11 | ![remove](assets/remove.png) 12 | 13 | ## Setup 14 | 15 | You need to clone the [caffe-deepsim-branch](https://github.com/dosovits/caffe-fr-chairs/tree/deepsim) and compile py-caffe. Then replcace your caffe path in both train.py and generate.py. 16 | 17 | ## Training 18 | 19 | - First we need to prepare the CelebA dataset since we need the annotation of face attributes. [link](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) 20 | - Divided the image list into two lists according to some specific attribute, e.g. Eyeglasses here. The two lists are in the following format: 21 | ``` 22 | Positive.txt 23 | 000053.jpg 0 24 | 000093.jpg 0 25 | ... 26 | 27 | Negative.txt 28 | 000001.jpg 0 29 | 000002.jpg 0 30 | ... 31 | ``` 32 | Note that the label 0 here is meaningless since we don't need them, we add labels here just for using the ImageDataLayer in Caffe. When the lists are ready you need to replace the source fileds in data_gen.prototxt and data_dual.prototxt with the prepared lists. 33 | 34 | - Run the training scripts 35 | 36 | $ python train.py 37 | 38 | ## Testing 39 | 40 | $ mkdir test_result 41 | $ python generate.py generator.prototxt ./snapshots_test/3000/generator.caffemodel 42 | 43 | ## TODO 44 | 45 | - Implement the perceptual loss.(I have implement a version but it seems not to work well) 46 | - Adding dual learning. 47 | - Change the network architecture to produce larger images(128x128), now the output is 64x64. 48 | 49 | -------------------------------------------------------------------------------- /assets/gen.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Zhongdao/FaceAttributeManipulation/075163247b7c493c3cf75d5ea43ca5935914afcc/assets/gen.png -------------------------------------------------------------------------------- /assets/remove.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Zhongdao/FaceAttributeManipulation/075163247b7c493c3cf75d5ea43ca5935914afcc/assets/remove.png --------------------------------------------------------------------------------