├── Images ├── Network_Overview_NEW.png └── test ├── README.md ├── datasets └── DES │ ├── HHA │ ├── RGBD_data_1.jpg │ ├── RGBD_data_2.jpg │ └── RGBD_data_3.jpg │ └── RGB │ ├── RGBD_data_1.jpg │ ├── RGBD_data_2.jpg │ └── RGBD_data_3.jpg ├── depth2HHA.zip ├── models └── test_RGBD.prototxt ├── salmaps └── DES │ ├── RGBD_data_1.png │ ├── RGBD_data_2.png │ └── RGBD_data_3.png └── test_matlab ├── test_DES_HHA.m └── test_ICNet.m /Images/Network_Overview_NEW.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/Images/Network_Overview_NEW.png -------------------------------------------------------------------------------- /Images/test: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ICNet-for-RGBD-SOD 2 | This project provides the codes and results for 'ICNet: Information Conversion Network for RGB-D Based Salient Object Detection', TIP 2020. [Paper link](https://ieeexplore.ieee.org/document/9024241) [Homepage](https://mathlee.github.io/) 3 | 4 | Our code is implemented based on the Caffe of FlowNet2. You should first install and compile the caffe according to the [FlowNet2](https://github.com/lmb-freiburg/flownet2). 5 | 6 | # Overview 7 | ![Image](https://github.com/MathLee/ICNet-for-RGBD-SOD/blob/master/Images/Network_Overview_NEW.png) 8 | 9 | # Saliency maps and Measure results on 8 Datasets 10 | We provide [saliency maps](https://pan.baidu.com/s/1Bkl7DYdt97orbQX66BufuQ) (code: bqvj) and [measure results](https://pan.baidu.com/s/1oUImRe0zRna0o3-_JLBlXQ) (code: r0b6) of our ICNet on 5 datasets (STEREO, NJU2K, LFSD, DES and NLPR). 11 | 12 | We also provide [saliency maps & measure results](https://pan.baidu.com/s/1S6jRKISkwTELmrEm8QtZsA) (code: ujdp) of our ICNet on DUT-RGBD, SIP and SSD datasets. 13 | 14 | You can use the [evaluation tool](https://github.com/MathLee/MatlabEvaluationTools) to evaluate the result maps. 15 | 16 | The parameter amount of our ICNet is 77.95M. 17 | 18 | The FLOPs of our ICNet is 125.72G. 19 | 20 | # Testing 21 | 1. `test_RGBD.prototxt/` is under `models/`. 22 | 2. Download the [trained model](https://pan.baidu.com/s/1N9kvRjMqNwUL6K1cr8gIGA) (code: 6jz7) (`RGBD_iter_25000.caffemodel`), and put it under `models/`. 23 | 3. The datasets are under `datasets/`, we provide some testing examples on DES dataset. 24 | 4. Download depth2HHA.zip and unzip it, run depth2HHA.m to convert depth map to HHA. 25 | 5. Run `test_matlab/test_ICNet.m`. 26 | 6. Saliency maps are saved under `salmaps/DES/`. 27 | 28 | # Related works on RGB-D SOD 29 | (**TIP_2021_HAINet**) [Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection](https://github.com/MathLee/HAINet). 30 | 31 | (**ECCV_2020_CMWNet**) [Cross-Modal Weighting Network for RGB-D Salient Object Detection](https://github.com/MathLee/CMWNet). 32 | 33 | (**Survey**) [RGB-D Salient Object Detection: A Survey](https://github.com/taozh2017/RGBD-SODsurvey). 34 | 35 | # Citation 36 | @ARTICLE{Li_2020_ICNet, 37 | author = {Li, Gongyang and Liu, Zhi and Ling, Haibin}, 38 | title = {ICNet: Information Conversion Network for RGB-D Based Salient Object Detection}, 39 | journal = {IEEE Transactions on Image Processing}, 40 | year = {2020}, 41 | volume = {29}, 42 | pages = {4873-4884},} 43 | 44 | If you encounter any problems with the code, want to report bugs, etc. 45 | 46 | Please contact me at lllmiemie@163.com or ligongyang@shu.edu.cn. 47 | -------------------------------------------------------------------------------- /datasets/DES/HHA/RGBD_data_1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/HHA/RGBD_data_1.jpg -------------------------------------------------------------------------------- /datasets/DES/HHA/RGBD_data_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/HHA/RGBD_data_2.jpg -------------------------------------------------------------------------------- /datasets/DES/HHA/RGBD_data_3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/HHA/RGBD_data_3.jpg -------------------------------------------------------------------------------- /datasets/DES/RGB/RGBD_data_1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/RGB/RGBD_data_1.jpg -------------------------------------------------------------------------------- /datasets/DES/RGB/RGBD_data_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/RGB/RGBD_data_2.jpg -------------------------------------------------------------------------------- /datasets/DES/RGB/RGBD_data_3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/datasets/DES/RGB/RGBD_data_3.jpg -------------------------------------------------------------------------------- /depth2HHA.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/depth2HHA.zip -------------------------------------------------------------------------------- /models/test_RGBD.prototxt: -------------------------------------------------------------------------------- 1 | name: "RGBD" 2 | input: "data_im" 3 | input_dim: 1 4 | input_dim: 3 5 | input_dim: 288 6 | input_dim: 288 7 | 8 | input: "data_Depth" 9 | input_dim: 1 10 | input_dim: 3 11 | input_dim: 288 12 | input_dim: 288 13 | 14 | ####################################################################### 15 | ## VGG16 ## 16 | ####################################################################### 17 | ######提取卷积特征层VGG16###### 18 | ###Data_im### 19 | layer { 20 | bottom: "data_im" 21 | top: "conv1_1" 22 | name: "conv1_1" 23 | type: "Convolution" 24 | param { 25 | name: "conv1_1_w" 26 | lr_mult: 1 27 | decay_mult: 1 28 | } 29 | param { 30 | name: "conv1_1_b" 31 | lr_mult: 2 32 | decay_mult: 0 33 | } 34 | convolution_param { 35 | weight_filler { 36 | type: "gaussian" 37 | std: 0.01 38 | } 39 | bias_filler { 40 | type: "constant" 41 | value: 0 42 | } 43 | num_output: 64 44 | pad: 1 45 | kernel_size: 3 46 | } 47 | } 48 | layer { 49 | bottom: "conv1_1" 50 | top: "conv1_1" 51 | name: "bn_conv1_1" 52 | type: "BatchNorm" 53 | param { 54 | name: "conv1_1_bn_w" 55 | lr_mult: 1 56 | decay_mult: 1 57 | } 58 | param { 59 | name: "conv1_1_bn_b" 60 | lr_mult: 2 61 | decay_mult: 0 62 | } 63 | batch_norm_param { 64 | use_global_stats: false 65 | } 66 | } 67 | layer { 68 | bottom: "conv1_1" 69 | top: "conv1_1" 70 | name: "scale_conv1_1" 71 | type: "Scale" 72 | param { 73 | name: "conv1_1_scale_w" 74 | lr_mult: 1 75 | decay_mult: 1 76 | } 77 | param { 78 | name: "conv1_1_scale_b" 79 | lr_mult: 2 80 | decay_mult: 0 81 | } 82 | scale_param { 83 | bias_term: true 84 | } 85 | } 86 | layer { 87 | bottom: "conv1_1" 88 | top: "conv1_1" 89 | name: "relu1_1" 90 | type: "ReLU" 91 | } 92 | layer { 93 | bottom: "conv1_1" 94 | top: "conv1_2" 95 | name: "conv1_2" 96 | type: "Convolution" 97 | param { 98 | name: "conv1_2_w" 99 | lr_mult: 1 100 | decay_mult: 1 101 | } 102 | param { 103 | name: "conv1_2_b" 104 | lr_mult: 2 105 | decay_mult: 0 106 | } 107 | convolution_param { 108 | weight_filler { 109 | type: "gaussian" 110 | std: 0.01 111 | } 112 | bias_filler { 113 | type: "constant" 114 | value: 0 115 | } 116 | num_output: 64 117 | pad: 1 118 | kernel_size: 3 119 | } 120 | } 121 | layer { 122 | bottom: "conv1_2" 123 | top: "conv1_2" 124 | name: "bn_conv1_2" 125 | type: "BatchNorm" 126 | param { 127 | name: "conv1_2_bn_w" 128 | lr_mult: 1 129 | decay_mult: 1 130 | } 131 | param { 132 | name: "conv1_2_bn_b" 133 | lr_mult: 2 134 | decay_mult: 0 135 | } 136 | batch_norm_param { 137 | use_global_stats: false 138 | } 139 | } 140 | layer { 141 | bottom: "conv1_2" 142 | top: "conv1_2" 143 | name: "scale_conv1_2" 144 | type: "Scale" 145 | param { 146 | name: "conv1_2_scale_w" 147 | lr_mult: 1 148 | decay_mult: 1 149 | } 150 | param { 151 | name: "conv1_2_scale_b" 152 | lr_mult: 2 153 | decay_mult: 0 154 | } 155 | scale_param { 156 | bias_term: true 157 | } 158 | } 159 | layer { 160 | bottom: "conv1_2" 161 | top: "conv1_2" 162 | name: "relu1_2" 163 | type: "ReLU" 164 | } 165 | layer { 166 | bottom: "conv1_2" 167 | top: "pool1" 168 | name: "pool1" 169 | type: "Pooling" 170 | pooling_param { 171 | pool: MAX 172 | kernel_size: 2 173 | stride: 2 174 | } 175 | } 176 | layer { 177 | bottom: "pool1" 178 | top: "conv2_1" 179 | name: "conv2_1" 180 | type: "Convolution" 181 | param { 182 | name: "conv2_1_w" 183 | lr_mult: 1 184 | decay_mult: 1 185 | } 186 | param { 187 | name: "conv2_1_b" 188 | lr_mult: 2 189 | decay_mult: 0 190 | } 191 | convolution_param { 192 | weight_filler { 193 | type: "gaussian" 194 | std: 0.01 195 | } 196 | bias_filler { 197 | type: "constant" 198 | value: 0 199 | } 200 | num_output: 128 201 | pad: 1 202 | kernel_size: 3 203 | } 204 | } 205 | layer { 206 | bottom: "conv2_1" 207 | top: "conv2_1" 208 | name: "bn_conv2_1" 209 | type: "BatchNorm" 210 | param { 211 | name: "conv2_1_bn_w" 212 | lr_mult: 1 213 | decay_mult: 1 214 | } 215 | param { 216 | name: "conv2_1_bn_b" 217 | lr_mult: 2 218 | decay_mult: 0 219 | } 220 | batch_norm_param { 221 | use_global_stats: false 222 | } 223 | } 224 | layer { 225 | bottom: "conv2_1" 226 | top: "conv2_1" 227 | name: "scale_conv2_1" 228 | type: "Scale" 229 | param { 230 | name: "conv2_1_scale_w" 231 | lr_mult: 1 232 | decay_mult: 1 233 | } 234 | param { 235 | name: "conv2_1_scale_b" 236 | lr_mult: 2 237 | decay_mult: 0 238 | } 239 | scale_param { 240 | bias_term: true 241 | } 242 | } 243 | layer { 244 | bottom: "conv2_1" 245 | top: "conv2_1" 246 | name: "relu2_1" 247 | type: "ReLU" 248 | } 249 | layer { 250 | bottom: "conv2_1" 251 | top: "conv2_2" 252 | name: "conv2_2" 253 | type: "Convolution" 254 | param { 255 | name: "conv2_2_w" 256 | lr_mult: 1 257 | decay_mult: 1 258 | } 259 | param { 260 | name: "conv2_2_b" 261 | lr_mult: 2 262 | decay_mult: 0 263 | } 264 | convolution_param { 265 | weight_filler { 266 | type: "gaussian" 267 | std: 0.01 268 | } 269 | bias_filler { 270 | type: "constant" 271 | value: 0 272 | } 273 | num_output: 128 274 | pad: 1 275 | kernel_size: 3 276 | } 277 | } 278 | layer { 279 | bottom: "conv2_2" 280 | top: "conv2_2" 281 | name: "bn_conv2_2" 282 | type: "BatchNorm" 283 | param { 284 | name: "conv2_2_bn_w" 285 | lr_mult: 1 286 | decay_mult: 1 287 | } 288 | param { 289 | name: "conv2_2_bn_b" 290 | lr_mult: 2 291 | decay_mult: 0 292 | } 293 | batch_norm_param { 294 | use_global_stats: false 295 | } 296 | } 297 | layer { 298 | bottom: "conv2_2" 299 | top: "conv2_2" 300 | name: "scale_conv2_2" 301 | type: "Scale" 302 | param { 303 | name: "conv2_2_scale_w" 304 | lr_mult: 1 305 | decay_mult: 1 306 | } 307 | param { 308 | name: "conv2_2_scale_b" 309 | lr_mult: 2 310 | decay_mult: 0 311 | } 312 | scale_param { 313 | bias_term: true 314 | } 315 | } 316 | layer { 317 | bottom: "conv2_2" 318 | top: "conv2_2" 319 | name: "relu2_2" 320 | type: "ReLU" 321 | } 322 | layer { 323 | bottom: "conv2_2" 324 | top: "pool2" 325 | name: "pool2" 326 | type: "Pooling" 327 | pooling_param { 328 | pool: MAX 329 | kernel_size: 2 330 | stride: 2 331 | } 332 | } 333 | layer { 334 | bottom: "pool2" 335 | top: "conv3_1" 336 | name: "conv3_1" 337 | type: "Convolution" 338 | param { 339 | name: "conv3_1_w" 340 | lr_mult: 1 341 | decay_mult: 1 342 | } 343 | param { 344 | name: "conv3_1_b" 345 | lr_mult: 2 346 | decay_mult: 0 347 | } 348 | convolution_param { 349 | weight_filler { 350 | type: "gaussian" 351 | std: 0.01 352 | } 353 | bias_filler { 354 | type: "constant" 355 | value: 0 356 | } 357 | num_output: 256 358 | pad: 1 359 | kernel_size: 3 360 | } 361 | } 362 | layer { 363 | bottom: "conv3_1" 364 | top: "conv3_1" 365 | name: "bn_conv3_1" 366 | type: "BatchNorm" 367 | param { 368 | name: "conv3_1_bn_w" 369 | lr_mult: 1 370 | decay_mult: 1 371 | } 372 | param { 373 | name: "conv3_1_bn_b" 374 | lr_mult: 2 375 | decay_mult: 0 376 | } 377 | batch_norm_param { 378 | use_global_stats: false 379 | } 380 | } 381 | layer { 382 | bottom: "conv3_1" 383 | top: "conv3_1" 384 | name: "scale_conv3_1" 385 | type: "Scale" 386 | param { 387 | name: "conv3_1_scale_w" 388 | lr_mult: 1 389 | decay_mult: 1 390 | } 391 | param { 392 | name: "conv3_1_scale_b" 393 | lr_mult: 2 394 | decay_mult: 0 395 | } 396 | scale_param { 397 | bias_term: true 398 | } 399 | } 400 | layer { 401 | bottom: "conv3_1" 402 | top: "conv3_1" 403 | name: "relu3_1" 404 | type: "ReLU" 405 | } 406 | layer { 407 | bottom: "conv3_1" 408 | top: "conv3_2" 409 | name: "conv3_2" 410 | type: "Convolution" 411 | param { 412 | name: "conv3_2_w" 413 | lr_mult: 1 414 | decay_mult: 1 415 | } 416 | param { 417 | name: "conv3_2_b" 418 | lr_mult: 2 419 | decay_mult: 0 420 | } 421 | convolution_param { 422 | weight_filler { 423 | type: "gaussian" 424 | std: 0.01 425 | } 426 | bias_filler { 427 | type: "constant" 428 | value: 0 429 | } 430 | num_output: 256 431 | pad: 1 432 | kernel_size: 3 433 | } 434 | } 435 | layer { 436 | bottom: "conv3_2" 437 | top: "conv3_2" 438 | name: "bn_conv3_2" 439 | type: "BatchNorm" 440 | param { 441 | name: "conv3_2_bn_w" 442 | lr_mult: 1 443 | decay_mult: 1 444 | } 445 | param { 446 | name: "conv3_2_bn_b" 447 | lr_mult: 2 448 | decay_mult: 0 449 | } 450 | batch_norm_param { 451 | use_global_stats: false 452 | } 453 | } 454 | layer { 455 | bottom: "conv3_2" 456 | top: "conv3_2" 457 | name: "scale_conv3_2" 458 | type: "Scale" 459 | param { 460 | name: "conv3_2_scale_w" 461 | lr_mult: 1 462 | decay_mult: 1 463 | } 464 | param { 465 | name: "conv3_2_scale_b" 466 | lr_mult: 2 467 | decay_mult: 0 468 | } 469 | scale_param { 470 | bias_term: true 471 | } 472 | } 473 | layer { 474 | bottom: "conv3_2" 475 | top: "conv3_2" 476 | name: "relu3_2" 477 | type: "ReLU" 478 | } 479 | layer { 480 | bottom: "conv3_2" 481 | top: "conv3_3" 482 | name: "conv3_3" 483 | type: "Convolution" 484 | param { 485 | name: "conv3_3_w" 486 | lr_mult: 1 487 | decay_mult: 1 488 | } 489 | param { 490 | name: "conv3_3_b" 491 | lr_mult: 2 492 | decay_mult: 0 493 | } 494 | convolution_param { 495 | weight_filler { 496 | type: "gaussian" 497 | std: 0.01 498 | } 499 | bias_filler { 500 | type: "constant" 501 | value: 0 502 | } 503 | num_output: 256 504 | pad: 1 505 | kernel_size: 3 506 | } 507 | } 508 | layer { 509 | bottom: "conv3_3" 510 | top: "conv3_3" 511 | name: "bn_conv3_3" 512 | type: "BatchNorm" 513 | param { 514 | name: "conv3_3_bn_w" 515 | lr_mult: 1 516 | decay_mult: 1 517 | } 518 | param { 519 | name: "conv3_3_bn_b" 520 | lr_mult: 2 521 | decay_mult: 0 522 | } 523 | batch_norm_param { 524 | use_global_stats: false 525 | } 526 | } 527 | layer { 528 | bottom: "conv3_3" 529 | top: "conv3_3" 530 | name: "scale_conv3_3" 531 | type: "Scale" 532 | param { 533 | name: "conv3_3_scale_w" 534 | lr_mult: 1 535 | decay_mult: 1 536 | } 537 | param { 538 | name: "conv3_3_scale_b" 539 | lr_mult: 2 540 | decay_mult: 0 541 | } 542 | scale_param { 543 | bias_term: true 544 | } 545 | } 546 | layer { 547 | bottom: "conv3_3" 548 | top: "conv3_3" 549 | name: "relu3_3" 550 | type: "ReLU" 551 | } 552 | layer { 553 | bottom: "conv3_3" 554 | top: "pool3" 555 | name: "pool3" 556 | type: "Pooling" 557 | pooling_param { 558 | pool: MAX 559 | kernel_size: 2 560 | stride: 2 561 | } 562 | } 563 | layer { 564 | bottom: "pool3" 565 | top: "conv4_1" 566 | name: "conv4_1" 567 | type: "Convolution" 568 | param { 569 | name: "conv4_1_w" 570 | lr_mult: 1 571 | decay_mult: 1 572 | } 573 | param { 574 | name: "conv4_1_b" 575 | lr_mult: 2 576 | decay_mult: 0 577 | } 578 | convolution_param { 579 | weight_filler { 580 | type: "gaussian" 581 | std: 0.01 582 | } 583 | bias_filler { 584 | type: "constant" 585 | value: 0 586 | } 587 | num_output: 512 588 | pad: 1 589 | kernel_size: 3 590 | } 591 | } 592 | layer { 593 | bottom: "conv4_1" 594 | top: "conv4_1" 595 | name: "bn_conv4_1" 596 | type: "BatchNorm" 597 | param { 598 | name: "conv4_1_bn_w" 599 | lr_mult: 1 600 | decay_mult: 1 601 | } 602 | param { 603 | name: "conv4_1_bn_b" 604 | lr_mult: 2 605 | decay_mult: 0 606 | } 607 | batch_norm_param { 608 | use_global_stats: false 609 | } 610 | } 611 | layer { 612 | bottom: "conv4_1" 613 | top: "conv4_1" 614 | name: "scale_conv4_1" 615 | type: "Scale" 616 | param { 617 | name: "conv4_1_scale_w" 618 | lr_mult: 1 619 | decay_mult: 1 620 | } 621 | param { 622 | name: "conv4_1_scale_b" 623 | lr_mult: 2 624 | decay_mult: 0 625 | } 626 | scale_param { 627 | bias_term: true 628 | } 629 | } 630 | layer { 631 | bottom: "conv4_1" 632 | top: "conv4_1" 633 | name: "relu4_1" 634 | type: "ReLU" 635 | } 636 | layer { 637 | bottom: "conv4_1" 638 | top: "conv4_2" 639 | name: "conv4_2" 640 | type: "Convolution" 641 | param { 642 | name: "conv4_2_w" 643 | lr_mult: 1 644 | decay_mult: 1 645 | } 646 | param { 647 | name: "conv4_2_b" 648 | lr_mult: 2 649 | decay_mult: 0 650 | } 651 | convolution_param { 652 | weight_filler { 653 | type: "gaussian" 654 | std: 0.01 655 | } 656 | bias_filler { 657 | type: "constant" 658 | value: 0 659 | } 660 | num_output: 512 661 | pad: 1 662 | kernel_size: 3 663 | } 664 | } 665 | layer { 666 | bottom: "conv4_2" 667 | top: "conv4_2" 668 | name: "bn_conv4_2" 669 | type: "BatchNorm" 670 | param { 671 | name: "conv4_2_bn_w" 672 | lr_mult: 1 673 | decay_mult: 1 674 | } 675 | param { 676 | name: "conv4_2_bn_b" 677 | lr_mult: 2 678 | decay_mult: 0 679 | } 680 | batch_norm_param { 681 | use_global_stats: false 682 | } 683 | } 684 | layer { 685 | bottom: "conv4_2" 686 | top: "conv4_2" 687 | name: "scale_conv4_2" 688 | type: "Scale" 689 | param { 690 | name: "conv4_2_scale_w" 691 | lr_mult: 1 692 | decay_mult: 1 693 | } 694 | param { 695 | name: "conv4_2_scale_b" 696 | lr_mult: 2 697 | decay_mult: 0 698 | } 699 | scale_param { 700 | bias_term: true 701 | } 702 | } 703 | layer { 704 | bottom: "conv4_2" 705 | top: "conv4_2" 706 | name: "relu4_2" 707 | type: "ReLU" 708 | } 709 | layer { 710 | bottom: "conv4_2" 711 | top: "conv4_3" 712 | name: "conv4_3" 713 | type: "Convolution" 714 | param { 715 | name: "conv4_3_w" 716 | lr_mult: 1 717 | decay_mult: 1 718 | } 719 | param { 720 | name: "conv4_3_b" 721 | lr_mult: 2 722 | decay_mult: 0 723 | } 724 | convolution_param { 725 | weight_filler { 726 | type: "gaussian" 727 | std: 0.01 728 | } 729 | bias_filler { 730 | type: "constant" 731 | value: 0 732 | } 733 | num_output: 512 734 | pad: 1 735 | kernel_size: 3 736 | } 737 | } 738 | layer { 739 | bottom: "conv4_3" 740 | top: "conv4_3" 741 | name: "bn_conv4_3" 742 | type: "BatchNorm" 743 | param { 744 | name: "conv4_3_bn_w" 745 | lr_mult: 1 746 | decay_mult: 1 747 | } 748 | param { 749 | name: "conv4_3_bn_b" 750 | lr_mult: 2 751 | decay_mult: 0 752 | } 753 | batch_norm_param { 754 | use_global_stats: false 755 | } 756 | } 757 | layer { 758 | bottom: "conv4_3" 759 | top: "conv4_3" 760 | name: "scale_conv4_3" 761 | type: "Scale" 762 | param { 763 | name: "conv4_3_scale_w" 764 | lr_mult: 1 765 | decay_mult: 1 766 | } 767 | param { 768 | name: "conv4_3_scale_b" 769 | lr_mult: 2 770 | decay_mult: 0 771 | } 772 | scale_param { 773 | bias_term: true 774 | } 775 | } 776 | layer { 777 | bottom: "conv4_3" 778 | top: "conv4_3" 779 | name: "relu4_3" 780 | type: "ReLU" 781 | } 782 | layer { 783 | bottom: "conv4_3" 784 | top: "pool4" 785 | name: "pool4" 786 | type: "Pooling" 787 | pooling_param { 788 | pool: MAX 789 | kernel_size: 2 790 | stride: 2 791 | } 792 | } 793 | layer { 794 | bottom: "pool4" 795 | top: "conv5_1" 796 | name: "conv5_1" 797 | type: "Convolution" 798 | param { 799 | name: "conv5_1_w" 800 | lr_mult: 1 801 | decay_mult: 1 802 | } 803 | param { 804 | name: "conv5_1_b" 805 | lr_mult: 2 806 | decay_mult: 0 807 | } 808 | convolution_param { 809 | weight_filler { 810 | type: "gaussian" 811 | std: 0.01 812 | } 813 | bias_filler { 814 | type: "constant" 815 | value: 0 816 | } 817 | num_output: 512 818 | pad: 1 819 | kernel_size: 3 820 | } 821 | } 822 | layer { 823 | bottom: "conv5_1" 824 | top: "conv5_1" 825 | name: "bn_conv5_1" 826 | type: "BatchNorm" 827 | param { 828 | name: "conv5_1_bn_w" 829 | lr_mult: 1 830 | decay_mult: 1 831 | } 832 | param { 833 | name: "conv5_1_bn_b" 834 | lr_mult: 2 835 | decay_mult: 0 836 | } 837 | batch_norm_param { 838 | use_global_stats: false 839 | } 840 | } 841 | layer { 842 | bottom: "conv5_1" 843 | top: "conv5_1" 844 | name: "scale_conv5_1" 845 | type: "Scale" 846 | param { 847 | name: "conv5_1_scale_w" 848 | lr_mult: 1 849 | decay_mult: 1 850 | } 851 | param { 852 | name: "conv5_1_scale_b" 853 | lr_mult: 2 854 | decay_mult: 0 855 | } 856 | scale_param { 857 | bias_term: true 858 | } 859 | } 860 | layer { 861 | bottom: "conv5_1" 862 | top: "conv5_1" 863 | name: "relu5_1" 864 | type: "ReLU" 865 | } 866 | layer { 867 | bottom: "conv5_1" 868 | top: "conv5_2" 869 | name: "conv5_2" 870 | type: "Convolution" 871 | param { 872 | name: "conv5_2_w" 873 | lr_mult: 1 874 | decay_mult: 1 875 | } 876 | param { 877 | name: "conv5_2_b" 878 | lr_mult: 2 879 | decay_mult: 0 880 | } 881 | convolution_param { 882 | weight_filler { 883 | type: "gaussian" 884 | std: 0.01 885 | } 886 | bias_filler { 887 | type: "constant" 888 | value: 0 889 | } 890 | num_output: 512 891 | pad: 1 892 | kernel_size: 3 893 | } 894 | } 895 | layer { 896 | bottom: "conv5_2" 897 | top: "conv5_2" 898 | name: "bn_conv5_2" 899 | type: "BatchNorm" 900 | param { 901 | name: "conv5_2_bn_w" 902 | lr_mult: 1 903 | decay_mult: 1 904 | } 905 | param { 906 | name: "conv5_2_bn_b" 907 | lr_mult: 2 908 | decay_mult: 0 909 | } 910 | batch_norm_param { 911 | use_global_stats: false 912 | } 913 | } 914 | layer { 915 | bottom: "conv5_2" 916 | top: "conv5_2" 917 | name: "scale_conv5_2" 918 | type: "Scale" 919 | param { 920 | name: "conv5_2_scale_w" 921 | lr_mult: 1 922 | decay_mult: 1 923 | } 924 | param { 925 | name: "conv5_2_scale_b" 926 | lr_mult: 2 927 | decay_mult: 0 928 | } 929 | scale_param { 930 | bias_term: true 931 | } 932 | } 933 | layer { 934 | bottom: "conv5_2" 935 | top: "conv5_2" 936 | name: "relu5_2" 937 | type: "ReLU" 938 | } 939 | layer { 940 | bottom: "conv5_2" 941 | top: "conv5_3" 942 | name: "conv5_3" 943 | type: "Convolution" 944 | param { 945 | name: "conv5_3_w" 946 | lr_mult: 1 947 | decay_mult: 1 948 | } 949 | param { 950 | name: "conv5_3_b" 951 | lr_mult: 2 952 | decay_mult: 0 953 | } 954 | convolution_param { 955 | weight_filler { 956 | type: "gaussian" 957 | std: 0.01 958 | } 959 | bias_filler { 960 | type: "constant" 961 | value: 0 962 | } 963 | num_output: 512 964 | pad: 1 965 | kernel_size: 3 966 | } 967 | } 968 | layer { 969 | bottom: "conv5_3" 970 | top: "conv5_3" 971 | name: "bn_conv5_3" 972 | type: "BatchNorm" 973 | param { 974 | name: "conv5_3_bn_w" 975 | lr_mult: 1 976 | decay_mult: 1 977 | } 978 | param { 979 | name: "conv5_3_bn_b" 980 | lr_mult: 2 981 | decay_mult: 0 982 | } 983 | batch_norm_param { 984 | use_global_stats: false 985 | } 986 | } 987 | layer { 988 | bottom: "conv5_3" 989 | top: "conv5_3" 990 | name: "scale_conv5_3" 991 | type: "Scale" 992 | param { 993 | name: "conv5_3_scale_w" 994 | lr_mult: 1 995 | decay_mult: 1 996 | } 997 | param { 998 | name: "conv5_3_scale_b" 999 | lr_mult: 2 1000 | decay_mult: 0 1001 | } 1002 | scale_param { 1003 | bias_term: true 1004 | } 1005 | } 1006 | layer { 1007 | bottom: "conv5_3" 1008 | top: "conv5_3" 1009 | name: "relu5_3" 1010 | type: "ReLU" 1011 | } 1012 | 1013 | ###Data_Depth### 1014 | ########## Depth conv1_1 ########## 1015 | layer { 1016 | bottom: "data_Depth" 1017 | top: "conv11_1" 1018 | name: "conv11_1" 1019 | type: "Convolution" 1020 | param { 1021 | name: "conv1_1_w" 1022 | lr_mult: 1 1023 | decay_mult: 1 1024 | } 1025 | param { 1026 | name: "conv1_1_b" 1027 | lr_mult: 2 1028 | decay_mult: 0 1029 | } 1030 | convolution_param { 1031 | weight_filler { 1032 | type: "gaussian" 1033 | std: 0.01 1034 | } 1035 | bias_filler { 1036 | type: "constant" 1037 | value: 0 1038 | } 1039 | num_output: 64 1040 | pad: 1 1041 | kernel_size: 3 1042 | } 1043 | } 1044 | layer { 1045 | bottom: "conv11_1" 1046 | top: "conv11_1" 1047 | name: "bn_conv11_1" 1048 | type: "BatchNorm" 1049 | param { 1050 | name: "conv1_1_bn_w" 1051 | lr_mult: 1 1052 | decay_mult: 1 1053 | } 1054 | param { 1055 | name: "conv1_1_bn_b" 1056 | lr_mult: 2 1057 | decay_mult: 0 1058 | } 1059 | batch_norm_param { 1060 | use_global_stats: false 1061 | } 1062 | } 1063 | layer { 1064 | bottom: "conv11_1" 1065 | top: "conv11_1" 1066 | name: "scale_conv11_1" 1067 | type: "Scale" 1068 | param { 1069 | name: "conv1_1_scale_w" 1070 | lr_mult: 1 1071 | decay_mult: 1 1072 | } 1073 | param { 1074 | name: "conv1_1_scale_b" 1075 | lr_mult: 2 1076 | decay_mult: 0 1077 | } 1078 | scale_param { 1079 | bias_term: true 1080 | } 1081 | } 1082 | layer { 1083 | bottom: "conv11_1" 1084 | top: "conv11_1" 1085 | name: "relu11_1" 1086 | type: "ReLU" 1087 | } 1088 | layer { 1089 | bottom: "conv11_1" 1090 | top: "conv11_2" 1091 | name: "conv11_2" 1092 | type: "Convolution" 1093 | param { 1094 | name: "conv1_2_w" 1095 | lr_mult: 1 1096 | decay_mult: 1 1097 | } 1098 | param { 1099 | name: "conv1_2_b" 1100 | lr_mult: 2 1101 | decay_mult: 0 1102 | } 1103 | convolution_param { 1104 | weight_filler { 1105 | type: "gaussian" 1106 | std: 0.01 1107 | } 1108 | bias_filler { 1109 | type: "constant" 1110 | value: 0 1111 | } 1112 | num_output: 64 1113 | pad: 1 1114 | kernel_size: 3 1115 | } 1116 | } 1117 | layer { 1118 | bottom: "conv11_2" 1119 | top: "conv11_2" 1120 | name: "bn_conv11_2" 1121 | type: "BatchNorm" 1122 | param { 1123 | name: "conv1_2_bn_w" 1124 | lr_mult: 1 1125 | decay_mult: 1 1126 | } 1127 | param { 1128 | name: "conv1_2_bn_b" 1129 | lr_mult: 2 1130 | decay_mult: 0 1131 | } 1132 | batch_norm_param { 1133 | use_global_stats: false 1134 | } 1135 | } 1136 | layer { 1137 | bottom: "conv11_2" 1138 | top: "conv11_2" 1139 | name: "scale_conv11_2" 1140 | type: "Scale" 1141 | param { 1142 | name: "conv1_2_scale_w" 1143 | lr_mult: 1 1144 | decay_mult: 1 1145 | } 1146 | param { 1147 | name: "conv1_2_scale_b" 1148 | lr_mult: 2 1149 | decay_mult: 0 1150 | } 1151 | scale_param { 1152 | bias_term: true 1153 | } 1154 | } 1155 | layer { 1156 | bottom: "conv11_2" 1157 | top: "conv11_2" 1158 | name: "relu11_2" 1159 | type: "ReLU" 1160 | } 1161 | ########## 1162 | layer { 1163 | bottom: "conv11_2" 1164 | top: "conv11_3" 1165 | name: "conv11_3" 1166 | type: "Convolution" 1167 | param { 1168 | lr_mult: 1 1169 | decay_mult: 1 1170 | } 1171 | param { 1172 | lr_mult: 2 1173 | decay_mult: 0 1174 | } 1175 | convolution_param { 1176 | weight_filler { 1177 | type: "xavier" 1178 | } 1179 | bias_filler { 1180 | type: "constant" 1181 | value: 0 1182 | } 1183 | num_output: 64 1184 | pad: 1 1185 | kernel_size: 3 1186 | } 1187 | } 1188 | layer { 1189 | bottom: "conv11_3" 1190 | top: "conv11_3" 1191 | name: "bn_conv11_3" 1192 | type: "BatchNorm" 1193 | param { 1194 | lr_mult: 1 1195 | decay_mult: 1 1196 | } 1197 | param { 1198 | lr_mult: 2 1199 | decay_mult: 0 1200 | } 1201 | batch_norm_param { 1202 | use_global_stats: false 1203 | } 1204 | } 1205 | layer { 1206 | bottom: "conv11_3" 1207 | top: "conv11_3" 1208 | name: "scale_conv11_3" 1209 | type: "Scale" 1210 | param { 1211 | lr_mult: 1 1212 | decay_mult: 1 1213 | } 1214 | param { 1215 | lr_mult: 2 1216 | decay_mult: 0 1217 | } 1218 | scale_param { 1219 | bias_term: true 1220 | } 1221 | } 1222 | layer { 1223 | bottom: "conv11_3" 1224 | top: "conv11_3" 1225 | name: "relu11_3" 1226 | type: "Sigmoid" 1227 | } 1228 | 1229 | layer { 1230 | bottom: "conv11_2" 1231 | top: "pool11" 1232 | name: "pool11" 1233 | type: "Pooling" 1234 | pooling_param { 1235 | pool: MAX 1236 | kernel_size: 2 1237 | stride: 2 1238 | } 1239 | } 1240 | ########## Depth conv2_1 ########## 1241 | layer { 1242 | bottom: "pool11" 1243 | top: "conv12_1" 1244 | name: "conv12_1" 1245 | type: "Convolution" 1246 | param { 1247 | name: "conv2_1_w" 1248 | lr_mult: 1 1249 | decay_mult: 1 1250 | } 1251 | param { 1252 | name: "conv2_1_b" 1253 | lr_mult: 2 1254 | decay_mult: 0 1255 | } 1256 | convolution_param { 1257 | weight_filler { 1258 | type: "gaussian" 1259 | std: 0.01 1260 | } 1261 | bias_filler { 1262 | type: "constant" 1263 | value: 0 1264 | } 1265 | num_output: 128 1266 | pad: 1 1267 | kernel_size: 3 1268 | } 1269 | } 1270 | layer { 1271 | bottom: "conv12_1" 1272 | top: "conv12_1" 1273 | name: "bn_conv12_1" 1274 | type: "BatchNorm" 1275 | param { 1276 | name: "conv2_1_bn_w" 1277 | lr_mult: 1 1278 | decay_mult: 1 1279 | } 1280 | param { 1281 | name: "conv2_1_bn_b" 1282 | lr_mult: 2 1283 | decay_mult: 0 1284 | } 1285 | batch_norm_param { 1286 | use_global_stats: false 1287 | } 1288 | } 1289 | layer { 1290 | bottom: "conv12_1" 1291 | top: "conv12_1" 1292 | name: "scale_conv12_1" 1293 | type: "Scale" 1294 | param { 1295 | name: "conv2_1_scale_w" 1296 | lr_mult: 1 1297 | decay_mult: 1 1298 | } 1299 | param { 1300 | name: "conv2_1_scale_b" 1301 | lr_mult: 2 1302 | decay_mult: 0 1303 | } 1304 | scale_param { 1305 | bias_term: true 1306 | } 1307 | } 1308 | layer { 1309 | bottom: "conv12_1" 1310 | top: "conv12_1" 1311 | name: "relu12_1" 1312 | type: "ReLU" 1313 | } 1314 | layer { 1315 | bottom: "conv12_1" 1316 | top: "conv12_2" 1317 | name: "conv12_2" 1318 | type: "Convolution" 1319 | param { 1320 | name: "conv2_2_w" 1321 | lr_mult: 1 1322 | decay_mult: 1 1323 | } 1324 | param { 1325 | name: "conv2_2_b" 1326 | lr_mult: 2 1327 | decay_mult: 0 1328 | } 1329 | convolution_param { 1330 | weight_filler { 1331 | type: "gaussian" 1332 | std: 0.01 1333 | } 1334 | bias_filler { 1335 | type: "constant" 1336 | value: 0 1337 | } 1338 | num_output: 128 1339 | pad: 1 1340 | kernel_size: 3 1341 | } 1342 | } 1343 | layer { 1344 | bottom: "conv12_2" 1345 | top: "conv12_2" 1346 | name: "bn_conv12_2" 1347 | type: "BatchNorm" 1348 | param { 1349 | name: "conv2_2_bn_w" 1350 | lr_mult: 1 1351 | decay_mult: 1 1352 | } 1353 | param { 1354 | name: "conv2_2_bn_b" 1355 | lr_mult: 2 1356 | decay_mult: 0 1357 | } 1358 | batch_norm_param { 1359 | use_global_stats: false 1360 | } 1361 | } 1362 | layer { 1363 | bottom: "conv12_2" 1364 | top: "conv12_2" 1365 | name: "scale_conv12_2" 1366 | type: "Scale" 1367 | param { 1368 | name: "conv2_2_scale_w" 1369 | lr_mult: 1 1370 | decay_mult: 1 1371 | } 1372 | param { 1373 | name: "conv2_2_scale_b" 1374 | lr_mult: 2 1375 | decay_mult: 0 1376 | } 1377 | scale_param { 1378 | bias_term: true 1379 | } 1380 | } 1381 | layer { 1382 | bottom: "conv12_2" 1383 | top: "conv12_2" 1384 | name: "relu12_2" 1385 | type: "ReLU" 1386 | } 1387 | ########## 1388 | layer { 1389 | bottom: "conv12_2" 1390 | top: "conv12_3" 1391 | name: "conv12_3" 1392 | type: "Convolution" 1393 | param { 1394 | lr_mult: 1 1395 | decay_mult: 1 1396 | } 1397 | param { 1398 | lr_mult: 2 1399 | decay_mult: 0 1400 | } 1401 | convolution_param { 1402 | weight_filler { 1403 | type: "xavier" 1404 | } 1405 | bias_filler { 1406 | type: "constant" 1407 | value: 0 1408 | } 1409 | num_output: 128 1410 | pad: 1 1411 | kernel_size: 3 1412 | } 1413 | } 1414 | layer { 1415 | bottom: "conv12_3" 1416 | top: "conv12_3" 1417 | name: "bn_conv12_3" 1418 | type: "BatchNorm" 1419 | param { 1420 | lr_mult: 1 1421 | decay_mult: 1 1422 | } 1423 | param { 1424 | lr_mult: 2 1425 | decay_mult: 0 1426 | } 1427 | batch_norm_param { 1428 | use_global_stats: false 1429 | } 1430 | } 1431 | layer { 1432 | bottom: "conv12_3" 1433 | top: "conv12_3" 1434 | name: "scale_conv12_3" 1435 | type: "Scale" 1436 | param { 1437 | lr_mult: 1 1438 | decay_mult: 1 1439 | } 1440 | param { 1441 | lr_mult: 2 1442 | decay_mult: 0 1443 | } 1444 | scale_param { 1445 | bias_term: true 1446 | } 1447 | } 1448 | layer { 1449 | bottom: "conv12_3" 1450 | top: "conv12_3" 1451 | name: "relu12_3" 1452 | type: "Sigmoid" 1453 | } 1454 | 1455 | layer { 1456 | bottom: "conv12_2" 1457 | top: "pool12" 1458 | name: "pool12" 1459 | type: "Pooling" 1460 | pooling_param { 1461 | pool: MAX 1462 | kernel_size: 2 1463 | stride: 2 1464 | } 1465 | } 1466 | ########## Depth conv3_1 ########## 1467 | layer { 1468 | bottom: "pool12" 1469 | top: "conv13_1" 1470 | name: "conv13_1" 1471 | type: "Convolution" 1472 | param { 1473 | name: "conv3_1_w" 1474 | lr_mult: 1 1475 | decay_mult: 1 1476 | } 1477 | param { 1478 | name: "conv3_1_b" 1479 | lr_mult: 2 1480 | decay_mult: 0 1481 | } 1482 | convolution_param { 1483 | weight_filler { 1484 | type: "gaussian" 1485 | std: 0.01 1486 | } 1487 | bias_filler { 1488 | type: "constant" 1489 | value: 0 1490 | } 1491 | num_output: 256 1492 | pad: 1 1493 | kernel_size: 3 1494 | } 1495 | } 1496 | layer { 1497 | bottom: "conv13_1" 1498 | top: "conv13_1" 1499 | name: "bn_conv13_1" 1500 | type: "BatchNorm" 1501 | param { 1502 | name: "conv3_1_bn_w" 1503 | lr_mult: 1 1504 | decay_mult: 1 1505 | } 1506 | param { 1507 | name: "conv3_1_bn_b" 1508 | lr_mult: 2 1509 | decay_mult: 0 1510 | } 1511 | batch_norm_param { 1512 | use_global_stats: false 1513 | } 1514 | } 1515 | layer { 1516 | bottom: "conv13_1" 1517 | top: "conv13_1" 1518 | name: "scale_conv13_1" 1519 | type: "Scale" 1520 | param { 1521 | name: "conv3_1_scale_w" 1522 | lr_mult: 1 1523 | decay_mult: 1 1524 | } 1525 | param { 1526 | name: "conv3_1_scale_b" 1527 | lr_mult: 2 1528 | decay_mult: 0 1529 | } 1530 | scale_param { 1531 | bias_term: true 1532 | } 1533 | } 1534 | layer { 1535 | bottom: "conv13_1" 1536 | top: "conv13_1" 1537 | name: "relu13_1" 1538 | type: "ReLU" 1539 | } 1540 | layer { 1541 | bottom: "conv13_1" 1542 | top: "conv13_2" 1543 | name: "conv13_2" 1544 | type: "Convolution" 1545 | param { 1546 | name: "conv3_2_w" 1547 | lr_mult: 1 1548 | decay_mult: 1 1549 | } 1550 | param { 1551 | name: "conv3_2_b" 1552 | lr_mult: 2 1553 | decay_mult: 0 1554 | } 1555 | convolution_param { 1556 | weight_filler { 1557 | type: "gaussian" 1558 | std: 0.01 1559 | } 1560 | bias_filler { 1561 | type: "constant" 1562 | value: 0 1563 | } 1564 | num_output: 256 1565 | pad: 1 1566 | kernel_size: 3 1567 | } 1568 | } 1569 | layer { 1570 | bottom: "conv13_2" 1571 | top: "conv13_2" 1572 | name: "bn_conv13_2" 1573 | type: "BatchNorm" 1574 | param { 1575 | name: "conv3_2_bn_w" 1576 | lr_mult: 1 1577 | decay_mult: 1 1578 | } 1579 | param { 1580 | name: "conv3_2_bn_b" 1581 | lr_mult: 2 1582 | decay_mult: 0 1583 | } 1584 | batch_norm_param { 1585 | use_global_stats: false 1586 | } 1587 | } 1588 | layer { 1589 | bottom: "conv13_2" 1590 | top: "conv13_2" 1591 | name: "scale_conv13_2" 1592 | type: "Scale" 1593 | param { 1594 | name: "conv3_2_scale_w" 1595 | lr_mult: 1 1596 | decay_mult: 1 1597 | } 1598 | param { 1599 | name: "conv3_2_scale_b" 1600 | lr_mult: 2 1601 | decay_mult: 0 1602 | } 1603 | scale_param { 1604 | bias_term: true 1605 | } 1606 | } 1607 | layer { 1608 | bottom: "conv13_2" 1609 | top: "conv13_2" 1610 | name: "relu13_2" 1611 | type: "ReLU" 1612 | } 1613 | layer { 1614 | bottom: "conv13_2" 1615 | top: "conv13_3" 1616 | name: "conv13_3" 1617 | type: "Convolution" 1618 | param { 1619 | name: "conv3_3_w" 1620 | lr_mult: 1 1621 | decay_mult: 1 1622 | } 1623 | param { 1624 | name: "conv3_3_b" 1625 | lr_mult: 2 1626 | decay_mult: 0 1627 | } 1628 | convolution_param { 1629 | weight_filler { 1630 | type: "gaussian" 1631 | std: 0.01 1632 | } 1633 | bias_filler { 1634 | type: "constant" 1635 | value: 0 1636 | } 1637 | num_output: 256 1638 | pad: 1 1639 | kernel_size: 3 1640 | } 1641 | } 1642 | layer { 1643 | bottom: "conv13_3" 1644 | top: "conv13_3" 1645 | name: "bn_conv13_3" 1646 | type: "BatchNorm" 1647 | param { 1648 | name: "conv3_3_bn_w" 1649 | lr_mult: 1 1650 | decay_mult: 1 1651 | } 1652 | param { 1653 | name: "conv3_3_bn_b" 1654 | lr_mult: 2 1655 | decay_mult: 0 1656 | } 1657 | batch_norm_param { 1658 | use_global_stats: false 1659 | } 1660 | } 1661 | layer { 1662 | bottom: "conv13_3" 1663 | top: "conv13_3" 1664 | name: "scale_conv13_3" 1665 | type: "Scale" 1666 | param { 1667 | name: "conv3_3_scale_w" 1668 | lr_mult: 1 1669 | decay_mult: 1 1670 | } 1671 | param { 1672 | name: "conv3_3_scale_b" 1673 | lr_mult: 2 1674 | decay_mult: 0 1675 | } 1676 | scale_param { 1677 | bias_term: true 1678 | } 1679 | } 1680 | layer { 1681 | bottom: "conv13_3" 1682 | top: "conv13_3" 1683 | name: "relu13_3" 1684 | type: "ReLU" 1685 | } 1686 | ########## 1687 | layer { 1688 | bottom: "conv13_3" 1689 | top: "conv13_4" 1690 | name: "conv13_4" 1691 | type: "Convolution" 1692 | param { 1693 | lr_mult: 1 1694 | decay_mult: 1 1695 | } 1696 | param { 1697 | lr_mult: 2 1698 | decay_mult: 0 1699 | } 1700 | convolution_param { 1701 | weight_filler { 1702 | type: "xavier" 1703 | } 1704 | bias_filler { 1705 | type: "constant" 1706 | value: 0 1707 | } 1708 | num_output: 256 1709 | pad: 1 1710 | kernel_size: 3 1711 | } 1712 | } 1713 | layer { 1714 | bottom: "conv13_4" 1715 | top: "conv13_4" 1716 | name: "bn_conv13_4" 1717 | type: "BatchNorm" 1718 | param { 1719 | lr_mult: 1 1720 | decay_mult: 1 1721 | } 1722 | param { 1723 | lr_mult: 2 1724 | decay_mult: 0 1725 | } 1726 | batch_norm_param { 1727 | use_global_stats: false 1728 | } 1729 | } 1730 | layer { 1731 | bottom: "conv13_4" 1732 | top: "conv13_4" 1733 | name: "scale_conv13_4" 1734 | type: "Scale" 1735 | param { 1736 | lr_mult: 1 1737 | decay_mult: 1 1738 | } 1739 | param { 1740 | lr_mult: 2 1741 | decay_mult: 0 1742 | } 1743 | scale_param { 1744 | bias_term: true 1745 | } 1746 | } 1747 | layer { 1748 | bottom: "conv13_4" 1749 | top: "conv13_4" 1750 | name: "relu13_4" 1751 | type: "Sigmoid" 1752 | } 1753 | 1754 | layer { 1755 | bottom: "conv13_3" 1756 | top: "pool13" 1757 | name: "pool13" 1758 | type: "Pooling" 1759 | pooling_param { 1760 | pool: MAX 1761 | kernel_size: 2 1762 | stride: 2 1763 | } 1764 | } 1765 | ########## Depth conv4_1 ########## 1766 | layer { 1767 | bottom: "pool13" 1768 | top: "conv14_1" 1769 | name: "conv14_1" 1770 | type: "Convolution" 1771 | param { 1772 | name: "conv4_1_w" 1773 | lr_mult: 1 1774 | decay_mult: 1 1775 | } 1776 | param { 1777 | name: "conv4_1_b" 1778 | lr_mult: 2 1779 | decay_mult: 0 1780 | } 1781 | convolution_param { 1782 | weight_filler { 1783 | type: "gaussian" 1784 | std: 0.01 1785 | } 1786 | bias_filler { 1787 | type: "constant" 1788 | value: 0 1789 | } 1790 | num_output: 512 1791 | pad: 1 1792 | kernel_size: 3 1793 | } 1794 | } 1795 | layer { 1796 | bottom: "conv14_1" 1797 | top: "conv14_1" 1798 | name: "bn_conv14_1" 1799 | type: "BatchNorm" 1800 | param { 1801 | name: "conv4_1_bn_w" 1802 | lr_mult: 1 1803 | decay_mult: 1 1804 | } 1805 | param { 1806 | name: "conv4_1_bn_b" 1807 | lr_mult: 2 1808 | decay_mult: 0 1809 | } 1810 | batch_norm_param { 1811 | use_global_stats: false 1812 | } 1813 | } 1814 | layer { 1815 | bottom: "conv14_1" 1816 | top: "conv14_1" 1817 | name: "scale_conv14_1" 1818 | type: "Scale" 1819 | param { 1820 | name: "conv4_1_scale_w" 1821 | lr_mult: 1 1822 | decay_mult: 1 1823 | } 1824 | param { 1825 | name: "conv4_1_scale_b" 1826 | lr_mult: 2 1827 | decay_mult: 0 1828 | } 1829 | scale_param { 1830 | bias_term: true 1831 | } 1832 | } 1833 | layer { 1834 | bottom: "conv14_1" 1835 | top: "conv14_1" 1836 | name: "relu14_1" 1837 | type: "ReLU" 1838 | } 1839 | layer { 1840 | bottom: "conv14_1" 1841 | top: "conv14_2" 1842 | name: "conv14_2" 1843 | type: "Convolution" 1844 | param { 1845 | name: "conv4_2_w" 1846 | lr_mult: 1 1847 | decay_mult: 1 1848 | } 1849 | param { 1850 | name: "conv4_2_b" 1851 | lr_mult: 2 1852 | decay_mult: 0 1853 | } 1854 | convolution_param { 1855 | weight_filler { 1856 | type: "gaussian" 1857 | std: 0.01 1858 | } 1859 | bias_filler { 1860 | type: "constant" 1861 | value: 0 1862 | } 1863 | num_output: 512 1864 | pad: 1 1865 | kernel_size: 3 1866 | } 1867 | } 1868 | layer { 1869 | bottom: "conv14_2" 1870 | top: "conv14_2" 1871 | name: "bn_conv14_2" 1872 | type: "BatchNorm" 1873 | param { 1874 | name: "conv4_2_bn_w" 1875 | lr_mult: 1 1876 | decay_mult: 1 1877 | } 1878 | param { 1879 | name: "conv4_2_bn_b" 1880 | lr_mult: 2 1881 | decay_mult: 0 1882 | } 1883 | batch_norm_param { 1884 | use_global_stats: false 1885 | } 1886 | } 1887 | layer { 1888 | bottom: "conv14_2" 1889 | top: "conv14_2" 1890 | name: "scale_conv14_2" 1891 | type: "Scale" 1892 | param { 1893 | name: "conv4_2_scale_w" 1894 | lr_mult: 1 1895 | decay_mult: 1 1896 | } 1897 | param { 1898 | name: "conv4_2_scale_b" 1899 | lr_mult: 2 1900 | decay_mult: 0 1901 | } 1902 | scale_param { 1903 | bias_term: true 1904 | } 1905 | } 1906 | layer { 1907 | bottom: "conv14_2" 1908 | top: "conv14_2" 1909 | name: "relu14_2" 1910 | type: "ReLU" 1911 | } 1912 | layer { 1913 | bottom: "conv14_2" 1914 | top: "conv14_3" 1915 | name: "conv14_3" 1916 | type: "Convolution" 1917 | param { 1918 | name: "conv4_3_w" 1919 | lr_mult: 1 1920 | decay_mult: 1 1921 | } 1922 | param { 1923 | name: "conv4_3_b" 1924 | lr_mult: 2 1925 | decay_mult: 0 1926 | } 1927 | convolution_param { 1928 | weight_filler { 1929 | type: "gaussian" 1930 | std: 0.01 1931 | } 1932 | bias_filler { 1933 | type: "constant" 1934 | value: 0 1935 | } 1936 | num_output: 512 1937 | pad: 1 1938 | kernel_size: 3 1939 | } 1940 | } 1941 | layer { 1942 | bottom: "conv14_3" 1943 | top: "conv14_3" 1944 | name: "bn_conv14_3" 1945 | type: "BatchNorm" 1946 | param { 1947 | name: "conv4_3_bn_w" 1948 | lr_mult: 1 1949 | decay_mult: 1 1950 | } 1951 | param { 1952 | name: "conv4_3_bn_b" 1953 | lr_mult: 2 1954 | decay_mult: 0 1955 | } 1956 | batch_norm_param { 1957 | use_global_stats: false 1958 | } 1959 | } 1960 | layer { 1961 | bottom: "conv14_3" 1962 | top: "conv14_3" 1963 | name: "scale_conv14_3" 1964 | type: "Scale" 1965 | param { 1966 | name: "conv4_3_scale_w" 1967 | lr_mult: 1 1968 | decay_mult: 1 1969 | } 1970 | param { 1971 | name: "conv4_3_scale_b" 1972 | lr_mult: 2 1973 | decay_mult: 0 1974 | } 1975 | scale_param { 1976 | bias_term: true 1977 | } 1978 | } 1979 | layer { 1980 | bottom: "conv14_3" 1981 | top: "conv14_3" 1982 | name: "relu14_3" 1983 | type: "ReLU" 1984 | } 1985 | 1986 | ########## 1987 | layer { 1988 | bottom: "conv14_3" 1989 | top: "conv14_4" 1990 | name: "conv14_4" 1991 | type: "Convolution" 1992 | param { 1993 | lr_mult: 1 1994 | decay_mult: 1 1995 | } 1996 | param { 1997 | lr_mult: 2 1998 | decay_mult: 0 1999 | } 2000 | convolution_param { 2001 | weight_filler { 2002 | type: "xavier" 2003 | } 2004 | bias_filler { 2005 | type: "constant" 2006 | value: 0 2007 | } 2008 | num_output: 512 2009 | pad: 1 2010 | kernel_size: 3 2011 | } 2012 | } 2013 | layer { 2014 | bottom: "conv14_4" 2015 | top: "conv14_4" 2016 | name: "bn_conv14_4" 2017 | type: "BatchNorm" 2018 | param { 2019 | lr_mult: 1 2020 | decay_mult: 1 2021 | } 2022 | param { 2023 | lr_mult: 2 2024 | decay_mult: 0 2025 | } 2026 | batch_norm_param { 2027 | use_global_stats: false 2028 | } 2029 | } 2030 | layer { 2031 | bottom: "conv14_4" 2032 | top: "conv14_4" 2033 | name: "scale_conv14_4" 2034 | type: "Scale" 2035 | param { 2036 | lr_mult: 1 2037 | decay_mult: 1 2038 | } 2039 | param { 2040 | lr_mult: 2 2041 | decay_mult: 0 2042 | } 2043 | scale_param { 2044 | bias_term: true 2045 | } 2046 | } 2047 | layer { 2048 | bottom: "conv14_4" 2049 | top: "conv14_4" 2050 | name: "relu14_4" 2051 | type: "Sigmoid" 2052 | } 2053 | 2054 | layer { 2055 | bottom: "conv14_3" 2056 | top: "pool14" 2057 | name: "pool14" 2058 | type: "Pooling" 2059 | pooling_param { 2060 | pool: MAX 2061 | kernel_size: 2 2062 | stride: 2 2063 | } 2064 | } 2065 | ########## Depth conv5_1 ########## 2066 | layer { 2067 | bottom: "pool14" 2068 | top: "conv15_1" 2069 | name: "conv15_1" 2070 | type: "Convolution" 2071 | param { 2072 | name: "conv5_1_w" 2073 | lr_mult: 1 2074 | decay_mult: 1 2075 | } 2076 | param { 2077 | name: "conv5_1_b" 2078 | lr_mult: 2 2079 | decay_mult: 0 2080 | } 2081 | convolution_param { 2082 | weight_filler { 2083 | type: "gaussian" 2084 | std: 0.01 2085 | } 2086 | bias_filler { 2087 | type: "constant" 2088 | value: 0 2089 | } 2090 | num_output: 512 2091 | pad: 1 2092 | kernel_size: 3 2093 | } 2094 | } 2095 | layer { 2096 | bottom: "conv15_1" 2097 | top: "conv15_1" 2098 | name: "bn_conv15_1" 2099 | type: "BatchNorm" 2100 | param { 2101 | name: "conv5_1_bn_w" 2102 | lr_mult: 1 2103 | decay_mult: 1 2104 | } 2105 | param { 2106 | name: "conv5_1_bn_b" 2107 | lr_mult: 2 2108 | decay_mult: 0 2109 | } 2110 | batch_norm_param { 2111 | use_global_stats: false 2112 | } 2113 | } 2114 | layer { 2115 | bottom: "conv15_1" 2116 | top: "conv15_1" 2117 | name: "scale_conv15_1" 2118 | type: "Scale" 2119 | param { 2120 | name: "conv5_1_scale_w" 2121 | lr_mult: 1 2122 | decay_mult: 1 2123 | } 2124 | param { 2125 | name: "conv5_1_scale_b" 2126 | lr_mult: 2 2127 | decay_mult: 0 2128 | } 2129 | scale_param { 2130 | bias_term: true 2131 | } 2132 | } 2133 | layer { 2134 | bottom: "conv15_1" 2135 | top: "conv15_1" 2136 | name: "relu15_1" 2137 | type: "ReLU" 2138 | } 2139 | layer { 2140 | bottom: "conv15_1" 2141 | top: "conv15_2" 2142 | name: "conv15_2" 2143 | type: "Convolution" 2144 | param { 2145 | name: "conv5_2_w" 2146 | lr_mult: 1 2147 | decay_mult: 1 2148 | } 2149 | param { 2150 | name: "conv5_2_b" 2151 | lr_mult: 2 2152 | decay_mult: 0 2153 | } 2154 | convolution_param { 2155 | weight_filler { 2156 | type: "gaussian" 2157 | std: 0.01 2158 | } 2159 | bias_filler { 2160 | type: "constant" 2161 | value: 0 2162 | } 2163 | num_output: 512 2164 | pad: 1 2165 | kernel_size: 3 2166 | } 2167 | } 2168 | layer { 2169 | bottom: "conv15_2" 2170 | top: "conv15_2" 2171 | name: "bn_conv15_2" 2172 | type: "BatchNorm" 2173 | param { 2174 | name: "conv5_2_bn_w" 2175 | lr_mult: 1 2176 | decay_mult: 1 2177 | } 2178 | param { 2179 | name: "conv5_2_bn_b" 2180 | lr_mult: 2 2181 | decay_mult: 0 2182 | } 2183 | batch_norm_param { 2184 | use_global_stats: false 2185 | } 2186 | } 2187 | layer { 2188 | bottom: "conv15_2" 2189 | top: "conv15_2" 2190 | name: "scale_conv15_2" 2191 | type: "Scale" 2192 | param { 2193 | name: "conv5_2_scale_w" 2194 | lr_mult: 1 2195 | decay_mult: 1 2196 | } 2197 | param { 2198 | name: "conv5_2_scale_b" 2199 | lr_mult: 2 2200 | decay_mult: 0 2201 | } 2202 | scale_param { 2203 | bias_term: true 2204 | } 2205 | } 2206 | layer { 2207 | bottom: "conv15_2" 2208 | top: "conv15_2" 2209 | name: "relu15_2" 2210 | type: "ReLU" 2211 | } 2212 | layer { 2213 | bottom: "conv15_2" 2214 | top: "conv15_3" 2215 | name: "conv15_3" 2216 | type: "Convolution" 2217 | param { 2218 | name: "conv5_3_w" 2219 | lr_mult: 1 2220 | decay_mult: 1 2221 | } 2222 | param { 2223 | name: "conv5_3_b" 2224 | lr_mult: 2 2225 | decay_mult: 0 2226 | } 2227 | convolution_param { 2228 | weight_filler { 2229 | type: "gaussian" 2230 | std: 0.01 2231 | } 2232 | bias_filler { 2233 | type: "constant" 2234 | value: 0 2235 | } 2236 | num_output: 512 2237 | pad: 1 2238 | kernel_size: 3 2239 | } 2240 | } 2241 | layer { 2242 | bottom: "conv15_3" 2243 | top: "conv15_3" 2244 | name: "bn_conv15_3" 2245 | type: "BatchNorm" 2246 | param { 2247 | name: "conv5_3_bn_w" 2248 | lr_mult: 1 2249 | decay_mult: 1 2250 | } 2251 | param { 2252 | name: "conv5_3_bn_b" 2253 | lr_mult: 2 2254 | decay_mult: 0 2255 | } 2256 | batch_norm_param { 2257 | use_global_stats: false 2258 | } 2259 | } 2260 | layer { 2261 | bottom: "conv15_3" 2262 | top: "conv15_3" 2263 | name: "scale_conv15_3" 2264 | type: "Scale" 2265 | param { 2266 | name: "conv5_3_scale_w" 2267 | lr_mult: 1 2268 | decay_mult: 1 2269 | } 2270 | param { 2271 | name: "conv5_3_scale_b" 2272 | lr_mult: 2 2273 | decay_mult: 0 2274 | } 2275 | scale_param { 2276 | bias_term: true 2277 | } 2278 | } 2279 | layer { 2280 | bottom: "conv15_3" 2281 | top: "conv15_3" 2282 | name: "relu15_3" 2283 | type: "ReLU" 2284 | } 2285 | ########## 2286 | layer { 2287 | bottom: "conv15_3" 2288 | top: "conv15_4" 2289 | name: "conv15_4" 2290 | type: "Convolution" 2291 | param { 2292 | lr_mult: 1 2293 | decay_mult: 1 2294 | } 2295 | param { 2296 | lr_mult: 2 2297 | decay_mult: 0 2298 | } 2299 | convolution_param { 2300 | weight_filler { 2301 | type: "xavier" 2302 | } 2303 | bias_filler { 2304 | type: "constant" 2305 | value: 0 2306 | } 2307 | num_output: 512 2308 | pad: 1 2309 | kernel_size: 3 2310 | } 2311 | } 2312 | layer { 2313 | bottom: "conv15_4" 2314 | top: "conv15_4" 2315 | name: "bn_conv15_4" 2316 | type: "BatchNorm" 2317 | param { 2318 | lr_mult: 1 2319 | decay_mult: 1 2320 | } 2321 | param { 2322 | lr_mult: 2 2323 | decay_mult: 0 2324 | } 2325 | batch_norm_param { 2326 | use_global_stats: false 2327 | } 2328 | } 2329 | layer { 2330 | bottom: "conv15_4" 2331 | top: "conv15_4" 2332 | name: "scale_conv15_4" 2333 | type: "Scale" 2334 | param { 2335 | lr_mult: 1 2336 | decay_mult: 1 2337 | } 2338 | param { 2339 | lr_mult: 2 2340 | decay_mult: 0 2341 | } 2342 | scale_param { 2343 | bias_term: true 2344 | } 2345 | } 2346 | layer { 2347 | bottom: "conv15_4" 2348 | top: "conv15_4" 2349 | name: "relu15_4" 2350 | type: "Sigmoid" 2351 | } 2352 | 2353 | ######correlation layer提取协同特征###### 2354 | layer { 2355 | name: "correlation_01" 2356 | type: "Correlation" 2357 | bottom: "conv5_3" 2358 | bottom: "conv15_3" 2359 | top: "correlation_01" 2360 | correlation_param { 2361 | pad: 15 2362 | kernel_size: 1 2363 | max_displacement: 15 2364 | stride_1: 1 2365 | stride_2: 1 2366 | } 2367 | } 2368 | layer { 2369 | name: "relu_correlation_01" 2370 | type: "ReLU" 2371 | bottom: "correlation_01" 2372 | top: "correlation_01" 2373 | relu_param { 2374 | negative_slope: 0.1 2375 | } 2376 | } 2377 | 2378 | layer { 2379 | name: "correlation_10" 2380 | type: "Correlation" 2381 | bottom: "conv15_3" 2382 | bottom: "conv5_3" 2383 | top: "correlation_10" 2384 | correlation_param { 2385 | pad: 15 2386 | kernel_size: 1 2387 | max_displacement: 15 2388 | stride_1: 1 2389 | stride_2: 1 2390 | } 2391 | } 2392 | layer { 2393 | name: "relu_correlation_10" 2394 | type: "ReLU" 2395 | bottom: "correlation_10" 2396 | top: "correlation_10" 2397 | relu_param { 2398 | negative_slope: 0.1 2399 | } 2400 | } 2401 | 2402 | layer { 2403 | name: "concat_correlation_conv5_3" 2404 | type: "Concat" 2405 | bottom: "correlation_01" 2406 | bottom: "correlation_10" 2407 | top: "concat_correlation_conv5_3" 2408 | } 2409 | layer { 2410 | bottom: "concat_correlation_conv5_3" 2411 | top: "correlation_conv5_3" 2412 | name: "correlation_conv5_3" 2413 | type: "Convolution" 2414 | param { 2415 | lr_mult: 10 2416 | decay_mult: 1 2417 | } 2418 | param { 2419 | lr_mult: 20 2420 | decay_mult: 0 2421 | } 2422 | convolution_param { 2423 | weight_filler { 2424 | type: "xavier" 2425 | } 2426 | bias_filler { 2427 | type: "constant" 2428 | value: 0 2429 | } 2430 | num_output: 512 2431 | pad: 1 2432 | kernel_size: 3 2433 | } 2434 | } 2435 | layer { 2436 | bottom: "correlation_conv5_3" 2437 | top: "correlation_conv5_3" 2438 | name: "bn_correlation_conv5_3" 2439 | type: "BatchNorm" 2440 | batch_norm_param { 2441 | use_global_stats: false 2442 | } 2443 | } 2444 | layer { 2445 | bottom: "correlation_conv5_3" 2446 | top: "correlation_conv5_3" 2447 | name: "scale_correlation_conv5_3" 2448 | type: "Scale" 2449 | scale_param { 2450 | bias_term: true 2451 | } 2452 | } 2453 | layer { 2454 | bottom: "correlation_conv5_3" 2455 | top: "correlation_conv5_3" 2456 | name: "relu_correlation_conv5_3" 2457 | type: "ReLU" 2458 | } 2459 | ######concat layer提取协同信息###### 2460 | layer { 2461 | name: "concat_concat5_3" 2462 | type: "Concat" 2463 | bottom: "conv15_3" 2464 | bottom: "conv5_3" 2465 | top: "concat_concat5_3" 2466 | } 2467 | layer { 2468 | bottom: "concat_concat5_3" 2469 | top: "concat5_3_pre" 2470 | name: "concat5_3_pre" 2471 | type: "Convolution" 2472 | param { 2473 | lr_mult: 10 2474 | decay_mult: 1 2475 | } 2476 | param { 2477 | lr_mult: 20 2478 | decay_mult: 0 2479 | } 2480 | convolution_param { 2481 | weight_filler { 2482 | type: "xavier" 2483 | } 2484 | bias_filler { 2485 | type: "constant" 2486 | value: 0 2487 | } 2488 | num_output: 512 2489 | pad: 1 2490 | kernel_size: 3 2491 | } 2492 | } 2493 | layer { 2494 | bottom: "concat5_3_pre" 2495 | top: "concat5_3_pre" 2496 | name: "bn_concat5_3_pre" 2497 | type: "BatchNorm" 2498 | batch_norm_param { 2499 | use_global_stats: false 2500 | } 2501 | } 2502 | layer { 2503 | bottom: "concat5_3_pre" 2504 | top: "concat5_3_pre" 2505 | name: "scale_concat5_3_pre" 2506 | type: "Scale" 2507 | scale_param { 2508 | bias_term: true 2509 | } 2510 | } 2511 | layer { 2512 | bottom: "concat5_3_pre" 2513 | top: "concat5_3_pre" 2514 | name: "relu_concat5_3_pre" 2515 | type: "ReLU" 2516 | } 2517 | 2518 | ######correlation+concat###### 2519 | layer { 2520 | name: "concat_correlation_concat" 2521 | type: "Concat" 2522 | bottom: "correlation_conv5_3" 2523 | bottom: "concat5_3_pre" 2524 | # bottom: "conv5_3" 2525 | # bottom: "conv15_3" 2526 | top: "concat_correlation_concat" 2527 | } 2528 | 2529 | ############################################### 2530 | #################### Decoder ################## 2531 | ##################### conv5 ################### 2532 | layer { 2533 | name: "eltwise_dep_weighted_RGB_conv5" 2534 | type: "Eltwise" 2535 | bottom: "conv5_3" 2536 | bottom: "conv15_4" 2537 | top: "eltwise_dep_weighted_RGB_conv5" 2538 | eltwise_param { 2539 | operation: PROD 2540 | } 2541 | } 2542 | layer { 2543 | name: "concat_weighted_RGB_conv5" 2544 | type: "Eltwise" 2545 | bottom: "conv5_3" 2546 | bottom: "eltwise_dep_weighted_RGB_conv5" 2547 | top: "concat_weighted_RGB_conv5" 2548 | eltwise_param { 2549 | operation: SUM 2550 | } 2551 | } 2552 | 2553 | layer { 2554 | name: "concat_conv5_correlation_concat" 2555 | type: "Concat" 2556 | bottom: "concat_weighted_RGB_conv5" 2557 | bottom: "conv15_3" 2558 | bottom: "concat_correlation_concat" 2559 | top: "concat_conv5_correlation_concat" 2560 | } 2561 | layer { 2562 | bottom: "concat_conv5_correlation_concat" 2563 | top: "conv5_3_D" 2564 | name: "conv5_3_D" 2565 | type: "Convolution" 2566 | param { 2567 | lr_mult: 10 2568 | decay_mult: 1 2569 | } 2570 | param { 2571 | lr_mult: 20 2572 | decay_mult: 0 2573 | } 2574 | convolution_param { 2575 | weight_filler { 2576 | type: "xavier" 2577 | } 2578 | bias_filler { 2579 | type: "constant" 2580 | value: 0 2581 | } 2582 | num_output: 512 2583 | pad: 1 2584 | kernel_size: 3 2585 | } 2586 | } 2587 | layer { 2588 | bottom: "conv5_3_D" 2589 | top: "conv5_3_D" 2590 | name: "conv5_3_D_bn" 2591 | type: "BatchNorm" 2592 | param { 2593 | lr_mult: 1 2594 | decay_mult: 1 2595 | } 2596 | param { 2597 | lr_mult: 2 2598 | decay_mult: 0 2599 | } 2600 | batch_norm_param { 2601 | use_global_stats: false 2602 | } 2603 | } 2604 | layer { 2605 | bottom: "conv5_3_D" 2606 | top: "conv5_3_D" 2607 | name: "scale_conv5_3_D" 2608 | type: "Scale" 2609 | param { 2610 | lr_mult: 1 2611 | decay_mult: 1 2612 | } 2613 | param { 2614 | lr_mult: 2 2615 | decay_mult: 0 2616 | } 2617 | scale_param { 2618 | bias_term: true 2619 | } 2620 | } 2621 | layer { 2622 | bottom: "conv5_3_D" 2623 | top: "conv5_3_D" 2624 | name: "relu5_3_D" 2625 | type: "ReLU" 2626 | } 2627 | 2628 | layer { 2629 | bottom: "conv5_3_D" 2630 | top: "conv5_2_D" 2631 | name: "conv5_2_D" 2632 | type: "Convolution" 2633 | param { 2634 | lr_mult: 10 2635 | decay_mult: 1 2636 | } 2637 | param { 2638 | lr_mult: 20 2639 | decay_mult: 0 2640 | } 2641 | convolution_param { 2642 | weight_filler { 2643 | type: "xavier" 2644 | } 2645 | bias_filler { 2646 | type: "constant" 2647 | value: 0 2648 | } 2649 | num_output: 512 2650 | pad: 1 2651 | kernel_size: 3 2652 | } 2653 | } 2654 | layer { 2655 | bottom: "conv5_2_D" 2656 | top: "conv5_2_D" 2657 | name: "conv5_2_D_bn" 2658 | type: "BatchNorm" 2659 | param { 2660 | lr_mult: 1 2661 | decay_mult: 1 2662 | } 2663 | param { 2664 | lr_mult: 2 2665 | decay_mult: 0 2666 | } 2667 | batch_norm_param { 2668 | use_global_stats: false 2669 | } 2670 | } 2671 | layer { 2672 | bottom: "conv5_2_D" 2673 | top: "conv5_2_D" 2674 | name: "scale_conv5_2_D" 2675 | type: "Scale" 2676 | param { 2677 | lr_mult: 1 2678 | decay_mult: 1 2679 | } 2680 | param { 2681 | lr_mult: 2 2682 | decay_mult: 0 2683 | } 2684 | scale_param { 2685 | bias_term: true 2686 | } 2687 | } 2688 | layer { 2689 | bottom: "conv5_2_D" 2690 | top: "conv5_2_D" 2691 | name: "relu5_2_D" 2692 | type: "ReLU" 2693 | } 2694 | layer { 2695 | bottom: "conv5_2_D" 2696 | top: "conv5_1_D" 2697 | name: "conv5_1_D" 2698 | type: "Convolution" 2699 | param { 2700 | lr_mult: 10 2701 | decay_mult: 1 2702 | } 2703 | param { 2704 | lr_mult: 20 2705 | decay_mult: 0 2706 | } 2707 | convolution_param { 2708 | weight_filler { 2709 | type: "xavier" 2710 | } 2711 | bias_filler { 2712 | type: "constant" 2713 | value: 0 2714 | } 2715 | num_output: 512 2716 | pad: 1 2717 | kernel_size: 3 2718 | } 2719 | } 2720 | layer { 2721 | bottom: "conv5_1_D" 2722 | top: "conv5_1_D" 2723 | name: "conv5_1_D_bn" 2724 | type: "BatchNorm" 2725 | param { 2726 | lr_mult: 1 2727 | decay_mult: 1 2728 | } 2729 | param { 2730 | lr_mult: 2 2731 | decay_mult: 0 2732 | } 2733 | batch_norm_param { 2734 | use_global_stats: false 2735 | } 2736 | } 2737 | layer { 2738 | bottom: "conv5_1_D" 2739 | top: "conv5_1_D" 2740 | name: "scale_conv5_1_D" 2741 | type: "Scale" 2742 | param { 2743 | lr_mult: 1 2744 | decay_mult: 1 2745 | } 2746 | param { 2747 | lr_mult: 2 2748 | decay_mult: 0 2749 | } 2750 | scale_param { 2751 | bias_term: true 2752 | } 2753 | } 2754 | layer { 2755 | bottom: "conv5_1_D" 2756 | top: "conv5_1_D" 2757 | name: "relu5_1_D" 2758 | type: "ReLU" 2759 | } 2760 | layer { 2761 | name: "conv5_drop" 2762 | type: "Dropout" 2763 | bottom: "conv5_1_D" 2764 | top: "conv5_1_D" 2765 | dropout_param { 2766 | dropout_ratio: 0.5 2767 | } 2768 | } 2769 | layer { 2770 | bottom: "conv5_1_D" 2771 | top: "Up_conv5_1_D" 2772 | name: "Up_conv5_1_D" 2773 | type: "Deconvolution" 2774 | param { 2775 | lr_mult: 10 2776 | decay_mult: 1 2777 | } 2778 | param { 2779 | lr_mult: 20 2780 | decay_mult: 0 2781 | } 2782 | convolution_param { 2783 | weight_filler { 2784 | type: "xavier" 2785 | } 2786 | bias_filler { 2787 | type: "constant" 2788 | value: 0 2789 | } 2790 | num_output: 512 2791 | kernel_size: 2 2792 | stride: 2 2793 | } 2794 | } 2795 | 2796 | ##################### conv4 ################### 2797 | layer { 2798 | name: "eltwise_dep_weighted_RGB_conv4" 2799 | type: "Eltwise" 2800 | bottom: "conv4_3" 2801 | bottom: "conv14_4" 2802 | top: "eltwise_dep_weighted_RGB_conv4" 2803 | eltwise_param { 2804 | operation: PROD 2805 | } 2806 | } 2807 | layer { 2808 | name: "concat_weighted_RGB_conv4" 2809 | type: "Eltwise" 2810 | bottom: "conv4_3" 2811 | bottom: "eltwise_dep_weighted_RGB_conv4" 2812 | top: "concat_weighted_RGB_conv4" 2813 | eltwise_param { 2814 | operation: SUM 2815 | } 2816 | } 2817 | layer { 2818 | name: "concat_decoder_conv45" 2819 | type: "Concat" 2820 | bottom: "Up_conv5_1_D" 2821 | bottom: "concat_weighted_RGB_conv4" 2822 | bottom: "conv14_3" 2823 | top: "concat_decoder_conv45" 2824 | } 2825 | 2826 | layer { 2827 | bottom: "concat_decoder_conv45" 2828 | top: "conv4_3_D" 2829 | name: "conv4_3_D" 2830 | type: "Convolution" 2831 | param { 2832 | lr_mult: 10 2833 | decay_mult: 1 2834 | } 2835 | param { 2836 | lr_mult: 20 2837 | decay_mult: 0 2838 | } 2839 | convolution_param { 2840 | weight_filler { 2841 | type: "xavier" 2842 | } 2843 | bias_filler { 2844 | type: "constant" 2845 | value: 0 2846 | } 2847 | num_output: 512 2848 | pad: 1 2849 | kernel_size: 3 2850 | } 2851 | } 2852 | layer { 2853 | bottom: "conv4_3_D" 2854 | top: "conv4_3_D" 2855 | name: "conv4_3_D_bn" 2856 | type: "BatchNorm" 2857 | param { 2858 | lr_mult: 1 2859 | decay_mult: 1 2860 | } 2861 | param { 2862 | lr_mult: 2 2863 | decay_mult: 0 2864 | } 2865 | batch_norm_param { 2866 | use_global_stats: false 2867 | } 2868 | } 2869 | layer { 2870 | bottom: "conv4_3_D" 2871 | top: "conv4_3_D" 2872 | name: "scale_conv4_3_D" 2873 | type: "Scale" 2874 | param { 2875 | lr_mult: 1 2876 | decay_mult: 1 2877 | } 2878 | param { 2879 | lr_mult: 2 2880 | decay_mult: 0 2881 | } 2882 | scale_param { 2883 | bias_term: true 2884 | } 2885 | } 2886 | layer { 2887 | bottom: "conv4_3_D" 2888 | top: "conv4_3_D" 2889 | name: "relu4_3_D" 2890 | type: "ReLU" 2891 | } 2892 | layer { 2893 | bottom: "conv4_3_D" 2894 | top: "conv4_2_D" 2895 | name: "conv4_2_D" 2896 | type: "Convolution" 2897 | param { 2898 | lr_mult: 10 2899 | decay_mult: 1 2900 | } 2901 | param { 2902 | lr_mult: 20 2903 | decay_mult: 0 2904 | } 2905 | convolution_param { 2906 | weight_filler { 2907 | type: "xavier" 2908 | } 2909 | bias_filler { 2910 | type: "constant" 2911 | value: 0 2912 | } 2913 | num_output: 512 2914 | pad: 1 2915 | kernel_size: 3 2916 | } 2917 | } 2918 | layer { 2919 | bottom: "conv4_2_D" 2920 | top: "conv4_2_D" 2921 | name: "conv4_2_D_bn" 2922 | type: "BatchNorm" 2923 | param { 2924 | lr_mult: 1 2925 | decay_mult: 1 2926 | } 2927 | param { 2928 | lr_mult: 2 2929 | decay_mult: 0 2930 | } 2931 | batch_norm_param { 2932 | use_global_stats: false 2933 | } 2934 | } 2935 | layer { 2936 | bottom: "conv4_2_D" 2937 | top: "conv4_2_D" 2938 | name: "scale_conv4_2_D" 2939 | type: "Scale" 2940 | param { 2941 | lr_mult: 1 2942 | decay_mult: 1 2943 | } 2944 | param { 2945 | lr_mult: 2 2946 | decay_mult: 0 2947 | } 2948 | scale_param { 2949 | bias_term: true 2950 | } 2951 | } 2952 | layer { 2953 | bottom: "conv4_2_D" 2954 | top: "conv4_2_D" 2955 | name: "relu4_2_D" 2956 | type: "ReLU" 2957 | } 2958 | layer { 2959 | bottom: "conv4_2_D" 2960 | top: "conv4_1_D" 2961 | name: "conv4_1_D" 2962 | type: "Convolution" 2963 | param { 2964 | lr_mult: 10 2965 | decay_mult: 1 2966 | } 2967 | param { 2968 | lr_mult: 20 2969 | decay_mult: 0 2970 | } 2971 | convolution_param { 2972 | weight_filler { 2973 | type: "xavier" 2974 | } 2975 | bias_filler { 2976 | type: "constant" 2977 | value: 0 2978 | } 2979 | num_output: 256 2980 | pad: 1 2981 | kernel_size: 3 2982 | } 2983 | } 2984 | layer { 2985 | bottom: "conv4_1_D" 2986 | top: "conv4_1_D" 2987 | name: "conv4_1_D_bn" 2988 | type: "BatchNorm" 2989 | param { 2990 | lr_mult: 1 2991 | decay_mult: 1 2992 | } 2993 | param { 2994 | lr_mult: 2 2995 | decay_mult: 0 2996 | } 2997 | batch_norm_param { 2998 | use_global_stats: false 2999 | } 3000 | } 3001 | layer { 3002 | bottom: "conv4_1_D" 3003 | top: "conv4_1_D" 3004 | name: "scale_conv4_1_D" 3005 | type: "Scale" 3006 | param { 3007 | lr_mult: 1 3008 | decay_mult: 1 3009 | } 3010 | param { 3011 | lr_mult: 2 3012 | decay_mult: 0 3013 | } 3014 | scale_param { 3015 | bias_term: true 3016 | } 3017 | } 3018 | layer { 3019 | bottom: "conv4_1_D" 3020 | top: "conv4_1_D" 3021 | name: "relu4_1_D" 3022 | type: "ReLU" 3023 | } 3024 | layer { 3025 | name: "conv4_drop" 3026 | type: "Dropout" 3027 | bottom: "conv4_1_D" 3028 | top: "conv4_1_D" 3029 | dropout_param { 3030 | dropout_ratio: 0.5 3031 | } 3032 | } 3033 | layer { 3034 | bottom: "conv4_1_D" 3035 | top: "Up_conv4_1_D" 3036 | name: "Up_conv4_1_D" 3037 | type: "Deconvolution" 3038 | param { 3039 | lr_mult: 10 3040 | decay_mult: 1 3041 | } 3042 | param { 3043 | lr_mult: 20 3044 | decay_mult: 0 3045 | } 3046 | convolution_param { 3047 | weight_filler { 3048 | type: "xavier" 3049 | } 3050 | bias_filler { 3051 | type: "constant" 3052 | value: 0 3053 | } 3054 | num_output: 256 3055 | kernel_size: 2 3056 | stride: 2 3057 | } 3058 | } 3059 | 3060 | ##################### conv3 ################### 3061 | layer { 3062 | name: "eltwise_dep_weighted_RGB_conv3" 3063 | type: "Eltwise" 3064 | bottom: "conv3_3" 3065 | bottom: "conv13_4" 3066 | top: "eltwise_dep_weighted_RGB_conv3" 3067 | eltwise_param { 3068 | operation: PROD 3069 | } 3070 | } 3071 | layer { 3072 | name: "concat_weighted_RGB_conv3" 3073 | type: "Eltwise" 3074 | bottom: "conv3_3" 3075 | bottom: "eltwise_dep_weighted_RGB_conv3" 3076 | top: "concat_weighted_RGB_conv3" 3077 | eltwise_param { 3078 | operation: SUM 3079 | } 3080 | } 3081 | layer { 3082 | name: "concat_decoder_conv34" 3083 | type: "Concat" 3084 | bottom: "Up_conv4_1_D" 3085 | bottom: "concat_weighted_RGB_conv3" 3086 | bottom: "conv13_3" 3087 | top: "concat_decoder_conv34" 3088 | } 3089 | layer { 3090 | bottom: "concat_decoder_conv34" 3091 | top: "conv3_3_D" 3092 | name: "conv3_3_D" 3093 | type: "Convolution" 3094 | param { 3095 | lr_mult: 10 3096 | decay_mult: 1 3097 | } 3098 | param { 3099 | lr_mult: 20 3100 | decay_mult: 0 3101 | } 3102 | convolution_param { 3103 | weight_filler { 3104 | type: "xavier" 3105 | } 3106 | bias_filler { 3107 | type: "constant" 3108 | value: 0 3109 | } 3110 | num_output: 256 3111 | pad: 1 3112 | kernel_size: 3 3113 | } 3114 | } 3115 | layer { 3116 | bottom: "conv3_3_D" 3117 | top: "conv3_3_D" 3118 | name: "conv3_3_D_bn" 3119 | type: "BatchNorm" 3120 | param { 3121 | lr_mult: 1 3122 | decay_mult: 1 3123 | } 3124 | param { 3125 | lr_mult: 2 3126 | decay_mult: 0 3127 | } 3128 | batch_norm_param { 3129 | use_global_stats: false 3130 | } 3131 | } 3132 | layer { 3133 | bottom: "conv3_3_D" 3134 | top: "conv3_3_D" 3135 | name: "scale_conv3_3_D" 3136 | type: "Scale" 3137 | param { 3138 | lr_mult: 1 3139 | decay_mult: 1 3140 | } 3141 | param { 3142 | lr_mult: 2 3143 | decay_mult: 0 3144 | } 3145 | scale_param { 3146 | bias_term: true 3147 | } 3148 | } 3149 | layer { 3150 | bottom: "conv3_3_D" 3151 | top: "conv3_3_D" 3152 | name: "relu3_3_D" 3153 | type: "ReLU" 3154 | } 3155 | layer { 3156 | bottom: "conv3_3_D" 3157 | top: "conv3_2_D" 3158 | name: "conv3_2_D" 3159 | type: "Convolution" 3160 | param { 3161 | lr_mult: 10 3162 | decay_mult: 1 3163 | } 3164 | param { 3165 | lr_mult: 20 3166 | decay_mult: 0 3167 | } 3168 | convolution_param { 3169 | weight_filler { 3170 | type: "xavier" 3171 | } 3172 | bias_filler { 3173 | type: "constant" 3174 | value: 0 3175 | } 3176 | num_output: 256 3177 | pad: 1 3178 | kernel_size: 3 3179 | } 3180 | } 3181 | layer { 3182 | bottom: "conv3_2_D" 3183 | top: "conv3_2_D" 3184 | name: "conv3_2_D_bn" 3185 | type: "BatchNorm" 3186 | param { 3187 | lr_mult: 1 3188 | decay_mult: 1 3189 | } 3190 | param { 3191 | lr_mult: 2 3192 | decay_mult: 0 3193 | } 3194 | batch_norm_param { 3195 | use_global_stats: false 3196 | } 3197 | } 3198 | layer { 3199 | bottom: "conv3_2_D" 3200 | top: "conv3_2_D" 3201 | name: "scale_conv3_2_D" 3202 | type: "Scale" 3203 | param { 3204 | lr_mult: 1 3205 | decay_mult: 1 3206 | } 3207 | param { 3208 | lr_mult: 2 3209 | decay_mult: 0 3210 | } 3211 | scale_param { 3212 | bias_term: true 3213 | } 3214 | } 3215 | layer { 3216 | bottom: "conv3_2_D" 3217 | top: "conv3_2_D" 3218 | name: "relu3_2_D" 3219 | type: "ReLU" 3220 | } 3221 | layer { 3222 | bottom: "conv3_2_D" 3223 | top: "conv3_1_D" 3224 | name: "conv3_1_D" 3225 | type: "Convolution" 3226 | param { 3227 | lr_mult: 10 3228 | decay_mult: 1 3229 | } 3230 | param { 3231 | lr_mult: 20 3232 | decay_mult: 0 3233 | } 3234 | convolution_param { 3235 | weight_filler { 3236 | type: "xavier" 3237 | } 3238 | bias_filler { 3239 | type: "constant" 3240 | value: 0 3241 | } 3242 | num_output: 128 3243 | pad: 1 3244 | kernel_size: 3 3245 | } 3246 | } 3247 | layer { 3248 | bottom: "conv3_1_D" 3249 | top: "conv3_1_D" 3250 | name: "conv3_1_D_bn" 3251 | type: "BatchNorm" 3252 | param { 3253 | lr_mult: 1 3254 | decay_mult: 1 3255 | } 3256 | param { 3257 | lr_mult: 2 3258 | decay_mult: 0 3259 | } 3260 | batch_norm_param { 3261 | use_global_stats: false 3262 | } 3263 | } 3264 | layer { 3265 | bottom: "conv3_1_D" 3266 | top: "conv3_1_D" 3267 | name: "scale_conv3_1_D" 3268 | type: "Scale" 3269 | param { 3270 | lr_mult: 1 3271 | decay_mult: 1 3272 | } 3273 | param { 3274 | lr_mult: 2 3275 | decay_mult: 0 3276 | } 3277 | scale_param { 3278 | bias_term: true 3279 | } 3280 | } 3281 | layer { 3282 | bottom: "conv3_1_D" 3283 | top: "conv3_1_D" 3284 | name: "relu3_1_D" 3285 | type: "ReLU" 3286 | } 3287 | layer { 3288 | name: "conv3_drop" 3289 | type: "Dropout" 3290 | bottom: "conv3_1_D" 3291 | top: "conv3_1_D" 3292 | dropout_param { 3293 | dropout_ratio: 0.5 3294 | } 3295 | } 3296 | layer { 3297 | bottom: "conv3_1_D" 3298 | top: "Up_conv3_1_D" 3299 | name: "Up_conv3_1_D" 3300 | type: "Deconvolution" 3301 | param { 3302 | lr_mult: 10 3303 | decay_mult: 1 3304 | } 3305 | param { 3306 | lr_mult: 20 3307 | decay_mult: 0 3308 | } 3309 | convolution_param { 3310 | weight_filler { 3311 | type: "xavier" 3312 | } 3313 | bias_filler { 3314 | type: "constant" 3315 | value: 0 3316 | } 3317 | num_output: 128 3318 | kernel_size: 2 3319 | stride: 2 3320 | } 3321 | } 3322 | 3323 | ##################### conv2 ################### 3324 | layer { 3325 | name: "eltwise_dep_weighted_RGB_conv2" 3326 | type: "Eltwise" 3327 | bottom: "conv2_2" 3328 | bottom: "conv12_3" 3329 | top: "eltwise_dep_weighted_RGB_conv2" 3330 | eltwise_param { 3331 | operation: PROD 3332 | } 3333 | } 3334 | layer { 3335 | name: "concat_weighted_RGB_conv2" 3336 | type: "Eltwise" 3337 | bottom: "conv2_2" 3338 | bottom: "eltwise_dep_weighted_RGB_conv2" 3339 | top: "concat_weighted_RGB_conv2" 3340 | eltwise_param { 3341 | operation: SUM 3342 | } 3343 | } 3344 | layer { 3345 | name: "concat_decoder_conv21" 3346 | type: "Concat" 3347 | bottom: "Up_conv3_1_D" 3348 | bottom: "concat_weighted_RGB_conv2" 3349 | bottom: "conv12_2" 3350 | top: "concat_decoder_conv21" 3351 | } 3352 | layer { 3353 | bottom: "concat_decoder_conv21" 3354 | top: "conv2_2_D" 3355 | name: "conv2_2_D" 3356 | type: "Convolution" 3357 | param { 3358 | lr_mult: 10 3359 | decay_mult: 1 3360 | } 3361 | param { 3362 | lr_mult: 20 3363 | decay_mult: 0 3364 | } 3365 | convolution_param { 3366 | weight_filler { 3367 | type: "xavier" 3368 | } 3369 | bias_filler { 3370 | type: "constant" 3371 | value: 0 3372 | } 3373 | num_output: 128 3374 | pad: 1 3375 | kernel_size: 3 3376 | } 3377 | } 3378 | layer { 3379 | bottom: "conv2_2_D" 3380 | top: "conv2_2_D" 3381 | name: "conv2_2_D_bn" 3382 | type: "BatchNorm" 3383 | param { 3384 | lr_mult: 1 3385 | decay_mult: 1 3386 | } 3387 | param { 3388 | lr_mult: 2 3389 | decay_mult: 0 3390 | } 3391 | batch_norm_param { 3392 | use_global_stats: false 3393 | } 3394 | } 3395 | layer { 3396 | bottom: "conv2_2_D" 3397 | top: "conv2_2_D" 3398 | name: "scale_conv2_2_D" 3399 | type: "Scale" 3400 | param { 3401 | lr_mult: 1 3402 | decay_mult: 1 3403 | } 3404 | param { 3405 | lr_mult: 2 3406 | decay_mult: 0 3407 | } 3408 | scale_param { 3409 | bias_term: true 3410 | } 3411 | } 3412 | layer { 3413 | bottom: "conv2_2_D" 3414 | top: "conv2_2_D" 3415 | name: "relu2_2_D" 3416 | type: "ReLU" 3417 | } 3418 | layer { 3419 | bottom: "conv2_2_D" 3420 | top: "conv2_1_D" 3421 | name: "conv2_1_D" 3422 | type: "Convolution" 3423 | param { 3424 | lr_mult: 10 3425 | decay_mult: 1 3426 | } 3427 | param { 3428 | lr_mult: 20 3429 | decay_mult: 0 3430 | } 3431 | convolution_param { 3432 | weight_filler { 3433 | type: "xavier" 3434 | } 3435 | bias_filler { 3436 | type: "constant" 3437 | value: 0 3438 | } 3439 | num_output: 64 3440 | pad: 1 3441 | kernel_size: 3 3442 | } 3443 | } 3444 | layer { 3445 | bottom: "conv2_1_D" 3446 | top: "conv2_1_D" 3447 | name: "conv2_1_D_bn" 3448 | type: "BatchNorm" 3449 | param { 3450 | lr_mult: 1 3451 | decay_mult: 1 3452 | } 3453 | param { 3454 | lr_mult: 2 3455 | decay_mult: 0 3456 | } 3457 | batch_norm_param { 3458 | use_global_stats: false 3459 | } 3460 | } 3461 | layer { 3462 | bottom: "conv2_1_D" 3463 | top: "conv2_1_D" 3464 | name: "scale_conv2_1_D" 3465 | type: "Scale" 3466 | param { 3467 | lr_mult: 1 3468 | decay_mult: 1 3469 | } 3470 | param { 3471 | lr_mult: 2 3472 | decay_mult: 0 3473 | } 3474 | scale_param { 3475 | bias_term: true 3476 | } 3477 | } 3478 | layer { 3479 | bottom: "conv2_1_D" 3480 | top: "conv2_1_D" 3481 | name: "relu2_1_D" 3482 | type: "ReLU" 3483 | } 3484 | layer { 3485 | name: "conv2_drop" 3486 | type: "Dropout" 3487 | bottom: "conv2_1_D" 3488 | top: "conv2_1_D" 3489 | dropout_param { 3490 | dropout_ratio: 0.5 3491 | } 3492 | } 3493 | layer { 3494 | bottom: "conv2_1_D" 3495 | top: "Up_conv2_1_D" 3496 | name: "Up_conv2_1_D" 3497 | type: "Deconvolution" 3498 | param { 3499 | lr_mult: 10 3500 | decay_mult: 1 3501 | } 3502 | param { 3503 | lr_mult: 20 3504 | decay_mult: 0 3505 | } 3506 | convolution_param { 3507 | weight_filler { 3508 | type: "xavier" 3509 | } 3510 | bias_filler { 3511 | type: "constant" 3512 | value: 0 3513 | } 3514 | num_output: 64 3515 | kernel_size: 2 3516 | stride: 2 3517 | } 3518 | } 3519 | 3520 | ##################### conv1 ################### 3521 | layer { 3522 | name: "eltwise_dep_weighted_RGB_conv1" 3523 | type: "Eltwise" 3524 | bottom: "conv1_2" 3525 | bottom: "conv11_3" 3526 | top: "eltwise_dep_weighted_RGB_conv1" 3527 | eltwise_param { 3528 | operation: PROD 3529 | } 3530 | } 3531 | layer { 3532 | name: "concat_weighted_RGB_conv1" 3533 | type: "Eltwise" 3534 | bottom: "conv1_2" 3535 | bottom: "eltwise_dep_weighted_RGB_conv1" 3536 | top: "concat_weighted_RGB_conv1" 3537 | eltwise_param { 3538 | operation: SUM 3539 | } 3540 | } 3541 | layer { 3542 | name: "concat_decoder_conv12" 3543 | type: "Concat" 3544 | bottom: "Up_conv2_1_D" 3545 | bottom: "concat_weighted_RGB_conv1" 3546 | bottom: "conv11_2" 3547 | top: "concat_decoder_conv12" 3548 | } 3549 | layer { 3550 | bottom: "concat_decoder_conv12" 3551 | top: "conv1_2_D" 3552 | name: "conv1_2_D" 3553 | type: "Convolution" 3554 | param { 3555 | lr_mult: 10 3556 | decay_mult: 1 3557 | } 3558 | param { 3559 | lr_mult: 20 3560 | decay_mult: 0 3561 | } 3562 | convolution_param { 3563 | weight_filler { 3564 | type: "xavier" 3565 | } 3566 | bias_filler { 3567 | type: "constant" 3568 | value: 0 3569 | } 3570 | num_output: 64 3571 | pad: 1 3572 | kernel_size: 3 3573 | } 3574 | } 3575 | layer { 3576 | bottom: "conv1_2_D" 3577 | top: "conv1_2_D" 3578 | name: "conv1_2_D_bn" 3579 | type: "BatchNorm" 3580 | param { 3581 | lr_mult: 1 3582 | decay_mult: 1 3583 | } 3584 | param { 3585 | lr_mult: 2 3586 | decay_mult: 0 3587 | } 3588 | batch_norm_param { 3589 | use_global_stats: false 3590 | } 3591 | } 3592 | layer { 3593 | bottom: "conv1_2_D" 3594 | top: "conv1_2_D" 3595 | name: "scale_conv1_2_D" 3596 | type: "Scale" 3597 | param { 3598 | lr_mult: 1 3599 | decay_mult: 1 3600 | } 3601 | param { 3602 | lr_mult: 2 3603 | decay_mult: 0 3604 | } 3605 | scale_param { 3606 | bias_term: true 3607 | } 3608 | } 3609 | layer { 3610 | bottom: "conv1_2_D" 3611 | top: "conv1_2_D" 3612 | name: "relu1_2_D" 3613 | type: "ReLU" 3614 | } 3615 | layer { 3616 | bottom: "conv1_2_D" 3617 | top: "conv1_1_D" 3618 | name: "conv1_1_D" 3619 | type: "Convolution" 3620 | param { 3621 | lr_mult: 10 3622 | decay_mult: 1 3623 | } 3624 | param { 3625 | lr_mult: 20 3626 | decay_mult: 0 3627 | } 3628 | convolution_param { 3629 | weight_filler { 3630 | type: "xavier" 3631 | } 3632 | bias_filler { 3633 | type: "constant" 3634 | value: 0 3635 | } 3636 | num_output: 2 3637 | pad: 1 3638 | kernel_size: 3 3639 | } 3640 | } 3641 | 3642 | layer { 3643 | name: "loss11" 3644 | type: "Softmax" 3645 | bottom: "conv1_1_D" 3646 | top: "loss11" 3647 | } 3648 | -------------------------------------------------------------------------------- /salmaps/DES/RGBD_data_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/salmaps/DES/RGBD_data_1.png -------------------------------------------------------------------------------- /salmaps/DES/RGBD_data_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/salmaps/DES/RGBD_data_2.png -------------------------------------------------------------------------------- /salmaps/DES/RGBD_data_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MathLee/ICNet-for-RGBD-SOD/c7cd8f40f54472b9c0e5cda63ff147e6af7b12d0/salmaps/DES/RGBD_data_3.png -------------------------------------------------------------------------------- /test_matlab/test_DES_HHA.m: -------------------------------------------------------------------------------- 1 | function test_DES_HHA(iters_num,gpu_id,resolution) 2 | %% setting caffe test config 3 | addpath('./flownet2-master/'); 4 | use_gpu= 1; 5 | % Set caffe mode 6 | if exist('use_gpu', 'var') && use_gpu 7 | caffe.set_mode_gpu(); 8 | % gpu_id = 0; % we will use the first gpu in this demo 9 | caffe.set_device(gpu_id); 10 | else 11 | caffe.set_mode_cpu(); 12 | end 13 | %% Initialize the network 14 | % change to your path... 15 | net_model = fullfile('./flownet2-master/models/test_RGBD.prototxt'); 16 | net_weights = fullfile('./flownet2-master/models/',['RGBD_iter_' num2str(iters_num) '.caffemodel']); 17 | 18 | end 19 | phase = 'test'; 20 | net = caffe.Net(net_model, net_weights, phase); 21 | %% load images from different Datasets 22 | % change to your path.... 23 | imPath = './flownet2-master/datasets/DES/RGB/'; 24 | DepthPath = './flownet2-master/datasets/DES/HHA/'; 25 | salPath = './flownet2-master/salmaps/DES/'; 26 | if ~exist(fullfile(salPath),'dir') 27 | mkdir(fullfile(salPath)); 28 | end 29 | files = dir([imPath '*.jpg']); 30 | num = length(files); 31 | time = 0; 32 | %% test each image 33 | for i = 1 : num 34 | im = imread([imPath files(i).name]); 35 | if size(im,3)==1 36 | im = cat(3,im,im,im); 37 | end 38 | Depthim = (imread([DepthPath files(i).name ])); 39 | 40 | % do forward pass to get scores 41 | res = net.forward({prepare_image(im,resolution), prepare_Depthimage(Depthim,resolution)}); 42 | salmap = permute(res{1}(:,:,2), [2 1 3]); 43 | salmap = imresize(salmap,[size(im,1) size(im,2)], 'bilinear'); 44 | imwrite(salmap, fullfile(salPath, [files(i).name(1:end-4) '.png'])); 45 | 46 | end 47 | caffe.reset_all(); 48 | end 49 | %% -------------------- prepare images --------------------------------- 50 | function images = prepare_image(im,resolution) 51 | % ------------------------------------------------------------------------ 52 | IMAGE_DIM = resolution; 53 | % resize to fixed input size 54 | im = single(im); 55 | im = imresize(im, [IMAGE_DIM IMAGE_DIM], 'bilinear'); 56 | % permute from RGB to BGR (IMAGE_MEAN is already BGR) 57 | im = im(:,:,[3 2 1]); 58 | im(:,:,1) = im(:,:,1) -104.00699 ; 59 | im(:,:,2) = im(:,:,2) -116.66877 ; 60 | im(:,:,3) = im(:,:,3) -122.67892 ; 61 | images = permute(im,[2 1 3]); 62 | % ------------------------------------------------------------------------ 63 | end 64 | function Depthimages = prepare_Depthimage(Depthim,resolution) 65 | % ------------------------------------------------------------------------ 66 | IMAGE_DIM = resolution; 67 | % resize to fixed input size 68 | Depthim = single(Depthim); 69 | Depthim = imresize(Depthim, [IMAGE_DIM IMAGE_DIM], 'bilinear'); 70 | Depthim = Depthim(:,:,[3 2 1]); 71 | Depthimages = permute(Depthim,[2 1 3]); 72 | % ------------------------------------------------------------------------ 73 | end 74 | -------------------------------------------------------------------------------- /test_matlab/test_ICNet.m: -------------------------------------------------------------------------------- 1 | clear; 2 | clc; 3 | close all; 4 | 5 | %################################### Inference Saliency Map ################################## 6 | % This code is used for ICnet 7 | % Code Author: Gongyang Li 8 | % Email: lllmiemie@163.com, ligongyang@shu.edu.cn 9 | % Date: 13/7/2020 10 | % The code is based on the following paper in TIP2020: 11 | % Title: ICNet: Information Conversion Network for RGB-D Based Salient Object Detection 12 | % Authors: Gongyang Li, Zhi Liu*, and Haibin Ling 13 | %############################################################################################# 14 | 15 | gpu_id = 1; 16 | resolution = 288; 17 | %% Producing the saliency map of DES dataset 18 | display(['Producing the saliency map of DES dataset.']); 19 | mIoU = test_DES_HHA(22500,gpu_id,resolution); 20 | 21 | --------------------------------------------------------------------------------