├── LICENSE ├── README.md ├── compile.sh ├── dataloader ├── data.py └── dataset.py ├── evaluation.py ├── evaluation.sh ├── illustration ├── Bicycle2.png ├── Bicycle2_disp.png ├── Crusade.png ├── Crusade_disp.png ├── cityscape_029736.png └── cityscape_disp.png ├── libs ├── GANet │ ├── functions │ │ ├── GANet.py │ │ └── __init__.py │ ├── modules │ │ ├── GANet.py │ │ └── __init__.py │ ├── setup.py │ └── src │ │ ├── GANet_cuda.cpp │ │ ├── GANet_cuda.h │ │ ├── GANet_kernel.cu │ │ ├── GANet_kernel.h │ │ └── GANet_kernel0.cu └── sync_bn │ ├── functions │ ├── __init__.py │ └── sync_bn.py │ ├── modules │ ├── __init__.py │ └── sync_bn.py │ ├── setup.py │ └── src │ ├── __init__.py │ ├── cpu │ ├── operator.cpp │ ├── operator.h │ └── sync_bn.cpp │ └── gpu │ ├── common.h │ ├── device_tensor.h │ ├── operator.cpp │ ├── operator.h │ └── sync_bn_cuda.cu ├── lists ├── debug.list ├── kitti2012_test.list ├── kitti2012_train.list ├── kitti2012_train170.list ├── kitti2012_val24.list ├── kitti2015_test.list ├── kitti2015_train.list ├── sceneflow_test.list ├── sceneflow_test_select.list └── sceneflow_train.list ├── models ├── GANet11.py └── GANet_deep.py ├── predict.py ├── predict.sh ├── train.py └── train.sh /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Feihu Zhang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GANet 2 | 3 | [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://arxiv.org/pdf/1904.06587.pdf) 4 | 5 | 6 | 7 | ## Brief Introduction 8 | We are formulating traditional geometric and optimization of stereo into deep neural networks ... 9 | 10 | ## Oral Presentation 11 | 12 | [Slides](http://www.feihuzhang.com/GANet/GANet.pptx), [Video](https://www.youtube.com/watch?v=tpyrxcGL_Zg&feature=youtu.be), [Poster](http://www.feihuzhang.com/GANet/GANet_poster.pdf) 13 | 14 | 15 | ## Building Requirements: 16 | 17 | gcc: >=5.3 18 | GPU mem: >=6.5G (for testing); >=11G (for training, >=22G is prefered) 19 | pytorch: >=1.0 20 | cuda: >=9.2 (9.0 doesn’t support well for the new pytorch version and may have “pybind11 errors”.) 21 | tested platform/settings: 22 | 1) ubuntu 16.04 + cuda 10.0 + python 3.6, 3.7 23 | 2) centos + cuda 9.2 + python 3.7 24 | 25 | 26 | ## Install Pytorch: 27 | You can easily install pytorch (>=1.0) by "pip install" to run the code. See this https://github.com/feihuzhang/GANet/issues/24 28 | 29 | But, if you have trouble (lib conflicts) when compiling cuda libs, 30 | installing pytorch from source would help solve most of the errors (lib conflicts). 31 | 32 | Please refer to https://github.com/pytorch/pytorch about how to reinstall pytorch from source. 33 | 34 | ## How to Use? 35 | 36 | Step 1: compile the libs by "sh compile.sh" 37 | - Change the environmental variable ($PATH, $LD_LIBRARY_PATH etc.), if it's not set correctly in your system environment (e.g. .bashrc). Examples are included in "compile.sh". 38 | - If you met the BN error, try to replace the sync-bn with another version: 39 | 1) Install NVIDIA-Apex package https://github.com/NVIDIA/apex 40 | $ git clone https://github.com/NVIDIA/apex 41 | $ cd apex 42 | $ pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./ 43 | 2) Revise the "GANet_deep.py": 44 | add `import apex` 45 | change all `BatchNorm2d` and `BatchNorm3d` to `apex.parallel.SyncBatchNorm` 46 | 47 | Step 2: download and prepare the dataset 48 | 49 | download SceneFLow dataset: "FlyingThings3D", "Driving" and "Monkaa" (final pass and disparity files). 50 | 51 | -mv all training images (totallty 29 folders) into ${your dataset PATH}/frames_finalpass/TRAIN/ 52 | -mv all corresponding disparity files (totallty 29 folders) into ${your dataset PATH}/disparity/TRAIN/ 53 | -make sure the following 29 folders are included in the "${your dataset PATH}/disparity/TRAIN/" and "${your dataset PATH}/frames_finalpass/TRAIN/": 54 | 55 | 15mm_focallength 35mm_focallength A a_rain_of_stones_x2 B C 56 | eating_camera2_x2 eating_naked_camera2_x2 eating_x2 family_x2 flower_storm_augmented0_x2 flower_storm_augmented1_x2 57 | flower_storm_x2 funnyworld_augmented0_x2 funnyworld_augmented1_x2 funnyworld_camera2_augmented0_x2 funnyworld_camera2_augmented1_x2 funnyworld_camera2_x2 58 | funnyworld_x2 lonetree_augmented0_x2 lonetree_augmented1_x2 lonetree_difftex2_x2 lonetree_difftex_x2 lonetree_winter_x2 59 | lonetree_x2 top_view_x2 treeflight_augmented0_x2 treeflight_augmented1_x2 treeflight_x2 60 | 61 | download and extract kitti and kitti2015 datasets. 62 | 63 | Step 3: revise parameter settings and run "train.sh" and "predict.sh" for training, finetuning and prediction/testing. Note that the “crop_width” and “crop_height” must be multiple of 48, "max_disp" must be multiple of 12 (default: 192). 64 | 65 | 66 | ## Pretrained models: 67 | 68 | - These pre-trained models use a batchsize of 8 on four P40 GPUs with a crop size of 240x624. 69 | - Eight 1080ti/Titan GPUs should also be able to achieve the similar accuracy. 70 | - Eight P40/V100/Titan RTX (22G) GPUs would be even better. 71 | 72 | | sceneflow (for fine-tuning, only 10 epoch) | kitti2012 (after fine-tuning) | kitti2015 (after fine-tuning)| 73 | |---|---|---| 74 | |[Google Drive](https://drive.google.com/open?id=1VkcBGkA_pXolgLhrWdpZPwfvzhQfWWJQ)|[Google Drive](https://drive.google.com/open?id=1WMfbEhzj-WLqYEI2jCH1YFUR6dYyzlVE)|[Google Drive](https://drive.google.com/open?id=19hVQXpcXwp7SrHgJ5Tlu7_iCYNi4Oj9u)| 75 | 76 | ## Results: 77 | 78 | The results of the deep model are better than those reported in the paper. 79 | 80 | #### Evaluations and Comparisons on SceneFlow Dataset (only 10 epoches) 81 | |Models|3D conv layers|GA layers |Avg. EPE (pixel)|1-pixel Error rate (%)| 82 | |---|---|---|---|---| 83 | |GC-Net|19|-|1.8|15.6| 84 | |PSMNet|35|-|1.09|12.1| 85 | |GANet-15|15|5|0.84|9.9| 86 | |GANet-deep|22|9|0.78|8.7| 87 | 88 | 89 | #### Evaluations on KITTI 2012 benchmark 90 | | Models | Non-Occluded | All Area | 91 | |---|---|---| 92 | | [GC-Net](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow_detail.php?benchmark=stereo&error=3&eval=all&result=8da072a8f49d792632b8940582d5578c7d86b747)| 1.77 | 2.30 | 93 | | [PSMNet](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow_detail.php?benchmark=stereo&error=3&eval=all&result=8da072a8f49d792632b8940582d5578c7d86b747) | 1.49 | 1.89 | 94 | | [GANet-15](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow_detail.php?benchmark=stereo&error=3&eval=all&result=b2d616a45b7b7bda1cb9d1fd834b5d7c70e9f4cc) | 1.36 | 1.80 | 95 | | [GANet-deep](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow_detail.php?benchmark=stereo&error=3&eval=all&result=95af4a21253204c14e9dc7ab8beb9d9b114cfb9d) | 1.19 | 1.60 | 96 | 97 | #### Evaluations on KITTI 2015 benchmark 98 | 99 | | Models | Non-Occluded | All Area | 100 | |---|---|---| 101 | | [GC-Net](http://www.cvlibs.net/datasets/kitti/eval_scene_flow_detail.php?benchmark=stereo&result=70b339586af7c573b33a4dad14ea4a7689dc9305) | 2.61 | 2.87 | 102 | | [PSMNet](http://www.cvlibs.net/datasets/kitti/eval_scene_flow_detail.php?benchmark=stereo&result=efb9db97938e12a20b9c95ce593f633dd63a2744) | 2.14 | 2.32 | 103 | | [GANet-15](http://www.cvlibs.net/datasets/kitti/eval_scene_flow_detail.php?benchmark=stereo&result=59cfbc4149e979b63b961f9daa3aa2bae021eff3) | 1.73 | 1.93 | 104 | | [GANet-deep](http://www.cvlibs.net/datasets/kitti/eval_scene_flow_detail.php?benchmark=stereo&result=ccb2b24d3e08ec968368f85a4eeab8b668e70b8c) | 1.63 | 1.81 | 105 | 106 | ## Great Generalization Abilities: 107 | GANet has great generalization abilities on other datasets/scenes. 108 | 109 | #### Cityscape 110 | 111 | 112 | #### Middlebury 113 | 114 | 115 | 116 | 117 | 118 | ## Reference: 119 | 120 | If you find the code useful, please cite our paper: 121 | 122 | @inproceedings{Zhang2019GANet, 123 | title={GA-Net: Guided Aggregation Net for End-to-end Stereo Matching}, 124 | author={Zhang, Feihu and Prisacariu, Victor and Yang, Ruigang and Torr, Philip HS}, 125 | booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, 126 | pages={185--194}, 127 | year={2019} 128 | } 129 | -------------------------------------------------------------------------------- /compile.sh: -------------------------------------------------------------------------------- 1 | #export LD_LIBRARY_PATH="/home/feihu/anaconda3/lib:$LD_LIBRARY_PATH" 2 | #export LD_INCLUDE_PATH="/home/feihu/anaconda3/include:$LD_INCLUDE_PATH" 3 | #export CUDA_HOME="/usr/local/cuda-10.0" 4 | #export PATH="/home/feihu/anaconda3/bin:/usr/local/cuda-10.0/bin:$PATH" 5 | #export CPATH="/usr/local/cuda-10.0/include" 6 | #export CUDNN_INCLUDE_DIR="/usr/local/cuda-10.0/include" 7 | #export CUDNN_LIB_DIR="/usr/local/cuda-10.0/lib64" 8 | 9 | #export LD_LIBRARY_PATH="/home/zhangfeihu/anaconda3/lib:$LD_LIBRARY_PATH" 10 | #export LD_INCLUDE_PATH="/home/zhangfeihu/anaconda3/include:$LD_INCLUDE_PATH" 11 | #export CUDA_HOME="/home/work/cuda-9.2" 12 | #export PATH="/home/zhangfeihu/anaconda3/bin:/home/work/cuda-9.2/bin:$PATH" 13 | #export CPATH="/home/work/cuda-9.2/include" 14 | #export CUDNN_INCLUDE_DIR="/home/work/cudnn/cudnn_v7/include" 15 | #export CUDNN_LIB_DIR="/home/work/cudnn/cudnn_v7/lib64" 16 | TORCH=$(python -c "import os; import torch; print(os.path.dirname(torch.__file__))") 17 | #echo $TORCH 18 | cd libs/GANet 19 | python setup.py clean 20 | rm -rf build 21 | python setup.py build 22 | cp -r build/lib* build/lib 23 | 24 | cd ../sync_bn 25 | python setup.py clean 26 | rm -rf build 27 | python setup.py build 28 | cp -r build/lib* build/lib 29 | -------------------------------------------------------------------------------- /dataloader/data.py: -------------------------------------------------------------------------------- 1 | from .dataset import DatasetFromList 2 | 3 | def get_training_set(data_path, train_list, crop_size=[256,256], left_right=False, kitti=False, kitti2015=False, shift=0): 4 | return DatasetFromList(data_path, train_list, 5 | crop_size, True, left_right, kitti, kitti2015, shift) 6 | 7 | 8 | def get_test_set(data_path, test_list, crop_size=[256,256], left_right=False, kitti=False, kitti2015=False): 9 | return DatasetFromList(data_path, test_list, 10 | crop_size, False, left_right, kitti, kitti2015) 11 | -------------------------------------------------------------------------------- /dataloader/dataset.py: -------------------------------------------------------------------------------- 1 | import torch.utils.data as data 2 | import skimage 3 | import skimage.io 4 | import skimage.transform 5 | 6 | from PIL import Image 7 | import numpy as np 8 | import random 9 | from struct import unpack 10 | import re 11 | import sys 12 | def readPFM(file): 13 | with open(file, "rb") as f: 14 | # Line 1: PF=>RGB (3 channels), Pf=>Greyscale (1 channel) 15 | type = f.readline().decode('latin-1') 16 | if "PF" in type: 17 | channels = 3 18 | elif "Pf" in type: 19 | channels = 1 20 | else: 21 | sys.exit(1) 22 | # Line 2: width height 23 | line = f.readline().decode('latin-1') 24 | width, height = re.findall('\d+', line) 25 | width = int(width) 26 | height = int(height) 27 | 28 | # Line 3: +ve number means big endian, negative means little endian 29 | line = f.readline().decode('latin-1') 30 | BigEndian = True 31 | if "-" in line: 32 | BigEndian = False 33 | # Slurp all binary data 34 | samples = width * height * channels; 35 | buffer = f.read(samples * 4) 36 | # Unpack floats with appropriate endianness 37 | if BigEndian: 38 | fmt = ">" 39 | else: 40 | fmt = "<" 41 | fmt = fmt + str(samples) + "f" 42 | img = unpack(fmt, buffer) 43 | img = np.reshape(img, (height, width)) 44 | img = np.flipud(img) 45 | # quit() 46 | return img, height, width 47 | 48 | def train_transform(temp_data, crop_height, crop_width, left_right=False, shift=0): 49 | _, h, w = np.shape(temp_data) 50 | 51 | if h > crop_height and w <= crop_width: 52 | temp = temp_data 53 | temp_data = np.zeros([8, h+shift, crop_width + shift], 'float32') 54 | temp_data[6:7,:,:] = 1000 55 | temp_data[:, h + shift - h: h + shift, crop_width + shift - w: crop_width + shift] = temp 56 | _, h, w = np.shape(temp_data) 57 | 58 | if h <= crop_height and w <= crop_width: 59 | temp = temp_data 60 | temp_data = np.zeros([8, crop_height + shift, crop_width + shift], 'float32') 61 | temp_data[6: 7, :, :] = 1000 62 | temp_data[:, crop_height + shift - h: crop_height + shift, crop_width + shift - w: crop_width + shift] = temp 63 | _, h, w = np.shape(temp_data) 64 | if shift > 0: 65 | start_x = random.randint(0, w - crop_width) 66 | shift_x = random.randint(-shift, shift) 67 | if shift_x + start_x < 0 or shift_x + start_x + crop_width > w: 68 | shift_x = 0 69 | start_y = random.randint(0, h - crop_height) 70 | left = temp_data[0: 3, start_y: start_y + crop_height, start_x + shift_x: start_x + shift_x + crop_width] 71 | right = temp_data[3: 6, start_y: start_y + crop_height, start_x: start_x + crop_width] 72 | target = temp_data[6: 7, start_y: start_y + crop_height, start_x + shift_x : start_x+shift_x + crop_width] 73 | target = target - shift_x 74 | return left, right, target 75 | if h <= crop_height and w <= crop_width: 76 | temp = temp_data 77 | temp_data = np.zeros([8, crop_height, crop_width], 'float32') 78 | temp_data[:, crop_height - h: crop_height, crop_width - w: crop_width] = temp 79 | else: 80 | start_x = random.randint(0, w - crop_width) 81 | start_y = random.randint(0, h - crop_height) 82 | temp_data = temp_data[:, start_y: start_y + crop_height, start_x: start_x + crop_width] 83 | if random.randint(0, 1) == 0 and left_right: 84 | right = temp_data[0: 3, :, :] 85 | left = temp_data[3: 6, :, :] 86 | target = temp_data[7: 8, :, :] 87 | return left, right, target 88 | else: 89 | left = temp_data[0: 3, :, :] 90 | right = temp_data[3: 6, :, :] 91 | target = temp_data[6: 7, :, :] 92 | return left, right, target 93 | 94 | 95 | 96 | def test_transform(temp_data, crop_height, crop_width, left_right=False): 97 | _, h, w = np.shape(temp_data) 98 | # if crop_height-h>20 or crop_width-w>20: 99 | # print 'crop_size over size!' 100 | if h <= crop_height and w <= crop_width: 101 | temp = temp_data 102 | temp_data = np.zeros([8,crop_height,crop_width], 'float32') 103 | temp_data[6: 7, :, :] = 1000 104 | temp_data[:, crop_height - h: crop_height, crop_width - w: crop_width] = temp 105 | else: 106 | start_x = (w-crop_width)/2 107 | start_y = (h-crop_height)/2 108 | temp_data = temp_data[:, start_y: start_y + crop_height, start_x: start_x + crop_width] 109 | 110 | left = temp_data[0: 3, :, :] 111 | right = temp_data[3: 6, :, :] 112 | target = temp_data[6: 7, :, :] 113 | # sign=np.ones([1,1,1],'float32')*-1 114 | return left, right, target 115 | 116 | 117 | def load_data(data_path, current_file): 118 | A = current_file 119 | filename = data_path + 'frames_finalpass/' + A[0: len(A) - 1] 120 | left =Image.open(filename) 121 | filename = data_path + 'frames_finalpass/' + A[0: len(A) - 14] + 'right/' + A[len(A) - 9:len(A) - 1] 122 | right = Image.open(filename) 123 | filename = data_path + 'disparity/' + A[0: len(A) - 4] + 'pfm' 124 | disp_left, height, width = readPFM(filename) 125 | filename = data_path + 'disparity/' + A[0: len(A) - 14] + 'right/' + A[len(A) - 9: len(A) - 4] + 'pfm' 126 | disp_right, height, width = readPFM(filename) 127 | size = np.shape(left) 128 | height = size[0] 129 | width = size[1] 130 | temp_data = np.zeros([8, height, width], 'float32') 131 | left = np.asarray(left) 132 | right = np.asarray(right) 133 | r = left[:, :, 0] 134 | g = left[:, :, 1] 135 | b = left[:,:,2] 136 | temp_data[0, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 137 | temp_data[1, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 138 | temp_data[2, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 139 | r=right[:, :, 0] 140 | g=right[:, :, 1] 141 | b=right[:, :, 2] 142 | temp_data[3, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 143 | temp_data[4, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 144 | temp_data[5, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 145 | temp_data[6: 7, :, :] = width * 2 146 | temp_data[6, :, :] = disp_left 147 | temp_data[7, :, :] = disp_right 148 | return temp_data 149 | 150 | 151 | def load_kitti_data(file_path, current_file): 152 | """ load current file from the list""" 153 | filename = file_path + 'colored_0/' + current_file[0: len(current_file) - 1] 154 | left = Image.open(filename) 155 | filename = file_path+'colored_1/' + current_file[0: len(current_file) - 1] 156 | right = Image.open(filename) 157 | filename = file_path+'disp_occ/' + current_file[0: len(current_file) - 1] 158 | 159 | disp_left = Image.open(filename) 160 | temp = np.asarray(disp_left) 161 | size = np.shape(left) 162 | 163 | height = size[0] 164 | width = size[1] 165 | temp_data = np.zeros([8, height, width], 'float32') 166 | left = np.asarray(left) 167 | right = np.asarray(right) 168 | disp_left = np.asarray(disp_left) 169 | r = left[:, :, 0] 170 | g = left[:, :, 1] 171 | b = left[:, :, 2] 172 | 173 | temp_data[0, :, :] = (r-np.mean(r[:])) / np.std(r[:]) 174 | temp_data[1, :, :] = (g-np.mean(g[:])) / np.std(g[:]) 175 | temp_data[2, :, :] = (b-np.mean(b[:])) / np.std(b[:]) 176 | r=right[:, :, 0] 177 | g=right[:, :, 1] 178 | b=right[:, :, 2] 179 | 180 | temp_data[3, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 181 | temp_data[4, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 182 | temp_data[5, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 183 | temp_data[6: 7, :, :] = width * 2 184 | temp_data[6, :, :] = disp_left[:, :] 185 | temp = temp_data[6, :, :] 186 | temp[temp < 0.1] = width * 2 * 256 187 | temp_data[6, :, :] = temp / 256. 188 | 189 | return temp_data 190 | def load_kitti2015_data(file_path, current_file): 191 | """ load current file from the list""" 192 | filename = file_path + 'image_2/' + current_file[0: len(current_file) - 1] 193 | left = Image.open(filename) 194 | filename = file_path + 'image_3/' + current_file[0: len(current_file) - 1] 195 | right = Image.open(filename) 196 | filename = file_path + 'disp_occ_0/' + current_file[0: len(current_file) - 1] 197 | 198 | disp_left = Image.open(filename) 199 | temp = np.asarray(disp_left) 200 | size = np.shape(left) 201 | 202 | height = size[0] 203 | width = size[1] 204 | temp_data = np.zeros([8, height, width], 'float32') 205 | left = np.asarray(left) 206 | right = np.asarray(right) 207 | disp_left = np.asarray(disp_left) 208 | r = left[:, :, 0] 209 | g = left[:, :, 1] 210 | b = left[:, :, 2] 211 | 212 | temp_data[0, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 213 | temp_data[1, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 214 | temp_data[2, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 215 | r = right[:, :, 0] 216 | g = right[:, :, 1] 217 | b = right[:, :, 2] 218 | 219 | temp_data[3, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 220 | temp_data[4, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 221 | temp_data[5, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 222 | temp_data[6: 7, :, :] = width * 2 223 | temp_data[6, :, :] = disp_left[:, :] 224 | temp = temp_data[6, :, :] 225 | temp[temp < 0.1] = width * 2 * 256 226 | temp_data[6, :, :] = temp / 256. 227 | 228 | return temp_data 229 | 230 | 231 | 232 | class DatasetFromList(data.Dataset): 233 | def __init__(self, data_path, file_list, crop_size=[256, 256], training=True, left_right=False, kitti=False, kitti2015=False, shift=0): 234 | super(DatasetFromList, self).__init__() 235 | #self.image_filenames = [join(image_dir, x) for x in listdir(image_dir) if is_image_file(x)] 236 | f = open(file_list, 'r') 237 | self.data_path = data_path 238 | self.file_list = f.readlines() 239 | self.training = training 240 | self.crop_height = crop_size[0] 241 | self.crop_width = crop_size[1] 242 | self.left_right = left_right 243 | self.kitti = kitti 244 | self.kitti2015 = kitti2015 245 | self.shift = shift 246 | 247 | def __getitem__(self, index): 248 | # print self.file_list[index] 249 | if self.kitti: #load kitti dataset 250 | temp_data = load_kitti_data(self.data_path, self.file_list[index]) 251 | elif self.kitti2015: #load kitti2015 dataset 252 | temp_data = load_kitti2015_data(self.data_path, self.file_list[index]) 253 | else: #load scene flow dataset 254 | temp_data = load_data(self.data_path, self.file_list[index]) 255 | # temp_data = load_data(self.data_path,self.file_list[index]) 256 | if self.training: 257 | input1, input2, target = train_transform(temp_data, self.crop_height, self.crop_width, self.left_right, self.shift) 258 | return input1, input2, target 259 | else: 260 | input1, input2, target = test_transform(temp_data, self.crop_height, self.crop_width) 261 | return input1, input2, target 262 | 263 | def __len__(self): 264 | return len(self.file_list) 265 | -------------------------------------------------------------------------------- /evaluation.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import skimage 4 | import skimage.io 5 | import skimage.transform 6 | from PIL import Image 7 | from math import log10 8 | import sys 9 | import shutil 10 | import os 11 | import re 12 | from struct import unpack 13 | import torch 14 | import torch.nn as nn 15 | import torch.nn.parallel 16 | import torch.backends.cudnn as cudnn 17 | import torch.optim as optim 18 | from torch.autograd import Variable 19 | from torch.utils.data import DataLoader 20 | from models.GANet_deep import GANet 21 | 22 | #from dataloader.data import get_test_set 23 | import numpy as np 24 | 25 | parser = argparse.ArgumentParser(description='PyTorch GANet Example') 26 | parser.add_argument('--crop_height', type=int, required=True, help="crop height") 27 | parser.add_argument('--crop_width', type=int, required=True, help="crop width") 28 | parser.add_argument('--max_disp', type=int, default=192, help="max disp") 29 | parser.add_argument('--resume', type=str, default='', help="resume from saved model") 30 | parser.add_argument('--cuda', type=bool, default=True, help='use cuda?') 31 | parser.add_argument('--kitti', type=int, default=0, help='kitti dataset? Default=False') 32 | parser.add_argument('--kitti2015', type=int, default=0, help='kitti 2015? Default=False') 33 | parser.add_argument('--data_path', type=str, required=True, help="data root") 34 | parser.add_argument('--test_list', type=str, required=True, help="training list") 35 | parser.add_argument('--save_path', type=str, default='./result/', help="location to save result") 36 | parser.add_argument('--threshold', type=float, default=3.0, help="threshold of error rates") 37 | parser.add_argument('--multi_gpu', type=int, default=0, help="multi_gpu choice") 38 | 39 | opt = parser.parse_args() 40 | 41 | 42 | print(opt) 43 | 44 | cuda = opt.cuda 45 | #cuda = True 46 | if cuda and not torch.cuda.is_available(): 47 | raise Exception("No GPU found, please run without --cuda") 48 | 49 | #print('===> Loading datasets') 50 | #test_set = get_test_set(opt.data_path, opt.test_list, [opt.crop_height, opt.crop_width], false, opt.kitti, opt.kitti2015) 51 | #testing_data_loader = DataLoader(dataset=test_set, num_workers=opt.threads, batch_size=opt.testBatchSize, shuffle=False) 52 | 53 | print('===> Building model') 54 | model = GANet(opt.max_disp) 55 | 56 | if cuda: 57 | model = torch.nn.DataParallel(model).cuda() 58 | 59 | if opt.resume: 60 | if os.path.isfile(opt.resume): 61 | print("=> loading checkpoint '{}'".format(opt.resume)) 62 | checkpoint = torch.load(opt.resume) 63 | model.load_state_dict(checkpoint['state_dict'], strict=False) 64 | 65 | else: 66 | print("=> no checkpoint found at '{}'".format(opt.resume)) 67 | 68 | 69 | def readPFM(file): 70 | with open(file, "rb") as f: 71 | # Line 1: PF=>RGB (3 channels), Pf=>Greyscale (1 channel) 72 | type = f.readline().decode('latin-1') 73 | if "PF" in type: 74 | channels = 3 75 | elif "Pf" in type: 76 | channels = 1 77 | else: 78 | sys.exit(1) 79 | # Line 2: width height 80 | line = f.readline().decode('latin-1') 81 | width, height = re.findall('\d+', line) 82 | width = int(width) 83 | height = int(height) 84 | 85 | # Line 3: +ve number means big endian, negative means little endian 86 | line = f.readline().decode('latin-1') 87 | BigEndian = True 88 | if "-" in line: 89 | BigEndian = False 90 | # Slurp all binary data 91 | samples = width * height * channels; 92 | buffer = f.read(samples * 4) 93 | # Unpack floats with appropriate endianness 94 | if BigEndian: 95 | fmt = ">" 96 | else: 97 | fmt = "<" 98 | fmt = fmt + str(samples) + "f" 99 | img = unpack(fmt, buffer) 100 | img = np.reshape(img, (height, width)) 101 | img = np.flipud(img) 102 | return img, height, width 103 | 104 | def test_transform(temp_data, crop_height, crop_width): 105 | _, h, w=np.shape(temp_data) 106 | 107 | if h <= crop_height and w <= crop_width: 108 | temp = temp_data 109 | temp_data = np.zeros([6, crop_height, crop_width], 'float32') 110 | temp_data[:, crop_height - h: crop_height, crop_width - w: crop_width] = temp 111 | else: 112 | start_x = int((w - crop_width) / 2) 113 | start_y = int((h - crop_height) / 2) 114 | temp_data = temp_data[:, start_y: start_y + crop_height, start_x: start_x + crop_width] 115 | left = np.ones([1, 3,crop_height,crop_width],'float32') 116 | left[0, :, :, :] = temp_data[0: 3, :, :] 117 | right = np.ones([1, 3, crop_height, crop_width], 'float32') 118 | right[0, :, :, :] = temp_data[3: 6, :, :] 119 | return torch.from_numpy(left).float(), torch.from_numpy(right).float(), h, w 120 | 121 | def load_data(leftname, rightname): 122 | left = Image.open(leftname) 123 | right = Image.open(rightname) 124 | size = np.shape(left) 125 | height = size[0] 126 | width = size[1] 127 | temp_data = np.zeros([6, height, width], 'float32') 128 | left = np.asarray(left) 129 | right = np.asarray(right) 130 | r = left[:, :, 0] 131 | g = left[:, :, 1] 132 | b = left[:, :, 2] 133 | temp_data[0, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 134 | temp_data[1, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 135 | temp_data[2, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 136 | r = right[:, :, 0] 137 | g = right[:, :, 1] 138 | b = right[:, :, 2] 139 | #r,g,b,_ = right.split() 140 | temp_data[3, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 141 | temp_data[4, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 142 | temp_data[5, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 143 | return temp_data 144 | 145 | def test(leftname, rightname, savename): 146 | input1, input2, height, width = test_transform(load_data(leftname, rightname), opt.crop_height, opt.crop_width) 147 | input1 = Variable(input1, requires_grad = False) 148 | input2 = Variable(input2, requires_grad = False) 149 | 150 | model.eval() 151 | if cuda: 152 | input1 = input1.cuda() 153 | input2 = input2.cuda() 154 | with torch.no_grad(): 155 | prediction = model(input1, input2) 156 | 157 | temp = prediction.cpu() 158 | temp = temp.detach().numpy() 159 | if height <= opt.crop_height and width <= opt.crop_width: 160 | temp = temp[0, opt.crop_height - height: opt.crop_height, opt.crop_width - width: opt.crop_width] 161 | else: 162 | temp = temp[0, :, :] 163 | skimage.io.imsave(savename, (temp * 256).astype('uint16')) 164 | return temp 165 | 166 | 167 | if __name__ == "__main__": 168 | file_path = opt.data_path 169 | file_list = opt.test_list 170 | f = open(file_list, 'r') 171 | filelist = f.readlines() 172 | avg_error = 0 173 | avg_rate = 0 174 | for index in range(len(filelist)): 175 | current_file = filelist[index] 176 | if opt.kitti2015: 177 | leftname = file_path + 'image_2/' + current_file[0: len(current_file) - 1] 178 | rightname = file_path + 'image_3/' + current_file[0: len(current_file) - 1] 179 | dispname = file_path + 'disp_occ_0/' + current_file[0: len(current_file) - 1] 180 | savename = opt.save_path + current_file[0: len(current_file) - 1] 181 | disp = Image.open(dispname) 182 | disp = np.asarray(disp) / 256.0 183 | elif opt.kitti: 184 | leftname = file_path + 'colored_0/' + current_file[0: len(current_file) - 1] 185 | rightname = file_path + 'colored_1/' + current_file[0: len(current_file) - 1] 186 | dispname = file_path + 'disp_occ/' + current_file[0: len(current_file) - 1] 187 | savename = opt.save_path + current_file[0: len(current_file) - 1] 188 | disp = Image.open(dispname) 189 | disp = np.asarray(disp) / 256.0 190 | 191 | else: 192 | leftname = opt.data_path + 'frames_finalpass/' + current_file[0: len(current_file) - 1] 193 | rightname = opt.data_path + 'frames_finalpass/' + current_file[0: len(current_file) - 14] + 'right/' + current_file[len(current_file) - 9:len(current_file) - 1] 194 | dispname = opt.data_path + 'disparity/' + current_file[0: len(current_file) - 4] + 'pfm' 195 | savename = opt.save_path + str(index) + '.png' 196 | disp, height, width = readPFM(dispname) 197 | 198 | prediction = test(leftname, rightname, savename) 199 | mask = np.logical_and(disp >= 0.001, disp <= opt.max_disp) 200 | 201 | error = np.mean(np.abs(prediction[mask] - disp[mask])) 202 | rate = np.sum(np.abs(prediction[mask] - disp[mask]) > opt.threshold) / np.sum(mask) 203 | avg_error += error 204 | avg_rate += rate 205 | print("===> Frame {}: ".format(index) + current_file[0:len(current_file)-1] + " ==> EPE Error: {:.4f}, Error Rate: {:.4f}".format(error, rate)) 206 | avg_error = avg_error / len(filelist) 207 | avg_rate = avg_rate / len(filelist) 208 | print("===> Total {} Frames ==> AVG EPE Error: {:.4f}, AVG Error Rate: {:.4f}".format(len(filelist), avg_error, avg_rate)) 209 | 210 | -------------------------------------------------------------------------------- /evaluation.sh: -------------------------------------------------------------------------------- 1 | CUDA_VISIBLE_DEVICES=0 python evaluation.py --crop_height=384 \ 2 | --crop_width=1248 \ 3 | --max_disp=192 \ 4 | --data_path='/ssd1/zhangfeihu/data/kitti2015/training/' \ 5 | --test_list='lists/kitti2015_train.list' \ 6 | --save_path='./result/' \ 7 | --resume='./checkpoint/kitti2015_final.pth' \ 8 | --threshold=3.0 \ 9 | --kitti2015=1 10 | # 2>&1 |tee logs/log_evaluation.txt 11 | exit 12 | CUDA_VISIBLE_DEVICES=0 python evaluation.py --crop_height=384 \ 13 | --crop_width=1248 \ 14 | --max_disp=192 \ 15 | --data_path='/ssd1/zhangfeihu/data/kitti2012/training/' \ 16 | --test_list='lists/kitti2012_train.list' \ 17 | --save_path='./result/' \ 18 | --resume='./checkpoint/kitti_final.pth' \ 19 | --threshold=3.0 \ 20 | --kitti=1 21 | # 2>&1 |tee logs/log_evaluation.txt 22 | exit 23 | CUDA_VISIBLE_DEVICES=0 python evaluation.py --crop_height=576 \ 24 | --crop_width=960 \ 25 | --max_disp=192 \ 26 | --data_path='/ssd1/zhangfeihu/data/sceneflow/' \ 27 | --test_list='lists/sceneflow_test.list' \ 28 | --save_path='./result/' \ 29 | --resume='./checkpoint/sceneflow_epoch_10.pth' \ 30 | --threshold=1.0 31 | # 2>&1 |tee logs/log_evaluation.txt 32 | exit 33 | 34 | -------------------------------------------------------------------------------- /illustration/Bicycle2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/Bicycle2.png -------------------------------------------------------------------------------- /illustration/Bicycle2_disp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/Bicycle2_disp.png -------------------------------------------------------------------------------- /illustration/Crusade.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/Crusade.png -------------------------------------------------------------------------------- /illustration/Crusade_disp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/Crusade_disp.png -------------------------------------------------------------------------------- /illustration/cityscape_029736.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/cityscape_029736.png -------------------------------------------------------------------------------- /illustration/cityscape_disp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feihuzhang/GANet/0d9d94fa9016c741eafc7044a9764ee2b16c59ab/illustration/cityscape_disp.png -------------------------------------------------------------------------------- /libs/GANet/functions/GANet.py: -------------------------------------------------------------------------------- 1 | import torch 2 | from torch.autograd import Function 3 | from ..build.lib import GANet 4 | from torch.autograd import Variable 5 | #import GANet 6 | 7 | 8 | class SgaFunction(Function): 9 | @staticmethod 10 | def forward(ctx, input, g0, g1, g2, g3): 11 | assert(input.is_contiguous() == True and g0.is_contiguous() == True and g1.is_contiguous() == True and g2.is_contiguous() == True and g3.is_contiguous() == True) 12 | with torch.cuda.device_of(input): 13 | num, channels, depth, height, width = input.size() 14 | output = input.new().resize_(num, channels, depth, height, width).zero_() 15 | temp_out = input.new().resize_(num, channels, depth, height, width).zero_() 16 | mask = input.new().resize_(num, channels, depth, height, width).zero_() 17 | GANet.sga_cuda_forward(input, g0, g1, g2, g3, temp_out, output, mask) 18 | # GANet.sga_cuda_forward(input, filters, output, radius) 19 | 20 | output = output.contiguous() 21 | ctx.save_for_backward(input, g0, g1, g2, g3, temp_out, mask) 22 | return output 23 | @staticmethod 24 | def backward(ctx, gradOutput): 25 | input, g0, g1, g2, g3, temp_out, mask = ctx.saved_tensors 26 | # print temp_out.size() 27 | # print mask.size() 28 | assert(gradOutput.is_contiguous() == True) 29 | with torch.cuda.device_of(gradOutput): 30 | num, channels, depth, height, width = input.size() 31 | # _, _, fsize, _, _ = g0.size() 32 | # print fsize 33 | gradInput = gradOutput.new().resize_(num, channels, depth, height, width).zero_() 34 | grad0 = gradOutput.new().resize_(g0.size()).zero_() 35 | grad1 = gradOutput.new().resize_(g1.size()).zero_() 36 | grad2 = gradOutput.new().resize_(g2.size()).zero_() 37 | grad3 = gradOutput.new().resize_(g3.size()).zero_() 38 | temp_grad = gradOutput.new().resize_(num, channels, depth, height, width).zero_() 39 | max_idx = gradOutput.new().resize_(num, channels, height, width).zero_() 40 | 41 | GANet.sga_cuda_backward(input, g0, g1, g2, g3, temp_out, mask, max_idx, gradOutput, temp_grad, gradInput, grad0, grad1, grad2, grad3) 42 | # GANet.lga_cuda_backward(input, filters, gradOutput, gradInput, gradFilters, radius) 43 | gradInput = gradInput.contiguous() 44 | grad0 = grad0.contiguous() 45 | grad1 = grad1.contiguous() 46 | grad2 = grad2.contiguous() 47 | grad3 = grad3.contiguous() 48 | return gradInput, grad0, grad1, grad2, grad3 49 | 50 | 51 | class Lga3d3Function(Function): 52 | @staticmethod 53 | def forward(ctx, input, filters, radius=1): 54 | ctx.radius = radius 55 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 56 | with torch.cuda.device_of(input): 57 | num, channels, depth, height, width = input.size() 58 | temp_out1 = input.new().resize_(num, channels, depth, height, width).zero_() 59 | temp_out2 = input.new().resize_(num, channels, depth, height, width).zero_() 60 | output = input.new().resize_(num, channels, depth, height, width).zero_() 61 | GANet.lga3d_cuda_forward(input, filters, temp_out1, radius) 62 | GANet.lga3d_cuda_forward(temp_out1, filters, temp_out2, radius) 63 | GANet.lga3d_cuda_forward(temp_out2, filters, output, radius) 64 | output = output.contiguous() 65 | ctx.save_for_backward(input, filters, temp_out1, temp_out2) 66 | return output 67 | @staticmethod 68 | def backward(ctx, gradOutput): 69 | input, filters, temp_out1, temp_out2 = ctx.saved_tensors 70 | assert(gradOutput.is_contiguous() == True) 71 | with torch.cuda.device_of(gradOutput): 72 | num, channels, depth, height, width = input.size() 73 | _, _, fsize, _, _ = filters.size() 74 | # gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 75 | gradFilters = gradOutput.new().resize_(num, channels, fsize, height, width).zero_() 76 | GANet.lga3d_cuda_backward(temp_out2, filters, gradOutput, temp_out2, gradFilters, ctx.radius) 77 | GANet.lga3d_cuda_backward(temp_out1, filters, temp_out2, temp_out1, gradFilters, ctx.radius) 78 | # temp_out[...] = 0 79 | GANet.lga3d_cuda_backward(input, filters, temp_out1, temp_out2, gradFilters, ctx.radius) 80 | # temp_out[...] = gradOutput[...] 81 | temp_out2 = temp_out2.contiguous() 82 | gradFilters = gradFilters.contiguous() 83 | return temp_out2, gradFilters, None 84 | class Lga3d2Function(Function): 85 | @staticmethod 86 | def forward(ctx, input, filters, radius=1): 87 | ctx.radius = radius 88 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 89 | with torch.cuda.device_of(input): 90 | num, channels, depth, height, width = input.size() 91 | temp_out = input.new().resize_(num, channels, depth, height, width).zero_() 92 | output = input.new().resize_(num, channels, depth, height, width).zero_() 93 | GANet.lga3d_cuda_forward(input, filters, temp_out, radius) 94 | GANet.lga3d_cuda_forward(temp_out, filters, output, radius) 95 | output = output.contiguous() 96 | ctx.save_for_backward(input, filters, temp_out) 97 | return output 98 | @staticmethod 99 | def backward(ctx, gradOutput): 100 | input, filters, temp_out = ctx.saved_tensors 101 | assert(gradOutput.is_contiguous() == True) 102 | with torch.cuda.device_of(gradOutput): 103 | num, channels, depth, height, width = input.size() 104 | _, _, fsize, _, _ = filters.size() 105 | # gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 106 | gradFilters = gradOutput.new().resize_(num, channels, fsize, height, width).zero_() 107 | GANet.lga3d_cuda_backward(temp_out, filters, gradOutput, temp_out, gradFilters, ctx.radius) 108 | # temp_out[...] = 0 109 | GANet.lga3d_cuda_backward(input, filters, temp_out, gradOutput, gradFilters, ctx.radius) 110 | temp_out[...] = gradOutput[...] 111 | temp_out = temp_out.contiguous() 112 | gradFilters = gradFilters.contiguous() 113 | return temp_out, gradFilters, None 114 | 115 | class Lga3dFunction(Function): 116 | @staticmethod 117 | def forward(ctx, input, filters, radius=1): 118 | ctx.radius = radius 119 | ctx.save_for_backward(input, filters) 120 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 121 | with torch.cuda.device_of(input): 122 | num, channels, depth, height, width = input.size() 123 | output = input.new().resize_(num, channels, depth, height, width).zero_() 124 | GANet.lga3d_cuda_forward(input, filters, output, radius) 125 | output = output.contiguous() 126 | return output 127 | @staticmethod 128 | def backward(ctx, gradOutput): 129 | input, filters = ctx.saved_tensors 130 | assert(gradOutput.is_contiguous() == True) 131 | with torch.cuda.device_of(gradOutput): 132 | num, channels, depth, height, width = input.size() 133 | _, _, fsize, _, _ = filters.size() 134 | gradInput = gradOutput.new().resize_(num, channels, depth, height, width).zero_() 135 | gradFilters = gradOutput.new().resize_(num, channels, fsize, height, width).zero_() 136 | GANet.lga3d_cuda_backward(input, filters, gradOutput, gradInput, gradFilters, ctx.radius) 137 | gradInput = gradInput.contiguous() 138 | gradFilters = gradFilters.contiguous() 139 | return gradInput, gradFilters, None 140 | 141 | class Lga3Function(Function): 142 | @staticmethod 143 | def forward(ctx, input, filters, radius=1): 144 | ctx.radius = radius 145 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 146 | with torch.cuda.device_of(input): 147 | num, channels, height, width = input.size() 148 | temp_out1 = input.new().resize_(num, channels, height, width).zero_() 149 | temp_out2 = input.new().resize_(num, channels, height, width).zero_() 150 | output = input.new().resize_(num, channels, height, width).zero_() 151 | GANet.lga_cuda_forward(input, filters, temp_out1, radius) 152 | GANet.lga_cuda_forward(temp_out1, filters, temp_out2, radius) 153 | GANet.lga_cuda_forward(temp_out2, filters, output, radius) 154 | output = output.contiguous() 155 | ctx.save_for_backward(input, fitlers, temp_out1, temp_out2) 156 | return output 157 | @staticmethod 158 | def backward(ctx, gradOutput): 159 | input, filters, temp_out1, temp_out2 = ctx.saved_tensors 160 | assert(gradOutput.is_contiguous() == True) 161 | with torch.cuda.device_of(gradOutput): 162 | num, channels, height, width = input.size() 163 | _, fsize, _, _ = filters.size() 164 | # gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 165 | gradFilters = gradOutput.new().resize_(num, fsize, height, width).zero_() 166 | GANet.lga_cuda_backward(temp_out2, filters, gradOutput, temp_out2, gradFilters, ctx.radius) 167 | GANet.lga_cuda_backward(temp_out1, filters, temp_out2, temp_out1, gradFilters, ctx.radius) 168 | # temp_out[...] = 0 169 | GANet.lga_cuda_backward(input, filters, temp_out1, temp_out2, gradFilters, ctx.radius) 170 | # temp_out[...] = gradOutput[...] 171 | temp_out2 = temp_out2.contiguous() 172 | gradFilters = gradFilters.contiguous() 173 | return temp_out2, gradFilters, None 174 | class Lga2Function(Function): 175 | @staticmethod 176 | def forward(ctx, input, filters, radius=1): 177 | ctx.radius = radius 178 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 179 | with torch.cuda.device_of(input): 180 | num, channels, height, width = input.size() 181 | temp_out = input.new().resize_(num, channels, height, width).zero_() 182 | output = input.new().resize_(num, channels, height, width).zero_() 183 | GANet.lga_cuda_forward(input, filters, temp_out, radius) 184 | GANet.lga_cuda_forward(temp_out, filters, output, radius) 185 | output = output.contiguous() 186 | ctx.save_for_backward(input, filters, temp_out) 187 | return output 188 | @staticmethod 189 | def backward(ctx, gradOutput): 190 | input, filters, temp_out = ctx.saved_tensors 191 | assert(gradOutput.is_contiguous() == True) 192 | with torch.cuda.device_of(gradOutput): 193 | num, channels, height, width = input.size() 194 | _, fsize, _, _ = filters.size() 195 | # gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 196 | gradFilters = gradOutput.new().resize_(num, fsize, height, width).zero_() 197 | GANet.lga_cuda_backward(temp_out, filters, gradOutput, temp_out, gradFilters, ctx.radius) 198 | # temp_out[...] = 0 199 | GANet.lga_cuda_backward(input, filters, temp_out, gradOutput, gradFilters, ctx.radius) 200 | temp_out[...] = gradOutput[...] 201 | temp_out = temp_out.contiguous() 202 | gradFilters = gradFilters.contiguous() 203 | return temp_out, gradFilters, None 204 | 205 | class Lgf2Function(Function): 206 | @staticmethod 207 | def forward(ctx, input, filters, radius=2): 208 | ctx.radius = radius 209 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 210 | with torch.cuda.device_of(input): 211 | # num, channels, depth, height, width = input.size() 212 | # temp_out = input.new().resize_(num, channels, depth, height, width).zero_() 213 | # output = input.new().resize_(num, channels, depth, height, width).zero_() 214 | temp_out = input.new().resize_(input.size()).zero_() 215 | output = input.new().resize_(input.size()).zero_() 216 | GANet.lgf_cuda_forward(input, filters, temp_out, radius) 217 | GANet.lgf_cuda_forward(temp_out, filters, output, radius) 218 | output = output.contiguous() 219 | ctx.save_for_backward(input, filters, temp_out) 220 | return output 221 | @staticmethod 222 | def backward(ctx, gradOutput): 223 | input, filters, temp_out = ctx.saved_tensors 224 | assert(gradOutput.is_contiguous() == True) 225 | with torch.cuda.device_of(gradOutput): 226 | # num, channels, depth, height, width = input.size() 227 | # _, fsize, _, _ = filters.size() 228 | # gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 229 | # gradFilters = gradOutput.new().resize_(num, fsize, height, width).zero_() 230 | gradFilters = gradOutput.new().resize_(filters.size()).zero_() 231 | GANet.lgf_cuda_backward(temp_out, filters, gradOutput, temp_out, gradFilters, ctx.radius) 232 | # temp_out[...] = 0 233 | GANet.lgf_cuda_backward(input, filters, temp_out, gradOutput, gradFilters, ctx.radius) 234 | temp_out[...] = gradOutput[...] 235 | temp_out = temp_out.contiguous() 236 | gradFilters = gradFilters.contiguous() 237 | return temp_out, gradFilters, None 238 | 239 | class LgaFunction(Function): 240 | @staticmethod 241 | def forward(ctx, input, filters): 242 | ctx.radius = radius 243 | assert(input.is_contiguous() == True and filters.is_contiguous() == True) 244 | with torch.cuda.device_of(input): 245 | num, channels, height, width = input.size() 246 | output = input.new().resize_(num, channels, height, width).zero_() 247 | GANet.lga_cuda_forward(input, filters, output, radius) 248 | output = output.contiguous() 249 | ctx.save_for_backward(input, filters) 250 | return output 251 | @staticmethod 252 | def backward(ctx, gradOutput): 253 | input, filters = ctx.saved_tensors 254 | assert(gradOutput.is_contiguous() == True) 255 | with torch.cuda.device_of(gradOutput): 256 | num, channels, height, width = input.size() 257 | _, fsize, _, _ = filters.size() 258 | gradInput = gradOutput.new().resize_(num, channels, height, width).zero_() 259 | gradFilters = gradOutput.new().resize_(num, fsize, height, width).zero_() 260 | GANet.lga_cuda_backward(input, filters, gradOutput, gradInput, gradFilters, ctx.radius) 261 | gradInput = gradInput.contiguous() 262 | gradFilters = gradFilters.contiguous() 263 | return gradInput, gradFilters, None 264 | class MyLoss2Function(Function): 265 | @staticmethod 266 | def forward(ctx, input1, input2, thresh=1, alpha=2): 267 | ctx.thresh = thresh 268 | ctx.alpha = alpha 269 | diff = input1 - input2 270 | temp=torch.abs(diff) 271 | temp[temp < thresh] = temp[temp < thresh] ** 2 / thresh 272 | tag = (temp <= thresh + alpha) & (temp >= thresh) 273 | temp[tag]=temp[tag] * 2 - (temp[tag] - thresh) ** 2 /(2.0 * alpha) - thresh 274 | temp[temp > thresh + alpha] += (alpha / 2.0) 275 | ctx.save_for_backward(diff) 276 | return torch.mean(temp) 277 | @staticmethod 278 | def backward(ctx, gradOutput): 279 | diff, = ctx.saved_tensors 280 | scale = torch.abs(diff) 281 | scale[scale > ctx.thresh + ctx.alpha] = 1 282 | tag = (scale <= ctx.thresh + ctx.alpha) & (scale >= ctx.thresh) 283 | scale[tag] = 2 - (scale[tag] - ctx.thresh) / ctx.alpha 284 | tag = scale < ctx.thresh 285 | scale[tag] = 2*scale[tag] / ctx.thresh 286 | diff[diff > 0] = 1.0 287 | diff[diff < 0] = -1.0 288 | diff = diff * scale * gradOutput / scale.numel() 289 | return diff, Variable(torch.Tensor([0])), None, None 290 | 291 | class MyLossFunction(Function): 292 | 293 | @staticmethod 294 | def forward(ctx, input1, input2, upper_thresh=5, lower_thresh=1): 295 | ctx.upper_thresh = upper_thresh 296 | ctx.lower_thresh = lower_thresh 297 | diff = input1 - input2 298 | ctx.save_for_backward(diff) 299 | return torch.mean(torch.abs(diff)) 300 | @staticmethod 301 | def backward(ctx, gradOutput): 302 | diff, = ctx.saved_tensors 303 | scale = torch.abs(diff) 304 | scale[scale > ctx.upper_thresh] = 1 305 | tag = (scale <= ctx.upper_thresh) & (scale >= ctx.lower_thresh) 306 | scale[tag] = 2 - torch.abs(scale[tag]-(ctx.upper_thresh + ctx.lower_thresh)/2.)/2. 307 | diff[diff > 0] = 1 308 | diff[diff < 0] = -1 309 | diff = diff * scale * gradOutput 310 | return diff, Variable(torch.Tensor([0])), None, None 311 | 312 | -------------------------------------------------------------------------------- /libs/GANet/functions/__init__.py: -------------------------------------------------------------------------------- 1 | from .GANet import * 2 | -------------------------------------------------------------------------------- /libs/GANet/modules/GANet.py: -------------------------------------------------------------------------------- 1 | from torch.nn.modules.module import Module 2 | import torch 3 | import numpy as np 4 | from torch.autograd import Variable 5 | from ..functions import * 6 | 7 | from ..functions.GANet import MyLossFunction 8 | from ..functions.GANet import SgaFunction 9 | from ..functions.GANet import LgaFunction 10 | from ..functions.GANet import Lga2Function 11 | from ..functions.GANet import Lga3Function 12 | from ..functions.GANet import Lga3dFunction 13 | from ..functions.GANet import Lga3d2Function 14 | from ..functions.GANet import Lga3d3Function 15 | from ..functions.GANet import MyLoss2Function 16 | 17 | 18 | class MyNormalize(Module): 19 | def __init__(self, dim): 20 | self.dim = dim 21 | super(MyNormalize, self).__init__() 22 | def forward(self, x): 23 | # assert(x.is_contiguous() == True) 24 | with torch.cuda.device_of(x): 25 | norm = torch.sum(torch.abs(x),self.dim) 26 | norm[norm <= 0] = norm[norm <= 0] - 1e-6 27 | norm[norm >= 0] = norm[norm >= 0] + 1e-6 28 | norm = torch.unsqueeze(norm, self.dim) 29 | size = np.ones(x.dim(), dtype='int') 30 | size[self.dim] = x.size()[self.dim] 31 | norm = norm.repeat(*size) 32 | x = torch.div(x, norm) 33 | return x 34 | class MyLoss2(Module): 35 | def __init__(self, thresh=1, alpha=2): 36 | super(MyLoss2, self).__init__() 37 | self.thresh = thresh 38 | self.alpha = alpha 39 | def forward(self, input1, input2): 40 | result = MyLoss2Function.apply(input1, input2, self.thresh, self.alpha) 41 | return result 42 | class MyLoss(Module): 43 | def __init__(self, upper_thresh=5, lower_thresh=1): 44 | super(MyLoss, self).__init__() 45 | self.upper_thresh = 5 46 | self.lower_thresh = 1 47 | def forward(self, input1, input2): 48 | result = MyLossFunction.apply(input1, input2, self.upper_thresh, self.lower_thresh) 49 | return result 50 | 51 | 52 | class SGA(Module): 53 | def __init__(self): 54 | super(SGA, self).__init__() 55 | 56 | def forward(self, input, g0, g1, g2, g3): 57 | result = SgaFunction.apply(input, g0, g1, g2, g3) 58 | return result 59 | 60 | 61 | 62 | class LGA3D3(Module): 63 | def __init__(self, radius=2): 64 | super(LGA3D3, self).__init__() 65 | self.radius = radius 66 | 67 | def forward(self, input1, input2): 68 | result = Lga3d3Function.apply(input1, input2, self.radius) 69 | return result 70 | class LGA3D2(Module): 71 | def __init__(self, radius=2): 72 | super(LGA3D2, self).__init__() 73 | self.radius = radius 74 | 75 | def forward(self, input1, input2): 76 | result = Lga3d2Function.apply(input1, input2, self.radius) 77 | return result 78 | class LGA3D(Module): 79 | def __init__(self, radius=2): 80 | super(LGA3D, self).__init__() 81 | self.radius = radius 82 | 83 | def forward(self, input1, input2): 84 | result = Lga3dFunction.apply(input1, input2, self.radius) 85 | return result 86 | 87 | class LGA3(Module): 88 | def __init__(self, radius=2): 89 | super(LGA3, self).__init__() 90 | self.radius = radius 91 | 92 | def forward(self, input1, input2): 93 | result = Lga3Function.apply(input1, input2, self.radius) 94 | return result 95 | class LGA2(Module): 96 | def __init__(self, radius=2): 97 | super(LGA2, self).__init__() 98 | self.radius = radius 99 | 100 | def forward(self, input1, input2): 101 | result = Lga2Function.apply(input1, input2, self.radius) 102 | return result 103 | class LGA(Module): 104 | def __init__(self, radius=2): 105 | super(LGA, self).__init__() 106 | self.radius = radius 107 | 108 | def forward(self, input1, input2): 109 | result = LgaFunction.apply(input1, input2, self.radius) 110 | return result 111 | 112 | 113 | 114 | class GetCostVolume(Module): 115 | def __init__(self, maxdisp): 116 | super(GetCostVolume, self).__init__() 117 | self.maxdisp = maxdisp + 1 118 | 119 | def forward(self, x, y): 120 | assert(x.is_contiguous() == True) 121 | with torch.cuda.device_of(x): 122 | num, channels, height, width = x.size() 123 | cost = x.new().resize_(num, channels * 2, self.maxdisp, height, width).zero_() 124 | # cost = Variable(torch.FloatTensor(x.size()[0], x.size()[1]*2, self.maxdisp, x.size()[2], x.size()[3]).zero_(), volatile= not self.training).cuda() 125 | for i in range(self.maxdisp): 126 | if i > 0 : 127 | cost[:, :x.size()[1], i, :,i:] = x[:,:,:,i:] 128 | cost[:, x.size()[1]:, i, :,i:] = y[:,:,:,:-i] 129 | else: 130 | cost[:, :x.size()[1], i, :,:] = x 131 | cost[:, x.size()[1]:, i, :,:] = y 132 | 133 | cost = cost.contiguous() 134 | return cost 135 | 136 | class DisparityRegression(Module): 137 | def __init__(self, maxdisp): 138 | super(DisparityRegression, self).__init__() 139 | self.maxdisp = maxdisp + 1 140 | # self.disp = Variable(torch.Tensor(np.reshape(np.array(range(self.maxdisp)),[1,self.maxdisp,1,1])).cuda(), requires_grad=False) 141 | 142 | def forward(self, x): 143 | assert(x.is_contiguous() == True) 144 | with torch.cuda.device_of(x): 145 | disp = Variable(torch.Tensor(np.reshape(np.array(range(self.maxdisp)),[1, self.maxdisp, 1, 1])).cuda(), requires_grad=False) 146 | disp = disp.repeat(x.size()[0], 1, x.size()[2], x.size()[3]) 147 | out = torch.sum(x * disp, 1) 148 | return out 149 | 150 | -------------------------------------------------------------------------------- /libs/GANet/modules/__init__.py: -------------------------------------------------------------------------------- 1 | from .GANet import * 2 | -------------------------------------------------------------------------------- /libs/GANet/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | from torch.utils.cpp_extension import CppExtension, BuildExtension, CUDAExtension 3 | 4 | 5 | setup( 6 | name='GANet', 7 | ext_modules=[ 8 | CUDAExtension('GANet', [ 9 | 'src/GANet_cuda.cpp', 10 | 'src/GANet_kernel.cu', 11 | ]) 12 | ], 13 | cmdclass={ 14 | 'build_ext': BuildExtension 15 | }) 16 | -------------------------------------------------------------------------------- /libs/GANet/src/GANet_cuda.cpp: -------------------------------------------------------------------------------- 1 | //#include 2 | #include 3 | #include "GANet_kernel.h" 4 | 5 | extern "C" int 6 | lga_cuda_backward (at::Tensor input, at::Tensor filters, 7 | at::Tensor gradOutput, at::Tensor gradInput, 8 | at::Tensor gradFilters, const int radius) 9 | { 10 | lga_backward (input, filters, gradOutput, gradInput, gradFilters, radius); 11 | return 1; 12 | } 13 | 14 | extern "C" int 15 | lga_cuda_forward (at::Tensor input, at::Tensor filters, at::Tensor output, 16 | const int radius) 17 | { 18 | lga_forward (input, filters, output, radius); 19 | return 1; 20 | } 21 | 22 | extern "C" int 23 | lga3d_cuda_backward (at::Tensor input, at::Tensor filters, 24 | at::Tensor gradOutput, at::Tensor gradInput, 25 | at::Tensor gradFilters, const int radius) 26 | { 27 | lga3d_backward (input, filters, gradOutput, gradInput, gradFilters, radius); 28 | return 1; 29 | } 30 | 31 | extern "C" int 32 | lga3d_cuda_forward (at::Tensor input, at::Tensor filters, at::Tensor output, 33 | const int radius) 34 | { 35 | lga3d_forward (input, filters, output, radius); 36 | return 1; 37 | } 38 | 39 | extern "C" int 40 | sga_cuda_forward (at::Tensor input, at::Tensor guidance_down, 41 | at::Tensor guidance_up, at::Tensor guidance_right, 42 | at::Tensor guidance_left, at::Tensor temp_out, 43 | at::Tensor output, at::Tensor mask) 44 | { 45 | sga_kernel_forward (input, guidance_down, guidance_up, guidance_right, 46 | guidance_left, temp_out, output, mask); 47 | return 1; 48 | } 49 | 50 | extern "C" int 51 | sga_cuda_backward (at::Tensor input, at::Tensor guidance_down, 52 | at::Tensor guidance_up, at::Tensor guidance_right, 53 | at::Tensor guidance_left, at::Tensor temp_out, 54 | at::Tensor mask, at::Tensor max_idx, at::Tensor gradOutput, 55 | at::Tensor temp_grad, at::Tensor gradInput, 56 | at::Tensor grad_down, at::Tensor grad_up, 57 | at::Tensor grad_right, at::Tensor grad_left) 58 | { 59 | sga_kernel_backward (input, guidance_down, guidance_up, guidance_right, 60 | guidance_left, temp_out, mask, max_idx, gradOutput, 61 | temp_grad, gradInput, grad_down, grad_up, grad_right, 62 | grad_left); 63 | return 1; 64 | } 65 | 66 | 67 | PYBIND11_MODULE (TORCH_EXTENSION_NAME, GANet) 68 | { 69 | GANet.def ("lga_cuda_forward", &lga_cuda_forward, "lga forward (CUDA)"); 70 | GANet.def ("lga_cuda_backward", &lga_cuda_backward, "lga backward (CUDA)"); 71 | GANet.def ("lga3d_cuda_forward", &lga3d_cuda_forward, "lga3d forward (CUDA)"); 72 | GANet.def ("lga3d_cuda_backward", &lga3d_cuda_backward, "lga3d backward (CUDA)"); 73 | GANet.def ("sga_cuda_backward", &sga_cuda_backward, "sga backward (CUDA)"); 74 | GANet.def ("sga_cuda_forward", &sga_cuda_forward, "sga forward (CUDA)"); 75 | } 76 | 77 | -------------------------------------------------------------------------------- /libs/GANet/src/GANet_cuda.h: -------------------------------------------------------------------------------- 1 | int lga_cuda_backward (at::Tensor input, at::Tensor filters, 2 | at::Tensor gradOutput, at::Tensor gradInput, 3 | at::Tensor gradFilters, const int radius); 4 | int lga_cuda_forward (at::Tensor input, at::Tensor filters, at::Tensor output, 5 | const int radius); 6 | int lga3d_cuda_backward (at::Tensor input, at::Tensor filters, 7 | at::Tensor gradOutput, at::Tensor gradInput, 8 | at::Tensor gradFilters, const int radius); 9 | int lga3d_cuda_forward (at::Tensor input, at::Tensor filters, 10 | at::Tensor output, const int radius); 11 | int sga_cuda_forward (at::Tensor input, at::Tensor guidance_down, 12 | at::Tensor guidance_up, at::Tensor guidance_right, 13 | at::Tensor guidance_left, at::Tensor temp_out, 14 | at::Tensor output, at::Tensor mask); 15 | int sga_cuda_backward (at::Tensor input, at::Tensor guidance_down, 16 | at::Tensor guidance_up, at::Tensor guidance_right, 17 | at::Tensor guidance_left, at::Tensor temp_out, 18 | at::Tensor mask, at::Tensor max_idx, 19 | at::Tensor gradOutput, at::Tensor temp_grad, 20 | at::Tensor gradInput, at::Tensor grad_down, 21 | at::Tensor grad_up, at::Tensor grad_right, 22 | at::Tensor grad_left); 23 | 24 | -------------------------------------------------------------------------------- /libs/GANet/src/GANet_kernel.h: -------------------------------------------------------------------------------- 1 | 2 | #include 3 | 4 | #ifdef __cplusplus 5 | extern "C" { 6 | #endif 7 | 8 | void sga_kernel_forward (at::Tensor input, at::Tensor guidance_down, 9 | at::Tensor guidance_up, at::Tensor guidance_right, 10 | at::Tensor guidance_left, at::Tensor temp_out, 11 | at::Tensor output, at::Tensor mask); 12 | void sga_kernel_backward (at::Tensor input, at::Tensor guidance_down, 13 | at::Tensor guidance_up, at::Tensor guidance_right, 14 | at::Tensor guidance_left, at::Tensor temp_out, 15 | at::Tensor mask, at::Tensor max_idx, 16 | at::Tensor gradOutput, at::Tensor temp_grad, 17 | at::Tensor gradInput, at::Tensor grad_down, 18 | at::Tensor grad_up, at::Tensor grad_right, 19 | at::Tensor grad_left); 20 | 21 | void lga_backward (at::Tensor input, at::Tensor filters, 22 | at::Tensor gradOutput, at::Tensor gradInput, 23 | at::Tensor gradFilters, const int radius); 24 | void lga_forward (at::Tensor input, at::Tensor filters, at::Tensor output, 25 | const int radius); 26 | 27 | void lga3d_backward (at::Tensor input, at::Tensor filters, 28 | at::Tensor gradOutput, at::Tensor gradInput, 29 | at::Tensor gradFilters, const int radius); 30 | void lga3d_forward (at::Tensor input, at::Tensor filters, at::Tensor output, 31 | const int radius); 32 | 33 | 34 | #ifdef __cplusplus 35 | } 36 | #endif 37 | -------------------------------------------------------------------------------- /libs/sync_bn/functions/__init__.py: -------------------------------------------------------------------------------- 1 | from .sync_bn import * 2 | -------------------------------------------------------------------------------- /libs/sync_bn/functions/sync_bn.py: -------------------------------------------------------------------------------- 1 | ##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 | ## Created by: Hang Zhang 3 | ## Email: zhanghang0704@gmail.com 4 | ## Copyright (c) 2018 5 | ## 6 | ## This source code is licensed under the MIT-style license found in the 7 | ## LICENSE file in the root directory of this source tree 8 | ##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 9 | 10 | """Synchronized Cross-GPU Batch Normalization functions""" 11 | import torch 12 | import torch.cuda.comm as comm 13 | from torch.autograd import Function 14 | from torch.autograd.function import once_differentiable 15 | from ..build.lib import sync_bn_gpu, sync_bn_cpu 16 | 17 | __all__ = ['moments', 'syncbatchnorm', 'inp_syncbatchnorm'] 18 | 19 | class moments(Function): 20 | @staticmethod 21 | def forward(ctx, x): 22 | if x.is_cuda: 23 | ex, ex2 = sync_bn_gpu.expectation_forward(x) 24 | else: 25 | raise NotImplemented 26 | return ex, ex2 27 | 28 | @staticmethod 29 | def backward(ctx, x, dex, dex2): 30 | if x.is_cuda: 31 | dx = sync_bn_gpu.expectation_backward(x, dex, dex2) 32 | else: 33 | raise NotImplemented 34 | return dx 35 | 36 | class syncbatchnorm_(Function): 37 | @classmethod 38 | def forward(cls, ctx, x, gamma, beta, running_mean, running_var, 39 | extra, sync=True, training=True, momentum=0.1, eps=1e-05, 40 | activation="none", slope=0.01): 41 | # save context 42 | cls._parse_extra(ctx, extra) 43 | ctx.sync = sync 44 | ctx.training = training 45 | ctx.momentum = momentum 46 | ctx.eps = eps 47 | ctx.activation = activation 48 | ctx.slope = slope 49 | assert activation == 'none' 50 | 51 | # continous inputs 52 | x = x.contiguous() 53 | gamma = gamma.contiguous() 54 | beta = beta.contiguous() 55 | 56 | if ctx.training: 57 | if x.is_cuda: 58 | _ex, _exs = sync_bn_gpu.expectation_forward(x) 59 | else: 60 | raise NotImplemented 61 | 62 | if ctx.sync: 63 | if ctx.is_master: 64 | _ex, _exs = [_ex.unsqueeze(0)], [_exs.unsqueeze(0)] 65 | for _ in range(ctx.master_queue.maxsize): 66 | _ex_w, _exs_w = ctx.master_queue.get() 67 | ctx.master_queue.task_done() 68 | _ex.append(_ex_w.unsqueeze(0)) 69 | _exs.append(_exs_w.unsqueeze(0)) 70 | 71 | _ex = comm.gather(_ex).mean(0) 72 | _exs = comm.gather(_exs).mean(0) 73 | 74 | tensors = comm.broadcast_coalesced((_ex, _exs), [_ex.get_device()] + ctx.worker_ids) 75 | for ts, queue in zip(tensors[1:], ctx.worker_queues): 76 | queue.put(ts) 77 | else: 78 | ctx.master_queue.put((_ex, _exs)) 79 | _ex, _exs = ctx.worker_queue.get() 80 | ctx.worker_queue.task_done() 81 | 82 | # Update running stats 83 | _var = _exs - _ex ** 2 84 | running_mean.mul_((1 - ctx.momentum)).add_(ctx.momentum * _ex) 85 | running_var.mul_((1 - ctx.momentum)).add_(ctx.momentum * _var) 86 | 87 | # Mark in-place modified tensors 88 | ctx.mark_dirty(running_mean, running_var) 89 | else: 90 | _ex, _var = running_mean.contiguous(), running_var.contiguous() 91 | _exs = _var + _ex ** 2 92 | 93 | # BN forward + activation 94 | if x.is_cuda: 95 | y = sync_bn_gpu.batchnorm_forward(x, _ex, _exs, gamma, beta, ctx.eps) 96 | else: 97 | y = sync_bn_cpu.batchnorm_forward(x, _ex, _exs, gamma, beta, ctx.eps) 98 | 99 | # Output 100 | ctx.save_for_backward(x, _ex, _exs, gamma, beta) 101 | return y 102 | 103 | @staticmethod 104 | @once_differentiable 105 | def backward(ctx, dz): 106 | x, _ex, _exs, gamma, beta = ctx.saved_tensors 107 | dz = dz.contiguous() 108 | 109 | # BN backward 110 | if dz.is_cuda: 111 | dx, _dex, _dexs, dgamma, dbeta = \ 112 | sync_bn_gpu.batchnorm_backward(dz, x, _ex, _exs, gamma, beta, ctx.eps) 113 | else: 114 | raise NotImplemented 115 | 116 | if ctx.training: 117 | if ctx.sync: 118 | if ctx.is_master: 119 | _dex, _dexs = [_dex.unsqueeze(0)], [_dexs.unsqueeze(0)] 120 | for _ in range(ctx.master_queue.maxsize): 121 | _dex_w, _dexs_w = ctx.master_queue.get() 122 | ctx.master_queue.task_done() 123 | _dex.append(_dex_w.unsqueeze(0)) 124 | _dexs.append(_dexs_w.unsqueeze(0)) 125 | 126 | _dex = comm.gather(_dex).mean(0) 127 | _dexs = comm.gather(_dexs).mean(0) 128 | 129 | tensors = comm.broadcast_coalesced((_dex, _dexs), [_dex.get_device()] + ctx.worker_ids) 130 | for ts, queue in zip(tensors[1:], ctx.worker_queues): 131 | queue.put(ts) 132 | else: 133 | ctx.master_queue.put((_dex, _dexs)) 134 | _dex, _dexs = ctx.worker_queue.get() 135 | ctx.worker_queue.task_done() 136 | 137 | if x.is_cuda: 138 | dx_ = sync_bn_gpu.expectation_backward(x, _dex, _dexs) 139 | else: 140 | raise NotImplemented 141 | dx = dx + dx_ 142 | 143 | return dx, dgamma, dbeta, None, None, None, None, None, None, None, None, None 144 | 145 | @staticmethod 146 | def _parse_extra(ctx, extra): 147 | ctx.is_master = extra["is_master"] 148 | if ctx.is_master: 149 | ctx.master_queue = extra["master_queue"] 150 | ctx.worker_queues = extra["worker_queues"] 151 | ctx.worker_ids = extra["worker_ids"] 152 | else: 153 | ctx.master_queue = extra["master_queue"] 154 | ctx.worker_queue = extra["worker_queue"] 155 | 156 | def _act_forward(ctx, x): 157 | if ctx.activation.lower() == "leaky_relu": 158 | if x.is_cuda: 159 | sync_bn_gpu.leaky_relu_forward(x, ctx.slope) 160 | else: 161 | raise NotImplemented 162 | else: 163 | assert ctx.activation == 'none' 164 | 165 | def _act_backward(ctx, x, dx): 166 | if ctx.activation.lower() == "leaky_relu": 167 | if x.is_cuda: 168 | sync_bn_gpu.leaky_relu_backward(x, dx, ctx.slope) 169 | else: 170 | raise NotImplemented 171 | else: 172 | assert ctx.activation == 'none' 173 | 174 | class inp_syncbatchnorm_(Function): 175 | @classmethod 176 | def forward(cls, ctx, x, gamma, beta, running_mean, running_var, 177 | extra, sync=True, training=True, momentum=0.1, eps=1e-05, 178 | activation="none", slope=0.01): 179 | # save context 180 | cls._parse_extra(ctx, extra) 181 | ctx.sync = sync 182 | ctx.training = training 183 | ctx.momentum = momentum 184 | ctx.eps = eps 185 | ctx.activation = activation 186 | ctx.slope = slope 187 | 188 | # continous inputs 189 | x = x.contiguous() 190 | gamma = gamma.contiguous() 191 | beta = beta.contiguous() 192 | 193 | if ctx.training: 194 | if x.is_cuda: 195 | _ex, _exs = sync_bn_gpu.expectation_forward(x) 196 | else: 197 | raise NotImplemented 198 | 199 | if ctx.sync: 200 | if ctx.is_master: 201 | _ex, _exs = [_ex.unsqueeze(0)], [_exs.unsqueeze(0)] 202 | for _ in range(ctx.master_queue.maxsize): 203 | _ex_w, _exs_w = ctx.master_queue.get() 204 | ctx.master_queue.task_done() 205 | _ex.append(_ex_w.unsqueeze(0)) 206 | _exs.append(_exs_w.unsqueeze(0)) 207 | 208 | _ex = comm.gather(_ex).mean(0) 209 | _exs = comm.gather(_exs).mean(0) 210 | 211 | tensors = comm.broadcast_coalesced((_ex, _exs), [_ex.get_device()] + ctx.worker_ids) 212 | for ts, queue in zip(tensors[1:], ctx.worker_queues): 213 | queue.put(ts) 214 | else: 215 | ctx.master_queue.put((_ex, _exs)) 216 | _ex, _exs = ctx.worker_queue.get() 217 | ctx.worker_queue.task_done() 218 | 219 | # Update running stats 220 | _var = _exs - _ex ** 2 221 | running_mean.mul_((1 - ctx.momentum)).add_(ctx.momentum * _ex) 222 | running_var.mul_((1 - ctx.momentum)).add_(ctx.momentum * _var) 223 | 224 | # Mark in-place modified tensors 225 | ctx.mark_dirty(x, running_mean, running_var) 226 | else: 227 | _ex, _var = running_mean.contiguous(), running_var.contiguous() 228 | _exs = _var + _ex ** 2 229 | ctx.mark_dirty(x) 230 | 231 | # BN forward + activation 232 | if x.is_cuda: 233 | sync_bn_gpu.batchnorm_inp_forward(x, _ex, _exs, gamma, beta, ctx.eps) 234 | else: 235 | raise NotImplemented 236 | 237 | _act_forward(ctx, x) 238 | 239 | # Output 240 | ctx.save_for_backward(x, _ex, _exs, gamma, beta) 241 | return x 242 | 243 | @staticmethod 244 | @once_differentiable 245 | def backward(ctx, dz): 246 | z, _ex, _exs, gamma, beta = ctx.saved_tensors 247 | dz = dz.contiguous() 248 | 249 | # Undo activation 250 | _act_backward(ctx, z, dz) 251 | 252 | # BN backward 253 | if dz.is_cuda: 254 | dx, _dex, _dexs, dgamma, dbeta = \ 255 | sync_bn_gpu.batchnorm_inp_backward(dz, z, _ex, _exs, gamma, beta, ctx.eps) 256 | else: 257 | raise NotImplemented 258 | 259 | if ctx.training: 260 | if ctx.sync: 261 | if ctx.is_master: 262 | _dex, _dexs = [_dex.unsqueeze(0)], [_dexs.unsqueeze(0)] 263 | for _ in range(ctx.master_queue.maxsize): 264 | _dex_w, _dexs_w = ctx.master_queue.get() 265 | ctx.master_queue.task_done() 266 | _dex.append(_dex_w.unsqueeze(0)) 267 | _dexs.append(_dexs_w.unsqueeze(0)) 268 | 269 | _dex = comm.gather(_dex).mean(0) 270 | _dexs = comm.gather(_dexs).mean(0) 271 | 272 | tensors = comm.broadcast_coalesced((_dex, _dexs), [_dex.get_device()] + ctx.worker_ids) 273 | for ts, queue in zip(tensors[1:], ctx.worker_queues): 274 | queue.put(ts) 275 | else: 276 | ctx.master_queue.put((_dex, _dexs)) 277 | _dex, _dexs = ctx.worker_queue.get() 278 | ctx.worker_queue.task_done() 279 | 280 | if z.is_cuda: 281 | sync_bn_gpu.expectation_inp_backward(dx, z, _dex, _dexs, _ex, _exs, gamma, beta, ctx.eps) 282 | else: 283 | raise NotImplemented 284 | 285 | return dx, dgamma, dbeta, None, None, None, None, None, None, None, None, None 286 | 287 | @staticmethod 288 | def _parse_extra(ctx, extra): 289 | ctx.is_master = extra["is_master"] 290 | if ctx.is_master: 291 | ctx.master_queue = extra["master_queue"] 292 | ctx.worker_queues = extra["worker_queues"] 293 | ctx.worker_ids = extra["worker_ids"] 294 | else: 295 | ctx.master_queue = extra["master_queue"] 296 | ctx.worker_queue = extra["worker_queue"] 297 | 298 | syncbatchnorm = syncbatchnorm_.apply 299 | inp_syncbatchnorm = inp_syncbatchnorm_.apply 300 | -------------------------------------------------------------------------------- /libs/sync_bn/modules/__init__.py: -------------------------------------------------------------------------------- 1 | from .sync_bn import * 2 | -------------------------------------------------------------------------------- /libs/sync_bn/modules/sync_bn.py: -------------------------------------------------------------------------------- 1 | ##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 | ## Created by: Hang Zhang 3 | ## ECE Department, Rutgers University 4 | ## Email: zhang.hang@rutgers.edu 5 | ## Copyright (c) 2017 6 | ## 7 | ## This source code is licensed under the MIT-style license found in the 8 | ## LICENSE file in the root directory of this source tree 9 | ##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 10 | 11 | """Synchronized Cross-GPU Batch Normalization Module""" 12 | import warnings 13 | try: 14 | from queue import Queue 15 | except ImportError: 16 | from Queue import Queue 17 | 18 | import torch 19 | from torch.nn.modules.batchnorm import _BatchNorm 20 | 21 | from ..functions import * 22 | 23 | 24 | __all__ = ['SyncBatchNorm', 'BatchNorm1d', 'BatchNorm2d', 'BatchNorm3d'] 25 | 26 | 27 | class SyncBatchNorm(_BatchNorm): 28 | r"""Cross-GPU Synchronized Batch normalization (SyncBN) 29 | 30 | Standard BN [1]_ implementation only normalize the data within each device (GPU). 31 | SyncBN normalizes the input within the whole mini-batch. 32 | We follow the sync-onece implmentation described in the paper [2]_ . 33 | Please see the design idea in the `notes <./notes/syncbn.html>`_. 34 | 35 | .. math:: 36 | 37 | y = \frac{x - mean[x]}{ \sqrt{Var[x] + \epsilon}} * gamma + beta 38 | 39 | The mean and standard-deviation are calculated per-channel over 40 | the mini-batches and gamma and beta are learnable parameter vectors 41 | of size C (where C is the input size). 42 | 43 | During training, this layer keeps a running estimate of its computed mean 44 | and variance. The running sum is kept with a default momentum of 0.1. 45 | 46 | During evaluation, this running mean/variance is used for normalization. 47 | 48 | Because the BatchNorm is done over the `C` dimension, computing statistics 49 | on `(N, H, W)` slices, it's common terminology to call this Spatial BatchNorm 50 | 51 | Args: 52 | num_features: num_features from an expected input of 53 | size batch_size x num_features x height x width 54 | eps: a value added to the denominator for numerical stability. 55 | Default: 1e-5 56 | momentum: the value used for the running_mean and running_var 57 | computation. Default: 0.1 58 | sync: a boolean value that when set to ``True``, synchronize across 59 | different gpus. Default: ``True`` 60 | activation : str 61 | Name of the activation functions, one of: `leaky_relu` or `none`. 62 | slope : float 63 | Negative slope for the `leaky_relu` activation. 64 | 65 | Shape: 66 | - Input: :math:`(N, C, H, W)` 67 | - Output: :math:`(N, C, H, W)` (same shape as input) 68 | 69 | Reference: 70 | .. [1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." *ICML 2015* 71 | .. [2] Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal. "Context Encoding for Semantic Segmentation." *CVPR 2018* 72 | 73 | Examples: 74 | >>> m = SyncBatchNorm(100) 75 | >>> net = torch.nn.DataParallel(m) 76 | >>> output = net(input) 77 | """ 78 | 79 | def __init__(self, num_features, eps=1e-5, momentum=0.1, sync=True, activation="none", slope=0.01, 80 | inplace=True): 81 | super(SyncBatchNorm, self).__init__(num_features, eps=eps, momentum=momentum, affine=True) 82 | self.activation = activation 83 | self.inplace = False if activation == 'none' else inplace 84 | #self.inplace = inplace 85 | self.slope = slope 86 | self.devices = list(range(torch.cuda.device_count())) 87 | self.sync = sync if len(self.devices) > 1 else False 88 | # Initialize queues 89 | self.worker_ids = self.devices[1:] 90 | self.master_queue = Queue(len(self.worker_ids)) 91 | self.worker_queues = [Queue(1) for _ in self.worker_ids] 92 | # running_exs 93 | #self.register_buffer('running_exs', torch.ones(num_features)) 94 | 95 | def forward(self, x): 96 | # Resize the input to (B, C, -1). 97 | input_shape = x.size() 98 | x = x.view(input_shape[0], self.num_features, -1) 99 | if x.get_device() == self.devices[0]: 100 | # Master mode 101 | extra = { 102 | "is_master": True, 103 | "master_queue": self.master_queue, 104 | "worker_queues": self.worker_queues, 105 | "worker_ids": self.worker_ids 106 | } 107 | else: 108 | # Worker mode 109 | extra = { 110 | "is_master": False, 111 | "master_queue": self.master_queue, 112 | "worker_queue": self.worker_queues[self.worker_ids.index(x.get_device())] 113 | } 114 | if self.inplace: 115 | return inp_syncbatchnorm(x, self.weight, self.bias, self.running_mean, self.running_var, 116 | extra, self.sync, self.training, self.momentum, self.eps, 117 | self.activation, self.slope).view(input_shape) 118 | else: 119 | return syncbatchnorm(x, self.weight, self.bias, self.running_mean, self.running_var, 120 | extra, self.sync, self.training, self.momentum, self.eps, 121 | self.activation, self.slope).view(input_shape) 122 | 123 | def extra_repr(self): 124 | if self.activation == 'none': 125 | return 'sync={}'.format(self.sync) 126 | else: 127 | return 'sync={}, act={}, slope={}, inplace={}'.format( 128 | self.sync, self.activation, self.slope, self.inplace 129 | ) 130 | 131 | class BatchNorm1d(SyncBatchNorm): 132 | r""" 133 | .. warning:: 134 | BatchNorm1d is deprecated in favor of :class:`encoding.nn.SyncBatchNorm`. 135 | """ 136 | def __init__(self, *args, **kwargs): 137 | super(BatchNorm1d, self).__init__(*args, **kwargs) 138 | 139 | class BatchNorm2d(SyncBatchNorm): 140 | r""" 141 | .. warning:: 142 | BatchNorm2d is deprecated in favor of :class:`encoding.nn.SyncBatchNorm`. 143 | """ 144 | def __init__(self, *args, **kwargs): 145 | super(BatchNorm2d, self).__init__(*args, **kwargs) 146 | 147 | class BatchNorm3d(SyncBatchNorm): 148 | r""" 149 | .. warning:: 150 | BatchNorm3d is deprecated in favor of :class:`encoding.nn.SyncBatchNorm`. 151 | """ 152 | def __init__(self, *args, **kwargs): 153 | super(BatchNorm3d, self).__init__(*args, **kwargs) 154 | -------------------------------------------------------------------------------- /libs/sync_bn/setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | from torch.utils.cpp_extension import CppExtension, BuildExtension, CUDAExtension 3 | 4 | setup( 5 | name='sync_bn_cpu', 6 | ext_modules=[ 7 | CppExtension('sync_bn_cpu', [ 8 | 'src/cpu/operator.cpp', 9 | 'src/cpu/sync_bn.cpp', 10 | ]) 11 | ], 12 | cmdclass={ 13 | 'build_ext': BuildExtension 14 | }) 15 | 16 | 17 | setup( 18 | name='sync_bn_gpu', 19 | ext_modules=[ 20 | CUDAExtension('sync_bn_gpu', [ 21 | 'src/gpu/operator.cpp', 22 | 'src/gpu/sync_bn_cuda.cu', 23 | ]) 24 | ], 25 | cmdclass={ 26 | 'build_ext': BuildExtension 27 | }) 28 | -------------------------------------------------------------------------------- /libs/sync_bn/src/__init__.py: -------------------------------------------------------------------------------- 1 | import os 2 | import torch 3 | from torch.utils.cpp_extension import load 4 | 5 | cwd = os.path.dirname(os.path.realpath(__file__)) 6 | cpu_path = os.path.join(cwd, 'cpu') 7 | gpu_path = os.path.join(cwd, 'gpu') 8 | 9 | cpu = load('sync_bn_cpu', [ 10 | os.path.join(cpu_path, 'operator.cpp'), 11 | os.path.join(cpu_path, 'sync_bn.cpp'), 12 | ], build_directory=cpu_path, verbose=False) 13 | 14 | if torch.cuda.is_available(): 15 | gpu = load('sync_bn_gpu', [ 16 | os.path.join(gpu_path, 'operator.cpp'), 17 | os.path.join(gpu_path, 'sync_bn_cuda.cu'), 18 | ], build_directory=gpu_path, verbose=False) 19 | -------------------------------------------------------------------------------- /libs/sync_bn/src/cpu/operator.cpp: -------------------------------------------------------------------------------- 1 | #include "operator.h" 2 | 3 | PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { 4 | m.def("batchnorm_forward", &BatchNorm_Forward_CPU, "BatchNorm forward (CPU)"); 5 | m.def("batchnorm_backward", &BatchNorm_Backward_CPU, "BatchNorm backward (CPU)"); 6 | m.def("sumsquare_forward", &Sum_Square_Forward_CPU, "SumSqu forward (CPU)"); 7 | m.def("sumsquare_backward", &Sum_Square_Backward_CPU, "SumSqu backward (CPU)"); 8 | } 9 | -------------------------------------------------------------------------------- /libs/sync_bn/src/cpu/operator.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | at::Tensor BatchNorm_Forward_CPU( 5 | const at::Tensor input_, 6 | const at::Tensor mean_, 7 | const at::Tensor std_, 8 | const at::Tensor gamma_, 9 | const at::Tensor beta_); 10 | 11 | std::vector BatchNorm_Backward_CPU( 12 | const at::Tensor gradoutput_, 13 | const at::Tensor input_, 14 | const at::Tensor mean_, 15 | const at::Tensor std_, 16 | const at::Tensor gamma_, 17 | const at::Tensor beta_, 18 | bool train); 19 | 20 | std::vector Sum_Square_Forward_CPU( 21 | const at::Tensor input_); 22 | 23 | at::Tensor Sum_Square_Backward_CPU( 24 | const at::Tensor input_, 25 | const at::Tensor gradSum_, 26 | const at::Tensor gradSquare_); 27 | -------------------------------------------------------------------------------- /libs/sync_bn/src/cpu/sync_bn.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | at::Tensor broadcast_to(at::Tensor v, at::Tensor x) { 6 | if (x.ndimension() == 2) { 7 | return v; 8 | } else { 9 | std::vector broadcast_size = {1, -1}; 10 | for (int64_t i = 2; i < x.ndimension(); ++i) 11 | broadcast_size.push_back(1); 12 | 13 | return v.view(broadcast_size); 14 | } 15 | } 16 | 17 | at::Tensor BatchNorm_Forward_CPU( 18 | const at::Tensor input, 19 | const at::Tensor mean, 20 | const at::Tensor std, 21 | const at::Tensor gamma, 22 | const at::Tensor beta) { 23 | auto output = (input - broadcast_to(mean, input)) / broadcast_to(std, input); 24 | output = output * broadcast_to(gamma, input) + broadcast_to(beta, input); 25 | return output; 26 | } 27 | 28 | // Not implementing CPU backward for now 29 | std::vector BatchNorm_Backward_CPU( 30 | const at::Tensor gradoutput, 31 | const at::Tensor input, 32 | const at::Tensor mean, 33 | const at::Tensor std, 34 | const at::Tensor gamma, 35 | const at::Tensor beta, 36 | bool train) { 37 | /* outputs*/ 38 | at::Tensor gradinput = at::zeros_like(input); 39 | at::Tensor gradgamma = at::zeros_like(gamma); 40 | at::Tensor gradbeta = at::zeros_like(beta); 41 | at::Tensor gradMean = at::zeros_like(mean); 42 | at::Tensor gradStd = at::zeros_like(std); 43 | return {gradinput, gradMean, gradStd, gradgamma, gradbeta}; 44 | } 45 | 46 | std::vector Sum_Square_Forward_CPU( 47 | const at::Tensor input) { 48 | /* outputs */ 49 | at::Tensor sum = torch::zeros({input.size(1)}, input.options()); 50 | at::Tensor square = torch::zeros({input.size(1)}, input.options()); 51 | return {sum, square}; 52 | } 53 | 54 | at::Tensor Sum_Square_Backward_CPU( 55 | const at::Tensor input, 56 | const at::Tensor gradSum, 57 | const at::Tensor gradSquare) { 58 | /* outputs */ 59 | at::Tensor gradInput = at::zeros_like(input); 60 | return gradInput; 61 | } 62 | -------------------------------------------------------------------------------- /libs/sync_bn/src/gpu/common.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | static const unsigned WARP_SIZE = 32; 5 | 6 | // The maximum number of threads in a block 7 | static const unsigned MAX_BLOCK_SIZE = 512U; 8 | 9 | template 10 | struct ScalarConvert { 11 | static __host__ __device__ __forceinline__ Out to(const In v) { return (Out) v; } 12 | }; 13 | 14 | // Number of threads in a block given an input size up to MAX_BLOCK_SIZE 15 | static int getNumThreads(int nElem) { 16 | int threadSizes[5] = { 32, 64, 128, 256, MAX_BLOCK_SIZE }; 17 | for (int i = 0; i != 5; ++i) { 18 | if (nElem <= threadSizes[i]) { 19 | return threadSizes[i]; 20 | } 21 | } 22 | return MAX_BLOCK_SIZE; 23 | } 24 | 25 | // Returns the index of the most significant 1 bit in `val`. 26 | __device__ __forceinline__ int getMSB(int val) { 27 | return 31 - __clz(val); 28 | } 29 | 30 | template 31 | __device__ __forceinline__ T WARP_SHFL_XOR(T value, int laneMask, int width = warpSize, unsigned int mask = 0xffffffff) 32 | { 33 | #if CUDA_VERSION >= 9000 34 | return __shfl_xor_sync(mask, value, laneMask, width); 35 | #else 36 | return __shfl_xor(value, laneMask, width); 37 | #endif 38 | } 39 | 40 | // Sum across all threads within a warp 41 | template 42 | static __device__ __forceinline__ T warpSum(T val) { 43 | #if __CUDA_ARCH__ >= 300 44 | for (int i = 0; i < getMSB(WARP_SIZE); ++i) { 45 | val += WARP_SHFL_XOR(val, 1 << i, WARP_SIZE); 46 | } 47 | #else 48 | __shared__ T values[MAX_BLOCK_SIZE]; 49 | values[threadIdx.x] = val; 50 | __threadfence_block(); 51 | const int base = (threadIdx.x / WARP_SIZE) * WARP_SIZE; 52 | for (int i = 1; i < WARP_SIZE; i++) { 53 | val += values[base + ((i + threadIdx.x) % WARP_SIZE)]; 54 | } 55 | #endif 56 | return val; 57 | } 58 | 59 | template 60 | struct Float2 { 61 | Acctype v1, v2; 62 | __device__ Float2() {} 63 | __device__ Float2(DType v1, DType v2) : v1(ScalarConvert::to(v1)), v2(ScalarConvert::to(v2)) {} 64 | __device__ Float2(DType v) : v1(ScalarConvert::to(v)), v2(ScalarConvert::to(v)) {} 65 | __device__ Float2(int v) : v1(ScalarConvert::to(v)), v2(ScalarConvert::to(v)) {} 66 | __device__ Float2& operator+=(const Float2& a) { 67 | v1 += a.v1; 68 | v2 += a.v2; 69 | return *this; 70 | } 71 | }; 72 | 73 | template 74 | static __device__ __forceinline__ Float2 warpSum(Float2 value) { 75 | value.v1 = warpSum(value.v1); 76 | value.v2 = warpSum(value.v2); 77 | return value; 78 | } 79 | 80 | template 81 | __device__ T reduceD( 82 | Op op, int b, int i, int k, int D) { 83 | T sum = 0; 84 | for (int x = threadIdx.x; x < D; x += blockDim.x) { 85 | sum += op(b,i,k,x); 86 | } 87 | // sum over NumThreads within a warp 88 | sum = warpSum(sum); 89 | 90 | // 'transpose', and reduce within warp again 91 | __shared__ T shared[32]; 92 | 93 | __syncthreads(); 94 | if (threadIdx.x % WARP_SIZE == 0) { 95 | if (threadIdx.x / WARP_SIZE < 32) { 96 | shared[threadIdx.x / WARP_SIZE] = sum; 97 | } 98 | } 99 | if (threadIdx.x >= blockDim.x / WARP_SIZE && threadIdx.x < WARP_SIZE) { 100 | // zero out the other entries in shared 101 | shared[threadIdx.x] = (T) 0; 102 | } 103 | __syncthreads(); 104 | if (threadIdx.x / WARP_SIZE == 0) { 105 | sum = warpSum(shared[threadIdx.x]); 106 | if (threadIdx.x == 0) { 107 | shared[0] = sum; 108 | } 109 | } 110 | __syncthreads(); 111 | 112 | // Everyone picks it up, should be broadcast into the whole gradInput 113 | return shared[0]; 114 | } 115 | 116 | template 117 | __device__ T reduceN( 118 | Op op, int b, int k, int d, int N) { 119 | T sum = 0; 120 | for (int x = threadIdx.x; x < N; x += blockDim.x) { 121 | sum += op(b,x,k,d); 122 | } 123 | // sum over NumThreads within a warp 124 | sum = warpSum(sum); 125 | 126 | // 'transpose', and reduce within warp again 127 | __shared__ T shared[32]; 128 | 129 | __syncthreads(); 130 | if (threadIdx.x % WARP_SIZE == 0) { 131 | if (threadIdx.x / WARP_SIZE < 32) { 132 | shared[threadIdx.x / WARP_SIZE] = sum; 133 | } 134 | } 135 | if (threadIdx.x >= blockDim.x / WARP_SIZE && threadIdx.x < WARP_SIZE) { 136 | // zero out the other entries in shared 137 | shared[threadIdx.x] = (T) 0; 138 | } 139 | __syncthreads(); 140 | if (threadIdx.x / WARP_SIZE == 0) { 141 | sum = warpSum(shared[threadIdx.x]); 142 | if (threadIdx.x == 0) { 143 | shared[0] = sum; 144 | } 145 | } 146 | __syncthreads(); 147 | 148 | // Everyone picks it up, should be broadcast into the whole gradInput 149 | return shared[0]; 150 | } 151 | 152 | template 153 | __device__ T reduceK( 154 | Op op, int b, int i, int d, int K) { 155 | T sum = 0; 156 | for (int x = threadIdx.x; x < K; x += blockDim.x) { 157 | sum += op(b,i,x,d); 158 | } 159 | // sum over NumThreads within a warp 160 | sum = warpSum(sum); 161 | 162 | // 'transpose', and reduce within warp again 163 | __shared__ T shared[32]; 164 | 165 | __syncthreads(); 166 | if (threadIdx.x % WARP_SIZE == 0) { 167 | if (threadIdx.x / WARP_SIZE < 32) { 168 | shared[threadIdx.x / WARP_SIZE] = sum; 169 | } 170 | } 171 | if (threadIdx.x >= blockDim.x / WARP_SIZE && threadIdx.x < WARP_SIZE) { 172 | // zero out the other entries in shared 173 | shared[threadIdx.x] = (T) 0; 174 | } 175 | __syncthreads(); 176 | if (threadIdx.x / WARP_SIZE == 0) { 177 | sum = warpSum(shared[threadIdx.x]); 178 | if (threadIdx.x == 0) { 179 | shared[0] = sum; 180 | } 181 | } 182 | __syncthreads(); 183 | 184 | // Everyone picks it up, should be broadcast into the whole gradInput 185 | return shared[0]; 186 | } 187 | 188 | template 189 | __device__ T reduceBN( 190 | Op op, 191 | int k, int d, int B, int N) { 192 | T sum = 0; 193 | for (int batch = 0; batch < B; ++batch) { 194 | for (int x = threadIdx.x; x < N; x += blockDim.x) { 195 | sum += op(batch,x,k,d); 196 | } 197 | } 198 | // sum over NumThreads within a warp 199 | sum = warpSum(sum); 200 | // 'transpose', and reduce within warp again 201 | __shared__ T shared[32]; 202 | 203 | __syncthreads(); 204 | if (threadIdx.x % WARP_SIZE == 0) { 205 | if (threadIdx.x / WARP_SIZE < 32) { 206 | shared[threadIdx.x / WARP_SIZE] = sum; 207 | } 208 | } 209 | if (threadIdx.x >= blockDim.x / WARP_SIZE && threadIdx.x < WARP_SIZE) { 210 | // zero out the other entries in shared 211 | shared[threadIdx.x] = (T) 0; 212 | } 213 | __syncthreads(); 214 | if (threadIdx.x / WARP_SIZE == 0) { 215 | sum = warpSum(shared[threadIdx.x]); 216 | if (threadIdx.x == 0) { 217 | shared[0] = sum; 218 | } 219 | } 220 | __syncthreads(); 221 | 222 | // Everyone picks it up, should be broadcast into the whole gradInput 223 | return shared[0]; 224 | } 225 | -------------------------------------------------------------------------------- /libs/sync_bn/src/gpu/device_tensor.h: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | template 4 | struct DeviceTensor { 5 | public: 6 | inline __device__ __host__ DeviceTensor(DType *p, const int *size) 7 | : dptr_(p) { 8 | for (int i = 0; i < Dim; ++i) { 9 | size_[i] = size ? size[i] : 0; 10 | } 11 | } 12 | 13 | inline __device__ __host__ unsigned getSize(const int i) const { 14 | assert(i < Dim); 15 | return size_[i]; 16 | } 17 | 18 | inline __device__ __host__ int numElements() const { 19 | int n = 1; 20 | for (int i = 0; i < Dim; ++i) { 21 | n *= size_[i]; 22 | } 23 | return n; 24 | } 25 | 26 | inline __device__ __host__ DeviceTensor select(const size_t x) const { 27 | assert(Dim > 1); 28 | int offset = x; 29 | for (int i = 1; i < Dim; ++i) { 30 | offset *= size_[i]; 31 | } 32 | DeviceTensor tensor(dptr_ + offset, nullptr); 33 | for (int i = 0; i < Dim - 1; ++i) { 34 | tensor.size_[i] = this->size_[i+1]; 35 | } 36 | return tensor; 37 | } 38 | 39 | inline __device__ __host__ DeviceTensor operator[](const size_t x) const { 40 | assert(Dim > 1); 41 | int offset = x; 42 | for (int i = 1; i < Dim; ++i) { 43 | offset *= size_[i]; 44 | } 45 | DeviceTensor tensor(dptr_ + offset, nullptr); 46 | for (int i = 0; i < Dim - 1; ++i) { 47 | tensor.size_[i] = this->size_[i+1]; 48 | } 49 | return tensor; 50 | } 51 | 52 | inline __device__ __host__ size_t InnerSize() const { 53 | assert(Dim >= 3); 54 | size_t sz = 1; 55 | for (size_t i = 2; i < Dim; ++i) { 56 | sz *= size_[i]; 57 | } 58 | return sz; 59 | } 60 | 61 | inline __device__ __host__ size_t ChannelCount() const { 62 | assert(Dim >= 3); 63 | return size_[1]; 64 | } 65 | 66 | inline __device__ __host__ DType* data_ptr() const { 67 | return dptr_; 68 | } 69 | 70 | DType *dptr_; 71 | int size_[Dim]; 72 | }; 73 | 74 | template 75 | struct DeviceTensor { 76 | inline __device__ __host__ DeviceTensor(DType *p, const int *size) 77 | : dptr_(p) { 78 | size_[0] = size ? size[0] : 0; 79 | } 80 | 81 | inline __device__ __host__ unsigned getSize(const int i) const { 82 | assert(i == 0); 83 | return size_[0]; 84 | } 85 | 86 | inline __device__ __host__ int numElements() const { 87 | return size_[0]; 88 | } 89 | 90 | inline __device__ __host__ DType &operator[](const size_t x) const { 91 | return *(dptr_ + x); 92 | } 93 | 94 | inline __device__ __host__ DType* data_ptr() const { 95 | return dptr_; 96 | } 97 | 98 | DType *dptr_; 99 | int size_[1]; 100 | }; 101 | 102 | template 103 | static DeviceTensor devicetensor(const at::Tensor &blob) { 104 | DType *data = blob.data(); 105 | DeviceTensor tensor(data, nullptr); 106 | for (int i = 0; i < Dim; ++i) { 107 | tensor.size_[i] = blob.size(i); 108 | } 109 | return tensor; 110 | } 111 | -------------------------------------------------------------------------------- /libs/sync_bn/src/gpu/operator.cpp: -------------------------------------------------------------------------------- 1 | #include "operator.h" 2 | 3 | PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { 4 | m.def("batchnorm_forward", &BatchNorm_Forward_CUDA, "BatchNorm forward (CUDA)"); 5 | m.def("batchnorm_inp_forward", &BatchNorm_Forward_Inp_CUDA, "BatchNorm forward (CUDA)"); 6 | m.def("batchnorm_backward", &BatchNorm_Backward_CUDA, "BatchNorm backward (CUDA)"); 7 | m.def("batchnorm_inp_backward", &BatchNorm_Inp_Backward_CUDA, "BatchNorm backward (CUDA)"); 8 | m.def("expectation_forward", &Expectation_Forward_CUDA, "Expectation forward (CUDA)"); 9 | m.def("expectation_backward", &Expectation_Backward_CUDA, "Expectation backward (CUDA)"); 10 | m.def("expectation_inp_backward", &Expectation_Inp_Backward_CUDA, 11 | "Inplace Expectation backward (CUDA)"); 12 | } 13 | -------------------------------------------------------------------------------- /libs/sync_bn/src/gpu/operator.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | at::Tensor BatchNorm_Forward_CUDA( 6 | const at::Tensor input_, 7 | const at::Tensor mean_, 8 | const at::Tensor std_, 9 | const at::Tensor gamma_, 10 | const at::Tensor beta_, 11 | float eps); 12 | 13 | at::Tensor BatchNorm_Forward_Inp_CUDA( 14 | const at::Tensor input_, 15 | const at::Tensor ex_, 16 | const at::Tensor exs_, 17 | const at::Tensor gamma_, 18 | const at::Tensor beta_, 19 | float eps); 20 | 21 | std::vector BatchNorm_Backward_CUDA( 22 | const at::Tensor gradoutput_, 23 | const at::Tensor input_, 24 | const at::Tensor ex_, 25 | const at::Tensor exs_, 26 | const at::Tensor gamma_, 27 | const at::Tensor beta_, 28 | float eps); 29 | 30 | std::vector BatchNorm_Inp_Backward_CUDA( 31 | const at::Tensor gradoutput_, 32 | const at::Tensor output_, 33 | const at::Tensor ex_, 34 | const at::Tensor exs_, 35 | const at::Tensor gamma_, 36 | const at::Tensor beta_, 37 | float eps); 38 | 39 | std::vector Expectation_Forward_CUDA( 40 | const at::Tensor input_); 41 | 42 | at::Tensor Expectation_Backward_CUDA( 43 | const at::Tensor input_, 44 | const at::Tensor gradEx_, 45 | const at::Tensor gradExs_); 46 | 47 | at::Tensor Expectation_Inp_Backward_CUDA( 48 | const at::Tensor gradInput_, 49 | const at::Tensor output_, 50 | const at::Tensor gradEx_, 51 | const at::Tensor gradExs_, 52 | const at::Tensor ex_, 53 | const at::Tensor exs_, 54 | const at::Tensor gamma_, 55 | const at::Tensor beta_, 56 | float eps); 57 | -------------------------------------------------------------------------------- /libs/sync_bn/src/gpu/sync_bn_cuda.cu: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | #include "common.h" 7 | #include "device_tensor.h" 8 | 9 | namespace { 10 | 11 | template 12 | struct GradOp { 13 | __device__ GradOp(Acctype m, const DeviceTensor3 i, const DeviceTensor3 g) 14 | : beta(m), output(i), gradOutput(g) {} 15 | __device__ __forceinline__ Float2 operator()(int batch, int plane, int n) { 16 | DType g = gradOutput[batch][plane][n]; 17 | DType c = ScalarConvert::to(output[batch][plane][n] - beta); 18 | return Float2(g, g * c); 19 | } 20 | const Acctype beta; 21 | const DeviceTensor3 output; 22 | const DeviceTensor3 gradOutput; 23 | }; 24 | 25 | template 26 | struct SumOp { 27 | __device__ SumOp(DeviceTensor i) : input(i){} 28 | __device__ __forceinline__ Float2 operator()(int batch, int plane, int n) { 29 | DType g = input[batch][plane][n]; 30 | return Float2(g, g * g); 31 | } 32 | DType mean; 33 | DeviceTensor input; 34 | }; 35 | 36 | // Sum across (batch, x/y/z) applying Op() pointwise 37 | template 38 | __device__ T reduce(Op op, DeviceTensor3 tensor, int plane) { 39 | T sum = (T)0; 40 | for (int batch = 0; batch < tensor.getSize(0); ++batch) { 41 | for (int x = threadIdx.x; x < tensor.getSize(2); x += blockDim.x) { 42 | sum += op(batch, plane, x); 43 | } 44 | } 45 | 46 | // sum over NumThreads within a warp 47 | sum = warpSum(sum); 48 | 49 | // 'transpose', and reduce within warp again 50 | __shared__ T shared[32]; 51 | __syncthreads(); 52 | if (threadIdx.x % WARP_SIZE == 0) { 53 | shared[threadIdx.x / WARP_SIZE] = sum; 54 | } 55 | if (threadIdx.x >= blockDim.x / WARP_SIZE && threadIdx.x < WARP_SIZE) { 56 | // zero out the other entries in shared 57 | shared[threadIdx.x] = (T)0; 58 | } 59 | __syncthreads(); 60 | if (threadIdx.x / WARP_SIZE == 0) { 61 | sum = warpSum(shared[threadIdx.x]); 62 | if (threadIdx.x == 0) { 63 | shared[0] = sum; 64 | } 65 | } 66 | __syncthreads(); 67 | 68 | // Everyone picks it up, should be broadcast into the whole gradInput 69 | return shared[0]; 70 | } 71 | 72 | template 73 | __global__ void BatchNorm_Forward_kernel ( 74 | DeviceTensor output, 75 | DeviceTensor input, 76 | DeviceTensor mean, 77 | DeviceTensor std, 78 | DeviceTensor gamma, 79 | DeviceTensor beta) { 80 | int c = blockIdx.x; 81 | /* main operation */ 82 | for (int b = 0; b < input.getSize(0); ++b) { 83 | for (int x = threadIdx.x; x < input.getSize(2); x += blockDim.x) { 84 | DType inp = input[b][c][x]; 85 | output[b][c][x] = gamma[c] * (inp - mean[c]) / 86 | std[c] + beta[c]; 87 | } 88 | } 89 | } 90 | 91 | template 92 | __global__ void BatchNorm_Forward_Inp_kernel ( 93 | DeviceTensor input, 94 | DeviceTensor mean, 95 | DeviceTensor std, 96 | DeviceTensor gamma, 97 | DeviceTensor beta) { 98 | int c = blockIdx.x; 99 | /* main operation */ 100 | for (int b = 0; b < input.getSize(0); ++b) { 101 | for (int x = threadIdx.x; x < input.getSize(2); x += blockDim.x) { 102 | DType inp = input[b][c][x]; 103 | input[b][c][x] = gamma[c] * (inp - mean[c]) / 104 | std[c] + beta[c]; 105 | } 106 | } 107 | } 108 | 109 | template 110 | __global__ void BatchNorm_Backward_Inp_kernel ( 111 | DeviceTensor gradoutput, 112 | DeviceTensor output, 113 | DeviceTensor gradinput, 114 | DeviceTensor gradgamma, 115 | DeviceTensor gradbeta, 116 | DeviceTensor mean, 117 | DeviceTensor std, 118 | DeviceTensor gamma, 119 | DeviceTensor beta, 120 | DeviceTensor gradEx, 121 | DeviceTensor gradExs) { 122 | /* declarations of the variables */ 123 | /* Get the index and channels */ 124 | int c = blockIdx.x; 125 | /* main operation */ 126 | GradOp> g(beta[c], output, gradoutput); 127 | Float2 res = reduce, 128 | GradOp>, 129 | DeviceTensor>(g, gradoutput, c); 130 | DType gradOutputSum = res.v1; 131 | DType dotP = res.v2; 132 | DType invstd = DType(1.0) / std[c]; 133 | DType gradScale = invstd * gamma[c]; 134 | if (threadIdx.x == 0) { 135 | gradEx[c] = - gradOutputSum * gradScale + mean[c] * invstd * invstd * dotP; 136 | gradExs[c] = - 0.5 * invstd * invstd * dotP; 137 | } 138 | if (gradinput.numElements() > 0) { 139 | for (int batch = 0; batch < gradoutput.getSize(0); ++batch) { 140 | for (int x = threadIdx.x; x < gradoutput.getSize(2); x += blockDim.x) { 141 | gradinput[batch][c][x] = gradoutput[batch][c][x] * gradScale; 142 | } 143 | } 144 | } 145 | if (gradgamma.numElements() > 0) { 146 | if (threadIdx.x == 0) { 147 | gradgamma[c] += dotP / gamma[c]; 148 | } 149 | } 150 | if (gradbeta.numElements() > 0) { 151 | if (threadIdx.x == 0) { 152 | gradbeta[c] += gradOutputSum; 153 | } 154 | } 155 | } 156 | 157 | template 158 | __global__ void BatchNorm_Backward_kernel ( 159 | DeviceTensor gradoutput, 160 | DeviceTensor input, 161 | DeviceTensor gradinput, 162 | DeviceTensor gradgamma, 163 | DeviceTensor gradbeta, 164 | DeviceTensor mean, 165 | DeviceTensor std, 166 | DeviceTensor gamma, 167 | DeviceTensor beta, 168 | DeviceTensor gradEx, 169 | DeviceTensor gradExs) { 170 | /* declarations of the variables */ 171 | /* Get the index and channels */ 172 | int c = blockIdx.x; 173 | /* main operation */ 174 | GradOp> g(mean[c], input, gradoutput); 175 | Float2 res = reduce, 176 | GradOp>, 177 | DeviceTensor>(g, gradoutput, c); 178 | DType gradOutputSum = res.v1; 179 | DType dotP = res.v2; 180 | DType invstd = DType(1.0) / std[c]; 181 | DType gradScale = invstd * gamma[c]; 182 | if (threadIdx.x == 0) { 183 | gradEx[c] = - gradOutputSum * gradScale + mean[c] * invstd * invstd * dotP * gradScale; 184 | gradExs[c] = - 0.5 * invstd * invstd * dotP * gradScale; 185 | } 186 | if (gradinput.numElements() > 0) { 187 | for (int batch = 0; batch < gradoutput.getSize(0); ++batch) { 188 | for (int x = threadIdx.x; x < gradoutput.getSize(2); x += blockDim.x) { 189 | gradinput[batch][c][x] = gradoutput[batch][c][x] * gradScale; 190 | } 191 | } 192 | } 193 | if (gradgamma.numElements() > 0) { 194 | if (threadIdx.x == 0) { 195 | gradgamma[c] += dotP * invstd; 196 | } 197 | } 198 | if (gradbeta.numElements() > 0) { 199 | if (threadIdx.x == 0) { 200 | gradbeta[c] += gradOutputSum; 201 | } 202 | } 203 | } 204 | 205 | 206 | template 207 | __global__ void Expectation_Forward_kernel ( 208 | DeviceTensor input, 209 | DeviceTensor ex, 210 | DeviceTensor exs, 211 | DType norm) { 212 | int c = blockIdx.x; 213 | /* main operation */ 214 | SumOp g(input); 215 | Float2 res = reduce, 216 | SumOp, DeviceTensor>(g, input, c); 217 | DType xsum = res.v1; 218 | DType xsquare = res.v2; 219 | if (threadIdx.x == 0) { 220 | ex[c] = xsum * norm; 221 | exs[c] = xsquare * norm; 222 | } 223 | } 224 | 225 | template 226 | __global__ void Expectation_Backward_kernel ( 227 | DeviceTensor gradInput, 228 | DeviceTensor input, 229 | DeviceTensor gradEx, 230 | DeviceTensor gradExs, 231 | DType norm) { 232 | int c = blockIdx.x; 233 | /* main operation */ 234 | for (int batch = 0; batch < gradInput.getSize(0); ++batch) { 235 | for (int x = threadIdx.x; x < gradInput.getSize(2); x += blockDim.x) { 236 | gradInput[batch][c][x] = gradEx[c] * norm + 2 * gradExs[c] * 237 | input[batch][c][x] * norm; 238 | } 239 | } 240 | } 241 | 242 | template 243 | __global__ void Expectation_Backward_Inp_kernel ( 244 | DeviceTensor gradInput, 245 | DeviceTensor output, 246 | DeviceTensor gradEx, 247 | DeviceTensor gradExs, 248 | DeviceTensor mean, 249 | DeviceTensor std, 250 | DeviceTensor gamma, 251 | DeviceTensor beta, 252 | DType norm) { 253 | int c = blockIdx.x; 254 | /* main operation */ 255 | for (int batch = 0; batch < gradInput.getSize(0); ++batch) { 256 | for (int x = threadIdx.x; x < gradInput.getSize(2); x += blockDim.x) { 257 | gradInput[batch][c][x] += gradEx[c] * norm + 2 * gradExs[c] * 258 | ((output[batch][c][x] - beta[c]) / gamma[c] * std[c] + mean[c]) * norm; 259 | } 260 | } 261 | } 262 | 263 | } // namespace 264 | 265 | at::Tensor BatchNorm_Forward_CUDA( 266 | const at::Tensor input_, 267 | const at::Tensor ex_, 268 | const at::Tensor exs_, 269 | const at::Tensor gamma_, 270 | const at::Tensor beta_, 271 | float eps) { 272 | auto output_ = at::zeros_like(input_); 273 | auto std_ = (exs_ - ex_ * ex_ + eps).sqrt(); 274 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 275 | dim3 blocks(input_.size(1)); 276 | dim3 threads(getNumThreads(input_.size(2))); 277 | AT_DISPATCH_FLOATING_TYPES(input_.type(), "BatchNorm_Forward_CUDA", ([&] { 278 | /* Device tensors */ 279 | DeviceTensor output = devicetensor(output_); 280 | DeviceTensor input = devicetensor(input_); 281 | DeviceTensor ex = devicetensor(ex_); 282 | DeviceTensor std = devicetensor(std_); 283 | DeviceTensor gamma = devicetensor(gamma_); 284 | DeviceTensor beta = devicetensor(beta_); 285 | /* kernel function */ 286 | BatchNorm_Forward_kernel<<>>( 287 | output, input, ex, std, gamma, beta); 288 | })); 289 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 290 | return output_; 291 | } 292 | 293 | at::Tensor BatchNorm_Forward_Inp_CUDA( 294 | const at::Tensor input_, 295 | const at::Tensor ex_, 296 | const at::Tensor exs_, 297 | const at::Tensor gamma_, 298 | const at::Tensor beta_, 299 | float eps) { 300 | auto std_ = (exs_ - ex_ * ex_ + eps).sqrt(); 301 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 302 | dim3 blocks(input_.size(1)); 303 | dim3 threads(getNumThreads(input_.size(2))); 304 | AT_DISPATCH_FLOATING_TYPES(input_.type(), "BatchNorm_Forward_CUDA", ([&] { 305 | /* Device tensors */ 306 | DeviceTensor input = devicetensor(input_); 307 | DeviceTensor ex = devicetensor(ex_); 308 | DeviceTensor std = devicetensor(std_); 309 | DeviceTensor gamma = devicetensor(gamma_); 310 | DeviceTensor beta = devicetensor(beta_); 311 | /* kernel function */ 312 | BatchNorm_Forward_Inp_kernel<<>>( 313 | input, ex, std, gamma, beta); 314 | })); 315 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 316 | return input_; 317 | } 318 | 319 | 320 | std::vector BatchNorm_Inp_Backward_CUDA( 321 | const at::Tensor gradoutput_, 322 | const at::Tensor output_, 323 | const at::Tensor ex_, 324 | const at::Tensor exs_, 325 | const at::Tensor gamma_, 326 | const at::Tensor beta_, 327 | float eps) { 328 | /* outputs*/ 329 | auto std_ = (exs_ - ex_ * ex_ + eps).sqrt(); 330 | auto gradinput_ = at::zeros_like(output_); 331 | auto gradgamma_ = at::zeros_like(gamma_); 332 | auto gradbeta_ = at::zeros_like(beta_); 333 | auto gradEx_ = at::zeros_like(ex_); 334 | auto gradExs_ = at::zeros_like(std_); 335 | /* cuda utils*/ 336 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 337 | dim3 blocks(output_.size(1)); 338 | dim3 threads(getNumThreads(output_.size(2))); 339 | AT_DISPATCH_FLOATING_TYPES(output_.type(), "BatchNorm_Inp_Backward_CUDA", ([&] { 340 | /* Device tensors */ 341 | DeviceTensor gradoutput = devicetensor(gradoutput_); 342 | DeviceTensor output = devicetensor(output_); 343 | DeviceTensor gradinput = devicetensor(gradinput_); 344 | DeviceTensor gradgamma = devicetensor(gradgamma_); 345 | DeviceTensor gradbeta = devicetensor(gradbeta_); 346 | DeviceTensor ex = devicetensor(ex_); 347 | DeviceTensor std = devicetensor(std_); 348 | DeviceTensor gamma = devicetensor(gamma_); 349 | DeviceTensor beta = devicetensor(beta_); 350 | DeviceTensor gradEx = devicetensor(gradEx_); 351 | DeviceTensor gradExs = devicetensor(gradExs_); 352 | /* kernel function */ 353 | BatchNorm_Backward_Inp_kernel 354 | <<>>( 355 | gradoutput, output, gradinput, gradgamma, gradbeta, ex, std, 356 | gamma, beta, gradEx, gradExs); 357 | })); 358 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 359 | return {gradinput_, gradEx_, gradExs_, gradgamma_, gradbeta_}; 360 | } 361 | 362 | 363 | std::vector BatchNorm_Backward_CUDA( 364 | const at::Tensor gradoutput_, 365 | const at::Tensor input_, 366 | const at::Tensor ex_, 367 | const at::Tensor exs_, 368 | const at::Tensor gamma_, 369 | const at::Tensor beta_, 370 | float eps) { 371 | /* outputs*/ 372 | auto std_ = (exs_ - ex_ * ex_ + eps).sqrt(); 373 | auto gradinput_ = at::zeros_like(input_); 374 | auto gradgamma_ = at::zeros_like(gamma_); 375 | auto gradbeta_ = at::zeros_like(beta_); 376 | auto gradEx_ = at::zeros_like(ex_); 377 | auto gradExs_ = at::zeros_like(std_); 378 | /* cuda utils*/ 379 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 380 | dim3 blocks(input_.size(1)); 381 | dim3 threads(getNumThreads(input_.size(2))); 382 | AT_DISPATCH_FLOATING_TYPES(input_.type(), "BatchNorm_Inp_Backward_CUDA", ([&] { 383 | /* Device tensors */ 384 | DeviceTensor gradoutput = devicetensor(gradoutput_); 385 | DeviceTensor input = devicetensor(input_); 386 | DeviceTensor gradinput = devicetensor(gradinput_); 387 | DeviceTensor gradgamma = devicetensor(gradgamma_); 388 | DeviceTensor gradbeta = devicetensor(gradbeta_); 389 | DeviceTensor ex = devicetensor(ex_); 390 | DeviceTensor std = devicetensor(std_); 391 | DeviceTensor gamma = devicetensor(gamma_); 392 | DeviceTensor beta = devicetensor(beta_); 393 | DeviceTensor gradEx = devicetensor(gradEx_); 394 | DeviceTensor gradExs = devicetensor(gradExs_); 395 | /* kernel function */ 396 | BatchNorm_Backward_kernel 397 | <<>>( 398 | gradoutput, input, gradinput, gradgamma, gradbeta, ex, std, 399 | gamma, beta, gradEx, gradExs); 400 | })); 401 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 402 | return {gradinput_, gradEx_, gradExs_, gradgamma_, gradbeta_}; 403 | } 404 | 405 | std::vector Expectation_Forward_CUDA( 406 | const at::Tensor input_) { 407 | /* outputs */ 408 | auto ex_ = torch::zeros({input_.size(1)}, input_.options()); 409 | auto exs_ = torch::zeros({input_.size(1)}, input_.options()); 410 | /* cuda utils*/ 411 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 412 | dim3 blocks(input_.size(1)); 413 | dim3 threads(getNumThreads(input_.size(2))); 414 | AT_DISPATCH_FLOATING_TYPES(input_.type(), "SumSquare_forward_CUDA", ([&] { 415 | scalar_t norm = scalar_t(1) / (input_.size(0) * input_.size(2)); 416 | /* Device tensors */ 417 | DeviceTensor input = devicetensor(input_); 418 | DeviceTensor ex = devicetensor(ex_); 419 | DeviceTensor exs = devicetensor(exs_); 420 | /* kernel function */ 421 | Expectation_Forward_kernel 422 | <<>>(input, ex, exs, norm); 423 | })); 424 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 425 | return {ex_, exs_}; 426 | } 427 | 428 | at::Tensor Expectation_Backward_CUDA( 429 | const at::Tensor input_, 430 | const at::Tensor gradEx_, 431 | const at::Tensor gradExs_) { 432 | /* outputs */ 433 | at::Tensor gradInput_ = at::zeros_like(input_); 434 | /* cuda utils*/ 435 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 436 | dim3 blocks(input_.size(1)); 437 | dim3 threads(getNumThreads(input_.size(2))); 438 | AT_DISPATCH_FLOATING_TYPES(input_.type(), "SumSquare_Backward_CUDA", ([&] { 439 | scalar_t norm = scalar_t(1) / (input_.size(0) * input_.size(2)); 440 | /* Device tensors */ 441 | DeviceTensor gradInput = devicetensor(gradInput_); 442 | DeviceTensor input = devicetensor(input_); 443 | DeviceTensor gradEx = devicetensor(gradEx_); 444 | DeviceTensor gradExs =devicetensor(gradExs_); 445 | /* kernel function */ 446 | Expectation_Backward_kernel 447 | <<>>(gradInput, input, gradEx, gradExs, norm); 448 | })); 449 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 450 | return gradInput_; 451 | } 452 | 453 | at::Tensor Expectation_Inp_Backward_CUDA( 454 | const at::Tensor gradInput_, 455 | const at::Tensor output_, 456 | const at::Tensor gradEx_, 457 | const at::Tensor gradExs_, 458 | const at::Tensor ex_, 459 | const at::Tensor exs_, 460 | const at::Tensor gamma_, 461 | const at::Tensor beta_, 462 | float eps) { 463 | /* outputs */ 464 | //auto gradInput_ = at::zeros_like(output_); 465 | auto std_ = (exs_ - ex_ * ex_ + eps).sqrt(); 466 | /* cuda utils*/ 467 | cudaStream_t stream = at::cuda::getCurrentCUDAStream(); 468 | dim3 blocks(output_.size(1)); 469 | dim3 threads(getNumThreads(output_.size(2))); 470 | AT_DISPATCH_FLOATING_TYPES(output_.type(), "SumSquare_Backward_CUDA", ([&] { 471 | scalar_t norm = scalar_t(1) / (output_.size(0) * output_.size(2)); 472 | /* Device tensors */ 473 | DeviceTensor gradInput = devicetensor(gradInput_); 474 | DeviceTensor input = devicetensor(output_); 475 | DeviceTensor gradEx = devicetensor(gradEx_); 476 | DeviceTensor gradExs =devicetensor(gradExs_); 477 | DeviceTensor ex = devicetensor(ex_); 478 | DeviceTensor std = devicetensor(std_); 479 | DeviceTensor gamma = devicetensor(gamma_); 480 | DeviceTensor beta = devicetensor(beta_); 481 | /* kernel function */ 482 | Expectation_Backward_Inp_kernel 483 | <<>>(gradInput, input, gradEx, gradExs, 484 | ex, std, gamma, beta, norm); 485 | })); 486 | AT_ASSERT(cudaGetLastError() == cudaSuccess); 487 | return gradInput_; 488 | } 489 | -------------------------------------------------------------------------------- /lists/debug.list: -------------------------------------------------------------------------------- 1 | TEST/B/0060/left/0009.png 2 | TEST/C/0048/left/0013.png 3 | TEST/C/0015/left/0007.png 4 | TEST/C/0128/left/0006.png 5 | TEST/B/0034/left/0014.png 6 | TEST/C/0066/left/0010.png 7 | TEST/A/0002/left/0012.png 8 | TEST/C/0021/left/0010.png 9 | TEST/B/0055/left/0013.png 10 | TEST/A/0117/left/0015.png 11 | TEST/C/0067/left/0013.png 12 | TEST/A/0115/left/0007.png 13 | TEST/A/0143/left/0007.png 14 | TEST/A/0086/left/0007.png 15 | TEST/A/0136/left/0015.png 16 | -------------------------------------------------------------------------------- /lists/kitti2012_test.list: -------------------------------------------------------------------------------- 1 | 000000_10.png 2 | 000001_10.png 3 | 000002_10.png 4 | 000003_10.png 5 | 000004_10.png 6 | 000005_10.png 7 | 000006_10.png 8 | 000007_10.png 9 | 000008_10.png 10 | 000009_10.png 11 | 000010_10.png 12 | 000011_10.png 13 | 000012_10.png 14 | 000013_10.png 15 | 000014_10.png 16 | 000015_10.png 17 | 000016_10.png 18 | 000017_10.png 19 | 000018_10.png 20 | 000019_10.png 21 | 000020_10.png 22 | 000021_10.png 23 | 000022_10.png 24 | 000023_10.png 25 | 000024_10.png 26 | 000025_10.png 27 | 000026_10.png 28 | 000027_10.png 29 | 000028_10.png 30 | 000029_10.png 31 | 000030_10.png 32 | 000031_10.png 33 | 000032_10.png 34 | 000033_10.png 35 | 000034_10.png 36 | 000035_10.png 37 | 000036_10.png 38 | 000037_10.png 39 | 000038_10.png 40 | 000039_10.png 41 | 000040_10.png 42 | 000041_10.png 43 | 000042_10.png 44 | 000043_10.png 45 | 000044_10.png 46 | 000045_10.png 47 | 000046_10.png 48 | 000047_10.png 49 | 000048_10.png 50 | 000049_10.png 51 | 000050_10.png 52 | 000051_10.png 53 | 000052_10.png 54 | 000053_10.png 55 | 000054_10.png 56 | 000055_10.png 57 | 000056_10.png 58 | 000057_10.png 59 | 000058_10.png 60 | 000059_10.png 61 | 000060_10.png 62 | 000061_10.png 63 | 000062_10.png 64 | 000063_10.png 65 | 000064_10.png 66 | 000065_10.png 67 | 000066_10.png 68 | 000067_10.png 69 | 000068_10.png 70 | 000069_10.png 71 | 000070_10.png 72 | 000071_10.png 73 | 000072_10.png 74 | 000073_10.png 75 | 000074_10.png 76 | 000075_10.png 77 | 000076_10.png 78 | 000077_10.png 79 | 000078_10.png 80 | 000079_10.png 81 | 000080_10.png 82 | 000081_10.png 83 | 000082_10.png 84 | 000083_10.png 85 | 000084_10.png 86 | 000085_10.png 87 | 000086_10.png 88 | 000087_10.png 89 | 000088_10.png 90 | 000089_10.png 91 | 000090_10.png 92 | 000091_10.png 93 | 000092_10.png 94 | 000093_10.png 95 | 000094_10.png 96 | 000095_10.png 97 | 000096_10.png 98 | 000097_10.png 99 | 000098_10.png 100 | 000099_10.png 101 | 000100_10.png 102 | 000101_10.png 103 | 000102_10.png 104 | 000103_10.png 105 | 000104_10.png 106 | 000105_10.png 107 | 000106_10.png 108 | 000107_10.png 109 | 000108_10.png 110 | 000109_10.png 111 | 000110_10.png 112 | 000111_10.png 113 | 000112_10.png 114 | 000113_10.png 115 | 000114_10.png 116 | 000115_10.png 117 | 000116_10.png 118 | 000117_10.png 119 | 000118_10.png 120 | 000119_10.png 121 | 000120_10.png 122 | 000121_10.png 123 | 000122_10.png 124 | 000123_10.png 125 | 000124_10.png 126 | 000125_10.png 127 | 000126_10.png 128 | 000127_10.png 129 | 000128_10.png 130 | 000129_10.png 131 | 000130_10.png 132 | 000131_10.png 133 | 000132_10.png 134 | 000133_10.png 135 | 000134_10.png 136 | 000135_10.png 137 | 000136_10.png 138 | 000137_10.png 139 | 000138_10.png 140 | 000139_10.png 141 | 000140_10.png 142 | 000141_10.png 143 | 000142_10.png 144 | 000143_10.png 145 | 000144_10.png 146 | 000145_10.png 147 | 000146_10.png 148 | 000147_10.png 149 | 000148_10.png 150 | 000149_10.png 151 | 000150_10.png 152 | 000151_10.png 153 | 000152_10.png 154 | 000153_10.png 155 | 000154_10.png 156 | 000155_10.png 157 | 000156_10.png 158 | 000157_10.png 159 | 000158_10.png 160 | 000159_10.png 161 | 000160_10.png 162 | 000161_10.png 163 | 000162_10.png 164 | 000163_10.png 165 | 000164_10.png 166 | 000165_10.png 167 | 000166_10.png 168 | 000167_10.png 169 | 000168_10.png 170 | 000169_10.png 171 | 000170_10.png 172 | 000171_10.png 173 | 000172_10.png 174 | 000173_10.png 175 | 000174_10.png 176 | 000175_10.png 177 | 000176_10.png 178 | 000177_10.png 179 | 000178_10.png 180 | 000179_10.png 181 | 000180_10.png 182 | 000181_10.png 183 | 000182_10.png 184 | 000183_10.png 185 | 000184_10.png 186 | 000185_10.png 187 | 000186_10.png 188 | 000187_10.png 189 | 000188_10.png 190 | 000189_10.png 191 | 000190_10.png 192 | 000191_10.png 193 | 000192_10.png 194 | 000193_10.png 195 | 000194_10.png 196 | -------------------------------------------------------------------------------- /lists/kitti2012_train.list: -------------------------------------------------------------------------------- 1 | 000153_10.png 2 | 000174_10.png 3 | 000176_10.png 4 | 000022_10.png 5 | 000169_10.png 6 | 000057_10.png 7 | 000013_10.png 8 | 000185_10.png 9 | 000072_10.png 10 | 000182_10.png 11 | 000012_10.png 12 | 000054_10.png 13 | 000064_10.png 14 | 000061_10.png 15 | 000017_10.png 16 | 000155_10.png 17 | 000044_10.png 18 | 000001_10.png 19 | 000030_10.png 20 | 000106_10.png 21 | 000007_10.png 22 | 000092_10.png 23 | 000142_10.png 24 | 000144_10.png 25 | 000193_10.png 26 | 000080_10.png 27 | 000097_10.png 28 | 000143_10.png 29 | 000042_10.png 30 | 000081_10.png 31 | 000109_10.png 32 | 000171_10.png 33 | 000175_10.png 34 | 000047_10.png 35 | 000014_10.png 36 | 000100_10.png 37 | 000015_10.png 38 | 000180_10.png 39 | 000016_10.png 40 | 000019_10.png 41 | 000139_10.png 42 | 000184_10.png 43 | 000050_10.png 44 | 000115_10.png 45 | 000136_10.png 46 | 000188_10.png 47 | 000034_10.png 48 | 000033_10.png 49 | 000114_10.png 50 | 000038_10.png 51 | 000063_10.png 52 | 000121_10.png 53 | 000004_10.png 54 | 000062_10.png 55 | 000160_10.png 56 | 000148_10.png 57 | 000120_10.png 58 | 000052_10.png 59 | 000045_10.png 60 | 000126_10.png 61 | 000065_10.png 62 | 000127_10.png 63 | 000113_10.png 64 | 000131_10.png 65 | 000163_10.png 66 | 000146_10.png 67 | 000028_10.png 68 | 000048_10.png 69 | 000179_10.png 70 | 000135_10.png 71 | 000053_10.png 72 | 000159_10.png 73 | 000117_10.png 74 | 000134_10.png 75 | 000181_10.png 76 | 000021_10.png 77 | 000067_10.png 78 | 000077_10.png 79 | 000177_10.png 80 | 000029_10.png 81 | 000070_10.png 82 | 000137_10.png 83 | 000090_10.png 84 | 000027_10.png 85 | 000123_10.png 86 | 000107_10.png 87 | 000026_10.png 88 | 000190_10.png 89 | 000133_10.png 90 | 000024_10.png 91 | 000119_10.png 92 | 000116_10.png 93 | 000023_10.png 94 | 000066_10.png 95 | 000152_10.png 96 | 000008_10.png 97 | 000079_10.png 98 | 000069_10.png 99 | 000187_10.png 100 | 000145_10.png 101 | 000124_10.png 102 | 000140_10.png 103 | 000150_10.png 104 | 000132_10.png 105 | 000025_10.png 106 | 000122_10.png 107 | 000110_10.png 108 | 000041_10.png 109 | 000098_10.png 110 | 000118_10.png 111 | 000084_10.png 112 | 000103_10.png 113 | 000086_10.png 114 | 000178_10.png 115 | 000056_10.png 116 | 000058_10.png 117 | 000006_10.png 118 | 000111_10.png 119 | 000186_10.png 120 | 000170_10.png 121 | 000141_10.png 122 | 000020_10.png 123 | 000073_10.png 124 | 000093_10.png 125 | 000009_10.png 126 | 000000_10.png 127 | 000094_10.png 128 | 000149_10.png 129 | 000151_10.png 130 | 000046_10.png 131 | 000002_10.png 132 | 000010_10.png 133 | 000003_10.png 134 | 000035_10.png 135 | 000183_10.png 136 | 000036_10.png 137 | 000112_10.png 138 | 000040_10.png 139 | 000192_10.png 140 | 000091_10.png 141 | 000156_10.png 142 | 000128_10.png 143 | 000164_10.png 144 | 000055_10.png 145 | 000130_10.png 146 | 000095_10.png 147 | 000162_10.png 148 | 000189_10.png 149 | 000088_10.png 150 | 000125_10.png 151 | 000078_10.png 152 | 000039_10.png 153 | 000157_10.png 154 | 000102_10.png 155 | 000138_10.png 156 | 000129_10.png 157 | 000165_10.png 158 | 000161_10.png 159 | 000089_10.png 160 | 000087_10.png 161 | 000011_10.png 162 | 000085_10.png 163 | 000191_10.png 164 | 000049_10.png 165 | 000168_10.png 166 | 000082_10.png 167 | 000154_10.png 168 | 000060_10.png 169 | 000018_10.png 170 | 000166_10.png 171 | 000173_10.png 172 | 000172_10.png 173 | 000147_10.png 174 | 000059_10.png 175 | 000043_10.png 176 | 000099_10.png 177 | 000031_10.png 178 | 000076_10.png 179 | 000032_10.png 180 | 000105_10.png 181 | 000101_10.png 182 | 000074_10.png 183 | 000071_10.png 184 | 000158_10.png 185 | 000051_10.png 186 | 000083_10.png 187 | 000037_10.png 188 | 000005_10.png 189 | 000104_10.png 190 | 000096_10.png 191 | 000068_10.png 192 | 000108_10.png 193 | 000075_10.png 194 | 000167_10.png 195 | -------------------------------------------------------------------------------- /lists/kitti2012_train170.list: -------------------------------------------------------------------------------- 1 | 000193_10.png 2 | 000080_10.png 3 | 000097_10.png 4 | 000143_10.png 5 | 000042_10.png 6 | 000081_10.png 7 | 000109_10.png 8 | 000171_10.png 9 | 000175_10.png 10 | 000047_10.png 11 | 000014_10.png 12 | 000100_10.png 13 | 000015_10.png 14 | 000180_10.png 15 | 000016_10.png 16 | 000019_10.png 17 | 000139_10.png 18 | 000184_10.png 19 | 000050_10.png 20 | 000115_10.png 21 | 000136_10.png 22 | 000188_10.png 23 | 000034_10.png 24 | 000033_10.png 25 | 000114_10.png 26 | 000038_10.png 27 | 000063_10.png 28 | 000121_10.png 29 | 000004_10.png 30 | 000062_10.png 31 | 000160_10.png 32 | 000148_10.png 33 | 000120_10.png 34 | 000052_10.png 35 | 000045_10.png 36 | 000126_10.png 37 | 000065_10.png 38 | 000127_10.png 39 | 000113_10.png 40 | 000131_10.png 41 | 000163_10.png 42 | 000146_10.png 43 | 000028_10.png 44 | 000048_10.png 45 | 000179_10.png 46 | 000135_10.png 47 | 000053_10.png 48 | 000159_10.png 49 | 000117_10.png 50 | 000134_10.png 51 | 000181_10.png 52 | 000021_10.png 53 | 000067_10.png 54 | 000077_10.png 55 | 000177_10.png 56 | 000029_10.png 57 | 000070_10.png 58 | 000137_10.png 59 | 000090_10.png 60 | 000027_10.png 61 | 000123_10.png 62 | 000107_10.png 63 | 000026_10.png 64 | 000190_10.png 65 | 000133_10.png 66 | 000024_10.png 67 | 000119_10.png 68 | 000116_10.png 69 | 000023_10.png 70 | 000066_10.png 71 | 000152_10.png 72 | 000008_10.png 73 | 000079_10.png 74 | 000069_10.png 75 | 000187_10.png 76 | 000145_10.png 77 | 000124_10.png 78 | 000140_10.png 79 | 000150_10.png 80 | 000132_10.png 81 | 000025_10.png 82 | 000122_10.png 83 | 000110_10.png 84 | 000041_10.png 85 | 000098_10.png 86 | 000118_10.png 87 | 000084_10.png 88 | 000103_10.png 89 | 000086_10.png 90 | 000178_10.png 91 | 000056_10.png 92 | 000058_10.png 93 | 000006_10.png 94 | 000111_10.png 95 | 000186_10.png 96 | 000170_10.png 97 | 000141_10.png 98 | 000020_10.png 99 | 000073_10.png 100 | 000093_10.png 101 | 000009_10.png 102 | 000000_10.png 103 | 000094_10.png 104 | 000149_10.png 105 | 000151_10.png 106 | 000046_10.png 107 | 000002_10.png 108 | 000010_10.png 109 | 000003_10.png 110 | 000035_10.png 111 | 000183_10.png 112 | 000036_10.png 113 | 000112_10.png 114 | 000040_10.png 115 | 000192_10.png 116 | 000091_10.png 117 | 000156_10.png 118 | 000128_10.png 119 | 000164_10.png 120 | 000055_10.png 121 | 000130_10.png 122 | 000095_10.png 123 | 000162_10.png 124 | 000189_10.png 125 | 000088_10.png 126 | 000125_10.png 127 | 000078_10.png 128 | 000039_10.png 129 | 000157_10.png 130 | 000102_10.png 131 | 000138_10.png 132 | 000129_10.png 133 | 000165_10.png 134 | 000161_10.png 135 | 000089_10.png 136 | 000087_10.png 137 | 000011_10.png 138 | 000085_10.png 139 | 000191_10.png 140 | 000049_10.png 141 | 000168_10.png 142 | 000082_10.png 143 | 000154_10.png 144 | 000060_10.png 145 | 000018_10.png 146 | 000166_10.png 147 | 000173_10.png 148 | 000172_10.png 149 | 000147_10.png 150 | 000059_10.png 151 | 000043_10.png 152 | 000099_10.png 153 | 000031_10.png 154 | 000076_10.png 155 | 000032_10.png 156 | 000105_10.png 157 | 000101_10.png 158 | 000074_10.png 159 | 000071_10.png 160 | 000158_10.png 161 | 000051_10.png 162 | 000083_10.png 163 | 000037_10.png 164 | 000005_10.png 165 | 000104_10.png 166 | 000096_10.png 167 | 000068_10.png 168 | 000108_10.png 169 | 000075_10.png 170 | 000167_10.png 171 | -------------------------------------------------------------------------------- /lists/kitti2012_val24.list: -------------------------------------------------------------------------------- 1 | 000153_10.png 2 | 000174_10.png 3 | 000176_10.png 4 | 000022_10.png 5 | 000169_10.png 6 | 000057_10.png 7 | 000013_10.png 8 | 000185_10.png 9 | 000072_10.png 10 | 000182_10.png 11 | 000012_10.png 12 | 000054_10.png 13 | 000064_10.png 14 | 000061_10.png 15 | 000017_10.png 16 | 000155_10.png 17 | 000044_10.png 18 | 000001_10.png 19 | 000030_10.png 20 | 000106_10.png 21 | 000007_10.png 22 | 000092_10.png 23 | 000142_10.png 24 | 000144_10.png 25 | -------------------------------------------------------------------------------- /lists/kitti2015_test.list: -------------------------------------------------------------------------------- 1 | 000000_10.png 2 | 000001_10.png 3 | 000002_10.png 4 | 000003_10.png 5 | 000004_10.png 6 | 000005_10.png 7 | 000006_10.png 8 | 000007_10.png 9 | 000008_10.png 10 | 000009_10.png 11 | 000010_10.png 12 | 000011_10.png 13 | 000012_10.png 14 | 000013_10.png 15 | 000014_10.png 16 | 000015_10.png 17 | 000016_10.png 18 | 000017_10.png 19 | 000018_10.png 20 | 000019_10.png 21 | 000020_10.png 22 | 000021_10.png 23 | 000022_10.png 24 | 000023_10.png 25 | 000024_10.png 26 | 000025_10.png 27 | 000026_10.png 28 | 000027_10.png 29 | 000028_10.png 30 | 000029_10.png 31 | 000030_10.png 32 | 000031_10.png 33 | 000032_10.png 34 | 000033_10.png 35 | 000034_10.png 36 | 000035_10.png 37 | 000036_10.png 38 | 000037_10.png 39 | 000038_10.png 40 | 000039_10.png 41 | 000040_10.png 42 | 000041_10.png 43 | 000042_10.png 44 | 000043_10.png 45 | 000044_10.png 46 | 000045_10.png 47 | 000046_10.png 48 | 000047_10.png 49 | 000048_10.png 50 | 000049_10.png 51 | 000050_10.png 52 | 000051_10.png 53 | 000052_10.png 54 | 000053_10.png 55 | 000054_10.png 56 | 000055_10.png 57 | 000056_10.png 58 | 000057_10.png 59 | 000058_10.png 60 | 000059_10.png 61 | 000060_10.png 62 | 000061_10.png 63 | 000062_10.png 64 | 000063_10.png 65 | 000064_10.png 66 | 000065_10.png 67 | 000066_10.png 68 | 000067_10.png 69 | 000068_10.png 70 | 000069_10.png 71 | 000070_10.png 72 | 000071_10.png 73 | 000072_10.png 74 | 000073_10.png 75 | 000074_10.png 76 | 000075_10.png 77 | 000076_10.png 78 | 000077_10.png 79 | 000078_10.png 80 | 000079_10.png 81 | 000080_10.png 82 | 000081_10.png 83 | 000082_10.png 84 | 000083_10.png 85 | 000084_10.png 86 | 000085_10.png 87 | 000086_10.png 88 | 000087_10.png 89 | 000088_10.png 90 | 000089_10.png 91 | 000090_10.png 92 | 000091_10.png 93 | 000092_10.png 94 | 000093_10.png 95 | 000094_10.png 96 | 000095_10.png 97 | 000096_10.png 98 | 000097_10.png 99 | 000098_10.png 100 | 000099_10.png 101 | 000100_10.png 102 | 000101_10.png 103 | 000102_10.png 104 | 000103_10.png 105 | 000104_10.png 106 | 000105_10.png 107 | 000106_10.png 108 | 000107_10.png 109 | 000108_10.png 110 | 000109_10.png 111 | 000110_10.png 112 | 000111_10.png 113 | 000112_10.png 114 | 000113_10.png 115 | 000114_10.png 116 | 000115_10.png 117 | 000116_10.png 118 | 000117_10.png 119 | 000118_10.png 120 | 000119_10.png 121 | 000120_10.png 122 | 000121_10.png 123 | 000122_10.png 124 | 000123_10.png 125 | 000124_10.png 126 | 000125_10.png 127 | 000126_10.png 128 | 000127_10.png 129 | 000128_10.png 130 | 000129_10.png 131 | 000130_10.png 132 | 000131_10.png 133 | 000132_10.png 134 | 000133_10.png 135 | 000134_10.png 136 | 000135_10.png 137 | 000136_10.png 138 | 000137_10.png 139 | 000138_10.png 140 | 000139_10.png 141 | 000140_10.png 142 | 000141_10.png 143 | 000142_10.png 144 | 000143_10.png 145 | 000144_10.png 146 | 000145_10.png 147 | 000146_10.png 148 | 000147_10.png 149 | 000148_10.png 150 | 000149_10.png 151 | 000150_10.png 152 | 000151_10.png 153 | 000152_10.png 154 | 000153_10.png 155 | 000154_10.png 156 | 000155_10.png 157 | 000156_10.png 158 | 000157_10.png 159 | 000158_10.png 160 | 000159_10.png 161 | 000160_10.png 162 | 000161_10.png 163 | 000162_10.png 164 | 000163_10.png 165 | 000164_10.png 166 | 000165_10.png 167 | 000166_10.png 168 | 000167_10.png 169 | 000168_10.png 170 | 000169_10.png 171 | 000170_10.png 172 | 000171_10.png 173 | 000172_10.png 174 | 000173_10.png 175 | 000174_10.png 176 | 000175_10.png 177 | 000176_10.png 178 | 000177_10.png 179 | 000178_10.png 180 | 000179_10.png 181 | 000180_10.png 182 | 000181_10.png 183 | 000182_10.png 184 | 000183_10.png 185 | 000184_10.png 186 | 000185_10.png 187 | 000186_10.png 188 | 000187_10.png 189 | 000188_10.png 190 | 000189_10.png 191 | 000190_10.png 192 | 000191_10.png 193 | 000192_10.png 194 | 000193_10.png 195 | 000194_10.png 196 | 000195_10.png 197 | 000196_10.png 198 | 000197_10.png 199 | 000198_10.png 200 | 000199_10.png 201 | -------------------------------------------------------------------------------- /lists/kitti2015_train.list: -------------------------------------------------------------------------------- 1 | 000179_10.png 2 | 000128_10.png 3 | 000122_10.png 4 | 000178_10.png 5 | 000173_10.png 6 | 000100_10.png 7 | 000114_10.png 8 | 000037_10.png 9 | 000071_10.png 10 | 000076_10.png 11 | 000031_10.png 12 | 000130_10.png 13 | 000191_10.png 14 | 000086_10.png 15 | 000099_10.png 16 | 000195_10.png 17 | 000005_10.png 18 | 000131_10.png 19 | 000083_10.png 20 | 000196_10.png 21 | 000001_10.png 22 | 000022_10.png 23 | 000189_10.png 24 | 000030_10.png 25 | 000096_10.png 26 | 000082_10.png 27 | 000101_10.png 28 | 000121_10.png 29 | 000046_10.png 30 | 000106_10.png 31 | 000015_10.png 32 | 000056_10.png 33 | 000021_10.png 34 | 000111_10.png 35 | 000070_10.png 36 | 000145_10.png 37 | 000197_10.png 38 | 000040_10.png 39 | 000092_10.png 40 | 000186_10.png 41 | 000140_10.png 42 | 000127_10.png 43 | 000165_10.png 44 | 000141_10.png 45 | 000193_10.png 46 | 000184_10.png 47 | 000154_10.png 48 | 000060_10.png 49 | 000107_10.png 50 | 000029_10.png 51 | 000038_10.png 52 | 000135_10.png 53 | 000091_10.png 54 | 000156_10.png 55 | 000112_10.png 56 | 000054_10.png 57 | 000065_10.png 58 | 000113_10.png 59 | 000016_10.png 60 | 000050_10.png 61 | 000089_10.png 62 | 000166_10.png 63 | 000142_10.png 64 | 000097_10.png 65 | 000042_10.png 66 | 000194_10.png 67 | 000190_10.png 68 | 000134_10.png 69 | 000080_10.png 70 | 000155_10.png 71 | 000055_10.png 72 | 000116_10.png 73 | 000069_10.png 74 | 000020_10.png 75 | 000180_10.png 76 | 000068_10.png 77 | 000129_10.png 78 | 000027_10.png 79 | 000167_10.png 80 | 000081_10.png 81 | 000115_10.png 82 | 000048_10.png 83 | 000061_10.png 84 | 000011_10.png 85 | 000105_10.png 86 | 000161_10.png 87 | 000062_10.png 88 | 000088_10.png 89 | 000098_10.png 90 | 000093_10.png 91 | 000064_10.png 92 | 000032_10.png 93 | 000102_10.png 94 | 000058_10.png 95 | 000188_10.png 96 | 000148_10.png 97 | 000183_10.png 98 | 000072_10.png 99 | 000103_10.png 100 | 000126_10.png 101 | 000034_10.png 102 | 000024_10.png 103 | 000185_10.png 104 | 000108_10.png 105 | 000181_10.png 106 | 000153_10.png 107 | 000174_10.png 108 | 000157_10.png 109 | 000151_10.png 110 | 000170_10.png 111 | 000118_10.png 112 | 000117_10.png 113 | 000139_10.png 114 | 000143_10.png 115 | 000077_10.png 116 | 000168_10.png 117 | 000012_10.png 118 | 000146_10.png 119 | 000152_10.png 120 | 000039_10.png 121 | 000053_10.png 122 | 000162_10.png 123 | 000041_10.png 124 | 000051_10.png 125 | 000035_10.png 126 | 000075_10.png 127 | 000138_10.png 128 | 000044_10.png 129 | 000036_10.png 130 | 000000_10.png 131 | 000164_10.png 132 | 000004_10.png 133 | 000175_10.png 134 | 000094_10.png 135 | 000109_10.png 136 | 000052_10.png 137 | 000047_10.png 138 | 000028_10.png 139 | 000149_10.png 140 | 000026_10.png 141 | 000192_10.png 142 | 000049_10.png 143 | 000014_10.png 144 | 000125_10.png 145 | 000136_10.png 146 | 000177_10.png 147 | 000084_10.png 148 | 000007_10.png 149 | 000085_10.png 150 | 000063_10.png 151 | 000019_10.png 152 | 000176_10.png 153 | 000079_10.png 154 | 000187_10.png 155 | 000172_10.png 156 | 000199_10.png 157 | 000003_10.png 158 | 000124_10.png 159 | 000095_10.png 160 | 000160_10.png 161 | 000009_10.png 162 | 000137_10.png 163 | 000087_10.png 164 | 000150_10.png 165 | 000090_10.png 166 | 000006_10.png 167 | 000182_10.png 168 | 000132_10.png 169 | 000078_10.png 170 | 000002_10.png 171 | 000169_10.png 172 | 000017_10.png 173 | 000110_10.png 174 | 000074_10.png 175 | 000119_10.png 176 | 000147_10.png 177 | 000159_10.png 178 | 000013_10.png 179 | 000045_10.png 180 | 000010_10.png 181 | 000198_10.png 182 | 000059_10.png 183 | 000067_10.png 184 | 000123_10.png 185 | 000133_10.png 186 | 000057_10.png 187 | 000073_10.png 188 | 000120_10.png 189 | 000018_10.png 190 | 000171_10.png 191 | 000033_10.png 192 | 000023_10.png 193 | 000043_10.png 194 | 000158_10.png 195 | 000163_10.png 196 | 000025_10.png 197 | 000144_10.png 198 | 000008_10.png 199 | 000066_10.png 200 | -------------------------------------------------------------------------------- /lists/sceneflow_test_select.list: -------------------------------------------------------------------------------- 1 | TEST/B/0060/left/0009.png 2 | TEST/C/0048/left/0013.png 3 | TEST/C/0015/left/0007.png 4 | TEST/C/0128/left/0006.png 5 | TEST/B/0034/left/0014.png 6 | TEST/C/0066/left/0010.png 7 | TEST/A/0002/left/0012.png 8 | TEST/C/0021/left/0010.png 9 | TEST/B/0055/left/0013.png 10 | TEST/A/0117/left/0015.png 11 | TEST/C/0067/left/0013.png 12 | TEST/A/0115/left/0007.png 13 | TEST/A/0143/left/0007.png 14 | TEST/A/0086/left/0007.png 15 | TEST/A/0136/left/0015.png 16 | TEST/B/0026/left/0008.png 17 | TEST/C/0111/left/0015.png 18 | TEST/C/0082/left/0013.png 19 | TEST/A/0039/left/0007.png 20 | TEST/B/0021/left/0007.png 21 | TEST/C/0076/left/0014.png 22 | TEST/A/0006/left/0009.png 23 | TEST/C/0116/left/0013.png 24 | TEST/C/0061/left/0012.png 25 | TEST/A/0112/left/0009.png 26 | TEST/C/0087/left/0012.png 27 | TEST/C/0098/left/0012.png 28 | TEST/C/0115/left/0011.png 29 | TEST/C/0050/left/0006.png 30 | TEST/A/0133/left/0007.png 31 | TEST/C/0079/left/0006.png 32 | TEST/C/0024/left/0012.png 33 | TEST/A/0034/left/0007.png 34 | TEST/C/0080/left/0010.png 35 | TEST/A/0025/left/0013.png 36 | TEST/A/0131/left/0009.png 37 | TEST/B/0032/left/0015.png 38 | TEST/C/0100/left/0012.png 39 | TEST/B/0120/left/0006.png 40 | TEST/A/0127/left/0013.png 41 | TEST/C/0135/left/0014.png 42 | TEST/A/0022/left/0014.png 43 | TEST/C/0138/left/0006.png 44 | TEST/B/0024/left/0008.png 45 | TEST/B/0067/left/0014.png 46 | TEST/B/0092/left/0014.png 47 | TEST/A/0088/left/0009.png 48 | TEST/A/0099/left/0013.png 49 | TEST/B/0076/left/0009.png 50 | TEST/B/0147/left/0007.png 51 | TEST/B/0113/left/0011.png 52 | TEST/B/0018/left/0008.png 53 | TEST/A/0080/left/0008.png 54 | TEST/B/0137/left/0015.png 55 | TEST/B/0085/left/0013.png 56 | TEST/C/0102/left/0011.png 57 | TEST/C/0078/left/0013.png 58 | TEST/C/0029/left/0012.png 59 | TEST/B/0015/left/0013.png 60 | TEST/B/0074/left/0010.png 61 | TEST/C/0147/left/0010.png 62 | TEST/B/0089/left/0012.png 63 | TEST/A/0076/left/0007.png 64 | TEST/C/0036/left/0015.png 65 | TEST/A/0041/left/0009.png 66 | TEST/C/0090/left/0013.png 67 | TEST/B/0038/left/0008.png 68 | TEST/B/0023/left/0012.png 69 | TEST/C/0137/left/0008.png 70 | TEST/B/0103/left/0006.png 71 | TEST/B/0049/left/0015.png 72 | TEST/A/0069/left/0012.png 73 | TEST/B/0043/left/0013.png 74 | TEST/A/0146/left/0008.png 75 | TEST/A/0008/left/0010.png 76 | TEST/B/0080/left/0008.png 77 | TEST/C/0124/left/0008.png 78 | TEST/B/0140/left/0007.png 79 | TEST/C/0089/left/0013.png 80 | TEST/A/0124/left/0014.png 81 | TEST/C/0015/left/0010.png 82 | TEST/C/0075/left/0007.png 83 | TEST/C/0103/left/0010.png 84 | TEST/A/0085/left/0011.png 85 | TEST/B/0043/left/0010.png 86 | TEST/C/0111/left/0013.png 87 | TEST/C/0072/left/0009.png 88 | TEST/C/0034/left/0015.png 89 | TEST/B/0083/left/0009.png 90 | TEST/A/0020/left/0009.png 91 | TEST/A/0089/left/0008.png 92 | TEST/A/0029/left/0014.png 93 | TEST/A/0031/left/0015.png 94 | TEST/B/0089/left/0015.png 95 | TEST/C/0089/left/0011.png 96 | TEST/B/0036/left/0010.png 97 | TEST/A/0122/left/0010.png 98 | TEST/B/0097/left/0011.png 99 | TEST/A/0143/left/0008.png 100 | TEST/C/0066/left/0006.png 101 | TEST/C/0077/left/0014.png 102 | TEST/C/0048/left/0009.png 103 | TEST/A/0078/left/0010.png 104 | TEST/B/0003/left/0011.png 105 | TEST/B/0076/left/0007.png 106 | TEST/C/0116/left/0010.png 107 | TEST/B/0010/left/0007.png 108 | TEST/C/0069/left/0015.png 109 | TEST/B/0032/left/0008.png 110 | TEST/C/0121/left/0013.png 111 | TEST/C/0047/left/0015.png 112 | TEST/C/0000/left/0006.png 113 | TEST/B/0030/left/0012.png 114 | TEST/B/0091/left/0010.png 115 | TEST/C/0110/left/0007.png 116 | TEST/A/0013/left/0010.png 117 | TEST/B/0133/left/0009.png 118 | TEST/B/0075/left/0010.png 119 | TEST/B/0127/left/0006.png 120 | TEST/A/0098/left/0013.png 121 | TEST/C/0033/left/0010.png 122 | TEST/C/0148/left/0007.png 123 | TEST/C/0054/left/0012.png 124 | TEST/A/0102/left/0014.png 125 | TEST/A/0012/left/0008.png 126 | TEST/C/0138/left/0015.png 127 | TEST/C/0115/left/0014.png 128 | TEST/C/0101/left/0006.png 129 | TEST/A/0065/left/0015.png 130 | TEST/A/0092/left/0008.png 131 | TEST/A/0000/left/0011.png 132 | TEST/C/0117/left/0013.png 133 | TEST/A/0087/left/0007.png 134 | TEST/C/0148/left/0008.png 135 | TEST/C/0137/left/0012.png 136 | TEST/A/0133/left/0008.png 137 | TEST/B/0058/left/0009.png 138 | TEST/C/0107/left/0014.png 139 | TEST/C/0125/left/0008.png 140 | TEST/A/0106/left/0006.png 141 | TEST/A/0083/left/0014.png 142 | TEST/C/0005/left/0015.png 143 | TEST/C/0019/left/0008.png 144 | TEST/A/0116/left/0009.png 145 | TEST/A/0118/left/0014.png 146 | TEST/B/0043/left/0007.png 147 | TEST/C/0017/left/0013.png 148 | TEST/C/0126/left/0009.png 149 | TEST/A/0028/left/0013.png 150 | TEST/C/0109/left/0014.png 151 | TEST/B/0101/left/0014.png 152 | TEST/B/0039/left/0015.png 153 | TEST/C/0144/left/0015.png 154 | TEST/B/0145/left/0006.png 155 | TEST/B/0071/left/0008.png 156 | TEST/A/0081/left/0006.png 157 | TEST/C/0064/left/0010.png 158 | TEST/B/0025/left/0011.png 159 | TEST/A/0130/left/0011.png 160 | TEST/C/0062/left/0007.png 161 | TEST/A/0146/left/0010.png 162 | TEST/C/0105/left/0013.png 163 | TEST/C/0133/left/0012.png 164 | TEST/A/0028/left/0006.png 165 | TEST/B/0119/left/0015.png 166 | TEST/C/0032/left/0006.png 167 | TEST/B/0004/left/0010.png 168 | TEST/C/0089/left/0010.png 169 | TEST/A/0137/left/0007.png 170 | TEST/C/0011/left/0009.png 171 | TEST/A/0128/left/0012.png 172 | TEST/B/0077/left/0007.png 173 | TEST/A/0039/left/0013.png 174 | TEST/C/0121/left/0012.png 175 | TEST/C/0137/left/0006.png 176 | TEST/C/0050/left/0008.png 177 | TEST/A/0114/left/0010.png 178 | TEST/C/0025/left/0010.png 179 | TEST/B/0034/left/0009.png 180 | TEST/A/0119/left/0014.png 181 | TEST/C/0051/left/0006.png 182 | TEST/A/0109/left/0014.png 183 | TEST/A/0127/left/0006.png 184 | TEST/C/0110/left/0014.png 185 | TEST/A/0130/left/0010.png 186 | TEST/C/0009/left/0009.png 187 | TEST/C/0141/left/0008.png 188 | TEST/B/0053/left/0010.png 189 | TEST/A/0052/left/0010.png 190 | TEST/C/0082/left/0011.png 191 | TEST/C/0131/left/0008.png 192 | TEST/C/0063/left/0012.png 193 | TEST/C/0012/left/0010.png 194 | TEST/A/0148/left/0013.png 195 | TEST/B/0128/left/0015.png 196 | TEST/A/0124/left/0011.png 197 | TEST/C/0144/left/0012.png 198 | TEST/A/0000/left/0014.png 199 | TEST/C/0102/left/0013.png 200 | TEST/B/0047/left/0014.png 201 | TEST/B/0112/left/0009.png 202 | TEST/C/0088/left/0009.png 203 | TEST/A/0032/left/0012.png 204 | TEST/C/0130/left/0010.png 205 | TEST/B/0053/left/0014.png 206 | TEST/A/0105/left/0013.png 207 | TEST/C/0104/left/0012.png 208 | TEST/A/0059/left/0014.png 209 | TEST/A/0129/left/0012.png 210 | TEST/C/0051/left/0009.png 211 | TEST/A/0032/left/0009.png 212 | TEST/B/0006/left/0007.png 213 | TEST/B/0113/left/0009.png 214 | TEST/A/0045/left/0006.png 215 | TEST/B/0109/left/0008.png 216 | TEST/C/0069/left/0008.png 217 | TEST/C/0034/left/0008.png 218 | TEST/A/0022/left/0006.png 219 | TEST/C/0004/left/0014.png 220 | TEST/A/0100/left/0008.png 221 | TEST/C/0085/left/0015.png 222 | TEST/A/0147/left/0010.png 223 | TEST/C/0078/left/0010.png 224 | TEST/C/0117/left/0012.png 225 | TEST/B/0129/left/0015.png 226 | TEST/C/0053/left/0011.png 227 | TEST/B/0144/left/0009.png 228 | TEST/B/0124/left/0012.png 229 | TEST/C/0072/left/0011.png 230 | TEST/B/0142/left/0008.png 231 | TEST/A/0048/left/0014.png 232 | TEST/A/0099/left/0010.png 233 | TEST/B/0088/left/0010.png 234 | TEST/B/0066/left/0015.png 235 | TEST/A/0109/left/0009.png 236 | TEST/C/0000/left/0010.png 237 | TEST/A/0108/left/0011.png 238 | TEST/B/0084/left/0011.png 239 | TEST/B/0003/left/0007.png 240 | TEST/C/0118/left/0013.png 241 | TEST/C/0052/left/0012.png 242 | TEST/C/0045/left/0006.png 243 | TEST/C/0073/left/0011.png 244 | TEST/C/0111/left/0007.png 245 | TEST/B/0018/left/0011.png 246 | TEST/B/0126/left/0013.png 247 | TEST/C/0080/left/0014.png 248 | TEST/A/0025/left/0007.png 249 | TEST/B/0021/left/0014.png 250 | TEST/A/0031/left/0014.png 251 | TEST/C/0067/left/0008.png 252 | TEST/B/0012/left/0009.png 253 | TEST/B/0134/left/0007.png 254 | TEST/C/0009/left/0008.png 255 | TEST/C/0063/left/0015.png 256 | TEST/A/0000/left/0015.png 257 | TEST/C/0073/left/0014.png 258 | TEST/C/0078/left/0012.png 259 | TEST/B/0052/left/0012.png 260 | TEST/A/0110/left/0008.png 261 | TEST/C/0105/left/0012.png 262 | TEST/A/0043/left/0008.png 263 | TEST/B/0137/left/0010.png 264 | TEST/C/0124/left/0012.png 265 | TEST/B/0089/left/0009.png 266 | TEST/B/0094/left/0013.png 267 | TEST/C/0131/left/0014.png 268 | TEST/A/0000/left/0009.png 269 | TEST/C/0026/left/0011.png 270 | TEST/B/0026/left/0009.png 271 | TEST/A/0088/left/0007.png 272 | TEST/C/0080/left/0012.png 273 | TEST/A/0087/left/0009.png 274 | TEST/C/0006/left/0013.png 275 | TEST/A/0145/left/0008.png 276 | TEST/B/0029/left/0013.png 277 | TEST/C/0085/left/0007.png 278 | TEST/A/0054/left/0013.png 279 | TEST/A/0107/left/0006.png 280 | TEST/B/0007/left/0007.png 281 | TEST/B/0058/left/0015.png 282 | TEST/A/0097/left/0014.png 283 | TEST/C/0130/left/0014.png 284 | TEST/B/0054/left/0014.png 285 | TEST/C/0127/left/0012.png 286 | TEST/C/0147/left/0011.png 287 | TEST/B/0104/left/0007.png 288 | TEST/A/0012/left/0006.png 289 | TEST/C/0062/left/0008.png 290 | TEST/A/0143/left/0006.png 291 | TEST/B/0139/left/0009.png 292 | TEST/A/0112/left/0015.png 293 | TEST/B/0146/left/0014.png 294 | TEST/A/0123/left/0006.png 295 | TEST/C/0035/left/0008.png 296 | TEST/C/0144/left/0014.png 297 | TEST/A/0071/left/0009.png 298 | TEST/B/0023/left/0014.png 299 | TEST/B/0106/left/0009.png 300 | TEST/B/0043/left/0014.png 301 | TEST/A/0095/left/0007.png 302 | TEST/B/0092/left/0007.png 303 | TEST/B/0054/left/0011.png 304 | TEST/B/0123/left/0012.png 305 | TEST/A/0119/left/0012.png 306 | TEST/B/0129/left/0006.png 307 | TEST/A/0090/left/0012.png 308 | TEST/C/0101/left/0008.png 309 | TEST/B/0082/left/0007.png 310 | TEST/C/0042/left/0007.png 311 | TEST/C/0091/left/0015.png 312 | TEST/C/0021/left/0011.png 313 | TEST/C/0061/left/0009.png 314 | TEST/C/0091/left/0010.png 315 | TEST/B/0063/left/0015.png 316 | TEST/B/0025/left/0013.png 317 | TEST/C/0086/left/0014.png 318 | TEST/B/0068/left/0010.png 319 | TEST/A/0074/left/0009.png 320 | TEST/B/0082/left/0009.png 321 | TEST/A/0047/left/0015.png 322 | TEST/C/0120/left/0009.png 323 | TEST/A/0022/left/0013.png 324 | TEST/A/0010/left/0013.png 325 | TEST/C/0076/left/0015.png 326 | TEST/A/0141/left/0015.png 327 | TEST/B/0133/left/0013.png 328 | TEST/C/0125/left/0013.png 329 | TEST/A/0111/left/0012.png 330 | TEST/A/0087/left/0012.png 331 | TEST/C/0045/left/0013.png 332 | TEST/C/0032/left/0014.png 333 | TEST/B/0133/left/0007.png 334 | TEST/B/0072/left/0010.png 335 | TEST/C/0072/left/0007.png 336 | TEST/B/0042/left/0006.png 337 | TEST/A/0065/left/0014.png 338 | TEST/C/0069/left/0013.png 339 | TEST/A/0037/left/0007.png 340 | TEST/C/0012/left/0008.png 341 | TEST/B/0037/left/0014.png 342 | TEST/B/0012/left/0006.png 343 | TEST/B/0144/left/0006.png 344 | TEST/C/0048/left/0015.png 345 | TEST/A/0051/left/0013.png 346 | TEST/C/0147/left/0012.png 347 | TEST/B/0104/left/0012.png 348 | TEST/B/0000/left/0006.png 349 | TEST/C/0143/left/0012.png 350 | TEST/A/0137/left/0015.png 351 | TEST/A/0006/left/0013.png 352 | TEST/A/0103/left/0008.png 353 | TEST/A/0091/left/0015.png 354 | TEST/C/0108/left/0008.png 355 | TEST/A/0125/left/0006.png 356 | TEST/C/0056/left/0012.png 357 | TEST/B/0104/left/0011.png 358 | TEST/C/0080/left/0007.png 359 | TEST/C/0072/left/0006.png 360 | TEST/B/0145/left/0015.png 361 | TEST/C/0099/left/0010.png 362 | TEST/C/0120/left/0013.png 363 | TEST/A/0115/left/0013.png 364 | TEST/A/0003/left/0013.png 365 | TEST/B/0028/left/0006.png 366 | TEST/B/0036/left/0014.png 367 | TEST/B/0068/left/0015.png 368 | TEST/A/0082/left/0012.png 369 | TEST/C/0059/left/0010.png 370 | TEST/A/0133/left/0006.png 371 | TEST/C/0087/left/0015.png 372 | TEST/A/0100/left/0006.png 373 | TEST/C/0147/left/0009.png 374 | TEST/C/0042/left/0013.png 375 | TEST/B/0007/left/0006.png 376 | TEST/A/0061/left/0014.png 377 | TEST/A/0057/left/0015.png 378 | TEST/B/0103/left/0009.png 379 | TEST/C/0054/left/0009.png 380 | TEST/B/0042/left/0008.png 381 | TEST/A/0092/left/0012.png 382 | TEST/B/0130/left/0007.png 383 | TEST/A/0038/left/0007.png 384 | TEST/C/0138/left/0008.png 385 | TEST/B/0017/left/0006.png 386 | TEST/C/0041/left/0010.png 387 | TEST/A/0019/left/0008.png 388 | TEST/B/0102/left/0011.png 389 | TEST/A/0074/left/0012.png 390 | TEST/C/0104/left/0014.png 391 | TEST/A/0095/left/0015.png 392 | TEST/B/0139/left/0007.png 393 | TEST/C/0046/left/0014.png 394 | TEST/B/0040/left/0007.png 395 | TEST/C/0011/left/0012.png 396 | TEST/A/0135/left/0014.png 397 | TEST/A/0134/left/0008.png 398 | TEST/B/0035/left/0015.png 399 | TEST/B/0000/left/0008.png 400 | TEST/B/0082/left/0013.png 401 | TEST/A/0087/left/0010.png 402 | TEST/C/0137/left/0011.png 403 | TEST/A/0024/left/0014.png 404 | TEST/B/0082/left/0012.png 405 | TEST/A/0083/left/0013.png 406 | TEST/C/0126/left/0014.png 407 | TEST/C/0070/left/0012.png 408 | TEST/C/0121/left/0010.png 409 | TEST/B/0065/left/0013.png 410 | TEST/B/0076/left/0014.png 411 | TEST/A/0110/left/0009.png 412 | TEST/C/0106/left/0006.png 413 | TEST/C/0087/left/0014.png 414 | TEST/B/0066/left/0010.png 415 | TEST/A/0060/left/0010.png 416 | TEST/B/0079/left/0007.png 417 | TEST/C/0093/left/0009.png 418 | TEST/B/0053/left/0012.png 419 | TEST/C/0089/left/0009.png 420 | TEST/B/0068/left/0012.png 421 | TEST/A/0040/left/0015.png 422 | TEST/B/0019/left/0006.png 423 | TEST/B/0036/left/0013.png 424 | TEST/C/0049/left/0012.png 425 | TEST/A/0059/left/0012.png 426 | TEST/C/0028/left/0009.png 427 | TEST/B/0075/left/0008.png 428 | TEST/B/0097/left/0013.png 429 | TEST/A/0062/left/0012.png 430 | TEST/A/0124/left/0007.png 431 | TEST/B/0106/left/0008.png 432 | TEST/B/0055/left/0006.png 433 | TEST/A/0078/left/0014.png 434 | TEST/A/0119/left/0006.png 435 | TEST/C/0091/left/0009.png 436 | TEST/B/0037/left/0007.png 437 | TEST/B/0032/left/0011.png 438 | TEST/B/0010/left/0011.png 439 | TEST/A/0022/left/0009.png 440 | TEST/B/0147/left/0006.png 441 | TEST/C/0102/left/0008.png 442 | TEST/B/0045/left/0014.png 443 | TEST/A/0106/left/0007.png 444 | TEST/B/0140/left/0010.png 445 | TEST/B/0105/left/0006.png 446 | TEST/A/0134/left/0010.png 447 | TEST/A/0120/left/0010.png 448 | TEST/A/0146/left/0007.png 449 | TEST/A/0118/left/0010.png 450 | TEST/C/0033/left/0011.png 451 | TEST/B/0106/left/0010.png 452 | TEST/C/0148/left/0009.png 453 | TEST/B/0008/left/0011.png 454 | TEST/B/0027/left/0009.png 455 | TEST/B/0148/left/0015.png 456 | TEST/B/0123/left/0007.png 457 | TEST/A/0091/left/0013.png 458 | TEST/A/0094/left/0015.png 459 | TEST/B/0016/left/0015.png 460 | TEST/B/0098/left/0013.png 461 | TEST/B/0096/left/0008.png 462 | TEST/B/0062/left/0013.png 463 | TEST/B/0060/left/0008.png 464 | TEST/C/0050/left/0015.png 465 | TEST/C/0093/left/0015.png 466 | TEST/A/0102/left/0007.png 467 | TEST/A/0089/left/0007.png 468 | TEST/B/0102/left/0010.png 469 | TEST/C/0019/left/0009.png 470 | TEST/C/0078/left/0009.png 471 | TEST/B/0064/left/0010.png 472 | TEST/C/0030/left/0011.png 473 | TEST/B/0055/left/0008.png 474 | TEST/C/0147/left/0007.png 475 | TEST/B/0105/left/0010.png 476 | TEST/C/0047/left/0012.png 477 | TEST/C/0127/left/0014.png 478 | TEST/C/0132/left/0007.png 479 | TEST/C/0122/left/0015.png 480 | TEST/B/0017/left/0015.png 481 | TEST/A/0017/left/0014.png 482 | TEST/A/0038/left/0006.png 483 | TEST/B/0083/left/0011.png 484 | TEST/A/0008/left/0007.png 485 | TEST/C/0033/left/0013.png 486 | TEST/A/0041/left/0012.png 487 | TEST/A/0051/left/0012.png 488 | TEST/B/0021/left/0012.png 489 | TEST/B/0004/left/0015.png 490 | TEST/B/0018/left/0010.png 491 | TEST/C/0053/left/0015.png 492 | TEST/C/0148/left/0011.png 493 | TEST/B/0091/left/0009.png 494 | TEST/B/0027/left/0007.png 495 | TEST/B/0088/left/0006.png 496 | TEST/A/0148/left/0010.png 497 | TEST/A/0063/left/0014.png 498 | TEST/C/0076/left/0011.png 499 | TEST/B/0024/left/0010.png 500 | TEST/B/0117/left/0009.png 501 | TEST/B/0038/left/0015.png 502 | TEST/C/0004/left/0013.png 503 | TEST/A/0017/left/0015.png 504 | TEST/A/0033/left/0006.png 505 | TEST/B/0128/left/0014.png 506 | TEST/B/0072/left/0012.png 507 | TEST/A/0046/left/0010.png 508 | TEST/B/0072/left/0007.png 509 | TEST/A/0015/left/0007.png 510 | TEST/C/0028/left/0013.png 511 | TEST/C/0122/left/0006.png 512 | TEST/C/0059/left/0012.png 513 | TEST/C/0006/left/0008.png 514 | TEST/C/0136/left/0007.png 515 | TEST/B/0113/left/0014.png 516 | TEST/C/0115/left/0007.png 517 | TEST/B/0083/left/0015.png 518 | TEST/C/0006/left/0014.png 519 | TEST/C/0067/left/0009.png 520 | TEST/B/0138/left/0013.png 521 | TEST/B/0128/left/0008.png 522 | TEST/B/0055/left/0012.png 523 | TEST/A/0034/left/0008.png 524 | TEST/A/0147/left/0015.png 525 | TEST/C/0038/left/0008.png 526 | TEST/A/0063/left/0007.png 527 | TEST/B/0149/left/0006.png 528 | TEST/A/0100/left/0009.png 529 | TEST/B/0129/left/0010.png 530 | TEST/A/0092/left/0009.png 531 | TEST/B/0096/left/0015.png 532 | TEST/C/0004/left/0007.png 533 | TEST/C/0076/left/0010.png 534 | TEST/B/0014/left/0014.png 535 | TEST/A/0086/left/0013.png 536 | TEST/C/0075/left/0008.png 537 | TEST/B/0073/left/0013.png 538 | TEST/A/0078/left/0011.png 539 | TEST/C/0058/left/0009.png 540 | TEST/C/0125/left/0014.png 541 | TEST/A/0090/left/0010.png 542 | TEST/C/0003/left/0015.png 543 | TEST/B/0032/left/0013.png 544 | TEST/C/0103/left/0011.png 545 | TEST/A/0105/left/0015.png 546 | TEST/A/0121/left/0010.png 547 | TEST/A/0065/left/0008.png 548 | TEST/A/0058/left/0010.png 549 | TEST/C/0068/left/0008.png 550 | TEST/C/0132/left/0013.png 551 | TEST/B/0078/left/0014.png 552 | TEST/C/0041/left/0012.png 553 | TEST/A/0135/left/0006.png 554 | TEST/A/0102/left/0013.png 555 | TEST/C/0115/left/0009.png 556 | TEST/B/0020/left/0013.png 557 | TEST/C/0095/left/0014.png 558 | TEST/B/0015/left/0011.png 559 | TEST/B/0032/left/0014.png 560 | TEST/B/0095/left/0010.png 561 | TEST/A/0134/left/0006.png 562 | TEST/A/0059/left/0009.png 563 | TEST/A/0002/left/0008.png 564 | TEST/A/0122/left/0006.png 565 | TEST/C/0049/left/0011.png 566 | TEST/A/0021/left/0009.png 567 | TEST/B/0135/left/0015.png 568 | TEST/A/0147/left/0013.png 569 | TEST/C/0083/left/0015.png 570 | TEST/C/0090/left/0015.png 571 | TEST/A/0047/left/0006.png 572 | TEST/A/0101/left/0013.png 573 | TEST/C/0113/left/0013.png 574 | TEST/B/0112/left/0012.png 575 | TEST/B/0019/left/0007.png 576 | TEST/A/0069/left/0013.png 577 | TEST/A/0121/left/0011.png 578 | TEST/C/0092/left/0008.png 579 | TEST/A/0015/left/0008.png 580 | TEST/A/0067/left/0011.png 581 | TEST/B/0134/left/0013.png 582 | TEST/C/0110/left/0012.png 583 | TEST/C/0017/left/0007.png 584 | TEST/A/0050/left/0008.png 585 | TEST/A/0009/left/0008.png 586 | TEST/C/0069/left/0010.png 587 | TEST/C/0136/left/0010.png 588 | TEST/B/0083/left/0010.png 589 | TEST/B/0067/left/0013.png 590 | TEST/A/0029/left/0006.png 591 | TEST/A/0137/left/0012.png 592 | TEST/A/0128/left/0013.png 593 | TEST/A/0085/left/0012.png 594 | TEST/B/0007/left/0012.png 595 | TEST/B/0048/left/0013.png 596 | TEST/A/0089/left/0009.png 597 | TEST/B/0033/left/0010.png 598 | TEST/A/0082/left/0014.png 599 | TEST/C/0090/left/0010.png 600 | TEST/C/0085/left/0008.png 601 | TEST/B/0043/left/0012.png 602 | TEST/A/0017/left/0013.png 603 | TEST/B/0047/left/0007.png 604 | TEST/B/0004/left/0006.png 605 | TEST/C/0066/left/0015.png 606 | TEST/A/0069/left/0014.png 607 | TEST/A/0054/left/0009.png 608 | TEST/B/0026/left/0011.png 609 | TEST/C/0045/left/0009.png 610 | TEST/C/0133/left/0014.png 611 | TEST/A/0015/left/0009.png 612 | TEST/A/0040/left/0013.png 613 | TEST/C/0110/left/0010.png 614 | TEST/C/0076/left/0007.png 615 | TEST/C/0070/left/0010.png 616 | TEST/B/0147/left/0015.png 617 | TEST/B/0098/left/0015.png 618 | TEST/B/0081/left/0009.png 619 | TEST/A/0077/left/0007.png 620 | TEST/B/0128/left/0012.png 621 | TEST/A/0047/left/0009.png 622 | TEST/C/0092/left/0013.png 623 | TEST/A/0002/left/0014.png 624 | TEST/A/0147/left/0006.png 625 | TEST/B/0008/left/0012.png 626 | TEST/B/0128/left/0009.png 627 | TEST/B/0052/left/0011.png 628 | TEST/A/0148/left/0011.png 629 | TEST/A/0010/left/0010.png 630 | TEST/B/0024/left/0013.png 631 | TEST/B/0080/left/0009.png 632 | TEST/C/0041/left/0007.png 633 | TEST/B/0122/left/0011.png 634 | TEST/C/0117/left/0011.png 635 | TEST/B/0054/left/0013.png 636 | TEST/B/0096/left/0010.png 637 | TEST/B/0063/left/0011.png 638 | TEST/B/0035/left/0006.png 639 | TEST/A/0014/left/0013.png 640 | TEST/A/0104/left/0008.png 641 | TEST/C/0112/left/0012.png 642 | TEST/A/0143/left/0009.png 643 | TEST/A/0062/left/0011.png 644 | TEST/A/0039/left/0008.png 645 | TEST/A/0012/left/0009.png 646 | TEST/A/0035/left/0011.png 647 | TEST/B/0071/left/0006.png 648 | TEST/B/0027/left/0012.png 649 | TEST/B/0052/left/0014.png 650 | TEST/C/0009/left/0006.png 651 | TEST/C/0037/left/0012.png 652 | TEST/C/0003/left/0013.png 653 | TEST/C/0054/left/0011.png 654 | TEST/A/0051/left/0010.png 655 | TEST/C/0027/left/0008.png 656 | TEST/A/0021/left/0010.png 657 | TEST/B/0069/left/0012.png 658 | TEST/B/0145/left/0011.png 659 | TEST/A/0030/left/0012.png 660 | TEST/C/0124/left/0006.png 661 | TEST/A/0131/left/0008.png 662 | TEST/B/0047/left/0011.png 663 | TEST/A/0068/left/0007.png 664 | TEST/B/0058/left/0008.png 665 | TEST/C/0093/left/0008.png 666 | TEST/A/0138/left/0013.png 667 | TEST/B/0019/left/0008.png 668 | TEST/C/0065/left/0015.png 669 | TEST/A/0129/left/0011.png 670 | TEST/C/0002/left/0007.png 671 | TEST/B/0033/left/0009.png 672 | TEST/B/0011/left/0013.png 673 | TEST/C/0038/left/0013.png 674 | TEST/B/0124/left/0007.png 675 | TEST/A/0094/left/0007.png 676 | TEST/B/0133/left/0011.png 677 | TEST/C/0093/left/0010.png 678 | TEST/A/0079/left/0008.png 679 | TEST/A/0057/left/0008.png 680 | TEST/C/0045/left/0012.png 681 | TEST/B/0128/left/0007.png 682 | TEST/B/0120/left/0014.png 683 | TEST/B/0108/left/0007.png 684 | TEST/B/0098/left/0006.png 685 | TEST/A/0010/left/0009.png 686 | TEST/B/0061/left/0014.png 687 | TEST/A/0012/left/0012.png 688 | TEST/B/0040/left/0013.png 689 | TEST/A/0122/left/0011.png 690 | TEST/B/0029/left/0015.png 691 | TEST/B/0023/left/0009.png 692 | TEST/A/0102/left/0009.png 693 | TEST/A/0062/left/0006.png 694 | TEST/C/0103/left/0009.png 695 | TEST/C/0097/left/0012.png 696 | TEST/A/0001/left/0011.png 697 | TEST/B/0086/left/0015.png 698 | TEST/B/0111/left/0014.png 699 | TEST/C/0146/left/0009.png 700 | TEST/C/0103/left/0008.png 701 | TEST/A/0105/left/0007.png 702 | TEST/C/0093/left/0014.png 703 | TEST/A/0141/left/0011.png 704 | TEST/B/0054/left/0008.png 705 | TEST/B/0124/left/0011.png 706 | TEST/B/0025/left/0008.png 707 | TEST/A/0035/left/0014.png 708 | TEST/C/0117/left/0010.png 709 | TEST/A/0053/left/0006.png 710 | TEST/C/0053/left/0010.png 711 | TEST/A/0138/left/0009.png 712 | TEST/C/0149/left/0013.png 713 | TEST/C/0149/left/0015.png 714 | TEST/B/0091/left/0008.png 715 | TEST/C/0116/left/0009.png 716 | TEST/C/0079/left/0012.png 717 | TEST/C/0015/left/0008.png 718 | TEST/B/0142/left/0014.png 719 | TEST/A/0099/left/0006.png 720 | TEST/B/0052/left/0015.png 721 | TEST/A/0064/left/0008.png 722 | TEST/A/0093/left/0013.png 723 | TEST/A/0050/left/0015.png 724 | TEST/A/0053/left/0013.png 725 | TEST/B/0133/left/0008.png 726 | TEST/B/0114/left/0008.png 727 | TEST/A/0118/left/0015.png 728 | TEST/B/0029/left/0009.png 729 | TEST/A/0102/left/0011.png 730 | TEST/B/0000/left/0014.png 731 | TEST/A/0099/left/0014.png 732 | TEST/B/0061/left/0010.png 733 | TEST/B/0099/left/0011.png 734 | TEST/B/0080/left/0011.png 735 | TEST/B/0130/left/0013.png 736 | TEST/A/0075/left/0013.png 737 | TEST/B/0064/left/0007.png 738 | TEST/A/0003/left/0014.png 739 | TEST/C/0143/left/0007.png 740 | TEST/A/0065/left/0010.png 741 | TEST/B/0042/left/0014.png 742 | TEST/B/0110/left/0012.png 743 | TEST/B/0041/left/0011.png 744 | TEST/A/0106/left/0008.png 745 | TEST/C/0012/left/0015.png 746 | TEST/B/0062/left/0006.png 747 | TEST/A/0085/left/0007.png 748 | TEST/A/0122/left/0007.png 749 | TEST/B/0136/left/0010.png 750 | TEST/B/0032/left/0010.png 751 | TEST/B/0068/left/0014.png 752 | TEST/B/0057/left/0008.png 753 | TEST/B/0099/left/0014.png 754 | TEST/C/0112/left/0006.png 755 | TEST/C/0073/left/0013.png 756 | TEST/A/0140/left/0008.png 757 | TEST/C/0118/left/0009.png 758 | TEST/C/0035/left/0009.png 759 | TEST/B/0064/left/0006.png 760 | TEST/B/0109/left/0007.png 761 | TEST/C/0103/left/0015.png 762 | TEST/B/0059/left/0008.png 763 | TEST/A/0115/left/0011.png 764 | TEST/B/0121/left/0009.png 765 | TEST/A/0039/left/0010.png 766 | TEST/A/0067/left/0007.png 767 | TEST/C/0031/left/0013.png 768 | TEST/B/0094/left/0012.png 769 | TEST/A/0008/left/0014.png 770 | TEST/A/0040/left/0010.png 771 | TEST/A/0015/left/0010.png 772 | TEST/A/0088/left/0013.png 773 | TEST/C/0033/left/0008.png 774 | TEST/B/0102/left/0015.png 775 | TEST/C/0133/left/0013.png 776 | TEST/A/0042/left/0006.png 777 | TEST/B/0013/left/0006.png 778 | TEST/A/0124/left/0009.png 779 | TEST/B/0110/left/0013.png 780 | TEST/B/0070/left/0012.png 781 | TEST/B/0010/left/0013.png 782 | TEST/A/0011/left/0010.png 783 | TEST/A/0104/left/0011.png 784 | TEST/A/0077/left/0015.png 785 | TEST/C/0085/left/0011.png 786 | TEST/A/0059/left/0006.png 787 | TEST/C/0087/left/0010.png 788 | TEST/C/0139/left/0009.png 789 | TEST/A/0007/left/0013.png 790 | TEST/B/0062/left/0011.png 791 | TEST/C/0061/left/0006.png 792 | TEST/A/0103/left/0015.png 793 | TEST/A/0117/left/0008.png 794 | TEST/C/0011/left/0011.png 795 | TEST/A/0110/left/0012.png 796 | TEST/A/0081/left/0010.png 797 | TEST/A/0063/left/0015.png 798 | TEST/C/0119/left/0007.png 799 | TEST/B/0122/left/0013.png 800 | TEST/A/0068/left/0008.png 801 | TEST/C/0041/left/0006.png 802 | TEST/C/0076/left/0006.png 803 | TEST/B/0148/left/0006.png 804 | TEST/C/0068/left/0015.png 805 | TEST/A/0036/left/0009.png 806 | TEST/B/0006/left/0015.png 807 | TEST/B/0051/left/0011.png 808 | TEST/C/0041/left/0008.png 809 | TEST/A/0068/left/0011.png 810 | TEST/B/0033/left/0011.png 811 | TEST/C/0021/left/0012.png 812 | TEST/B/0045/left/0009.png 813 | TEST/C/0005/left/0012.png 814 | TEST/B/0032/left/0009.png 815 | TEST/C/0091/left/0011.png 816 | TEST/B/0097/left/0006.png 817 | TEST/C/0093/left/0012.png 818 | TEST/B/0074/left/0011.png 819 | TEST/A/0032/left/0006.png 820 | TEST/A/0046/left/0008.png 821 | TEST/C/0138/left/0007.png 822 | TEST/C/0065/left/0007.png 823 | TEST/B/0101/left/0011.png 824 | TEST/A/0105/left/0011.png 825 | TEST/B/0044/left/0006.png 826 | TEST/A/0036/left/0007.png 827 | TEST/C/0097/left/0015.png 828 | TEST/C/0111/left/0006.png 829 | TEST/B/0045/left/0006.png 830 | TEST/C/0121/left/0009.png 831 | TEST/B/0050/left/0008.png 832 | TEST/C/0006/left/0011.png 833 | TEST/A/0019/left/0007.png 834 | TEST/A/0053/left/0011.png 835 | TEST/A/0098/left/0007.png 836 | TEST/C/0131/left/0015.png 837 | TEST/A/0062/left/0010.png 838 | TEST/A/0123/left/0010.png 839 | TEST/C/0005/left/0014.png 840 | TEST/A/0055/left/0012.png 841 | TEST/A/0008/left/0008.png 842 | TEST/C/0139/left/0010.png 843 | TEST/B/0021/left/0015.png 844 | TEST/C/0013/left/0012.png 845 | TEST/B/0130/left/0009.png 846 | TEST/C/0008/left/0014.png 847 | TEST/A/0056/left/0010.png 848 | TEST/B/0113/left/0010.png 849 | TEST/C/0108/left/0010.png 850 | TEST/C/0075/left/0011.png 851 | TEST/A/0088/left/0010.png 852 | TEST/C/0030/left/0012.png 853 | TEST/A/0033/left/0014.png 854 | TEST/C/0060/left/0012.png 855 | TEST/C/0149/left/0014.png 856 | TEST/C/0053/left/0007.png 857 | TEST/A/0069/left/0010.png 858 | TEST/A/0120/left/0011.png 859 | TEST/C/0002/left/0013.png 860 | TEST/A/0037/left/0006.png 861 | TEST/A/0130/left/0008.png 862 | TEST/A/0113/left/0007.png 863 | TEST/C/0044/left/0011.png 864 | TEST/B/0002/left/0006.png 865 | TEST/A/0094/left/0008.png 866 | TEST/C/0012/left/0012.png 867 | TEST/B/0095/left/0012.png 868 | TEST/A/0058/left/0006.png 869 | TEST/B/0050/left/0010.png 870 | TEST/A/0094/left/0011.png 871 | TEST/C/0060/left/0013.png 872 | TEST/B/0000/left/0011.png 873 | TEST/C/0084/left/0013.png 874 | TEST/B/0024/left/0011.png 875 | TEST/A/0091/left/0014.png 876 | TEST/A/0111/left/0011.png 877 | TEST/A/0006/left/0006.png 878 | TEST/C/0104/left/0011.png 879 | TEST/A/0076/left/0014.png 880 | TEST/A/0110/left/0014.png 881 | TEST/A/0123/left/0008.png 882 | TEST/A/0006/left/0014.png 883 | TEST/B/0023/left/0013.png 884 | TEST/A/0032/left/0014.png 885 | TEST/A/0046/left/0007.png 886 | TEST/C/0048/left/0007.png 887 | TEST/A/0109/left/0013.png 888 | TEST/A/0028/left/0010.png 889 | TEST/A/0147/left/0012.png 890 | TEST/C/0046/left/0015.png 891 | TEST/C/0022/left/0015.png 892 | TEST/A/0111/left/0010.png 893 | TEST/B/0048/left/0011.png 894 | TEST/B/0067/left/0009.png 895 | TEST/C/0025/left/0009.png 896 | TEST/A/0081/left/0009.png 897 | TEST/B/0017/left/0010.png 898 | TEST/C/0129/left/0006.png 899 | TEST/C/0127/left/0008.png 900 | TEST/B/0146/left/0015.png 901 | TEST/A/0101/left/0008.png 902 | TEST/C/0102/left/0014.png 903 | TEST/A/0005/left/0015.png 904 | TEST/C/0081/left/0015.png 905 | TEST/A/0079/left/0015.png 906 | TEST/C/0144/left/0010.png 907 | TEST/A/0109/left/0010.png 908 | TEST/B/0033/left/0008.png 909 | TEST/C/0008/left/0008.png 910 | TEST/A/0035/left/0008.png 911 | TEST/C/0023/left/0011.png 912 | TEST/C/0132/left/0011.png 913 | TEST/A/0044/left/0014.png 914 | TEST/C/0024/left/0011.png 915 | TEST/B/0135/left/0012.png 916 | TEST/A/0112/left/0014.png 917 | TEST/C/0056/left/0007.png 918 | TEST/C/0040/left/0012.png 919 | TEST/C/0015/left/0011.png 920 | TEST/C/0089/left/0015.png 921 | TEST/C/0105/left/0010.png 922 | TEST/A/0094/left/0009.png 923 | TEST/A/0105/left/0010.png 924 | TEST/B/0085/left/0007.png 925 | TEST/C/0050/left/0014.png 926 | TEST/C/0017/left/0006.png 927 | TEST/C/0045/left/0010.png 928 | TEST/A/0086/left/0015.png 929 | TEST/B/0120/left/0007.png 930 | TEST/A/0135/left/0010.png 931 | TEST/A/0001/left/0015.png 932 | TEST/C/0084/left/0009.png 933 | TEST/B/0072/left/0014.png 934 | TEST/A/0018/left/0006.png 935 | TEST/A/0026/left/0008.png 936 | TEST/A/0126/left/0011.png 937 | TEST/B/0046/left/0007.png 938 | TEST/C/0134/left/0010.png 939 | TEST/C/0004/left/0015.png 940 | TEST/A/0116/left/0014.png 941 | TEST/C/0139/left/0012.png 942 | TEST/C/0052/left/0015.png 943 | TEST/A/0081/left/0014.png 944 | TEST/C/0120/left/0012.png 945 | TEST/A/0080/left/0013.png 946 | TEST/C/0068/left/0009.png 947 | TEST/C/0009/left/0010.png 948 | TEST/C/0035/left/0014.png 949 | TEST/C/0002/left/0012.png 950 | TEST/C/0003/left/0012.png 951 | TEST/B/0005/left/0008.png 952 | TEST/A/0107/left/0010.png 953 | TEST/B/0061/left/0006.png 954 | TEST/C/0065/left/0008.png 955 | TEST/B/0053/left/0008.png 956 | TEST/B/0042/left/0015.png 957 | TEST/C/0046/left/0007.png 958 | TEST/B/0085/left/0011.png 959 | TEST/C/0081/left/0014.png 960 | TEST/B/0139/left/0014.png 961 | TEST/C/0061/left/0011.png 962 | TEST/C/0109/left/0013.png 963 | TEST/A/0127/left/0007.png 964 | TEST/A/0132/left/0013.png 965 | TEST/C/0127/left/0009.png 966 | TEST/B/0026/left/0012.png 967 | TEST/A/0012/left/0011.png 968 | TEST/A/0021/left/0012.png 969 | TEST/B/0149/left/0009.png 970 | TEST/A/0148/left/0012.png 971 | TEST/A/0099/left/0007.png 972 | TEST/C/0120/left/0007.png 973 | TEST/A/0041/left/0015.png 974 | TEST/B/0145/left/0009.png 975 | TEST/A/0082/left/0009.png 976 | TEST/C/0033/left/0006.png 977 | TEST/B/0084/left/0007.png 978 | TEST/B/0049/left/0008.png 979 | TEST/A/0014/left/0009.png 980 | TEST/A/0122/left/0013.png 981 | TEST/C/0041/left/0014.png 982 | TEST/B/0050/left/0007.png 983 | TEST/A/0138/left/0011.png 984 | TEST/C/0067/left/0015.png 985 | TEST/B/0062/left/0009.png 986 | TEST/C/0099/left/0011.png 987 | TEST/C/0096/left/0007.png 988 | TEST/A/0006/left/0011.png 989 | TEST/B/0016/left/0011.png 990 | TEST/B/0054/left/0007.png 991 | TEST/C/0027/left/0009.png 992 | TEST/B/0130/left/0011.png 993 | TEST/C/0097/left/0009.png 994 | TEST/A/0011/left/0013.png 995 | TEST/C/0141/left/0010.png 996 | TEST/B/0037/left/0011.png 997 | TEST/A/0070/left/0008.png 998 | TEST/B/0123/left/0009.png 999 | TEST/B/0046/left/0012.png 1000 | TEST/C/0008/left/0007.png 1001 | -------------------------------------------------------------------------------- /models/GANet11.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.init as init 4 | from libs.GANet.modules.GANet import DisparityRegression, GetCostVolume 5 | from libs.GANet.modules.GANet import MyNormalize 6 | from libs.GANet.modules.GANet import SGA 7 | from libs.GANet.modules.GANet import LGA, LGA2, LGA3 8 | from libs.sync_bn.modules.sync_bn import BatchNorm2d, BatchNorm3d 9 | 10 | import torch.nn.functional as F 11 | from torch.autograd import Variable 12 | import numpy as np 13 | class BasicConv(nn.Module): 14 | 15 | def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, bn=True, relu=True, **kwargs): 16 | super(BasicConv, self).__init__() 17 | # print(in_channels, out_channels, deconv, is_3d, bn, relu, kwargs) 18 | self.relu = relu 19 | self.use_bn = bn 20 | if is_3d: 21 | if deconv: 22 | self.conv = nn.ConvTranspose3d(in_channels, out_channels, bias=False, **kwargs) 23 | else: 24 | self.conv = nn.Conv3d(in_channels, out_channels, bias=False, **kwargs) 25 | self.bn = BatchNorm3d(out_channels) 26 | else: 27 | if deconv: 28 | self.conv = nn.ConvTranspose2d(in_channels, out_channels, bias=False, **kwargs) 29 | else: 30 | self.conv = nn.Conv2d(in_channels, out_channels, bias=False, **kwargs) 31 | self.bn = BatchNorm2d(out_channels) 32 | 33 | def forward(self, x): 34 | x = self.conv(x) 35 | if self.use_bn: 36 | x = self.bn(x) 37 | if self.relu: 38 | x = F.relu(x, inplace=True) 39 | return x 40 | 41 | 42 | class Conv2x(nn.Module): 43 | 44 | def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, concat=True, bn=True, relu=True): 45 | super(Conv2x, self).__init__() 46 | self.concat = concat 47 | 48 | if deconv and is_3d: 49 | kernel = (3, 4, 4) 50 | elif deconv: 51 | kernel = 4 52 | else: 53 | kernel = 3 54 | self.conv1 = BasicConv(in_channels, out_channels, deconv, is_3d, bn=True, relu=True, kernel_size=kernel, stride=2, padding=1) 55 | 56 | if self.concat: 57 | self.conv2 = BasicConv(out_channels*2, out_channels, False, is_3d, bn, relu, kernel_size=3, stride=1, padding=1) 58 | else: 59 | self.conv2 = BasicConv(out_channels, out_channels, False, is_3d, bn, relu, kernel_size=3, stride=1, padding=1) 60 | 61 | def forward(self, x, rem): 62 | x = self.conv1(x) 63 | assert(x.size() == rem.size()) 64 | if self.concat: 65 | x = torch.cat((x, rem), 1) 66 | else: 67 | x = x + rem 68 | x = self.conv2(x) 69 | return x 70 | 71 | 72 | class Feature(nn.Module): 73 | def __init__(self): 74 | super(Feature, self).__init__() 75 | 76 | self.conv_start = nn.Sequential( 77 | BasicConv(3, 32, kernel_size=3, padding=1), 78 | BasicConv(32, 32, kernel_size=5, stride=3, padding=2), 79 | BasicConv(32, 32, kernel_size=3, padding=1)) 80 | self.conv1a = BasicConv(32, 48, kernel_size=3, stride=2, padding=1) 81 | self.conv2a = BasicConv(48, 64, kernel_size=3, stride=2, padding=1) 82 | self.conv3a = BasicConv(64, 96, kernel_size=3, stride=2, padding=1) 83 | self.conv4a = BasicConv(96, 128, kernel_size=3, stride=2, padding=1) 84 | 85 | self.deconv4a = Conv2x(128, 96, deconv=True) 86 | self.deconv3a = Conv2x(96, 64, deconv=True) 87 | self.deconv2a = Conv2x(64, 48, deconv=True) 88 | self.deconv1a = Conv2x(48, 32, deconv=True) 89 | 90 | self.conv1b = Conv2x(32, 48) 91 | self.conv2b = Conv2x(48, 64) 92 | self.conv3b = Conv2x(64, 96) 93 | self.conv4b = Conv2x(96, 128) 94 | 95 | self.deconv4b = Conv2x(128, 96, deconv=True) 96 | self.deconv3b = Conv2x(96, 64, deconv=True) 97 | self.deconv2b = Conv2x(64, 48, deconv=True) 98 | self.deconv1b = Conv2x(48, 32, deconv=True) 99 | 100 | def forward(self, x): 101 | x = self.conv_start(x) 102 | rem0 = x 103 | x = self.conv1a(x) 104 | rem1 = x 105 | x = self.conv2a(x) 106 | rem2 = x 107 | x = self.conv3a(x) 108 | rem3 = x 109 | x = self.conv4a(x) 110 | rem4 = x 111 | x = self.deconv4a(x, rem3) 112 | rem3 = x 113 | 114 | x = self.deconv3a(x, rem2) 115 | rem2 = x 116 | x = self.deconv2a(x, rem1) 117 | rem1 = x 118 | x = self.deconv1a(x, rem0) 119 | rem0 = x 120 | 121 | x = self.conv1b(x, rem1) 122 | rem1 = x 123 | x = self.conv2b(x, rem2) 124 | rem2 = x 125 | x = self.conv3b(x, rem3) 126 | rem3 = x 127 | x = self.conv4b(x, rem4) 128 | 129 | x = self.deconv4b(x, rem3) 130 | x = self.deconv3b(x, rem2) 131 | x = self.deconv2b(x, rem1) 132 | x = self.deconv1b(x, rem0) 133 | 134 | return x 135 | 136 | class Guidance(nn.Module): 137 | def __init__(self): 138 | super(Guidance, self).__init__() 139 | 140 | self.conv0 = BasicConv(64, 16, kernel_size=3, padding=1) 141 | self.conv1 = nn.Sequential( 142 | BasicConv(16, 32, kernel_size=5, stride=3, padding=2), 143 | BasicConv(32, 32, kernel_size=3, padding=1)) 144 | 145 | self.conv2 = BasicConv(32, 32, kernel_size=3, padding=1) 146 | 147 | self.conv11 = nn.Sequential(BasicConv(32, 48, kernel_size=3, stride=2, padding=1), 148 | BasicConv(48, 48, kernel_size=3, padding=1)) 149 | self.conv12 = BasicConv(48, 48, kernel_size=3, padding=1) 150 | 151 | self.weight_sg1 = nn.Conv2d(32, 640, (3, 3), (1, 1), (1, 1), bias=False) 152 | self.weight_sg2 = nn.Conv2d(32, 640, (3, 3), (1, 1), (1, 1), bias=False) 153 | 154 | self.weight_sg11 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 155 | self.weight_sg12 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 156 | 157 | self.weight_lg1 = nn.Sequential(BasicConv(16, 16, kernel_size=3, padding=1), 158 | nn.Conv2d(16, 75, (3, 3), (1, 1), (1, 1) ,bias=False)) 159 | self.weight_lg2 = nn.Sequential(BasicConv(16, 16, kernel_size=3, padding=1), 160 | nn.Conv2d(16, 75, (3, 3), (1, 1), (1, 1) ,bias=False)) 161 | 162 | def forward(self, x): 163 | x = self.conv0(x) 164 | rem = x 165 | x = self.conv1(x) 166 | sg1 = self.weight_sg1(x) 167 | x = self.conv2(x) 168 | sg2 = self.weight_sg2(x) 169 | 170 | x = self.conv11(x) 171 | sg11 = self.weight_sg11(x) 172 | x = self.conv12(x) 173 | sg12 = self.weight_sg12(x) 174 | 175 | lg1 = self.weight_lg1(rem) 176 | lg2 = self.weight_lg2(rem) 177 | 178 | return dict([ 179 | ('sg1', sg1), 180 | ('sg2', sg2), 181 | ('sg11', sg11), 182 | ('sg12', sg12), 183 | ('lg1', lg1), 184 | ('lg2', lg2)]) 185 | 186 | class Disp(nn.Module): 187 | 188 | def __init__(self, maxdisp=192): 189 | super(Disp, self).__init__() 190 | self.maxdisp = maxdisp 191 | self.softmax = nn.Softmin(dim=1) 192 | self.disparity = DisparityRegression(maxdisp=self.maxdisp) 193 | # self.conv32x1 = BasicConv(32, 1, kernel_size=3) 194 | self.conv32x1 = nn.Conv3d(32, 1, (3, 3, 3), (1, 1, 1), (1, 1, 1), bias=False) 195 | 196 | def forward(self, x): 197 | x = F.interpolate(self.conv32x1(x), [self.maxdisp+1, x.size()[3]*3, x.size()[4]*3], mode='trilinear', align_corners=False) 198 | x = torch.squeeze(x, 1) 199 | x = self.softmax(x) 200 | 201 | return self.disparity(x) 202 | 203 | class DispAgg(nn.Module): 204 | 205 | def __init__(self, maxdisp=192): 206 | super(DispAgg, self).__init__() 207 | self.maxdisp = maxdisp 208 | self.LGA3 = LGA3(radius=2) 209 | self.LGA2 = LGA2(radius=2) 210 | self.LGA = LGA(radius=2) 211 | self.softmax = nn.Softmin(dim=1) 212 | self.disparity = DisparityRegression(maxdisp=self.maxdisp) 213 | # self.conv32x1 = BasicConv(32, 1, kernel_size=3) 214 | self.conv32x1=nn.Conv3d(32, 1, (3, 3, 3), (1, 1, 1), (1, 1, 1), bias=False) 215 | 216 | def lga(self, x, g): 217 | g = F.normalize(g, p=1, dim=1) 218 | x = self.LGA2(x, g) 219 | return x 220 | 221 | def forward(self, x, lg1, lg2): 222 | x = F.interpolate(self.conv32x1(x), [self.maxdisp+1, x.size()[3]*3, x.size()[4]*3], mode='trilinear', align_corners=False) 223 | x = torch.squeeze(x, 1) 224 | assert(lg1.size() == lg2.size()) 225 | x = self.lga(x, lg1) 226 | x = self.softmax(x) 227 | x = self.lga(x, lg2) 228 | x = F.normalize(x, p=1, dim=1) 229 | return self.disparity(x) 230 | 231 | class SGABlock(nn.Module): 232 | def __init__(self, channels=32, refine=False): 233 | super(SGABlock, self).__init__() 234 | self.refine = refine 235 | if self.refine: 236 | self.bn_relu = nn.Sequential(BatchNorm3d(channels), 237 | nn.ReLU(inplace=True)) 238 | self.conv_refine = BasicConv(channels, channels, is_3d=True, kernel_size=3, padding=1, relu=False) 239 | # self.conv_refine1 = BasicConv(8, 8, is_3d=True, kernel_size=1, padding=1) 240 | else: 241 | self.bn = BatchNorm3d(channels) 242 | self.SGA=SGA() 243 | self.relu = nn.ReLU(inplace=True) 244 | def forward(self, x, g): 245 | rem = x 246 | k1, k2, k3, k4 = torch.split(g, (x.size()[1]*5, x.size()[1]*5, x.size()[1]*5, x.size()[1]*5), 1) 247 | k1 = F.normalize(k1.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 248 | k2 = F.normalize(k2.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 249 | k3 = F.normalize(k3.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 250 | k4 = F.normalize(k4.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 251 | x = self.SGA(x, k1, k2, k3, k4) 252 | if self.refine: 253 | x = self.bn_relu(x) 254 | x = self.conv_refine(x) 255 | else: 256 | x = self.bn(x) 257 | assert(x.size() == rem.size()) 258 | x += rem 259 | return self.relu(x) 260 | # return self.bn_relu(x) 261 | 262 | 263 | class CostAggregation(nn.Module): 264 | def __init__(self, maxdisp=192): 265 | super(CostAggregation, self).__init__() 266 | self.maxdisp = maxdisp 267 | self.conv_start = BasicConv(64, 32, is_3d=True, kernel_size=3, padding=1, relu=False) 268 | 269 | self.conv1a = BasicConv(32, 48, is_3d=True, kernel_size=3, stride=2, padding=1) 270 | self.conv2a = BasicConv(48, 64, is_3d=True, kernel_size=3, stride=2, padding=1) 271 | # self.conv3a = BasicConv(64, 96, is_3d=True, kernel_size=3, stride=2, padding=1) 272 | 273 | self.deconv1a = Conv2x(48, 32, deconv=True, is_3d=True, relu=False) 274 | self.deconv2a = Conv2x(64, 48, deconv=True, is_3d=True) 275 | # self.deconv0a = Conv2x(8, 8, deconv=True, is_3d=True) 276 | 277 | self.sga1 = SGABlock(refine=True) 278 | self.sga2 = SGABlock(refine=True) 279 | 280 | self.sga11 = SGABlock(channels=48, refine=True) 281 | self.sga12 = SGABlock(channels=48, refine=True) 282 | 283 | self.disp0 = Disp(self.maxdisp) 284 | self.disp1 = DispAgg(self.maxdisp) 285 | 286 | 287 | def forward(self, x, g): 288 | 289 | x = self.conv_start(x) 290 | x = self.sga1(x, g['sg1']) 291 | rem0 = x 292 | 293 | if self.training: 294 | disp0 = self.disp0(x) 295 | 296 | x = self.conv1a(x) 297 | x = self.sga11(x, g['sg11']) 298 | rem1 = x 299 | x = self.conv2a(x) 300 | x = self.deconv2a(x, rem1) 301 | x = self.sga12(x, g['sg12']) 302 | x = self.deconv1a(x, rem0) 303 | x = self.sga2(x, g['sg2']) 304 | disp1 = self.disp1(x, g['lg1'], g['lg2']) 305 | 306 | if self.training: 307 | return disp0, disp1 308 | else: 309 | return disp1 310 | 311 | class GANet(nn.Module): 312 | def __init__(self, maxdisp=192): 313 | super(GANet, self).__init__() 314 | self.maxdisp = maxdisp 315 | self.conv_start = nn.Sequential(BasicConv(3, 16, kernel_size=3, padding=1), 316 | BasicConv(16, 32, kernel_size=3, padding=1)) 317 | 318 | self.conv_x = BasicConv(32, 32, kernel_size=3, padding=1) 319 | self.conv_y = BasicConv(32, 32, kernel_size=3, padding=1) 320 | self.conv_refine = nn.Conv2d(32, 32, (3, 3), (1, 1), (1,1), bias=False) 321 | self.bn_relu = nn.Sequential(BatchNorm2d(32), 322 | nn.ReLU(inplace=True)) 323 | self.feature = Feature() 324 | self.guidance = Guidance() 325 | self.cost_agg = CostAggregation(self.maxdisp) 326 | self.cv = GetCostVolume(int(self.maxdisp/3)) 327 | 328 | for m in self.modules(): 329 | if isinstance(m, (nn.Conv2d, nn.Conv3d)): 330 | nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu') 331 | elif isinstance(m, (BatchNorm2d, BatchNorm3d)): 332 | nn.init.constant_(m.weight, 1) 333 | nn.init.constant_(m.bias, 0) 334 | 335 | def forward(self, x, y): 336 | g = self.conv_start(x) 337 | x = self.feature(x) 338 | 339 | rem = x 340 | x = self.conv_x(x) 341 | 342 | y = self.feature(y) 343 | y = self.conv_y(y) 344 | 345 | x = self.cv(x,y) 346 | 347 | x1 = self.conv_refine(rem) 348 | x1 = F.interpolate(x1, [x1.size()[2] * 3,x1.size()[3] * 3], mode='bilinear', align_corners=False) 349 | x1 = self.bn_relu(x1) 350 | g = torch.cat((g, x1), 1) 351 | g = self.guidance(g) 352 | 353 | return self.cost_agg(x, g) 354 | -------------------------------------------------------------------------------- /models/GANet_deep.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.init as init 4 | from libs.GANet.modules.GANet import DisparityRegression, GetCostVolume 5 | from libs.GANet.modules.GANet import MyNormalize 6 | from libs.GANet.modules.GANet import SGA 7 | from libs.GANet.modules.GANet import LGA, LGA2, LGA3 8 | from libs.sync_bn.modules.sync_bn import BatchNorm2d, BatchNorm3d 9 | 10 | import torch.nn.functional as F 11 | from torch.autograd import Variable 12 | import numpy as np 13 | 14 | 15 | class BasicConv(nn.Module): 16 | 17 | def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, bn=True, relu=True, **kwargs): 18 | super(BasicConv, self).__init__() 19 | # print(in_channels, out_channels, deconv, is_3d, bn, relu, kwargs) 20 | self.relu = relu 21 | self.use_bn = bn 22 | if is_3d: 23 | if deconv: 24 | self.conv = nn.ConvTranspose3d(in_channels, out_channels, bias=False, **kwargs) 25 | else: 26 | self.conv = nn.Conv3d(in_channels, out_channels, bias=False, **kwargs) 27 | self.bn = BatchNorm3d(out_channels) 28 | else: 29 | if deconv: 30 | self.conv = nn.ConvTranspose2d(in_channels, out_channels, bias=False, **kwargs) 31 | else: 32 | self.conv = nn.Conv2d(in_channels, out_channels, bias=False, **kwargs) 33 | self.bn = BatchNorm2d(out_channels) 34 | 35 | def forward(self, x): 36 | x = self.conv(x) 37 | if self.use_bn: 38 | x = self.bn(x) 39 | if self.relu: 40 | x = F.relu(x, inplace=True) 41 | return x 42 | 43 | 44 | class Conv2x(nn.Module): 45 | 46 | def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, concat=True, bn=True, relu=True): 47 | super(Conv2x, self).__init__() 48 | self.concat = concat 49 | 50 | if deconv and is_3d: 51 | kernel = (3, 4, 4) 52 | elif deconv: 53 | kernel = 4 54 | else: 55 | kernel = 3 56 | self.conv1 = BasicConv(in_channels, out_channels, deconv, is_3d, bn=True, relu=True, kernel_size=kernel, stride=2, padding=1) 57 | 58 | if self.concat: 59 | self.conv2 = BasicConv(out_channels*2, out_channels, False, is_3d, bn, relu, kernel_size=3, stride=1, padding=1) 60 | else: 61 | self.conv2 = BasicConv(out_channels, out_channels, False, is_3d, bn, relu, kernel_size=3, stride=1, padding=1) 62 | 63 | def forward(self, x, rem): 64 | x = self.conv1(x) 65 | assert(x.size() == rem.size()) 66 | if self.concat: 67 | x = torch.cat((x, rem), 1) 68 | else: 69 | x = x + rem 70 | x = self.conv2(x) 71 | return x 72 | 73 | 74 | class Feature(nn.Module): 75 | def __init__(self): 76 | super(Feature, self).__init__() 77 | 78 | self.conv_start = nn.Sequential( 79 | BasicConv(3, 32, kernel_size=3, padding=1), 80 | BasicConv(32, 32, kernel_size=5, stride=3, padding=2), 81 | BasicConv(32, 32, kernel_size=3, padding=1)) 82 | self.conv1a = BasicConv(32, 48, kernel_size=3, stride=2, padding=1) 83 | self.conv2a = BasicConv(48, 64, kernel_size=3, stride=2, padding=1) 84 | self.conv3a = BasicConv(64, 96, kernel_size=3, stride=2, padding=1) 85 | self.conv4a = BasicConv(96, 128, kernel_size=3, stride=2, padding=1) 86 | 87 | self.deconv4a = Conv2x(128, 96, deconv=True) 88 | self.deconv3a = Conv2x(96, 64, deconv=True) 89 | self.deconv2a = Conv2x(64, 48, deconv=True) 90 | self.deconv1a = Conv2x(48, 32, deconv=True) 91 | 92 | self.conv1b = Conv2x(32, 48) 93 | self.conv2b = Conv2x(48, 64) 94 | self.conv3b = Conv2x(64, 96) 95 | self.conv4b = Conv2x(96, 128) 96 | 97 | self.deconv4b = Conv2x(128, 96, deconv=True) 98 | self.deconv3b = Conv2x(96, 64, deconv=True) 99 | self.deconv2b = Conv2x(64, 48, deconv=True) 100 | self.deconv1b = Conv2x(48, 32, deconv=True) 101 | 102 | def forward(self, x): 103 | x = self.conv_start(x) 104 | rem0 = x 105 | x = self.conv1a(x) 106 | rem1 = x 107 | x = self.conv2a(x) 108 | rem2 = x 109 | x = self.conv3a(x) 110 | rem3 = x 111 | x = self.conv4a(x) 112 | rem4 = x 113 | x = self.deconv4a(x, rem3) 114 | rem3 = x 115 | 116 | x = self.deconv3a(x, rem2) 117 | rem2 = x 118 | x = self.deconv2a(x, rem1) 119 | rem1 = x 120 | x = self.deconv1a(x, rem0) 121 | rem0 = x 122 | 123 | x = self.conv1b(x, rem1) 124 | rem1 = x 125 | x = self.conv2b(x, rem2) 126 | rem2 = x 127 | x = self.conv3b(x, rem3) 128 | rem3 = x 129 | x = self.conv4b(x, rem4) 130 | 131 | x = self.deconv4b(x, rem3) 132 | x = self.deconv3b(x, rem2) 133 | x = self.deconv2b(x, rem1) 134 | x = self.deconv1b(x, rem0) 135 | 136 | return x 137 | 138 | class Guidance(nn.Module): 139 | def __init__(self): 140 | super(Guidance, self).__init__() 141 | 142 | self.conv0 = BasicConv(64, 16, kernel_size=3, padding=1) 143 | self.conv1 = nn.Sequential( 144 | BasicConv(16, 32, kernel_size=5, stride=3, padding=2), 145 | BasicConv(32, 32, kernel_size=3, padding=1)) 146 | 147 | self.conv2 = BasicConv(32, 32, kernel_size=3, padding=1) 148 | self.conv3 = BasicConv(32, 32, kernel_size=3, padding=1) 149 | 150 | # self.conv11 = Conv2x(32, 48) 151 | self.conv11 = nn.Sequential(BasicConv(32, 48, kernel_size=3, stride=2, padding=1), 152 | BasicConv(48, 48, kernel_size=3, padding=1)) 153 | self.conv12 = BasicConv(48, 48, kernel_size=3, padding=1) 154 | self.conv13 = BasicConv(48, 48, kernel_size=3, padding=1) 155 | self.conv14 = BasicConv(48, 48, kernel_size=3, padding=1) 156 | 157 | self.weight_sg1 = nn.Conv2d(32, 640, (3, 3), (1, 1), (1, 1), bias=False) 158 | self.weight_sg2 = nn.Conv2d(32, 640, (3, 3), (1, 1), (1, 1), bias=False) 159 | self.weight_sg3 = nn.Conv2d(32, 640, (3, 3), (1, 1), (1, 1), bias=False) 160 | 161 | self.weight_sg11 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 162 | self.weight_sg12 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 163 | self.weight_sg13 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 164 | self.weight_sg14 = nn.Conv2d(48, 960, (3, 3), (1, 1), (1, 1), bias=False) 165 | 166 | self.weight_lg1 = nn.Sequential(BasicConv(16, 16, kernel_size=3, padding=1), 167 | nn.Conv2d(16, 75, (3, 3), (1, 1), (1, 1) ,bias=False)) 168 | self.weight_lg2 = nn.Sequential(BasicConv(16, 16, kernel_size=3, padding=1), 169 | nn.Conv2d(16, 75, (3, 3), (1, 1), (1, 1) ,bias=False)) 170 | 171 | def forward(self, x): 172 | x = self.conv0(x) 173 | rem = x 174 | x = self.conv1(x) 175 | sg1 = self.weight_sg1(x) 176 | x = self.conv2(x) 177 | sg2 = self.weight_sg2(x) 178 | x = self.conv3(x) 179 | sg3 = self.weight_sg3(x) 180 | 181 | x = self.conv11(x) 182 | sg11 = self.weight_sg11(x) 183 | x = self.conv12(x) 184 | sg12 = self.weight_sg12(x) 185 | x = self.conv13(x) 186 | sg13 = self.weight_sg13(x) 187 | x = self.conv14(x) 188 | sg14 = self.weight_sg14(x) 189 | 190 | lg1 = self.weight_lg1(rem) 191 | lg2 = self.weight_lg2(rem) 192 | 193 | return dict([ 194 | ('sg1', sg1), 195 | ('sg2', sg2), 196 | ('sg3', sg3), 197 | ('sg11', sg11), 198 | ('sg12', sg12), 199 | ('sg13', sg13), 200 | ('sg14', sg14), 201 | ('lg1', lg1), 202 | ('lg2', lg2)]) 203 | 204 | class Disp(nn.Module): 205 | 206 | def __init__(self, maxdisp=192): 207 | super(Disp, self).__init__() 208 | self.maxdisp = maxdisp 209 | self.softmax = nn.Softmin(dim=1) 210 | self.disparity = DisparityRegression(maxdisp=self.maxdisp) 211 | # self.conv32x1 = BasicConv(32, 1, kernel_size=3) 212 | self.conv32x1 = nn.Conv3d(32, 1, (3, 3, 3), (1, 1, 1), (1, 1, 1), bias=False) 213 | 214 | def forward(self, x): 215 | x = F.interpolate(self.conv32x1(x), [self.maxdisp+1, x.size()[3]*3, x.size()[4]*3], mode='trilinear', align_corners=False) 216 | x = torch.squeeze(x, 1) 217 | x = self.softmax(x) 218 | 219 | return self.disparity(x) 220 | 221 | class DispAgg(nn.Module): 222 | 223 | def __init__(self, maxdisp=192): 224 | super(DispAgg, self).__init__() 225 | self.maxdisp = maxdisp 226 | self.LGA3 = LGA3(radius=2) 227 | self.LGA2 = LGA2(radius=2) 228 | self.LGA = LGA(radius=2) 229 | self.softmax = nn.Softmin(dim=1) 230 | self.disparity = DisparityRegression(maxdisp=self.maxdisp) 231 | # self.conv32x1 = BasicConv(32, 1, kernel_size=3) 232 | self.conv32x1=nn.Conv3d(32, 1, (3, 3, 3), (1, 1, 1), (1, 1, 1), bias=False) 233 | 234 | def lga(self, x, g): 235 | g = F.normalize(g, p=1, dim=1) 236 | x = self.LGA2(x, g) 237 | return x 238 | 239 | def forward(self, x, lg1, lg2): 240 | x = F.interpolate(self.conv32x1(x), [self.maxdisp+1, x.size()[3]*3, x.size()[4]*3], mode='trilinear', align_corners=False) 241 | x = torch.squeeze(x, 1) 242 | assert(lg1.size() == lg2.size()) 243 | x = self.lga(x, lg1) 244 | x = self.softmax(x) 245 | x = self.lga(x, lg2) 246 | x = F.normalize(x, p=1, dim=1) 247 | return self.disparity(x) 248 | 249 | class SGABlock(nn.Module): 250 | def __init__(self, channels=32, refine=False): 251 | super(SGABlock, self).__init__() 252 | self.refine = refine 253 | if self.refine: 254 | self.bn_relu = nn.Sequential(BatchNorm3d(channels), 255 | nn.ReLU(inplace=True)) 256 | self.conv_refine = BasicConv(channels, channels, is_3d=True, kernel_size=3, padding=1, relu=False) 257 | # self.conv_refine1 = BasicConv(8, 8, is_3d=True, kernel_size=1, padding=1) 258 | else: 259 | self.bn = BatchNorm3d(channels) 260 | self.SGA=SGA() 261 | self.relu = nn.ReLU(inplace=True) 262 | def forward(self, x, g): 263 | rem = x 264 | k1, k2, k3, k4 = torch.split(g, (x.size()[1]*5, x.size()[1]*5, x.size()[1]*5, x.size()[1]*5), 1) 265 | k1 = F.normalize(k1.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 266 | k2 = F.normalize(k2.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 267 | k3 = F.normalize(k3.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 268 | k4 = F.normalize(k4.view(x.size()[0], x.size()[1], 5, x.size()[3], x.size()[4]), p=1, dim=2) 269 | x = self.SGA(x, k1, k2, k3, k4) 270 | if self.refine: 271 | x = self.bn_relu(x) 272 | x = self.conv_refine(x) 273 | else: 274 | x = self.bn(x) 275 | assert(x.size() == rem.size()) 276 | x += rem 277 | return self.relu(x) 278 | # return self.bn_relu(x) 279 | 280 | 281 | class CostAggregation(nn.Module): 282 | def __init__(self, maxdisp=192): 283 | super(CostAggregation, self).__init__() 284 | self.maxdisp = maxdisp 285 | self.conv_start = BasicConv(64, 32, is_3d=True, kernel_size=3, padding=1, relu=False) 286 | 287 | self.conv1a = BasicConv(32, 48, is_3d=True, kernel_size=3, stride=2, padding=1) 288 | self.conv2a = BasicConv(48, 64, is_3d=True, kernel_size=3, stride=2, padding=1) 289 | # self.conv3a = BasicConv(64, 96, is_3d=True, kernel_size=3, stride=2, padding=1) 290 | 291 | self.deconv1a = Conv2x(48, 32, deconv=True, is_3d=True, relu=False) 292 | self.deconv2a = Conv2x(64, 48, deconv=True, is_3d=True) 293 | # self.deconv3a = Conv2x(96, 64, deconv=True, is_3d=True) 294 | 295 | self.conv1b = Conv2x(32, 48, is_3d=True) 296 | self.conv2b = Conv2x(48, 64, is_3d=True) 297 | # self.conv3b = Conv2x(64, 96, is_3d=True) 298 | 299 | self.deconv1b = Conv2x(48, 32, deconv=True, is_3d=True, relu=False) 300 | self.deconv2b = Conv2x(64, 48, deconv=True, is_3d=True) 301 | # self.deconv3b = Conv2x(96, 64, deconv=True, is_3d=True) 302 | self.deconv0b = Conv2x(8, 8, deconv=True, is_3d=True) 303 | 304 | self.sga1 = SGABlock(refine=True) 305 | self.sga2 = SGABlock(refine=True) 306 | self.sga3 = SGABlock(refine=True) 307 | 308 | self.sga11 = SGABlock(channels=48, refine=True) 309 | self.sga12 = SGABlock(channels=48, refine=True) 310 | self.sga13 = SGABlock(channels=48, refine=True) 311 | self.sga14 = SGABlock(channels=48, refine=True) 312 | 313 | self.disp0 = Disp(self.maxdisp) 314 | self.disp1 = Disp(self.maxdisp) 315 | self.disp2 = DispAgg(self.maxdisp) 316 | 317 | 318 | def forward(self, x, g): 319 | 320 | x = self.conv_start(x) 321 | x = self.sga1(x, g['sg1']) 322 | rem0 = x 323 | 324 | if self.training: 325 | disp0 = self.disp0(x) 326 | 327 | x = self.conv1a(x) 328 | x = self.sga11(x, g['sg11']) 329 | rem1 = x 330 | x = self.conv2a(x) 331 | rem2 = x 332 | # x = self.conv3a(x) 333 | # rem3 = x 334 | 335 | # x = self.deconv3a(x, rem2) 336 | # rem2 = x 337 | x = self.deconv2a(x, rem1) 338 | x = self.sga12(x, g['sg12']) 339 | rem1 = x 340 | x = self.deconv1a(x, rem0) 341 | x = self.sga2(x, g['sg2']) 342 | rem0 = x 343 | if self.training: 344 | disp1 = self.disp1(x) 345 | 346 | x = self.conv1b(x, rem1) 347 | x = self.sga13(x, g['sg13']) 348 | rem1 = x 349 | x = self.conv2b(x, rem2) 350 | # rem2 = x 351 | # x = self.conv3b(x, rem3) 352 | 353 | # x = self.deconv3b(x, rem2) 354 | x = self.deconv2b(x, rem1) 355 | x = self.sga14(x, g['sg14']) 356 | x = self.deconv1b(x, rem0) 357 | x = self.sga3(x, g['sg3']) 358 | 359 | disp2 = self.disp2(x, g['lg1'], g['lg2']) 360 | if self.training: 361 | return disp0, disp1, disp2 362 | else: 363 | return disp2 364 | 365 | class GANet(nn.Module): 366 | def __init__(self, maxdisp=192): 367 | super(GANet, self).__init__() 368 | self.maxdisp = maxdisp 369 | self.conv_start = nn.Sequential(BasicConv(3, 16, kernel_size=3, padding=1), 370 | BasicConv(16, 32, kernel_size=3, padding=1)) 371 | 372 | self.conv_x = BasicConv(32, 32, kernel_size=3, padding=1) 373 | self.conv_y = BasicConv(32, 32, kernel_size=3, padding=1) 374 | self.conv_refine = nn.Conv2d(32, 32, (3, 3), (1, 1), (1,1), bias=False) 375 | self.bn_relu = nn.Sequential(BatchNorm2d(32), 376 | nn.ReLU(inplace=True)) 377 | self.feature = Feature() 378 | self.guidance = Guidance() 379 | self.cost_agg = CostAggregation(self.maxdisp) 380 | self.cv = GetCostVolume(int(self.maxdisp/3)) 381 | 382 | for m in self.modules(): 383 | if isinstance(m, (nn.Conv2d, nn.Conv3d)): 384 | nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu') 385 | elif isinstance(m, (BatchNorm2d, BatchNorm3d)): 386 | nn.init.constant_(m.weight, 1) 387 | nn.init.constant_(m.bias, 0) 388 | 389 | def forward(self, x, y): 390 | g = self.conv_start(x) 391 | x = self.feature(x) 392 | 393 | rem = x 394 | x = self.conv_x(x) 395 | 396 | y = self.feature(y) 397 | y = self.conv_y(y) 398 | 399 | x = self.cv(x,y) 400 | 401 | x1 = self.conv_refine(rem) 402 | x1 = F.interpolate(x1, [x1.size()[2]*3,x1.size()[3]*3], mode='bilinear', align_corners=False) 403 | x1 = self.bn_relu(x1) 404 | g = torch.cat((g, x1), 1) 405 | g = self.guidance(g) 406 | if self.training: 407 | disp0, disp1, disp2 = self.cost_agg(x, g) 408 | return disp0, disp1, disp2 409 | else: 410 | return self.cost_agg(x, g) 411 | -------------------------------------------------------------------------------- /predict.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | import skimage 4 | import skimage.io 5 | import skimage.transform 6 | from PIL import Image 7 | from math import log10 8 | #from GCNet.modules.GCNet import L1Loss 9 | import sys 10 | import shutil 11 | import os 12 | import torch 13 | import torch.nn as nn 14 | import torch.nn.parallel 15 | import torch.backends.cudnn as cudnn 16 | import torch.optim as optim 17 | from torch.autograd import Variable 18 | from torch.utils.data import DataLoader 19 | #from models.GANet_deep import GANet 20 | from dataloader.data import get_test_set 21 | import numpy as np 22 | 23 | # Training settings 24 | parser = argparse.ArgumentParser(description='PyTorch GANet Example') 25 | parser.add_argument('--crop_height', type=int, required=True, help="crop height") 26 | parser.add_argument('--crop_width', type=int, required=True, help="crop width") 27 | parser.add_argument('--max_disp', type=int, default=192, help="max disp") 28 | parser.add_argument('--resume', type=str, default='', help="resume from saved model") 29 | parser.add_argument('--cuda', type=bool, default=True, help='use cuda?') 30 | parser.add_argument('--kitti', type=int, default=0, help='kitti dataset? Default=False') 31 | parser.add_argument('--kitti2015', type=int, default=0, help='kitti 2015? Default=False') 32 | parser.add_argument('--data_path', type=str, required=True, help="data root") 33 | parser.add_argument('--test_list', type=str, required=True, help="training list") 34 | parser.add_argument('--save_path', type=str, default='./result/', help="location to save result") 35 | parser.add_argument('--model', type=str, default='GANet_deep', help="model to train") 36 | 37 | opt = parser.parse_args() 38 | 39 | 40 | print(opt) 41 | if opt.model == 'GANet11': 42 | from models.GANet11 import GANet 43 | elif opt.model == 'GANet_deep': 44 | from models.GANet_deep import GANet 45 | else: 46 | raise Exception("No suitable model found ...") 47 | 48 | cuda = opt.cuda 49 | #cuda = True 50 | if cuda and not torch.cuda.is_available(): 51 | raise Exception("No GPU found, please run without --cuda") 52 | 53 | #torch.manual_seed(opt.seed) 54 | #if cuda: 55 | # torch.cuda.manual_seed(opt.seed) 56 | #print('===> Loading datasets') 57 | 58 | 59 | print('===> Building model') 60 | model = GANet(opt.max_disp) 61 | 62 | if cuda: 63 | model = torch.nn.DataParallel(model).cuda() 64 | 65 | if opt.resume: 66 | if os.path.isfile(opt.resume): 67 | print("=> loading checkpoint '{}'".format(opt.resume)) 68 | checkpoint = torch.load(opt.resume) 69 | model.load_state_dict(checkpoint['state_dict'], strict=False) 70 | 71 | else: 72 | print("=> no checkpoint found at '{}'".format(opt.resume)) 73 | 74 | 75 | def test_transform(temp_data, crop_height, crop_width): 76 | _, h, w=np.shape(temp_data) 77 | 78 | if h <= crop_height and w <= crop_width: 79 | temp = temp_data 80 | temp_data = np.zeros([6, crop_height, crop_width], 'float32') 81 | temp_data[:, crop_height - h: crop_height, crop_width - w: crop_width] = temp 82 | else: 83 | start_x = int((w - crop_width) / 2) 84 | start_y = int((h - crop_height) / 2) 85 | temp_data = temp_data[:, start_y: start_y + crop_height, start_x: start_x + crop_width] 86 | left = np.ones([1, 3,crop_height,crop_width],'float32') 87 | left[0, :, :, :] = temp_data[0: 3, :, :] 88 | right = np.ones([1, 3, crop_height, crop_width], 'float32') 89 | right[0, :, :, :] = temp_data[3: 6, :, :] 90 | return torch.from_numpy(left).float(), torch.from_numpy(right).float(), h, w 91 | 92 | def load_data(leftname, rightname): 93 | left = Image.open(leftname) 94 | right = Image.open(rightname) 95 | size = np.shape(left) 96 | height = size[0] 97 | width = size[1] 98 | temp_data = np.zeros([6, height, width], 'float32') 99 | left = np.asarray(left) 100 | right = np.asarray(right) 101 | r = left[:, :, 0] 102 | g = left[:, :, 1] 103 | b = left[:, :, 2] 104 | temp_data[0, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 105 | temp_data[1, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 106 | temp_data[2, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 107 | r = right[:, :, 0] 108 | g = right[:, :, 1] 109 | b = right[:, :, 2] 110 | #r,g,b,_ = right.split() 111 | temp_data[3, :, :] = (r - np.mean(r[:])) / np.std(r[:]) 112 | temp_data[4, :, :] = (g - np.mean(g[:])) / np.std(g[:]) 113 | temp_data[5, :, :] = (b - np.mean(b[:])) / np.std(b[:]) 114 | return temp_data 115 | 116 | def test(leftname, rightname, savename): 117 | # count=0 118 | 119 | input1, input2, height, width = test_transform(load_data(leftname, rightname), opt.crop_height, opt.crop_width) 120 | 121 | 122 | input1 = Variable(input1, requires_grad = False) 123 | input2 = Variable(input2, requires_grad = False) 124 | 125 | model.eval() 126 | if cuda: 127 | input1 = input1.cuda() 128 | input2 = input2.cuda() 129 | with torch.no_grad(): 130 | prediction = model(input1, input2) 131 | 132 | temp = prediction.cpu() 133 | temp = temp.detach().numpy() 134 | if height <= opt.crop_height and width <= opt.crop_width: 135 | temp = temp[0, opt.crop_height - height: opt.crop_height, opt.crop_width - width: opt.crop_width] 136 | else: 137 | temp = temp[0, :, :] 138 | skimage.io.imsave(savename, (temp * 256).astype('uint16')) 139 | 140 | 141 | if __name__ == "__main__": 142 | file_path = opt.data_path 143 | file_list = opt.test_list 144 | f = open(file_list, 'r') 145 | filelist = f.readlines() 146 | for index in range(len(filelist)): 147 | current_file = filelist[index] 148 | if opt.kitti2015: 149 | leftname = file_path + 'image_2/' + current_file[0: len(current_file) - 1] 150 | rightname = file_path + 'image_3/' + current_file[0: len(current_file) - 1] 151 | if opt.kitti: 152 | leftname = file_path + 'colored_0/' + current_file[0: len(current_file) - 1] 153 | rightname = file_path + 'colored_1/' + current_file[0: len(current_file) - 1] 154 | 155 | savename = opt.save_path + current_file[0: len(current_file) - 1] 156 | test(leftname, rightname, savename) 157 | 158 | -------------------------------------------------------------------------------- /predict.sh: -------------------------------------------------------------------------------- 1 | 2 | python predict.py --crop_height=384 \ 3 | --crop_width=1248 \ 4 | --max_disp=192 \ 5 | --data_path='/ssd1/zhangfeihu/data/kitti/2015//testing/' \ 6 | --test_list='lists/kitti2015_test.list' \ 7 | --save_path='./result/' \ 8 | --kitti2015=1 \ 9 | --resume='./checkpoint/kitti2015_final.pth' 10 | exit 11 | 12 | python predict.py --crop_height=384 \ 13 | --crop_width=1248 \ 14 | --max_disp=192 \ 15 | --data_path='/media/feihu/Storage/stereo/kitti/testing/' \ 16 | --test_list='lists/kitti2012_test.list' \ 17 | --save_path='./result/' \ 18 | --kitti=1 \ 19 | --resume='./checkpoint/kitti2012_final.pth' 20 | 21 | 22 | 23 | -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import argparse 3 | from math import log10 4 | 5 | from libs.GANet.modules.GANet import MyLoss2 6 | import sys 7 | import shutil 8 | import os 9 | import torch 10 | import torch.nn as nn 11 | import torch.nn.parallel 12 | import torch.backends.cudnn as cudnn 13 | import torch.optim as optim 14 | from torch.autograd import Variable 15 | from torch.utils.data import DataLoader 16 | #from models.GANet_deep import GANet 17 | import torch.nn.functional as F 18 | from dataloader.data import get_training_set, get_test_set 19 | 20 | # Training settings 21 | parser = argparse.ArgumentParser(description='PyTorch GANet Example') 22 | parser.add_argument('--crop_height', type=int, required=True, help="crop height") 23 | parser.add_argument('--max_disp', type=int, default=192, help="max disp") 24 | parser.add_argument('--crop_width', type=int, required=True, help="crop width") 25 | parser.add_argument('--resume', type=str, default='', help="resume from saved model") 26 | parser.add_argument('--left_right', type=int, default=0, help="use right view for training. Default=False") 27 | parser.add_argument('--batchSize', type=int, default=1, help='training batch size') 28 | parser.add_argument('--testBatchSize', type=int, default=1, help='testing batch size') 29 | parser.add_argument('--nEpochs', type=int, default=2048, help='number of epochs to train for') 30 | parser.add_argument('--lr', type=float, default=0.001, help='Learning Rate. Default=0.001') 31 | parser.add_argument('--cuda', type=int, default=1, help='use cuda? Default=True') 32 | parser.add_argument('--threads', type=int, default=1, help='number of threads for data loader to use') 33 | parser.add_argument('--seed', type=int, default=123, help='random seed to use. Default=123') 34 | parser.add_argument('--shift', type=int, default=0, help='random shift of left image. Default=0') 35 | parser.add_argument('--kitti', type=int, default=0, help='kitti dataset? Default=False') 36 | parser.add_argument('--kitti2015', type=int, default=0, help='kitti 2015? Default=False') 37 | parser.add_argument('--data_path', type=str, default='/ssd1/zhangfeihu/data/stereo/', help="data root") 38 | parser.add_argument('--training_list', type=str, default='./lists/sceneflow_train.list', help="training list") 39 | parser.add_argument('--val_list', type=str, default='./lists/sceneflow_test_select.list', help="validation list") 40 | parser.add_argument('--save_path', type=str, default='./checkpoint/', help="location to save models") 41 | parser.add_argument('--model', type=str, default='GANet_deep', help="model to train") 42 | 43 | opt = parser.parse_args() 44 | 45 | print(opt) 46 | if opt.model == 'GANet11': 47 | from models.GANet11 import GANet 48 | elif opt.model == 'GANet_deep': 49 | from models.GANet_deep import GANet 50 | else: 51 | raise Exception("No suitable model found ...") 52 | 53 | cuda = opt.cuda 54 | #cuda = True 55 | if cuda and not torch.cuda.is_available(): 56 | raise Exception("No GPU found, please run without --cuda") 57 | 58 | torch.manual_seed(opt.seed) 59 | if cuda: 60 | torch.cuda.manual_seed(opt.seed) 61 | 62 | print('===> Loading datasets') 63 | train_set = get_training_set(opt.data_path, opt.training_list, [opt.crop_height, opt.crop_width], opt.left_right, opt.kitti, opt.kitti2015, opt.shift) 64 | test_set = get_test_set(opt.data_path, opt.val_list, [576,960], opt.left_right, opt.kitti, opt.kitti2015) 65 | training_data_loader = DataLoader(dataset=train_set, num_workers=opt.threads, batch_size=opt.batchSize, shuffle=True, drop_last=True) 66 | testing_data_loader = DataLoader(dataset=test_set, num_workers=opt.threads, batch_size=opt.testBatchSize, shuffle=False) 67 | 68 | print('===> Building model') 69 | model = GANet(opt.max_disp) 70 | 71 | criterion = MyLoss2(thresh=3, alpha=2) 72 | if cuda: 73 | model = torch.nn.DataParallel(model).cuda() 74 | optimizer=optim.Adam(model.parameters(), lr=opt.lr,betas=(0.9,0.999)) 75 | if opt.resume: 76 | if os.path.isfile(opt.resume): 77 | print("=> loading checkpoint '{}'".format(opt.resume)) 78 | checkpoint = torch.load(opt.resume) 79 | model.load_state_dict(checkpoint['state_dict'], strict=False) 80 | # optimizer.load_state_dict(checkpoint['optimizer']) 81 | else: 82 | print("=> no checkpoint found at '{}'".format(opt.resume)) 83 | 84 | 85 | def train(epoch): 86 | epoch_loss = 0 87 | epoch_error0 = 0 88 | epoch_error1 = 0 89 | epoch_error2 = 0 90 | valid_iteration = 0 91 | model.train() 92 | for iteration, batch in enumerate(training_data_loader): 93 | input1, input2, target = Variable(batch[0], requires_grad=True), Variable(batch[1], requires_grad=True), Variable(batch[2], requires_grad=False) 94 | if cuda: 95 | input1 = input1.cuda() 96 | input2 = input2.cuda() 97 | target = target.cuda() 98 | 99 | target=torch.squeeze(target,1) 100 | mask = target < opt.max_disp 101 | mask.detach_() 102 | valid = target[mask].size()[0] 103 | if valid > 0: 104 | optimizer.zero_grad() 105 | 106 | if opt.model == 'GANet11': 107 | disp1, disp2 = model(input1, input2) 108 | disp0 = (disp1 + disp2)/2. 109 | if opt.kitti or opt.kitti2015: 110 | loss = 0.4 * F.smooth_l1_loss(disp1[mask], target[mask], reduction='mean') + 1.2 * criterion(disp2[mask], target[mask]) 111 | else: 112 | loss = 0.4 * F.smooth_l1_loss(disp1[mask], target[mask], reduction='mean') + 1.2 * F.smooth_l1_loss(disp2[mask], target[mask], reduction='mean') 113 | elif opt.model == 'GANet_deep': 114 | disp0, disp1, disp2 = model(input1, input2) 115 | if opt.kitti or opt.kitti2015: 116 | loss = 0.2 * F.smooth_l1_loss(disp0[mask], target[mask], reduction='mean') + 0.6 * F.smooth_l1_loss(disp1[mask], target[mask], reduction='mean') + criterion(disp2[mask], target[mask]) 117 | else: 118 | loss = 0.2 * F.smooth_l1_loss(disp0[mask], target[mask], reduction='mean') + 0.6 * F.smooth_l1_loss(disp1[mask], target[mask], reduction='mean') + F.smooth_l1_loss(disp2[mask], target[mask], reduction='mean') 119 | else: 120 | raise Exception("No suitable model found ...") 121 | 122 | loss.backward() 123 | optimizer.step() 124 | error0 = torch.mean(torch.abs(disp0[mask] - target[mask])) 125 | error1 = torch.mean(torch.abs(disp1[mask] - target[mask])) 126 | error2 = torch.mean(torch.abs(disp2[mask] - target[mask])) 127 | 128 | epoch_loss += loss.item() 129 | valid_iteration += 1 130 | epoch_error0 += error0.item() 131 | epoch_error1 += error1.item() 132 | epoch_error2 += error2.item() 133 | print("===> Epoch[{}]({}/{}): Loss: {:.4f}, Error: ({:.4f} {:.4f} {:.4f})".format(epoch, iteration, len(training_data_loader), loss.item(), error0.item(), error1.item(), error2.item())) 134 | sys.stdout.flush() 135 | 136 | print("===> Epoch {} Complete: Avg. Loss: {:.4f}, Avg. Error: ({:.4f} {:.4f} {:.4f})".format(epoch, epoch_loss / valid_iteration,epoch_error0/valid_iteration,epoch_error1/valid_iteration,epoch_error2/valid_iteration)) 137 | 138 | def val(): 139 | epoch_error2 = 0 140 | 141 | valid_iteration = 0 142 | model.eval() 143 | for iteration, batch in enumerate(testing_data_loader): 144 | input1, input2, target = Variable(batch[0],requires_grad=False), Variable(batch[1], requires_grad=False), Variable(batch[2], requires_grad=False) 145 | if cuda: 146 | input1 = input1.cuda() 147 | input2 = input2.cuda() 148 | target = target.cuda() 149 | target = torch.squeeze(target, 1) 150 | mask = target < opt.max_disp 151 | mask.detach_() 152 | valid = target[mask].size()[0] 153 | if valid>0: 154 | with torch.no_grad(): 155 | disp2 = model(input1,input2) 156 | error2 = torch.mean(torch.abs(disp2[mask] - target[mask])) 157 | valid_iteration += 1 158 | epoch_error2 += error2.item() 159 | print("===> Test({}/{}): Error: ({:.4f})".format(iteration, len(testing_data_loader), error2.item())) 160 | 161 | print("===> Test: Avg. Error: ({:.4f})".format(epoch_error2 / valid_iteration)) 162 | return epoch_error2 / valid_iteration 163 | 164 | def save_checkpoint(save_path, epoch,state, is_best): 165 | filename = save_path + "_epoch_{}.pth".format(epoch) 166 | torch.save(state, filename) 167 | if is_best: 168 | shutil.copyfile(filename, save_path + '_best.pth') 169 | print("Checkpoint saved to {}".format(filename)) 170 | 171 | def adjust_learning_rate(optimizer, epoch): 172 | if epoch <= 400: 173 | lr = opt.lr 174 | else: 175 | lr = opt.lr*0.1 176 | print(lr) 177 | for param_group in optimizer.param_groups: 178 | param_group['lr'] = lr 179 | 180 | if __name__ == '__main__': 181 | error=100 182 | for epoch in range(1, opt.nEpochs + 1): 183 | # if opt.kitti or opt.kitti2015: 184 | adjust_learning_rate(optimizer, epoch) 185 | train(epoch) 186 | is_best = False 187 | # loss=val() 188 | # if loss < error: 189 | # error=loss 190 | # is_best = True 191 | if opt.kitti or opt.kitti2015: 192 | if epoch%50 == 0 and epoch >= 300: 193 | save_checkpoint(opt.save_path, epoch,{ 194 | 'epoch': epoch, 195 | 'state_dict': model.state_dict(), 196 | 'optimizer' : optimizer.state_dict(), 197 | }, is_best) 198 | else: 199 | if epoch>=8: 200 | save_checkpoint(opt.save_path, epoch,{ 201 | 'epoch': epoch, 202 | 'state_dict': model.state_dict(), 203 | 'optimizer' : optimizer.state_dict(), 204 | }, is_best) 205 | 206 | 207 | save_checkpoint(opt.save_path, opt.nEpochs,{ 208 | 'epoch': opt.nEpochs, 209 | 'state_dict': model.state_dict(), 210 | 'optimizer' : optimizer.state_dict(), 211 | }, is_best) 212 | 213 | 214 | -------------------------------------------------------------------------------- /train.sh: -------------------------------------------------------------------------------- 1 | CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py --batchSize=16 \ 2 | --crop_height=240 \ 3 | --crop_width=528 \ 4 | --max_disp=192 \ 5 | --thread=16 \ 6 | --data_path='/ssd1/zhangfeihu/data/stereo/' \ 7 | --training_list='lists/sceneflow_train.list' \ 8 | --save_path='./checkpoint/sceneflow' \ 9 | --resume='' \ 10 | --model='GANet_deep' \ 11 | --nEpochs=11 2>&1 |tee logs/log_train_sceneflow.txt 12 | 13 | exit 14 | #Fine tuning for kitti 2015 15 | CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py --batchSize=16 \ 16 | --crop_height=240 \ 17 | --crop_width=528 \ 18 | --max_disp=192 \ 19 | --thread=16 \ 20 | --data_path='/media/feihu/Storage/stereo/data_scene_flow/training/' \ 21 | --training_list='lists/kitti2015_train.list' \ 22 | --save_path='./checkpoint/finetune_kitti2015' \ 23 | --kitti2015=1 \ 24 | --shift=3 \ 25 | --resume='./checkpoint/sceneflow_epoch_10.pth' \ 26 | --nEpochs=800 2>&1 |tee logs/log_finetune_kitti2015.txt 27 | 28 | CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py --batchSize=8 \ 29 | --crop_height=240 \ 30 | --crop_width=1248 \ 31 | --max_disp=192 \ 32 | --thread=16 \ 33 | --data_path='/media/feihu/Storage/stereo/data_scene_flow/training/' \ 34 | --training_list='lists/kitti2015_train.list' \ 35 | --save_path='./checkpoint/finetune2_kitti2015' \ 36 | --kitti2015=1 \ 37 | --shift=3 \ 38 | --lr=0.0001 \ 39 | --resume='./checkpoint/finetune_kitti2015_epoch_800.pth' \ 40 | --nEpochs=8 2>&1 |tee logs/log_finetune_kitti2015.txt 41 | 42 | #Fine tuning for kitti 2012 43 | 44 | CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py --batchSize=16 \ 45 | --crop_height=240 \ 46 | --crop_width=528 \ 47 | --max_disp=192 \ 48 | --thread=16 \ 49 | --data_path='/media/feihu/Storage/stereo/kitti/training/' \ 50 | --training_list='lists/kitti2012_train.list' \ 51 | --save_path='./checkpoint/finetune_kitti' \ 52 | --kitti=1 \ 53 | --shift=3 \ 54 | --resume='./checkpoint/sceneflow_epoch_10.pth' \ 55 | --nEpochs=800 2>&1 |tee logs/log_finetune2_kitti.txt 56 | 57 | CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py --batchSize=8 \ 58 | --crop_height=240 \ 59 | --crop_width=1248 \ 60 | --max_disp=192 \ 61 | --thread=16 \ 62 | --data_path='/media/feihu/Storage/stereo/kitti/training/' \ 63 | --training_list='lists/kitti2012_train.list' \ 64 | --save_path='./checkpoint/finetune2_kitti' \ 65 | --kitti=1 \ 66 | --shift=3 \ 67 | --lr=0.0001 \ 68 | --resume='./checkpoint/finetune_kitti_epoch_800.pth' \ 69 | --nEpochs=8 2>&1 |tee logs/log_finetune2_kitti.txt 70 | 71 | 72 | 73 | 74 | --------------------------------------------------------------------------------