├── LICENSE
├── README.md
├── data
    ├── __init__.py
    ├── cityscapes
    │   ├── __init__.py
    │   └── cityscapes_loader.py
    ├── kitti
    │   ├── __init__.py
    │   ├── kitti_odom_loader.py
    │   ├── kitti_raw_loader.py
    │   ├── static_frames.txt
    │   ├── test_files_eigen.txt
    │   ├── test_files_stereo.txt
    │   ├── test_scenes_eigen.txt
    │   └── test_scenes_stereo.txt
    └── prepare_train_data.py
├── data_loader.py
├── geonet_main.py
├── geonet_model.py
├── geonet_nets.py
├── geonet_test_depth.py
├── geonet_test_flow.py
├── geonet_test_pose.py
├── kitti_eval
    ├── __init__.py
    ├── depth_evaluation_utils.py
    ├── eval_depth.py
    ├── eval_flow.py
    ├── eval_pose.py
    ├── flow_tool
    │   ├── flowlib.py
    │   ├── pfm.py
    │   └── png.py
    ├── generate_multiview_extension.py
    ├── generate_pose_snippets.py
    └── pose_evaluation_utils.py
├── misc
    └── overview.jpg
└── utils.py


/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2018 Zhichao
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # GeoNet
  2 | 
  3 | This is a Tensorflow implementation of our paper:
  4 | 
  5 | GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose (CVPR 2018)
  6 | 
  7 | Zhichao Yin and Jianping Shi
  8 | 
  9 | arxiv preprint: (https://arxiv.org/abs/1803.02276)
 10 | 
 11 | <img src="misc/overview.jpg" width="550">
 12 | 
 13 | ## Requirements
 14 | 
 15 | This code has been tested with Python 2.7, TensorFlow 1.1 and CUDA 8.0 on Ubuntu 16.04.
 16 | 
 17 | ## Data preparation
 18 | 
 19 | For replicating our results in all of the three tasks (monocular depth, camera pose and optical flow), 
 20 | you need to download the following datasets, and preprocess them into certain formats:
 21 | 
 22 | ### [KITTI](http://www.cvlibs.net/datasets/kitti/index.php)
 23 | For **depth** and **flow** tasks, the training data is [KITTI raw dataset](http://www.cvlibs.net/datasets/kitti/raw_data.php) 
 24 | and you can download them by the [official script](http://www.cvlibs.net/download.php?file=raw_data_downloader.zip);
 25 | 
 26 | For **pose** task, the training data is [KITTI odometry dataset](http://www.cvlibs.net/download.php?file=data_odometry_color.zip) 
 27 | and you should download the calibration files as well as ground truth poses (for evaluation).
 28 | 
 29 | After downloaded the data, you can run the following command for preprocessing:
 30 | ```bash
 31 | python data/prepare_train_data.py --dataset_dir=/path/to/kitti/dataset/ --dataset_name=kitti_split --dump_root=/path/to/formatted/data/ --seq_length=3 --img_height=128 --img_width=416 --num_threads=16 --remove_static
 32 | ```
 33 | 
 34 | For **depth** task, the `--dataset_name` should be `kitti_raw_eigen` and `--seq_length` is set to `3`;
 35 | 
 36 | For **flow** task, the `--dataset_name` should be `kitti_raw_stereo` and `--seq_length` is set to `3`;
 37 | 
 38 | For **pose** task, the `--dataset_name` should be `kitti_odom` and `--seq_length` is set to `5`.
 39 | 
 40 | ### [Cityscapes](https://www.cityscapes-dataset.com/)
 41 | You can optionally pretrain the model on Cityscapes dataset for any of the three tasks. The required training 
 42 | data is image sequence `leftImg8bit_sequence_trainvaltest.zip` and calibration file `camera_trainvaltest.zip`. 
 43 | After downloaded them, simply run:
 44 | ```bash
 45 | python data/prepare_train_data.py --dataset_dir=/path/to/cityscapes/dataset/ --dataset_name='cityscapes' --dump_root=/path/to/formatted/data/ --seq_length=3 --img_height=171 --img_width=416 --num_threads=16
 46 | ```
 47 | 
 48 | ## Training
 49 | Our code supports two training modes, corresponding to our stage-wise training strategy. 
 50 | The `train_rigid` mode is mainly for learning depth and pose, while `train_flow` mode supports direct or residual flow learning.
 51 | 
 52 | For ``train_rigid`` mode (**depth** and **pose** tasks), run the command
 53 | ```bash
 54 | python geonet_main.py --mode=train_rigid --dataset_dir=/path/to/formatted/data/ --checkpoint_dir=/path/to/save/ckpts/ --learning_rate=0.0002 --seq_length=3 --batch_size=4 --max_steps=350000 
 55 | ```
 56 | You can switch the network encoder by setting `--dispnet_encoder` flag, or perform depth scale normalization (see [this paper](https://arxiv.org/abs/1712.00175) for details) by setting `--scale_normalize` as True.
 57 | Note that for replicating depth and pose results, the `--seq_length` is suggested to be 3 and 5 respectively.
 58 | 
 59 | For ``train_flow`` mode (**flow** task), run the command
 60 | ```bash
 61 | python geonet_main.py --mode=train_flow --dataset_dir=/path/to/formatted/data/ --checkpoint_dir=/path/to/save/ckpts/ --learning_rate=0.0002 --seq_length=3 --flownet_type=direct --max_steps=400000
 62 | ```
 63 | You can choose to learn direct or residual flow by setting `--flownet_type` flag. **Note** that when the `--flownet_type` is `residual`, the `--init_ckpt_file` should be specified to point
 64 | at a model pretrained on the same dataset with mode of `train_rigid`. Also a `max_steps` more than 200 epochs is preferred for learning residual flow.
 65 | 
 66 | ### Pretrained models
 67 | You can download our pretrained models as well as their predictions in all of the three tasks from [[Google Drive](https://drive.google.com/open?id=1VSGpdMrQ3dFKdher_2RteDfz7F0g57ZH)]. **Note** that they were trained according to **different splits** of KITTI as described in the paper. Following the testing and evaluation instructions below, you should obtain similar results as reported in the paper.
 68 | 
 69 | #### Notes about depth scale normalization
 70 | Following most of the original hyperparameters but setting `--scale_normalize` as True, we have trained our DepthNet better on the Eigen's split of KITTI. The pretrained model is also provided, namely **model_sn** in **geonet_depthnet** subfolder. Note this is not included in our paper, but the performance is further improved:
 71 | 
 72 | | Abs Rel | Sq Rel | RMSE  | RMSE(log) | Acc.1 | Acc.2 | Acc.3 |
 73 | |---------|--------|-------|-----------|-------|-------|-------|
 74 | | 0.149   | 1.060  | 5.567 | 0.226     | 0.796 | 0.935 | 0.975 |
 75 | 
 76 | ## Testing
 77 | We provide testing and evaluation scripts for all of the three tasks.
 78 | 
 79 | ### Monocular Depth
 80 | Run the following command
 81 | ```bash
 82 | python geonet_main.py --mode=test_depth --dataset_dir=/path/to/kitti/raw/dataset/ --init_ckpt_file=/path/to/trained/model/ --batch_size=1 --depth_test_split=eigen --output_dir=/path/to/save/predictions/
 83 | ```
 84 | Then you can evaluate the prediction by running
 85 | ```bash
 86 | python kitti_eval/eval_depth.py --split=eigen --kitti_dir=/path/to/kitti/raw/dataset/ --pred_file=/path/to/predictions/
 87 | ```
 88 | 
 89 | ### Camera Pose
 90 | Firstly assuming you have downloaded the KITTI odometry dataset (including groundtruth poses), run
 91 | ```bash
 92 | python geonet_main.py --mode=test_pose --dataset_dir=/path/to/kitti/odom/dataset/ --init_ckpt_file=/path/to/trained/model/ --batch_size=1 --seq_length=5 --pose_test_seq=9 --output_dir=/path/to/save/predictions/
 93 | ```
 94 | Now you have predicted pose snippets. You can **generate the groundtruth pose snippets** by running
 95 | ```bash
 96 | python kitti_eval/generate_pose_snippets.py --dataset_dir=/path/to/kitti/odom/dataset/ --output_dir=/path/to/save/gtruth/pose/snippets/ --seq_id=09 --seq_length=5
 97 | ```
 98 | Then you can evaluate your predictions by
 99 | ```bash
100 | python kitti_eval/eval_pose.py --gtruth_dir=/path/to/gtruth/pose/snippets/ --pred_dir=/path/to/predicted/pose/snippets/
101 | ```
102 | 
103 | ### Optical Flow
104 | Firstly you need to download the [KITTI flow 2015 dataset](http://www.cvlibs.net/download.php?file=data_scene_flow.zip) and its [multi-view extension](http://www.cvlibs.net/download.php?file=data_scene_flow_multiview.zip). 
105 | For replicating our flow results in the paper, a `seq_length` of 3 is recommended. You need to format the testing data by running
106 | ```bash
107 | python kitti_eval/generate_multiview_extension.py --dataset_dir=/path/to/data_scene_flow_multiview/ --calib_dir=/path/to/data_scene_flow_calib/ --dump_root=/path/to/formatted/testdata/ --cam_id=02 --seq_length=3
108 | ```
109 | Then you can test your trained model by
110 | ```bash
111 | python geonet_main.py --mode=test_flow --dataset_dir=/path/to/formatted/testdata/ --init_ckpt_file=/path/to/trained/model/ --flownet_type=direct --batch_size=1 --output_dir=/path/to/save/predictions/
112 | ```
113 | We again provide evaluation script:
114 | ```bash
115 | python kitti_eval/eval_flow.py --dataset_dir=/path/to/kitti_stereo_2015/ --pred_dir=/path/to/predictions/
116 | ```
117 | 
118 | ## Acknowledgements
119 | We thank [Tinghui Zhou](https://github.com/tinghuiz/SfMLearner) and [Clément Godard](https://github.com/mrharicot/monodepth) for their great works and repos.
120 | 
121 | ## Reference
122 | If you find our work useful in your research please consider citing our paper:
123 | ```
124 | @inproceedings{yin2018geonet,
125 |   title     = {GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose},
126 |   author    = {Yin, Zhichao and Shi, Jianping},
127 |   booktitle = {CVPR},
128 |   year = {2018}
129 | }
130 | ```
131 | 


--------------------------------------------------------------------------------
/data/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/yzcjtr/GeoNet/5b176025bc63563ef53297aa3d20cc6e575fb833/data/__init__.py


--------------------------------------------------------------------------------
/data/cityscapes/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/yzcjtr/GeoNet/5b176025bc63563ef53297aa3d20cc6e575fb833/data/cityscapes/__init__.py


--------------------------------------------------------------------------------
/data/cityscapes/cityscapes_loader.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/data/cityscapes/cityscapes_loader.py
  3 | from __future__ import division
  4 | import json
  5 | import os
  6 | import numpy as np
  7 | import scipy.misc
  8 | from glob import glob
  9 | 
 10 | class cityscapes_loader(object):
 11 |     def __init__(self, 
 12 |                  dataset_dir,
 13 |                  split='train',
 14 |                  crop_bottom=True, # Get rid of the car logo
 15 |                  sample_gap=2,  # Sample every two frames to match KITTI frame rate
 16 |                  img_height=171, 
 17 |                  img_width=416,
 18 |                  seq_length=5):
 19 |         self.dataset_dir = dataset_dir
 20 |         self.split = split
 21 |         # Crop out the bottom 25% of the image to remove the car logo
 22 |         self.crop_bottom = crop_bottom
 23 |         self.sample_gap = sample_gap
 24 |         self.img_height = img_height
 25 |         self.img_width = img_width
 26 |         self.seq_length = seq_length
 27 |         assert seq_length % 2 != 0, 'seq_length must be odd!'
 28 |         self.frames = self.collect_frames(split)
 29 |         self.num_frames = len(self.frames)
 30 |         if split == 'train':
 31 |             self.num_train = self.num_frames
 32 |         else:
 33 |             self.num_test = self.num_frames
 34 |         print('Total frames collected: %d' % self.num_frames)
 35 |         
 36 |     def collect_frames(self, split):
 37 |         img_dir = self.dataset_dir + '/leftImg8bit_sequence/' + split + '/'
 38 |         city_list = os.listdir(img_dir)
 39 |         frames = []
 40 |         for city in city_list:
 41 |             img_files = glob(img_dir + city + '/*.png')
 42 |             for f in img_files:
 43 |                 frame_id = os.path.basename(f).split('leftImg8bit')[0]
 44 |                 frames.append(frame_id)
 45 |         return frames
 46 | 
 47 |     def get_train_example_with_idx(self, tgt_idx):
 48 |         tgt_frame_id = self.frames[tgt_idx]
 49 |         if not self.is_valid_example(tgt_frame_id):
 50 |             return False
 51 |         example = self.load_example(self.frames[tgt_idx])
 52 |         return example
 53 | 
 54 |     def load_intrinsics(self, frame_id, split):
 55 |         city, seq, _, _ = frame_id.split('_')
 56 |         camera_file = os.path.join(self.dataset_dir, 'camera',
 57 |                                    split, city, city + '_' + seq + '_*_camera.json')
 58 |         camera_file = glob(camera_file)[0]
 59 |         with open(camera_file, 'r') as f: 
 60 |             camera = json.load(f)
 61 |         fx = camera['intrinsic']['fx']
 62 |         fy = camera['intrinsic']['fy']
 63 |         u0 = camera['intrinsic']['u0']
 64 |         v0 = camera['intrinsic']['v0']
 65 |         intrinsics = np.array([[fx, 0, u0],
 66 |                                [0, fy, v0],
 67 |                                [0,  0,  1]])
 68 |         return intrinsics
 69 | 
 70 |     def is_valid_example(self, tgt_frame_id):
 71 |         city, snippet_id, tgt_local_frame_id, _ = tgt_frame_id.split('_')
 72 |         half_offset = int((self.seq_length - 1)/2 * self.sample_gap)
 73 |         for o in range(-half_offset, half_offset + 1, self.sample_gap):
 74 |             curr_local_frame_id = '%.6d' % (int(tgt_local_frame_id) + o)
 75 |             curr_frame_id = '%s_%s_%s_' % (city, snippet_id, curr_local_frame_id)
 76 |             curr_image_file = os.path.join(self.dataset_dir, 'leftImg8bit_sequence', 
 77 |                                 self.split, city, curr_frame_id + 'leftImg8bit.png')
 78 |             if not os.path.exists(curr_image_file):
 79 |                 return False
 80 |         return True
 81 | 
 82 |     def load_image_sequence(self, tgt_frame_id, seq_length, crop_bottom):
 83 |         city, snippet_id, tgt_local_frame_id, _ = tgt_frame_id.split('_')
 84 |         half_offset = int((self.seq_length - 1)/2 * self.sample_gap)
 85 |         image_seq = []
 86 |         for o in range(-half_offset, half_offset + 1, self.sample_gap):
 87 |             curr_local_frame_id = '%.6d' % (int(tgt_local_frame_id) + o)
 88 |             curr_frame_id = '%s_%s_%s_' % (city, snippet_id, curr_local_frame_id)
 89 |             curr_image_file = os.path.join(self.dataset_dir, 'leftImg8bit_sequence', 
 90 |                                 self.split, city, curr_frame_id + 'leftImg8bit.png')
 91 |             curr_img = scipy.misc.imread(curr_image_file)
 92 |             raw_shape = np.copy(curr_img.shape)
 93 |             if o == 0:
 94 |                 zoom_y = self.img_height/raw_shape[0]
 95 |                 zoom_x = self.img_width/raw_shape[1]
 96 |             curr_img = scipy.misc.imresize(curr_img, (self.img_height, self.img_width))
 97 |             if crop_bottom:
 98 |                 ymax = int(curr_img.shape[0] * 0.75)
 99 |                 curr_img = curr_img[:ymax]
100 |             image_seq.append(curr_img)
101 |         return image_seq, zoom_x, zoom_y
102 |     
103 |     def load_example(self, tgt_frame_id, load_gt_pose=False):
104 |         image_seq, zoom_x, zoom_y = self.load_image_sequence(tgt_frame_id, self.seq_length, self.crop_bottom)
105 |         intrinsics = self.load_intrinsics(tgt_frame_id, self.split)
106 |         intrinsics = self.scale_intrinsics(intrinsics, zoom_x, zoom_y)
107 |         example = {}
108 |         example['intrinsics'] = intrinsics
109 |         example['image_seq'] = image_seq
110 |         example['folder_name'] = tgt_frame_id.split('_')[0]
111 |         example['file_name'] = tgt_frame_id[:-1]
112 |         return example
113 | 
114 |     def scale_intrinsics(self, mat, sx, sy):
115 |         out = np.copy(mat)
116 |         out[0,0] *= sx
117 |         out[0,2] *= sx
118 |         out[1,1] *= sy
119 |         out[1,2] *= sy
120 |         return out


--------------------------------------------------------------------------------
/data/kitti/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/yzcjtr/GeoNet/5b176025bc63563ef53297aa3d20cc6e575fb833/data/kitti/__init__.py


--------------------------------------------------------------------------------
/data/kitti/kitti_odom_loader.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/data/kitti/kitti_odom_loader.py
  3 | from __future__ import division
  4 | import numpy as np
  5 | from glob import glob
  6 | import os
  7 | import scipy.misc
  8 | 
  9 | class kitti_odom_loader(object):
 10 |     def __init__(self,
 11 |                  dataset_dir,
 12 |                  img_height=128,
 13 |                  img_width=416,
 14 |                  seq_length=5):
 15 |         self.dataset_dir = dataset_dir
 16 |         self.img_height = img_height
 17 |         self.img_width = img_width
 18 |         self.seq_length = seq_length
 19 |         self.train_seqs = [0, 1, 2, 3, 4, 5, 6, 7, 8]
 20 |         self.test_seqs = [9, 10]
 21 | 
 22 |         self.collect_test_frames()
 23 |         self.collect_train_frames()
 24 | 
 25 |     def collect_test_frames(self):
 26 |         self.test_frames = []
 27 |         for seq in self.test_seqs:
 28 |             seq_dir = os.path.join(self.dataset_dir, 'sequences', '%.2d' % seq)
 29 |             img_dir = os.path.join(seq_dir, 'image_2')
 30 |             N = len(glob(img_dir + '/*.png'))
 31 |             for n in range(N):
 32 |                 self.test_frames.append('%.2d %.6d' % (seq, n))
 33 |         self.num_test = len(self.test_frames)
 34 |         
 35 |     def collect_train_frames(self):
 36 |         self.train_frames = []
 37 |         for seq in self.train_seqs:
 38 |             seq_dir = os.path.join(self.dataset_dir, 'sequences', '%.2d' % seq)
 39 |             img_dir = os.path.join(seq_dir, 'image_2')
 40 |             N = len(glob(img_dir + '/*.png'))
 41 |             for n in range(N):
 42 |                 self.train_frames.append('%.2d %.6d' % (seq, n))
 43 |         self.num_train = len(self.train_frames)
 44 | 
 45 |     def is_valid_sample(self, frames, tgt_idx):
 46 |         N = len(frames)
 47 |         tgt_drive, _ = frames[tgt_idx].split(' ')
 48 |         half_offset = int((self.seq_length - 1)/2)
 49 |         min_src_idx = tgt_idx - half_offset
 50 |         max_src_idx = tgt_idx + half_offset
 51 |         if min_src_idx < 0 or max_src_idx >= N:
 52 |             return False
 53 |         min_src_drive, _ = frames[min_src_idx].split(' ')
 54 |         max_src_drive, _ = frames[max_src_idx].split(' ')
 55 |         if tgt_drive == min_src_drive and tgt_drive == max_src_drive:
 56 |             return True
 57 |         return False
 58 | 
 59 |     def load_image_sequence(self, frames, tgt_idx, seq_length):
 60 |         half_offset = int((seq_length - 1)/2)
 61 |         image_seq = []
 62 |         for o in range(-half_offset, half_offset+1):
 63 |             curr_idx = tgt_idx + o
 64 |             curr_drive, curr_frame_id = frames[curr_idx].split(' ')
 65 |             curr_img = self.load_image(curr_drive, curr_frame_id)
 66 |             if o == 0:
 67 |                 zoom_y = self.img_height/curr_img.shape[0]
 68 |                 zoom_x = self.img_width/curr_img.shape[1]
 69 |             curr_img = scipy.misc.imresize(curr_img, (self.img_height, self.img_width))
 70 |             image_seq.append(curr_img)
 71 |         return image_seq, zoom_x, zoom_y
 72 | 
 73 |     def load_example(self, frames, tgt_idx):
 74 |         image_seq, zoom_x, zoom_y = self.load_image_sequence(frames, tgt_idx, self.seq_length)
 75 |         tgt_drive, tgt_frame_id = frames[tgt_idx].split(' ')
 76 |         intrinsics = self.load_intrinsics(tgt_drive, tgt_frame_id)
 77 |         intrinsics = self.scale_intrinsics(intrinsics, zoom_x, zoom_y)        
 78 |         example = {}
 79 |         example['intrinsics'] = intrinsics
 80 |         example['image_seq'] = image_seq
 81 |         example['folder_name'] = tgt_drive
 82 |         example['file_name'] = tgt_frame_id
 83 |         return example
 84 | 
 85 |     def get_train_example_with_idx(self, tgt_idx):
 86 |         if not self.is_valid_sample(self.train_frames, tgt_idx):
 87 |             return False
 88 |         example = self.load_example(self.train_frames, tgt_idx)
 89 |         return example
 90 | 
 91 |     def load_image(self, drive, frame_id):
 92 |         img_file = os.path.join(self.dataset_dir, 'sequences', '%s/image_2/%s.png' % (drive, frame_id))
 93 |         img = scipy.misc.imread(img_file)
 94 |         return img
 95 | 
 96 |     def load_intrinsics(self, drive, frame_id):
 97 |         calib_file = os.path.join(self.dataset_dir, 'sequences', '%s/calib.txt' % drive)
 98 |         proj_c2p, _ = self.read_calib_file(calib_file)
 99 |         intrinsics = proj_c2p[:3, :3]
100 |         return intrinsics
101 | 
102 |     def read_calib_file(self, filepath, cid=2):
103 |         """Read in a calibration file and parse into a dictionary."""
104 |         with open(filepath, 'r') as f:
105 |             C = f.readlines()
106 |         def parseLine(L, shape):
107 |             data = L.split()
108 |             data = np.array(data[1:]).reshape(shape).astype(np.float32)
109 |             return data
110 |         proj_c2p = parseLine(C[cid], shape=(3,4))
111 |         proj_v2c = parseLine(C[-1], shape=(3,4))
112 |         filler = np.array([0, 0, 0, 1]).reshape((1,4))
113 |         proj_v2c = np.concatenate((proj_v2c, filler), axis=0)
114 |         return proj_c2p, proj_v2c
115 | 
116 |     def scale_intrinsics(self,mat, sx, sy):
117 |         out = np.copy(mat)
118 |         out[0,0] *= sx
119 |         out[0,2] *= sx
120 |         out[1,1] *= sy
121 |         out[1,2] *= sy
122 |         return out
123 | 


--------------------------------------------------------------------------------
/data/kitti/kitti_raw_loader.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/data/kitti/kitti_raw_loader.py
  3 | from __future__ import division
  4 | import numpy as np
  5 | from glob import glob
  6 | import os
  7 | import scipy.misc
  8 | 
  9 | class kitti_raw_loader(object):
 10 |     def __init__(self, 
 11 |                  dataset_dir,
 12 |                  split,
 13 |                  img_height=128,
 14 |                  img_width=416,
 15 |                  seq_length=5,
 16 |                  remove_static=True):
 17 |         dir_path = os.path.dirname(os.path.realpath(__file__))        
 18 |         test_scene_file = dir_path + '/test_scenes_' + split + '.txt'
 19 |         with open(test_scene_file, 'r') as f:
 20 |             test_scenes = f.readlines()
 21 |         self.test_scenes = [t[:-1] for t in test_scenes]
 22 |         self.dataset_dir = dataset_dir
 23 |         self.img_height = img_height
 24 |         self.img_width = img_width
 25 |         self.seq_length = seq_length
 26 |         self.cam_ids = ['02', '03']
 27 |         self.date_list = ['2011_09_26', '2011_09_28', '2011_09_29', 
 28 |                           '2011_09_30', '2011_10_03']
 29 |         if remove_static:
 30 |             static_frames_file = dir_path + '/static_frames.txt'
 31 |             self.collect_static_frames(static_frames_file)
 32 |         self.collect_train_frames(remove_static)
 33 | 
 34 |     def collect_static_frames(self, static_frames_file):
 35 |         with open(static_frames_file, 'r') as f:
 36 |             frames = f.readlines()
 37 |         self.static_frames = []
 38 |         for fr in frames:
 39 |             if fr == '\n':
 40 |                 continue
 41 |             date, drive, frame_id = fr.split(' ')
 42 |             curr_fid = '%.10d' % (np.int(frame_id[:-1]))
 43 |             for cid in self.cam_ids:
 44 |                 self.static_frames.append(drive + ' ' + cid + ' ' + curr_fid)
 45 |         
 46 |     def collect_train_frames(self, remove_static):
 47 |         all_frames = []
 48 |         for date in self.date_list:
 49 |             drive_set = os.listdir(self.dataset_dir + date + '/')
 50 |             for dr in drive_set:
 51 |                 drive_dir = os.path.join(self.dataset_dir, date, dr)
 52 |                 if os.path.isdir(drive_dir):
 53 |                     if dr[:-5] in self.test_scenes:
 54 |                         continue
 55 |                     for cam in self.cam_ids:
 56 |                         img_dir = os.path.join(drive_dir, 'image_' + cam, 'data')
 57 |                         N = len(glob(img_dir + '/*.png'))
 58 |                         for n in range(N):
 59 |                             frame_id = '%.10d' % n
 60 |                             all_frames.append(dr + ' ' + cam + ' ' + frame_id)
 61 |                         
 62 |         if remove_static:
 63 |             for s in self.static_frames:
 64 |                 try:
 65 |                     all_frames.remove(s)
 66 |                 except:
 67 |                     pass
 68 | 
 69 |         self.train_frames = all_frames
 70 |         self.num_train = len(self.train_frames)
 71 | 
 72 |     def is_valid_sample(self, frames, tgt_idx):
 73 |         N = len(frames)
 74 |         tgt_drive, cid, _ = frames[tgt_idx].split(' ')
 75 |         half_offset = int((self.seq_length - 1)/2)
 76 |         min_src_idx = tgt_idx - half_offset
 77 |         max_src_idx = tgt_idx + half_offset
 78 |         if min_src_idx < 0 or max_src_idx >= N:
 79 |             return False
 80 |         min_src_drive, min_src_cid, _ = frames[min_src_idx].split(' ')
 81 |         max_src_drive, max_src_cid, _ = frames[max_src_idx].split(' ')
 82 |         if tgt_drive == min_src_drive and tgt_drive == max_src_drive and cid == min_src_cid and cid == max_src_cid:
 83 |             return True
 84 |         return False
 85 | 
 86 |     def get_train_example_with_idx(self, tgt_idx):
 87 |         if not self.is_valid_sample(self.train_frames, tgt_idx):
 88 |             return False
 89 |         example = self.load_example(self.train_frames, tgt_idx)
 90 |         return example
 91 | 
 92 |     def load_image_sequence(self, frames, tgt_idx, seq_length):
 93 |         half_offset = int((seq_length - 1)/2)
 94 |         image_seq = []
 95 |         for o in range(-half_offset, half_offset + 1):
 96 |             curr_idx = tgt_idx + o
 97 |             curr_drive, curr_cid, curr_frame_id = frames[curr_idx].split(' ')
 98 |             curr_img = self.load_image_raw(curr_drive, curr_cid, curr_frame_id)
 99 |             if o == 0:
100 |                 zoom_y = self.img_height/curr_img.shape[0]
101 |                 zoom_x = self.img_width/curr_img.shape[1]
102 |             curr_img = scipy.misc.imresize(curr_img, (self.img_height, self.img_width))
103 |             image_seq.append(curr_img)
104 |         return image_seq, zoom_x, zoom_y
105 | 
106 |     def load_example(self, frames, tgt_idx):
107 |         image_seq, zoom_x, zoom_y = self.load_image_sequence(frames, tgt_idx, self.seq_length)
108 |         tgt_drive, tgt_cid, tgt_frame_id = frames[tgt_idx].split(' ')
109 |         intrinsics = self.load_intrinsics_raw(tgt_drive, tgt_cid, tgt_frame_id)
110 |         intrinsics = self.scale_intrinsics(intrinsics, zoom_x, zoom_y)
111 |         example = {}
112 |         example['intrinsics'] = intrinsics
113 |         example['image_seq'] = image_seq
114 |         example['folder_name'] = tgt_drive + '_' + tgt_cid + '/'
115 |         example['file_name'] = tgt_frame_id
116 |         return example
117 | 
118 |     def load_image_raw(self, drive, cid, frame_id):
119 |         date = drive[:10]
120 |         img_file = os.path.join(self.dataset_dir, date, drive, 'image_' + cid, 'data', frame_id + '.png')
121 |         img = scipy.misc.imread(img_file)
122 |         return img
123 | 
124 |     def load_intrinsics_raw(self, drive, cid, frame_id):
125 |         date = drive[:10]
126 |         calib_file = os.path.join(self.dataset_dir, date, 'calib_cam_to_cam.txt')
127 | 
128 |         filedata = self.read_raw_calib_file(calib_file)
129 |         P_rect = np.reshape(filedata['P_rect_' + cid], (3, 4))
130 |         intrinsics = P_rect[:3, :3]
131 |         return intrinsics
132 | 
133 |     def read_raw_calib_file(self,filepath):
134 |         # From https://github.com/utiasSTARS/pykitti/blob/master/pykitti/utils.py
135 |         """Read in a calibration file and parse into a dictionary."""
136 |         data = {}
137 | 
138 |         with open(filepath, 'r') as f:
139 |             for line in f.readlines():
140 |                 key, value = line.split(':', 1)
141 |                 # The only non-float values in these files are dates, which
142 |                 # we don't care about anyway
143 |                 try:
144 |                         data[key] = np.array([float(x) for x in value.split()])
145 |                 except ValueError:
146 |                         pass
147 |         return data
148 | 
149 |     def scale_intrinsics(self, mat, sx, sy):
150 |         out = np.copy(mat)
151 |         out[0,0] *= sx
152 |         out[0,2] *= sx
153 |         out[1,1] *= sy
154 |         out[1,2] *= sy
155 |         return out
156 | 
157 | 
158 | 


--------------------------------------------------------------------------------
/data/kitti/test_files_eigen.txt:
--------------------------------------------------------------------------------
  1 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000069.png
  2 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000054.png
  3 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000042.png
  4 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000057.png
  5 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000030.png
  6 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000027.png
  7 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000012.png
  8 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000075.png
  9 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000036.png
 10 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000033.png
 11 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000015.png
 12 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000072.png
 13 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000003.png
 14 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000039.png
 15 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000009.png
 16 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000051.png
 17 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000060.png
 18 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000021.png
 19 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000000.png
 20 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000024.png
 21 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000045.png
 22 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000018.png
 23 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000048.png
 24 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000006.png
 25 | 2011_09_26/2011_09_26_drive_0002_sync/image_02/data/0000000063.png
 26 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000000.png
 27 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000016.png
 28 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000032.png
 29 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000048.png
 30 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000064.png
 31 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000080.png
 32 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000096.png
 33 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000112.png
 34 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000128.png
 35 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000144.png
 36 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000160.png
 37 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000176.png
 38 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000196.png
 39 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000212.png
 40 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000228.png
 41 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000244.png
 42 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000260.png
 43 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000276.png
 44 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000292.png
 45 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000308.png
 46 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000324.png
 47 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000340.png
 48 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000356.png
 49 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000372.png
 50 | 2011_09_26/2011_09_26_drive_0009_sync/image_02/data/0000000388.png
 51 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000090.png
 52 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000050.png
 53 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000110.png
 54 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000115.png
 55 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000060.png
 56 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000105.png
 57 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000125.png
 58 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000020.png
 59 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000140.png
 60 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000085.png
 61 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000070.png
 62 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000080.png
 63 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000065.png
 64 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000095.png
 65 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000130.png
 66 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000100.png
 67 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000010.png
 68 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000030.png
 69 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000000.png
 70 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000135.png
 71 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000040.png
 72 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000005.png
 73 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000120.png
 74 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000045.png
 75 | 2011_09_26/2011_09_26_drive_0013_sync/image_02/data/0000000035.png
 76 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000003.png
 77 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000069.png
 78 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000057.png
 79 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000012.png
 80 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000072.png
 81 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000018.png
 82 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000063.png
 83 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000000.png
 84 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000084.png
 85 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000015.png
 86 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000066.png
 87 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000006.png
 88 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000048.png
 89 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000060.png
 90 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000009.png
 91 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000033.png
 92 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000021.png
 93 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000075.png
 94 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000027.png
 95 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000045.png
 96 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000078.png
 97 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000036.png
 98 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000051.png
 99 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000054.png
100 | 2011_09_26/2011_09_26_drive_0020_sync/image_02/data/0000000042.png
101 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000018.png
102 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000090.png
103 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000126.png
104 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000378.png
105 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000036.png
106 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000288.png
107 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000198.png
108 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000450.png
109 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000144.png
110 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000072.png
111 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000252.png
112 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000180.png
113 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000432.png
114 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000396.png
115 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000054.png
116 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000468.png
117 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000306.png
118 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000108.png
119 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000162.png
120 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000342.png
121 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000270.png
122 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000414.png
123 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000216.png
124 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000360.png
125 | 2011_09_26/2011_09_26_drive_0023_sync/image_02/data/0000000324.png
126 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000077.png
127 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000035.png
128 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000091.png
129 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000112.png
130 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000007.png
131 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000175.png
132 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000042.png
133 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000098.png
134 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000133.png
135 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000161.png
136 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000014.png
137 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000126.png
138 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000168.png
139 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000070.png
140 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000084.png
141 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000140.png
142 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000049.png
143 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000000.png
144 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000182.png
145 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000147.png
146 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000056.png
147 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000063.png
148 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000021.png
149 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000119.png
150 | 2011_09_26/2011_09_26_drive_0027_sync/image_02/data/0000000028.png
151 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000380.png
152 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000394.png
153 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000324.png
154 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000000.png
155 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000268.png
156 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000366.png
157 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000296.png
158 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000014.png
159 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000028.png
160 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000182.png
161 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000168.png
162 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000196.png
163 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000140.png
164 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000084.png
165 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000056.png
166 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000112.png
167 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000352.png
168 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000126.png
169 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000070.png
170 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000310.png
171 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000154.png
172 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000098.png
173 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000408.png
174 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000042.png
175 | 2011_09_26/2011_09_26_drive_0029_sync/image_02/data/0000000338.png
176 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000000.png
177 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000128.png
178 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000192.png
179 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000032.png
180 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000352.png
181 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000608.png
182 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000224.png
183 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000576.png
184 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000672.png
185 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000064.png
186 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000448.png
187 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000704.png
188 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000640.png
189 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000512.png
190 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000768.png
191 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000160.png
192 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000416.png
193 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000480.png
194 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000800.png
195 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000288.png
196 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000544.png
197 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000096.png
198 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000384.png
199 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000256.png
200 | 2011_09_26/2011_09_26_drive_0036_sync/image_02/data/0000000320.png
201 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000000.png
202 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000005.png
203 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000010.png
204 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000015.png
205 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000020.png
206 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000025.png
207 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000030.png
208 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000035.png
209 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000040.png
210 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000045.png
211 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000050.png
212 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000055.png
213 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000060.png
214 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000065.png
215 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000070.png
216 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000075.png
217 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000080.png
218 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000085.png
219 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000090.png
220 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000095.png
221 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000100.png
222 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000105.png
223 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000110.png
224 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000115.png
225 | 2011_09_26/2011_09_26_drive_0046_sync/image_02/data/0000000120.png
226 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000000.png
227 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000001.png
228 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000002.png
229 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000003.png
230 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000004.png
231 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000005.png
232 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000006.png
233 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000007.png
234 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000008.png
235 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000009.png
236 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000010.png
237 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000011.png
238 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000012.png
239 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000013.png
240 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000014.png
241 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000015.png
242 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000016.png
243 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000017.png
244 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000018.png
245 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000019.png
246 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000020.png
247 | 2011_09_26/2011_09_26_drive_0048_sync/image_02/data/0000000021.png
248 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000046.png
249 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000014.png
250 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000036.png
251 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000028.png
252 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000026.png
253 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000050.png
254 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000040.png
255 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000008.png
256 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000016.png
257 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000044.png
258 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000018.png
259 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000032.png
260 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000042.png
261 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000010.png
262 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000020.png
263 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000048.png
264 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000052.png
265 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000006.png
266 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000030.png
267 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000012.png
268 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000038.png
269 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000000.png
270 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000002.png
271 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000004.png
272 | 2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000022.png
273 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000011.png
274 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000033.png
275 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000242.png
276 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000253.png
277 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000286.png
278 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000154.png
279 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000099.png
280 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000220.png
281 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000022.png
282 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000077.png
283 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000187.png
284 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000143.png
285 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000066.png
286 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000176.png
287 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000110.png
288 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000275.png
289 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000264.png
290 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000198.png
291 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000055.png
292 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000088.png
293 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000121.png
294 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000209.png
295 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000165.png
296 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000231.png
297 | 2011_09_26/2011_09_26_drive_0056_sync/image_02/data/0000000044.png
298 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000056.png
299 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000000.png
300 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000344.png
301 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000358.png
302 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000316.png
303 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000238.png
304 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000098.png
305 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000112.png
306 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000028.png
307 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000014.png
308 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000330.png
309 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000154.png
310 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000042.png
311 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000302.png
312 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000182.png
313 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000288.png
314 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000140.png
315 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000274.png
316 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000224.png
317 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000372.png
318 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000196.png
319 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000126.png
320 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000084.png
321 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000210.png
322 | 2011_09_26/2011_09_26_drive_0059_sync/image_02/data/0000000070.png
323 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000528.png
324 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000308.png
325 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000044.png
326 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000352.png
327 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000066.png
328 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000000.png
329 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000506.png
330 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000176.png
331 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000022.png
332 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000242.png
333 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000462.png
334 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000418.png
335 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000110.png
336 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000440.png
337 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000396.png
338 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000154.png
339 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000374.png
340 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000088.png
341 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000286.png
342 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000550.png
343 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000264.png
344 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000220.png
345 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000330.png
346 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000484.png
347 | 2011_09_26/2011_09_26_drive_0064_sync/image_02/data/0000000198.png
348 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000283.png
349 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000361.png
350 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000270.png
351 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000127.png
352 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000205.png
353 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000218.png
354 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000153.png
355 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000335.png
356 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000192.png
357 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000348.png
358 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000101.png
359 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000049.png
360 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000179.png
361 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000140.png
362 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000374.png
363 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000322.png
364 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000309.png
365 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000244.png
366 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000062.png
367 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000257.png
368 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000088.png
369 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000114.png
370 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000075.png
371 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000296.png
372 | 2011_09_26/2011_09_26_drive_0084_sync/image_02/data/0000000231.png
373 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000007.png
374 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000196.png
375 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000439.png
376 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000169.png
377 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000115.png
378 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000034.png
379 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000304.png
380 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000331.png
381 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000277.png
382 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000520.png
383 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000682.png
384 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000628.png
385 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000088.png
386 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000601.png
387 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000574.png
388 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000223.png
389 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000655.png
390 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000358.png
391 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000412.png
392 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000142.png
393 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000385.png
394 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000061.png
395 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000493.png
396 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000466.png
397 | 2011_09_26/2011_09_26_drive_0086_sync/image_02/data/0000000250.png
398 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000000.png
399 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000016.png
400 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000032.png
401 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000048.png
402 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000064.png
403 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000080.png
404 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000096.png
405 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000112.png
406 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000128.png
407 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000144.png
408 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000160.png
409 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000176.png
410 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000192.png
411 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000208.png
412 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000224.png
413 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000240.png
414 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000256.png
415 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000305.png
416 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000321.png
417 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000337.png
418 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000353.png
419 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000369.png
420 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000385.png
421 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000401.png
422 | 2011_09_26/2011_09_26_drive_0093_sync/image_02/data/0000000417.png
423 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000000.png
424 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000019.png
425 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000038.png
426 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000057.png
427 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000076.png
428 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000095.png
429 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000114.png
430 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000133.png
431 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000152.png
432 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000171.png
433 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000190.png
434 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000209.png
435 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000228.png
436 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000247.png
437 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000266.png
438 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000285.png
439 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000304.png
440 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000323.png
441 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000342.png
442 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000361.png
443 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000380.png
444 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000399.png
445 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000418.png
446 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000437.png
447 | 2011_09_26/2011_09_26_drive_0096_sync/image_02/data/0000000456.png
448 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000692.png
449 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000930.png
450 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000760.png
451 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000896.png
452 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000284.png
453 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000148.png
454 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000522.png
455 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000794.png
456 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000624.png
457 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000726.png
458 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000216.png
459 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000318.png
460 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000488.png
461 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000590.png
462 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000454.png
463 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000862.png
464 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000386.png
465 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000352.png
466 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000420.png
467 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000658.png
468 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000828.png
469 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000556.png
470 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000114.png
471 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000182.png
472 | 2011_09_26/2011_09_26_drive_0101_sync/image_02/data/0000000080.png
473 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000015.png
474 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000035.png
475 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000043.png
476 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000051.png
477 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000059.png
478 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000067.png
479 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000075.png
480 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000083.png
481 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000091.png
482 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000099.png
483 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000107.png
484 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000115.png
485 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000123.png
486 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000131.png
487 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000139.png
488 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000147.png
489 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000155.png
490 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000163.png
491 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000171.png
492 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000179.png
493 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000187.png
494 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000195.png
495 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000203.png
496 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000211.png
497 | 2011_09_26/2011_09_26_drive_0106_sync/image_02/data/0000000219.png
498 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000312.png
499 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000494.png
500 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000104.png
501 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000130.png
502 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000156.png
503 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000182.png
504 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000598.png
505 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000416.png
506 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000364.png
507 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000026.png
508 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000078.png
509 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000572.png
510 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000468.png
511 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000260.png
512 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000624.png
513 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000234.png
514 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000442.png
515 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000390.png
516 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000546.png
517 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000286.png
518 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000000.png
519 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000338.png
520 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000208.png
521 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000650.png
522 | 2011_09_26/2011_09_26_drive_0117_sync/image_02/data/0000000052.png
523 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000024.png
524 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000021.png
525 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000036.png
526 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000000.png
527 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000051.png
528 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000018.png
529 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000033.png
530 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000090.png
531 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000045.png
532 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000054.png
533 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000012.png
534 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000039.png
535 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000009.png
536 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000003.png
537 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000030.png
538 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000078.png
539 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000060.png
540 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000048.png
541 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000084.png
542 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000081.png
543 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000006.png
544 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000057.png
545 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000072.png
546 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000087.png
547 | 2011_09_28/2011_09_28_drive_0002_sync/image_02/data/0000000063.png
548 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000252.png
549 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000540.png
550 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000001054.png
551 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000036.png
552 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000360.png
553 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000807.png
554 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000879.png
555 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000288.png
556 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000771.png
557 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000000.png
558 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000216.png
559 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000951.png
560 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000324.png
561 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000432.png
562 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000504.png
563 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000576.png
564 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000108.png
565 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000180.png
566 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000072.png
567 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000612.png
568 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000915.png
569 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000735.png
570 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000144.png
571 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000396.png
572 | 2011_09_29/2011_09_29_drive_0071_sync/image_02/data/0000000468.png
573 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000132.png
574 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000011.png
575 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000154.png
576 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000022.png
577 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000242.png
578 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000198.png
579 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000176.png
580 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000231.png
581 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000275.png
582 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000220.png
583 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000088.png
584 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000143.png
585 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000055.png
586 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000033.png
587 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000187.png
588 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000110.png
589 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000044.png
590 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000077.png
591 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000066.png
592 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000000.png
593 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000165.png
594 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000264.png
595 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000253.png
596 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000209.png
597 | 2011_09_30/2011_09_30_drive_0016_sync/image_02/data/0000000121.png
598 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000107.png
599 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002247.png
600 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001391.png
601 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000535.png
602 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001819.png
603 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001177.png
604 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000428.png
605 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001926.png
606 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000749.png
607 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001284.png
608 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002140.png
609 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001605.png
610 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001498.png
611 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000642.png
612 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002740.png
613 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002419.png
614 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000856.png
615 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002526.png
616 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001712.png
617 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000001070.png
618 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000000.png
619 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002033.png
620 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000214.png
621 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000000963.png
622 | 2011_09_30/2011_09_30_drive_0018_sync/image_02/data/0000002633.png
623 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000533.png
624 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000001040.png
625 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000082.png
626 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000205.png
627 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000835.png
628 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000451.png
629 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000164.png
630 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000794.png
631 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000328.png
632 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000615.png
633 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000917.png
634 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000369.png
635 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000287.png
636 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000123.png
637 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000876.png
638 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000410.png
639 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000492.png
640 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000958.png
641 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000656.png
642 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000000.png
643 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000753.png
644 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000574.png
645 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000001081.png
646 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000041.png
647 | 2011_09_30/2011_09_30_drive_0027_sync/image_02/data/0000000246.png
648 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000002906.png
649 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000002544.png
650 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000362.png
651 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000004535.png
652 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000734.png
653 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000001096.png
654 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000004173.png
655 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000543.png
656 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000001277.png
657 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000004354.png
658 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000001458.png
659 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000001820.png
660 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003449.png
661 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003268.png
662 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000915.png
663 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000002363.png
664 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000002725.png
665 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000181.png
666 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000001639.png
667 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003992.png
668 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003087.png
669 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000002001.png
670 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003811.png
671 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000003630.png
672 | 2011_10_03/2011_10_03_drive_0027_sync/image_02/data/0000000000.png
673 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000096.png
674 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000800.png
675 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000320.png
676 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000576.png
677 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000000.png
678 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000480.png
679 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000640.png
680 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000032.png
681 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000384.png
682 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000160.png
683 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000704.png
684 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000736.png
685 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000672.png
686 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000064.png
687 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000288.png
688 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000352.png
689 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000512.png
690 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000544.png
691 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000608.png
692 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000128.png
693 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000224.png
694 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000416.png
695 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000192.png
696 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000448.png
697 | 2011_10_03/2011_10_03_drive_0047_sync/image_02/data/0000000768.png
698 | 


--------------------------------------------------------------------------------
/data/kitti/test_files_stereo.txt:
--------------------------------------------------------------------------------
  1 | training/image_2/000000_10.png
  2 | training/image_2/000001_10.png
  3 | training/image_2/000002_10.png
  4 | training/image_2/000003_10.png
  5 | training/image_2/000004_10.png
  6 | training/image_2/000005_10.png
  7 | training/image_2/000006_10.png
  8 | training/image_2/000007_10.png
  9 | training/image_2/000008_10.png
 10 | training/image_2/000009_10.png
 11 | training/image_2/000010_10.png
 12 | training/image_2/000011_10.png
 13 | training/image_2/000012_10.png
 14 | training/image_2/000013_10.png
 15 | training/image_2/000014_10.png
 16 | training/image_2/000015_10.png
 17 | training/image_2/000016_10.png
 18 | training/image_2/000017_10.png
 19 | training/image_2/000018_10.png
 20 | training/image_2/000019_10.png
 21 | training/image_2/000020_10.png
 22 | training/image_2/000021_10.png
 23 | training/image_2/000022_10.png
 24 | training/image_2/000023_10.png
 25 | training/image_2/000024_10.png
 26 | training/image_2/000025_10.png
 27 | training/image_2/000026_10.png
 28 | training/image_2/000027_10.png
 29 | training/image_2/000028_10.png
 30 | training/image_2/000029_10.png
 31 | training/image_2/000030_10.png
 32 | training/image_2/000031_10.png
 33 | training/image_2/000032_10.png
 34 | training/image_2/000033_10.png
 35 | training/image_2/000034_10.png
 36 | training/image_2/000035_10.png
 37 | training/image_2/000036_10.png
 38 | training/image_2/000037_10.png
 39 | training/image_2/000038_10.png
 40 | training/image_2/000039_10.png
 41 | training/image_2/000040_10.png
 42 | training/image_2/000041_10.png
 43 | training/image_2/000042_10.png
 44 | training/image_2/000043_10.png
 45 | training/image_2/000044_10.png
 46 | training/image_2/000045_10.png
 47 | training/image_2/000046_10.png
 48 | training/image_2/000047_10.png
 49 | training/image_2/000048_10.png
 50 | training/image_2/000049_10.png
 51 | training/image_2/000050_10.png
 52 | training/image_2/000051_10.png
 53 | training/image_2/000052_10.png
 54 | training/image_2/000053_10.png
 55 | training/image_2/000054_10.png
 56 | training/image_2/000055_10.png
 57 | training/image_2/000056_10.png
 58 | training/image_2/000057_10.png
 59 | training/image_2/000058_10.png
 60 | training/image_2/000059_10.png
 61 | training/image_2/000060_10.png
 62 | training/image_2/000061_10.png
 63 | training/image_2/000062_10.png
 64 | training/image_2/000063_10.png
 65 | training/image_2/000064_10.png
 66 | training/image_2/000065_10.png
 67 | training/image_2/000066_10.png
 68 | training/image_2/000067_10.png
 69 | training/image_2/000068_10.png
 70 | training/image_2/000069_10.png
 71 | training/image_2/000070_10.png
 72 | training/image_2/000071_10.png
 73 | training/image_2/000072_10.png
 74 | training/image_2/000073_10.png
 75 | training/image_2/000074_10.png
 76 | training/image_2/000075_10.png
 77 | training/image_2/000076_10.png
 78 | training/image_2/000077_10.png
 79 | training/image_2/000078_10.png
 80 | training/image_2/000079_10.png
 81 | training/image_2/000080_10.png
 82 | training/image_2/000081_10.png
 83 | training/image_2/000082_10.png
 84 | training/image_2/000083_10.png
 85 | training/image_2/000084_10.png
 86 | training/image_2/000085_10.png
 87 | training/image_2/000086_10.png
 88 | training/image_2/000087_10.png
 89 | training/image_2/000088_10.png
 90 | training/image_2/000089_10.png
 91 | training/image_2/000090_10.png
 92 | training/image_2/000091_10.png
 93 | training/image_2/000092_10.png
 94 | training/image_2/000093_10.png
 95 | training/image_2/000094_10.png
 96 | training/image_2/000095_10.png
 97 | training/image_2/000096_10.png
 98 | training/image_2/000097_10.png
 99 | training/image_2/000098_10.png
100 | training/image_2/000099_10.png
101 | training/image_2/000100_10.png
102 | training/image_2/000101_10.png
103 | training/image_2/000102_10.png
104 | training/image_2/000103_10.png
105 | training/image_2/000104_10.png
106 | training/image_2/000105_10.png
107 | training/image_2/000106_10.png
108 | training/image_2/000107_10.png
109 | training/image_2/000108_10.png
110 | training/image_2/000109_10.png
111 | training/image_2/000110_10.png
112 | training/image_2/000111_10.png
113 | training/image_2/000112_10.png
114 | training/image_2/000113_10.png
115 | training/image_2/000114_10.png
116 | training/image_2/000115_10.png
117 | training/image_2/000116_10.png
118 | training/image_2/000117_10.png
119 | training/image_2/000118_10.png
120 | training/image_2/000119_10.png
121 | training/image_2/000120_10.png
122 | training/image_2/000121_10.png
123 | training/image_2/000122_10.png
124 | training/image_2/000123_10.png
125 | training/image_2/000124_10.png
126 | training/image_2/000125_10.png
127 | training/image_2/000126_10.png
128 | training/image_2/000127_10.png
129 | training/image_2/000128_10.png
130 | training/image_2/000129_10.png
131 | training/image_2/000130_10.png
132 | training/image_2/000131_10.png
133 | training/image_2/000132_10.png
134 | training/image_2/000133_10.png
135 | training/image_2/000134_10.png
136 | training/image_2/000135_10.png
137 | training/image_2/000136_10.png
138 | training/image_2/000137_10.png
139 | training/image_2/000138_10.png
140 | training/image_2/000139_10.png
141 | training/image_2/000140_10.png
142 | training/image_2/000141_10.png
143 | training/image_2/000142_10.png
144 | training/image_2/000143_10.png
145 | training/image_2/000144_10.png
146 | training/image_2/000145_10.png
147 | training/image_2/000146_10.png
148 | training/image_2/000147_10.png
149 | training/image_2/000148_10.png
150 | training/image_2/000149_10.png
151 | training/image_2/000150_10.png
152 | training/image_2/000151_10.png
153 | training/image_2/000152_10.png
154 | training/image_2/000153_10.png
155 | training/image_2/000154_10.png
156 | training/image_2/000155_10.png
157 | training/image_2/000156_10.png
158 | training/image_2/000157_10.png
159 | training/image_2/000158_10.png
160 | training/image_2/000159_10.png
161 | training/image_2/000160_10.png
162 | training/image_2/000161_10.png
163 | training/image_2/000162_10.png
164 | training/image_2/000163_10.png
165 | training/image_2/000164_10.png
166 | training/image_2/000165_10.png
167 | training/image_2/000166_10.png
168 | training/image_2/000167_10.png
169 | training/image_2/000168_10.png
170 | training/image_2/000169_10.png
171 | training/image_2/000170_10.png
172 | training/image_2/000171_10.png
173 | training/image_2/000172_10.png
174 | training/image_2/000173_10.png
175 | training/image_2/000174_10.png
176 | training/image_2/000175_10.png
177 | training/image_2/000176_10.png
178 | training/image_2/000177_10.png
179 | training/image_2/000178_10.png
180 | training/image_2/000179_10.png
181 | training/image_2/000180_10.png
182 | training/image_2/000181_10.png
183 | training/image_2/000182_10.png
184 | training/image_2/000183_10.png
185 | training/image_2/000184_10.png
186 | training/image_2/000185_10.png
187 | training/image_2/000186_10.png
188 | training/image_2/000187_10.png
189 | training/image_2/000188_10.png
190 | training/image_2/000189_10.png
191 | training/image_2/000190_10.png
192 | training/image_2/000191_10.png
193 | training/image_2/000192_10.png
194 | training/image_2/000193_10.png
195 | training/image_2/000194_10.png
196 | training/image_2/000195_10.png
197 | training/image_2/000196_10.png
198 | training/image_2/000197_10.png
199 | training/image_2/000198_10.png
200 | training/image_2/000199_10.png
201 | 


--------------------------------------------------------------------------------
/data/kitti/test_scenes_eigen.txt:
--------------------------------------------------------------------------------
 1 | 2011_09_26_drive_0117
 2 | 2011_09_28_drive_0002
 3 | 2011_09_26_drive_0052
 4 | 2011_09_30_drive_0016
 5 | 2011_09_26_drive_0059
 6 | 2011_09_26_drive_0027
 7 | 2011_09_26_drive_0020
 8 | 2011_09_26_drive_0009
 9 | 2011_09_26_drive_0013
10 | 2011_09_26_drive_0101
11 | 2011_09_26_drive_0046
12 | 2011_09_26_drive_0029
13 | 2011_09_26_drive_0064
14 | 2011_09_26_drive_0048
15 | 2011_10_03_drive_0027
16 | 2011_09_26_drive_0002
17 | 2011_09_26_drive_0036
18 | 2011_09_29_drive_0071
19 | 2011_10_03_drive_0047
20 | 2011_09_30_drive_0027
21 | 2011_09_26_drive_0086
22 | 2011_09_26_drive_0084
23 | 2011_09_26_drive_0096
24 | 2011_09_30_drive_0018
25 | 2011_09_26_drive_0106
26 | 2011_09_26_drive_0056
27 | 2011_09_26_drive_0023
28 | 2011_09_26_drive_0093
29 | 


--------------------------------------------------------------------------------
/data/kitti/test_scenes_stereo.txt:
--------------------------------------------------------------------------------
 1 | 2011_09_26_drive_0005
 2 | 2011_09_26_drive_0009
 3 | 2011_09_26_drive_0011
 4 | 2011_09_26_drive_0013
 5 | 2011_09_26_drive_0014
 6 | 2011_09_26_drive_0015
 7 | 2011_09_26_drive_0017
 8 | 2011_09_26_drive_0018
 9 | 2011_09_26_drive_0019
10 | 2011_09_26_drive_0022
11 | 2011_09_26_drive_0027
12 | 2011_09_26_drive_0028
13 | 2011_09_26_drive_0029
14 | 2011_09_26_drive_0032
15 | 2011_09_26_drive_0036
16 | 2011_09_26_drive_0046
17 | 2011_09_26_drive_0051
18 | 2011_09_26_drive_0056
19 | 2011_09_26_drive_0057
20 | 2011_09_26_drive_0059
21 | 2011_09_26_drive_0070
22 | 2011_09_26_drive_0084
23 | 2011_09_26_drive_0096
24 | 2011_09_26_drive_0101
25 | 2011_09_26_drive_0104
26 | 2011_09_28_drive_0002
27 | 2011_09_29_drive_0004
28 | 2011_09_29_drive_0071
29 | 2011_10_03_drive_0047
30 | 


--------------------------------------------------------------------------------
/data/prepare_train_data.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/data/prepare_train_data.py
  3 | from __future__ import division
  4 | import argparse
  5 | import scipy.misc
  6 | import numpy as np
  7 | from glob import glob
  8 | from joblib import Parallel, delayed
  9 | import os
 10 | 
 11 | parser = argparse.ArgumentParser()
 12 | parser.add_argument("--dataset_dir",   type=str, required=True, help="where the dataset is stored")
 13 | parser.add_argument("--dataset_name",  type=str, required=True, choices=["kitti_raw_eigen", "kitti_raw_stereo", "kitti_odom", "cityscapes"])
 14 | parser.add_argument("--dump_root",     type=str, required=True, help="where to dump the data")
 15 | parser.add_argument("--seq_length",    type=int, required=True, help="length of each training sequence")
 16 | parser.add_argument("--img_height",    type=int, default=128,   help="image height")
 17 | parser.add_argument("--img_width",     type=int, default=416,   help="image width")
 18 | parser.add_argument("--num_threads",   type=int, default=4,     help="number of threads to use")
 19 | parser.add_argument("--remove_static", help="remove static frames from kitti raw data", action='store_true')
 20 | args = parser.parse_args()
 21 | 
 22 | def concat_image_seq(seq):
 23 |     for i, im in enumerate(seq):
 24 |         if i == 0:
 25 |             res = im
 26 |         else:
 27 |             res = np.hstack((res, im))
 28 |     return res
 29 | 
 30 | def dump_example(n, args):
 31 |     if n % 2000 == 0:
 32 |         print('Progress %d/%d....' % (n, data_loader.num_train))
 33 |     example = data_loader.get_train_example_with_idx(n)
 34 |     if example == False:
 35 |         return
 36 |     image_seq = concat_image_seq(example['image_seq'])
 37 |     intrinsics = example['intrinsics']
 38 |     fx = intrinsics[0, 0]
 39 |     fy = intrinsics[1, 1]
 40 |     cx = intrinsics[0, 2]
 41 |     cy = intrinsics[1, 2]
 42 |     dump_dir = os.path.join(args.dump_root, example['folder_name'])
 43 | 
 44 |     try: 
 45 |         os.makedirs(dump_dir)
 46 |     except OSError:
 47 |         if not os.path.isdir(dump_dir):
 48 |             raise
 49 |     dump_img_file = dump_dir + '/%s.jpg' % example['file_name']
 50 |     scipy.misc.imsave(dump_img_file, image_seq.astype(np.uint8))
 51 |     dump_cam_file = dump_dir + '/%s_cam.txt' % example['file_name']
 52 |     with open(dump_cam_file, 'w') as f:
 53 |         f.write('%f,0.,%f,0.,%f,%f,0.,0.,1.' % (fx, cx, fy, cy))
 54 | 
 55 | def main():
 56 |     if not os.path.exists(args.dump_root):
 57 |         os.makedirs(args.dump_root)
 58 | 
 59 |     global data_loader
 60 |     if args.dataset_name == 'kitti_odom':
 61 |         from kitti.kitti_odom_loader import kitti_odom_loader
 62 |         data_loader = kitti_odom_loader(args.dataset_dir,
 63 |                                         img_height=args.img_height,
 64 |                                         img_width=args.img_width,
 65 |                                         seq_length=args.seq_length)
 66 | 
 67 |     if args.dataset_name == 'kitti_raw_eigen':
 68 |         from kitti.kitti_raw_loader import kitti_raw_loader
 69 |         data_loader = kitti_raw_loader(args.dataset_dir,
 70 |                                        split='eigen',
 71 |                                        img_height=args.img_height,
 72 |                                        img_width=args.img_width,
 73 |                                        seq_length=args.seq_length,
 74 |                                        remove_static=args.remove_static)
 75 | 
 76 |     if args.dataset_name == 'kitti_raw_stereo':
 77 |         from kitti.kitti_raw_loader import kitti_raw_loader
 78 |         data_loader = kitti_raw_loader(args.dataset_dir,
 79 |                                        split='stereo',
 80 |                                        img_height=args.img_height,
 81 |                                        img_width=args.img_width,
 82 |                                        seq_length=args.seq_length,
 83 |                                        remove_static=args.remove_static)
 84 | 
 85 |     if args.dataset_name == 'cityscapes':
 86 |         from cityscapes.cityscapes_loader import cityscapes_loader
 87 |         data_loader = cityscapes_loader(args.dataset_dir,
 88 |                                         img_height=args.img_height,
 89 |                                         img_width=args.img_width,
 90 |                                         seq_length=args.seq_length)
 91 | 
 92 |     Parallel(n_jobs=args.num_threads)(delayed(dump_example)(n, args) for n in range(data_loader.num_train))
 93 | 
 94 |     # Split into train/val
 95 |     np.random.seed(8964)
 96 |     subfolders = os.listdir(args.dump_root)
 97 |     with open(os.path.join(args.dump_root, 'train.txt'), 'w') as tf:
 98 |         with open(os.path.join(args.dump_root, 'val.txt'), 'w') as vf:
 99 |             for s in subfolders:
100 |                 if not os.path.isdir(args.dump_root + '/%s' % s):
101 |                     continue
102 |                 imfiles = glob(os.path.join(args.dump_root, s, '*.jpg'))
103 |                 frame_ids = [os.path.basename(fi).split('.')[0] for fi in imfiles]
104 |                 for frame in frame_ids:
105 |                     if np.random.random() < 0.1:
106 |                         vf.write('%s %s\n' % (s, frame))
107 |                     else:
108 |                         tf.write('%s %s\n' % (s, frame))
109 | 
110 | main()
111 | 
112 | 


--------------------------------------------------------------------------------
/data_loader.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou & Clement Godard: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/data_loader.py
  3 | # https://github.com/mrharicot/monodepth/blob/master/monodepth_dataloader.py
  4 | from __future__ import division
  5 | import os
  6 | import random
  7 | import tensorflow as tf
  8 | 
  9 | class DataLoader(object):
 10 |     def __init__(self, opt=None):
 11 |         self.opt = opt
 12 | 
 13 |     def load_train_batch(self):
 14 |         """Load a batch of training instances.
 15 |         """
 16 |         opt = self.opt
 17 | 
 18 |         # Load the list of training files into queues
 19 |         file_list = self.format_file_list(opt.dataset_dir, 'train')
 20 |         image_paths_queue = tf.train.string_input_producer(
 21 |             file_list['image_file_list'], shuffle=False)
 22 |         cam_paths_queue = tf.train.string_input_producer(
 23 |             file_list['cam_file_list'], shuffle=False)
 24 | 
 25 |         # Load images
 26 |         img_reader = tf.WholeFileReader()
 27 |         _, image_contents = img_reader.read(image_paths_queue)
 28 |         image_seq = tf.image.decode_jpeg(image_contents)
 29 |         tgt_image, src_image_stack = \
 30 |             self.unpack_image_sequence(
 31 |                 image_seq, opt.img_height, opt.img_width, opt.num_source)
 32 | 
 33 |         # Load camera intrinsics
 34 |         cam_reader = tf.TextLineReader()
 35 |         _, raw_cam_contents = cam_reader.read(cam_paths_queue)
 36 |         rec_def = []
 37 |         for i in range(9):
 38 |             rec_def.append([1.])
 39 |         raw_cam_vec = tf.decode_csv(raw_cam_contents, 
 40 |                                     record_defaults=rec_def)
 41 |         raw_cam_vec = tf.stack(raw_cam_vec)
 42 |         intrinsics = tf.reshape(raw_cam_vec, [3, 3])
 43 | 
 44 |         # Form training batches
 45 |         seed = random.randint(0, 2**31 - 1)
 46 | 
 47 |         min_after_dequeue = 2048
 48 |         capacity = min_after_dequeue + opt.num_threads * opt.batch_size
 49 |         src_image_stack, tgt_image, intrinsics = \
 50 |             tf.train.shuffle_batch([src_image_stack, tgt_image, intrinsics], opt.batch_size, 
 51 |                                     capacity, min_after_dequeue, opt.num_threads, seed)
 52 | 
 53 |         # Data augmentation
 54 |         image_all = tf.concat([tgt_image, src_image_stack], axis=3)
 55 |         image_all, intrinsics = self.data_augmentation(
 56 |             image_all, intrinsics, opt.img_height, opt.img_width)
 57 |         tgt_image = image_all[:, :, :, :3]
 58 |         src_image_stack = image_all[:, :, :, 3:]
 59 |         intrinsics = self.get_multi_scale_intrinsics(
 60 |             intrinsics, opt.num_scales)
 61 |         return tgt_image, src_image_stack, intrinsics
 62 | 
 63 |     def make_intrinsics_matrix(self, fx, fy, cx, cy):
 64 |         # Assumes batch input
 65 |         batch_size = fx.get_shape().as_list()[0]
 66 |         zeros = tf.zeros_like(fx)
 67 |         r1 = tf.stack([fx, zeros, cx], axis=1)
 68 |         r2 = tf.stack([zeros, fy, cy], axis=1)
 69 |         r3 = tf.constant([0.,0.,1.], shape=[1, 3])
 70 |         r3 = tf.tile(r3, [batch_size, 1])
 71 |         intrinsics = tf.stack([r1, r2, r3], axis=1)
 72 |         return intrinsics
 73 | 
 74 |     def data_augmentation(self, im, intrinsics, out_h, out_w):
 75 |         # Random scaling
 76 |         def random_scaling(im, intrinsics):
 77 |             batch_size, in_h, in_w, _ = im.get_shape().as_list()
 78 |             scaling = tf.random_uniform([2], 1, 1.15)
 79 |             x_scaling = scaling[0]
 80 |             y_scaling = scaling[1]
 81 |             out_h = tf.cast(in_h * y_scaling, dtype=tf.int32)
 82 |             out_w = tf.cast(in_w * x_scaling, dtype=tf.int32)
 83 |             im = tf.image.resize_area(im, [out_h, out_w])
 84 |             fx = intrinsics[:,0,0] * x_scaling
 85 |             fy = intrinsics[:,1,1] * y_scaling
 86 |             cx = intrinsics[:,0,2] * x_scaling
 87 |             cy = intrinsics[:,1,2] * y_scaling
 88 |             intrinsics = self.make_intrinsics_matrix(fx, fy, cx, cy)
 89 |             return im, intrinsics
 90 | 
 91 |         # Random cropping
 92 |         def random_cropping(im, intrinsics, out_h, out_w):
 93 |             # batch_size, in_h, in_w, _ = im.get_shape().as_list()
 94 |             batch_size, in_h, in_w, _ = tf.unstack(tf.shape(im))
 95 |             offset_y = tf.random_uniform([1], 0, in_h - out_h + 1, dtype=tf.int32)[0]
 96 |             offset_x = tf.random_uniform([1], 0, in_w - out_w + 1, dtype=tf.int32)[0]
 97 |             im = tf.image.crop_to_bounding_box(
 98 |                 im, offset_y, offset_x, out_h, out_w)
 99 |             fx = intrinsics[:,0,0]
100 |             fy = intrinsics[:,1,1]
101 |             cx = intrinsics[:,0,2] - tf.cast(offset_x, dtype=tf.float32)
102 |             cy = intrinsics[:,1,2] - tf.cast(offset_y, dtype=tf.float32)
103 |             intrinsics = self.make_intrinsics_matrix(fx, fy, cx, cy)
104 |             return im, intrinsics
105 | 
106 |         # Random coloring
107 |         def random_coloring(im):
108 |             batch_size, in_h, in_w, in_c = im.get_shape().as_list()
109 |             im_f = tf.image.convert_image_dtype(im, tf.float32)
110 | 
111 |             # randomly shift gamma
112 |             random_gamma = tf.random_uniform([], 0.8, 1.2)
113 |             im_aug  = im_f  ** random_gamma
114 | 
115 |             # randomly shift brightness
116 |             random_brightness = tf.random_uniform([], 0.5, 2.0)
117 |             im_aug  =  im_aug * random_brightness
118 | 
119 |             # randomly shift color
120 |             random_colors = tf.random_uniform([in_c], 0.8, 1.2)
121 |             white = tf.ones([batch_size, in_h, in_w])
122 |             color_image = tf.stack([white * random_colors[i] for i in range(in_c)], axis=3)
123 |             im_aug  *= color_image
124 | 
125 |             # saturate
126 |             im_aug  = tf.clip_by_value(im_aug,  0, 1)
127 | 
128 |             im_aug = tf.image.convert_image_dtype(im_aug, tf.uint8)
129 | 
130 |             return im_aug
131 |         im, intrinsics = random_scaling(im, intrinsics)
132 |         im, intrinsics = random_cropping(im, intrinsics, out_h, out_w)
133 |         im = tf.cast(im, dtype=tf.uint8)
134 |         do_augment  = tf.random_uniform([], 0, 1)
135 |         im = tf.cond(do_augment > 0.5, lambda: random_coloring(im), lambda: im)
136 |         return im, intrinsics
137 | 
138 |     def format_file_list(self, data_root, split):
139 |         with open(data_root + '/%s.txt' % split, 'r') as f:
140 |             frames = f.readlines()
141 |         subfolders = [x.split(' ')[0] for x in frames]
142 |         frame_ids = [x.split(' ')[1][:-1] for x in frames]
143 |         image_file_list = [os.path.join(data_root, subfolders[i], 
144 |             frame_ids[i] + '.jpg') for i in range(len(frames))]
145 |         cam_file_list = [os.path.join(data_root, subfolders[i], 
146 |             frame_ids[i] + '_cam.txt') for i in range(len(frames))]
147 |         all_list = {}
148 |         all_list['image_file_list'] = image_file_list
149 |         all_list['cam_file_list'] = cam_file_list
150 |         return all_list
151 | 
152 |     def unpack_image_sequence(self, image_seq, img_height, img_width, num_source):
153 |         # Assuming the center image is the target frame
154 |         tgt_start_idx = int(img_width * (num_source//2))
155 |         tgt_image = tf.slice(image_seq, 
156 |                              [0, tgt_start_idx, 0], 
157 |                              [-1, img_width, -1])
158 |         # Source frames before the target frame
159 |         src_image_1 = tf.slice(image_seq, 
160 |                                [0, 0, 0], 
161 |                                [-1, int(img_width * (num_source//2)), -1])
162 |         # Source frames after the target frame
163 |         src_image_2 = tf.slice(image_seq, 
164 |                                [0, int(tgt_start_idx + img_width), 0], 
165 |                                [-1, int(img_width * (num_source//2)), -1])
166 |         src_image_seq = tf.concat([src_image_1, src_image_2], axis=1)
167 |         # Stack source frames along the color channels (i.e. [H, W, N*3])
168 |         src_image_stack = tf.concat([tf.slice(src_image_seq, 
169 |                                     [0, i*img_width, 0], 
170 |                                     [-1, img_width, -1]) 
171 |                                     for i in range(num_source)], axis=2)
172 |         src_image_stack.set_shape([img_height, 
173 |                                    img_width, 
174 |                                    num_source * 3])
175 |         tgt_image.set_shape([img_height, img_width, 3])
176 |         return tgt_image, src_image_stack
177 | 
178 |     def get_multi_scale_intrinsics(self, intrinsics, num_scales):
179 |         intrinsics_mscale = []
180 |         # Scale the intrinsics accordingly for each scale
181 |         for s in range(num_scales):
182 |             fx = intrinsics[:,0,0]/(2 ** s)
183 |             fy = intrinsics[:,1,1]/(2 ** s)
184 |             cx = intrinsics[:,0,2]/(2 ** s)
185 |             cy = intrinsics[:,1,2]/(2 ** s)
186 |             intrinsics_mscale.append(
187 |                 self.make_intrinsics_matrix(fx, fy, cx, cy))
188 |         intrinsics_mscale = tf.stack(intrinsics_mscale, axis=1)
189 |         return intrinsics_mscale


--------------------------------------------------------------------------------
/geonet_main.py:
--------------------------------------------------------------------------------
  1 | from __future__ import division
  2 | import os
  3 | import time
  4 | import random
  5 | import pprint
  6 | import numpy as np
  7 | import tensorflow as tf
  8 | import tensorflow.contrib.slim as slim
  9 | 
 10 | from geonet_model import *
 11 | from geonet_test_depth import *
 12 | from geonet_test_pose import *
 13 | from geonet_test_flow import *
 14 | from data_loader import DataLoader
 15 | 
 16 | flags = tf.app.flags
 17 | flags.DEFINE_string("mode",                         "",    "(train_rigid, train_flow) or (test_depth, test_pose, test_flow)")
 18 | flags.DEFINE_string("dataset_dir",                  "",    "Dataset directory")
 19 | flags.DEFINE_string("init_ckpt_file",             None,    "Specific checkpoint file to initialize from")
 20 | flags.DEFINE_integer("batch_size",                   4,    "The size of of a sample batch")
 21 | flags.DEFINE_integer("num_threads",                 32,    "Number of threads for data loading")
 22 | flags.DEFINE_integer("img_height",                 128,    "Image height")
 23 | flags.DEFINE_integer("img_width",                  416,    "Image width")
 24 | flags.DEFINE_integer("seq_length",                   3,    "Sequence length for each example")
 25 | 
 26 | ##### Training Configurations #####
 27 | flags.DEFINE_string("checkpoint_dir",               "",    "Directory name to save the checkpoints")
 28 | flags.DEFINE_float("learning_rate",             0.0002,    "Learning rate for adam")
 29 | flags.DEFINE_integer("max_to_keep",                 20,    "Maximum number of checkpoints to save")
 30 | flags.DEFINE_integer("max_steps",               300000,    "Maximum number of training iterations")
 31 | flags.DEFINE_integer("save_ckpt_freq",            5000,    "Save the checkpoint model every save_ckpt_freq iterations")
 32 | flags.DEFINE_float("alpha_recon_image",           0.85,    "Alpha weight between SSIM and L1 in reconstruction loss")
 33 | 
 34 | ##### Configurations about DepthNet & PoseNet of GeoNet #####
 35 | flags.DEFINE_string("dispnet_encoder",      "resnet50",    "Type of encoder for dispnet, vgg or resnet50")
 36 | flags.DEFINE_boolean("scale_normalize",          False,    "Spatially normalize depth prediction")
 37 | flags.DEFINE_float("rigid_warp_weight",            1.0,    "Weight for warping by rigid flow")
 38 | flags.DEFINE_float("disp_smooth_weight",           0.5,    "Weight for disp smoothness")
 39 | 
 40 | ##### Configurations about ResFlowNet of GeoNet (or DirFlowNetS) #####
 41 | flags.DEFINE_string("flownet_type",         "residual",    "type of flownet, residual or direct")
 42 | flags.DEFINE_float("flow_warp_weight",             1.0,    "Weight for warping by full flow")
 43 | flags.DEFINE_float("flow_smooth_weight",           0.2,    "Weight for flow smoothness")
 44 | flags.DEFINE_float("flow_consistency_weight",      0.2,    "Weight for bidirectional flow consistency")
 45 | flags.DEFINE_float("flow_consistency_alpha",       3.0,    "Alpha for flow consistency check")
 46 | flags.DEFINE_float("flow_consistency_beta",       0.05,    "Beta for flow consistency check")
 47 | 
 48 | ##### Testing Configurations #####
 49 | flags.DEFINE_string("output_dir",                 None,    "Test result output directory")
 50 | flags.DEFINE_string("depth_test_split",        "eigen",    "KITTI depth split, eigen or stereo")
 51 | flags.DEFINE_integer("pose_test_seq",                9,    "KITTI Odometry Sequence ID to test")
 52 | 
 53 | 
 54 | opt = flags.FLAGS
 55 | 
 56 | def train():
 57 | 
 58 |     seed = 8964
 59 |     tf.set_random_seed(seed)
 60 |     np.random.seed(seed)
 61 |     random.seed(seed)
 62 | 
 63 |     pp = pprint.PrettyPrinter()
 64 |     pp.pprint(flags.FLAGS.__flags)
 65 | 
 66 |     if not os.path.exists(opt.checkpoint_dir):
 67 |         os.makedirs(opt.checkpoint_dir)
 68 | 
 69 |     with tf.Graph().as_default():
 70 |         # Data Loader
 71 |         loader = DataLoader(opt)
 72 |         tgt_image, src_image_stack, intrinsics = loader.load_train_batch()
 73 | 
 74 |         # Build Model
 75 |         model = GeoNetModel(opt, tgt_image, src_image_stack, intrinsics)
 76 |         loss = model.total_loss
 77 | 
 78 |         # Train Op
 79 |         if opt.mode == 'train_flow' and opt.flownet_type == "residual":
 80 |             # we pretrain DepthNet & PoseNet, then finetune ResFlowNetS
 81 |             train_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, "flow_net")
 82 |             vars_to_restore = slim.get_variables_to_restore(include=["depth_net", "pose_net"])
 83 |         else:
 84 |             train_vars = [var for var in tf.trainable_variables()]
 85 |             vars_to_restore = slim.get_model_variables()
 86 | 
 87 |         if opt.init_ckpt_file != None:
 88 |             init_assign_op, init_feed_dict = slim.assign_from_checkpoint(
 89 |                                             opt.init_ckpt_file, vars_to_restore)
 90 | 
 91 |         optim = tf.train.AdamOptimizer(opt.learning_rate, 0.9)
 92 |         train_op = slim.learning.create_train_op(loss, optim,
 93 |                                                  variables_to_train=train_vars)
 94 | 
 95 |         # Global Step
 96 |         global_step = tf.Variable(0,
 97 |                                 name='global_step',
 98 |                                 trainable=False)
 99 |         incr_global_step = tf.assign(global_step,
100 |                                      global_step+1)
101 | 
102 |         # Parameter Count
103 |         parameter_count = tf.reduce_sum([tf.reduce_prod(tf.shape(v)) \
104 |                                         for v in train_vars])
105 | 
106 |         # Saver
107 |         saver = tf.train.Saver([var for var in tf.model_variables()] + \
108 |                                 [global_step],
109 |                                 max_to_keep=opt.max_to_keep)
110 | 
111 |         # Session
112 |         sv = tf.train.Supervisor(logdir=opt.checkpoint_dir,
113 |                                  save_summaries_secs=0,
114 |                                  saver=None)
115 |         config = tf.ConfigProto()
116 |         config.gpu_options.allow_growth = True
117 | 
118 |         with sv.managed_session(config=config) as sess:
119 |             print('Trainable variables: ')
120 |             for var in train_vars:
121 |                 print(var.name)
122 |             print("parameter_count =", sess.run(parameter_count))
123 | 
124 |             if opt.init_ckpt_file != None:
125 |                 sess.run(init_assign_op, init_feed_dict)
126 |             start_time = time.time()
127 | 
128 |             for step in range(1, opt.max_steps):
129 |                 fetches = {
130 |                     "train": train_op,
131 |                     "global_step": global_step,
132 |                     "incr_global_step": incr_global_step
133 |                 }
134 |                 if step % 100 == 0:
135 |                     fetches["loss"] = loss
136 |                 results = sess.run(fetches)
137 |                 if step % 100 == 0:
138 |                     time_per_iter = (time.time() - start_time) / 100
139 |                     start_time = time.time()
140 |                     print('Iteration: [%7d] | Time: %4.4fs/iter | Loss: %.3f' \
141 |                           % (step, time_per_iter, results["loss"]))
142 |                 if step % opt.save_ckpt_freq == 0:
143 |                     saver.save(sess, os.path.join(opt.checkpoint_dir, 'model'), global_step=step)
144 | 
145 | def main(_):
146 | 
147 |     opt.num_source = opt.seq_length - 1
148 |     opt.num_scales = 4
149 | 
150 |     opt.add_flownet = opt.mode in ['train_flow', 'test_flow']
151 |     opt.add_dispnet = opt.add_flownet and opt.flownet_type == 'residual' \
152 |                       or opt.mode in ['train_rigid', 'test_depth']
153 |     opt.add_posenet = opt.add_flownet and opt.flownet_type == 'residual' \
154 |                       or opt.mode in ['train_rigid', 'test_pose']
155 | 
156 |     if opt.mode in ['train_rigid', 'train_flow']:
157 |         train()
158 |     elif opt.mode == 'test_depth':
159 |         test_depth(opt)
160 |     elif opt.mode == 'test_pose':
161 |         test_pose(opt)
162 |     elif opt.mode == 'test_flow':
163 |         test_flow(opt)
164 | 
165 | if __name__ == '__main__':
166 |     tf.app.run()
167 | 


--------------------------------------------------------------------------------
/geonet_model.py:
--------------------------------------------------------------------------------
  1 | from __future__ import division
  2 | import os
  3 | import time
  4 | import math
  5 | import numpy as np
  6 | import tensorflow as tf
  7 | import tensorflow.contrib.slim as slim
  8 | from geonet_nets import *
  9 | from utils import *
 10 | 
 11 | class GeoNetModel(object):
 12 | 
 13 |     def __init__(self, opt, tgt_image, src_image_stack, intrinsics):
 14 |         self.opt = opt
 15 |         self.tgt_image = self.preprocess_image(tgt_image)
 16 |         self.src_image_stack = self.preprocess_image(src_image_stack)
 17 |         self.intrinsics = intrinsics
 18 | 
 19 |         self.build_model()
 20 | 
 21 |         if not opt.mode in ['train_rigid', 'train_flow']:
 22 |             return
 23 | 
 24 |         self.build_losses()
 25 | 
 26 |     def build_model(self):
 27 |         opt = self.opt
 28 |         self.tgt_image_pyramid = self.scale_pyramid(self.tgt_image, opt.num_scales)
 29 |         self.tgt_image_tile_pyramid = [tf.tile(img, [opt.num_source, 1, 1, 1]) \
 30 |                                       for img in self.tgt_image_pyramid]
 31 |         # src images concated along batch dimension
 32 |         if self.src_image_stack != None:
 33 |             self.src_image_concat = tf.concat([self.src_image_stack[:,:,:,3*i:3*(i+1)] \
 34 |                                     for i in range(opt.num_source)], axis=0)
 35 |             self.src_image_concat_pyramid = self.scale_pyramid(self.src_image_concat, opt.num_scales)
 36 | 
 37 |         if opt.add_dispnet:
 38 |             self.build_dispnet()
 39 | 
 40 |         if opt.add_posenet:
 41 |             self.build_posenet()
 42 | 
 43 |         if opt.add_dispnet and opt.add_posenet:
 44 |             self.build_rigid_flow_warping()
 45 | 
 46 |         if opt.add_flownet:
 47 |             self.build_flownet()
 48 |             if opt.mode == 'train_flow':
 49 |                 self.build_full_flow_warping()
 50 |                 if opt.flow_consistency_weight > 0:
 51 |                     self.build_flow_consistency()
 52 | 
 53 |     def build_dispnet(self):
 54 |         opt = self.opt
 55 | 
 56 |         # build dispnet_inputs
 57 |         if opt.mode == 'test_depth':
 58 |             # for test_depth mode we only predict the depth of the target image
 59 |             self.dispnet_inputs = self.tgt_image
 60 |         else:
 61 |             # multiple depth predictions; tgt: disp[:bs,:,:,:] src.i: disp[bs*(i+1):bs*(i+2),:,:,:]
 62 |             self.dispnet_inputs = self.tgt_image
 63 |             for i in range(opt.num_source):
 64 |                 self.dispnet_inputs = tf.concat([self.dispnet_inputs, self.src_image_stack[:,:,:,3*i:3*(i+1)]], axis=0)
 65 |         
 66 |         # build dispnet
 67 |         self.pred_disp = disp_net(opt, self.dispnet_inputs)
 68 | 
 69 |         if opt.scale_normalize:
 70 |             # As proposed in https://arxiv.org/abs/1712.00175, this can 
 71 |             # bring improvement in depth estimation, but not included in our paper.
 72 |             self.pred_disp = [self.spatial_normalize(disp) for disp in self.pred_disp]
 73 | 
 74 |         self.pred_depth = [1./d for d in self.pred_disp]
 75 | 
 76 |     def build_posenet(self):
 77 |         opt = self.opt
 78 | 
 79 |         # build posenet_inputs
 80 |         self.posenet_inputs = tf.concat([self.tgt_image, self.src_image_stack], axis=3)
 81 |         
 82 |         # build posenet
 83 |         self.pred_poses = pose_net(opt, self.posenet_inputs)
 84 | 
 85 |     def build_rigid_flow_warping(self):
 86 |         opt = self.opt
 87 |         bs = opt.batch_size
 88 | 
 89 |         # build rigid flow (fwd: tgt->src, bwd: src->tgt)
 90 |         self.fwd_rigid_flow_pyramid = []
 91 |         self.bwd_rigid_flow_pyramid = []
 92 |         for s in range(opt.num_scales):
 93 |             for i in range(opt.num_source):
 94 |                 fwd_rigid_flow = compute_rigid_flow(tf.squeeze(self.pred_depth[s][:bs], axis=3),
 95 |                                  self.pred_poses[:,i,:], self.intrinsics[:,s,:,:], False)
 96 |                 bwd_rigid_flow = compute_rigid_flow(tf.squeeze(self.pred_depth[s][bs*(i+1):bs*(i+2)], axis=3),
 97 |                                  self.pred_poses[:,i,:], self.intrinsics[:,s,:,:], True)
 98 |                 if not i:
 99 |                     fwd_rigid_flow_concat = fwd_rigid_flow
100 |                     bwd_rigid_flow_concat = bwd_rigid_flow
101 |                 else:
102 |                     fwd_rigid_flow_concat = tf.concat([fwd_rigid_flow_concat, fwd_rigid_flow], axis=0)
103 |                     bwd_rigid_flow_concat = tf.concat([bwd_rigid_flow_concat, bwd_rigid_flow], axis=0)
104 |             self.fwd_rigid_flow_pyramid.append(fwd_rigid_flow_concat)
105 |             self.bwd_rigid_flow_pyramid.append(bwd_rigid_flow_concat)
106 | 
107 |         # warping by rigid flow
108 |         self.fwd_rigid_warp_pyramid = [flow_warp(self.src_image_concat_pyramid[s], self.fwd_rigid_flow_pyramid[s]) \
109 |                                       for s in range(opt.num_scales)]
110 |         self.bwd_rigid_warp_pyramid = [flow_warp(self.tgt_image_tile_pyramid[s], self.bwd_rigid_flow_pyramid[s]) \
111 |                                       for s in range(opt.num_scales)]
112 | 
113 |         # compute reconstruction error  
114 |         self.fwd_rigid_error_pyramid = [self.image_similarity(self.fwd_rigid_warp_pyramid[s], self.tgt_image_tile_pyramid[s]) \
115 |                                        for s in range(opt.num_scales)]      
116 |         self.bwd_rigid_error_pyramid = [self.image_similarity(self.bwd_rigid_warp_pyramid[s], self.src_image_concat_pyramid[s]) \
117 |                                        for s in range(opt.num_scales)]
118 |     
119 |     def build_flownet(self):
120 |         opt = self.opt
121 | 
122 |         # build flownet_inputs
123 |         self.fwd_flownet_inputs = tf.concat([self.tgt_image_tile_pyramid[0], self.src_image_concat_pyramid[0]], axis=3)
124 |         self.bwd_flownet_inputs = tf.concat([self.src_image_concat_pyramid[0], self.tgt_image_tile_pyramid[0]], axis=3)
125 |         if opt.flownet_type == 'residual':
126 |             self.fwd_flownet_inputs = tf.concat([self.fwd_flownet_inputs,
127 |                                       self.fwd_rigid_warp_pyramid[0],
128 |                                       self.fwd_rigid_flow_pyramid[0],
129 |                                       self.L2_norm(self.fwd_rigid_error_pyramid[0])], axis=3)
130 |             self.bwd_flownet_inputs = tf.concat([self.bwd_flownet_inputs,
131 |                                       self.bwd_rigid_warp_pyramid[0],
132 |                                       self.bwd_rigid_flow_pyramid[0],
133 |                                       self.L2_norm(self.bwd_rigid_error_pyramid[0])], axis=3)
134 |         self.flownet_inputs = tf.concat([self.fwd_flownet_inputs, self.bwd_flownet_inputs], axis=0)
135 |         
136 |         # build flownet
137 |         self.pred_flow = flow_net(opt, self.flownet_inputs)
138 | 
139 |         # unnormalize pyramid flow back into pixel metric
140 |         for s in range(opt.num_scales):
141 |             curr_bs, curr_h, curr_w, _ = self.pred_flow[s].get_shape().as_list()
142 |             scale_factor = tf.cast(tf.constant([curr_w, curr_h], shape=[1,1,1,2]), 'float32')
143 |             scale_factor = tf.tile(scale_factor, [curr_bs, curr_h, curr_w, 1])
144 |             self.pred_flow[s] = self.pred_flow[s] * scale_factor
145 | 
146 |         # split forward/backward flows
147 |         self.fwd_full_flow_pyramid = [self.pred_flow[s][:opt.batch_size*opt.num_source] for s in range(opt.num_scales)]
148 |         self.bwd_full_flow_pyramid = [self.pred_flow[s][opt.batch_size*opt.num_source:] for s in range(opt.num_scales)]
149 | 
150 |         # residual flow postprocessing
151 |         if opt.flownet_type == 'residual':
152 |             self.fwd_full_flow_pyramid = [self.fwd_full_flow_pyramid[s] + self.fwd_rigid_flow_pyramid[s] for s in range(opt.num_scales)]
153 |             self.bwd_full_flow_pyramid = [self.bwd_full_flow_pyramid[s] + self.bwd_rigid_flow_pyramid[s] for s in range(opt.num_scales)]   
154 | 
155 |     def build_full_flow_warping(self):
156 |         opt = self.opt
157 |         
158 |         # warping by full flow
159 |         self.fwd_full_warp_pyramid = [flow_warp(self.src_image_concat_pyramid[s], self.fwd_full_flow_pyramid[s]) \
160 |                                       for s in range(opt.num_scales)]
161 |         self.bwd_full_warp_pyramid = [flow_warp(self.tgt_image_tile_pyramid[s], self.bwd_full_flow_pyramid[s]) \
162 |                                       for s in range(opt.num_scales)]
163 | 
164 |         # compute reconstruction error  
165 |         self.fwd_full_error_pyramid = [self.image_similarity(self.fwd_full_warp_pyramid[s], self.tgt_image_tile_pyramid[s]) \
166 |                                        for s in range(opt.num_scales)]      
167 |         self.bwd_full_error_pyramid = [self.image_similarity(self.bwd_full_warp_pyramid[s], self.src_image_concat_pyramid[s]) \
168 |                                        for s in range(opt.num_scales)]    
169 | 
170 |     def build_flow_consistency(self):
171 |         opt = self.opt
172 | 
173 |         # warp pyramid full flow
174 |         self.bwd2fwd_flow_pyramid = [flow_warp(self.bwd_full_flow_pyramid[s], self.fwd_full_flow_pyramid[s]) \
175 |                                     for s in range(opt.num_scales)]
176 |         self.fwd2bwd_flow_pyramid = [flow_warp(self.fwd_full_flow_pyramid[s], self.bwd_full_flow_pyramid[s]) \
177 |                                     for s in range(opt.num_scales)]
178 | 
179 |         # calculate flow consistency
180 |         self.fwd_flow_diff_pyramid = [tf.abs(self.bwd2fwd_flow_pyramid[s] + self.fwd_full_flow_pyramid[s]) for s in range(opt.num_scales)]
181 |         self.bwd_flow_diff_pyramid = [tf.abs(self.fwd2bwd_flow_pyramid[s] + self.bwd_full_flow_pyramid[s]) for s in range(opt.num_scales)]
182 | 
183 |         # build flow consistency condition
184 |         self.fwd_consist_bound = [opt.flow_consistency_beta * self.L2_norm(self.fwd_full_flow_pyramid[s]) * 2**s for s in range(opt.num_scales)]
185 |         self.bwd_consist_bound = [opt.flow_consistency_beta * self.L2_norm(self.bwd_full_flow_pyramid[s]) * 2**s for s in range(opt.num_scales)]
186 |         self.fwd_consist_bound = [tf.stop_gradient(tf.maximum(v, opt.flow_consistency_alpha)) for v in self.fwd_consist_bound]
187 |         self.bwd_consist_bound = [tf.stop_gradient(tf.maximum(v, opt.flow_consistency_alpha)) for v in self.bwd_consist_bound]
188 | 
189 |         # build flow consistency mask
190 |         self.noc_masks_src = [tf.cast(tf.less(self.L2_norm(self.bwd_flow_diff_pyramid[s]) * 2**s, 
191 |                              self.bwd_consist_bound[s]), tf.float32) for s in range(opt.num_scales)]
192 |         self.noc_masks_tgt = [tf.cast(tf.less(self.L2_norm(self.fwd_flow_diff_pyramid[s]) * 2**s,
193 |                              self.fwd_consist_bound[s]), tf.float32) for s in range(opt.num_scales)]
194 | 
195 |     def build_losses(self):
196 |         opt = self.opt
197 |         bs = opt.batch_size
198 |         rigid_warp_loss = 0
199 |         disp_smooth_loss = 0
200 |         flow_warp_loss = 0
201 |         flow_smooth_loss = 0
202 |         flow_consistency_loss = 0
203 | 
204 |         for s in range(opt.num_scales):
205 |             # rigid_warp_loss
206 |             if opt.mode == 'train_rigid' and opt.rigid_warp_weight > 0:
207 |                 rigid_warp_loss += opt.rigid_warp_weight*opt.num_source/2 * \
208 |                                 (tf.reduce_mean(self.fwd_rigid_error_pyramid[s]) + \
209 |                                  tf.reduce_mean(self.bwd_rigid_error_pyramid[s]))
210 | 
211 |             # disp_smooth_loss
212 |             if opt.mode == 'train_rigid' and opt.disp_smooth_weight > 0:
213 |                 disp_smooth_loss += opt.disp_smooth_weight/(2**s) * self.compute_smooth_loss(self.pred_disp[s],
214 |                                 tf.concat([self.tgt_image_pyramid[s], self.src_image_concat_pyramid[s]], axis=0))
215 | 
216 |             # flow_warp_loss
217 |             if opt.mode == 'train_flow' and opt.flow_warp_weight > 0:
218 |                 if opt.flow_consistency_weight == 0:
219 |                     flow_warp_loss += opt.flow_warp_weight*opt.num_source/2 * \
220 |                                 (tf.reduce_mean(self.fwd_full_error_pyramid[s]) + tf.reduce_mean(self.bwd_full_error_pyramid[s]))
221 |                 else:
222 |                     flow_warp_loss += opt.flow_warp_weight*opt.num_source/2 * \
223 |                                 (tf.reduce_sum(tf.reduce_mean(self.fwd_full_error_pyramid[s], axis=3, keep_dims=True) * \
224 |                                  self.noc_masks_tgt[s]) / tf.reduce_sum(self.noc_masks_tgt[s]) + \
225 |                                  tf.reduce_sum(tf.reduce_mean(self.bwd_full_error_pyramid[s], axis=3, keep_dims=True) * \
226 |                                  self.noc_masks_src[s]) / tf.reduce_sum(self.noc_masks_src[s]))
227 | 
228 |             # flow_smooth_loss
229 |             if opt.mode == 'train_flow' and opt.flow_smooth_weight > 0:
230 |                 flow_smooth_loss += opt.flow_smooth_weight/(2**(s+1)) * \
231 |                                 (self.compute_flow_smooth_loss(self.fwd_full_flow_pyramid[s], self.tgt_image_tile_pyramid[s]) +
232 |                                 self.compute_flow_smooth_loss(self.bwd_full_flow_pyramid[s], self.src_image_concat_pyramid[s]))
233 | 
234 |             # flow_consistency_loss
235 |             if opt.mode == 'train_flow' and opt.flow_consistency_weight > 0:
236 |                 flow_consistency_loss += opt.flow_consistency_weight/2 * \
237 |                                 (tf.reduce_sum(tf.reduce_mean(self.fwd_flow_diff_pyramid[s] , axis=3, keep_dims=True) * \
238 |                                  self.noc_masks_tgt[s]) / tf.reduce_sum(self.noc_masks_tgt[s]) + \
239 |                                  tf.reduce_sum(tf.reduce_mean(self.bwd_flow_diff_pyramid[s] , axis=3, keep_dims=True) * \
240 |                                  self.noc_masks_src[s]) / tf.reduce_sum(self.noc_masks_src[s]))
241 | 
242 |         regularization_loss = tf.add_n(tf.losses.get_regularization_losses())
243 |         self.total_loss = 0  # regularization_loss
244 |         if opt.mode == 'train_rigid':
245 |             self.total_loss += rigid_warp_loss + disp_smooth_loss
246 |         if opt.mode == 'train_flow':
247 |             self.total_loss += flow_warp_loss + flow_smooth_loss + flow_consistency_loss
248 | 
249 |     def SSIM(self, x, y):
250 |         C1 = 0.01 ** 2
251 |         C2 = 0.03 ** 2
252 | 
253 |         mu_x = slim.avg_pool2d(x, 3, 1, 'SAME')
254 |         mu_y = slim.avg_pool2d(y, 3, 1, 'SAME')
255 | 
256 |         sigma_x  = slim.avg_pool2d(x ** 2, 3, 1, 'SAME') - mu_x ** 2
257 |         sigma_y  = slim.avg_pool2d(y ** 2, 3, 1, 'SAME') - mu_y ** 2
258 |         sigma_xy = slim.avg_pool2d(x * y , 3, 1, 'SAME') - mu_x * mu_y
259 | 
260 |         SSIM_n = (2 * mu_x * mu_y + C1) * (2 * sigma_xy + C2)
261 |         SSIM_d = (mu_x ** 2 + mu_y ** 2 + C1) * (sigma_x + sigma_y + C2)
262 | 
263 |         SSIM = SSIM_n / SSIM_d
264 | 
265 |         return tf.clip_by_value((1 - SSIM) / 2, 0, 1)
266 | 
267 |     def image_similarity(self, x, y):
268 |         return self.opt.alpha_recon_image * self.SSIM(x, y) + (1-self.opt.alpha_recon_image) * tf.abs(x-y)
269 | 
270 |     def L2_norm(self, x, axis=3, keep_dims=True):
271 |         curr_offset = 1e-10
272 |         l2_norm = tf.norm(tf.abs(x) + curr_offset, axis=axis, keep_dims=keep_dims)
273 |         return l2_norm
274 | 
275 |     def spatial_normalize(self, disp):
276 |         _, curr_h, curr_w, curr_c = disp.get_shape().as_list()
277 |         disp_mean = tf.reduce_mean(disp, axis=[1,2,3], keep_dims=True)
278 |         disp_mean = tf.tile(disp_mean, [1, curr_h, curr_w, curr_c])
279 |         return disp/disp_mean
280 | 
281 |     def scale_pyramid(self, img, num_scales):
282 |         if img == None:
283 |             return None
284 |         else:
285 |             scaled_imgs = [img]
286 |             _, h, w, _ = img.get_shape().as_list()
287 |             for i in range(num_scales - 1):
288 |                 ratio = 2 ** (i + 1)
289 |                 nh = int(h / ratio)
290 |                 nw = int(w / ratio)
291 |                 scaled_imgs.append(tf.image.resize_area(img, [nh, nw]))
292 |             return scaled_imgs
293 | 
294 |     def gradient_x(self, img):
295 |         gx = img[:,:,:-1,:] - img[:,:,1:,:]
296 |         return gx
297 | 
298 |     def gradient_y(self, img):
299 |         gy = img[:,:-1,:,:] - img[:,1:,:,:]
300 |         return gy
301 | 
302 |     def compute_smooth_loss(self, disp, img):
303 |         disp_gradients_x = self.gradient_x(disp)
304 |         disp_gradients_y = self.gradient_y(disp)
305 | 
306 |         image_gradients_x = self.gradient_x(img)
307 |         image_gradients_y = self.gradient_y(img)
308 | 
309 |         weights_x = tf.exp(-tf.reduce_mean(tf.abs(image_gradients_x), 3, keep_dims=True))
310 |         weights_y = tf.exp(-tf.reduce_mean(tf.abs(image_gradients_y), 3, keep_dims=True))
311 | 
312 |         smoothness_x = disp_gradients_x * weights_x
313 |         smoothness_y = disp_gradients_y * weights_y
314 | 
315 |         return tf.reduce_mean(tf.abs(smoothness_x)) + tf.reduce_mean(tf.abs(smoothness_y))
316 | 
317 |     def compute_flow_smooth_loss(self, flow, img):
318 |         smoothness = 0
319 |         for i in range(2):
320 |             smoothness += self.compute_smooth_loss(tf.expand_dims(flow[:,:,:,i], -1), img)
321 |         return smoothness/2
322 | 
323 |     def preprocess_image(self, image):
324 |         # Assuming input image is uint8
325 |         if image == None:
326 |             return None
327 |         else:
328 |             image = tf.image.convert_image_dtype(image, dtype=tf.float32)
329 |             return image * 2. -1.
330 | 
331 |     def deprocess_image(self, image):
332 |         # Assuming input image is float32
333 |         image = (image + 1.)/2.
334 |         return tf.image.convert_image_dtype(image, dtype=tf.uint8)
335 | 


--------------------------------------------------------------------------------
/geonet_nets.py:
--------------------------------------------------------------------------------
  1 | # The network design is based on Tinghui Zhou & Clement Godard's works:
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/nets.py
  3 | # https://github.com/mrharicot/monodepth/blob/master/monodepth_model.py
  4 | from __future__ import division
  5 | import tensorflow as tf
  6 | import tensorflow.contrib.slim as slim
  7 | import numpy as np
  8 | 
  9 | # Range of disparity/inverse depth values
 10 | DISP_SCALING_RESNET50 = 5
 11 | DISP_SCALING_VGG = 10
 12 | FLOW_SCALING = 0.1
 13 | 
 14 | def disp_net(opt, dispnet_inputs):
 15 |     is_training = opt.mode == 'train_rigid'
 16 |     if opt.dispnet_encoder == 'vgg':
 17 |         return build_vgg(dispnet_inputs, get_disp_vgg, is_training, 'depth_net')
 18 |     else:
 19 |         return build_resnet50(dispnet_inputs, get_disp_resnet50, is_training, 'depth_net')
 20 | 
 21 | def flow_net(opt, flownet_inputs):
 22 |     is_training = opt.mode == 'train_flow'
 23 |     return build_resnet50(flownet_inputs, get_flow, is_training, 'flow_net')
 24 | 
 25 | def pose_net(opt, posenet_inputs):
 26 |     is_training = opt.mode == 'train_rigid'
 27 |     batch_norm_params = {'is_training': is_training}
 28 |     with tf.variable_scope('pose_net') as sc:
 29 |         with slim.arg_scope([slim.conv2d],
 30 |                             normalizer_fn=slim.batch_norm,
 31 |                             normalizer_params=batch_norm_params,
 32 |                             weights_regularizer=slim.l2_regularizer(0.0001),
 33 |                             activation_fn=tf.nn.relu):
 34 |             conv1  = slim.conv2d(posenet_inputs, 16,  7, 2)
 35 |             conv2  = slim.conv2d(conv1, 32,  5, 2)
 36 |             conv3  = slim.conv2d(conv2, 64,  3, 2)
 37 |             conv4  = slim.conv2d(conv3, 128, 3, 2)
 38 |             conv5  = slim.conv2d(conv4, 256, 3, 2)
 39 |             conv6  = slim.conv2d(conv5, 256, 3, 2)
 40 |             conv7  = slim.conv2d(conv6, 256, 3, 2)
 41 |             pose_pred = slim.conv2d(conv7, 6*opt.num_source, 1, 1,
 42 |                                     normalizer_fn=None, activation_fn=None)
 43 |             pose_avg = tf.reduce_mean(pose_pred, [1, 2])
 44 |             pose_final = 0.01 * tf.reshape(pose_avg, [-1, opt.num_source, 6])
 45 |             return pose_final
 46 | 
 47 | def build_vgg(inputs, get_pred, is_training, var_scope):
 48 |     batch_norm_params = {'is_training': is_training}
 49 |     H = inputs.get_shape()[1].value
 50 |     W = inputs.get_shape()[2].value
 51 |     with tf.variable_scope(var_scope) as sc:
 52 |         with slim.arg_scope([slim.conv2d, slim.conv2d_transpose],
 53 |                             normalizer_fn=slim.batch_norm,
 54 |                             normalizer_params=batch_norm_params,
 55 |                             weights_regularizer=slim.l2_regularizer(0.0001),
 56 |                             activation_fn=tf.nn.relu):
 57 |             # ENCODING
 58 |             conv1  = slim.conv2d(inputs, 32,  7, 2)
 59 |             conv1b = slim.conv2d(conv1,  32,  7, 1)
 60 |             conv2  = slim.conv2d(conv1b, 64,  5, 2)
 61 |             conv2b = slim.conv2d(conv2,  64,  5, 1)
 62 |             conv3  = slim.conv2d(conv2b, 128, 3, 2)
 63 |             conv3b = slim.conv2d(conv3,  128, 3, 1)
 64 |             conv4  = slim.conv2d(conv3b, 256, 3, 2)
 65 |             conv4b = slim.conv2d(conv4,  256, 3, 1)
 66 |             conv5  = slim.conv2d(conv4b, 512, 3, 2)
 67 |             conv5b = slim.conv2d(conv5,  512, 3, 1)
 68 |             conv6  = slim.conv2d(conv5b, 512, 3, 2)
 69 |             conv6b = slim.conv2d(conv6,  512, 3, 1)
 70 |             conv7  = slim.conv2d(conv6b, 512, 3, 2)
 71 |             conv7b = slim.conv2d(conv7,  512, 3, 1)
 72 | 
 73 |             # DECODING
 74 |             upconv7 = upconv(conv7b, 512, 3, 2)
 75 |             # There might be dimension mismatch due to uneven down/up-sampling
 76 |             upconv7 = resize_like(upconv7, conv6b)
 77 |             i7_in  = tf.concat([upconv7, conv6b], axis=3)
 78 |             iconv7  = slim.conv2d(i7_in, 512, 3, 1)
 79 | 
 80 |             upconv6 = upconv(iconv7, 512, 3, 2)
 81 |             upconv6 = resize_like(upconv6, conv5b)
 82 |             i6_in  = tf.concat([upconv6, conv5b], axis=3)
 83 |             iconv6  = slim.conv2d(i6_in, 512, 3, 1)
 84 | 
 85 |             upconv5 = upconv(iconv6, 256, 3, 2)
 86 |             upconv5 = resize_like(upconv5, conv4b)
 87 |             i5_in  = tf.concat([upconv5, conv4b], axis=3)
 88 |             iconv5  = slim.conv2d(i5_in, 256, 3, 1)
 89 | 
 90 |             upconv4 = upconv(iconv5, 128, 3, 2)
 91 |             i4_in  = tf.concat([upconv4, conv3b], axis=3)
 92 |             iconv4  = slim.conv2d(i4_in, 128, 3, 1)
 93 |             pred4  = get_pred(iconv4)
 94 |             pred4_up = tf.image.resize_bilinear(pred4, [np.int(H/4), np.int(W/4)])
 95 | 
 96 |             upconv3 = upconv(iconv4, 64,  3, 2)
 97 |             i3_in  = tf.concat([upconv3, conv2b, pred4_up], axis=3)
 98 |             iconv3  = slim.conv2d(i3_in, 64,  3, 1)
 99 |             pred3  = get_pred(iconv3)
100 |             pred3_up = tf.image.resize_bilinear(pred3, [np.int(H/2), np.int(W/2)])
101 | 
102 |             upconv2 = upconv(iconv3, 32,  3, 2)
103 |             i2_in  = tf.concat([upconv2, conv1b, pred3_up], axis=3)
104 |             iconv2  = slim.conv2d(i2_in, 32,  3, 1)
105 |             pred2  = get_pred(iconv2)
106 |             pred2_up = tf.image.resize_bilinear(pred2, [H, W])
107 | 
108 |             upconv1 = upconv(iconv2, 16,  3, 2)
109 |             i1_in  = tf.concat([upconv1, pred2_up], axis=3)
110 |             iconv1  = slim.conv2d(i1_in, 16,  3, 1)
111 |             pred1  = get_pred(iconv1)
112 |             
113 |             return [pred1, pred2, pred3, pred4]
114 | 
115 | def build_resnet50(inputs, get_pred, is_training, var_scope):
116 |     batch_norm_params = {'is_training': is_training}
117 |     with tf.variable_scope(var_scope) as sc:
118 |         with slim.arg_scope([slim.conv2d, slim.conv2d_transpose],
119 |                             normalizer_fn=slim.batch_norm,
120 |                             normalizer_params=batch_norm_params,
121 |                             weights_regularizer=slim.l2_regularizer(0.0001),
122 |                             activation_fn=tf.nn.relu):
123 |             conv1 = conv(inputs, 64, 7, 2) # H/2  -   64D
124 |             pool1 = maxpool(conv1,           3) # H/4  -   64D
125 |             conv2 = resblock(pool1,      64, 3) # H/8  -  256D
126 |             conv3 = resblock(conv2,     128, 4) # H/16 -  512D
127 |             conv4 = resblock(conv3,     256, 6) # H/32 - 1024D
128 |             conv5 = resblock(conv4,     512, 3) # H/64 - 2048D
129 | 
130 |             skip1 = conv1
131 |             skip2 = pool1
132 |             skip3 = conv2
133 |             skip4 = conv3
134 |             skip5 = conv4
135 |             
136 |             # DECODING
137 |             upconv6 = upconv(conv5,   512, 3, 2) #H/32
138 |             upconv6 = resize_like(upconv6, skip5)
139 |             concat6 = tf.concat([upconv6, skip5], 3)
140 |             iconv6  = conv(concat6,   512, 3, 1)
141 | 
142 |             upconv5 = upconv(iconv6, 256, 3, 2) #H/16
143 |             upconv5 = resize_like(upconv5, skip4)
144 |             concat5 = tf.concat([upconv5, skip4], 3)
145 |             iconv5  = conv(concat5,   256, 3, 1)
146 | 
147 |             upconv4 = upconv(iconv5,  128, 3, 2) #H/8
148 |             upconv4 = resize_like(upconv4, skip3)
149 |             concat4 = tf.concat([upconv4, skip3], 3)
150 |             iconv4  = conv(concat4,   128, 3, 1)
151 |             pred4 = get_pred(iconv4)
152 |             upred4  = upsample_nn(pred4, 2)
153 | 
154 |             upconv3 = upconv(iconv4,   64, 3, 2) #H/4
155 |             concat3 = tf.concat([upconv3, skip2, upred4], 3)
156 |             iconv3  = conv(concat3,    64, 3, 1)
157 |             pred3 = get_pred(iconv3)
158 |             upred3  = upsample_nn(pred3, 2)
159 | 
160 |             upconv2 = upconv(iconv3,   32, 3, 2) #H/2
161 |             concat2 = tf.concat([upconv2, skip1, upred3], 3)
162 |             iconv2  = conv(concat2,    32, 3, 1)
163 |             pred2 = get_pred(iconv2)
164 |             upred2  = upsample_nn(pred2, 2)
165 | 
166 |             upconv1 = upconv(iconv2,  16, 3, 2) #H
167 |             concat1 = tf.concat([upconv1, upred2], 3)
168 |             iconv1  = conv(concat1,   16, 3, 1)
169 |             pred1 = get_pred(iconv1)
170 | 
171 |             return [pred1, pred2, pred3, pred4]
172 | 
173 | def conv(x, num_out_layers, kernel_size, stride, activation_fn=tf.nn.relu, normalizer_fn=slim.batch_norm):
174 |     p = np.floor((kernel_size - 1) / 2).astype(np.int32)
175 |     p_x = tf.pad(x, [[0, 0], [p, p], [p, p], [0, 0]])
176 |     return slim.conv2d(p_x, num_out_layers, kernel_size, stride, 'VALID', activation_fn=activation_fn, normalizer_fn=normalizer_fn)
177 | 
178 | def maxpool(x, kernel_size):
179 |     p = np.floor((kernel_size - 1) / 2).astype(np.int32)
180 |     p_x = tf.pad(x, [[0, 0], [p, p], [p, p], [0, 0]])
181 |     return slim.max_pool2d(p_x, kernel_size)
182 | 
183 | def get_disp_vgg(x):
184 |     disp = DISP_SCALING_VGG * slim.conv2d(x, 1, 3, 1, activation_fn=tf.nn.sigmoid, normalizer_fn=None) + 0.01
185 |     return disp
186 | 
187 | def get_disp_resnet50(x):
188 |     disp = DISP_SCALING_RESNET50 * conv(x, 1, 3, 1, activation_fn=tf.nn.sigmoid, normalizer_fn=None) + 0.01
189 |     return disp
190 | 
191 | def get_flow(x):
192 |     # Output flow value is normalized by image height/width
193 |     flow = FLOW_SCALING * slim.conv2d(x, 2, 3, 1, activation_fn=None, normalizer_fn=None)
194 |     return flow
195 | 
196 | def resize_like(inputs, ref):
197 |     iH, iW = inputs.get_shape()[1], inputs.get_shape()[2]
198 |     rH, rW = ref.get_shape()[1], ref.get_shape()[2]
199 |     if iH == rH and iW == rW:
200 |         return inputs
201 |     return tf.image.resize_nearest_neighbor(inputs, [rH.value, rW.value])
202 | 
203 | def upsample_nn(x, ratio):
204 |     h = x.get_shape()[1].value
205 |     w = x.get_shape()[2].value
206 |     return tf.image.resize_nearest_neighbor(x, [h * ratio, w * ratio])
207 | 
208 | def upconv(x, num_out_layers, kernel_size, scale):
209 |     upsample = upsample_nn(x, scale)
210 |     cnv = conv(upsample, num_out_layers, kernel_size, 1)
211 |     return cnv
212 | 
213 | def resconv(x, num_layers, stride):
214 |     # Actually here exists a bug: tf.shape(x)[3] != num_layers is always true,
215 |     # but we preserve it here for consistency with Godard's implementation.
216 |     do_proj = tf.shape(x)[3] != num_layers or stride == 2
217 |     shortcut = []
218 |     conv1 = conv(x,         num_layers, 1, 1)
219 |     conv2 = conv(conv1,     num_layers, 3, stride)
220 |     conv3 = conv(conv2, 4 * num_layers, 1, 1, None)
221 |     if do_proj:
222 |         shortcut = conv(x, 4 * num_layers, 1, stride, None)
223 |     else:
224 |         shortcut = x
225 |     return tf.nn.relu(conv3 + shortcut)
226 | 
227 | def resblock(x, num_layers, num_blocks):
228 |     out = x
229 |     for i in range(num_blocks - 1):
230 |         out = resconv(out, num_layers, 1)
231 |     out = resconv(out, num_layers, 2)
232 |     return out
233 | 
234 | # def resconv(x, num_layers, stride):
235 | #     shortcut = []
236 | #     conv1 = conv(x,         num_layers, 1, 1)
237 | #     conv2 = conv(conv1,     num_layers, 3, stride)
238 | #     conv3 = conv(conv2, 4 * num_layers, 1, 1, None)
239 | #     if stride == 2:
240 | #         shortcut = conv(x, 4 * num_layers, 1, stride, None)
241 | #     else:
242 | #         shortcut = x
243 | #     return tf.nn.relu(conv3 + shortcut)
244 | 
245 | # def resblock(x, num_layers, num_blocks):
246 | #     out = x
247 | #     out = resconv(out, num_layers, 2)
248 | #     for i in range(num_blocks - 1):
249 | #         out = resconv(out, num_layers, 1)
250 | #     return out
251 | 


--------------------------------------------------------------------------------
/geonet_test_depth.py:
--------------------------------------------------------------------------------
 1 | from __future__ import division
 2 | import tensorflow as tf
 3 | import numpy as np
 4 | import os
 5 | import PIL.Image as pil
 6 | from geonet_model import *
 7 | 
 8 | def test_depth(opt):
 9 |     ##### load testing list #####
10 |     with open('data/kitti/test_files_%s.txt' % opt.depth_test_split, 'r') as f:
11 |         test_files = f.readlines()
12 |         test_files = [opt.dataset_dir + t[:-1] for t in test_files]
13 |     if not os.path.exists(opt.output_dir):
14 |         os.makedirs(opt.output_dir)
15 | 
16 |     ##### init #####
17 |     input_uint8 = tf.placeholder(tf.uint8, [opt.batch_size,
18 |                 opt.img_height, opt.img_width, 3], name='raw_input')
19 | 
20 |     model = GeoNetModel(opt, input_uint8, None, None)
21 |     fetches = { "depth": model.pred_depth[0] }
22 | 
23 |     saver = tf.train.Saver([var for var in tf.model_variables()])
24 |     config = tf.ConfigProto()
25 |     config.gpu_options.allow_growth = True
26 | 
27 |     ##### Go #####
28 |     with tf.Session(config=config) as sess:
29 |         saver.restore(sess, opt.init_ckpt_file)
30 |         pred_all = []
31 |         for t in range(0, len(test_files), opt.batch_size):
32 |             if t % 100 == 0:
33 |                 print('processing: %d/%d' % (t, len(test_files)))
34 |             inputs = np.zeros(
35 |                 (opt.batch_size, opt.img_height, opt.img_width, 3),
36 |                 dtype=np.uint8)
37 | 
38 |             for b in range(opt.batch_size):
39 |                 idx = t + b
40 |                 if idx >= len(test_files):
41 |                     break
42 |                 fh = open(test_files[idx], 'r')
43 |                 raw_im = pil.open(fh)
44 |                 scaled_im = raw_im.resize((opt.img_width, opt.img_height), pil.ANTIALIAS)
45 |                 inputs[b] = np.array(scaled_im)
46 | 
47 |             pred = sess.run(fetches, feed_dict={input_uint8: inputs})
48 |             for b in range(opt.batch_size):
49 |                 idx = t + b
50 |                 if idx >= len(test_files):
51 |                     break
52 |                 pred_all.append(pred['depth'][b,:,:,0])
53 | 
54 |         np.save(opt.output_dir + '/' + os.path.basename(opt.init_ckpt_file), pred_all)
55 | 


--------------------------------------------------------------------------------
/geonet_test_flow.py:
--------------------------------------------------------------------------------
  1 | from __future__ import division
  2 | import tensorflow as tf
  3 | import numpy as np
  4 | import os
  5 | import PIL.Image as pil
  6 | import cv2
  7 | from geonet_model import *
  8 | from data_loader import DataLoader
  9 | import sys
 10 | sys.path.insert(0, './kitti_eval/flow_tool/')
 11 | import flowlib as fl
 12 | 
 13 | def test_flow(opt):
 14 | 
 15 |     ##### load testing list #####
 16 |     with open(opt.dataset_dir + "test_flow.txt", 'r') as f:
 17 |         test_files = f.readlines()
 18 |         input_list = []
 19 |         for seq in test_files:
 20 |             seq = seq.split(' ')
 21 |             input_list.append(opt.dataset_dir+seq[0]+'/'+seq[1][:-1])
 22 | 
 23 |     if not os.path.exists(opt.output_dir):
 24 |         os.makedirs(opt.output_dir)
 25 | 
 26 |     ##### init #####
 27 |     # TODO: currently assuming batch_size = 1
 28 |     assert opt.batch_size == 1
 29 |     
 30 |     tgt_image_uint8 = tf.placeholder(tf.uint8, [opt.batch_size,
 31 |                         opt.img_height, opt.img_width, 3],
 32 |                         name='tgt_input')
 33 |     src_image_stack_uint8 = tf.placeholder(tf.uint8, [opt.batch_size,
 34 |                         opt.img_height, opt.img_width, opt.num_source * 3],
 35 |                         name='src_stack_input')
 36 |     intrinsics = tf.placeholder(tf.float32, [opt.batch_size, 3, 3],
 37 |                         name='intrinsics_input')
 38 |     loader = DataLoader(opt)
 39 |     intrinsics_ms = loader.get_multi_scale_intrinsics(intrinsics, opt.num_scales)
 40 | 
 41 |     # currently assume a sequence is fed and the tgt->src_id flow is computed
 42 |     src_id = int(opt.num_source // 2)
 43 |     bs = opt.batch_size
 44 |     model = GeoNetModel(opt, tgt_image_uint8, src_image_stack_uint8, intrinsics_ms)
 45 |     fetches = {}
 46 |     fetches["pred_flow"] = model.fwd_full_flow_pyramid[0][bs*src_id:bs*(src_id+1)]
 47 | 
 48 |     saver = tf.train.Saver([var for var in tf.model_variables()])
 49 |     config = tf.ConfigProto()
 50 |     config.gpu_options.allow_growth = True
 51 | 
 52 |     ##### Go! #####
 53 |     output_file = opt.output_dir + '/' + os.path.basename(opt.init_ckpt_file)
 54 |     if not os.path.exists(output_file):
 55 |         os.makedirs(output_file)
 56 |     binary_dir = os.path.join(output_file, 'binary')
 57 |     color_dir  = os.path.join(output_file, 'color')
 58 |     png_dir    = os.path.join(output_file, 'png')
 59 |     if (not os.path.exists(binary_dir)):
 60 |         os.makedirs(binary_dir)
 61 |     if (not os.path.exists(color_dir)):
 62 |         os.makedirs(color_dir)
 63 |     if (not os.path.exists(png_dir)):
 64 |         os.makedirs(png_dir)
 65 | 
 66 |     with tf.Session(config=config) as sess:
 67 |         saver.restore(sess, opt.init_ckpt_file)
 68 |         pred_all = []
 69 |         img_num = len(input_list)
 70 | 
 71 |         for tgt_idx in range(img_num):
 72 |             if (tgt_idx+1) % 100 == 0:
 73 |                 print('processing: %d/%d' % (tgt_idx+1, img_num))            
 74 |             image_seq = cv2.imread(input_list[tgt_idx]+'.jpg')
 75 |             tgt_image, src_image_stack = unpack_image_sequence(image_seq, 
 76 |                                          opt.img_height, opt.img_width, opt.num_source)
 77 |             with open(input_list[tgt_idx]+'_cam.txt', 'r') as cf:
 78 |                 cam_file = cf.readlines()
 79 |             cam_file = cam_file[0].split(',')
 80 |             cam_file = np.array([float(d) for d in cam_file])
 81 |             cam_file = np.reshape(cam_file, (3,3))
 82 | 
 83 |             pred = sess.run(fetches, feed_dict={tgt_image_uint8: tgt_image[None, :, :, :], 
 84 |                                                 src_image_stack_uint8: src_image_stack[None, :, :, :],
 85 |                                                 intrinsics: cam_file[None,:,:]})
 86 |             pred_flow=pred['pred_flow'][0]
 87 | 
 88 |             # save flow
 89 |             flow_fn = '%.6d.png' % tgt_idx
 90 |             color_fn    = os.path.join(color_dir, flow_fn)
 91 |             color_flow  = fl.flow_to_image(pred_flow)
 92 |             color_flow  = cv2.cvtColor(color_flow, cv2.COLOR_RGB2BGR)
 93 |             color_flow  = cv2.imwrite(color_fn, color_flow)
 94 | 
 95 |             png_fn      = os.path.join(png_dir, flow_fn)
 96 |             mask_blob   = np.ones((opt.img_height, opt.img_width), dtype = np.uint16)
 97 |             fl.write_kitti_png_file(png_fn, pred_flow, mask_blob)
 98 | 
 99 |             binary_fn   = flow_fn.replace('.png', '.flo')
100 |             binary_fn   = os.path.join(binary_dir, binary_fn)
101 |             fl.write_flow(pred_flow, binary_fn)
102 | 
103 | def unpack_image_sequence(image_seq, img_height, img_width, num_source):
104 |     # Assuming the center image is the target frame
105 |     half_seq_width = int(img_width * (num_source//2))
106 |     tgt_image = image_seq[:, half_seq_width:half_seq_width+img_width, :]
107 |     # Source frames before the target frame
108 |     src_image_1 = image_seq[:, :half_seq_width, :]
109 |     # Source frames after the target frame
110 |     src_image_2 = image_seq[:, half_seq_width+img_width:, :]
111 |     src_image_seq = np.hstack((src_image_1, src_image_2))
112 |     # Stack source frames along the color channels (i.e. [H, W, N*3]
113 |     src_image_stack = np.dstack([src_image_seq[:,i*img_width:(i+1)*img_width,:] \
114 |                       for i in range(num_source)])
115 |     return tgt_image, src_image_stack


--------------------------------------------------------------------------------
/geonet_test_pose.py:
--------------------------------------------------------------------------------
 1 | from __future__ import division
 2 | import os
 3 | import math
 4 | import scipy.misc
 5 | import tensorflow as tf
 6 | import numpy as np
 7 | from glob import glob
 8 | from geonet_model import *
 9 | from kitti_eval.pose_evaluation_utils import dump_pose_seq_TUM
10 | 
11 | def test_pose(opt):
12 | 
13 |     if not os.path.isdir(opt.output_dir):
14 |         os.makedirs(opt.output_dir)
15 | 
16 |     ##### init #####
17 |     input_uint8 = tf.placeholder(tf.uint8, [opt.batch_size, 
18 |         opt.img_height, opt.img_width, opt.seq_length * 3], 
19 |         name='raw_input')
20 |     tgt_image = input_uint8[:,:,:,:3]
21 |     src_image_stack = input_uint8[:,:,:,3:]
22 | 
23 |     model = GeoNetModel(opt, tgt_image, src_image_stack, None)
24 |     fetches = { "pose": model.pred_poses }
25 | 
26 |     saver = tf.train.Saver([var for var in tf.model_variables()]) 
27 | 
28 |     ##### load test frames #####
29 |     seq_dir = os.path.join(opt.dataset_dir, 'sequences', '%.2d' % opt.pose_test_seq)
30 |     img_dir = os.path.join(seq_dir, 'image_2')
31 |     N = len(glob(img_dir + '/*.png'))
32 |     test_frames = ['%.2d %.6d' % (opt.pose_test_seq, n) for n in range(N)]
33 | 
34 |     ##### load time file #####
35 |     with open(opt.dataset_dir + 'sequences/%.2d/times.txt' % opt.pose_test_seq, 'r') as f:
36 |         times = f.readlines()
37 |     times = np.array([float(s[:-1]) for s in times])
38 | 
39 |     ##### Go! #####
40 |     max_src_offset = (opt.seq_length - 1) // 2
41 |     with tf.Session() as sess:
42 |         saver.restore(sess, opt.init_ckpt_file)
43 |         for tgt_idx in range(max_src_offset, N-max_src_offset, opt.batch_size):            
44 |             if (tgt_idx-max_src_offset) % 100 == 0:
45 |                 print('Progress: %d/%d' % (tgt_idx-max_src_offset, N))
46 | 
47 |             inputs = np.zeros((opt.batch_size, opt.img_height,
48 |                      opt.img_width, 3*opt.seq_length), dtype=np.uint8)
49 | 
50 |             for b in range(opt.batch_size):
51 |                 idx = tgt_idx + b
52 |                 if idx >= N-max_src_offset:
53 |                     break
54 |                 image_seq = load_image_sequence(opt.dataset_dir,
55 |                                                 test_frames,
56 |                                                 idx,
57 |                                                 opt.seq_length,
58 |                                                 opt.img_height,
59 |                                                 opt.img_width)
60 |                 inputs[b] = image_seq
61 | 
62 |             pred = sess.run(fetches, feed_dict={input_uint8: inputs})
63 |             pred_poses = pred['pose']
64 |             # Insert the target pose [0, 0, 0, 0, 0, 0] 
65 |             pred_poses = np.insert(pred_poses, max_src_offset, np.zeros((1,6)), axis=1)
66 | 
67 |             for b in range(opt.batch_size):
68 |                 idx = tgt_idx + b
69 |                 if idx >=N-max_src_offset:
70 |                     break
71 |                 pred_pose = pred_poses[b]                
72 |                 curr_times = times[idx - max_src_offset:idx + max_src_offset + 1]
73 |                 out_file = opt.output_dir + '%.6d.txt' % (idx - max_src_offset)
74 |                 dump_pose_seq_TUM(out_file, pred_pose, curr_times)
75 | 
76 | def load_image_sequence(dataset_dir, 
77 |                         frames, 
78 |                         tgt_idx, 
79 |                         seq_length, 
80 |                         img_height, 
81 |                         img_width):
82 |     half_offset = int((seq_length - 1)/2)
83 |     for o in range(-half_offset, half_offset+1):
84 |         curr_idx = tgt_idx + o
85 |         curr_drive, curr_frame_id = frames[curr_idx].split(' ')
86 |         img_file = os.path.join(
87 |             dataset_dir, 'sequences', '%s/image_2/%s.png' % (curr_drive, curr_frame_id))
88 |         curr_img = scipy.misc.imread(img_file)
89 |         curr_img = scipy.misc.imresize(curr_img, (img_height, img_width))
90 |         if o == -half_offset:
91 |             image_seq = curr_img
92 |         elif o == 0:
93 |             image_seq = np.dstack((curr_img, image_seq))
94 |         else:
95 |             image_seq = np.dstack((image_seq, curr_img))
96 |     return image_seq
97 | 


--------------------------------------------------------------------------------
/kitti_eval/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/yzcjtr/GeoNet/5b176025bc63563ef53297aa3d20cc6e575fb833/kitti_eval/__init__.py


--------------------------------------------------------------------------------
/kitti_eval/depth_evaluation_utils.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Clement Godard: 
  2 | # https://github.com/mrharicot/monodepth/blob/master/utils/evaluation_utils.py
  3 | import numpy as np
  4 | import os
  5 | import cv2
  6 | from collections import Counter
  7 | import pickle
  8 | 
  9 | def compute_errors(gt, pred):
 10 |     thresh = np.maximum((gt / pred), (pred / gt))
 11 |     a1 = (thresh < 1.25   ).mean()
 12 |     a2 = (thresh < 1.25 ** 2).mean()
 13 |     a3 = (thresh < 1.25 ** 3).mean()
 14 | 
 15 |     rmse = (gt - pred) ** 2
 16 |     rmse = np.sqrt(rmse.mean())
 17 | 
 18 |     rmse_log = (np.log(gt) - np.log(pred)) ** 2
 19 |     rmse_log = np.sqrt(rmse_log.mean())
 20 | 
 21 |     abs_rel = np.mean(np.abs(gt - pred) / gt)
 22 | 
 23 |     sq_rel = np.mean(((gt - pred)**2) / gt)
 24 | 
 25 |     return abs_rel, sq_rel, rmse, rmse_log, a1, a2, a3
 26 | 
 27 | ###############################################################################
 28 | #######################  KITTI
 29 | 
 30 | width_to_focal = dict()
 31 | width_to_focal[1242] = 721.5377
 32 | width_to_focal[1241] = 718.856
 33 | width_to_focal[1224] = 707.0493
 34 | width_to_focal[1238] = 718.3351
 35 | 
 36 | def load_gt_disp_kitti(path):
 37 |     gt_disparities = []
 38 |     for i in range(200):
 39 |         disp = cv2.imread(path + "/training/disp_noc_0/" + str(i).zfill(6) + "_10.png", -1)
 40 |         disp = disp.astype(np.float32) / 256
 41 |         gt_disparities.append(disp)
 42 |     return gt_disparities
 43 | 
 44 | def convert_disps_to_depths_kitti(gt_disparities, pred_disparities):
 45 |     gt_depths = []
 46 |     pred_depths = []
 47 |     pred_disparities_resized = []
 48 |     
 49 |     for i in range(len(gt_disparities)):
 50 |         gt_disp = gt_disparities[i]
 51 |         height, width = gt_disp.shape
 52 | 
 53 |         pred_disp = pred_disparities[i]
 54 |         pred_disp = width * cv2.resize(pred_disp, (width, height), interpolation=cv2.INTER_LINEAR)
 55 | 
 56 |         pred_disparities_resized.append(pred_disp) 
 57 | 
 58 |         mask = gt_disp > 0
 59 | 
 60 |         gt_depth = width_to_focal[width] * 0.54 / (gt_disp + (1.0 - mask))
 61 |         pred_depth = width_to_focal[width] * 0.54 / pred_disp
 62 | 
 63 |         gt_depths.append(gt_depth)
 64 |         pred_depths.append(pred_depth)
 65 |     return gt_depths, pred_depths, pred_disparities_resized
 66 | 
 67 | 
 68 | ###############################################################################
 69 | #######################  EIGEN
 70 | 
 71 | def read_text_lines(file_path):
 72 |     f = open(file_path, 'r')
 73 |     lines = f.readlines()
 74 |     f.close()
 75 |     lines = [l.rstrip() for l in lines]
 76 |     return lines
 77 | 
 78 | def read_file_data(files, data_root):
 79 |     gt_files = []
 80 |     gt_calib = []
 81 |     im_sizes = []
 82 |     im_files = []
 83 |     cams = []
 84 |     num_probs = 0
 85 |     for filename in files:
 86 |         filename = filename.split()[0]
 87 |         splits = filename.split('/')
 88 | #         camera_id = filename[-1]   # 2 is left, 3 is right
 89 |         date = splits[0]
 90 |         im_id = splits[4][:10]
 91 |         file_root = '{}/{}'
 92 |         
 93 |         im = filename
 94 |         vel = '{}/{}/velodyne_points/data/{}.bin'.format(splits[0], splits[1], im_id)
 95 | 
 96 |         if os.path.isfile(data_root + im):
 97 |             gt_files.append(data_root + vel)
 98 |             gt_calib.append(data_root + date + '/')
 99 |             im_sizes.append(cv2.imread(data_root + im).shape[:2])
100 |             im_files.append(data_root + im)
101 |             cams.append(2)
102 |         else:
103 |             num_probs += 1
104 |             print('{} missing'.format(data_root + im))
105 |     # print(num_probs, 'files missing')
106 | 
107 |     return gt_files, gt_calib, im_sizes, im_files, cams
108 | 
109 | def load_velodyne_points(file_name):
110 |     # adapted from https://github.com/hunse/kitti
111 |     points = np.fromfile(file_name, dtype=np.float32).reshape(-1, 4)
112 |     points[:, 3] = 1.0  # homogeneous
113 |     return points
114 | 
115 | 
116 | def lin_interp(shape, xyd):
117 |     # taken from https://github.com/hunse/kitti
118 |     m, n = shape
119 |     ij, d = xyd[:, 1::-1], xyd[:, 2]
120 |     f = LinearNDInterpolator(ij, d, fill_value=0)
121 |     J, I = np.meshgrid(np.arange(n), np.arange(m))
122 |     IJ = np.vstack([I.flatten(), J.flatten()]).T
123 |     disparity = f(IJ).reshape(shape)
124 |     return disparity
125 | 
126 | 
127 | def read_calib_file(path):
128 |     # taken from https://github.com/hunse/kitti
129 |     float_chars = set("0123456789.e+- ")
130 |     data = {}
131 |     with open(path, 'r') as f:
132 |         for line in f.readlines():
133 |             key, value = line.split(':', 1)
134 |             value = value.strip()
135 |             data[key] = value
136 |             if float_chars.issuperset(value):
137 |                 # try to cast to float array
138 |                 try:
139 |                     data[key] = np.array(map(float, value.split(' ')))
140 |                 except ValueError:
141 |                     # casting error: data[key] already eq. value, so pass
142 |                     pass
143 | 
144 |     return data
145 | 
146 | 
147 | def get_focal_length_baseline(calib_dir, cam=2):
148 |     cam2cam = read_calib_file(calib_dir + 'calib_cam_to_cam.txt')
149 |     P2_rect = cam2cam['P_rect_02'].reshape(3,4)
150 |     P3_rect = cam2cam['P_rect_03'].reshape(3,4)
151 | 
152 |     # cam 2 is left of camera 0  -6cm
153 |     # cam 3 is to the right  +54cm
154 |     b2 = P2_rect[0,3] / -P2_rect[0,0]
155 |     b3 = P3_rect[0,3] / -P3_rect[0,0]
156 |     baseline = b3-b2
157 | 
158 |     if cam==2:
159 |         focal_length = P2_rect[0,0]
160 |     elif cam==3:
161 |         focal_length = P3_rect[0,0]
162 | 
163 |     return focal_length, baseline
164 | 
165 | 
166 | def sub2ind(matrixSize, rowSub, colSub):
167 |     m, n = matrixSize
168 |     return rowSub * (n-1) + colSub - 1
169 | 
170 | def generate_depth_map(calib_dir, velo_file_name, im_shape, cam=2, interp=False, vel_depth=False):
171 |     # load calibration files
172 |     cam2cam = read_calib_file(calib_dir + 'calib_cam_to_cam.txt')
173 |     velo2cam = read_calib_file(calib_dir + 'calib_velo_to_cam.txt')
174 |     velo2cam = np.hstack((velo2cam['R'].reshape(3,3), velo2cam['T'][..., np.newaxis]))
175 |     velo2cam = np.vstack((velo2cam, np.array([0, 0, 0, 1.0])))
176 | 
177 |     # compute projection matrix velodyne->image plane
178 |     R_cam2rect = np.eye(4)
179 |     R_cam2rect[:3,:3] = cam2cam['R_rect_00'].reshape(3,3)
180 |     P_rect = cam2cam['P_rect_0'+str(cam)].reshape(3,4)
181 |     P_velo2im = np.dot(np.dot(P_rect, R_cam2rect), velo2cam)
182 | 
183 |     # load velodyne points and remove all behind image plane (approximation)
184 |     # each row of the velodyne data is forward, left, up, reflectance
185 |     velo = load_velodyne_points(velo_file_name)
186 |     velo = velo[velo[:, 0] >= 0, :]
187 | 
188 |     # project the points to the camera
189 |     velo_pts_im = np.dot(P_velo2im, velo.T).T
190 |     velo_pts_im[:, :2] = velo_pts_im[:,:2] / velo_pts_im[:,2][..., np.newaxis]
191 | 
192 |     if vel_depth:
193 |         velo_pts_im[:, 2] = velo[:, 0]
194 | 
195 |     # check if in bounds
196 |     # use minus 1 to get the exact same value as KITTI matlab code
197 |     velo_pts_im[:, 0] = np.round(velo_pts_im[:,0]) - 1
198 |     velo_pts_im[:, 1] = np.round(velo_pts_im[:,1]) - 1
199 |     val_inds = (velo_pts_im[:, 0] >= 0) & (velo_pts_im[:, 1] >= 0)
200 |     val_inds = val_inds & (velo_pts_im[:,0] < im_shape[1]) & (velo_pts_im[:,1] < im_shape[0])
201 |     velo_pts_im = velo_pts_im[val_inds, :]
202 | 
203 |     # project to image
204 |     depth = np.zeros((im_shape))
205 |     depth[velo_pts_im[:, 1].astype(np.int), velo_pts_im[:, 0].astype(np.int)] = velo_pts_im[:, 2]
206 | 
207 |     # find the duplicate points and choose the closest depth
208 |     inds = sub2ind(depth.shape, velo_pts_im[:, 1], velo_pts_im[:, 0])
209 |     dupe_inds = [item for item, count in Counter(inds).iteritems() if count > 1]
210 |     for dd in dupe_inds:
211 |         pts = np.where(inds==dd)[0]
212 |         x_loc = int(velo_pts_im[pts[0], 0])
213 |         y_loc = int(velo_pts_im[pts[0], 1])
214 |         depth[y_loc, x_loc] = velo_pts_im[pts, 2].min()
215 |     depth[depth<0] = 0
216 | 
217 |     if interp:
218 |         # interpolate the depth map to fill in holes
219 |         depth_interp = lin_interp(im_shape, velo_pts_im)
220 |         return depth, depth_interp
221 |     else:
222 |         return depth
223 | 
224 | 
225 | 
226 | 


--------------------------------------------------------------------------------
/kitti_eval/eval_depth.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Clement Godard: 
  2 | # https://github.com/mrharicot/monodepth/blob/master/utils/evaluate_kitti.py
  3 | from __future__ import division
  4 | import sys
  5 | import cv2
  6 | import os
  7 | import numpy as np
  8 | import argparse
  9 | from depth_evaluation_utils import *
 10 | 
 11 | parser = argparse.ArgumentParser()
 12 | parser.add_argument("--split", type=str, default='eigen', help='eigen or stereo split')
 13 | parser.add_argument("--kitti_dir", type=str, help='Path to the KITTI dataset directory')
 14 | parser.add_argument("--pred_file", type=str, help="Path to the prediction file")
 15 | parser.add_argument('--min_depth', type=float, default=1e-3, help="Threshold for minimum depth")
 16 | parser.add_argument('--max_depth', type=float, default=80, help="Threshold for maximum depth")
 17 | args = parser.parse_args()
 18 | 
 19 | def convert_disps_to_depths_stereo(gt_disparities, pred_depths):
 20 |     gt_depths = []
 21 |     pred_depths_resized = []
 22 |     pred_disparities_resized = []
 23 |     
 24 |     for i in range(len(gt_disparities)):
 25 |         gt_disp = gt_disparities[i]
 26 |         height, width = gt_disp.shape
 27 | 
 28 |         pred_depth = pred_depths[i]
 29 |         pred_depth = cv2.resize(pred_depth, (width, height), interpolation=cv2.INTER_LINEAR)
 30 | 
 31 |         pred_disparities_resized.append(1./pred_depth) 
 32 | 
 33 |         mask = gt_disp > 0
 34 | 
 35 |         gt_depth = width_to_focal[width] * 0.54 / (gt_disp + (1.0 - mask))
 36 |         #pred_depth = width_to_focal[width] * 0.54 / pred_disp
 37 | 
 38 |         gt_depths.append(gt_depth)
 39 |         pred_depths_resized.append(pred_depth)
 40 |     return gt_depths, pred_depths_resized, pred_disparities_resized
 41 | 
 42 | def main():
 43 | 
 44 |     pred_depths = np.load(args.pred_file)
 45 |     args.test_file_list = './data/kitti/test_files_%s.txt' % args.split
 46 | 
 47 |     print 'evaluating ' + args.pred_file + '...'
 48 | 
 49 |     if args.split == 'eigen':
 50 |         test_files = read_text_lines(args.test_file_list)
 51 |         gt_files, gt_calib, im_sizes, im_files, cams = \
 52 |             read_file_data(test_files, args.kitti_dir)
 53 |         num_test = len(im_files)
 54 |         gt_depths = []
 55 |         pred_depths_resized = []
 56 |         for t_id in range(num_test):
 57 |             camera_id = cams[t_id]  # 2 is left, 3 is right
 58 |             pred_depths_resized.append(
 59 |                 cv2.resize(pred_depths[t_id], 
 60 |                            (im_sizes[t_id][1], im_sizes[t_id][0]), 
 61 |                            interpolation=cv2.INTER_LINEAR))
 62 |             depth = generate_depth_map(gt_calib[t_id], 
 63 |                                        gt_files[t_id], 
 64 |                                        im_sizes[t_id], 
 65 |                                        camera_id, 
 66 |                                        False, 
 67 |                                        True)
 68 |             gt_depths.append(depth.astype(np.float32))
 69 |         pred_depths = pred_depths_resized
 70 |     else:
 71 |         num_test = 200
 72 |         gt_disparities = load_gt_disp_kitti(args.kitti_dir)
 73 |         gt_depths, pred_depths, pred_disparities_resized = convert_disps_to_depths_stereo(gt_disparities, pred_depths)
 74 | 
 75 |     rms     = np.zeros(num_test, np.float32)
 76 |     log_rms = np.zeros(num_test, np.float32)
 77 |     abs_rel = np.zeros(num_test, np.float32)
 78 |     sq_rel  = np.zeros(num_test, np.float32)
 79 |     d1_all  = np.zeros(num_test, np.float32)
 80 |     a1      = np.zeros(num_test, np.float32)
 81 |     a2      = np.zeros(num_test, np.float32)
 82 |     a3      = np.zeros(num_test, np.float32)
 83 |     for i in range(num_test):    
 84 |         gt_depth = gt_depths[i]
 85 |         pred_depth = np.copy(pred_depths[i])
 86 | 
 87 |         if args.split == 'eigen':
 88 | 
 89 |             mask = np.logical_and(gt_depth > args.min_depth, 
 90 |                                   gt_depth < args.max_depth)
 91 |             # crop used by Garg ECCV16 to reprocude Eigen NIPS14 results
 92 |             # if used on gt_size 370x1224 produces a crop of [-218, -3, 44, 1180]
 93 |             gt_height, gt_width = gt_depth.shape
 94 |             crop = np.array([0.40810811 * gt_height,  0.99189189 * gt_height,   
 95 |                              0.03594771 * gt_width,   0.96405229 * gt_width]).astype(np.int32)
 96 | 
 97 |             crop_mask = np.zeros(mask.shape)
 98 |             crop_mask[crop[0]:crop[1],crop[2]:crop[3]] = 1
 99 |             mask = np.logical_and(mask, crop_mask)
100 | 
101 |         if args.split == 'stereo':
102 |             gt_disp = gt_disparities[i]
103 |             mask = gt_disp > 0
104 |             pred_disp = pred_disparities_resized[i]
105 | 
106 |             disp_diff = np.abs(gt_disp[mask] - pred_disp[mask])
107 |             bad_pixels = np.logical_and(disp_diff >= 3, (disp_diff / gt_disp[mask]) >= 0.05)
108 |             d1_all[i] = 100.0 * bad_pixels.sum() / mask.sum()
109 | 
110 |         # Scale matching
111 |         scalor = np.median(gt_depth[mask])/np.median(pred_depth[mask])
112 |         pred_depth[mask] *= scalor
113 | 
114 |         pred_depth[pred_depth < args.min_depth] = args.min_depth
115 |         pred_depth[pred_depth > args.max_depth] = args.max_depth
116 |         abs_rel[i], sq_rel[i], rms[i], log_rms[i], a1[i], a2[i], a3[i] = \
117 |             compute_errors(gt_depth[mask], pred_depth[mask])
118 | 
119 |     print("{:>10}, {:>10}, {:>10}, {:>10}, {:>10}, {:>10}, {:>10}, {:>10}".format('abs_rel', 'sq_rel', 'rms', 'log_rms', 'd1_all', 'a1', 'a2', 'a3'))
120 |     print("{:10.4f}, {:10.4f}, {:10.4f}, {:10.4f}, {:10.4f}, {:10.4f}, {:10.4f}, {:10.4f}".format(abs_rel.mean(), sq_rel.mean(), rms.mean(), log_rms.mean(), d1_all.mean(), a1.mean(), a2.mean(), a3.mean()))    
121 | 
122 | main()
123 | 


--------------------------------------------------------------------------------
/kitti_eval/eval_flow.py:
--------------------------------------------------------------------------------
 1 | from __future__ import division
 2 | import cv2
 3 | import os
 4 | import numpy as np
 5 | import argparse
 6 | import sys
 7 | root_path = os.path.dirname(os.path.abspath(__file__))
 8 | sys.path.insert(0, root_path + '/flow_tool/')
 9 | import flowlib as fl
10 | 
11 | parser = argparse.ArgumentParser()
12 | parser.add_argument("--dataset_dir", type=str, help="Path to kitti stereo dataset")
13 | parser.add_argument("--pred_dir",    type=str, help="Path to the flow prediction")
14 | args = parser.parse_args()
15 | 
16 | def main():
17 |     img_num = 200
18 |     noc_epe = np.zeros(img_num, dtype=np.float)
19 |     noc_acc = np.zeros(img_num, dtype=np.float)
20 |     occ_epe = np.zeros(img_num, dtype=np.float)
21 |     occ_acc = np.zeros(img_num, dtype=np.float)
22 | 
23 |     eval_log = os.path.join(args.pred_dir, 'flow_result.txt')
24 |     with open(eval_log, 'w') as el:
25 |         for idx in range(img_num):
26 |             # read groundtruth flow
27 |             gt_noc_fn = args.dataset_dir + 'training/flow_noc/%.6d_10.png' % idx
28 |             gt_occ_fn = args.dataset_dir + 'training/flow_occ/%.6d_10.png' % idx
29 |             gt_noc_flow = fl.read_flow(gt_noc_fn)
30 |             gt_occ_flow = fl.read_flow(gt_occ_fn)
31 | 
32 |             # read predicted flow (in png format)
33 |             pred_flow_fn = args.pred_dir + 'png/%.6d.png' % idx
34 |             pred_flow = fl.read_flow(pred_flow_fn)
35 | 
36 |             # resize pred_flow to the same size as gt_flow
37 |             dst_h = gt_noc_flow.shape[0]
38 |             dst_w = gt_noc_flow.shape[1]
39 |             pred_flow = fl.resize_flow(pred_flow, dst_w, dst_h)
40 | 
41 |             # evaluation
42 |             (single_noc_epe, single_noc_acc) = fl.evaluate_kitti_flow(gt_noc_flow, pred_flow, None)
43 |             (single_occ_epe, single_occ_acc) = fl.evaluate_kitti_flow(gt_occ_flow, pred_flow, None)
44 |             noc_epe[idx] = single_noc_epe
45 |             noc_acc[idx] = single_noc_acc
46 |             occ_epe[idx] = single_occ_epe
47 |             occ_acc[idx] = single_occ_acc
48 |             output_line = 'Flow %.6d Noc EPE = %.4f' + ' Noc ACC = %.4f' + ' Occ EPE = %.4f' + ' Occ ACC = %.4f\n';
49 |             el.write(output_line % (idx, noc_epe[idx], noc_acc[idx], occ_epe[idx], occ_acc[idx]))
50 | 
51 |     noc_mean_epe = np.mean(noc_epe)
52 |     noc_mean_acc = np.mean(noc_acc)
53 |     occ_mean_epe = np.mean(occ_epe)
54 |     occ_mean_acc = np.mean(occ_acc)
55 | 
56 |     print ('Mean Noc EPE = %.4f ' % noc_mean_epe)
57 |     print ('Mean Noc ACC = %.4f ' % noc_mean_acc)
58 |     print ('Mean Occ EPE = %.4f ' % occ_mean_epe)
59 |     print ('Mean Occ ACC = %.4f ' % occ_mean_acc)
60 | 
61 | main()


--------------------------------------------------------------------------------
/kitti_eval/eval_pose.py:
--------------------------------------------------------------------------------
 1 | # Mostly based on the code written by Tinghui Zhou: 
 2 | # https://github.com/tinghuiz/SfMLearner/blob/master/kitti_eval/eval_pose.py
 3 | from __future__ import division
 4 | import os
 5 | import numpy as np
 6 | import argparse
 7 | from glob import glob
 8 | from pose_evaluation_utils import *
 9 | 
10 | parser = argparse.ArgumentParser()
11 | parser.add_argument("--gtruth_dir", type=str, 
12 |     help='Path to the directory with ground-truth trajectories')
13 | parser.add_argument("--pred_dir", type=str, 
14 |     help="Path to the directory with predicted trajectories")
15 | args = parser.parse_args()
16 | 
17 | def main():
18 |     pred_files = glob(args.pred_dir + '/*.txt')
19 |     ate_all = []
20 |     for i in range(len(pred_files)):
21 |         gtruth_file = args.gtruth_dir + os.path.basename(pred_files[i])
22 |         if not os.path.exists(gtruth_file):
23 |             continue
24 |         ate = compute_ate(gtruth_file, pred_files[i])
25 |         if ate == False:
26 |             continue
27 |         ate_all.append(ate)
28 |     ate_all = np.array(ate_all)
29 |     print("Predictions dir: %s" % args.pred_dir)
30 |     print("ATE mean: %.4f, std: %.4f" % (np.mean(ate_all), np.std(ate_all)))
31 | main()


--------------------------------------------------------------------------------
/kitti_eval/flow_tool/flowlib.py:
--------------------------------------------------------------------------------
  1 | 
  2 | 
  3 | #!/usr/bin/python
  4 | """
  5 | # ==============================
  6 | # flowlib.py
  7 | # library for optical flow processing
  8 | # Author: Ruoteng Li
  9 | # Date: 6th Aug 2016
 10 | # ==============================
 11 | """
 12 | import png
 13 | import pfm
 14 | import numpy as np
 15 | import matplotlib.colors as cl
 16 | # import matplotlib.pyplot as plt
 17 | from PIL import Image
 18 | from scipy import misc
 19 | import cv2
 20 | 
 21 | UNKNOWN_FLOW_THRESH = 1e7
 22 | SMALLFLOW = 0.0
 23 | LARGEFLOW = 1e8
 24 | 
 25 | """
 26 | =============
 27 | Flow Section
 28 | =============
 29 | """
 30 | 
 31 | 
 32 | #def show_flow(filename):
 33 | #    """
 34 | #    visualize optical flow map using matplotlib
 35 | #    :param filename: optical flow file
 36 | #    :return: None
 37 | #    """
 38 | #    flow = read_flow(filename)
 39 | #    img = flow_to_image(flow)
 40 | #    plt.imshow(img)
 41 | #    plt.show()
 42 | 
 43 | 
 44 | # def visualize_flow(flow, mode='Y'):
 45 | #    """
 46 | #    this function visualize the input flow
 47 | #    :param flow: input flow in array
 48 | #    :param mode: choose which color mode to visualize the flow (Y: Ccbcr, RGB: RGB color)
 49 | #    :return: None
 50 | #    """
 51 | #    if mode == 'Y':
 52 | #        # Ccbcr color wheel
 53 | #        img = flow_to_image(flow)
 54 | #        plt.imshow(img)
 55 | #        plt.show()
 56 | #    elif mode == 'RGB':
 57 | #        (h, w) = flow.shape[0:2]
 58 | #        du = flow[:, :, 0]
 59 | #        dv = flow[:, :, 1]
 60 | #        valid = flow[:, :, 2]
 61 | #        max_flow = max(np.max(du), np.max(dv))
 62 | #        img = np.zeros((h, w, 3), dtype=np.float64)
 63 | #        # angle layer
 64 | #        img[:, :, 0] = np.arctan2(dv, du) / (2 * np.pi)
 65 | #        # magnitude layer, normalized to 1
 66 | #        img[:, :, 1] = np.sqrt(du * du + dv * dv) * 8 / max_flow
 67 | #        # phase layer
 68 | #        img[:, :, 2] = 8 - img[:, :, 1]
 69 | #        # clip to [0,1]
 70 | #        small_idx = img[:, :, 0:3] < 0
 71 | #        large_idx = img[:, :, 0:3] > 1
 72 | #        img[small_idx] = 0
 73 | #        img[large_idx] = 1
 74 | #        # convert to rgb
 75 | #        img = cl.hsv_to_rgb(img)
 76 | #        # remove invalid point
 77 | #        img[:, :, 0] = img[:, :, 0] * valid
 78 | #        img[:, :, 1] = img[:, :, 1] * valid
 79 | #        img[:, :, 2] = img[:, :, 2] * valid
 80 | #        # show
 81 | #        plt.imshow(img)
 82 | #        plt.show()
 83 | #
 84 | #    return None
 85 | 
 86 | 
 87 | def read_flow(filename):
 88 |     """
 89 |     read optical flow data from flow file
 90 |     :param filename: name of the flow file
 91 |     :return: optical flow data in numpy array
 92 |     """
 93 |     if filename.endswith('.flo'):
 94 |         flow = read_flo_file(filename)
 95 |     elif filename.endswith('.png'):
 96 |         # flow = read_png_file(filename)
 97 |         flow = read_kitti_png_file(filename)
 98 |     elif filename.endswith('.pfm'):
 99 |         flow = read_pfm_file(filename)
100 |     else:
101 |         raise Exception('Invalid flow file format!')
102 | 
103 |     return flow
104 | 
105 | 
106 | def write_flow(flow, filename):
107 |     """
108 |     write optical flow in Middlebury .flo format
109 |     :param flow: optical flow map
110 |     :param filename: optical flow file path to be saved
111 |     :return: None
112 |     """
113 |     f = open(filename, 'wb')
114 |     magic = np.array([202021.25], dtype=np.float32)
115 |     (height, width) = flow.shape[0:2]
116 |     w = np.array([width], dtype=np.int32)
117 |     h = np.array([height], dtype=np.int32)
118 |     magic.tofile(f)
119 |     w.tofile(f)
120 |     h.tofile(f)
121 |     flow.tofile(f)
122 |     f.close()
123 | 
124 | 
125 | def save_flow_image(flow, image_file):
126 |     """
127 |     save flow visualization into image file
128 |     :param flow: optical flow data
129 |     :param flow_fil
130 |     :return: None
131 |     """
132 |     # print flow.shape
133 |     flow_img = flow_to_image(flow)
134 |     img_out = Image.fromarray(flow_img)
135 |     img_out.save(image_file)
136 | 
137 | 
138 | def flowfile_to_imagefile(flow_file, image_file):
139 |     """
140 |     convert flowfile into image file
141 |     :param flow: optical flow data
142 |     :param flow_fil
143 |     :return: None
144 |     """
145 |     flow = read_flow(flow_file)
146 |     save_flow_image(flow, image_file)
147 | 
148 | 
149 | def segment_flow(flow):
150 |     h = flow.shape[0]
151 |     w = flow.shape[1]
152 |     u = flow[:, :, 0]
153 |     v = flow[:, :, 1]
154 | 
155 |     idx = ((abs(u) > LARGEFLOW) | (abs(v) > LARGEFLOW))
156 |     idx2 = (abs(u) == SMALLFLOW)
157 |     class0 = (v == 0) & (u == 0)
158 |     u[idx2] = 0.00001
159 |     tan_value = v / u
160 | 
161 |     class1 = (tan_value < 1) & (tan_value >= 0) & (u > 0) & (v >= 0)
162 |     class2 = (tan_value >= 1) & (u >= 0) & (v >= 0)
163 |     class3 = (tan_value < -1) & (u <= 0) & (v >= 0)
164 |     class4 = (tan_value < 0) & (tan_value >= -1) & (u < 0) & (v >= 0)
165 |     class8 = (tan_value >= -1) & (tan_value < 0) & (u > 0) & (v <= 0)
166 |     class7 = (tan_value < -1) & (u >= 0) & (v <= 0)
167 |     class6 = (tan_value >= 1) & (u <= 0) & (v <= 0)
168 |     class5 = (tan_value >= 0) & (tan_value < 1) & (u < 0) & (v <= 0)
169 | 
170 |     seg = np.zeros((h, w))
171 | 
172 |     seg[class1] = 1
173 |     seg[class2] = 2
174 |     seg[class3] = 3
175 |     seg[class4] = 4
176 |     seg[class5] = 5
177 |     seg[class6] = 6
178 |     seg[class7] = 7
179 |     seg[class8] = 8
180 |     seg[class0] = 0
181 |     seg[idx] = 0
182 | 
183 |     return seg
184 | 
185 | 
186 | def flow_error(tu, tv, u, v):
187 |     """
188 |     Calculate average end point error
189 |     :param tu: ground-truth horizontal flow map
190 |     :param tv: ground-truth vertical flow map
191 |     :param u:  estimated horizontal flow map
192 |     :param v:  estimated vertical flow map
193 |     :return: End point error of the estimated flow
194 |     """
195 |     smallflow = 0.0
196 |     '''
197 |     stu = tu[bord+1:end-bord,bord+1:end-bord]
198 |     stv = tv[bord+1:end-bord,bord+1:end-bord]
199 |     su = u[bord+1:end-bord,bord+1:end-bord]
200 |     sv = v[bord+1:end-bord,bord+1:end-bord]
201 |     '''
202 |     stu = tu[:]
203 |     stv = tv[:]
204 |     su = u[:]
205 |     sv = v[:]
206 | 
207 |     idxUnknow = (abs(stu) > UNKNOWN_FLOW_THRESH) | (abs(stv) > UNKNOWN_FLOW_THRESH)
208 |     stu[idxUnknow] = 0
209 |     stv[idxUnknow] = 0
210 |     su[idxUnknow] = 0
211 |     sv[idxUnknow] = 0
212 | 
213 |     ind2 = [(np.absolute(stu) > smallflow) | (np.absolute(stv) > smallflow)]
214 |     index_su = su[ind2]
215 |     index_sv = sv[ind2]
216 |     an = 1.0 / np.sqrt(index_su ** 2 + index_sv ** 2 + 1)
217 |     un = index_su * an
218 |     vn = index_sv * an
219 | 
220 |     index_stu = stu[ind2]
221 |     index_stv = stv[ind2]
222 |     tn = 1.0 / np.sqrt(index_stu ** 2 + index_stv ** 2 + 1)
223 |     tun = index_stu * tn
224 |     tvn = index_stv * tn
225 | 
226 |     '''
227 |     angle = un * tun + vn * tvn + (an * tn)
228 |     index = [angle == 1.0]
229 |     angle[index] = 0.999
230 |     ang = np.arccos(angle)
231 |     mang = np.mean(ang)
232 |     mang = mang * 180 / np.pi
233 |     '''
234 | 
235 |     epe = np.sqrt((stu - su) ** 2 + (stv - sv) ** 2)
236 |     epe = epe[ind2]
237 |     mepe = np.mean(epe)
238 |     return mepe
239 | 
240 | def flow_kitti_error(tu, tv, u, v, mask, ru = None, rv = None):
241 |     """
242 |     Calculate average end point error
243 |     :param tu: ground-truth horizontal flow map
244 |     :param tv: ground-truth vertical flow map
245 |     :param u:  estimated horizontal flow map
246 |     :param v:  estimated vertical flow map
247 |     :param mask: ground-truth mask
248 |     :return: End point error of the estimated flow
249 |     """
250 |     tau = [3, 0.05]
251 |     '''
252 |     stu = tu[bord+1:end-bord,bord+1:end-bord]
253 |     stv = tv[bord+1:end-bord,bord+1:end-bord]
254 |     su = u[bord+1:end-bord,bord+1:end-bord]
255 |     sv = v[bord+1:end-bord,bord+1:end-bord]
256 |     '''
257 |     stu = tu[:]
258 |     stv = tv[:]
259 |     su = u[:]
260 |     sv = v[:]
261 |     smask = mask[:]
262 | 
263 |     ind_valid = (smask != 0)
264 |     n_total = np.sum(ind_valid)
265 |     # print stu.size
266 |     # print n_total
267 | 
268 |     epe = np.sqrt((stu - su) ** 2 + (stv - sv) ** 2)
269 |     mag = np.sqrt(stu ** 2 + stv ** 2) + 1e-5
270 | 
271 |     epe = epe[ind_valid]
272 |     mag = mag[ind_valid]
273 | 
274 |     if ru != None and rv != None:
275 |         sru = ru[:]
276 |         srv = rv[:]
277 |         rig_mag = np.sqrt((stu - sru) ** 2 + (stv - srv) ** 2) + 1e-5
278 |         rig_mag = rig_mag[ind_valid]
279 | 
280 |     phased_epe = []
281 |     '''
282 |     phased_error = range(0, 256, 5)
283 |     for i in range(len(phased_error)-1):
284 |         filter_mask = rig_mag < phased_error[i+1] #np.logical_and((mag > phased_error[i]), (mag < phased_error[i+1]))
285 |         tmp_epe = np.mean(epe[filter_mask])
286 |         phased_epe.append(tmp_epe)
287 |     '''
288 | 
289 |     err = np.logical_and((epe > tau[0]), (epe / mag) > tau[1])
290 |     n_err = np.sum(err)
291 | 
292 |     # print n_err
293 |     # print n_total
294 |     mean_epe = np.mean(epe)
295 |     mean_acc = 1 - (float(n_err) / float(n_total))
296 |     # print mean_epe
297 |     # print mean_acc
298 |     if ru != None and rv != None:
299 |         return (mean_epe, mean_acc, phased_epe)
300 |     else:
301 |         return (mean_epe, mean_acc)
302 | 
303 | 
304 | def flow_to_image(flow):
305 |     """
306 |     Convert flow into middlebury color code image
307 |     :param flow: optical flow map
308 |     :return: optical flow image in middlebury color
309 |     """
310 |     u = flow[:, :, 0]
311 |     v = flow[:, :, 1]
312 | 
313 |     maxu = -999.
314 |     maxv = -999.
315 |     minu = 999.
316 |     minv = 999.
317 | 
318 |     idxUnknow = (abs(u) > UNKNOWN_FLOW_THRESH) | (abs(v) > UNKNOWN_FLOW_THRESH)
319 |     u[idxUnknow] = 0
320 |     v[idxUnknow] = 0
321 | 
322 |     maxu = max(maxu, np.max(u))
323 |     minu = min(minu, np.min(u))
324 | 
325 |     maxv = max(maxv, np.max(v))
326 |     minv = min(minv, np.min(v))
327 | 
328 |     rad = np.sqrt(u ** 2 + v ** 2)
329 |     maxrad = max(-1, np.max(rad))
330 | 
331 |     # print "max flow: %.4f\nflow range:\nu = %.3f .. %.3f\nv = %.3f .. %.3f" % (maxrad, minu,maxu, minv, maxv)
332 | 
333 |     u = u/(maxrad + np.finfo(float).eps)
334 |     v = v/(maxrad + np.finfo(float).eps)
335 | 
336 |     img = compute_color(u, v)
337 | 
338 |     idx = np.repeat(idxUnknow[:, :, np.newaxis], 3, axis=2)
339 |     img[idx] = 0
340 | 
341 |     return np.uint8(img)
342 | 
343 | 
344 | def evaluate_flow_file(gt_file, pred_file):
345 |     """
346 |     evaluate the estimated optical flow end point error according to ground truth provided
347 |     :param gt_file: ground truth file path
348 |     :param pred_file: estimated optical flow file path
349 |     :return: end point error, float32
350 |     """
351 |     # Read flow files and calculate the errors
352 |     gt_flow = read_flow(gt_file)        # ground truth flow
353 |     eva_flow = read_flow(pred_file)     # predicted flow
354 |     # Calculate errors
355 |     average_pe = flow_error(gt_flow[:, :, 0], gt_flow[:, :, 1], eva_flow[:, :, 0], eva_flow[:, :, 1])
356 |     return average_pe
357 | 
358 | 
359 | def evaluate_flow(gt_flow, pred_flow):
360 |     """
361 |     gt: ground-truth flow
362 |     pred: estimated flow
363 |     """
364 |     average_pe = flow_error(gt_flow[:, :, 0], gt_flow[:, :, 1], pred_flow[:, :, 0], pred_flow[:, :, 1])
365 |     return average_pe
366 | 
367 | def evaluate_kitti_flow(gt_flow, pred_flow, rigid_flow = None):
368 |     # print gt_flow.shape
369 |     if gt_flow.shape[2] == 2:
370 |         gt_mask = np.ones((gt_flow.shape[0], gt_flow.shape[1]))
371 |         epe, acc = flow_kitti_error(gt_flow[:, :, 0], gt_flow[:, :, 1], pred_flow[:, :, 0], pred_flow[:, :, 1], gt_mask)
372 |         #epe, acc, phase = flow_kitti_error(gt_flow[:, :, 0], gt_flow[:, :, 1], pred_flow[:, :, 0], pred_flow[:, :, 1], gt_mask, rigid_flow[:,:,0], rigid_flow[:,:,1])
373 |     elif gt_flow.shape[2] == 3:
374 |         epe, acc = flow_kitti_error(gt_flow[:, :, 0], gt_flow[:, :, 1], pred_flow[:, :, 0], pred_flow[:, :, 1], gt_flow[:, :, 2])
375 |         #epe, acc, phase = flow_kitti_error(gt_flow[:, :, 0], gt_flow[:, :, 1], pred_flow[:, :, 0], pred_flow[:, :, 1], gt_flow[:, :, 2], rigid_flow[:,:,0], rigid_flow[:,:,1])
376 |     #return (epe, acc, phase)
377 |     return (epe, acc)
378 | 
379 | """
380 | ==============
381 | Disparity Section
382 | ==============
383 | """
384 | 
385 | 
386 | def read_disp_png(file_name):
387 |     """
388 |     Read optical flow from KITTI .png file
389 |     :param file_name: name of the flow file
390 |     :return: optical flow data in matrix
391 |     """
392 |     image_object = png.Reader(filename=file_name)
393 |     image_direct = image_object.asDirect()
394 |     image_data = list(image_direct[2])
395 |     (w, h) = image_direct[3]['size']
396 |     channel = len(image_data[0]) / w
397 |     flow = np.zeros((h, w, channel), dtype=np.uint16)
398 |     for i in range(len(image_data)):
399 |         for j in range(channel):
400 |             flow[i, :, j] = image_data[i][j::channel]
401 |     return flow[:, :, 0] / 256
402 | 
403 | 
404 | def disp_to_flowfile(disp, filename):
405 |     """
406 |     Read KITTI disparity file in png format
407 |     :param disp: disparity matrix
408 |     :param filename: the flow file name to save
409 |     :return: None
410 |     """
411 |     f = open(filename, 'wb')
412 |     magic = np.array([202021.25], dtype=np.float32)
413 |     (height, width) = disp.shape[0:2]
414 |     w = np.array([width], dtype=np.int32)
415 |     h = np.array([height], dtype=np.int32)
416 |     empty_map = np.zeros((height, width), dtype=np.float32)
417 |     data = np.dstack((disp, empty_map))
418 |     magic.tofile(f)
419 |     w.tofile(f)
420 |     h.tofile(f)
421 |     data.tofile(f)
422 |     f.close()
423 | 
424 | 
425 | """
426 | ==============
427 | Image Section
428 | ==============
429 | """
430 | 
431 | 
432 | def read_image(filename):
433 |     """
434 |     Read normal image of any format
435 |     :param filename: name of the image file
436 |     :return: image data in matrix uint8 type
437 |     """
438 |     img = Image.open(filename)
439 |     im = np.array(img)
440 |     return im
441 | 
442 | 
443 | def warp_image(im, flow):
444 |     """
445 |     Use optical flow to warp image to the next
446 |     :param im: image to warp
447 |     :param flow: optical flow
448 |     :return: warped image
449 |     """
450 |     from scipy import interpolate
451 |     image_height = im.shape[0]
452 |     image_width = im.shape[1]
453 |     flow_height = flow.shape[0]
454 |     flow_width = flow.shape[1]
455 |     n = image_height * image_width
456 |     (iy, ix) = np.mgrid[0:image_height, 0:image_width]
457 |     (fy, fx) = np.mgrid[0:flow_height, 0:flow_width]
458 |     # fx = fx.astype('float32')
459 |     # fy = fy.astype('float32')
460 |     flow = np.rint(flow).astype('int32')
461 |     fx += flow[:,:,0]
462 |     fy += flow[:,:,1]
463 |     mask = np.logical_or(fx <0 , fx > flow_width)
464 |     mask = np.logical_or(mask, fy < 0)
465 |     mask = np.logical_or(mask, fy > flow_height)
466 |     fx = np.minimum(np.maximum(fx, 0), flow_width)
467 |     fy = np.minimum(np.maximum(fy, 0), flow_height)
468 |     points = np.concatenate((ix.reshape(n,1), iy.reshape(n,1)), axis=1)
469 |     xi = np.concatenate((fx.reshape(n, 1), fy.reshape(n,1)), axis=1)
470 |     warp = np.zeros((image_height, image_width, im.shape[2]))
471 |     for i in range(im.shape[2]):
472 |         channel = im[:, :, i]
473 |         # plt.imshow(channel, cmap='gray')
474 |         values = channel.reshape(n, 1)
475 |         new_channel = interpolate.griddata(points, values, xi, method='cubic')
476 |         new_channel = np.reshape(new_channel, [flow_height, flow_width])
477 |         new_channel[mask] = 1
478 |         warp[:, :, i] = new_channel.astype(np.uint8)
479 | 
480 |     return warp.astype(np.uint8)
481 | 
482 | 
483 | """
484 | ==============
485 | Others
486 | ==============
487 | """
488 | 
489 | def pfm_to_flo(pfm_file):
490 |     flow_filename = pfm_file[0:pfm_file.find('.pfm')] + '.flo'
491 |     (data, scale) = pfm.readPFM(pfm_file)
492 |     flow = data[:, :, 0:2]
493 |     write_flow(flow, flow_filename)
494 | 
495 | 
496 | def scale_image(image, new_range):
497 |     """
498 |     Linearly scale the image into desired range
499 |     :param image: input image
500 |     :param new_range: the new range to be aligned
501 |     :return: image normalized in new range
502 |     """
503 |     min_val = np.min(image).astype(np.float32)
504 |     max_val = np.max(image).astype(np.float32)
505 |     min_val_new = np.array(min(new_range), dtype=np.float32)
506 |     max_val_new = np.array(max(new_range), dtype=np.float32)
507 |     scaled_image = (image - min_val) / (max_val - min_val) * (max_val_new - min_val_new) + min_val_new
508 |     return scaled_image.astype(np.uint8)
509 | 
510 | 
511 | def compute_color(u, v):
512 |     """
513 |     compute optical flow color map
514 |     :param u: optical flow horizontal map
515 |     :param v: optical flow vertical map
516 |     :return: optical flow in color code
517 |     """
518 |     [h, w] = u.shape
519 |     img = np.zeros([h, w, 3])
520 |     nanIdx = np.isnan(u) | np.isnan(v)
521 |     u[nanIdx] = 0
522 |     v[nanIdx] = 0
523 | 
524 |     colorwheel = make_color_wheel()
525 |     ncols = np.size(colorwheel, 0)
526 | 
527 |     rad = np.sqrt(u**2+v**2)
528 | 
529 |     a = np.arctan2(-v, -u) / np.pi
530 | 
531 |     fk = (a+1) / 2 * (ncols - 1) + 1
532 | 
533 |     k0 = np.floor(fk).astype(int)
534 | 
535 |     k1 = k0 + 1
536 |     k1[k1 == ncols+1] = 1
537 |     f = fk - k0
538 | 
539 |     for i in range(0, np.size(colorwheel,1)):
540 |         tmp = colorwheel[:, i]
541 |         col0 = tmp[k0-1] / 255
542 |         col1 = tmp[k1-1] / 255
543 |         col = (1-f) * col0 + f * col1
544 | 
545 |         idx = rad <= 1
546 |         col[idx] = 1-rad[idx]*(1-col[idx])
547 |         notidx = np.logical_not(idx)
548 | 
549 |         col[notidx] *= 0.75
550 |         img[:, :, i] = np.uint8(np.floor(255 * col*(1-nanIdx)))
551 | 
552 |     return img
553 | 
554 | 
555 | def make_color_wheel():
556 |     """
557 |     Generate color wheel according Middlebury color code
558 |     :return: Color wheel
559 |     """
560 |     RY = 15
561 |     YG = 6
562 |     GC = 4
563 |     CB = 11
564 |     BM = 13
565 |     MR = 6
566 | 
567 |     ncols = RY + YG + GC + CB + BM + MR
568 | 
569 |     colorwheel = np.zeros([ncols, 3])
570 | 
571 |     col = 0
572 | 
573 |     # RY
574 |     colorwheel[0:RY, 0] = 255
575 |     colorwheel[0:RY, 1] = np.transpose(np.floor(255*np.arange(0, RY) / RY))
576 |     col += RY
577 | 
578 |     # YG
579 |     colorwheel[col:col+YG, 0] = 255 - np.transpose(np.floor(255*np.arange(0, YG) / YG))
580 |     colorwheel[col:col+YG, 1] = 255
581 |     col += YG
582 | 
583 |     # GC
584 |     colorwheel[col:col+GC, 1] = 255
585 |     colorwheel[col:col+GC, 2] = np.transpose(np.floor(255*np.arange(0, GC) / GC))
586 |     col += GC
587 | 
588 |     # CB
589 |     colorwheel[col:col+CB, 1] = 255 - np.transpose(np.floor(255*np.arange(0, CB) / CB))
590 |     colorwheel[col:col+CB, 2] = 255
591 |     col += CB
592 | 
593 |     # BM
594 |     colorwheel[col:col+BM, 2] = 255
595 |     colorwheel[col:col+BM, 0] = np.transpose(np.floor(255*np.arange(0, BM) / BM))
596 |     col += + BM
597 | 
598 |     # MR
599 |     colorwheel[col:col+MR, 2] = 255 - np.transpose(np.floor(255 * np.arange(0, MR) / MR))
600 |     colorwheel[col:col+MR, 0] = 255
601 | 
602 |     return colorwheel
603 | 
604 | 
605 | def read_flo_file(filename):
606 |     """
607 |     Read from Middlebury .flo file
608 |     :param flow_file: name of the flow file
609 |     :return: optical flow data in matrix
610 |     """
611 |     f = open(filename, 'rb')
612 |     magic = np.fromfile(f, np.float32, count=1)
613 |     data2d = None
614 | 
615 |     if 202021.25 != magic:
616 |         print 'Magic number incorrect. Invalid .flo file'
617 |     else:
618 |         w = np.fromfile(f, np.int32, count=1)
619 |         h = np.fromfile(f, np.int32, count=1)
620 |         # print "Reading %d x %d flow file in .flo format" % (h, w)
621 |         data2d = np.fromfile(f, np.float32, count=2 * w * h)
622 |         # reshape data into 3D array (columns, rows, channels)
623 |         data2d = np.resize(data2d, (h[0], w[0], 2))
624 |     f.close()
625 |     return data2d
626 | 
627 | 
628 | def read_png_file(flow_file):
629 |     """
630 |     Read from KITTI .png file
631 |     :param flow_file: name of the flow file
632 |     :return: optical flow data in matrix
633 |     """
634 |     flow_object = png.Reader(filename=flow_file)
635 |     flow_direct = flow_object.asDirect()
636 |     flow_data = list(flow_direct[2])
637 |     (w, h) = flow_direct[3]['size']
638 |     print "Reading %d x %d flow file in .png format" % (h, w)
639 |     flow = np.zeros((h, w, 3), dtype=np.float64)
640 |     for i in range(len(flow_data)):
641 |         flow[i, :, 0] = flow_data[i][0::3]
642 |         flow[i, :, 1] = flow_data[i][1::3]
643 |         flow[i, :, 2] = flow_data[i][2::3]
644 | 
645 |     invalid_idx = (flow[:, :, 2] == 0)
646 |     flow[:, :, 0:2] = (flow[:, :, 0:2] - 2 ** 15) / 64.0
647 |     flow[invalid_idx, 0] = 0
648 |     flow[invalid_idx, 1] = 0
649 |     return flow
650 | 
651 | def read_kitti_png_file(flow_file):
652 |     # print flow_file
653 |     flow_img = cv2.imread(flow_file, cv2.CV_LOAD_IMAGE_UNCHANGED)
654 |     flow_img = flow_img.astype(float)
655 |     # print flow_img.shape
656 |     flow_data = np.zeros(flow_img.shape, dtype = np.float)
657 |     flow_data[:, :, 0] = (flow_img[:, :, 2] - 2 ** 15) / 64.0
658 |     flow_data[:, :, 1] = (flow_img[:, :, 1] - 2 ** 15) / 64.0
659 |     flow_data[:, :, 2] = flow_img[:, :, 0]
660 |     return flow_data
661 | 
662 | def read_pfm_file(flow_file):
663 |     """
664 |     Read from .pfm file
665 |     :param flow_file: name of the flow file
666 |     :return: optical flow data in matrix
667 |     """
668 |     import pfm
669 |     (data, scale) = pfm.readPFM(flow_file)
670 |     return data
671 | 
672 | def resize_flow(flow, des_width, des_height):
673 |     src_height  = flow.shape[0]
674 |     src_width   = flow.shape[1]
675 |     ratio_height    = float(des_height) / float(src_height)
676 |     ratio_width     = float(des_width) / float(src_width)
677 |     # print ratio_width
678 |     # print ratio_height
679 |     flow = cv2.resize(flow, (des_width, des_height), interpolation=cv2.INTER_NEAREST)
680 |     flow[:, :, 0] = flow[:, :, 0] * ratio_width
681 |     flow[:, : ,1] = flow[:, :, 1] * ratio_height
682 |     return flow
683 | 
684 | def remove_ambiguity_flow(flow_img, err_img, threshold_err = 10.0):
685 |     thre_flow   = flow_img
686 |     mask_img    = np.ones(err_img.shape, dtype = np.uint8)
687 |     mask_img[err_img > threshold_err] = 0.0
688 |     thre_flow[err_img > threshold_err] = 0.0
689 |     return (thre_flow, mask_img)
690 | 
691 | def write_kitti_png_file(flow_fn, flow_data, mask_data):
692 |     flow_img = np.zeros((flow_data.shape[0], flow_data.shape[1], 3), dtype = np.uint16)
693 |     flow_img[:, :, 2] = flow_data[:, :, 0] * 64.0 + 2 ** 15
694 |     flow_img[:, :, 1] = flow_data[:, :, 1] * 64.0 + 2 ** 15
695 |     flow_img[:, :, 0] = mask_data[:, :]
696 |     cv2.imwrite(flow_fn, flow_img)
697 | 
698 | def flow_kitti_mask_error(tu, tv, gt_mask, u, v, pd_mask):
699 |     """
700 |     Calculate average end point error
701 |     :param tu: ground-truth horizontal flow map
702 |     :param tv: ground-truth vertical flow map
703 |     :param gt_mask: ground-truth mask
704 | 
705 |     :param u:  estimated horizontal flow map
706 |     :param v:  estimated vertical flow map
707 |     :param pd_mask: estimated flow mask
708 |     :return: End point error of the estimated flow
709 |     """
710 |     tau = [3, 0.05]
711 |     '''
712 |     stu = tu[bord+1:end-bord,bord+1:end-bord]
713 |     stv = tv[bord+1:end-bord,bord+1:end-bord]
714 |     su = u[bord+1:end-bord,bord+1:end-bord]
715 |     sv = v[bord+1:end-bord,bord+1:end-bord]
716 |     '''
717 |     stu = tu[:]
718 |     stv = tv[:]
719 |     su = u[:]
720 |     sv = v[:]
721 |     s_gt_mask = gt_mask[:]
722 |     s_pd_mask = pd_mask[:]
723 | 
724 |     ind_valid = np.logical_and(s_gt_mask != 0, s_pd_mask != 0)
725 |     n_total = np.sum(ind_valid)
726 |     # print stu.size
727 |     # print n_total
728 | 
729 |     epe = np.sqrt((stu - su) ** 2 + (stv - sv) ** 2)
730 |     mag = np.sqrt(stu ** 2 + stv ** 2) + 1e-5
731 | 
732 |     epe = epe[ind_valid]
733 |     mag = mag[ind_valid]
734 | 
735 |     err = np.logical_and((epe > tau[0]), (epe / mag) > tau[1])
736 |     n_err = np.sum(err)
737 | 
738 |     # print n_err
739 |     # print n_total
740 |     mean_epe = np.mean(epe)
741 |     mean_acc = 1 - (float(n_err) / float(n_total))
742 |     # print mean_epe
743 |     # print mean_acc
744 |     return (mean_epe, mean_acc)
745 | 
746 | 


--------------------------------------------------------------------------------
/kitti_eval/flow_tool/pfm.py:
--------------------------------------------------------------------------------
 1 | import re
 2 | import numpy as np
 3 | import sys
 4 | 
 5 | 
 6 | def readPFM(file):
 7 |     file = open(file, 'rb')
 8 | 
 9 |     color = None
10 |     width = None
11 |     height = None
12 |     scale = None
13 |     endian = None
14 | 
15 |     header = file.readline().rstrip()
16 |     if header == 'PF':
17 |         color = True
18 |     elif header == 'Pf':
19 |         color = False
20 |     else:
21 |         raise Exception('Not a PFM file.')
22 | 
23 |     dim_match = re.match(r'^(\d+)\s(\d+)\s$', file.readline())
24 |     if dim_match:
25 |         width, height = map(int, dim_match.groups())
26 |     else:
27 |         raise Exception('Malformed PFM header.')
28 | 
29 |     scale = float(file.readline().rstrip())
30 |     if scale < 0:  # little-endian
31 |         endian = '<'
32 |         scale = -scale
33 |     else:
34 |         endian = '>'  # big-endian
35 | 
36 |     data = np.fromfile(file, endian + 'f')
37 |     shape = (height, width, 3) if color else (height, width)
38 | 
39 |     data = np.reshape(data, shape)
40 |     data = np.flipud(data)
41 |     return data, scale
42 | 
43 | 
44 | def writePFM(file, image, scale=1):
45 |     file = open(file, 'wb')
46 | 
47 |     color = None
48 | 
49 |     if image.dtype.name != 'float32':
50 |         raise Exception('Image dtype must be float32.')
51 | 
52 |     image = np.flipud(image)
53 | 
54 |     if len(image.shape) == 3 and image.shape[2] == 3:  # color image
55 |         color = True
56 |     elif len(image.shape) == 2 or len(image.shape) == 3 and image.shape[2] == 1:  # greyscale
57 |         color = False
58 |     else:
59 |         raise Exception('Image must have H x W x 3, H x W x 1 or H x W dimensions.')
60 | 
61 |     file.write('PF\n' if color else 'Pf\n')
62 |     file.write('%d %d\n' % (image.shape[1], image.shape[0]))
63 | 
64 |     endian = image.dtype.byteorder
65 | 
66 |     if endian == '<' or endian == '=' and sys.byteorder == 'little':
67 |         scale = -scale
68 | 
69 |     file.write('%f\n' % scale)
70 | 
71 |     image.tofile(file)
72 | 
73 | 


--------------------------------------------------------------------------------
/kitti_eval/generate_multiview_extension.py:
--------------------------------------------------------------------------------
 1 | from __future__ import division
 2 | import numpy as np
 3 | import os
 4 | import cv2
 5 | import argparse
 6 | 
 7 | parser = argparse.ArgumentParser()
 8 | parser.add_argument("--dataset_dir", type=str, help="path to kitti scene flow multiview dataset")
 9 | parser.add_argument("--calib_dir",   type=str, help="path to data_scene_flow_calib")
10 | parser.add_argument("--dump_root",   type=str, help="where to dump the data")
11 | parser.add_argument("--cam_id",      type=str, default='02', help="camera id")
12 | parser.add_argument("--seq_length",  type=int, default=3, help="sequence length of pose snippets")
13 | parser.add_argument("--img_height",  type=int, default=128, help="image height")
14 | parser.add_argument("--img_width",   type=int, default=416, help="image width")
15 | args = parser.parse_args()
16 | 
17 | def read_raw_calib_file(filepath):
18 |     # From https://github.com/utiasSTARS/pykitti/blob/master/pykitti/utils.py
19 |     """Read in a calibration file and parse into a dictionary."""
20 |     data = {}
21 | 
22 |     with open(filepath, 'r') as f:
23 |         for line in f.readlines():
24 |             key, value = line.split(':', 1)
25 |             # The only non-float values in these files are dates, which
26 |             # we don't care about anyway
27 |             try:
28 |                     data[key] = np.array([float(x) for x in value.split()])
29 |             except ValueError:
30 |                     pass
31 |     return data
32 | 
33 | def load_intrinsics_raw(dataset_dir, fr_id, cid):
34 |     calib_file = os.path.join(dataset_dir, '%.6d.txt' % fr_id)
35 | 
36 |     filedata = read_raw_calib_file(calib_file)
37 |     P_rect = np.reshape(filedata['P_rect_' + cid], (3, 4))
38 |     intrinsics = P_rect[:3, :3]
39 |     return intrinsics
40 | 
41 | def scale_intrinsics(mat, sx, sy):
42 |     out = np.copy(mat)
43 |     out[0,0] *= sx
44 |     out[0,2] *= sx
45 |     out[1,1] *= sy
46 |     out[1,2] *= sy
47 |     return out
48 | 
49 | def concat_image_seq(seq):
50 |     for i, im in enumerate(seq):
51 |         if i == 0:
52 |             res = im
53 |         else:
54 |             res = np.hstack((res, im))
55 |     return res
56 | 
57 | def main():
58 |     frame_list = range(200)
59 |     # read calib files
60 |     calib_path = os.path.join(args.calib_dir, 'training', 'calib_cam_to_cam')
61 |     intri_list = []
62 |     for i in frame_list:
63 |         intri = load_intrinsics_raw(calib_path, i, args.cam_id)
64 |         intri_list.append(intri)
65 | 
66 |     # generate test examples
67 |     if not os.path.exists(args.dump_root):
68 |         os.makedirs(args.dump_root)
69 |     with open(os.path.join(args.dump_root, 'test_flow.txt'), 'w') as tf:
70 |         for i in frame_list:
71 |             half_offset = int((args.seq_length-1)/2)
72 |             img_seq = []
73 |             for o in range(-half_offset, half_offset+1):
74 |                 curr_img_path = args.dataset_dir + 'training/image_%s/%.6d_%.2d.png' % (args.cam_id[-1], i, 10+o)
75 |                 curr_img = cv2.imread(curr_img_path)
76 |                 if o==0:
77 |                     zoom_y = args.img_height/(curr_img.shape[0])
78 |                     zoom_x = args.img_width/(curr_img.shape[1])
79 |                 curr_img = cv2.resize(curr_img, (args.img_width, args.img_height))
80 |                 img_seq.append(curr_img)
81 |             intrinsics = scale_intrinsics(intri_list[i], zoom_x, zoom_y)
82 |             img_seq = concat_image_seq(img_seq)
83 |             fx = intrinsics[0, 0]
84 |             fy = intrinsics[1, 1]
85 |             cx = intrinsics[0, 2]
86 |             cy = intrinsics[1, 2]
87 |             dump_dir = os.path.join(args.dump_root, 'sub_folder_%s' % args.cam_id)
88 |             if not os.path.exists(dump_dir):
89 |                 os.makedirs(dump_dir)
90 |             dump_img_file = dump_dir + '/%.6d.jpg' % i
91 |             cv2.imwrite(dump_img_file, img_seq.astype(np.uint8))
92 |             dump_cam_file = dump_dir + '/%.6d_cam.txt' % i
93 |             with open(dump_cam_file, 'w') as f:
94 |                 f.write('%f,0.,%f,0.,%f,%f,0.,0.,1.' % (fx, cx, fy, cy))
95 |             tf.write('sub_folder_%s %.6d\n' % (args.cam_id, i))
96 | 
97 | main()
98 | 


--------------------------------------------------------------------------------
/kitti_eval/generate_pose_snippets.py:
--------------------------------------------------------------------------------
 1 | from __future__ import division
 2 | import os
 3 | import math
 4 | import scipy.misc
 5 | import numpy as np
 6 | import argparse
 7 | from glob import glob
 8 | from pose_evaluation_utils import mat2euler, dump_pose_seq_TUM
 9 | 
10 | parser = argparse.ArgumentParser()
11 | parser.add_argument("--dataset_dir", type=str, help="path to kitti odometry dataset")
12 | parser.add_argument("--output_dir",  type=str, help="path to output pose snippets")
13 | parser.add_argument("--seq_id",      type=int, default=9, help="sequence id to generate groundtruth pose snippets")
14 | parser.add_argument("--seq_length",  type=int, default=5, help="sequence length of pose snippets")
15 | args = parser.parse_args()
16 | 
17 | def is_valid_sample(frames, tgt_idx, seq_length):
18 |     N = len(frames)
19 |     tgt_drive, _ = frames[tgt_idx].split(' ')
20 |     max_src_offset = int((seq_length - 1)/2)
21 |     min_src_idx = tgt_idx - max_src_offset
22 |     max_src_idx = tgt_idx + max_src_offset
23 |     if min_src_idx < 0 or max_src_idx >= N:
24 |         return False
25 |     min_src_drive, _ = frames[min_src_idx].split(' ')
26 |     max_src_drive, _ = frames[max_src_idx].split(' ')
27 |     if tgt_drive == min_src_drive and tgt_drive == max_src_drive:
28 |         return True
29 |     return False
30 | 
31 | def main():
32 |     pose_gt_dir = args.dataset_dir + 'poses/'
33 |     if not os.path.isdir(args.output_dir):
34 |         os.makedirs(args.output_dir)
35 |     seq_dir = os.path.join(args.dataset_dir, 'sequences', '%.2d' % args.seq_id)
36 |     img_dir = os.path.join(seq_dir, 'image_2')
37 |     N = len(glob(img_dir + '/*.png'))
38 |     test_frames = ['%.2d %.6d' % (args.seq_id, n) for n in range(N)]
39 |     with open(args.dataset_dir + 'sequences/%.2d/times.txt' % args.seq_id, 'r') as f:
40 |         times = f.readlines()
41 |     times = np.array([float(s[:-1]) for s in times])
42 | 
43 |     with open(pose_gt_dir + '%.2d.txt' % args.seq_id, 'r') as f:
44 |         poses = f.readlines()
45 |     poses_gt = []
46 |     for pose in poses:
47 |         pose = np.array([float(s) for s in pose[:-1].split(' ')]).reshape((3,4))
48 |         rot = np.linalg.inv(pose[:,:3])
49 |         tran = -np.dot(rot, pose[:,3].transpose())
50 |         rz, ry, rx = mat2euler(rot)
51 |         poses_gt.append(tran.tolist() + [rx, ry, rz])
52 |     poses_gt = np.array(poses_gt)
53 | 
54 |     max_src_offset = (args.seq_length - 1)//2
55 |     for tgt_idx in range(N):
56 |         if not is_valid_sample(test_frames, tgt_idx, args.seq_length):
57 |             continue
58 |         if tgt_idx % 100 == 0:
59 |             print('Progress: %d/%d' % (tgt_idx, N))
60 |         pred_poses = poses_gt[tgt_idx - max_src_offset:tgt_idx + max_src_offset + 1]
61 |         curr_times = times[tgt_idx - max_src_offset:tgt_idx + max_src_offset + 1]
62 |         out_file = args.output_dir + '%.6d.txt' % (tgt_idx - max_src_offset)
63 |         dump_pose_seq_TUM(out_file, pred_poses, curr_times)
64 | 
65 | main()
66 | 
67 | 


--------------------------------------------------------------------------------
/kitti_eval/pose_evaluation_utils.py:
--------------------------------------------------------------------------------
  1 | # Some of the code are from the TUM evaluation toolkit:
  2 | # https://vision.in.tum.de/data/datasets/rgbd-dataset/tools#absolute_trajectory_error_ate
  3 | 
  4 | import math
  5 | import numpy as np
  6 | 
  7 | def compute_ate(gtruth_file, pred_file):
  8 |     gtruth_list = read_file_list(gtruth_file)
  9 |     pred_list = read_file_list(pred_file)
 10 |     matches = associate(gtruth_list, pred_list, 0, 0.01)
 11 |     if len(matches) < 2:
 12 |         return False
 13 | 
 14 |     gtruth_xyz = np.array([[float(value) for value in gtruth_list[a][0:3]] for a,b in matches])
 15 |     pred_xyz = np.array([[float(value) for value in pred_list[b][0:3]] for a,b in matches])
 16 |     
 17 |     # Make sure that the first matched frames align (no need for rotational alignment as
 18 |     # all the predicted/ground-truth snippets have been converted to use the same coordinate
 19 |     # system with the first frame of the snippet being the origin).
 20 |     offset = gtruth_xyz[0] - pred_xyz[0]
 21 |     pred_xyz += offset[None,:]
 22 | 
 23 |     # Optimize the scaling factor
 24 |     scale = np.sum(gtruth_xyz * pred_xyz)/np.sum(pred_xyz ** 2)
 25 |     alignment_error = pred_xyz * scale - gtruth_xyz
 26 |     rmse = np.sqrt(np.sum(alignment_error ** 2))/len(matches)
 27 |     return rmse
 28 | 
 29 | def read_file_list(filename):
 30 |     """
 31 |     Reads a trajectory from a text file. 
 32 |     
 33 |     File format:
 34 |     The file format is "stamp d1 d2 d3 ...", where stamp denotes the time stamp (to be matched)
 35 |     and "d1 d2 d3.." is arbitary data (e.g., a 3D position and 3D orientation) associated to this timestamp. 
 36 |     
 37 |     Input:
 38 |     filename -- File name
 39 |     
 40 |     Output:
 41 |     dict -- dictionary of (stamp,data) tuples
 42 |     
 43 |     """
 44 |     file = open(filename)
 45 |     data = file.read()
 46 |     lines = data.replace(","," ").replace("\t"," ").split("\n") 
 47 |     list = [[v.strip() for v in line.split(" ") if v.strip()!=""] for line in lines if len(line)>0 and line[0]!="#"]
 48 |     list = [(float(l[0]),l[1:]) for l in list if len(l)>1]
 49 |     return dict(list)
 50 | 
 51 | def associate(first_list, second_list,offset,max_difference):
 52 |     """
 53 |     Associate two dictionaries of (stamp,data). As the time stamps never match exactly, we aim 
 54 |     to find the closest match for every input tuple.
 55 |     
 56 |     Input:
 57 |     first_list -- first dictionary of (stamp,data) tuples
 58 |     second_list -- second dictionary of (stamp,data) tuples
 59 |     offset -- time offset between both dictionaries (e.g., to model the delay between the sensors)
 60 |     max_difference -- search radius for candidate generation
 61 | 
 62 |     Output:
 63 |     matches -- list of matched tuples ((stamp1,data1),(stamp2,data2))
 64 |     
 65 |     """
 66 |     first_keys = list(first_list.keys())
 67 |     second_keys = list(second_list.keys())
 68 |     potential_matches = [(abs(a - (b + offset)), a, b) 
 69 |                          for a in first_keys 
 70 |                          for b in second_keys 
 71 |                          if abs(a - (b + offset)) < max_difference]
 72 |     potential_matches.sort()
 73 |     matches = []
 74 |     for diff, a, b in potential_matches:
 75 |         if a in first_keys and b in second_keys:
 76 |             first_keys.remove(a)
 77 |             second_keys.remove(b)
 78 |             matches.append((a, b))
 79 |     
 80 |     matches.sort()
 81 |     return matches
 82 | 
 83 | def rot2quat(R):
 84 |     rz, ry, rx = mat2euler(R)
 85 |     qw, qx, qy, qz = euler2quat(rz, ry, rx)
 86 |     return qw, qx, qy, qz
 87 | 
 88 | def quat2mat(q):
 89 |     ''' Calculate rotation matrix corresponding to quaternion
 90 |     https://afni.nimh.nih.gov/pub/dist/src/pkundu/meica.libs/nibabel/quaternions.py
 91 |     Parameters
 92 |     ----------
 93 |     q : 4 element array-like
 94 | 
 95 |     Returns
 96 |     -------
 97 |     M : (3,3) array
 98 |       Rotation matrix corresponding to input quaternion *q*
 99 | 
100 |     Notes
101 |     -----
102 |     Rotation matrix applies to column vectors, and is applied to the
103 |     left of coordinate vectors.  The algorithm here allows non-unit
104 |     quaternions.
105 | 
106 |     References
107 |     ----------
108 |     Algorithm from
109 |     http://en.wikipedia.org/wiki/Rotation_matrix#Quaternion
110 | 
111 |     Examples
112 |     --------
113 |     >>> import numpy as np
114 |     >>> M = quat2mat([1, 0, 0, 0]) # Identity quaternion
115 |     >>> np.allclose(M, np.eye(3))
116 |     True
117 |     >>> M = quat2mat([0, 1, 0, 0]) # 180 degree rotn around axis 0
118 |     >>> np.allclose(M, np.diag([1, -1, -1]))
119 |     True
120 |     '''
121 |     w, x, y, z = q
122 |     Nq = w*w + x*x + y*y + z*z
123 |     if Nq < 1e-8:
124 |         return np.eye(3)
125 |     s = 2.0/Nq
126 |     X = x*s
127 |     Y = y*s
128 |     Z = z*s
129 |     wX = w*X; wY = w*Y; wZ = w*Z
130 |     xX = x*X; xY = x*Y; xZ = x*Z
131 |     yY = y*Y; yZ = y*Z; zZ = z*Z
132 |     return np.array(
133 |            [[ 1.0-(yY+zZ), xY-wZ, xZ+wY ],
134 |             [ xY+wZ, 1.0-(xX+zZ), yZ-wX ],
135 |             [ xZ-wY, yZ+wX, 1.0-(xX+yY) ]])
136 | 
137 | def mat2euler(M, cy_thresh=None, seq='zyx'):
138 |     '''
139 |     Taken From: http://afni.nimh.nih.gov/pub/dist/src/pkundu/meica.libs/nibabel/eulerangles.py
140 |     Discover Euler angle vector from 3x3 matrix
141 |     Uses the conventions above.
142 |     Parameters
143 |     ----------
144 |     M : array-like, shape (3,3)
145 |     cy_thresh : None or scalar, optional
146 |      threshold below which to give up on straightforward arctan for
147 |      estimating x rotation.  If None (default), estimate from
148 |      precision of input.
149 |     Returns
150 |     -------
151 |     z : scalar
152 |     y : scalar
153 |     x : scalar
154 |      Rotations in radians around z, y, x axes, respectively
155 |     Notes
156 |     -----
157 |     If there was no numerical error, the routine could be derived using
158 |     Sympy expression for z then y then x rotation matrix, which is::
159 |     [                       cos(y)*cos(z),                       -cos(y)*sin(z),         sin(y)],
160 |     [cos(x)*sin(z) + cos(z)*sin(x)*sin(y), cos(x)*cos(z) - sin(x)*sin(y)*sin(z), -cos(y)*sin(x)],
161 |     [sin(x)*sin(z) - cos(x)*cos(z)*sin(y), cos(z)*sin(x) + cos(x)*sin(y)*sin(z),  cos(x)*cos(y)]
162 |     with the obvious derivations for z, y, and x
163 |      z = atan2(-r12, r11)
164 |      y = asin(r13)
165 |      x = atan2(-r23, r33)
166 |     for x,y,z order
167 |     y = asin(-r31)
168 |     x = atan2(r32, r33)
169 |     z = atan2(r21, r11)
170 |     Problems arise when cos(y) is close to zero, because both of::
171 |      z = atan2(cos(y)*sin(z), cos(y)*cos(z))
172 |      x = atan2(cos(y)*sin(x), cos(x)*cos(y))
173 |     will be close to atan2(0, 0), and highly unstable.
174 |     The ``cy`` fix for numerical instability below is from: *Graphics
175 |     Gems IV*, Paul Heckbert (editor), Academic Press, 1994, ISBN:
176 |     0123361559.  Specifically it comes from EulerAngles.c by Ken
177 |     Shoemake, and deals with the case where cos(y) is close to zero:
178 |     See: http://www.graphicsgems.org/
179 |     The code appears to be licensed (from the website) as "can be used
180 |     without restrictions".
181 |     '''
182 |     M = np.asarray(M)
183 |     if cy_thresh is None:
184 |         try:
185 |             cy_thresh = np.finfo(M.dtype).eps * 4
186 |         except ValueError:
187 |             cy_thresh = _FLOAT_EPS_4
188 |     r11, r12, r13, r21, r22, r23, r31, r32, r33 = M.flat
189 |     # cy: sqrt((cos(y)*cos(z))**2 + (cos(x)*cos(y))**2)
190 |     cy = math.sqrt(r33*r33 + r23*r23)
191 |     if seq=='zyx':
192 |         if cy > cy_thresh: # cos(y) not close to zero, standard form
193 |             z = math.atan2(-r12,  r11) # atan2(cos(y)*sin(z), cos(y)*cos(z))
194 |             y = math.atan2(r13,  cy) # atan2(sin(y), cy)
195 |             x = math.atan2(-r23, r33) # atan2(cos(y)*sin(x), cos(x)*cos(y))
196 |         else: # cos(y) (close to) zero, so x -> 0.0 (see above)
197 |             # so r21 -> sin(z), r22 -> cos(z) and
198 |             z = math.atan2(r21,  r22)
199 |             y = math.atan2(r13,  cy) # atan2(sin(y), cy)
200 |             x = 0.0
201 |     elif seq=='xyz':
202 |         if cy > cy_thresh:
203 |             y = math.atan2(-r31, cy)
204 |             x = math.atan2(r32, r33)
205 |             z = math.atan2(r21, r11)
206 |         else:
207 |             z = 0.0
208 |             if r31 < 0:
209 |                 y = np.pi/2
210 |                 x = atan2(r12, r13)
211 |             else:
212 |                 y = -np.pi/2
213 |     else:
214 |         raise Exception('Sequence not recognized')
215 |     return z, y, x
216 | 
217 | import functools
218 | def euler2mat(z=0, y=0, x=0, isRadian=True):
219 |     ''' Return matrix for rotations around z, y and x axes
220 |     Uses the z, then y, then x convention above
221 |     Parameters
222 |     ----------
223 |     z : scalar
224 |          Rotation angle in radians around z-axis (performed first)
225 |     y : scalar
226 |          Rotation angle in radians around y-axis
227 |     x : scalar
228 |          Rotation angle in radians around x-axis (performed last)
229 |     Returns
230 |     -------
231 |     M : array shape (3,3)
232 |          Rotation matrix giving same rotation as for given angles
233 |     Examples
234 |     --------
235 |     >>> zrot = 1.3 # radians
236 |     >>> yrot = -0.1
237 |     >>> xrot = 0.2
238 |     >>> M = euler2mat(zrot, yrot, xrot)
239 |     >>> M.shape == (3, 3)
240 |     True
241 |     The output rotation matrix is equal to the composition of the
242 |     individual rotations
243 |     >>> M1 = euler2mat(zrot)
244 |     >>> M2 = euler2mat(0, yrot)
245 |     >>> M3 = euler2mat(0, 0, xrot)
246 |     >>> composed_M = np.dot(M3, np.dot(M2, M1))
247 |     >>> np.allclose(M, composed_M)
248 |     True
249 |     You can specify rotations by named arguments
250 |     >>> np.all(M3 == euler2mat(x=xrot))
251 |     True
252 |     When applying M to a vector, the vector should column vector to the
253 |     right of M.  If the right hand side is a 2D array rather than a
254 |     vector, then each column of the 2D array represents a vector.
255 |     >>> vec = np.array([1, 0, 0]).reshape((3,1))
256 |     >>> v2 = np.dot(M, vec)
257 |     >>> vecs = np.array([[1, 0, 0],[0, 1, 0]]).T # giving 3x2 array
258 |     >>> vecs2 = np.dot(M, vecs)
259 |     Rotations are counter-clockwise.
260 |     >>> zred = np.dot(euler2mat(z=np.pi/2), np.eye(3))
261 |     >>> np.allclose(zred, [[0, -1, 0],[1, 0, 0], [0, 0, 1]])
262 |     True
263 |     >>> yred = np.dot(euler2mat(y=np.pi/2), np.eye(3))
264 |     >>> np.allclose(yred, [[0, 0, 1],[0, 1, 0], [-1, 0, 0]])
265 |     True
266 |     >>> xred = np.dot(euler2mat(x=np.pi/2), np.eye(3))
267 |     >>> np.allclose(xred, [[1, 0, 0],[0, 0, -1], [0, 1, 0]])
268 |     True
269 |     Notes
270 |     -----
271 |     The direction of rotation is given by the right-hand rule (orient
272 |     the thumb of the right hand along the axis around which the rotation
273 |     occurs, with the end of the thumb at the positive end of the axis;
274 |     curl your fingers; the direction your fingers curl is the direction
275 |     of rotation).  Therefore, the rotations are counterclockwise if
276 |     looking along the axis of rotation from positive to negative.
277 |     '''
278 | 
279 |     if not isRadian:
280 |         z = ((np.pi)/180.) * z
281 |         y = ((np.pi)/180.) * y
282 |         x = ((np.pi)/180.) * x
283 |     assert z>=(-np.pi) and z < np.pi, 'Inapprorpriate z: %f' % z
284 |     assert y>=(-np.pi) and y < np.pi, 'Inapprorpriate y: %f' % y
285 |     assert x>=(-np.pi) and x < np.pi, 'Inapprorpriate x: %f' % x    
286 | 
287 |     Ms = []
288 |     if z:
289 |             cosz = math.cos(z)
290 |             sinz = math.sin(z)
291 |             Ms.append(np.array(
292 |                             [[cosz, -sinz, 0],
293 |                              [sinz, cosz, 0],
294 |                              [0, 0, 1]]))
295 |     if y:
296 |             cosy = math.cos(y)
297 |             siny = math.sin(y)
298 |             Ms.append(np.array(
299 |                             [[cosy, 0, siny],
300 |                              [0, 1, 0],
301 |                              [-siny, 0, cosy]]))
302 |     if x:
303 |             cosx = math.cos(x)
304 |             sinx = math.sin(x)
305 |             Ms.append(np.array(
306 |                             [[1, 0, 0],
307 |                              [0, cosx, -sinx],
308 |                              [0, sinx, cosx]]))
309 |     if Ms:
310 |             return functools.reduce(np.dot, Ms[::-1])
311 |     return np.eye(3)
312 | 
313 | def euler2quat(z=0, y=0, x=0, isRadian=True):
314 |     ''' Return quaternion corresponding to these Euler angles
315 |     Uses the z, then y, then x convention above
316 |     Parameters
317 |     ----------
318 |     z : scalar
319 |          Rotation angle in radians around z-axis (performed first)
320 |     y : scalar
321 |          Rotation angle in radians around y-axis
322 |     x : scalar
323 |          Rotation angle in radians around x-axis (performed last)
324 |     Returns
325 |     -------
326 |     quat : array shape (4,)
327 |          Quaternion in w, x, y z (real, then vector) format
328 |     Notes
329 |     -----
330 |     We can derive this formula in Sympy using:
331 |     1. Formula giving quaternion corresponding to rotation of theta radians
332 |          about arbitrary axis:
333 |          http://mathworld.wolfram.com/EulerParameters.html
334 |     2. Generated formulae from 1.) for quaternions corresponding to
335 |          theta radians rotations about ``x, y, z`` axes
336 |     3. Apply quaternion multiplication formula -
337 |          http://en.wikipedia.org/wiki/Quaternions#Hamilton_product - to
338 |          formulae from 2.) to give formula for combined rotations.
339 |     '''
340 |   
341 |     if not isRadian:
342 |         z = ((np.pi)/180.) * z
343 |         y = ((np.pi)/180.) * y
344 |         x = ((np.pi)/180.) * x
345 |     z = z/2.0
346 |     y = y/2.0
347 |     x = x/2.0
348 |     cz = math.cos(z)
349 |     sz = math.sin(z)
350 |     cy = math.cos(y)
351 |     sy = math.sin(y)
352 |     cx = math.cos(x)
353 |     sx = math.sin(x)
354 |     return np.array([
355 |                      cx*cy*cz - sx*sy*sz,
356 |                      cx*sy*sz + cy*cz*sx,
357 |                      cx*cz*sy - sx*cy*sz,
358 |                      cx*cy*sz + sx*cz*sy])
359 | 
360 | def pose_vec_to_mat(vec):
361 |     tx = vec[0]
362 |     ty = vec[1]
363 |     tz = vec[2]
364 |     trans = np.array([tx, ty, tz]).reshape((3,1))
365 |     rot = euler2mat(vec[5], vec[4], vec[3])
366 |     Tmat = np.concatenate((rot, trans), axis=1)
367 |     hfiller = np.array([0, 0, 0, 1]).reshape((1,4))
368 |     Tmat = np.concatenate((Tmat, hfiller), axis=0)
369 |     return Tmat
370 | 
371 | def dump_pose_seq_TUM(out_file, poses, times):
372 |     # First frame as the origin
373 |     first_pose = pose_vec_to_mat(poses[0])
374 |     with open(out_file, 'w') as f:
375 |         for p in range(len(times)):
376 |             this_pose = pose_vec_to_mat(poses[p])
377 |             # this_pose = np.dot(this_pose, np.linalg.inv(first_pose))
378 |             this_pose = np.dot(first_pose, np.linalg.inv(this_pose))
379 |             tx = this_pose[0, 3]
380 |             ty = this_pose[1, 3]
381 |             tz = this_pose[2, 3]
382 |             rot = this_pose[:3, :3]
383 |             qw, qx, qy, qz = rot2quat(rot)
384 |             f.write('%f %f %f %f %f %f %f %f\n' % (times[p], tx, ty, tz, qx, qy, qz, qw))


--------------------------------------------------------------------------------
/misc/overview.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/yzcjtr/GeoNet/5b176025bc63563ef53297aa3d20cc6e575fb833/misc/overview.jpg


--------------------------------------------------------------------------------
/utils.py:
--------------------------------------------------------------------------------
  1 | # Mostly based on the code written by Tinghui Zhou: 
  2 | # https://github.com/tinghuiz/SfMLearner/blob/master/utils.py
  3 | from __future__ import division
  4 | import numpy as np
  5 | import tensorflow as tf
  6 | 
  7 | def euler2mat(z, y, x):
  8 |   """Converts euler angles to rotation matrix
  9 |    TODO: remove the dimension for 'N' (deprecated for converting all source
 10 |          poses altogether)
 11 |    Reference: https://github.com/pulkitag/pycaffe-utils/blob/master/rot_utils.py#L174
 12 |   Args:
 13 |       z: rotation angle along z axis (in radians) -- size = [B, N]
 14 |       y: rotation angle along y axis (in radians) -- size = [B, N]
 15 |       x: rotation angle along x axis (in radians) -- size = [B, N]
 16 |   Returns:
 17 |       Rotation matrix corresponding to the euler angles -- size = [B, N, 3, 3]
 18 |   """
 19 |   B = tf.shape(z)[0]
 20 |   N = 1
 21 |   z = tf.clip_by_value(z, -np.pi, np.pi)
 22 |   y = tf.clip_by_value(y, -np.pi, np.pi)
 23 |   x = tf.clip_by_value(x, -np.pi, np.pi)
 24 | 
 25 |   # Expand to B x N x 1 x 1
 26 |   z = tf.expand_dims(tf.expand_dims(z, -1), -1)
 27 |   y = tf.expand_dims(tf.expand_dims(y, -1), -1)
 28 |   x = tf.expand_dims(tf.expand_dims(x, -1), -1)
 29 | 
 30 |   zeros = tf.zeros([B, N, 1, 1])
 31 |   ones  = tf.ones([B, N, 1, 1])
 32 | 
 33 |   cosz = tf.cos(z)
 34 |   sinz = tf.sin(z)
 35 |   rotz_1 = tf.concat([cosz, -sinz, zeros], axis=3)
 36 |   rotz_2 = tf.concat([sinz,  cosz, zeros], axis=3)
 37 |   rotz_3 = tf.concat([zeros, zeros, ones], axis=3)
 38 |   zmat = tf.concat([rotz_1, rotz_2, rotz_3], axis=2)
 39 | 
 40 |   cosy = tf.cos(y)
 41 |   siny = tf.sin(y)
 42 |   roty_1 = tf.concat([cosy, zeros, siny], axis=3)
 43 |   roty_2 = tf.concat([zeros, ones, zeros], axis=3)
 44 |   roty_3 = tf.concat([-siny,zeros, cosy], axis=3)
 45 |   ymat = tf.concat([roty_1, roty_2, roty_3], axis=2)
 46 | 
 47 |   cosx = tf.cos(x)
 48 |   sinx = tf.sin(x)
 49 |   rotx_1 = tf.concat([ones, zeros, zeros], axis=3)
 50 |   rotx_2 = tf.concat([zeros, cosx, -sinx], axis=3)
 51 |   rotx_3 = tf.concat([zeros, sinx, cosx], axis=3)
 52 |   xmat = tf.concat([rotx_1, rotx_2, rotx_3], axis=2)
 53 | 
 54 |   rotMat = tf.matmul(tf.matmul(xmat, ymat), zmat)
 55 |   return rotMat
 56 | 
 57 | def pose_vec2mat(vec):
 58 |   """Converts 6DoF parameters to transformation matrix
 59 |   Args:
 60 |       vec: 6DoF parameters in the order of tx, ty, tz, rx, ry, rz -- [B, 6]
 61 |   Returns:
 62 |       A transformation matrix -- [B, 4, 4]
 63 |   """
 64 |   batch_size, _ = vec.get_shape().as_list()
 65 |   translation = tf.slice(vec, [0, 0], [-1, 3])
 66 |   translation = tf.expand_dims(translation, -1)
 67 |   rx = tf.slice(vec, [0, 3], [-1, 1])
 68 |   ry = tf.slice(vec, [0, 4], [-1, 1])
 69 |   rz = tf.slice(vec, [0, 5], [-1, 1])
 70 |   rot_mat = euler2mat(rz, ry, rx)
 71 |   rot_mat = tf.squeeze(rot_mat, axis=[1])
 72 |   filler = tf.constant([0.0, 0.0, 0.0, 1.0], shape=[1, 1, 4])
 73 |   filler = tf.tile(filler, [batch_size, 1, 1])
 74 |   transform_mat = tf.concat([rot_mat, translation], axis=2)
 75 |   transform_mat = tf.concat([transform_mat, filler], axis=1)
 76 |   return transform_mat
 77 | 
 78 | def pixel2cam(depth, pixel_coords, intrinsics, is_homogeneous=True):
 79 |   """Transforms coordinates in the pixel frame to the camera frame.
 80 | 
 81 |   Args:
 82 |     depth: [batch, height, width]
 83 |     pixel_coords: homogeneous pixel coordinates [batch, 3, height, width]
 84 |     intrinsics: camera intrinsics [batch, 3, 3]
 85 |     is_homogeneous: return in homogeneous coordinates
 86 |   Returns:
 87 |     Coords in the camera frame [batch, 3 (4 if homogeneous), height, width]
 88 |   """
 89 |   batch, height, width = depth.get_shape().as_list()
 90 |   depth = tf.reshape(depth, [batch, 1, -1])
 91 |   pixel_coords = tf.reshape(pixel_coords, [batch, 3, -1])
 92 |   cam_coords = tf.matmul(tf.matrix_inverse(intrinsics), pixel_coords) * depth
 93 |   if is_homogeneous:
 94 |     ones = tf.ones([batch, 1, height*width])
 95 |     cam_coords = tf.concat([cam_coords, ones], axis=1)
 96 |   cam_coords = tf.reshape(cam_coords, [batch, -1, height, width])
 97 |   return cam_coords
 98 | 
 99 | def cam2pixel(cam_coords, proj):
100 |   """Transforms coordinates in a camera frame to the pixel frame.
101 | 
102 |   Args:
103 |     cam_coords: [batch, 4, height, width]
104 |     proj: [batch, 4, 4]
105 |   Returns:
106 |     Pixel coordinates projected from the camera frame [batch, height, width, 2]
107 |   """
108 |   batch, _, height, width = cam_coords.get_shape().as_list()
109 |   cam_coords = tf.reshape(cam_coords, [batch, 4, -1])
110 |   unnormalized_pixel_coords = tf.matmul(proj, cam_coords)
111 |   x_u = tf.slice(unnormalized_pixel_coords, [0, 0, 0], [-1, 1, -1])
112 |   y_u = tf.slice(unnormalized_pixel_coords, [0, 1, 0], [-1, 1, -1])
113 |   z_u = tf.slice(unnormalized_pixel_coords, [0, 2, 0], [-1, 1, -1])
114 |   x_n = x_u / (z_u + 1e-10)
115 |   y_n = y_u / (z_u + 1e-10)
116 |   pixel_coords = tf.concat([x_n, y_n], axis=1)
117 |   pixel_coords = tf.reshape(pixel_coords, [batch, 2, height, width])
118 |   return tf.transpose(pixel_coords, perm=[0, 2, 3, 1])
119 | 
120 | def meshgrid(batch, height, width, is_homogeneous=True):
121 |   """Construct a 2D meshgrid.
122 | 
123 |   Args:
124 |     batch: batch size
125 |     height: height of the grid
126 |     width: width of the grid
127 |     is_homogeneous: whether to return in homogeneous coordinates
128 |   Returns:
129 |     x,y grid coordinates [batch, 2 (3 if homogeneous), height, width]
130 |   """
131 |   x_t = tf.matmul(tf.ones(shape=tf.stack([height, 1])),
132 |                   tf.transpose(tf.expand_dims(
133 |                       tf.linspace(-1.0, 1.0, width), 1), [1, 0]))
134 |   y_t = tf.matmul(tf.expand_dims(tf.linspace(-1.0, 1.0, height), 1),
135 |                   tf.ones(shape=tf.stack([1, width])))
136 |   x_t = (x_t + 1.0) * 0.5 * tf.cast(width - 1, tf.float32)
137 |   y_t = (y_t + 1.0) * 0.5 * tf.cast(height - 1, tf.float32)
138 |   if is_homogeneous:
139 |     ones = tf.ones_like(x_t)
140 |     coords = tf.stack([x_t, y_t, ones], axis=0)
141 |   else:
142 |     coords = tf.stack([x_t, y_t], axis=0)
143 |   coords = tf.tile(tf.expand_dims(coords, 0), [batch, 1, 1, 1])
144 |   return coords
145 | 
146 | def flow_warp(src_img, flow):
147 |   """ inverse warp a source image to the target image plane based on flow field
148 |   Args:
149 |     src_img: the source  image [batch, height_s, width_s, 3]
150 |     flow: target image to source image flow [batch, height_t, width_t, 2]
151 |   Returns:
152 |     Source image inverse warped to the target image plane [batch, height_t, width_t, 3]
153 |   """
154 |   batch, height, width, _ = src_img.get_shape().as_list()
155 |   tgt_pixel_coords = tf.transpose(meshgrid(batch, height, width, False),
156 |                      [0, 2, 3, 1])
157 |   src_pixel_coords = tgt_pixel_coords + flow
158 |   output_img = bilinear_sampler(src_img, src_pixel_coords)
159 |   return output_img
160 | 
161 | def compute_rigid_flow(depth, pose, intrinsics, reverse_pose=False):
162 |   """Compute the rigid flow from target image plane to source image
163 | 
164 |   Args:
165 |     depth: depth map of the target image [batch, height_t, width_t]
166 |     pose: target to source (or source to target if reverse_pose=True) 
167 |           camera transformation matrix [batch, 6], in the order of 
168 |           tx, ty, tz, rx, ry, rz; 
169 |     intrinsics: camera intrinsics [batch, 3, 3]
170 |   Returns:
171 |     Rigid flow from target image to source image [batch, height_t, width_t, 2]
172 |   """
173 |   batch, height, width = depth.get_shape().as_list()
174 |   # Convert pose vector to matrix
175 |   pose = pose_vec2mat(pose)
176 |   if reverse_pose:
177 |     pose = tf.matrix_inverse(pose)
178 |   # Construct pixel grid coordinates
179 |   pixel_coords = meshgrid(batch, height, width)
180 |   tgt_pixel_coords = tf.transpose(pixel_coords[:,:2,:,:], [0, 2, 3, 1])
181 |   # Convert pixel coordinates to the camera frame
182 |   cam_coords = pixel2cam(depth, pixel_coords, intrinsics)
183 |   # Construct a 4x4 intrinsic matrix
184 |   filler = tf.constant([0.0, 0.0, 0.0, 1.0], shape=[1, 1, 4])
185 |   filler = tf.tile(filler, [batch, 1, 1])
186 |   intrinsics = tf.concat([intrinsics, tf.zeros([batch, 3, 1])], axis=2)
187 |   intrinsics = tf.concat([intrinsics, filler], axis=1)
188 |   # Get a 4x4 transformation matrix from 'target' camera frame to 'source'
189 |   # pixel frame.
190 |   proj_tgt_cam_to_src_pixel = tf.matmul(intrinsics, pose)
191 |   src_pixel_coords = cam2pixel(cam_coords, proj_tgt_cam_to_src_pixel)
192 |   rigid_flow = src_pixel_coords - tgt_pixel_coords
193 |   return rigid_flow
194 | 
195 | def bilinear_sampler(imgs, coords):
196 |   """Construct a new image by bilinear sampling from the input image.
197 | 
198 |   Points falling outside the source image boundary have value 0.
199 | 
200 |   Args:
201 |     imgs: source image to be sampled from [batch, height_s, width_s, channels]
202 |     coords: coordinates of source pixels to sample from [batch, height_t,
203 |       width_t, 2]. height_t/width_t correspond to the dimensions of the output
204 |       image (don't need to be the same as height_s/width_s). The two channels
205 |       correspond to x and y coordinates respectively.
206 |   Returns:
207 |     A new sampled image [batch, height_t, width_t, channels]
208 |   """
209 |   def _repeat(x, n_repeats):
210 |     rep = tf.transpose(
211 |         tf.expand_dims(tf.ones(shape=tf.stack([
212 |             n_repeats,
213 |         ])), 1), [1, 0])
214 |     rep = tf.cast(rep, 'float32')
215 |     x = tf.matmul(tf.reshape(x, (-1, 1)), rep)
216 |     return tf.reshape(x, [-1])
217 | 
218 |   with tf.name_scope('image_sampling'):
219 |     coords_x, coords_y = tf.split(coords, [1, 1], axis=3)
220 |     inp_size = imgs.get_shape()
221 |     coord_size = coords.get_shape()
222 |     out_size = coords.get_shape().as_list()
223 |     out_size[3] = imgs.get_shape().as_list()[3]
224 | 
225 |     coords_x = tf.cast(coords_x, 'float32')
226 |     coords_y = tf.cast(coords_y, 'float32')
227 | 
228 |     x0 = tf.floor(coords_x)
229 |     x1 = x0 + 1
230 |     y0 = tf.floor(coords_y)
231 |     y1 = y0 + 1
232 | 
233 |     y_max = tf.cast(tf.shape(imgs)[1] - 1, 'float32')
234 |     x_max = tf.cast(tf.shape(imgs)[2] - 1, 'float32')
235 |     zero = tf.zeros([1], dtype='float32')
236 | 
237 |     x0_safe = tf.clip_by_value(x0, zero, x_max)
238 |     y0_safe = tf.clip_by_value(y0, zero, y_max)
239 |     x1_safe = tf.clip_by_value(x1, zero, x_max)
240 |     y1_safe = tf.clip_by_value(y1, zero, y_max)
241 | 
242 |     ## bilinear interp weights, with points outside the grid having weight 0
243 |     # wt_x0 = (x1 - coords_x) * tf.cast(tf.equal(x0, x0_safe), 'float32')
244 |     # wt_x1 = (coords_x - x0) * tf.cast(tf.equal(x1, x1_safe), 'float32')
245 |     # wt_y0 = (y1 - coords_y) * tf.cast(tf.equal(y0, y0_safe), 'float32')
246 |     # wt_y1 = (coords_y - y0) * tf.cast(tf.equal(y1, y1_safe), 'float32')
247 | 
248 |     wt_x0 = x1_safe - coords_x
249 |     wt_x1 = coords_x - x0_safe
250 |     wt_y0 = y1_safe - coords_y
251 |     wt_y1 = coords_y - y0_safe
252 | 
253 |     ## indices in the flat image to sample from
254 |     dim2 = tf.cast(inp_size[2], 'float32')
255 |     dim1 = tf.cast(inp_size[2] * inp_size[1], 'float32')
256 |     base = tf.reshape(
257 |         _repeat(
258 |             tf.cast(tf.range(coord_size[0]), 'float32') * dim1,
259 |             coord_size[1] * coord_size[2]),
260 |         [out_size[0], out_size[1], out_size[2], 1])
261 | 
262 |     base_y0 = base + y0_safe * dim2
263 |     base_y1 = base + y1_safe * dim2
264 |     idx00 = tf.reshape(x0_safe + base_y0, [-1])
265 |     idx01 = x0_safe + base_y1
266 |     idx10 = x1_safe + base_y0
267 |     idx11 = x1_safe + base_y1
268 | 
269 |     ## sample from imgs
270 |     imgs_flat = tf.reshape(imgs, tf.stack([-1, inp_size[3]]))
271 |     imgs_flat = tf.cast(imgs_flat, 'float32')
272 |     im00 = tf.reshape(tf.gather(imgs_flat, tf.cast(idx00, 'int32')), out_size)
273 |     im01 = tf.reshape(tf.gather(imgs_flat, tf.cast(idx01, 'int32')), out_size)
274 |     im10 = tf.reshape(tf.gather(imgs_flat, tf.cast(idx10, 'int32')), out_size)
275 |     im11 = tf.reshape(tf.gather(imgs_flat, tf.cast(idx11, 'int32')), out_size)
276 | 
277 |     w00 = wt_x0 * wt_y0
278 |     w01 = wt_x0 * wt_y1
279 |     w10 = wt_x1 * wt_y0
280 |     w11 = wt_x1 * wt_y1
281 | 
282 |     output = tf.add_n([
283 |         w00 * im00, w01 * im01,
284 |         w10 * im10, w11 * im11
285 |     ])
286 |     return output
287 | 


--------------------------------------------------------------------------------