├── .gitignore ├── LICENSE ├── README.md ├── classifier ├── dgcnn.py └── pointnet.py ├── prepare_data ├── README.md ├── intrinsics.txt ├── normalize_modelnet.py ├── parallel_render.py ├── render_single.py ├── test_models.txt └── traj2pcd.py ├── requirements.txt ├── segmenter ├── dgcnn.py └── pointnet.py ├── test_cls.py ├── test_pose.py ├── test_seg.py ├── train_cls.py ├── train_pose.py ├── train_seg.py ├── transformer ├── it_net.py ├── it_net_dgcnn.py └── t_net.py └── util ├── data_util.py ├── log_util.py ├── tf_util.py └── visu_util.py /.gitignore: -------------------------------------------------------------------------------- 1 | *__pycache__ 2 | data 3 | log 4 | results -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Wentao Yuan 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Iterative Transformer Network for 3D Point Cloud 2 | #### [[paper]](https://arxiv.org/pdf/1811.11209.pdf) [[data]](https://drive.google.com/open?id=178Ykn-_GDAbfBlKDAs-9MBwRFI53bznF) 3 | 4 | ### Introduction 5 | This repository contains the implementation of [Iterative Transformer Network](https://arxiv.org/abs/1811.11209) (IT-Net), a network module that predicts 3D rigid transformations from partial point clouds in an iterative fashion. IT-Net can be used independently for canonical pose estimation or jointly with downstream networks for shape classification and part segmentation. Please refer to our [paper](https://arxiv.org/abs/1811.11209) for more details. 6 | 7 | ### Citation 8 | If you find our work useful, please consider citing our paper. 9 | ``` 10 | @article{yuan2018iterative, 11 | title = {Iterative Transformer Network for 3D Point Cloud}, 12 | author = {Yuan, Wentao and Held, David and Mertz, Christoph and Hebert, Martial}, 13 | journal = {arXiv preprint arXiv:1811.11209}, 14 | year = {2018} 15 | } 16 | ``` 17 | 18 | ### Usage 19 | #### 1) Setup 20 | 1. Install dependencies by running `pip install -r requirments.txt`. 21 | 2. Download data and pre-trained models from [Google Drive](https://drive.google.com/open?id=178Ykn-_GDAbfBlKDAs-9MBwRFI53bznF). 22 | 23 | This code is tested on Ubuntu 16.04 with CUDA 9.0 and python 3.6. 24 | 25 | #### 2) Object Pose Estimation 26 | Here is an example command to get results with pretrained model. 27 | ``` 28 | python test_pose.py --transformer it_net --n_iter 10 --lmdb data/shapenet_pose/car/test.lmdb --checkpoint data/trained_models/pose_estimation/car-5_iter --results results/pose_estimation/car 29 | ``` 30 | This will generate statistics as well as visualizations shown in the paper for car pose estimation. The default setting uses 5 unrolled iterations during training and 10 during testing (controlled by the `n_iter` option). Besides the car category, we also provide data and pre-trained model for the chair category. 31 | 32 | We provide implementations for three different 3D transformer networks, including T-Net (baseline), IT-Net with PointNet backbone (evaluated in the paper) and IT-Net with DGCNN backbone (achieves slightly better results at the cost of more computation and memory). Use the `transformer` option to switch between different architectures. 33 | 34 | #### 3) 3D Shape Classification 35 | Here is an example command to get results with pretrained model. 36 | ``` 37 | python test_cls.py --classifier pointnet --transformer it_net --n_iter 2 --lmdb data/partial_modelnet40/test.lmdb --checkpoint data/trained_models/classification/pointnet-2_iter --results results/classification/pointnet 38 | ``` 39 | This will evaluate the 2-iteration IT-Net trained jointly with PointNet classifier on the partial ModelNet40 dataset. The `classifier` option selects the downstream classification network (PointNet or DGCNN). The `transformer` option selects the 3D transformer model. The `n_iter` options specifies the number of unrolled iterations for the 3D transformer. We also provide pre-trained 2-iteration IT-Net with DGCNN classifier. 40 | 41 | #### 4) Object Part Segmentation 42 | Here is an example command to get results with pretrained model. 43 | ``` 44 | python test_seg.py --classifier pointnet --segmenter it_net --n_iter 2 --lmdb data/shapenet_part/test.lmdb --checkpoint data/trained_models/segmentation/pointnet-2_iter --results results/segmentation/pointnet 45 | ``` 46 | The options are similar to classification, except that the downstream networks are point segmentation networks (controlled via the `segmenter` option) and the dataset is ShapeNet Part. We provide two pre-trained models: one with PointNet and one with DGCNN as the segmentation backbone. 47 | 48 | #### 5) Data Generation 49 | The `prepare_data` folder contains scripts to generate partial point clouds from CAD datasets. Feel free to use these scripts to generate your own partial point cloud data. 50 | 51 | ### License 52 | This project code is released under the MIT License. 53 | -------------------------------------------------------------------------------- /classifier/dgcnn.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 4 | sys.path.append(os.path.join(BASE_DIR, '../util')) 5 | import tensorflow as tf 6 | import tf_util 7 | 8 | 9 | def get_model(point_cloud, is_training, bn_decay, k=20): 10 | """ Classification DGCNN, input is BxNx3, output Bx40 """ 11 | 12 | # EdgeConv functions (MLP implemented as conv2d) 13 | adj_matrix = tf_util.pairwise_distance(point_cloud) 14 | nn_idx = tf_util.knn(adj_matrix, k=k) 15 | edge_feature = tf_util.get_edge_feature(point_cloud, nn_idx=nn_idx, k=k) 16 | 17 | net = tf_util.conv2d(edge_feature, 64, [1,1], 18 | padding='VALID', stride=[1,1], 19 | bn=True, is_training=is_training, 20 | scope='dgcnn1', bn_decay=bn_decay) 21 | net = tf.reduce_max(net, axis=-2, keepdims=True) 22 | net1 = net 23 | 24 | adj_matrix = tf_util.pairwise_distance(net) 25 | nn_idx = tf_util.knn(adj_matrix, k=k) 26 | edge_feature = tf_util.get_edge_feature(net, nn_idx=nn_idx, k=k) 27 | 28 | net = tf_util.conv2d(edge_feature, 64, [1,1], 29 | padding='VALID', stride=[1,1], 30 | bn=True, is_training=is_training, 31 | scope='dgcnn2', bn_decay=bn_decay) 32 | net = tf.reduce_max(net, axis=-2, keepdims=True) 33 | net2 = net 34 | 35 | adj_matrix = tf_util.pairwise_distance(net) 36 | nn_idx = tf_util.knn(adj_matrix, k=k) 37 | edge_feature = tf_util.get_edge_feature(net, nn_idx=nn_idx, k=k) 38 | 39 | net = tf_util.conv2d(edge_feature, 64, [1,1], 40 | padding='VALID', stride=[1,1], 41 | bn=True, is_training=is_training, 42 | scope='dgcnn3', bn_decay=bn_decay) 43 | net = tf.reduce_max(net, axis=-2, keepdims=True) 44 | net3 = net 45 | 46 | adj_matrix = tf_util.pairwise_distance(net) 47 | nn_idx = tf_util.knn(adj_matrix, k=k) 48 | edge_feature = tf_util.get_edge_feature(net, nn_idx=nn_idx, k=k) 49 | 50 | net = tf_util.conv2d(edge_feature, 128, [1,1], 51 | padding='VALID', stride=[1,1], 52 | bn=True, is_training=is_training, 53 | scope='dgcnn4', bn_decay=bn_decay) 54 | net = tf.reduce_max(net, axis=-2, keepdims=True) 55 | net4 = net 56 | 57 | net = tf_util.conv2d(tf.concat([net1, net2, net3, net4], axis=-1), 1024, [1, 1], 58 | padding='VALID', stride=[1,1], 59 | bn=True, is_training=is_training, 60 | scope='agg', bn_decay=bn_decay) 61 | net = tf.squeeze(net, -2) 62 | 63 | # Symmetric function: max pooling 64 | net = tf.reduce_max(net, axis=1, name='maxpool') 65 | 66 | # MLP on global point cloud vector 67 | net = tf_util.fully_connected(net, 512, bn=True, is_training=is_training, 68 | scope='fc1', bn_decay=bn_decay) 69 | net = tf_util.dropout(net, keep_prob=0.5, is_training=is_training, scope='dp1') 70 | net = tf_util.fully_connected(net, 256, bn=True, is_training=is_training, 71 | scope='fc2', bn_decay=bn_decay) 72 | net = tf_util.dropout(net, keep_prob=0.5, is_training=is_training, scope='dp2') 73 | net = tf_util.fully_connected(net, 40, activation_fn=None, scope='fc3') 74 | 75 | return net 76 | 77 | 78 | def get_loss(pred, label): 79 | """ pred: B x NUM_CLASSES, 80 | label: B, """ 81 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 82 | classify_loss = tf.reduce_mean(loss) 83 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 84 | return classify_loss 85 | 86 | 87 | def get_loss_reg(pred, label, transform, reg_weight=0.001): 88 | """ pred: B x NUM_CLASSES, 89 | label: B, 90 | transform: B x 3 x 3 """ 91 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 92 | classify_loss = tf.reduce_mean(loss) 93 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 94 | 95 | # Enforce the transformation as orthogonal matrix 96 | mat_diff = tf.matmul(transform, tf.transpose(transform, perm=[0,2,1])) - tf.eye(3) 97 | mat_diff_loss = tf.nn.l2_loss(mat_diff) 98 | tf.summary.scalar('train/mat loss', mat_diff_loss, collections=['train']) 99 | 100 | return classify_loss + mat_diff_loss * reg_weight 101 | -------------------------------------------------------------------------------- /classifier/pointnet.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 4 | sys.path.append(os.path.join(BASE_DIR, '../util')) 5 | import tensorflow as tf 6 | import tf_util 7 | 8 | 9 | def get_model(point_cloud, is_training, bn_decay): 10 | """ Classification PointNet, input is BxNx3, output Bx40 """ 11 | point_cloud = tf.expand_dims(point_cloud, -2) 12 | 13 | # Point functions (MLP implemented as conv2d) 14 | net = tf_util.conv2d(point_cloud, 64, [1,1], 15 | padding='VALID', stride=[1,1], 16 | bn=True, is_training=is_training, 17 | scope='conv1', bn_decay=bn_decay) 18 | net = tf_util.conv2d(net, 64, [1,1], 19 | padding='VALID', stride=[1,1], 20 | bn=True, is_training=is_training, 21 | scope='conv2', bn_decay=bn_decay) 22 | net = tf_util.conv2d(net, 64, [1,1], 23 | padding='VALID', stride=[1,1], 24 | bn=True, is_training=is_training, 25 | scope='conv3', bn_decay=bn_decay) 26 | net = tf_util.conv2d(net, 128, [1,1], 27 | padding='VALID', stride=[1,1], 28 | bn=True, is_training=is_training, 29 | scope='conv4', bn_decay=bn_decay) 30 | net = tf_util.conv2d(net, 1024, [1,1], 31 | padding='VALID', stride=[1,1], 32 | bn=True, is_training=is_training, 33 | scope='conv5', bn_decay=bn_decay) 34 | net = tf.squeeze(net, -2) 35 | 36 | # Symmetric function: max pooling 37 | net = tf.reduce_max(net, axis=1, name='maxpool') 38 | 39 | # MLP on global point cloud vector 40 | net = tf_util.fully_connected(net, 512, bn=True, is_training=is_training, 41 | scope='fc1', bn_decay=bn_decay) 42 | net = tf_util.fully_connected(net, 256, bn=True, is_training=is_training, 43 | scope='fc2', bn_decay=bn_decay) 44 | net = tf_util.dropout(net, keep_prob=0.7, is_training=is_training, 45 | scope='dp1') 46 | net = tf_util.fully_connected(net, 40, activation_fn=None, scope='fc3') 47 | 48 | return net 49 | 50 | 51 | def get_loss(pred, label): 52 | """ pred: B x NUM_CLASSES, 53 | label: B, """ 54 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 55 | classify_loss = tf.reduce_mean(loss) 56 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 57 | return classify_loss 58 | 59 | 60 | def get_loss_reg(pred, label, transform, reg_weight=0.001): 61 | """ pred: B x NUM_CLASSES, 62 | label: B, 63 | transform: B x 3 x 3 """ 64 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 65 | classify_loss = tf.reduce_mean(loss) 66 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 67 | 68 | # Enforce the transformation as orthogonal matrix 69 | mat_diff = tf.matmul(transform, tf.transpose(transform, perm=[0,2,1])) - tf.eye(3) 70 | mat_diff_loss = tf.nn.l2_loss(mat_diff) 71 | tf.summary.scalar('train/mat loss', mat_diff_loss, collections=['train']) 72 | 73 | return classify_loss + mat_diff_loss * reg_weight 74 | -------------------------------------------------------------------------------- /prepare_data/README.md: -------------------------------------------------------------------------------- 1 | This directory contains code that generates partial point clouds from [ModelNet](https://modelnet.cs.princeton.edu) models. To use it: 2 | 1. Install [Blender](https://blender.org/download) and [Blender OFF Addon](https://github.com/alextsui05/blender-off-addon). 3 | 2. Download aligned [ModelNet40](https://lmb.informatik.uni-freiburg.de/resources/datasets/ORION/modelnet40_manually_aligned.tar). 4 | 3. Create a list of model IDs. Each line of the model list should have the format `airplane_0001`. 5 | 4. Run `python parallel_render.py [model list] [intrinsics file] [output directory]` to render depth scan trajectories. The depth scans will be stored in OpenEXR format. Use `-n` to control the number of scan trajectories to generate for each model and `-p` to control the number of parallel threads. 6 | 5. Run `python traj2pcd.py [model list] [intrinsics file] [depth trajectory directory] [output directory] [number of trajectories per model]` to fuse the depth scans into partial point clouds in the initial camera's coordinate frame. 7 | 6. There are a couple of parameters such as distance from the camera to the object center and number of scans per trajectories in `render_single.py` that can be adjusted. 8 | -------------------------------------------------------------------------------- /prepare_data/intrinsics.txt: -------------------------------------------------------------------------------- 1 | 200 0 160 2 | 0 200 120 3 | 0 0 1 -------------------------------------------------------------------------------- /prepare_data/normalize_modelnet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import numpy as np 26 | import os 27 | 28 | 29 | def read_off(filepath): 30 | with open(filepath) as file: 31 | if 'OFF' != file.readline().strip(): 32 | raise('Not a valid OFF header') 33 | n_verts, n_faces, _ = [int(s) for s in file.readline().strip().split(' ')] 34 | verts = [[float(s) for s in file.readline().strip().split(' ')] for i in range(n_verts)] 35 | faces = [[int(s) for s in file.readline().strip().split(' ')][1:] for i in range(n_faces)] 36 | return np.array(verts), np.array(faces) 37 | 38 | 39 | def write_off(filepath, verts, faces): 40 | with open(filepath, 'w') as file: 41 | file.write('OFF\n') 42 | file.write('%d %d 0\n' % (verts.shape[0], faces.shape[0])) 43 | for vert in verts: 44 | file.write('%f %f %f\n' % tuple(vert)) 45 | for face in faces: 46 | file.write('3 %d %d %d\n' % tuple(face)) 47 | 48 | 49 | input_dir = '/usr0/home/Datasets/modelnet40_aligned' 50 | output_dir = '/usr0/home/Datasets/modelnet40_normalized' 51 | mode = 'train' 52 | with open('%s/%s.txt' % (input_dir, mode)) as file: 53 | model_list = file.read().splitlines() 54 | 55 | for model_id in model_list: 56 | cat = model_id.rsplit('_', 1)[0] 57 | verts, faces = read_off('%s/%s/%s/%s.off' % (input_dir, cat, mode, model_id)) 58 | verts -= (verts.max(0) + verts.min(0)) / 2 59 | verts /= np.linalg.norm(verts, axis=1).max() 60 | os.makedirs('%s/%s' % (output_dir, cat), exist_ok=True) 61 | write_off('%s/%s/%s.off' % (output_dir, cat, model_id), verts, faces) 62 | -------------------------------------------------------------------------------- /prepare_data/parallel_render.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import os 27 | import subprocess 28 | from functools import partial 29 | from multiprocessing.dummy import Pool 30 | from termcolor import colored 31 | 32 | 33 | if __name__ == '__main__': 34 | parser = argparse.ArgumentParser() 35 | parser.add_argument('list_path') 36 | parser.add_argument('intrinsics') 37 | parser.add_argument('output_dir') 38 | parser.add_argument('-n', type=int, default=1, help='number of trajectories') 39 | parser.add_argument('-p', type=int, default=8, help='number of processes') 40 | args = parser.parse_args() 41 | 42 | with open(os.path.join(args.list_path)) as file: 43 | model_list = [line.strip() for line in file] 44 | 45 | commands = [['/opt/blender/blender', '-b', '-P', 'render_single.py', 46 | model_id, args.intrinsics, args.output_dir, '%d' % args.n] 47 | for model_id in model_list] 48 | 49 | pool = Pool(args.p) 50 | print(colored('=== Rendering %d models on %d workers...' % (len(commands), args.p), 'white', 'on_blue')) 51 | for idx, completed in enumerate(pool.imap(partial(subprocess.run), commands)): 52 | pass 53 | -------------------------------------------------------------------------------- /prepare_data/render_single.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import bpy 26 | import addon_utils 27 | from mathutils import Matrix 28 | import numpy as np 29 | import os 30 | import sys 31 | import time 32 | addon_utils.enable('import_off') 33 | 34 | 35 | def setup_blender(width, height, focal_length): 36 | # camera 37 | camera = bpy.data.objects['Camera'] 38 | camera.data.sensor_height = camera.data.sensor_width 39 | camera.data.angle = np.arctan(width / 2 / focal_length) * 2 40 | 41 | # render layer 42 | scene = bpy.context.scene 43 | scene.render.alpha_mode = 'TRANSPARENT' 44 | scene.render.image_settings.color_depth = '16' 45 | scene.render.image_settings.use_zbuffer = True 46 | scene.render.resolution_x = width 47 | scene.render.resolution_y = height 48 | scene.render.resolution_percentage = 100 49 | 50 | # compositor nodes 51 | scene.use_nodes = True 52 | tree = scene.node_tree 53 | for n in tree.nodes: 54 | tree.nodes.remove(n) 55 | rl = tree.nodes.new('CompositorNodeRLayers') 56 | output = tree.nodes.new('CompositorNodeOutputFile') 57 | output.format.file_format = 'OPEN_EXR' 58 | tree.links.new(rl.outputs['Depth'], output.inputs[0]) 59 | 60 | # remove default cube 61 | bpy.data.objects['Cube'].select = True 62 | bpy.ops.object.delete() 63 | 64 | return scene, camera, output 65 | 66 | 67 | if __name__ == '__main__': 68 | model_dir = sys.argv[-5] 69 | model_id = sys.argv[-4] 70 | intrinsics_file = sys.argv[-3] 71 | output_dir = sys.argv[-2] 72 | num_trajs = int(sys.argv[-1]) 73 | 74 | max_num_scans = 4 75 | min_dist = 2 76 | max_dist = 4 77 | 78 | intrinsics = np.loadtxt(intrinsics_file) 79 | focal = intrinsics[0, 0] 80 | width = int(intrinsics[0, 2] * 2) 81 | height = int(intrinsics[1, 2] * 2) 82 | scene, camera, output = setup_blender(width, height, focal) 83 | 84 | category = model_id.rsplit('_', 1)[0] 85 | output_dir = os.path.join(output_dir, category, model_id) 86 | os.makedirs(output_dir, exist_ok=True) 87 | output.base_path = output_dir 88 | scene.render.filepath = os.path.join(output_dir, 'buffer.png') 89 | log_path = os.path.join(output_dir, 'blender_render.log') 90 | 91 | # Redirect output to log file 92 | open(log_path, 'a').close() 93 | old = os.dup(1) 94 | os.close(1) 95 | os.open(log_path, os.O_WRONLY) 96 | 97 | start = time.time() 98 | model_path = os.path.join(model_dir, category, '%s.off' % model_id) 99 | bpy.ops.import_mesh.off(filepath=model_path) 100 | 101 | num_scans = np.random.randint(1, max_num_scans+1, num_trajs) 102 | np.savetxt(os.path.join(output_dir, 'num_scans.txt'), np.array(num_scans), '%d') 103 | 104 | for i in range(num_trajs): 105 | traj_dir = os.path.join(output_dir, '%d' % i) 106 | os.makedirs(traj_dir, exist_ok=True) 107 | output.file_slots[0].path = os.path.join('%d/#.exr' % i) 108 | 109 | axis = np.random.normal(0, 1, 3) 110 | axis /= np.linalg.norm(axis) 111 | angle = np.random.rand() * np.pi 112 | rot = Matrix.Rotation(angle, 4, axis) 113 | 114 | dist = np.random.uniform(min_dist, max_dist) 115 | R = np.array(rot) 116 | trans = Matrix.Translation(R[:3, 2] * dist) 117 | 118 | for j in range(num_scans[i]): 119 | camera.matrix_world = trans * rot 120 | np.savetxt(os.path.join(traj_dir, '%d.txt' % j), 121 | np.array(camera.matrix_world), '%.20f') 122 | 123 | scene.frame_set(j) 124 | bpy.ops.render.render(write_still=True) 125 | 126 | axis = np.random.normal(0, 1, 3) 127 | axis /= np.linalg.norm(axis) 128 | angle = np.random.rand() * np.pi / 6 129 | rot *= Matrix.Rotation(angle, 4, axis) 130 | R = np.array(rot) 131 | trans = Matrix.Translation(R[:3, 2] * dist) 132 | 133 | # clean up 134 | bpy.ops.object.delete() 135 | 136 | # show time 137 | os.close(1) 138 | os.dup(old) 139 | os.close(old) 140 | print('{1} done, time={0:.4f} sec'.format(time.time() - start, model_id)) 141 | -------------------------------------------------------------------------------- /prepare_data/test_models.txt: -------------------------------------------------------------------------------- 1 | airplane_0627 2 | airplane_0628 3 | airplane_0629 4 | airplane_0630 5 | airplane_0631 6 | airplane_0632 7 | airplane_0633 8 | airplane_0634 9 | airplane_0635 10 | airplane_0636 11 | airplane_0637 12 | airplane_0638 13 | airplane_0639 14 | airplane_0640 15 | airplane_0641 16 | airplane_0642 17 | airplane_0643 18 | airplane_0644 19 | airplane_0645 20 | airplane_0646 21 | airplane_0647 22 | airplane_0648 23 | airplane_0649 24 | airplane_0650 25 | airplane_0651 26 | airplane_0652 27 | airplane_0653 28 | airplane_0654 29 | airplane_0655 30 | airplane_0656 31 | airplane_0657 32 | airplane_0658 33 | airplane_0659 34 | airplane_0660 35 | airplane_0661 36 | airplane_0662 37 | airplane_0663 38 | airplane_0664 39 | airplane_0665 40 | airplane_0666 41 | airplane_0667 42 | airplane_0668 43 | airplane_0669 44 | airplane_0670 45 | airplane_0671 46 | airplane_0672 47 | airplane_0673 48 | airplane_0674 49 | airplane_0675 50 | airplane_0676 51 | airplane_0677 52 | airplane_0678 53 | airplane_0679 54 | airplane_0680 55 | airplane_0681 56 | airplane_0682 57 | airplane_0683 58 | airplane_0684 59 | airplane_0685 60 | airplane_0686 61 | airplane_0687 62 | airplane_0688 63 | airplane_0689 64 | airplane_0690 65 | airplane_0691 66 | airplane_0692 67 | airplane_0693 68 | airplane_0694 69 | airplane_0695 70 | airplane_0696 71 | airplane_0697 72 | airplane_0698 73 | airplane_0699 74 | airplane_0700 75 | airplane_0701 76 | airplane_0702 77 | airplane_0703 78 | airplane_0704 79 | airplane_0705 80 | airplane_0706 81 | airplane_0707 82 | airplane_0708 83 | airplane_0709 84 | airplane_0710 85 | airplane_0711 86 | airplane_0712 87 | airplane_0713 88 | airplane_0714 89 | airplane_0715 90 | airplane_0716 91 | airplane_0717 92 | airplane_0718 93 | airplane_0719 94 | airplane_0720 95 | airplane_0721 96 | airplane_0722 97 | airplane_0723 98 | airplane_0724 99 | airplane_0725 100 | airplane_0726 101 | bathtub_0107 102 | bathtub_0108 103 | bathtub_0109 104 | bathtub_0110 105 | bathtub_0111 106 | bathtub_0112 107 | bathtub_0113 108 | bathtub_0114 109 | bathtub_0115 110 | bathtub_0116 111 | bathtub_0117 112 | bathtub_0118 113 | bathtub_0119 114 | bathtub_0120 115 | bathtub_0121 116 | bathtub_0122 117 | bathtub_0123 118 | bathtub_0124 119 | bathtub_0125 120 | bathtub_0126 121 | bathtub_0127 122 | bathtub_0128 123 | bathtub_0129 124 | bathtub_0130 125 | bathtub_0131 126 | bathtub_0132 127 | bathtub_0133 128 | bathtub_0134 129 | bathtub_0135 130 | bathtub_0136 131 | bathtub_0137 132 | bathtub_0138 133 | bathtub_0139 134 | bathtub_0140 135 | bathtub_0141 136 | bathtub_0142 137 | bathtub_0143 138 | bathtub_0144 139 | bathtub_0145 140 | bathtub_0146 141 | bathtub_0147 142 | bathtub_0148 143 | bathtub_0149 144 | bathtub_0150 145 | bathtub_0151 146 | bathtub_0152 147 | bathtub_0153 148 | bathtub_0154 149 | bathtub_0155 150 | bathtub_0156 151 | bed_0516 152 | bed_0517 153 | bed_0518 154 | bed_0519 155 | bed_0520 156 | bed_0521 157 | bed_0522 158 | bed_0523 159 | bed_0524 160 | bed_0525 161 | bed_0526 162 | bed_0527 163 | bed_0528 164 | bed_0529 165 | bed_0530 166 | bed_0531 167 | bed_0532 168 | bed_0533 169 | bed_0534 170 | bed_0535 171 | bed_0536 172 | bed_0537 173 | bed_0538 174 | bed_0539 175 | bed_0540 176 | bed_0541 177 | bed_0542 178 | bed_0543 179 | bed_0544 180 | bed_0545 181 | bed_0546 182 | bed_0547 183 | bed_0548 184 | bed_0549 185 | bed_0550 186 | bed_0551 187 | bed_0552 188 | bed_0553 189 | bed_0554 190 | bed_0555 191 | bed_0556 192 | bed_0557 193 | bed_0558 194 | bed_0559 195 | bed_0560 196 | bed_0561 197 | bed_0562 198 | bed_0563 199 | bed_0564 200 | bed_0565 201 | bed_0566 202 | bed_0567 203 | bed_0568 204 | bed_0569 205 | bed_0570 206 | bed_0571 207 | bed_0572 208 | bed_0573 209 | bed_0574 210 | bed_0575 211 | bed_0576 212 | bed_0577 213 | bed_0578 214 | bed_0579 215 | bed_0580 216 | bed_0581 217 | bed_0582 218 | bed_0583 219 | bed_0584 220 | bed_0585 221 | bed_0586 222 | bed_0587 223 | bed_0588 224 | bed_0589 225 | bed_0590 226 | bed_0591 227 | bed_0592 228 | bed_0593 229 | bed_0594 230 | bed_0595 231 | bed_0596 232 | bed_0597 233 | bed_0598 234 | bed_0599 235 | bed_0600 236 | bed_0601 237 | bed_0602 238 | bed_0603 239 | bed_0604 240 | bed_0605 241 | bed_0606 242 | bed_0607 243 | bed_0608 244 | bed_0609 245 | bed_0610 246 | bed_0611 247 | bed_0612 248 | bed_0613 249 | bed_0614 250 | bed_0615 251 | bench_0174 252 | bench_0175 253 | bench_0176 254 | bench_0177 255 | bench_0178 256 | bench_0179 257 | bench_0180 258 | bench_0181 259 | bench_0182 260 | bench_0183 261 | bench_0184 262 | bench_0185 263 | bench_0186 264 | bench_0187 265 | bench_0188 266 | bench_0189 267 | bench_0190 268 | bench_0191 269 | bench_0192 270 | bench_0193 271 | bookshelf_0573 272 | bookshelf_0574 273 | bookshelf_0575 274 | bookshelf_0576 275 | bookshelf_0577 276 | bookshelf_0578 277 | bookshelf_0579 278 | bookshelf_0580 279 | bookshelf_0581 280 | bookshelf_0582 281 | bookshelf_0583 282 | bookshelf_0584 283 | bookshelf_0585 284 | bookshelf_0586 285 | bookshelf_0587 286 | bookshelf_0588 287 | bookshelf_0589 288 | bookshelf_0590 289 | bookshelf_0591 290 | bookshelf_0592 291 | bookshelf_0593 292 | bookshelf_0594 293 | bookshelf_0595 294 | bookshelf_0596 295 | bookshelf_0597 296 | bookshelf_0598 297 | bookshelf_0599 298 | bookshelf_0600 299 | bookshelf_0601 300 | bookshelf_0602 301 | bookshelf_0603 302 | bookshelf_0604 303 | bookshelf_0605 304 | bookshelf_0606 305 | bookshelf_0607 306 | bookshelf_0608 307 | bookshelf_0609 308 | bookshelf_0610 309 | bookshelf_0611 310 | bookshelf_0612 311 | bookshelf_0613 312 | bookshelf_0614 313 | bookshelf_0615 314 | bookshelf_0616 315 | bookshelf_0617 316 | bookshelf_0618 317 | bookshelf_0619 318 | bookshelf_0620 319 | bookshelf_0621 320 | bookshelf_0622 321 | bookshelf_0623 322 | bookshelf_0624 323 | bookshelf_0625 324 | bookshelf_0626 325 | bookshelf_0627 326 | bookshelf_0628 327 | bookshelf_0629 328 | bookshelf_0630 329 | bookshelf_0631 330 | bookshelf_0632 331 | bookshelf_0633 332 | bookshelf_0634 333 | bookshelf_0635 334 | bookshelf_0636 335 | bookshelf_0637 336 | bookshelf_0638 337 | bookshelf_0639 338 | bookshelf_0640 339 | bookshelf_0641 340 | bookshelf_0642 341 | bookshelf_0643 342 | bookshelf_0644 343 | bookshelf_0645 344 | bookshelf_0646 345 | bookshelf_0647 346 | bookshelf_0648 347 | bookshelf_0649 348 | bookshelf_0650 349 | bookshelf_0651 350 | bookshelf_0652 351 | bookshelf_0653 352 | bookshelf_0654 353 | bookshelf_0655 354 | bookshelf_0656 355 | bookshelf_0657 356 | bookshelf_0658 357 | bookshelf_0659 358 | bookshelf_0660 359 | bookshelf_0661 360 | bookshelf_0662 361 | bookshelf_0663 362 | bookshelf_0664 363 | bookshelf_0665 364 | bookshelf_0666 365 | bookshelf_0667 366 | bookshelf_0668 367 | bookshelf_0669 368 | bookshelf_0670 369 | bookshelf_0671 370 | bookshelf_0672 371 | bottle_0336 372 | bottle_0337 373 | bottle_0338 374 | bottle_0339 375 | bottle_0340 376 | bottle_0341 377 | bottle_0342 378 | bottle_0343 379 | bottle_0344 380 | bottle_0345 381 | bottle_0346 382 | bottle_0347 383 | bottle_0348 384 | bottle_0349 385 | bottle_0350 386 | bottle_0351 387 | bottle_0352 388 | bottle_0353 389 | bottle_0354 390 | bottle_0355 391 | bottle_0356 392 | bottle_0357 393 | bottle_0358 394 | bottle_0359 395 | bottle_0360 396 | bottle_0361 397 | bottle_0362 398 | bottle_0363 399 | bottle_0364 400 | bottle_0365 401 | bottle_0366 402 | bottle_0367 403 | bottle_0368 404 | bottle_0369 405 | bottle_0370 406 | bottle_0371 407 | bottle_0372 408 | bottle_0373 409 | bottle_0374 410 | bottle_0375 411 | bottle_0376 412 | bottle_0377 413 | bottle_0378 414 | bottle_0379 415 | bottle_0380 416 | bottle_0381 417 | bottle_0382 418 | bottle_0383 419 | bottle_0384 420 | bottle_0385 421 | bottle_0386 422 | bottle_0387 423 | bottle_0388 424 | bottle_0389 425 | bottle_0390 426 | bottle_0391 427 | bottle_0392 428 | bottle_0393 429 | bottle_0394 430 | bottle_0395 431 | bottle_0396 432 | bottle_0397 433 | bottle_0398 434 | bottle_0399 435 | bottle_0400 436 | bottle_0401 437 | bottle_0402 438 | bottle_0403 439 | bottle_0404 440 | bottle_0405 441 | bottle_0406 442 | bottle_0407 443 | bottle_0408 444 | bottle_0409 445 | bottle_0410 446 | bottle_0411 447 | bottle_0412 448 | bottle_0413 449 | bottle_0414 450 | bottle_0415 451 | bottle_0416 452 | bottle_0417 453 | bottle_0418 454 | bottle_0419 455 | bottle_0420 456 | bottle_0421 457 | bottle_0422 458 | bottle_0423 459 | bottle_0424 460 | bottle_0425 461 | bottle_0426 462 | bottle_0427 463 | bottle_0428 464 | bottle_0429 465 | bottle_0430 466 | bottle_0431 467 | bottle_0432 468 | bottle_0433 469 | bottle_0434 470 | bottle_0435 471 | bowl_0065 472 | bowl_0066 473 | bowl_0067 474 | bowl_0068 475 | bowl_0069 476 | bowl_0070 477 | bowl_0071 478 | bowl_0072 479 | bowl_0073 480 | bowl_0074 481 | bowl_0075 482 | bowl_0076 483 | bowl_0077 484 | bowl_0078 485 | bowl_0079 486 | bowl_0080 487 | bowl_0081 488 | bowl_0082 489 | bowl_0083 490 | bowl_0084 491 | car_0198 492 | car_0199 493 | car_0200 494 | car_0201 495 | car_0202 496 | car_0203 497 | car_0204 498 | car_0205 499 | car_0206 500 | car_0207 501 | car_0208 502 | car_0209 503 | car_0210 504 | car_0211 505 | car_0212 506 | car_0213 507 | car_0214 508 | car_0215 509 | car_0216 510 | car_0217 511 | car_0218 512 | car_0219 513 | car_0220 514 | car_0221 515 | car_0222 516 | car_0223 517 | car_0224 518 | car_0225 519 | car_0226 520 | car_0227 521 | car_0228 522 | car_0229 523 | car_0230 524 | car_0231 525 | car_0232 526 | car_0233 527 | car_0234 528 | car_0235 529 | car_0236 530 | car_0237 531 | car_0238 532 | car_0239 533 | car_0240 534 | car_0241 535 | car_0242 536 | car_0243 537 | car_0244 538 | car_0245 539 | car_0246 540 | car_0247 541 | car_0248 542 | car_0249 543 | car_0250 544 | car_0251 545 | car_0252 546 | car_0253 547 | car_0254 548 | car_0255 549 | car_0256 550 | car_0257 551 | car_0258 552 | car_0259 553 | car_0260 554 | car_0261 555 | car_0262 556 | car_0263 557 | car_0264 558 | car_0265 559 | car_0266 560 | car_0267 561 | car_0268 562 | car_0269 563 | car_0270 564 | car_0271 565 | car_0272 566 | car_0273 567 | car_0274 568 | car_0275 569 | car_0276 570 | car_0277 571 | car_0278 572 | car_0279 573 | car_0280 574 | car_0281 575 | car_0282 576 | car_0283 577 | car_0284 578 | car_0285 579 | car_0286 580 | car_0287 581 | car_0288 582 | car_0289 583 | car_0290 584 | car_0291 585 | car_0292 586 | car_0293 587 | car_0294 588 | car_0295 589 | car_0296 590 | car_0297 591 | chair_0890 592 | chair_0891 593 | chair_0892 594 | chair_0893 595 | chair_0894 596 | chair_0895 597 | chair_0896 598 | chair_0897 599 | chair_0898 600 | chair_0899 601 | chair_0900 602 | chair_0901 603 | chair_0902 604 | chair_0903 605 | chair_0904 606 | chair_0905 607 | chair_0906 608 | chair_0907 609 | chair_0908 610 | chair_0909 611 | chair_0910 612 | chair_0911 613 | chair_0912 614 | chair_0913 615 | chair_0914 616 | chair_0915 617 | chair_0916 618 | chair_0917 619 | chair_0918 620 | chair_0919 621 | chair_0920 622 | chair_0921 623 | chair_0922 624 | chair_0923 625 | chair_0924 626 | chair_0925 627 | chair_0926 628 | chair_0927 629 | chair_0928 630 | chair_0929 631 | chair_0930 632 | chair_0931 633 | chair_0932 634 | chair_0933 635 | chair_0934 636 | chair_0935 637 | chair_0936 638 | chair_0937 639 | chair_0938 640 | chair_0939 641 | chair_0940 642 | chair_0941 643 | chair_0942 644 | chair_0943 645 | chair_0944 646 | chair_0945 647 | chair_0946 648 | chair_0947 649 | chair_0948 650 | chair_0949 651 | chair_0950 652 | chair_0951 653 | chair_0952 654 | chair_0953 655 | chair_0954 656 | chair_0955 657 | chair_0956 658 | chair_0957 659 | chair_0958 660 | chair_0959 661 | chair_0960 662 | chair_0961 663 | chair_0962 664 | chair_0963 665 | chair_0964 666 | chair_0965 667 | chair_0966 668 | chair_0967 669 | chair_0968 670 | chair_0969 671 | chair_0970 672 | chair_0971 673 | chair_0972 674 | chair_0973 675 | chair_0974 676 | chair_0975 677 | chair_0976 678 | chair_0977 679 | chair_0978 680 | chair_0979 681 | chair_0980 682 | chair_0981 683 | chair_0982 684 | chair_0983 685 | chair_0984 686 | chair_0985 687 | chair_0986 688 | chair_0987 689 | chair_0988 690 | chair_0989 691 | cone_0168 692 | cone_0169 693 | cone_0170 694 | cone_0171 695 | cone_0172 696 | cone_0173 697 | cone_0174 698 | cone_0175 699 | cone_0176 700 | cone_0177 701 | cone_0178 702 | cone_0179 703 | cone_0180 704 | cone_0181 705 | cone_0182 706 | cone_0183 707 | cone_0184 708 | cone_0185 709 | cone_0186 710 | cone_0187 711 | cup_0080 712 | cup_0081 713 | cup_0082 714 | cup_0083 715 | cup_0084 716 | cup_0085 717 | cup_0086 718 | cup_0087 719 | cup_0088 720 | cup_0089 721 | cup_0090 722 | cup_0091 723 | cup_0092 724 | cup_0093 725 | cup_0094 726 | cup_0095 727 | cup_0096 728 | cup_0097 729 | cup_0098 730 | cup_0099 731 | curtain_0139 732 | curtain_0140 733 | curtain_0141 734 | curtain_0142 735 | curtain_0143 736 | curtain_0144 737 | curtain_0145 738 | curtain_0146 739 | curtain_0147 740 | curtain_0148 741 | curtain_0149 742 | curtain_0150 743 | curtain_0151 744 | curtain_0152 745 | curtain_0153 746 | curtain_0154 747 | curtain_0155 748 | curtain_0156 749 | curtain_0157 750 | curtain_0158 751 | desk_0201 752 | desk_0202 753 | desk_0203 754 | desk_0204 755 | desk_0205 756 | desk_0206 757 | desk_0207 758 | desk_0208 759 | desk_0209 760 | desk_0210 761 | desk_0211 762 | desk_0212 763 | desk_0213 764 | desk_0214 765 | desk_0215 766 | desk_0216 767 | desk_0217 768 | desk_0218 769 | desk_0219 770 | desk_0220 771 | desk_0221 772 | desk_0222 773 | desk_0223 774 | desk_0224 775 | desk_0225 776 | desk_0226 777 | desk_0227 778 | desk_0228 779 | desk_0229 780 | desk_0230 781 | desk_0231 782 | desk_0232 783 | desk_0233 784 | desk_0234 785 | desk_0235 786 | desk_0236 787 | desk_0237 788 | desk_0238 789 | desk_0239 790 | desk_0240 791 | desk_0241 792 | desk_0242 793 | desk_0243 794 | desk_0244 795 | desk_0245 796 | desk_0246 797 | desk_0247 798 | desk_0248 799 | desk_0249 800 | desk_0250 801 | desk_0251 802 | desk_0252 803 | desk_0253 804 | desk_0254 805 | desk_0255 806 | desk_0256 807 | desk_0257 808 | desk_0258 809 | desk_0259 810 | desk_0260 811 | desk_0261 812 | desk_0262 813 | desk_0263 814 | desk_0264 815 | desk_0265 816 | desk_0266 817 | desk_0267 818 | desk_0268 819 | desk_0269 820 | desk_0270 821 | desk_0271 822 | desk_0272 823 | desk_0273 824 | desk_0274 825 | desk_0275 826 | desk_0276 827 | desk_0277 828 | desk_0278 829 | desk_0279 830 | desk_0280 831 | desk_0281 832 | desk_0282 833 | desk_0283 834 | desk_0284 835 | desk_0285 836 | desk_0286 837 | door_0110 838 | door_0111 839 | door_0112 840 | door_0113 841 | door_0114 842 | door_0115 843 | door_0116 844 | door_0117 845 | door_0118 846 | door_0119 847 | door_0120 848 | door_0121 849 | door_0122 850 | door_0123 851 | door_0124 852 | door_0125 853 | door_0126 854 | door_0127 855 | door_0128 856 | door_0129 857 | dresser_0201 858 | dresser_0202 859 | dresser_0203 860 | dresser_0204 861 | dresser_0205 862 | dresser_0206 863 | dresser_0207 864 | dresser_0208 865 | dresser_0209 866 | dresser_0210 867 | dresser_0211 868 | dresser_0212 869 | dresser_0213 870 | dresser_0214 871 | dresser_0215 872 | dresser_0216 873 | dresser_0217 874 | dresser_0218 875 | dresser_0219 876 | dresser_0220 877 | dresser_0221 878 | dresser_0222 879 | dresser_0223 880 | dresser_0224 881 | dresser_0225 882 | dresser_0226 883 | dresser_0227 884 | dresser_0228 885 | dresser_0229 886 | dresser_0230 887 | dresser_0231 888 | dresser_0232 889 | dresser_0233 890 | dresser_0234 891 | dresser_0235 892 | dresser_0236 893 | dresser_0237 894 | dresser_0238 895 | dresser_0239 896 | dresser_0240 897 | dresser_0241 898 | dresser_0242 899 | dresser_0243 900 | dresser_0244 901 | dresser_0245 902 | dresser_0246 903 | dresser_0247 904 | dresser_0248 905 | dresser_0249 906 | dresser_0250 907 | dresser_0251 908 | dresser_0252 909 | dresser_0253 910 | dresser_0254 911 | dresser_0255 912 | dresser_0256 913 | dresser_0257 914 | dresser_0258 915 | dresser_0259 916 | dresser_0260 917 | dresser_0261 918 | dresser_0262 919 | dresser_0263 920 | dresser_0264 921 | dresser_0265 922 | dresser_0266 923 | dresser_0267 924 | dresser_0268 925 | dresser_0269 926 | dresser_0270 927 | dresser_0271 928 | dresser_0272 929 | dresser_0273 930 | dresser_0274 931 | dresser_0275 932 | dresser_0276 933 | dresser_0277 934 | dresser_0278 935 | dresser_0279 936 | dresser_0280 937 | dresser_0281 938 | dresser_0282 939 | dresser_0283 940 | dresser_0284 941 | dresser_0285 942 | dresser_0286 943 | flower_pot_0150 944 | flower_pot_0151 945 | flower_pot_0152 946 | flower_pot_0153 947 | flower_pot_0154 948 | flower_pot_0155 949 | flower_pot_0156 950 | flower_pot_0157 951 | flower_pot_0158 952 | flower_pot_0159 953 | flower_pot_0160 954 | flower_pot_0161 955 | flower_pot_0162 956 | flower_pot_0163 957 | flower_pot_0164 958 | flower_pot_0165 959 | flower_pot_0166 960 | flower_pot_0167 961 | flower_pot_0168 962 | flower_pot_0169 963 | glass_box_0172 964 | glass_box_0173 965 | glass_box_0174 966 | glass_box_0175 967 | glass_box_0176 968 | glass_box_0177 969 | glass_box_0178 970 | glass_box_0179 971 | glass_box_0180 972 | glass_box_0181 973 | glass_box_0182 974 | glass_box_0183 975 | glass_box_0184 976 | glass_box_0185 977 | glass_box_0186 978 | glass_box_0187 979 | glass_box_0188 980 | glass_box_0189 981 | glass_box_0190 982 | glass_box_0191 983 | glass_box_0192 984 | glass_box_0193 985 | glass_box_0194 986 | glass_box_0195 987 | glass_box_0196 988 | glass_box_0197 989 | glass_box_0198 990 | glass_box_0199 991 | glass_box_0200 992 | glass_box_0201 993 | glass_box_0202 994 | glass_box_0203 995 | glass_box_0204 996 | glass_box_0205 997 | glass_box_0206 998 | glass_box_0207 999 | glass_box_0208 1000 | glass_box_0209 1001 | glass_box_0210 1002 | glass_box_0211 1003 | glass_box_0212 1004 | glass_box_0213 1005 | glass_box_0214 1006 | glass_box_0215 1007 | glass_box_0216 1008 | glass_box_0217 1009 | glass_box_0218 1010 | glass_box_0219 1011 | glass_box_0220 1012 | glass_box_0221 1013 | glass_box_0222 1014 | glass_box_0223 1015 | glass_box_0224 1016 | glass_box_0225 1017 | glass_box_0226 1018 | glass_box_0227 1019 | glass_box_0228 1020 | glass_box_0229 1021 | glass_box_0230 1022 | glass_box_0231 1023 | glass_box_0232 1024 | glass_box_0233 1025 | glass_box_0234 1026 | glass_box_0235 1027 | glass_box_0236 1028 | glass_box_0237 1029 | glass_box_0238 1030 | glass_box_0239 1031 | glass_box_0240 1032 | glass_box_0241 1033 | glass_box_0242 1034 | glass_box_0243 1035 | glass_box_0244 1036 | glass_box_0245 1037 | glass_box_0246 1038 | glass_box_0247 1039 | glass_box_0248 1040 | glass_box_0249 1041 | glass_box_0250 1042 | glass_box_0251 1043 | glass_box_0252 1044 | glass_box_0253 1045 | glass_box_0254 1046 | glass_box_0255 1047 | glass_box_0256 1048 | glass_box_0257 1049 | glass_box_0258 1050 | glass_box_0259 1051 | glass_box_0260 1052 | glass_box_0261 1053 | glass_box_0262 1054 | glass_box_0263 1055 | glass_box_0264 1056 | glass_box_0265 1057 | glass_box_0266 1058 | glass_box_0267 1059 | glass_box_0268 1060 | glass_box_0269 1061 | glass_box_0270 1062 | glass_box_0271 1063 | guitar_0156 1064 | guitar_0157 1065 | guitar_0158 1066 | guitar_0159 1067 | guitar_0160 1068 | guitar_0161 1069 | guitar_0162 1070 | guitar_0163 1071 | guitar_0164 1072 | guitar_0165 1073 | guitar_0166 1074 | guitar_0167 1075 | guitar_0168 1076 | guitar_0169 1077 | guitar_0170 1078 | guitar_0171 1079 | guitar_0172 1080 | guitar_0173 1081 | guitar_0174 1082 | guitar_0175 1083 | guitar_0176 1084 | guitar_0177 1085 | guitar_0178 1086 | guitar_0179 1087 | guitar_0180 1088 | guitar_0181 1089 | guitar_0182 1090 | guitar_0183 1091 | guitar_0184 1092 | guitar_0185 1093 | guitar_0186 1094 | guitar_0187 1095 | guitar_0188 1096 | guitar_0189 1097 | guitar_0190 1098 | guitar_0191 1099 | guitar_0192 1100 | guitar_0193 1101 | guitar_0194 1102 | guitar_0195 1103 | guitar_0196 1104 | guitar_0197 1105 | guitar_0198 1106 | guitar_0199 1107 | guitar_0200 1108 | guitar_0201 1109 | guitar_0202 1110 | guitar_0203 1111 | guitar_0204 1112 | guitar_0205 1113 | guitar_0206 1114 | guitar_0207 1115 | guitar_0208 1116 | guitar_0209 1117 | guitar_0210 1118 | guitar_0211 1119 | guitar_0212 1120 | guitar_0213 1121 | guitar_0214 1122 | guitar_0215 1123 | guitar_0216 1124 | guitar_0217 1125 | guitar_0218 1126 | guitar_0219 1127 | guitar_0220 1128 | guitar_0221 1129 | guitar_0222 1130 | guitar_0223 1131 | guitar_0224 1132 | guitar_0225 1133 | guitar_0226 1134 | guitar_0227 1135 | guitar_0228 1136 | guitar_0229 1137 | guitar_0230 1138 | guitar_0231 1139 | guitar_0232 1140 | guitar_0233 1141 | guitar_0234 1142 | guitar_0235 1143 | guitar_0236 1144 | guitar_0237 1145 | guitar_0238 1146 | guitar_0239 1147 | guitar_0240 1148 | guitar_0241 1149 | guitar_0242 1150 | guitar_0243 1151 | guitar_0244 1152 | guitar_0245 1153 | guitar_0246 1154 | guitar_0247 1155 | guitar_0248 1156 | guitar_0249 1157 | guitar_0250 1158 | guitar_0251 1159 | guitar_0252 1160 | guitar_0253 1161 | guitar_0254 1162 | guitar_0255 1163 | keyboard_0146 1164 | keyboard_0147 1165 | keyboard_0148 1166 | keyboard_0149 1167 | keyboard_0150 1168 | keyboard_0151 1169 | keyboard_0152 1170 | keyboard_0153 1171 | keyboard_0154 1172 | keyboard_0155 1173 | keyboard_0156 1174 | keyboard_0157 1175 | keyboard_0158 1176 | keyboard_0159 1177 | keyboard_0160 1178 | keyboard_0161 1179 | keyboard_0162 1180 | keyboard_0163 1181 | keyboard_0164 1182 | keyboard_0165 1183 | lamp_0125 1184 | lamp_0126 1185 | lamp_0127 1186 | lamp_0128 1187 | lamp_0129 1188 | lamp_0130 1189 | lamp_0131 1190 | lamp_0132 1191 | lamp_0133 1192 | lamp_0134 1193 | lamp_0135 1194 | lamp_0136 1195 | lamp_0137 1196 | lamp_0138 1197 | lamp_0139 1198 | lamp_0140 1199 | lamp_0141 1200 | lamp_0142 1201 | lamp_0143 1202 | lamp_0144 1203 | laptop_0150 1204 | laptop_0151 1205 | laptop_0152 1206 | laptop_0153 1207 | laptop_0154 1208 | laptop_0155 1209 | laptop_0156 1210 | laptop_0157 1211 | laptop_0158 1212 | laptop_0159 1213 | laptop_0160 1214 | laptop_0161 1215 | laptop_0162 1216 | laptop_0163 1217 | laptop_0164 1218 | laptop_0165 1219 | laptop_0166 1220 | laptop_0167 1221 | laptop_0168 1222 | laptop_0169 1223 | mantel_0285 1224 | mantel_0286 1225 | mantel_0287 1226 | mantel_0288 1227 | mantel_0289 1228 | mantel_0290 1229 | mantel_0291 1230 | mantel_0292 1231 | mantel_0293 1232 | mantel_0294 1233 | mantel_0295 1234 | mantel_0296 1235 | mantel_0297 1236 | mantel_0298 1237 | mantel_0299 1238 | mantel_0300 1239 | mantel_0301 1240 | mantel_0302 1241 | mantel_0303 1242 | mantel_0304 1243 | mantel_0305 1244 | mantel_0306 1245 | mantel_0307 1246 | mantel_0308 1247 | mantel_0309 1248 | mantel_0310 1249 | mantel_0311 1250 | mantel_0312 1251 | mantel_0313 1252 | mantel_0314 1253 | mantel_0315 1254 | mantel_0316 1255 | mantel_0317 1256 | mantel_0318 1257 | mantel_0319 1258 | mantel_0320 1259 | mantel_0321 1260 | mantel_0322 1261 | mantel_0323 1262 | mantel_0324 1263 | mantel_0325 1264 | mantel_0326 1265 | mantel_0327 1266 | mantel_0328 1267 | mantel_0329 1268 | mantel_0330 1269 | mantel_0331 1270 | mantel_0332 1271 | mantel_0333 1272 | mantel_0334 1273 | mantel_0335 1274 | mantel_0336 1275 | mantel_0337 1276 | mantel_0338 1277 | mantel_0339 1278 | mantel_0340 1279 | mantel_0341 1280 | mantel_0342 1281 | mantel_0343 1282 | mantel_0344 1283 | mantel_0345 1284 | mantel_0346 1285 | mantel_0347 1286 | mantel_0348 1287 | mantel_0349 1288 | mantel_0350 1289 | mantel_0351 1290 | mantel_0352 1291 | mantel_0353 1292 | mantel_0354 1293 | mantel_0355 1294 | mantel_0356 1295 | mantel_0357 1296 | mantel_0358 1297 | mantel_0359 1298 | mantel_0360 1299 | mantel_0361 1300 | mantel_0362 1301 | mantel_0363 1302 | mantel_0364 1303 | mantel_0365 1304 | mantel_0366 1305 | mantel_0367 1306 | mantel_0368 1307 | mantel_0369 1308 | mantel_0370 1309 | mantel_0371 1310 | mantel_0372 1311 | mantel_0373 1312 | mantel_0374 1313 | mantel_0375 1314 | mantel_0376 1315 | mantel_0377 1316 | mantel_0378 1317 | mantel_0379 1318 | mantel_0380 1319 | mantel_0381 1320 | mantel_0382 1321 | mantel_0383 1322 | mantel_0384 1323 | monitor_0466 1324 | monitor_0467 1325 | monitor_0468 1326 | monitor_0469 1327 | monitor_0470 1328 | monitor_0471 1329 | monitor_0472 1330 | monitor_0473 1331 | monitor_0474 1332 | monitor_0475 1333 | monitor_0476 1334 | monitor_0477 1335 | monitor_0478 1336 | monitor_0479 1337 | monitor_0480 1338 | monitor_0481 1339 | monitor_0482 1340 | monitor_0483 1341 | monitor_0484 1342 | monitor_0485 1343 | monitor_0486 1344 | monitor_0487 1345 | monitor_0488 1346 | monitor_0489 1347 | monitor_0490 1348 | monitor_0491 1349 | monitor_0492 1350 | monitor_0493 1351 | monitor_0494 1352 | monitor_0495 1353 | monitor_0496 1354 | monitor_0497 1355 | monitor_0498 1356 | monitor_0499 1357 | monitor_0500 1358 | monitor_0501 1359 | monitor_0502 1360 | monitor_0503 1361 | monitor_0504 1362 | monitor_0505 1363 | monitor_0506 1364 | monitor_0507 1365 | monitor_0508 1366 | monitor_0509 1367 | monitor_0510 1368 | monitor_0511 1369 | monitor_0512 1370 | monitor_0513 1371 | monitor_0514 1372 | monitor_0515 1373 | monitor_0516 1374 | monitor_0517 1375 | monitor_0518 1376 | monitor_0519 1377 | monitor_0520 1378 | monitor_0521 1379 | monitor_0522 1380 | monitor_0523 1381 | monitor_0524 1382 | monitor_0525 1383 | monitor_0526 1384 | monitor_0527 1385 | monitor_0528 1386 | monitor_0529 1387 | monitor_0530 1388 | monitor_0531 1389 | monitor_0532 1390 | monitor_0533 1391 | monitor_0534 1392 | monitor_0535 1393 | monitor_0536 1394 | monitor_0537 1395 | monitor_0538 1396 | monitor_0539 1397 | monitor_0540 1398 | monitor_0541 1399 | monitor_0542 1400 | monitor_0543 1401 | monitor_0544 1402 | monitor_0545 1403 | monitor_0546 1404 | monitor_0547 1405 | monitor_0548 1406 | monitor_0549 1407 | monitor_0550 1408 | monitor_0551 1409 | monitor_0552 1410 | monitor_0553 1411 | monitor_0554 1412 | monitor_0555 1413 | monitor_0556 1414 | monitor_0557 1415 | monitor_0558 1416 | monitor_0559 1417 | monitor_0560 1418 | monitor_0561 1419 | monitor_0562 1420 | monitor_0563 1421 | monitor_0564 1422 | monitor_0565 1423 | night_stand_0201 1424 | night_stand_0202 1425 | night_stand_0203 1426 | night_stand_0204 1427 | night_stand_0205 1428 | night_stand_0206 1429 | night_stand_0207 1430 | night_stand_0208 1431 | night_stand_0209 1432 | night_stand_0210 1433 | night_stand_0211 1434 | night_stand_0212 1435 | night_stand_0213 1436 | night_stand_0214 1437 | night_stand_0215 1438 | night_stand_0216 1439 | night_stand_0217 1440 | night_stand_0218 1441 | night_stand_0219 1442 | night_stand_0220 1443 | night_stand_0221 1444 | night_stand_0222 1445 | night_stand_0223 1446 | night_stand_0224 1447 | night_stand_0225 1448 | night_stand_0226 1449 | night_stand_0227 1450 | night_stand_0228 1451 | night_stand_0229 1452 | night_stand_0230 1453 | night_stand_0231 1454 | night_stand_0232 1455 | night_stand_0233 1456 | night_stand_0234 1457 | night_stand_0235 1458 | night_stand_0236 1459 | night_stand_0237 1460 | night_stand_0238 1461 | night_stand_0239 1462 | night_stand_0240 1463 | night_stand_0241 1464 | night_stand_0242 1465 | night_stand_0243 1466 | night_stand_0244 1467 | night_stand_0245 1468 | night_stand_0246 1469 | night_stand_0247 1470 | night_stand_0248 1471 | night_stand_0249 1472 | night_stand_0250 1473 | night_stand_0251 1474 | night_stand_0252 1475 | night_stand_0253 1476 | night_stand_0254 1477 | night_stand_0255 1478 | night_stand_0256 1479 | night_stand_0257 1480 | night_stand_0258 1481 | night_stand_0259 1482 | night_stand_0260 1483 | night_stand_0261 1484 | night_stand_0262 1485 | night_stand_0263 1486 | night_stand_0264 1487 | night_stand_0265 1488 | night_stand_0266 1489 | night_stand_0267 1490 | night_stand_0268 1491 | night_stand_0269 1492 | night_stand_0270 1493 | night_stand_0271 1494 | night_stand_0272 1495 | night_stand_0273 1496 | night_stand_0274 1497 | night_stand_0275 1498 | night_stand_0276 1499 | night_stand_0277 1500 | night_stand_0278 1501 | night_stand_0279 1502 | night_stand_0280 1503 | night_stand_0281 1504 | night_stand_0282 1505 | night_stand_0283 1506 | night_stand_0284 1507 | night_stand_0285 1508 | night_stand_0286 1509 | person_0089 1510 | person_0090 1511 | person_0091 1512 | person_0092 1513 | person_0093 1514 | person_0094 1515 | person_0095 1516 | person_0096 1517 | person_0097 1518 | person_0098 1519 | person_0099 1520 | person_0100 1521 | person_0101 1522 | person_0102 1523 | person_0103 1524 | person_0104 1525 | person_0105 1526 | person_0106 1527 | person_0107 1528 | person_0108 1529 | piano_0232 1530 | piano_0233 1531 | piano_0234 1532 | piano_0235 1533 | piano_0236 1534 | piano_0237 1535 | piano_0238 1536 | piano_0239 1537 | piano_0240 1538 | piano_0241 1539 | piano_0242 1540 | piano_0243 1541 | piano_0244 1542 | piano_0245 1543 | piano_0246 1544 | piano_0247 1545 | piano_0248 1546 | piano_0249 1547 | piano_0250 1548 | piano_0251 1549 | piano_0252 1550 | piano_0253 1551 | piano_0254 1552 | piano_0255 1553 | piano_0256 1554 | piano_0257 1555 | piano_0258 1556 | piano_0259 1557 | piano_0260 1558 | piano_0261 1559 | piano_0262 1560 | piano_0263 1561 | piano_0264 1562 | piano_0265 1563 | piano_0266 1564 | piano_0267 1565 | piano_0268 1566 | piano_0269 1567 | piano_0270 1568 | piano_0271 1569 | piano_0272 1570 | piano_0273 1571 | piano_0274 1572 | piano_0275 1573 | piano_0276 1574 | piano_0277 1575 | piano_0278 1576 | piano_0279 1577 | piano_0280 1578 | piano_0281 1579 | piano_0282 1580 | piano_0283 1581 | piano_0284 1582 | piano_0285 1583 | piano_0286 1584 | piano_0287 1585 | piano_0288 1586 | piano_0289 1587 | piano_0290 1588 | piano_0291 1589 | piano_0292 1590 | piano_0293 1591 | piano_0294 1592 | piano_0295 1593 | piano_0296 1594 | piano_0297 1595 | piano_0298 1596 | piano_0299 1597 | piano_0300 1598 | piano_0301 1599 | piano_0302 1600 | piano_0303 1601 | piano_0304 1602 | piano_0305 1603 | piano_0306 1604 | piano_0307 1605 | piano_0308 1606 | piano_0309 1607 | piano_0310 1608 | piano_0311 1609 | piano_0312 1610 | piano_0313 1611 | piano_0314 1612 | piano_0315 1613 | piano_0316 1614 | piano_0317 1615 | piano_0318 1616 | piano_0319 1617 | piano_0320 1618 | piano_0321 1619 | piano_0322 1620 | piano_0323 1621 | piano_0324 1622 | piano_0325 1623 | piano_0326 1624 | piano_0327 1625 | piano_0328 1626 | piano_0329 1627 | piano_0330 1628 | piano_0331 1629 | plant_0241 1630 | plant_0242 1631 | plant_0243 1632 | plant_0244 1633 | plant_0245 1634 | plant_0246 1635 | plant_0247 1636 | plant_0248 1637 | plant_0249 1638 | plant_0250 1639 | plant_0251 1640 | plant_0252 1641 | plant_0253 1642 | plant_0254 1643 | plant_0255 1644 | plant_0256 1645 | plant_0257 1646 | plant_0258 1647 | plant_0259 1648 | plant_0260 1649 | plant_0261 1650 | plant_0262 1651 | plant_0263 1652 | plant_0264 1653 | plant_0265 1654 | plant_0266 1655 | plant_0267 1656 | plant_0268 1657 | plant_0269 1658 | plant_0270 1659 | plant_0271 1660 | plant_0272 1661 | plant_0273 1662 | plant_0274 1663 | plant_0275 1664 | plant_0276 1665 | plant_0277 1666 | plant_0278 1667 | plant_0279 1668 | plant_0280 1669 | plant_0281 1670 | plant_0282 1671 | plant_0283 1672 | plant_0284 1673 | plant_0285 1674 | plant_0286 1675 | plant_0287 1676 | plant_0288 1677 | plant_0289 1678 | plant_0290 1679 | plant_0291 1680 | plant_0292 1681 | plant_0293 1682 | plant_0294 1683 | plant_0295 1684 | plant_0296 1685 | plant_0297 1686 | plant_0298 1687 | plant_0299 1688 | plant_0300 1689 | plant_0301 1690 | plant_0302 1691 | plant_0303 1692 | plant_0304 1693 | plant_0305 1694 | plant_0306 1695 | plant_0307 1696 | plant_0308 1697 | plant_0309 1698 | plant_0310 1699 | plant_0311 1700 | plant_0312 1701 | plant_0313 1702 | plant_0314 1703 | plant_0315 1704 | plant_0316 1705 | plant_0317 1706 | plant_0318 1707 | plant_0319 1708 | plant_0320 1709 | plant_0321 1710 | plant_0322 1711 | plant_0323 1712 | plant_0324 1713 | plant_0325 1714 | plant_0326 1715 | plant_0327 1716 | plant_0328 1717 | plant_0329 1718 | plant_0330 1719 | plant_0331 1720 | plant_0332 1721 | plant_0333 1722 | plant_0334 1723 | plant_0335 1724 | plant_0336 1725 | plant_0337 1726 | plant_0338 1727 | plant_0339 1728 | plant_0340 1729 | radio_0105 1730 | radio_0106 1731 | radio_0107 1732 | radio_0108 1733 | radio_0109 1734 | radio_0110 1735 | radio_0111 1736 | radio_0112 1737 | radio_0113 1738 | radio_0114 1739 | radio_0115 1740 | radio_0116 1741 | radio_0117 1742 | radio_0118 1743 | radio_0119 1744 | radio_0120 1745 | radio_0121 1746 | radio_0122 1747 | radio_0123 1748 | radio_0124 1749 | range_hood_0116 1750 | range_hood_0117 1751 | range_hood_0118 1752 | range_hood_0119 1753 | range_hood_0120 1754 | range_hood_0121 1755 | range_hood_0122 1756 | range_hood_0123 1757 | range_hood_0124 1758 | range_hood_0125 1759 | range_hood_0126 1760 | range_hood_0127 1761 | range_hood_0128 1762 | range_hood_0129 1763 | range_hood_0130 1764 | range_hood_0131 1765 | range_hood_0132 1766 | range_hood_0133 1767 | range_hood_0134 1768 | range_hood_0135 1769 | range_hood_0136 1770 | range_hood_0137 1771 | range_hood_0138 1772 | range_hood_0139 1773 | range_hood_0140 1774 | range_hood_0141 1775 | range_hood_0142 1776 | range_hood_0143 1777 | range_hood_0144 1778 | range_hood_0145 1779 | range_hood_0146 1780 | range_hood_0147 1781 | range_hood_0148 1782 | range_hood_0149 1783 | range_hood_0150 1784 | range_hood_0151 1785 | range_hood_0152 1786 | range_hood_0153 1787 | range_hood_0154 1788 | range_hood_0155 1789 | range_hood_0156 1790 | range_hood_0157 1791 | range_hood_0158 1792 | range_hood_0159 1793 | range_hood_0160 1794 | range_hood_0161 1795 | range_hood_0162 1796 | range_hood_0163 1797 | range_hood_0164 1798 | range_hood_0165 1799 | range_hood_0166 1800 | range_hood_0167 1801 | range_hood_0168 1802 | range_hood_0169 1803 | range_hood_0170 1804 | range_hood_0171 1805 | range_hood_0172 1806 | range_hood_0173 1807 | range_hood_0174 1808 | range_hood_0175 1809 | range_hood_0176 1810 | range_hood_0177 1811 | range_hood_0178 1812 | range_hood_0179 1813 | range_hood_0180 1814 | range_hood_0181 1815 | range_hood_0182 1816 | range_hood_0183 1817 | range_hood_0184 1818 | range_hood_0185 1819 | range_hood_0186 1820 | range_hood_0187 1821 | range_hood_0188 1822 | range_hood_0189 1823 | range_hood_0190 1824 | range_hood_0191 1825 | range_hood_0192 1826 | range_hood_0193 1827 | range_hood_0194 1828 | range_hood_0195 1829 | range_hood_0196 1830 | range_hood_0197 1831 | range_hood_0198 1832 | range_hood_0199 1833 | range_hood_0200 1834 | range_hood_0201 1835 | range_hood_0202 1836 | range_hood_0203 1837 | range_hood_0204 1838 | range_hood_0205 1839 | range_hood_0206 1840 | range_hood_0207 1841 | range_hood_0208 1842 | range_hood_0209 1843 | range_hood_0210 1844 | range_hood_0211 1845 | range_hood_0212 1846 | range_hood_0213 1847 | range_hood_0214 1848 | range_hood_0215 1849 | sink_0129 1850 | sink_0130 1851 | sink_0131 1852 | sink_0132 1853 | sink_0133 1854 | sink_0134 1855 | sink_0135 1856 | sink_0136 1857 | sink_0137 1858 | sink_0138 1859 | sink_0139 1860 | sink_0140 1861 | sink_0141 1862 | sink_0142 1863 | sink_0143 1864 | sink_0144 1865 | sink_0145 1866 | sink_0146 1867 | sink_0147 1868 | sink_0148 1869 | sofa_0681 1870 | sofa_0682 1871 | sofa_0683 1872 | sofa_0684 1873 | sofa_0685 1874 | sofa_0686 1875 | sofa_0687 1876 | sofa_0688 1877 | sofa_0689 1878 | sofa_0690 1879 | sofa_0691 1880 | sofa_0692 1881 | sofa_0693 1882 | sofa_0694 1883 | sofa_0695 1884 | sofa_0696 1885 | sofa_0697 1886 | sofa_0698 1887 | sofa_0699 1888 | sofa_0700 1889 | sofa_0701 1890 | sofa_0702 1891 | sofa_0703 1892 | sofa_0704 1893 | sofa_0705 1894 | sofa_0706 1895 | sofa_0707 1896 | sofa_0708 1897 | sofa_0709 1898 | sofa_0710 1899 | sofa_0711 1900 | sofa_0712 1901 | sofa_0713 1902 | sofa_0714 1903 | sofa_0715 1904 | sofa_0716 1905 | sofa_0717 1906 | sofa_0718 1907 | sofa_0719 1908 | sofa_0720 1909 | sofa_0721 1910 | sofa_0722 1911 | sofa_0723 1912 | sofa_0724 1913 | sofa_0725 1914 | sofa_0726 1915 | sofa_0727 1916 | sofa_0728 1917 | sofa_0729 1918 | sofa_0730 1919 | sofa_0731 1920 | sofa_0732 1921 | sofa_0733 1922 | sofa_0734 1923 | sofa_0735 1924 | sofa_0736 1925 | sofa_0737 1926 | sofa_0738 1927 | sofa_0739 1928 | sofa_0740 1929 | sofa_0741 1930 | sofa_0742 1931 | sofa_0743 1932 | sofa_0744 1933 | sofa_0745 1934 | sofa_0746 1935 | sofa_0747 1936 | sofa_0748 1937 | sofa_0749 1938 | sofa_0750 1939 | sofa_0751 1940 | sofa_0752 1941 | sofa_0753 1942 | sofa_0754 1943 | sofa_0755 1944 | sofa_0756 1945 | sofa_0757 1946 | sofa_0758 1947 | sofa_0759 1948 | sofa_0760 1949 | sofa_0761 1950 | sofa_0762 1951 | sofa_0763 1952 | sofa_0764 1953 | sofa_0765 1954 | sofa_0766 1955 | sofa_0767 1956 | sofa_0768 1957 | sofa_0769 1958 | sofa_0770 1959 | sofa_0771 1960 | sofa_0772 1961 | sofa_0773 1962 | sofa_0774 1963 | sofa_0775 1964 | sofa_0776 1965 | sofa_0777 1966 | sofa_0778 1967 | sofa_0779 1968 | sofa_0780 1969 | stairs_0125 1970 | stairs_0126 1971 | stairs_0127 1972 | stairs_0128 1973 | stairs_0129 1974 | stairs_0130 1975 | stairs_0131 1976 | stairs_0132 1977 | stairs_0133 1978 | stairs_0134 1979 | stairs_0135 1980 | stairs_0136 1981 | stairs_0137 1982 | stairs_0138 1983 | stairs_0139 1984 | stairs_0140 1985 | stairs_0141 1986 | stairs_0142 1987 | stairs_0143 1988 | stairs_0144 1989 | stool_0091 1990 | stool_0092 1991 | stool_0093 1992 | stool_0094 1993 | stool_0095 1994 | stool_0096 1995 | stool_0097 1996 | stool_0098 1997 | stool_0099 1998 | stool_0100 1999 | stool_0101 2000 | stool_0102 2001 | stool_0103 2002 | stool_0104 2003 | stool_0105 2004 | stool_0106 2005 | stool_0107 2006 | stool_0108 2007 | stool_0109 2008 | stool_0110 2009 | table_0393 2010 | table_0394 2011 | table_0395 2012 | table_0396 2013 | table_0397 2014 | table_0398 2015 | table_0399 2016 | table_0400 2017 | table_0401 2018 | table_0402 2019 | table_0403 2020 | table_0404 2021 | table_0405 2022 | table_0406 2023 | table_0407 2024 | table_0408 2025 | table_0409 2026 | table_0410 2027 | table_0411 2028 | table_0412 2029 | table_0413 2030 | table_0414 2031 | table_0415 2032 | table_0416 2033 | table_0417 2034 | table_0418 2035 | table_0419 2036 | table_0420 2037 | table_0421 2038 | table_0422 2039 | table_0423 2040 | table_0424 2041 | table_0425 2042 | table_0426 2043 | table_0427 2044 | table_0428 2045 | table_0429 2046 | table_0430 2047 | table_0431 2048 | table_0432 2049 | table_0433 2050 | table_0434 2051 | table_0435 2052 | table_0436 2053 | table_0437 2054 | table_0438 2055 | table_0439 2056 | table_0440 2057 | table_0441 2058 | table_0442 2059 | table_0443 2060 | table_0444 2061 | table_0445 2062 | table_0446 2063 | table_0447 2064 | table_0448 2065 | table_0449 2066 | table_0450 2067 | table_0451 2068 | table_0452 2069 | table_0453 2070 | table_0454 2071 | table_0455 2072 | table_0456 2073 | table_0457 2074 | table_0458 2075 | table_0459 2076 | table_0460 2077 | table_0461 2078 | table_0462 2079 | table_0463 2080 | table_0464 2081 | table_0465 2082 | table_0466 2083 | table_0467 2084 | table_0468 2085 | table_0469 2086 | table_0470 2087 | table_0471 2088 | table_0472 2089 | table_0473 2090 | table_0474 2091 | table_0475 2092 | table_0476 2093 | table_0477 2094 | table_0478 2095 | table_0479 2096 | table_0480 2097 | table_0481 2098 | table_0482 2099 | table_0483 2100 | table_0484 2101 | table_0485 2102 | table_0486 2103 | table_0487 2104 | table_0488 2105 | table_0489 2106 | table_0490 2107 | table_0491 2108 | table_0492 2109 | tent_0164 2110 | tent_0165 2111 | tent_0166 2112 | tent_0167 2113 | tent_0168 2114 | tent_0169 2115 | tent_0170 2116 | tent_0171 2117 | tent_0172 2118 | tent_0173 2119 | tent_0174 2120 | tent_0175 2121 | tent_0176 2122 | tent_0177 2123 | tent_0178 2124 | tent_0179 2125 | tent_0180 2126 | tent_0181 2127 | tent_0182 2128 | tent_0183 2129 | toilet_0345 2130 | toilet_0346 2131 | toilet_0347 2132 | toilet_0348 2133 | toilet_0349 2134 | toilet_0350 2135 | toilet_0351 2136 | toilet_0352 2137 | toilet_0353 2138 | toilet_0354 2139 | toilet_0355 2140 | toilet_0356 2141 | toilet_0357 2142 | toilet_0358 2143 | toilet_0359 2144 | toilet_0360 2145 | toilet_0361 2146 | toilet_0362 2147 | toilet_0363 2148 | toilet_0364 2149 | toilet_0365 2150 | toilet_0366 2151 | toilet_0367 2152 | toilet_0368 2153 | toilet_0369 2154 | toilet_0370 2155 | toilet_0371 2156 | toilet_0372 2157 | toilet_0373 2158 | toilet_0374 2159 | toilet_0375 2160 | toilet_0376 2161 | toilet_0377 2162 | toilet_0378 2163 | toilet_0379 2164 | toilet_0380 2165 | toilet_0381 2166 | toilet_0382 2167 | toilet_0383 2168 | toilet_0384 2169 | toilet_0385 2170 | toilet_0386 2171 | toilet_0387 2172 | toilet_0388 2173 | toilet_0389 2174 | toilet_0390 2175 | toilet_0391 2176 | toilet_0392 2177 | toilet_0393 2178 | toilet_0394 2179 | toilet_0395 2180 | toilet_0396 2181 | toilet_0397 2182 | toilet_0398 2183 | toilet_0399 2184 | toilet_0400 2185 | toilet_0401 2186 | toilet_0402 2187 | toilet_0403 2188 | toilet_0404 2189 | toilet_0405 2190 | toilet_0406 2191 | toilet_0407 2192 | toilet_0408 2193 | toilet_0409 2194 | toilet_0410 2195 | toilet_0411 2196 | toilet_0412 2197 | toilet_0413 2198 | toilet_0414 2199 | toilet_0415 2200 | toilet_0416 2201 | toilet_0417 2202 | toilet_0418 2203 | toilet_0419 2204 | toilet_0420 2205 | toilet_0421 2206 | toilet_0422 2207 | toilet_0423 2208 | toilet_0424 2209 | toilet_0425 2210 | toilet_0426 2211 | toilet_0427 2212 | toilet_0428 2213 | toilet_0429 2214 | toilet_0430 2215 | toilet_0431 2216 | toilet_0432 2217 | toilet_0433 2218 | toilet_0434 2219 | toilet_0435 2220 | toilet_0436 2221 | toilet_0437 2222 | toilet_0438 2223 | toilet_0439 2224 | toilet_0440 2225 | toilet_0441 2226 | toilet_0442 2227 | toilet_0443 2228 | toilet_0444 2229 | tv_stand_0268 2230 | tv_stand_0269 2231 | tv_stand_0270 2232 | tv_stand_0271 2233 | tv_stand_0272 2234 | tv_stand_0273 2235 | tv_stand_0274 2236 | tv_stand_0275 2237 | tv_stand_0276 2238 | tv_stand_0277 2239 | tv_stand_0278 2240 | tv_stand_0279 2241 | tv_stand_0280 2242 | tv_stand_0281 2243 | tv_stand_0282 2244 | tv_stand_0283 2245 | tv_stand_0284 2246 | tv_stand_0285 2247 | tv_stand_0286 2248 | tv_stand_0287 2249 | tv_stand_0288 2250 | tv_stand_0289 2251 | tv_stand_0290 2252 | tv_stand_0291 2253 | tv_stand_0292 2254 | tv_stand_0293 2255 | tv_stand_0294 2256 | tv_stand_0295 2257 | tv_stand_0296 2258 | tv_stand_0297 2259 | tv_stand_0298 2260 | tv_stand_0299 2261 | tv_stand_0300 2262 | tv_stand_0301 2263 | tv_stand_0302 2264 | tv_stand_0303 2265 | tv_stand_0304 2266 | tv_stand_0305 2267 | tv_stand_0306 2268 | tv_stand_0307 2269 | tv_stand_0308 2270 | tv_stand_0309 2271 | tv_stand_0310 2272 | tv_stand_0311 2273 | tv_stand_0312 2274 | tv_stand_0313 2275 | tv_stand_0314 2276 | tv_stand_0315 2277 | tv_stand_0316 2278 | tv_stand_0317 2279 | tv_stand_0318 2280 | tv_stand_0319 2281 | tv_stand_0320 2282 | tv_stand_0321 2283 | tv_stand_0322 2284 | tv_stand_0323 2285 | tv_stand_0324 2286 | tv_stand_0325 2287 | tv_stand_0326 2288 | tv_stand_0327 2289 | tv_stand_0328 2290 | tv_stand_0329 2291 | tv_stand_0330 2292 | tv_stand_0331 2293 | tv_stand_0332 2294 | tv_stand_0333 2295 | tv_stand_0334 2296 | tv_stand_0335 2297 | tv_stand_0336 2298 | tv_stand_0337 2299 | tv_stand_0338 2300 | tv_stand_0339 2301 | tv_stand_0340 2302 | tv_stand_0341 2303 | tv_stand_0342 2304 | tv_stand_0343 2305 | tv_stand_0344 2306 | tv_stand_0345 2307 | tv_stand_0346 2308 | tv_stand_0347 2309 | tv_stand_0348 2310 | tv_stand_0349 2311 | tv_stand_0350 2312 | tv_stand_0351 2313 | tv_stand_0352 2314 | tv_stand_0353 2315 | tv_stand_0354 2316 | tv_stand_0355 2317 | tv_stand_0356 2318 | tv_stand_0357 2319 | tv_stand_0358 2320 | tv_stand_0359 2321 | tv_stand_0360 2322 | tv_stand_0361 2323 | tv_stand_0362 2324 | tv_stand_0363 2325 | tv_stand_0364 2326 | tv_stand_0365 2327 | tv_stand_0366 2328 | tv_stand_0367 2329 | vase_0476 2330 | vase_0477 2331 | vase_0478 2332 | vase_0479 2333 | vase_0480 2334 | vase_0481 2335 | vase_0482 2336 | vase_0483 2337 | vase_0484 2338 | vase_0485 2339 | vase_0486 2340 | vase_0487 2341 | vase_0488 2342 | vase_0489 2343 | vase_0490 2344 | vase_0491 2345 | vase_0492 2346 | vase_0493 2347 | vase_0494 2348 | vase_0495 2349 | vase_0496 2350 | vase_0497 2351 | vase_0498 2352 | vase_0499 2353 | vase_0500 2354 | vase_0501 2355 | vase_0502 2356 | vase_0503 2357 | vase_0504 2358 | vase_0505 2359 | vase_0506 2360 | vase_0507 2361 | vase_0508 2362 | vase_0509 2363 | vase_0510 2364 | vase_0511 2365 | vase_0512 2366 | vase_0513 2367 | vase_0514 2368 | vase_0515 2369 | vase_0516 2370 | vase_0517 2371 | vase_0518 2372 | vase_0519 2373 | vase_0520 2374 | vase_0521 2375 | vase_0522 2376 | vase_0523 2377 | vase_0524 2378 | vase_0525 2379 | vase_0526 2380 | vase_0527 2381 | vase_0528 2382 | vase_0529 2383 | vase_0530 2384 | vase_0531 2385 | vase_0532 2386 | vase_0533 2387 | vase_0534 2388 | vase_0535 2389 | vase_0536 2390 | vase_0537 2391 | vase_0538 2392 | vase_0539 2393 | vase_0540 2394 | vase_0541 2395 | vase_0542 2396 | vase_0543 2397 | vase_0544 2398 | vase_0545 2399 | vase_0546 2400 | vase_0547 2401 | vase_0548 2402 | vase_0549 2403 | vase_0550 2404 | vase_0551 2405 | vase_0552 2406 | vase_0553 2407 | vase_0554 2408 | vase_0555 2409 | vase_0556 2410 | vase_0557 2411 | vase_0558 2412 | vase_0559 2413 | vase_0560 2414 | vase_0561 2415 | vase_0562 2416 | vase_0563 2417 | vase_0564 2418 | vase_0565 2419 | vase_0566 2420 | vase_0567 2421 | vase_0568 2422 | vase_0569 2423 | vase_0570 2424 | vase_0571 2425 | vase_0572 2426 | vase_0573 2427 | vase_0574 2428 | vase_0575 2429 | wardrobe_0088 2430 | wardrobe_0089 2431 | wardrobe_0090 2432 | wardrobe_0091 2433 | wardrobe_0092 2434 | wardrobe_0093 2435 | wardrobe_0094 2436 | wardrobe_0095 2437 | wardrobe_0096 2438 | wardrobe_0097 2439 | wardrobe_0098 2440 | wardrobe_0099 2441 | wardrobe_0100 2442 | wardrobe_0101 2443 | wardrobe_0102 2444 | wardrobe_0103 2445 | wardrobe_0104 2446 | wardrobe_0105 2447 | wardrobe_0106 2448 | wardrobe_0107 2449 | xbox_0104 2450 | xbox_0105 2451 | xbox_0106 2452 | xbox_0107 2453 | xbox_0108 2454 | xbox_0109 2455 | xbox_0110 2456 | xbox_0111 2457 | xbox_0112 2458 | xbox_0113 2459 | xbox_0114 2460 | xbox_0115 2461 | xbox_0116 2462 | xbox_0117 2463 | xbox_0118 2464 | xbox_0119 2465 | xbox_0120 2466 | xbox_0121 2467 | xbox_0122 2468 | xbox_0123 2469 | -------------------------------------------------------------------------------- /prepare_data/traj2pcd.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import Imath 26 | import OpenEXR 27 | import argparse 28 | import array 29 | import numpy as np 30 | import os 31 | import matplotlib.pyplot as plt 32 | from mpl_toolkits.mplot3d import Axes3D 33 | from open3d import * 34 | 35 | 36 | def read_exr(exr_path, width, height): 37 | file = OpenEXR.InputFile(exr_path) 38 | depth_arr = array.array('f', file.channel('R', Imath.PixelType(Imath.PixelType.FLOAT))) 39 | depth = np.array(depth_arr).reshape((height, width)) 40 | return depth 41 | 42 | 43 | def depth2pcd(depth, intrinsics, pose=None): 44 | # Camera coordinate system in Blender is x: right, y: up, z: inwards 45 | inv_K = np.linalg.inv(intrinsics) 46 | inv_K[2, 2] = -1 47 | depth = np.flipud(depth) 48 | y, x = np.where(depth > 0) 49 | points = np.dot(inv_K, np.stack([x, y, np.ones_like(x)] * depth[y, x], 0)) 50 | if pose is not None: 51 | points = np.dot(pose, np.concatenate([points, np.ones((1, points.shape[1]))], 0))[:3, :] 52 | return points.T 53 | 54 | 55 | if __name__ == '__main__': 56 | parser = argparse.ArgumentParser() 57 | parser.add_argument('list_file') 58 | parser.add_argument('intrinsics_file') 59 | parser.add_argument('input_dir') 60 | parser.add_argument('output_dir') 61 | parser.add_argument('num_traj', type=int) 62 | args = parser.parse_args() 63 | 64 | with open(args.list_file) as file: 65 | model_list = file.read().splitlines() 66 | intrinsics = np.loadtxt(args.intrinsics_file) 67 | width = int(intrinsics[0, 2] * 2) 68 | height = int(intrinsics[1, 2] * 2) 69 | 70 | for model_id in model_list: 71 | category = model_id.rsplit('_', 1)[0] 72 | in_dir = os.path.join(args.input_dir, category, model_id) 73 | out_dir = os.path.join(args.output_dir, category, model_id) 74 | num_scans = np.loadtxt(os.path.join(in_dir, 'num_scans.txt'), dtype=np.int) 75 | if len(num_scans.shape) == 0: 76 | num_scans = [num_scans] 77 | os.makedirs(out_dir, exist_ok=True) 78 | for i in range(args.num_traj): 79 | for j in range(num_scans[i]): 80 | depth = read_exr(os.path.join(in_dir, '%d/%d.exr' % (i, j)), width, height) 81 | depth[np.isinf(depth)] = 0 82 | 83 | pose = np.loadtxt(os.path.join(in_dir, '%d/%d.txt' % (i, j))) 84 | if j == 0: 85 | pose0 = pose 86 | points = depth2pcd(depth, intrinsics) 87 | else: 88 | points = np.concatenate([points, 89 | depth2pcd(depth, intrinsics, np.dot(np.linalg.inv(pose0), pose))], axis=0) 90 | 91 | if points.shape[0] == 0: 92 | print(model_id, i, j) 93 | # fig = plt.figure() 94 | # ax = fig.add_subplot(111, projection='3d') 95 | # print(num_scans[i], points.shape[0], points.min(0), points.max(0)) 96 | # ax.scatter(points[:, 0], points[:, 1], points[:, 2], s=2) 97 | # ax.set_xlim(-1, 1) 98 | # ax.set_ylim(-1, 1) 99 | # ax.set_zlim(-3, -1) 100 | # plt.show() 101 | 102 | pcd = PointCloud() 103 | pcd.points = Vector3dVector(points) 104 | write_point_cloud(os.path.join(out_dir, '%d.pcd' % i), pcd) 105 | np.savetxt(os.path.join(out_dir, '%d.txt' % i), np.linalg.inv(pose0), '%.20f') 106 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | lmdb>=0.9 2 | matplotlib>=2.1 3 | tensorflow-gpu>=1.12 4 | tensorpack==0.9.0 5 | -------------------------------------------------------------------------------- /segmenter/dgcnn.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 4 | sys.path.append(os.path.join(BASE_DIR, '../util')) 5 | import tensorflow as tf 6 | import tf_util 7 | 8 | 9 | def get_model(point_cloud, input_label, is_training, bn_decay, k=20): 10 | """ Classification DGCNN, input is BxNx3, output BxNx50 """ 11 | cat_num = 16 12 | part_num = 50 13 | batch_size = point_cloud.get_shape()[0].value 14 | num_point = point_cloud.get_shape()[1].value 15 | 16 | adj = tf_util.pairwise_distance(point_cloud) 17 | nn_idx = tf_util.knn(adj, k=k) 18 | edge_feature = tf_util.get_edge_feature(point_cloud, nn_idx=nn_idx, k=k) 19 | 20 | out1 = tf_util.conv2d(edge_feature, 64, [1,1], 21 | padding='VALID', stride=[1,1], 22 | bn=True, is_training=is_training, 23 | scope='adj_conv1', bn_decay=bn_decay) 24 | 25 | out2 = tf_util.conv2d(out1, 64, [1,1], 26 | padding='VALID', stride=[1,1], 27 | bn=True, is_training=is_training, 28 | scope='adj_conv2', bn_decay=bn_decay) 29 | 30 | net_max_1 = tf.reduce_max(out2, axis=-2, keepdims=True, name='maxpool1') 31 | net_mean_1 = tf.reduce_mean(out2, axis=-2, keepdims=True, name='meanpool1') 32 | 33 | out3 = tf_util.conv2d(tf.concat([net_max_1, net_mean_1], axis=-1), 64, [1,1], 34 | padding='VALID', stride=[1,1], 35 | bn=True, is_training=is_training, 36 | scope='adj_conv3', bn_decay=bn_decay) 37 | 38 | adj = tf_util.pairwise_distance(tf.squeeze(out3, axis=-2)) 39 | nn_idx = tf_util.knn(adj, k=k) 40 | edge_feature = tf_util.get_edge_feature(tf.squeeze(out3, axis=-2), nn_idx=nn_idx, k=k) 41 | 42 | out4 = tf_util.conv2d(edge_feature, 64, [1,1], 43 | padding='VALID', stride=[1,1], 44 | bn=True, is_training=is_training, 45 | scope='adj_conv4', bn_decay=bn_decay) 46 | 47 | net_max_2 = tf.reduce_max(out4, axis=-2, keepdims=True, name='maxpool2') 48 | net_mean_2 = tf.reduce_mean(out4, axis=-2, keepdims=True, name='meanpool2') 49 | 50 | out5 = tf_util.conv2d(tf.concat([net_max_2, net_mean_2], axis=-1), 64, [1,1], 51 | padding='VALID', stride=[1,1], 52 | bn=True, is_training=is_training, 53 | scope='adj_conv5', bn_decay=bn_decay) 54 | 55 | adj = tf_util.pairwise_distance(tf.squeeze(out5, axis=-2)) 56 | nn_idx = tf_util.knn(adj, k=k) 57 | edge_feature = tf_util.get_edge_feature(tf.squeeze(out5, axis=-2), nn_idx=nn_idx, k=k) 58 | 59 | out6 = tf_util.conv2d(edge_feature, 64, [1,1], 60 | padding='VALID', stride=[1,1], 61 | bn=True, is_training=is_training, 62 | scope='adj_conv6', bn_decay=bn_decay) 63 | 64 | net_max_3 = tf.reduce_max(out6, axis=-2, keepdims=True, name='maxpool3') 65 | net_mean_3 = tf.reduce_mean(out6, axis=-2, keepdims=True, name='meanpool3') 66 | 67 | out7 = tf_util.conv2d(tf.concat([net_max_3, net_mean_3], axis=-1), 64, [1,1], 68 | padding='VALID', stride=[1,1], 69 | bn=True, is_training=is_training, 70 | scope='adj_conv7', bn_decay=bn_decay) 71 | 72 | out8 = tf_util.conv2d(tf.concat([out3, out5, out7], axis=-1), 1024, [1, 1], 73 | padding='VALID', stride=[1,1], 74 | bn=True, is_training=is_training, 75 | scope='adj_conv8', bn_decay=bn_decay) 76 | 77 | out_max = tf.reduce_max(out8, axis=1, keepdims=True, name='maxpool4') 78 | 79 | one_hot_label = tf.one_hot(input_label, cat_num) 80 | one_hot_label_expand = tf.reshape(one_hot_label, [batch_size, 1, 1, cat_num]) 81 | one_hot_label_expand = tf_util.conv2d(one_hot_label_expand, 128, [1, 1], 82 | padding='VALID', stride=[1,1], 83 | bn=True, is_training=is_training, 84 | scope='one_hot_label_expand', bn_decay=bn_decay) 85 | out_max = tf.concat(axis=3, values=[out_max, one_hot_label_expand]) 86 | expand = tf.tile(out_max, [1, num_point, 1, 1]) 87 | 88 | concat = tf.concat(axis=3, values=[expand, 89 | net_max_1, 90 | net_mean_1, 91 | out3, 92 | net_max_2, 93 | net_mean_2, 94 | out5, 95 | net_max_3, 96 | net_mean_3, 97 | out7, 98 | out8]) 99 | 100 | net2 = tf_util.conv2d(concat, 256, [1,1], padding='VALID', stride=[1,1], bn_decay=bn_decay, 101 | bn=True, is_training=is_training, scope='seg/conv1') 102 | net2 = tf_util.dropout(net2, keep_prob=0.6, is_training=is_training, scope='seg/dp1') 103 | net2 = tf_util.conv2d(net2, 256, [1,1], padding='VALID', stride=[1,1], bn_decay=bn_decay, 104 | bn=True, is_training=is_training, scope='seg/conv2') 105 | net2 = tf_util.dropout(net2, keep_prob=0.6, is_training=is_training, scope='seg/dp2') 106 | net2 = tf_util.conv2d(net2, 128, [1,1], padding='VALID', stride=[1,1], bn_decay=bn_decay, 107 | bn=True, is_training=is_training, scope='seg/conv3') 108 | net2 = tf_util.conv2d(net2, part_num, [1,1], padding='VALID', stride=[1,1], activation_fn=None, 109 | bn=False, scope='seg/conv4') 110 | 111 | net2 = tf.reshape(net2, [batch_size, num_point, part_num]) 112 | 113 | return net2 114 | 115 | 116 | def get_loss(pred, label): 117 | """ pred: BxNxC, 118 | label: BxN, """ 119 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 120 | classify_loss = tf.reduce_mean(loss) 121 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 122 | return classify_loss 123 | -------------------------------------------------------------------------------- /segmenter/pointnet.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 4 | sys.path.append(os.path.join(BASE_DIR, '../util')) 5 | import tensorflow as tf 6 | import tf_util 7 | 8 | 9 | def get_model(point_cloud, input_label, is_training, bn_decay=None): 10 | """ Classification PointNet, input is BxNx3, output BxNx50 """ 11 | part_num = 50 12 | batch_size = point_cloud.get_shape()[0].value 13 | num_point = point_cloud.get_shape()[1].value 14 | 15 | input_image = tf.expand_dims(point_cloud, -1) 16 | net = tf_util.conv2d(input_image, 64, [1,3], 17 | padding='VALID', stride=[1,1], 18 | bn=True, is_training=is_training, 19 | scope='conv1', bn_decay=bn_decay) 20 | net = tf_util.conv2d(net, 64, [1,1], 21 | padding='VALID', stride=[1,1], 22 | bn=True, is_training=is_training, 23 | scope='conv2', bn_decay=bn_decay) 24 | point_feat = net 25 | 26 | net = tf_util.conv2d(net, 64, [1,1], 27 | padding='VALID', stride=[1,1], 28 | bn=True, is_training=is_training, 29 | scope='conv3', bn_decay=bn_decay) 30 | net = tf_util.conv2d(net, 128, [1,1], 31 | padding='VALID', stride=[1,1], 32 | bn=True, is_training=is_training, 33 | scope='conv4', bn_decay=bn_decay) 34 | net = tf_util.conv2d(net, 1024, [1,1], 35 | padding='VALID', stride=[1,1], 36 | bn=True, is_training=is_training, 37 | scope='conv5', bn_decay=bn_decay) 38 | global_feat = tf.reduce_max(net, axis=1, name='maxpool', keep_dims=True) 39 | 40 | global_feat_expand = tf.tile(global_feat, [1, num_point, 1, 1]) 41 | concat_feat = tf.concat([point_feat, global_feat_expand], axis=3) 42 | 43 | net = tf_util.conv2d(concat_feat, 512, [1,1], 44 | padding='VALID', stride=[1,1], 45 | bn=True, is_training=is_training, 46 | scope='conv6', bn_decay=bn_decay) 47 | net = tf_util.conv2d(net, 256, [1,1], 48 | padding='VALID', stride=[1,1], 49 | bn=True, is_training=is_training, 50 | scope='conv7', bn_decay=bn_decay) 51 | net = tf_util.conv2d(net, 128, [1,1], 52 | padding='VALID', stride=[1,1], 53 | bn=True, is_training=is_training, 54 | scope='conv8', bn_decay=bn_decay) 55 | net = tf_util.conv2d(net, 128, [1,1], 56 | padding='VALID', stride=[1,1], 57 | bn=True, is_training=is_training, 58 | scope='conv9', bn_decay=bn_decay) 59 | 60 | net = tf_util.conv2d(net, part_num, [1,1], 61 | padding='VALID', stride=[1,1], activation_fn=None, 62 | scope='conv10') 63 | net = tf.squeeze(net, 2) 64 | 65 | return net 66 | 67 | 68 | def get_loss(pred, label): 69 | """ pred: BxNxC, 70 | label: BxN, """ 71 | loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label) 72 | classify_loss = tf.reduce_mean(loss) 73 | tf.summary.scalar('train/classify loss', classify_loss, collections=['train']) 74 | return classify_loss 75 | -------------------------------------------------------------------------------- /test_cls.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import numpy as np 28 | import tensorflow as tf 29 | import time 30 | from tensorpack import dataflow 31 | 32 | import os 33 | import sys 34 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 35 | sys.path.append(os.path.join(BASE_DIR, 'classifier')) 36 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 37 | sys.path.append(os.path.join(BASE_DIR, 'util')) 38 | from data_util import ResampleData 39 | from visu_util import plot_iters, plot_conf 40 | 41 | n_classes = 40 42 | cat_names = ['airplane','bathtub','bed','bench','bookshelf', 43 | 'bottle','bowl','car','chair','cone', 44 | 'cup','curtain','desk','door','dresser', 45 | 'flower_pot','glass_box','guitar','keyboard','lamp', 46 | 'laptop','mantel','monitor','night_stand','person', 47 | 'piano','plant','radio','range_hood','sink', 48 | 'sofa','stairs','stool','table','tent', 49 | 'toilet','tv_stand','vase','wardrobe','xbox'] 50 | 51 | 52 | if __name__ == '__main__': 53 | parser = argparse.ArgumentParser() 54 | parser.add_argument('--lmdb_path') 55 | parser.add_argument('--checkpoint') 56 | parser.add_argument('--results_dir') 57 | parser.add_argument('--classifier', choices=['pointnet', 'dgcnn']) 58 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 59 | parser.add_argument('--n_iter', type=int, default=2) 60 | parser.add_argument('--num_points', type=int, default=1024) 61 | parser.add_argument('--plot_freq', type=int, default=40) 62 | parser.add_argument('--plot_lim', type=float, default=0.7) 63 | parser.add_argument('--plot_size', type=float, default=5) 64 | args = parser.parse_args() 65 | 66 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 67 | points_pl = tf.placeholder(tf.float32, (1, args.num_points, 3), 'points') 68 | 69 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 70 | transformer = importlib.import_module(args.transformer) 71 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 72 | is_training_pl, 0.99) 73 | with tf.variable_scope('classifier'): 74 | classifier = importlib.import_module(args.classifier) 75 | logits = classifier.get_model(transformed_points, is_training_pl, 0.99) 76 | prediction = tf.argmax(logits, axis=1) 77 | 78 | config = tf.ConfigProto() 79 | config.gpu_options.allow_growth = True 80 | config.allow_soft_placement = True 81 | sess = tf.Session(config=config) 82 | 83 | saver = tf.train.Saver() 84 | saver.restore(sess, args.checkpoint) 85 | os.makedirs(os.path.join(args.results_dir, 'plots'), exist_ok=True) 86 | 87 | df = dataflow.LMDBSerializer.load(args.lmdb_path, shuffle=False) 88 | if args.num_points is not None: 89 | df = ResampleData(df, args.num_points, 'cls') 90 | 91 | times = [] 92 | conf = np.zeros((n_classes, n_classes)) 93 | rotation = [[[] for i in range(args.n_iter+1)] for j in range(n_classes)] 94 | translation = [[[] for i in range(args.n_iter+1)] for j in range(n_classes)] 95 | is_correct = [[] for i in range(n_classes)] 96 | r_mag = [[] for i in range(args.n_iter)] 97 | t_mag = [[] for i in range(args.n_iter)] 98 | for i, (model_id, points, label, init_pose) in enumerate(df.get_data()): 99 | start = time.time() 100 | pred, ts = sess.run([prediction, Ts], 101 | feed_dict={is_training_pl: False, points_pl: [points]}) 102 | times.append(time.time() - start) 103 | 104 | conf[pred[0], label] += 1 105 | is_correct[label].append([pred[0] == label]) 106 | 107 | T = np.eye(4) 108 | transforms = [] 109 | titles = [] 110 | for j in range(args.n_iter+1): 111 | transforms.append(T) 112 | titles.append('Iteration %d' % j) 113 | if j < args.n_iter: 114 | T = np.dot(ts[j][0], T) 115 | 116 | if (i+1) % args.plot_freq == 0: 117 | figpath = os.path.join(args.results_dir, 'plots', '%s.png' % model_id) 118 | titles[0] = 'Input' 119 | suptitle = 'Predicted %s Actual %s' % (cat_names[pred[0]], cat_names[label]) 120 | figpath = os.path.join(args.results_dir, 'plots', '%s.png' % model_id) 121 | plot_iters(figpath, points, transforms, titles, args.plot_lim, args.plot_size, suptitle) 122 | 123 | print('Average time', np.mean(times)) 124 | print('Average accuracy', np.trace(conf) / np.sum(conf)) 125 | print('Average class accuracy', np.nanmean(np.diag(conf) / np.sum(conf, axis=0))) 126 | plot_conf(os.path.join(args.results_dir, 'confusion.png'), conf) 127 | np.savetxt(os.path.join(args.results_dir, 'confusion.txt'), conf, '%d') 128 | with open(os.path.join(args.results_dir, 'precision.txt'), 'w') as file: 129 | file.write('\n'.join(['%s: %.4f' % (cat_names[i], conf[i, i] / np.sum(conf[i, :])) for i in range(n_classes)])) 130 | with open(os.path.join(args.results_dir, 'recall.txt'), 'w') as file: 131 | file.write('\n'.join(['%s: %.4f' % (cat_names[i], conf[i, i] / np.sum(conf[:, i])) for i in range(n_classes)])) 132 | -------------------------------------------------------------------------------- /test_pose.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import numpy as np 28 | import tensorflow as tf 29 | import time 30 | from matplotlib import pyplot as plt 31 | from mpl_toolkits.mplot3d import Axes3D 32 | from tensorpack import dataflow 33 | 34 | import os 35 | import sys 36 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 37 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 38 | sys.path.append(os.path.join(BASE_DIR, 'util')) 39 | from tf_util import rotation_error, translation_error 40 | from visu_util import plot_iters, plot_mean_std, plot_cdf 41 | 42 | 43 | if __name__ == '__main__': 44 | parser = argparse.ArgumentParser() 45 | parser.add_argument('--lmdb_path') 46 | parser.add_argument('--checkpoint') 47 | parser.add_argument('--results_dir') 48 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 49 | parser.add_argument('--randinit', action='store_true') 50 | parser.add_argument('--nostop', action='store_true') 51 | parser.add_argument('--n_iter', type=int, default=10) 52 | parser.add_argument('--iter_plot_freq', type=int, default=3) 53 | parser.add_argument('--plot_freq', type=int, default=50) 54 | parser.add_argument('--plot_lim', type=float, default=0.3) 55 | parser.add_argument('--plot_size', type=float, default=7) 56 | parser.add_argument('--plot_nbins', type=int, default=100) 57 | args = parser.parse_args() 58 | 59 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 60 | points_pl = tf.placeholder(tf.float32, (1, None, 3), 'points') 61 | 62 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 63 | transformer = importlib.import_module(args.transformer) 64 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 65 | is_training_pl, 0.99, 66 | args.randinit, args.nostop) 67 | 68 | R_pl = tf.placeholder(tf.float32, (3, 3), 'rotation') 69 | R_gt_pl = tf.placeholder(tf.float32, (3, 3), 'rotation_gt') 70 | t_pl = tf.placeholder(tf.float32, (3,), 'translation') 71 | t_gt_pl = tf.placeholder(tf.float32, (3,), 'translation_gt') 72 | rot_error_op = rotation_error(R_pl, R_gt_pl) 73 | trans_error_op = translation_error(t_pl, t_gt_pl) 74 | 75 | config = tf.ConfigProto() 76 | config.gpu_options.allow_growth = True 77 | config.allow_soft_placement = True 78 | sess = tf.Session(config=config) 79 | 80 | saver = tf.train.Saver() 81 | saver.restore(sess, args.checkpoint) 82 | 83 | df = dataflow.LMDBSerializer.load(args.lmdb_path, shuffle=False) 84 | os.makedirs(os.path.join(args.results_dir, 'plots'), exist_ok=True) 85 | 86 | times = [] 87 | r_errors = [[] for i in range(args.n_iter + 1)] 88 | t_errors = [[] for i in range(args.n_iter + 1)] 89 | g_errors = [[] for i in range(args.n_iter + 1)] 90 | r_mag = [[] for i in range(args.n_iter)] 91 | t_mag = [[] for i in range(args.n_iter)] 92 | n_correct = np.zeros(args.n_iter + 1) 93 | for n, (model_id, pcd, T_gt) in enumerate(df.get_data()): 94 | start = time.time() 95 | ts = sess.run(Ts, {is_training_pl: False, points_pl: [pcd]}) 96 | times.append(time.time() - start) 97 | 98 | T = np.eye(4) 99 | transforms = [] 100 | titles = [] 101 | for i in range(args.n_iter + 1): 102 | r_error = sess.run(rot_error_op, {R_pl: T[:3, :3], R_gt_pl: T_gt[:3, :3]}) 103 | t_error = sess.run(trans_error_op, {t_pl: T[:3, 3], t_gt_pl: T_gt[:3, 3]}) 104 | r_errors[i].append(r_error) 105 | t_errors[i].append(t_error) 106 | if i % args.iter_plot_freq == 0: 107 | error_str = 'Rotation error %.4f\nTranslation error %.4f\n' % (r_error, t_error) 108 | transforms.append(T) 109 | if i == 0: 110 | titles.append('Input\n%s' % error_str) 111 | else: 112 | titles.append('Iteration %d\n%s' % (i, error_str)) 113 | if r_error < 10 and t_error < 0.1: 114 | n_correct[i] += 1 115 | if i < args.n_iter: 116 | T = np.dot(ts[i][0], T) 117 | r_mag = sess.run(rot_error_op, {R_pl: ts[i][0, :3, :3], R_gt_pl: np.eye(3, dtype=np.float32)}) 118 | t_mag = sess.run(trans_error_op, {t_pl: ts[i][0, :3, 3], t_gt_pl: np.zeros(3, dtype=np.float32)}) 119 | r_mag[i].append(r_mag) 120 | t_mag[i].append(t_mag) 121 | transforms.append(T_gt) 122 | titles.append('Ground truth\n\n') 123 | 124 | if n % args.plot_freq == 0: 125 | figpath = os.path.join(args.results_dir, 'plots', '%s.png' % model_id) 126 | plot_iters(figpath, pcd, transforms, titles, args.plot_lim, args.plot_size) 127 | 128 | np.savetxt(os.path.join(args.results_dir, 'r_err.txt'), r_errors) 129 | np.savetxt(os.path.join(args.results_dir, 't_err.txt'), t_errors) 130 | 131 | print('Average time', np.mean(times), 'std', np.std(times)) 132 | print('Percentage < 10 degree, 0.1', n_correct / (n+1)) 133 | 134 | plt.figure() 135 | plt.plot(np.arange(args.n_iter + 1), n_correct / (n+1)) 136 | plt.xlabel('number of iterations') 137 | plt.ylabel('percentage < 10 degree, 0.1') 138 | plt.savefig(os.path.join(args.results_dir, 'pose_accuracy.png')) 139 | 140 | plot_mean_std(os.path.join(args.results_dir, 'r_err.png'), 141 | np.arange(args.n_iter + 1), r_errors, 'rotation error') 142 | plot_mean_std(os.path.join(args.results_dir, 't_err.png'), 143 | np.arange(args.n_iter + 1), t_errors, 'translation error') 144 | plot_mean_std(os.path.join(args.results_dir, 'r_mag.png'), 145 | np.arange(args.n_iter) + 1, r_mag, 'Predicted rotation magnitude') 146 | plot_mean_std(os.path.join(args.results_dir, 't_mag.png'), 147 | np.arange(args.n_iter) + 1, t_mag, 'Predicted translation magnitude') 148 | 149 | for i in range(args.n_iter + 1): 150 | if i % args.iter_plot_freq == 0: 151 | plot_cdf(os.path.join(args.results_dir, 'r_err_cdf_iter-%d.png' % i), 152 | r_errors[i], 'Rotation error') 153 | plot_cdf(os.path.join(args.results_dir, 't_err_cdf_iter-%d.png' % i), 154 | t_errors[i], 'Translation error') 155 | -------------------------------------------------------------------------------- /test_seg.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import json 28 | import numpy as np 29 | import tensorflow as tf 30 | import time 31 | from tensorpack import dataflow 32 | 33 | import os 34 | import sys 35 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 36 | sys.path.append(os.path.join(BASE_DIR, 'segmenter')) 37 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 38 | sys.path.append(os.path.join(BASE_DIR, 'util')) 39 | from data_util import random_idx 40 | from visu_util import plot_iters_seg 41 | 42 | n_classes = 16 43 | n_parts = 50 44 | synsets = ['02691156', '02773838', '02954340', '02958343', 45 | '03001627', '03261776', '03467517', '03624134', 46 | '03636649', '03642806', '03790512', '03797390', 47 | '03948459', '04099429', '04225987', '04379243'] 48 | 49 | 50 | def mean_iou(pred, labels, num_classes): 51 | conf = np.zeros((num_classes, num_classes)) 52 | for i in range(pred.shape[0]): 53 | conf[pred[i], labels[i]] += 1 54 | tp = np.diag(conf) 55 | total = np.sum(conf, 0) + np.sum(conf, 1) - tp 56 | return np.mean(tp[total > 0] / total[total > 0]) 57 | 58 | 59 | if __name__ == '__main__': 60 | parser = argparse.ArgumentParser() 61 | parser.add_argument('--lmdb_path') 62 | parser.add_argument('--checkpoint') 63 | parser.add_argument('--results_dir') 64 | parser.add_argument('--segmenter', choices=['pointnet', 'dgcnn']) 65 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 66 | parser.add_argument('--n_iter', type=int, default=2) 67 | parser.add_argument('--num_points', type=int, default=1024) 68 | parser.add_argument('--plot_freq', type=int, default=40) 69 | parser.add_argument('--plot_lim', type=float, default=0.3) 70 | parser.add_argument('--plot_size', type=float, default=5) 71 | args = parser.parse_args() 72 | 73 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 74 | points_pl = tf.placeholder(tf.float32, (1, args.num_points, 3), 'points') 75 | cat_labels_pl = tf.placeholder(tf.int32, (1,), 'cat_labels') 76 | 77 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 78 | transformer = importlib.import_module(args.transformer) 79 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 80 | is_training_pl, 0.99) 81 | with tf.variable_scope('segmenter'): 82 | segmenter = importlib.import_module(args.segmenter) 83 | logits = segmenter.get_model(transformed_points, cat_labels_pl, is_training_pl, 0.99) 84 | prediction = tf.argmax(logits, axis=2) 85 | 86 | config = tf.ConfigProto() 87 | config.gpu_options.allow_growth = True 88 | config.allow_soft_placement = True 89 | sess = tf.Session(config=config) 90 | 91 | saver = tf.train.Saver() 92 | saver.restore(sess, args.checkpoint) 93 | os.makedirs(os.path.join(args.results_dir, 'plots'), exist_ok=True) 94 | 95 | df = dataflow.LMDBSerializer.load(args.lmdb_path, shuffle=False) 96 | 97 | times = [] 98 | acc = {synset_id: [] for synset_id in synsets} 99 | iou = {synset_id: [] for synset_id in synsets} 100 | rotation = [[[] for i in range(args.n_iter+1)] for j in range(n_classes)] 101 | translation = [[[] for i in range(args.n_iter+1)] for j in range(n_classes)] 102 | r_mag = [[] for i in range(args.n_iter)] 103 | t_mag = [[] for i in range(args.n_iter)] 104 | for i, (model_id, points, labels, cat_label, init_pose) in enumerate(df.get_data()): 105 | orig_size = points.shape[0] 106 | idx, perm = random_idx(orig_size, args.num_points) 107 | 108 | start = time.time() 109 | pred, ts = sess.run([prediction, Ts], 110 | feed_dict={is_training_pl: False, points_pl: [points[idx]], 111 | cat_labels_pl: [cat_label]}) 112 | times.append(time.time() - start) 113 | 114 | points = points[perm] 115 | labels = labels[perm] 116 | pred = pred[0][:orig_size] 117 | 118 | acc[synsets[cat_label]].append(np.sum(pred == labels) / orig_size) 119 | iou[synsets[cat_label]].append(mean_iou(pred, labels, n_parts)) 120 | 121 | T = np.eye(4) 122 | transforms = [] 123 | part_ids = [] 124 | titles = [] 125 | for j in range(args.n_iter+1): 126 | transforms.append(T) 127 | part_ids.append(pred) 128 | titles.append('Iteration %d' % j) 129 | if j < args.n_iter: 130 | T = np.dot(ts[j][0], T) 131 | transforms.append(T) 132 | part_ids.append(labels) 133 | titles.append('Ground truth') 134 | if (i+1) % args.plot_freq == 0: 135 | titles[0] = 'Input' 136 | figpath = os.path.join(args.results_dir, 'plots', '%s.png' % model_id) 137 | plot_iters_seg(figpath, points, transforms, part_ids, titles, args.plot_lim, args.plot_size) 138 | 139 | total_acc = 0 140 | n_shapes = 0 141 | for synset_id in acc: 142 | n_shapes += len(acc[synset_id]) 143 | total_acc += np.sum(acc[synset_id]) 144 | acc[synset_id] = np.mean(acc[synset_id]) 145 | avg_acc = total_acc / n_shapes 146 | total_iou = 0 147 | for synset_id in iou: 148 | total_iou += np.sum(iou[synset_id]) 149 | iou[synset_id] = np.mean(iou[synset_id]) 150 | avg_iou = total_iou / n_shapes 151 | print('Average time', np.mean(times)) 152 | print('Average accuracy', avg_acc) 153 | print('Average IOU', avg_iou) 154 | with open(os.path.join(args.results_dir, 'accuracy.txt'), 'w') as file: 155 | file.write('\n'.join(['%s: %.4f' % (synsets[i], acc[synsets[i]]) for i in range(n_classes)])) 156 | file.write('\naverage: %.4f' % avg_acc) 157 | with open(os.path.join(args.results_dir, 'iou.txt'), 'w') as file: 158 | file.write('\n'.join(['%s: %.4f' % (synsets[i], iou[synsets[i]]) for i in range(n_classes)])) 159 | file.write('\naverage: %.4f' % avg_iou) 160 | -------------------------------------------------------------------------------- /train_cls.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import numpy as np 28 | import tensorflow as tf 29 | from termcolor import colored 30 | 31 | import os 32 | import sys 33 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 34 | sys.path.append(os.path.join(BASE_DIR, 'classifier')) 35 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 36 | sys.path.append(os.path.join(BASE_DIR, 'util')) 37 | from data_util import lmdb_dataflow 38 | from log_util import create_log_dir 39 | from visu_util import plot_iters 40 | 41 | n_classes = 40 42 | cat_names = ['airplane','bathtub','bed','bench','bookshelf', 43 | 'bottle','bowl','car','chair','cone', 44 | 'cup','curtain','desk','door','dresser', 45 | 'flower_pot','glass_box','guitar','keyboard','lamp', 46 | 'laptop','mantel','monitor','night_stand','person', 47 | 'piano','plant','radio','range_hood','sink', 48 | 'sofa','stairs','stool','table','tent', 49 | 'toilet','tv_stand','vase','wardrobe','xbox'] 50 | 51 | 52 | if __name__ == '__main__': 53 | parser = argparse.ArgumentParser() 54 | parser.add_argument('--data_dir') 55 | parser.add_argument('--log_dir') 56 | parser.add_argument('--classifier', choices=['pointnet', 'dgcnn']) 57 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 58 | parser.add_argument('--regularize', action='store_true') 59 | parser.add_argument('--n_iter', type=int, default=2) 60 | parser.add_argument('--batch_size', type=int, default=32) 61 | parser.add_argument('--num_points', type=int, default=1024) 62 | parser.add_argument('--init_lr', type=float, default=0.001) 63 | parser.add_argument('--lr_decay_rate', type=float, default=0.7) 64 | parser.add_argument('--lr_decay_steps', type=int, default=400000) 65 | parser.add_argument('--lr_clip', type=float, default=0.00001) 66 | parser.add_argument('--init_bn_decay', type=float, default=0.5) 67 | parser.add_argument('--bn_decay_decay_rate', type=float, default=0.5) 68 | parser.add_argument('--bn_decay_decay_steps', type=int, default=400000) 69 | parser.add_argument('--bn_decay_clip', type=float, default=0.99) 70 | parser.add_argument('--grad_clip', type=float, default=30.0) 71 | parser.add_argument('--max_epoch', type=int, default=60) 72 | parser.add_argument('--epoch_per_save', type=int, default=10) 73 | parser.add_argument('--steps_per_print', type=int, default=100) 74 | parser.add_argument('--steps_per_eval', type=int, default=300) 75 | parser.add_argument('--steps_per_plot', type=int, default=3000) 76 | parser.add_argument('--val_steps_per_plot', type=int, default=2) 77 | parser.add_argument('--plot_lim', type=float, default=0.7) 78 | parser.add_argument('--plot_size', type=float, default=2) 79 | parser.add_argument('--iter_plot_freq', type=int, default=1) 80 | parser.add_argument('--restore', action='store_true') 81 | parser.add_argument('--checkpoint') 82 | args = parser.parse_args() 83 | 84 | global_step = tf.Variable(0, trainable=False, name='global_step') 85 | learning_rate = tf.train.exponential_decay(args.init_lr, global_step * args.batch_size, 86 | args.lr_decay_steps, args.lr_decay_rate, 87 | staircase=True, name='learning_rate') 88 | learning_rate = tf.maximum(learning_rate, args.lr_clip) 89 | bn_decay = 1 - tf.train.exponential_decay(args.init_bn_decay, global_step * args.batch_size, 90 | args.bn_decay_decay_steps, args.bn_decay_decay_rate, 91 | staircase=True, name='bn_decay') 92 | bn_decay = tf.minimum(bn_decay, args.bn_decay_clip) 93 | 94 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 95 | points_pl = tf.placeholder(tf.float32, (args.batch_size, args.num_points, 3), 'points') 96 | labels_pl = tf.placeholder(tf.int32, (args.batch_size,), 'labels') 97 | 98 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 99 | transformer = importlib.import_module(args.transformer) 100 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 101 | is_training_pl, bn_decay) 102 | with tf.variable_scope('classifier'): 103 | classifier = importlib.import_module(args.classifier) 104 | logits = classifier.get_model(transformed_points, is_training_pl, bn_decay) 105 | prediction = tf.argmax(logits, axis=1) 106 | 107 | if args.regularize: 108 | loss_op = classifier.get_loss_reg(logits, labels_pl, T_out[:, :3, :3]) 109 | else: 110 | loss_op = classifier.get_loss(logits, labels_pl) 111 | 112 | trainer = tf.train.AdamOptimizer(learning_rate) 113 | grad, var = zip(*trainer.compute_gradients(loss_op, tf.trainable_variables())) 114 | grad, global_norm = tf.clip_by_global_norm(grad, args.grad_clip) 115 | train_op = trainer.apply_gradients(zip(grad, var), global_step) 116 | 117 | avg_loss, loss_update = tf.metrics.mean(loss_op) 118 | avg_acc, acc_update = tf.metrics.accuracy(labels_pl, prediction) 119 | 120 | tf.summary.scalar('train/learning rate', learning_rate, collections=['train']) 121 | tf.summary.scalar('train/gradient norm', global_norm, collections=['train']) 122 | tf.summary.scalar('train/bn decay', bn_decay, collections=['train']) 123 | tf.summary.scalar('valid/loss', avg_loss, collections=['valid']) 124 | tf.summary.scalar('valid/accuracy', avg_acc, collections=['valid']) 125 | update_ops = [loss_update, acc_update] 126 | train_summary = tf.summary.merge_all('train') 127 | valid_summary = tf.summary.merge_all('valid') 128 | 129 | lmdb_train = os.path.join(args.data_dir, 'train.lmdb') 130 | lmdb_valid = os.path.join(args.data_dir, 'valid.lmdb') 131 | df_train, num_train = lmdb_dataflow(lmdb_train, args.batch_size, args.num_points, 132 | shuffle=True, task='cls') 133 | df_valid, num_valid = lmdb_dataflow(lmdb_valid, args.batch_size, args.num_points, 134 | shuffle=False, task='cls') 135 | train_gen = df_train.get_data() 136 | valid_gen = df_valid.get_data() 137 | 138 | config = tf.ConfigProto() 139 | config.gpu_options.allow_growth = True 140 | config.allow_soft_placement = True 141 | sess = tf.Session(config=config) 142 | sess.run(tf.global_variables_initializer()) 143 | 144 | writer = create_log_dir(args, sess) 145 | 146 | saver = tf.train.Saver() 147 | step = sess.run(global_step) 148 | epoch = step * args.batch_size // num_train + 1 149 | next_epoch = epoch 150 | while next_epoch <= args.max_epoch: 151 | step += 1 152 | epoch = step * args.batch_size // num_train + 1 153 | _, points, labels, poses = next(train_gen) 154 | _, loss, summary = sess.run([train_op, loss_op, train_summary], 155 | feed_dict={is_training_pl: True, 156 | points_pl: points, 157 | labels_pl: labels}) 158 | writer.add_summary(summary, step) 159 | if step % args.steps_per_print == 0: 160 | print('Epoch %d Step %d Loss %f' % (epoch, step, loss)) 161 | if step % args.steps_per_eval == 0: 162 | sess.run(tf.local_variables_initializer()) 163 | for i in range(num_valid // args.batch_size): 164 | instance_id, points, labels, poses = next(valid_gen) 165 | ts, pred, _ = sess.run([Ts, prediction, update_ops], 166 | feed_dict={is_training_pl: False, 167 | points_pl: points, 168 | labels_pl: labels}) 169 | if step % args.steps_per_plot == 0 and (i + 1) % args.val_steps_per_plot == 0: 170 | T = np.eye(4) 171 | transforms = [] 172 | titles = [] 173 | for j in range(args.n_iter+1): 174 | if j > 0: 175 | T = np.dot(ts[j-1][0], T) 176 | if j % args.iter_plot_freq == 0: 177 | transforms.append(T) 178 | titles.append('Iteration %d' % j) 179 | suptitle = 'Predicted %s Actual %s' % (cat_names[pred[0]], cat_names[labels[0]]) 180 | figpath = '%s/plots/%s_step_%d.png' % (args.log_dir, instance_id[0], step) 181 | plot_iters(figpath, points[0], transforms, titles, args.plot_lim, args.plot_size, suptitle) 182 | loss, acc = sess.run([avg_loss, avg_acc]) 183 | print(colored('Loss %f Accuracy %.4f ' % (loss, acc), 'white', 'on_blue')) 184 | summary = sess.run(valid_summary) 185 | writer.add_summary(summary, step) 186 | next_epoch = (step+1) * args.batch_size // num_train + 1 187 | if epoch % args.epoch_per_save == 0 and epoch < next_epoch: 188 | saver.save(sess, os.path.join(args.log_dir, 'model_epoch_%d' % epoch)) 189 | -------------------------------------------------------------------------------- /train_pose.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import numpy as np 28 | import tensorflow as tf 29 | from termcolor import colored 30 | 31 | import os 32 | import sys 33 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 34 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 35 | sys.path.append(os.path.join(BASE_DIR, 'util')) 36 | from data_util import lmdb_dataflow 37 | from log_util import create_log_dir 38 | from tf_util import geometric_error, rotation_error, translation_error 39 | from visu_util import plot_iters 40 | 41 | 42 | if __name__ == '__main__': 43 | parser = argparse.ArgumentParser() 44 | parser.add_argument('--data_dir') 45 | parser.add_argument('--log_dir') 46 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 47 | parser.add_argument('--nostop', action='store_true') 48 | parser.add_argument('--randinit', action='store_true') 49 | parser.add_argument('--n_iter', type=int, default=5) 50 | parser.add_argument('--batch_size', type=int, default=100) 51 | parser.add_argument('--num_points', type=int, default=256) 52 | parser.add_argument('--max_step', type=int, default=20000) 53 | parser.add_argument('--init_lr', type=float, default=0.001) 54 | parser.add_argument('--lr_decay_rate', type=float, default=0.7) 55 | parser.add_argument('--lr_decay_steps', type=float, default=200000) 56 | parser.add_argument('--lr_clip', type=float, default=0.00001) 57 | parser.add_argument('--use_bn', action='store_true') 58 | parser.add_argument('--init_bn_decay', type=float, default=0.5) 59 | parser.add_argument('--bn_decay_decay_rate', type=float, default=0.5) 60 | parser.add_argument('--bn_decay_decay_steps', type=int, default=200000) 61 | parser.add_argument('--bn_decay_clip', type=float, default=0.99) 62 | parser.add_argument('--steps_per_print', type=int, default=50) 63 | parser.add_argument('--steps_per_eval', type=int, default=100) 64 | parser.add_argument('--steps_per_plot', type=int, default=400) 65 | parser.add_argument('--steps_per_save', type=int, default=4000) 66 | parser.add_argument('--plot_lim', type=float, default=0.4) 67 | parser.add_argument('--plot_size', type=float, default=5) 68 | parser.add_argument('--iter_plot_freq', type=int, default=1) 69 | parser.add_argument('--restore', action='store_true') 70 | parser.add_argument('--checkpoint') 71 | args = parser.parse_args() 72 | 73 | global_step = tf.Variable(0, trainable=False, name='global_step') 74 | learning_rate = tf.train.exponential_decay(args.init_lr, global_step * args.batch_size, 75 | args.lr_decay_steps, args.lr_decay_rate, 76 | staircase=True, name='learning_rate') 77 | learning_rate = tf.maximum(learning_rate, args.lr_clip) 78 | bn_decay = 1 - tf.train.exponential_decay(args.init_bn_decay, global_step * args.batch_size, 79 | args.bn_decay_decay_steps, args.bn_decay_decay_rate, 80 | staircase=True, name='bn_decay') 81 | bn_decay = tf.minimum(bn_decay, args.bn_decay_clip) 82 | 83 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 84 | points_pl = tf.placeholder(tf.float32, (args.batch_size, args.num_points, 3), 'points') 85 | transforms_pl = tf.placeholder(tf.float32, (args.batch_size, 4, 4), 'transform') 86 | 87 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 88 | transformer = importlib.import_module(args.transformer) 89 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 90 | is_training_pl, bn_decay, 91 | args.randinit, args.nostop) 92 | 93 | g_error = geometric_error(points_pl, T_out, transforms_pl) 94 | r_error = rotation_error(T_out[:, :3, :3], transforms_pl[:, :3, :3]) 95 | t_error = translation_error(T_out[:, :3, 3], transforms_pl[:, :3, 3]) 96 | loss_op = tf.reduce_mean(g_error) 97 | 98 | trainer = tf.train.AdamOptimizer(learning_rate) 99 | train_op = trainer.minimize(loss_op, global_step) 100 | 101 | avg_loss, update0 = tf.metrics.mean(loss_op) 102 | percent_10deg, update1 = tf.metrics.percentage_below(r_error, 10) 103 | percent_01, update2 = tf.metrics.percentage_below(t_error, 0.1) 104 | update_ops = [update0, update1, update2] 105 | 106 | tf.summary.scalar('train/learning rate', learning_rate, collections=['train']) 107 | tf.summary.scalar('train/bn decay', bn_decay, collections=['train']) 108 | tf.summary.scalar('train/loss', loss_op, collections=['train']) 109 | tf.summary.scalar('valid/loss', avg_loss, collections=['valid']) 110 | tf.summary.scalar('valid/percent 10deg', percent_10deg, collections=['valid']) 111 | tf.summary.scalar('valid/percent 0.1', percent_01, collections=['valid']) 112 | 113 | train_summary = tf.summary.merge_all('train') 114 | valid_summary = tf.summary.merge_all('valid') 115 | 116 | lmdb_train = os.path.join(args.data_dir, 'train.lmdb') 117 | lmdb_valid = os.path.join(args.data_dir, 'test.lmdb') 118 | df_train, num_train = lmdb_dataflow(lmdb_train, args.batch_size, args.num_points, 119 | shuffle=True, task='pose') 120 | df_valid, num_valid = lmdb_dataflow(lmdb_valid, args.batch_size, args.num_points, 121 | shuffle=False, task='pose') 122 | train_gen = df_train.get_data() 123 | valid_gen = df_valid.get_data() 124 | 125 | config = tf.ConfigProto() 126 | config.gpu_options.allow_growth = True 127 | config.allow_soft_placement = True 128 | sess = tf.Session(config=config) 129 | sess.run(tf.global_variables_initializer()) 130 | 131 | writer = create_log_dir(args, sess) 132 | 133 | saver = tf.train.Saver() 134 | last_step = sess.run(global_step) 135 | for step in range(last_step, args.max_step): 136 | epoch = step * args.batch_size // num_train + 1 137 | instance_id, points, transforms = next(train_gen) 138 | _, loss, summary = sess.run([train_op, loss_op, train_summary], 139 | feed_dict={is_training_pl: True, 140 | points_pl: points, 141 | transforms_pl: transforms}) 142 | writer.add_summary(summary, step + 1) 143 | if (step + 1) % args.steps_per_print == 0: 144 | print('Epoch %d Step %d Loss %f' % (epoch, step + 1, loss)) 145 | if (step + 1) % args.steps_per_eval == 0: 146 | sess.run(tf.local_variables_initializer()) 147 | for i in range(num_valid // args.batch_size): 148 | instance_id, points, transforms = next(valid_gen) 149 | ts, _ = sess.run([Ts, update_ops], 150 | feed_dict={is_training_pl: False, 151 | points_pl: points, 152 | transforms_pl: transforms}) 153 | if (step + 1) % args.steps_per_plot == 0: 154 | T = np.eye(4) 155 | T_gt = transforms[0] 156 | plot_ts = [] 157 | titles = [] 158 | for j in range(args.n_iter+1): 159 | if j > 0: 160 | T = np.dot(ts[j-1][0], T) 161 | if j % args.iter_plot_freq == 0: 162 | rerr = sess.run(rotation_error(T[:3, :3], T_gt[:3, :3])) 163 | terr = sess.run(translation_error(T[:3, 3], T_gt[:3, 3])) 164 | gerr = sess.run(geometric_error(points[0], T, T_gt)) 165 | plot_ts.append(T) 166 | titles.append('Iteration %d\nRotation error %.4f\nTranslation error %.4f\n' 167 | 'Geometric error %.4f' % (j, rerr, terr, gerr)) 168 | plot_ts.append(T_gt) 169 | titles.append('Ground truth\nRotation error %.4f\nTranslation error %.4f\n' 170 | 'Geometric error %.4f' % (0, 0, 0)) 171 | figpath = os.path.join(args.log_dir, 'plots', '%s_step_%d.png' % (instance_id[0], step + 1)) 172 | plot_iters(figpath, points[0], plot_ts, titles, args.plot_lim, args.plot_size) 173 | l, p1, p2 = sess.run([avg_loss, percent_10deg, percent_01]) 174 | print(colored('Validation loss %f percentage < 10 degree %.4f percentage < 0.1 %.4f' % (l, p1, p2), 175 | 'white', 'on_blue')) 176 | summary = sess.run(valid_summary) 177 | writer.add_summary(summary, step + 1) 178 | if (step + 1) % args.steps_per_save == 0: 179 | saver.save(sess, os.path.join(args.log_dir, 'model'), step + 1) 180 | -------------------------------------------------------------------------------- /train_seg.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import argparse 26 | import importlib 27 | import numpy as np 28 | import tensorflow as tf 29 | from termcolor import colored 30 | 31 | import os 32 | import sys 33 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 34 | sys.path.append(os.path.join(BASE_DIR, 'segmenter')) 35 | sys.path.append(os.path.join(BASE_DIR, 'transformer')) 36 | sys.path.append(os.path.join(BASE_DIR, 'util')) 37 | from data_util import lmdb_dataflow 38 | from log_util import create_log_dir 39 | from visu_util import plot_iters_seg 40 | 41 | n_classes = 16 42 | n_parts = 50 43 | parts_idx = {'02691156': 0, '02773838': 4, '02954340': 6, '02958343': 8, 44 | '03001627': 12, '03261776': 16, '03467517': 19, '03624134': 22, 45 | '03636649': 24, '03642806': 28, '03790512': 30, '03797390': 36, 46 | '03948459': 38, '04099429': 41, '04225987': 44, '04379243': 47} 47 | 48 | 49 | def mean_iou(pred, labels, num_classes): 50 | conf = np.zeros((num_classes, num_classes)) 51 | for i in range(pred.shape[0]): 52 | conf[pred[i], labels[i]] += 1 53 | tp = np.diag(conf) 54 | total = np.sum(conf, 0) + np.sum(conf, 1) - tp 55 | return np.mean(tp[total > 0] / total[total > 0]) 56 | 57 | 58 | if __name__ == '__main__': 59 | parser = argparse.ArgumentParser() 60 | parser.add_argument('--data_dir') 61 | parser.add_argument('--log_dir') 62 | parser.add_argument('--segmenter', choices=['pointnet', 'dgcnn']) 63 | parser.add_argument('--transformer', choices=['t_net', 'it_net', 'it_net_dgcnn']) 64 | parser.add_argument('--n_iter', type=int, default=2) 65 | parser.add_argument('--batch_size', type=int, default=32) 66 | parser.add_argument('--num_points', type=int, default=1024) 67 | parser.add_argument('--num_parts', type=int, default=50) 68 | parser.add_argument('--init_lr', type=float, default=0.0005) 69 | parser.add_argument('--lr_decay_rate', type=float, default=0.7) 70 | parser.add_argument('--lr_decay_steps', type=int, default=200000) 71 | parser.add_argument('--lr_clip', type=float, default=0.00001) 72 | parser.add_argument('--init_bn_decay', type=float, default=0.5) 73 | parser.add_argument('--bn_decay_decay_rate', type=float, default=0.5) 74 | parser.add_argument('--bn_decay_decay_steps', type=int, default=200000) 75 | parser.add_argument('--bn_decay_clip', type=float, default=0.99) 76 | parser.add_argument('--grad_clip', type=float, default=30.0) 77 | parser.add_argument('--max_epoch', type=int, default=200) 78 | parser.add_argument('--epoch_per_save', type=int, default=50) 79 | parser.add_argument('--steps_per_print', type=int, default=100) 80 | parser.add_argument('--steps_per_eval', type=int, default=500) 81 | parser.add_argument('--steps_per_plot', type=int, default=3000) 82 | parser.add_argument('--val_steps_per_plot', type=int, default=7) 83 | parser.add_argument('--plot_lim', type=float, default=0.5) 84 | parser.add_argument('--plot_size', type=float, default=5) 85 | parser.add_argument('--iter_plot_freq', type=int, default=1) 86 | parser.add_argument('--restore', action='store_true') 87 | parser.add_argument('--checkpoint') 88 | args = parser.parse_args() 89 | 90 | global_step = tf.Variable(0, trainable=False, name='global_step') 91 | learning_rate = tf.train.exponential_decay(args.init_lr, global_step * args.batch_size, 92 | args.lr_decay_steps, args.lr_decay_rate, 93 | staircase=True, name='learning_rate') 94 | learning_rate = tf.maximum(learning_rate, args.lr_clip) 95 | bn_decay = 1 - tf.train.exponential_decay(args.init_bn_decay, global_step * args.batch_size, 96 | args.bn_decay_decay_steps, args.bn_decay_decay_rate, 97 | staircase=True, name='bn_decay') 98 | bn_decay = tf.minimum(bn_decay, args.bn_decay_clip) 99 | 100 | is_training_pl = tf.placeholder(tf.bool, (), 'is_training') 101 | points_pl = tf.placeholder(tf.float32, (args.batch_size, args.num_points, 3), 'points') 102 | labels_pl = tf.placeholder(tf.int32, (args.batch_size, args.num_points), 'labels') 103 | cat_labels_pl = tf.placeholder(tf.int32, (args.batch_size,), 'cat_labels') 104 | 105 | with tf.variable_scope('transformer', reuse=tf.AUTO_REUSE): 106 | transformer = importlib.import_module(args.transformer) 107 | transformed_points, T_out, Ts = transformer.get_model(points_pl, args.n_iter, 108 | is_training_pl, bn_decay) 109 | with tf.variable_scope('segmenter'): 110 | segmenter = importlib.import_module(args.segmenter) 111 | logits = segmenter.get_model(transformed_points, cat_labels_pl, is_training_pl, bn_decay) 112 | prediction = tf.argmax(logits, axis=2) 113 | 114 | loss_op = segmenter.get_loss(logits, labels_pl) 115 | 116 | trainer = tf.train.AdamOptimizer(learning_rate) 117 | grad, var = zip(*trainer.compute_gradients(loss_op, tf.trainable_variables())) 118 | grad, global_norm = tf.clip_by_global_norm(grad, args.grad_clip) 119 | train_op = trainer.apply_gradients(zip(grad, var), global_step) 120 | 121 | avg_loss, loss_update = tf.metrics.mean(loss_op) 122 | avg_acc, acc_update = tf.metrics.accuracy(labels_pl, prediction) 123 | avg_iou, iou_update = tf.metrics.mean_iou(labels_pl, prediction, n_parts) 124 | 125 | tf.summary.scalar('train/learning rate', learning_rate, collections=['train']) 126 | tf.summary.scalar('train/gradient norm', global_norm, collections=['train']) 127 | tf.summary.scalar('train/bn decay', bn_decay, collections=['train']) 128 | tf.summary.scalar('valid/loss', avg_loss, collections=['valid']) 129 | tf.summary.scalar('valid/accuracy', avg_acc, collections=['valid']) 130 | tf.summary.scalar('valid/mean iou', avg_iou, collections=['valid']) 131 | update_ops = [loss_update, acc_update, iou_update] 132 | train_summary = tf.summary.merge_all('train') 133 | valid_summary = tf.summary.merge_all('valid') 134 | 135 | lmdb_train = os.path.join(args.data_dir, 'train.lmdb') 136 | lmdb_valid = os.path.join(args.data_dir, 'valid.lmdb') 137 | df_train, num_train = lmdb_dataflow(lmdb_train, args.batch_size, args.num_points, 138 | shuffle=True, render=True, task='seg') 139 | df_valid, num_valid = lmdb_dataflow(lmdb_valid, args.batch_size, args.num_points, 140 | shuffle=False, task='seg') 141 | train_gen = df_train.get_data() 142 | valid_gen = df_valid.get_data() 143 | 144 | config = tf.ConfigProto() 145 | config.gpu_options.allow_growth = True 146 | config.allow_soft_placement = True 147 | sess = tf.Session(config=config) 148 | sess.run(tf.global_variables_initializer()) 149 | 150 | writer = create_log_dir(args, sess) 151 | 152 | saver = tf.train.Saver() 153 | step = sess.run(global_step) 154 | epoch = step * args.batch_size // num_train + 1 155 | next_epoch = epoch 156 | while next_epoch <= args.max_epoch: 157 | step += 1 158 | epoch = step * args.batch_size // num_train + 1 159 | model_id, points, labels, cat_labels, poses = next(train_gen) 160 | _, loss, summary = sess.run([train_op, loss_op, train_summary], 161 | feed_dict={is_training_pl: True, 162 | points_pl: points, 163 | labels_pl: labels, 164 | cat_labels_pl: cat_labels}) 165 | writer.add_summary(summary, step) 166 | if step % args.steps_per_print == 0: 167 | print('Epoch %d Step %d Loss %f' % (epoch, step, loss)) 168 | if step % args.steps_per_eval == 0: 169 | sess.run(tf.local_variables_initializer()) 170 | for i in range(num_valid // args.batch_size): 171 | instance_id, points, labels, cat_labels, poses = next(valid_gen) 172 | ts, pred, _ = sess.run([Ts, prediction, update_ops], 173 | feed_dict={is_training_pl: False, 174 | points_pl: points, 175 | labels_pl: labels, 176 | cat_labels_pl: cat_labels}) 177 | if step % args.steps_per_plot == 0 and (i + 1) % args.val_steps_per_plot == 0: 178 | synset_id = instance_id[0].split('_')[0] 179 | T = np.eye(4) 180 | transforms = [] 181 | part_ids = [] 182 | titles = [] 183 | for j in range(args.n_iter+1): 184 | if j > 0: 185 | T = np.dot(ts[j-1][0], T) 186 | if j % args.iter_plot_freq == 0: 187 | transforms.append(T) 188 | part_ids.append(pred[0]) 189 | titles.append('Iteration %d' % j) 190 | titles[0] = 'Input' 191 | transforms.append(T) 192 | part_ids.append(labels[0]) 193 | titles.append('Ground truth') 194 | acc = np.sum(pred[0] == labels[0]) / args.num_points 195 | iou = mean_iou(pred[0], labels[0], n_parts) 196 | suptitle = 'Accuracy %.4f Mean IOU %.4f' % (acc, iou) 197 | figpath = '%s/plots/%s_step_%d.png' % (args.log_dir, instance_id[0], step) 198 | plot_iters_seg(figpath, points[0], transforms, part_ids, titles, 199 | args.plot_lim, args.plot_size, suptitle) 200 | loss, acc, iou = sess.run([avg_loss, avg_acc, avg_iou]) 201 | print(colored('Loss %f Accuracy %.4f Mean IOU %.4f' % (loss, acc, iou), 'white', 'on_blue')) 202 | summary = sess.run(valid_summary) 203 | writer.add_summary(summary, step) 204 | next_epoch = (step+1) * args.batch_size // num_train + 1 205 | if epoch % args.epoch_per_save == 0 and epoch < next_epoch: 206 | saver.save(sess, os.path.join(args.log_dir, 'model_epoch_%d' % epoch)) 207 | -------------------------------------------------------------------------------- /transformer/it_net.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import sys 26 | import os 27 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 28 | sys.path.append(os.path.join(BASE_DIR, '../util')) 29 | import tensorflow as tf 30 | import tf_util 31 | 32 | 33 | def input_transform_net(point_cloud, is_training, bn_decay, randinit): 34 | """ Input (XYZ) Transform Net, input is BxNx3 gray image 35 | Return: Transformation matrix of size 3xK """ 36 | point_cloud = tf.expand_dims(point_cloud, -2) 37 | net = tf_util.conv2d(point_cloud, 64, [1,1], 38 | padding='VALID', stride=[1,1], 39 | bn=True, is_training=is_training, 40 | scope='tconv1', bn_decay=bn_decay) 41 | net = tf_util.conv2d(net, 128, [1,1], 42 | padding='VALID', stride=[1,1], 43 | bn=True, is_training=is_training, 44 | scope='tconv2', bn_decay=bn_decay) 45 | net = tf_util.conv2d(net, 1024, [1,1], 46 | padding='VALID', stride=[1,1], 47 | bn=True, is_training=is_training, 48 | scope='tconv3', bn_decay=bn_decay) 49 | net = tf.squeeze(net, -2) 50 | 51 | net = tf.reduce_max(net, axis=1, name='tmaxpool') 52 | 53 | net = tf_util.fully_connected(net, 512, bn=True, is_training=is_training, 54 | scope='tfc1', bn_decay=bn_decay) 55 | net = tf_util.fully_connected(net, 256, bn=True, is_training=is_training, 56 | scope='tfc2', bn_decay=bn_decay) 57 | 58 | with tf.variable_scope('transform') as sc: 59 | weights_init = tf.contrib.layers.xavier_initializer() if randinit \ 60 | else tf.zeros_initializer() 61 | weights = tf.get_variable('weights', [256, 7], 62 | initializer=tf.zeros_initializer(), 63 | dtype=tf.float32) 64 | biases = tf.get_variable('biases', [7], 65 | initializer=tf.zeros_initializer(), 66 | dtype=tf.float32) 67 | if not randinit: 68 | biases = biases + tf.constant([1,0,0,0,0,0,0], dtype=tf.float32) 69 | transform = tf.matmul(net, weights) 70 | transform = tf.nn.bias_add(transform, biases) 71 | return transform 72 | 73 | 74 | def get_model(points, n_iter, is_training, bn_decay, randinit=False, nostop=False): 75 | T = tf.eye(4, batch_shape=(points.shape[0],)) 76 | T_deltas = [] 77 | for i in range(n_iter): 78 | transformed_points = tf_util.transform_points(points, T) 79 | if not nostop: 80 | transformed_points = tf.stop_gradient(transformed_points) 81 | qt = input_transform_net(transformed_points, is_training, bn_decay, randinit) 82 | T_delta = tf.map_fn(tf_util.qt2mat, qt, dtype=tf.float32) 83 | T_deltas.append(T_delta) 84 | T = tf.matmul(T_delta, T) 85 | transformed_points = tf_util.transform_points(points, T) 86 | return transformed_points, T, T_deltas 87 | -------------------------------------------------------------------------------- /transformer/it_net_dgcnn.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import sys 26 | import os 27 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 28 | sys.path.append(os.path.join(BASE_DIR, '../util')) 29 | import tensorflow as tf 30 | import tf_util 31 | 32 | 33 | def input_transform_net(edge_feature, is_training, bn_decay, randinit): 34 | """ Input (XYZ) Transform Net, input is BxNx3 gray image 35 | Return: Transformation matrix of size 3xK """ 36 | net = tf_util.conv2d(edge_feature, 64, [1,1], 37 | padding='VALID', stride=[1,1], 38 | bn=True, is_training=is_training, 39 | scope='tconv1', bn_decay=bn_decay) 40 | net = tf_util.conv2d(net, 128, [1,1], 41 | padding='VALID', stride=[1,1], 42 | bn=True, is_training=is_training, 43 | scope='tconv2', bn_decay=bn_decay) 44 | net = tf.reduce_max(net, axis=-2, keepdims=True) 45 | net = tf_util.conv2d(net, 1024, [1,1], 46 | padding='VALID', stride=[1,1], 47 | bn=True, is_training=is_training, 48 | scope='tconv3', bn_decay=bn_decay) 49 | net = tf.squeeze(net, -2) 50 | 51 | net = tf.reduce_max(net, axis=1, name='tmaxpool') 52 | 53 | net = tf_util.fully_connected(net, 512, bn=True, is_training=is_training, 54 | scope='tfc1', bn_decay=bn_decay) 55 | net = tf_util.fully_connected(net, 256, bn=True, is_training=is_training, 56 | scope='tfc2', bn_decay=bn_decay) 57 | 58 | with tf.variable_scope('transform') as sc: 59 | weights_init = tf.contrib.layers.xavier_initializer() if randinit \ 60 | else tf.zeros_initializer() 61 | weights = tf.get_variable('weights', [256, 7], 62 | initializer=tf.zeros_initializer(), 63 | dtype=tf.float32) 64 | biases = tf.get_variable('biases', [7], 65 | initializer=tf.zeros_initializer(), 66 | dtype=tf.float32) 67 | if not randinit: 68 | biases = biases + tf.constant([1,0,0,0,0,0,0], dtype=tf.float32) 69 | transform = tf.matmul(net, weights) 70 | transform = tf.nn.bias_add(transform, biases) 71 | return transform 72 | 73 | 74 | def get_model(points, n_iter, is_training, bn_decay, randinit=False, nostop=False, k=10): 75 | T = tf.eye(4, batch_shape=(points.shape[0],)) 76 | T_deltas = [] 77 | for i in range(n_iter): 78 | transformed_points = tf_util.transform_points(points, T) 79 | if not nostop: 80 | transformed_points = tf.stop_gradient(transformed_points) 81 | 82 | adj_matrix = tf_util.pairwise_distance(transformed_points) 83 | nn_idx = tf_util.knn(adj_matrix, k=k) 84 | edge_feature = tf_util.get_edge_feature(points, nn_idx=nn_idx, k=k) 85 | 86 | qt = input_transform_net(edge_feature, is_training, bn_decay, randinit) 87 | T_delta = tf.map_fn(tf_util.qt2mat, qt, dtype=tf.float32) 88 | T_deltas.append(T_delta) 89 | T = tf.matmul(T_delta, T) 90 | transformed_points = tf_util.transform_points(points, T) 91 | return transformed_points, T, T_deltas 92 | -------------------------------------------------------------------------------- /transformer/t_net.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import sys 26 | import os 27 | BASE_DIR = os.path.dirname(os.path.abspath(__file__)) 28 | sys.path.append(os.path.join(BASE_DIR, '../util')) 29 | import tensorflow as tf 30 | import tf_util 31 | 32 | 33 | def input_transform_net(point_cloud, is_training, bn_decay, randinit): 34 | """ Input (XYZ) Transform Net, input is BxNx3 gray image 35 | Return: Transformation matrix of size 3xK """ 36 | point_cloud = tf.expand_dims(point_cloud, -2) 37 | net = tf_util.conv2d(point_cloud, 64, [1,1], 38 | padding='VALID', stride=[1,1], 39 | bn=True, is_training=is_training, 40 | scope='tconv1', bn_decay=bn_decay) 41 | net = tf_util.conv2d(net, 128, [1,1], 42 | padding='VALID', stride=[1,1], 43 | bn=True, is_training=is_training, 44 | scope='tconv2', bn_decay=bn_decay) 45 | net = tf_util.conv2d(net, 1024, [1,1], 46 | padding='VALID', stride=[1,1], 47 | bn=True, is_training=is_training, 48 | scope='tconv3', bn_decay=bn_decay) 49 | net = tf.squeeze(net, -2) 50 | 51 | net = tf.reduce_max(net, axis=1, name='tmaxpool') 52 | 53 | net = tf_util.fully_connected(net, 512, bn=True, is_training=is_training, 54 | scope='tfc1', bn_decay=bn_decay) 55 | net = tf_util.fully_connected(net, 256, bn=True, is_training=is_training, 56 | scope='tfc2', bn_decay=bn_decay) 57 | 58 | with tf.variable_scope('transform') as sc: 59 | weights_init = tf.contrib.layers.xavier_initializer() if randinit \ 60 | else tf.zeros_initializer() 61 | weights = tf.get_variable('weights', [256, 9], 62 | initializer=tf.zeros_initializer(), 63 | dtype=tf.float32) 64 | biases = tf.get_variable('biases', [9], 65 | initializer=tf.zeros_initializer(), 66 | dtype=tf.float32) 67 | if not randinit: 68 | biases = biases + tf.constant([1,0,0,0,1,0,0,0,1], dtype=tf.float32) 69 | transform = tf.matmul(net, weights) 70 | transform = tf.nn.bias_add(transform, biases) 71 | return transform 72 | 73 | 74 | def get_model(points, n_iter, is_training, bn_decay, randinit=False, nostop=False): 75 | T = tf.eye(4, batch_shape=(points.shape[0],)) 76 | T_deltas = [] 77 | for i in range(n_iter): 78 | transformed_points = tf_util.transform_points(points, T) 79 | if not nostop: 80 | transformed_points = tf.stop_gradient(transformed_points) 81 | affine = input_transform_net(transformed_points, is_training, bn_decay, randinit) 82 | T_delta = tf.map_fn(tf_util.affine2mat, affine, dtype=tf.float32) 83 | T_deltas.append(T_delta) 84 | T = tf.matmul(T_delta, T) 85 | transformed_points = tf_util.transform_points(points, T) 86 | return transformed_points, T, T_deltas 87 | -------------------------------------------------------------------------------- /util/data_util.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import numpy as np 26 | from tensorpack import dataflow 27 | 28 | 29 | def random_idx(m, n): 30 | """Drop or duplicate points so that pcd has exactly n points""" 31 | perm = np.random.permutation(m) 32 | idx = perm.copy() 33 | while idx.shape[0] < n: 34 | idx = np.concatenate([idx, idx]) 35 | return idx[:n], perm 36 | 37 | 38 | def random_rotation(): 39 | axis = np.random.randn(3) 40 | axis /= np.linalg.norm(axis) 41 | angle = np.random.rand() * np.pi 42 | A = np.array([[0, -axis[2], axis[1]], [axis[2], 0, -axis[0]], [-axis[1], axis[0], 0]]) 43 | R = np.eye(3) + np.sin(angle) * A + (1 - np.cos(angle)) * np.dot(A, A) 44 | return R 45 | 46 | 47 | class VirtualRenderData(dataflow.ProxyDataFlow): 48 | def __init__(self, ds): 49 | super(VirtualRenderData, self).__init__(ds) 50 | 51 | def get_data(self): 52 | for model_id, points, labels, cat_label in self.ds.get_data(): 53 | pose = np.eye(4) 54 | R = random_rotation() 55 | pose[:3, :3] = R 56 | points = np.dot(points, R.T) 57 | d = np.ones((500, 500)) 58 | ids = -np.ones((500, 500), dtype=np.int) 59 | for i, point in enumerate(points): 60 | x = int((point[0] + 0.5) / 0.02) 61 | y = int((point[1] + 0.5) / 0.02) 62 | z = point[2] + 0.5 63 | if z < d[x, y]: 64 | d[x, y] = z 65 | ids[x, y] = i 66 | ids = np.ravel(ids[ids > 0]) 67 | points = points[ids] 68 | labels = labels[ids] 69 | pose[:3, 3] = -points.mean(axis=0) 70 | points -= points.mean(axis=0) 71 | yield model_id, points, labels, cat_label, pose 72 | 73 | 74 | class ResampleData(dataflow.ProxyDataFlow): 75 | def __init__(self, ds, num_points, task): 76 | super(ResampleData, self).__init__(ds) 77 | self.num_points = num_points 78 | self.task = task 79 | 80 | def get_data(self): 81 | for data in self.ds.get_data(): 82 | data = list(data) 83 | idx, _ = random_idx(data[1].shape[0], self.num_points) 84 | # For pose estimation, data = [id, points, pose] 85 | # For classfication, data = [id, points, cls_label, pose] 86 | # For segmentation, data = [id, points, part_labels, cls_label, pose] 87 | data[1] = data[1][idx] 88 | if self.task == 'seg': 89 | data[2] = data[2][idx] 90 | yield data 91 | 92 | 93 | def lmdb_dataflow(lmdb_path, batch_size, num_points, shuffle, task, render=False): 94 | df = dataflow.LMDBSerializer.load(lmdb_path, shuffle=False) 95 | size = df.size() 96 | if render: 97 | df = VirtualRenderData(df) 98 | if num_points is not None: 99 | df = ResampleData(df, num_points, task) 100 | if shuffle: 101 | df = dataflow.LocallyShuffleData(df, 1000) 102 | df = dataflow.PrefetchDataZMQ(df, 8) 103 | df = dataflow.BatchData(df, batch_size, use_list=True) 104 | df = dataflow.RepeatedData(df, -1) 105 | df.reset_state() 106 | return df, size 107 | -------------------------------------------------------------------------------- /util/log_util.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import os 26 | import tensorflow as tf 27 | from termcolor import colored 28 | 29 | 30 | def create_log_dir(args, sess): 31 | if args.restore: 32 | restorer = tf.train.Saver() 33 | if args.checkpoint is not None: 34 | restorer.restore(sess, args.checkpoint) 35 | else: 36 | restorer.restore(sess, tf.train.latest_checkpoint(args.log_dir)) 37 | writer = tf.summary.FileWriter(args.log_dir) 38 | else: 39 | if os.path.exists(args.log_dir): 40 | delete_key = input(colored('%s exists. Delete? [y (or enter)/N]' 41 | % args.log_dir, 'white', 'on_red')) 42 | if delete_key == 'y' or delete_key == "": 43 | os.system('rm -rf %s/*' % args.log_dir) 44 | os.makedirs(os.path.join(args.log_dir, 'plots')) 45 | else: 46 | os.makedirs(os.path.join(args.log_dir, 'plots')) 47 | with open(os.path.join(args.log_dir, 'args.txt'), 'w') as log: 48 | for arg in sorted(vars(args)): 49 | log.write(arg + ': ' + str(getattr(args, arg)) + '\n') 50 | writer = tf.summary.FileWriter(args.log_dir, sess.graph) 51 | return writer 52 | -------------------------------------------------------------------------------- /util/tf_util.py: -------------------------------------------------------------------------------- 1 | """ Wrapper functions for TensorFlow layers. 2 | 3 | Author: Charles R. Qi 4 | Date: November 2017 5 | 6 | Updated by Yue Wang and Yongbin Sun 7 | Date: Feburary 2018 8 | 9 | Updated by Wentao Yuan 10 | Date: May 2019 11 | """ 12 | 13 | import numpy as np 14 | import tensorflow as tf 15 | 16 | 17 | def _variable_on_cpu(name, shape, initializer, use_fp16=False): 18 | """Helper to create a Variable stored on CPU memory. 19 | Args: 20 | name: name of the variable 21 | shape: list of ints 22 | initializer: initializer for Variable 23 | Returns: 24 | Variable Tensor 25 | """ 26 | with tf.device("/cpu:0"): 27 | dtype = tf.float16 if use_fp16 else tf.float32 28 | var = tf.get_variable(name, shape, initializer=initializer, dtype=dtype) 29 | return var 30 | 31 | 32 | def _variable_with_weight_decay(name, shape, stddev, wd, use_xavier=True): 33 | """Helper to create an initialized Variable with weight decay. 34 | 35 | Note that the Variable is initialized with a truncated normal distribution. 36 | A weight decay is added only if one is specified. 37 | 38 | Args: 39 | name: name of the variable 40 | shape: list of ints 41 | stddev: standard deviation of a truncated Gaussian 42 | wd: add L2Loss weight decay multiplied by this float. If None, weight 43 | decay is not added for this Variable. 44 | use_xavier: bool, whether to use xavier initializer 45 | 46 | Returns: 47 | Variable Tensor 48 | """ 49 | if use_xavier: 50 | initializer = tf.contrib.layers.xavier_initializer() 51 | else: 52 | initializer = tf.truncated_normal_initializer(stddev=stddev) 53 | var = _variable_on_cpu(name, shape, initializer) 54 | if wd is not None: 55 | weight_decay = tf.multiply(tf.nn.l2_loss(var), wd, name='weight_loss') 56 | tf.add_to_collection('losses', weight_decay) 57 | return var 58 | 59 | 60 | def conv1d(inputs, 61 | num_output_channels, 62 | kernel_size, 63 | scope, 64 | stride=1, 65 | padding='SAME', 66 | data_format='NHWC', 67 | use_xavier=True, 68 | stddev=1e-3, 69 | weight_decay=None, 70 | activation_fn=tf.nn.relu, 71 | bn=False, 72 | bn_decay=None, 73 | is_training=None): 74 | """ 1D convolution with non-linear operation. 75 | 76 | Args: 77 | inputs: 3-D tensor variable BxLxC 78 | num_output_channels: int 79 | kernel_size: int 80 | scope: string 81 | stride: int 82 | padding: 'SAME' or 'VALID' 83 | data_format: 'NHWC' or 'NCHW' 84 | use_xavier: bool, use xavier_initializer if true 85 | stddev: float, stddev for truncated_normal init 86 | weight_decay: float 87 | activation_fn: function 88 | bn: bool, whether to use batch norm 89 | bn_decay: float or float tensor variable in [0,1] 90 | is_training: bool Tensor variable 91 | 92 | Returns: 93 | Variable tensor 94 | """ 95 | with tf.variable_scope(scope) as sc: 96 | assert(data_format=='NHWC' or data_format=='NCHW') 97 | if data_format == 'NHWC': 98 | num_in_channels = inputs.get_shape()[-1].value 99 | elif data_format=='NCHW': 100 | num_in_channels = inputs.get_shape()[1].value 101 | kernel_shape = [kernel_size, 102 | num_in_channels, num_output_channels] 103 | kernel = _variable_with_weight_decay('weights', 104 | shape=kernel_shape, 105 | use_xavier=use_xavier, 106 | stddev=stddev, 107 | wd=weight_decay) 108 | outputs = tf.nn.conv1d(inputs, kernel, 109 | stride=stride, 110 | padding=padding, 111 | data_format=data_format) 112 | biases = _variable_on_cpu('biases', [num_output_channels], 113 | tf.constant_initializer(0.0)) 114 | outputs = tf.nn.bias_add(outputs, biases, data_format=data_format) 115 | 116 | if bn: 117 | outputs = batch_norm_for_conv1d(outputs, is_training, 118 | bn_decay=bn_decay, scope='bn', 119 | data_format=data_format) 120 | 121 | if activation_fn is not None: 122 | outputs = activation_fn(outputs) 123 | return outputs 124 | 125 | 126 | def conv2d(inputs, 127 | num_output_channels, 128 | kernel_size, 129 | scope, 130 | stride=[1, 1], 131 | padding='SAME', 132 | data_format='NHWC', 133 | use_xavier=True, 134 | stddev=1e-3, 135 | weight_decay=None, 136 | activation_fn=tf.nn.relu, 137 | bn=False, 138 | bn_decay=None, 139 | is_training=None): 140 | """ 2D convolution with non-linear operation. 141 | 142 | Args: 143 | inputs: 4-D tensor variable BxHxWxC 144 | num_output_channels: int 145 | kernel_size: a list of 2 ints 146 | scope: string 147 | stride: a list of 2 ints 148 | padding: 'SAME' or 'VALID' 149 | data_format: 'NHWC' or 'NCHW' 150 | use_xavier: bool, use xavier_initializer if true 151 | stddev: float, stddev for truncated_normal init 152 | weight_decay: float 153 | activation_fn: function 154 | bn: bool, whether to use batch norm 155 | bn_decay: float or float tensor variable in [0,1] 156 | is_training: bool Tensor variable 157 | 158 | Returns: 159 | Variable tensor 160 | """ 161 | with tf.variable_scope(scope) as sc: 162 | kernel_h, kernel_w = kernel_size 163 | assert(data_format=='NHWC' or data_format=='NCHW') 164 | if data_format == 'NHWC': 165 | num_in_channels = inputs.get_shape()[-1].value 166 | elif data_format=='NCHW': 167 | num_in_channels = inputs.get_shape()[1].value 168 | kernel_shape = [kernel_h, kernel_w, 169 | num_in_channels, num_output_channels] 170 | kernel = _variable_with_weight_decay('weights', 171 | shape=kernel_shape, 172 | use_xavier=use_xavier, 173 | stddev=stddev, 174 | wd=weight_decay) 175 | stride_h, stride_w = stride 176 | outputs = tf.nn.conv2d(inputs, kernel, 177 | [1, stride_h, stride_w, 1], 178 | padding=padding, 179 | data_format=data_format) 180 | biases = _variable_on_cpu('biases', [num_output_channels], 181 | tf.constant_initializer(0.0)) 182 | outputs = tf.nn.bias_add(outputs, biases, data_format=data_format) 183 | 184 | if bn: 185 | outputs = batch_norm_for_conv2d(outputs, is_training, 186 | bn_decay=bn_decay, scope='bn', 187 | data_format=data_format) 188 | 189 | if activation_fn is not None: 190 | outputs = activation_fn(outputs) 191 | return outputs 192 | 193 | 194 | def fully_connected(inputs, 195 | num_outputs, 196 | scope, 197 | use_xavier=True, 198 | stddev=1e-3, 199 | weight_decay=None, 200 | activation_fn=tf.nn.relu, 201 | bn=False, 202 | bn_decay=None, 203 | is_training=None): 204 | """ Fully connected layer with non-linear operation. 205 | 206 | Args: 207 | inputs: 2-D tensor BxN 208 | num_outputs: int 209 | 210 | Returns: 211 | Variable tensor of size B x num_outputs. 212 | """ 213 | with tf.variable_scope(scope) as sc: 214 | num_input_units = inputs.get_shape()[-1].value 215 | weights = _variable_with_weight_decay('weights', 216 | shape=[num_input_units, num_outputs], 217 | use_xavier=use_xavier, 218 | stddev=stddev, 219 | wd=weight_decay) 220 | outputs = tf.matmul(inputs, weights) 221 | biases = _variable_on_cpu('biases', [num_outputs], 222 | tf.constant_initializer(0.0)) 223 | outputs = tf.nn.bias_add(outputs, biases) 224 | 225 | if bn: 226 | outputs = batch_norm_for_fc(outputs, is_training, bn_decay, 'bn') 227 | 228 | if activation_fn is not None: 229 | outputs = activation_fn(outputs) 230 | return outputs 231 | 232 | 233 | def batch_norm_template(inputs, is_training, scope, moments_dims_unused, bn_decay, data_format='NHWC'): 234 | """ Batch normalization on convolutional maps and beyond... 235 | Ref.: http://stackoverflow.com/questions/33949786/how-could-i-use-batch-normalization-in-tensorflow 236 | 237 | Args: 238 | inputs: Tensor, k-D input ... x C could be BC or BHWC or BDHWC 239 | is_training: boolean tf.Varialbe, true indicates training phase 240 | scope: string, variable scope 241 | moments_dims: a list of ints, indicating dimensions for moments calculation 242 | bn_decay: float or float tensor variable, controling moving average weight 243 | data_format: 'NHWC' or 'NCHW' 244 | Return: 245 | normed: batch-normalized maps 246 | """ 247 | bn_decay = bn_decay if bn_decay is not None else 0.9 248 | return tf.contrib.layers.batch_norm(inputs, 249 | center=True, scale=True, 250 | is_training=is_training, decay=bn_decay,updates_collections=None, 251 | scope=scope, 252 | data_format=data_format) 253 | 254 | 255 | def batch_norm_for_fc(inputs, is_training, bn_decay, scope): 256 | """ Batch normalization on FC data. 257 | 258 | Args: 259 | inputs: Tensor, 2D BxC input 260 | is_training: boolean tf.Varialbe, true indicates training phase 261 | bn_decay: float or float tensor variable, controling moving average weight 262 | scope: string, variable scope 263 | Return: 264 | normed: batch-normalized maps 265 | """ 266 | return batch_norm_template(inputs, is_training, scope, [0,], bn_decay) 267 | 268 | 269 | def batch_norm_for_conv1d(inputs, is_training, bn_decay, scope, data_format): 270 | """ Batch normalization on 1D convolutional maps. 271 | 272 | Args: 273 | inputs: Tensor, 3D BLC input maps 274 | is_training: boolean tf.Varialbe, true indicates training phase 275 | bn_decay: float or float tensor variable, controling moving average weight 276 | scope: string, variable scope 277 | data_format: 'NHWC' or 'NCHW' 278 | Return: 279 | normed: batch-normalized maps 280 | """ 281 | return batch_norm_template(inputs, is_training, scope, [0,1], bn_decay, data_format) 282 | 283 | 284 | 285 | def batch_norm_for_conv2d(inputs, is_training, bn_decay, scope, data_format): 286 | """ Batch normalization on 2D convolutional maps. 287 | 288 | Args: 289 | inputs: Tensor, 4D BHWC input maps 290 | is_training: boolean tf.Varialbe, true indicates training phase 291 | bn_decay: float or float tensor variable, controling moving average weight 292 | scope: string, variable scope 293 | data_format: 'NHWC' or 'NCHW' 294 | Return: 295 | normed: batch-normalized maps 296 | """ 297 | return batch_norm_template(inputs, is_training, scope, [0,1,2], bn_decay, data_format) 298 | 299 | 300 | def dropout(inputs, 301 | is_training, 302 | scope, 303 | keep_prob=0.5, 304 | noise_shape=None): 305 | """ Dropout layer. 306 | 307 | Args: 308 | inputs: tensor 309 | is_training: boolean tf.Variable 310 | scope: string 311 | keep_prob: float in [0,1] 312 | noise_shape: list of ints 313 | 314 | Returns: 315 | tensor variable 316 | """ 317 | with tf.variable_scope(scope) as sc: 318 | outputs = tf.cond(is_training, 319 | lambda: tf.nn.dropout(inputs, keep_prob, noise_shape), 320 | lambda: inputs) 321 | return outputs 322 | 323 | 324 | def pairwise_distance(point_cloud): 325 | """Compute pairwise distance of a point cloud. 326 | 327 | Args: 328 | point_cloud: tensor (batch_size, num_points, num_dims) 329 | 330 | Returns: 331 | pairwise distance: (batch_size, num_points, num_points) 332 | """ 333 | og_batch_size = point_cloud.get_shape().as_list()[0] 334 | point_cloud = tf.squeeze(point_cloud) 335 | if og_batch_size == 1: 336 | point_cloud = tf.expand_dims(point_cloud, 0) 337 | 338 | point_cloud_transpose = tf.transpose(point_cloud, perm=[0, 2, 1]) 339 | point_cloud_inner = tf.matmul(point_cloud, point_cloud_transpose) 340 | point_cloud_inner = -2*point_cloud_inner 341 | point_cloud_square = tf.reduce_sum(tf.square(point_cloud), axis=-1, keepdims=True) 342 | point_cloud_square_tranpose = tf.transpose(point_cloud_square, perm=[0, 2, 1]) 343 | return point_cloud_square + point_cloud_inner + point_cloud_square_tranpose 344 | 345 | 346 | def knn(adj_matrix, k=20): 347 | """Get KNN based on the pairwise distance. 348 | Args: 349 | pairwise distance: (batch_size, num_points, num_points) 350 | k: int 351 | 352 | Returns: 353 | nearest neighbors: (batch_size, num_points, k) 354 | """ 355 | neg_adj = -adj_matrix 356 | _, nn_idx = tf.nn.top_k(neg_adj, k=k) 357 | return nn_idx 358 | 359 | 360 | def get_edge_feature(point_cloud, nn_idx, k=20): 361 | """Construct edge feature for each point 362 | Args: 363 | point_cloud: (batch_size, num_points, 1, num_dims) 364 | nn_idx: (batch_size, num_points, k) 365 | k: int 366 | 367 | Returns: 368 | edge features: (batch_size, num_points, k, num_dims) 369 | """ 370 | og_batch_size = point_cloud.get_shape().as_list()[0] 371 | point_cloud = tf.squeeze(point_cloud) 372 | if og_batch_size == 1: 373 | point_cloud = tf.expand_dims(point_cloud, 0) 374 | 375 | point_cloud_central = point_cloud 376 | 377 | point_cloud_shape = point_cloud.get_shape() 378 | batch_size = point_cloud_shape[0].value 379 | num_points = point_cloud_shape[1].value 380 | num_dims = point_cloud_shape[2].value 381 | 382 | idx_ = tf.range(batch_size) * num_points 383 | idx_ = tf.reshape(idx_, [batch_size, 1, 1]) 384 | 385 | point_cloud_flat = tf.reshape(point_cloud, [-1, num_dims]) 386 | point_cloud_neighbors = tf.gather(point_cloud_flat, nn_idx+idx_) 387 | point_cloud_central = tf.expand_dims(point_cloud_central, axis=-2) 388 | 389 | point_cloud_central = tf.tile(point_cloud_central, [1, 1, k, 1]) 390 | 391 | edge_feature = tf.concat([point_cloud_central, point_cloud_neighbors-point_cloud_central], axis=-1) 392 | return edge_feature 393 | 394 | 395 | def affine2mat(A): 396 | T = [A[0], A[1], A[2], 0, A[3], A[4], A[5], 0, A[6], A[7], A[8], 0, 0, 0, 0, 1] 397 | return tf.reshape(T, (4, 4)) 398 | 399 | 400 | def qt2mat(qt): 401 | q = qt[:4] / tf.norm(qt[:4]) 402 | t = qt[4:] 403 | T = [q[0] ** 2 + q[1] ** 2 - q[2] ** 2 - q[3] ** 2, 404 | 2 * (q[1] * q[2] - q[0] * q[3]), 405 | 2 * (q[1] * q[3] + q[0] * q[2]), 406 | t[0], 407 | 2 * (q[1] * q[2] + q[0] * q[3]), 408 | q[0] ** 2 - q[1] ** 2 + q[2] ** 2 - q[3] ** 2, 409 | 2 * (q[2] * q[3] - q[0] * q[1]), 410 | t[1], 411 | 2 * (q[1] * q[3] - q[0] * q[2]), 412 | 2 * (q[2] * q[3] + q[0] * q[1]), 413 | q[0] ** 2 - q[1] ** 2 - q[2] ** 2 + q[3] ** 2, 414 | t[2], 415 | 0, 0, 0, 1] 416 | return tf.reshape(T, (4, 4)) 417 | 418 | 419 | def mat2qt(T): 420 | t = T[:3, :] 421 | R = T[:3, :3] 422 | w = tf.sqrt(1 + tf.trace(R)) / 2 423 | x = (R[2, 1] - R[1, 2]) / (4 * w) 424 | y = (R[0, 2] - R[2, 0]) / (4 * w) 425 | z = (R[1, 0] - R[0, 1]) / (4 * w) 426 | return tf.stack([w, x, y, z], 0) 427 | 428 | 429 | def transform_points(points, transform): 430 | if len(transform.shape) == 3: 431 | rot = transform[:, :3, :3] 432 | trans = transform[:, :3, 3] 433 | else: 434 | rot = transform[:3, :3] 435 | trans = transform[:3, 3] 436 | points = tf.matmul(points, rot, transpose_b=True) + tf.expand_dims(trans, axis=-2) 437 | return points 438 | 439 | 440 | def geometric_error(points, T, T_gt): 441 | trans_points = transform_points(points, T) 442 | trans_points_gt = transform_points(points, T_gt) 443 | dist = tf.norm(trans_points - trans_points_gt, axis=-1) 444 | return tf.reduce_mean(dist, axis=-1) 445 | 446 | 447 | def rotation_error(R, R_gt): 448 | cos_theta = (tf.trace(tf.matmul(R, tf.matrix_transpose(R_gt))) - 1) / 2 449 | return tf.acos(cos_theta) * 180 / np.pi 450 | 451 | 452 | def translation_error(t, t_gt): 453 | return tf.norm(t - t_gt, axis=-1) 454 | -------------------------------------------------------------------------------- /util/visu_util.py: -------------------------------------------------------------------------------- 1 | ''' 2 | MIT License 3 | 4 | Copyright (c) 2019 Wentao Yuan 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | ''' 24 | 25 | import numpy as np 26 | from matplotlib import pyplot as plt 27 | from mpl_toolkits.mplot3d import Axes3D 28 | 29 | part_colormap = np.array([[0.65, 0.95, 0.05], [0.35, 0.05, 0.35], [0.65, 0.35, 0.65], 30 | [0.95, 0.95, 0.65], [0.95, 0.65, 0.05], [0.35, 0.05, 0.05], [0.65, 0.05, 0.05], 31 | [0.65, 0.35, 0.95], [0.05, 0.05, 0.65], [0.65, 0.05, 0.35], [0.05, 0.35, 0.35], 32 | [0.65, 0.65, 0.35], [0.35, 0.95, 0.05], [0.05, 0.35, 0.65], [0.95, 0.95, 0.35], 33 | [0.65, 0.65, 0.65], [0.95, 0.95, 0.05], [0.65, 0.35, 0.05], [0.35, 0.65, 0.05], 34 | [0.95, 0.65, 0.95], [0.95, 0.35, 0.65], [0.05, 0.65, 0.95], [0.65, 0.95, 0.65], 35 | [0.95, 0.35, 0.95], [0.05, 0.05, 0.95], [0.65, 0.05, 0.95], [0.65, 0.05, 0.65], 36 | [0.35, 0.35, 0.95], [0.95, 0.95, 0.95], [0.05, 0.05, 0.05], [0.05, 0.35, 0.95], 37 | [0.65, 0.95, 0.95], [0.95, 0.05, 0.05], [0.35, 0.95, 0.35], [0.05, 0.35, 0.05], 38 | [0.05, 0.65, 0.35], [0.05, 0.95, 0.05], [0.95, 0.65, 0.65], [0.35, 0.95, 0.95], 39 | [0.05, 0.95, 0.35], [0.95, 0.35, 0.05], [0.65, 0.35, 0.35], [0.35, 0.95, 0.65], 40 | [0.35, 0.35, 0.65], [0.65, 0.95, 0.35], [0.05, 0.95, 0.65], [0.65, 0.65, 0.95], 41 | [0.35, 0.05, 0.95], [0.35, 0.65, 0.95], [0.35, 0.05, 0.65]]) 42 | 43 | 44 | def plot_pcd(ax, pcd, size, azim, elev, lim, color=None, cmap='Blues'): 45 | if color is None: 46 | color = pcd[:, 0] 47 | vmax = color.max() 48 | vmin = color.min() - (color.max() - color.min()) / 2 49 | ax.scatter(pcd[:, 0], pcd[:, 1], pcd[:, 2], c=color, s=size, cmap=cmap, vmin=vmin, vmax=vmax) 50 | else: 51 | ax.scatter(pcd[:, 0], pcd[:, 1], pcd[:, 2], c=color, s=size) 52 | ax.view_init(elev, azim) 53 | for axis in 'xyz': 54 | getattr(ax, 'set_{}lim'.format(axis))((-lim, lim)) 55 | 56 | 57 | def plot_iters(figpath, pcd, transforms, titles, lim, size, suptitle=None, axis_on=False): 58 | fig = plt.figure(figsize=(len(transforms) * 4, 3 * 4)) 59 | if not isinstance(lim, list): 60 | lim = [lim] * len(transforms) 61 | for i in range(3): 62 | azim = 80 * i - 30 63 | elev = 15 * (i+1) 64 | for j in range(len(transforms)): 65 | ax = fig.add_subplot(3, len(transforms), i*len(transforms)+j+1, projection='3d') 66 | pcd_trans = np.dot(np.concatenate([pcd, np.ones((pcd.shape[0], 1))], axis=1), transforms[j].T) 67 | plot_pcd(ax, pcd_trans, size, azim, elev, lim[j]) 68 | if axis_on: 69 | for axis in 'xyz': 70 | getattr(ax, 'set_{}ticks'.format(axis))([-lim[j], 0, lim[j]]) 71 | ax.tick_params(labelsize='large') 72 | else: 73 | ax.set_axis_off() 74 | if i == 0: 75 | ax.set_title(titles[j], fontsize=20) 76 | if suptitle is not None: 77 | plt.suptitle(suptitle) 78 | plt.subplots_adjust(left=0, right=1, bottom=0, top=0.9, wspace=0, hspace=0) 79 | fig.savefig(figpath) 80 | plt.close(fig) 81 | 82 | 83 | def plot_iters_seg(figpath, pcd, transforms, part_ids, titles, lim, size, suptitle=None, axis_on=False): 84 | fig = plt.figure(figsize=(len(transforms) * 4, 3 * 4)) 85 | if not isinstance(lim, list): 86 | lim = [lim] * len(transforms) 87 | for i in range(3): 88 | azim = 80 * i - 30 89 | elev = 15 * (i+1) 90 | for j in range(len(transforms)): 91 | ax = fig.add_subplot(3, len(transforms), i*len(transforms)+j+1, projection='3d') 92 | pcd_trans = np.dot(np.concatenate([pcd, np.ones((pcd.shape[0], 1))], axis=1), transforms[j].T) 93 | plot_pcd(ax, pcd_trans, size, azim, elev, lim[j], part_colormap[part_ids[j]]) 94 | if axis_on: 95 | for axis in 'xyz': 96 | getattr(ax, 'set_{}ticks'.format(axis))([-lim[j], 0, lim[j]]) 97 | ax.tick_params(labelsize='large') 98 | else: 99 | ax.set_axis_off() 100 | if i == 0: 101 | ax.set_title(titles[j], fontsize=20) 102 | if suptitle is not None: 103 | plt.suptitle(suptitle) 104 | plt.subplots_adjust(left=0, right=1, bottom=0, top=0.95, wspace=0, hspace=0) 105 | fig.savefig(figpath) 106 | plt.close(fig) 107 | 108 | 109 | def plot_conf(figpath, conf): 110 | plt.figure(figsize=(6, 6)) 111 | plt.matshow(conf) 112 | plt.colorbar() 113 | plt.xlabel('Ground truth', verticalalignment='top') 114 | plt.ylabel('Prediction') 115 | plt.subplots_adjust(left=0, right=1, bottom=0, top=1) 116 | plt.savefig(figpath) 117 | 118 | 119 | def plot_cdf(figpath, data, label): 120 | data = sorted(data) 121 | fig = plt.figure() 122 | plt.plot(data, np.arange(0, 1, 1 / len(data))) 123 | plt.ylabel('Percentage', fontsize=18) 124 | plt.tick_params(labelsize=18) 125 | plt.xlabel(label, fontsize=18) 126 | plt.subplots_adjust(left=0.16, right=0.98, bottom=0.13, top=0.98) 127 | plt.savefig(figpath) 128 | plt.close(fig) 129 | 130 | 131 | def plot_mean_std(figpath, x, y, label): 132 | mean = np.mean(y, axis=1) 133 | std = np.std(y, axis=1) 134 | fig = plt.figure() 135 | plt.plot(x, mean) 136 | plt.fill_between(x, mean-std, mean+std, alpha=0.3) 137 | plt.xlabel('Iteration', fontsize=18) 138 | plt.xticks(x) 139 | plt.ylabel(label, fontsize=18) 140 | plt.ylim(bottom=0) 141 | plt.tick_params(labelsize=18) 142 | plt.subplots_adjust(left=0.16, right=0.98, bottom=0.13, top=0.98) 143 | plt.savefig(figpath) 144 | plt.close(fig) 145 | --------------------------------------------------------------------------------