├── figs ├── blensor.jpg └── teaser.png └── README.md /figs/blensor.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ai4ce/BAGSFit/HEAD/figs/blensor.jpg -------------------------------------------------------------------------------- /figs/teaser.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ai4ce/BAGSFit/HEAD/figs/teaser.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Primitive Fitting Using Deep Boundary Aware Geometric Segmentation 2 | 3 | 4 | By [Duanshun Li](https://duanshun.github.io/) and [Chen Feng](https://simbaforrest.github.io/). 5 | 6 | 7 | 8 | ## Introduction 9 | 10 | To identify and fit geometric primitives (e.g., planes, spheres, cylinders, cones) in a noisy point cloud 11 | is a challenging yet beneficial task for fields such as robotics and reverse engineering. As a multi-model 12 | multi-instance fitting problem, it has been tackled with different approaches including RANSAC, which however 13 | often fit inferior models in practice with noisy inputs of cluttered scenes. Inspired by the corresponding human 14 | recognition process, and benefiting from the recent advancements in image semantic segmentation using deep neural 15 | networks, we propose BAGSFit as a new framework addressing this problem. 16 | Firstly, through a fully convolutional neural network, the input point cloud is point-wisely 17 | segmented into multiple classes divided by jointly detected instance boundaries without any geometric fitting. 18 | Thus, segments can serve as primitive hypotheses with a probability estimation of associating primitive classes. 19 | Finally, all hypotheses are sent through a geometric verification to correct any misclassification by 20 | fitting primitives respectively. We performed training using simulated range images and tested it with both simulated 21 | and real-world point clouds. Quantitative and qualitative experiments demonstrated the superiority of BAGSFit. 22 | 23 | ![primfit](./figs/teaser.png) 24 | 25 | Primitive fitting on a simulated test range image (top left) with BAGSFit (top right) vs. RANSAC (top middle). 26 | Estimated normals (bottom left) and ground truth labels (bottom middle) are used to train a fully convolutional 27 | segmentation network in BAGSFit. During testing, a boundary-aware and thus instance-aware segmentation (bottom right) 28 | is predicted, and sent through a geometric verification to fit final primitives (randomly colored). Comparing with BAGSFit, 29 | the RANSAC-based method produces more misses and false detections of primitives (shown as transparent or wire-frame), 30 | and thus a less appealing visual result. 31 | 32 | ## Dataset 33 | 34 | ### Download 35 | 36 | The dataset used in our [paper](https://arxiv.org/pdf/1810.01604.pdf) is provided 37 | on [Google Drive](https://drive.google.com/open?id=1JNjK5eQaVQn7w_gyvd2nszg_qCHSjqvl). 38 | 39 | ### Details 40 | The dataset is created with [Blensor](http://www.blensor.org/) on simulated scenes. 41 | 42 | ![blensor](./figs/blensor.jpg) 43 | 44 | In our paper, the first 18 scenes in Train-20s dataset is used for training, 45 | the last two scenes are used for valication. 46 | The Test-20s dataset is used for testing. 47 | 48 | In each scene, there are several types of data: 49 | + The `.npz` files store the simulated data and their label; 50 | + The `primitives.prim` file stores the parameters of the objects in the scene; 51 | + The `cam.pos` file stores the pose of the camera for each image; 52 | + The `.prim` files record the parameters of the primitive in the camera's coordinate system. 53 | 54 | Each `npz` file has following fields: 55 | + `data`: the xyz coordinates of simulated data with noise; 56 | + `scan`: the xyz coordinates of simulated data without noise; 57 | + `ins`: the instance label 58 | + `cls`: the class label 59 | 60 | The primitives are parameterized with following parameters: 61 | + `Plane`: a point on the plane and its norm; i.e. x, y, z, nx, ny, nz. 62 | + `Sphere`: a point on the center and its radius; i.e. x, y, z, r. 63 | + `Cylinder`: a point on the axis, the axis, and the radius; i.e. x, y, z, nx, ny, nz, r. 64 | + `Cone`: the apex, the axis, and the open angle in radius; i.e. x, y, z, nx, ny, nz, \\theta 65 | 66 | ## Citation 67 | 68 | You can reference the following [article](https://arxiv.org/pdf/1810.01604.pdf) if the dataset is used. 69 | 70 | Li, Duanshun, and Chen Feng. "Primitive Fitting Using Deep Boundary Aware Geometric Segmentation." arXiv preprint arXiv:1810.01604 (2018). 71 | 72 | BibTex: 73 | ``` 74 | @article{li2018primitive, 75 | title={Primitive Fitting Using Deep Boundary Aware Geometric Segmentation}, 76 | author={Li, Duanshun and Feng, Chen}, 77 | journal={arXiv preprint arXiv:1810.01604}, 78 | year={2018} 79 | } 80 | ``` 81 | --------------------------------------------------------------------------------