├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Barry Kui 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 3d-deep-learning 2 | 3D Deep Learning works 3 | 4 | 5 | ## Tasks 6 | 7 | 8 | ### 3D Representation 9 | 10 | #### Spherical CNNs 11 | - Taco S. Cohen, Spherical CNNs, ICLR 2018 Best paper \[[paper](https://openreview.net/forum?id=Hkbd5xZRb)\] 12 | - Learning SO\(3\) Equivariant Representations with Spherical CNNs \[[paper](https://arxiv.org/pdf/1711.06721v2.pdf)] [[code](https://github.com/daniilidis-group/spherical-cnn)] 13 | - Deep Learning Advances on Different 3D Data 14 | Representations: A Survey \[[paper](https://arxiv.org/pdf/1808.01462.pdf)\] 15 | 16 | 17 | ### 3D Classification 18 | 19 | #### Datasets 20 | 21 | - [ModelNet10/40](http://3dshapenets.cs.princeton.edu) 22 | 23 | #### Networks 24 | 25 | - 3D CNN 26 | - [3D-DenseNet](https://github.com/barrykui/3ddensenet.torch) 27 | - Voxnet: A 3d convolutional neural network for real-time object recognition, IROS 2015. \[[code](https://github.com/dimatura/voxnet)\] \[[paper](http://arxiv.org/abs/1505.00880)\] 28 | - [3D-NIN, network in network] 29 | - VRN Ensemble, Generative and discriminative voxel modeling with convolutional neural networks, arxiv \[[paper](https://arxiv.org/pdf/1608.04236.pdf)] \[[code](https://github.com/ajbrock/Generative-and-Discriminative-Voxel-Modeling)\] 30 | - Voxception-Resnet Blocks 31 | - 2D CNN 32 | - MVCNN, Learned-Miller.Multi- view convolutional neural networks for 3d shape recognition, ICCV2015 \[[project](http://vis-www.cs.umass.edu/mvcnn/)\] \[[code](https://github.com/suhangpro/mvcnn)\] \[[paper](http://arxiv.org/abs/1505.00880)\]\[[data](http://maxwell.cs.umass.edu/mvcnn-data/)\] \[[video](http://vis-www.cs.umass.edu/mvcnn/docs/1694_video.mp4)\] 33 | - Point 34 | - PointNet \[[project](http://stanford.edu/~rqi/pointnet/)]\[[paper](http://arxiv.org/abs/1612.00593)]\[[code](https://github.com/charlesq34/pointnet)]\[[video](https://www.youtube.com/watch?v=Cge-hot0Oc0)][[slides](http://stanford.edu/~rqi/pointnet/docs/cvpr17_pointnet_slides.pdf)] 35 | - global pooling 36 | - T-net 37 | - PointNet++ \[[paper](https://arxiv.org/pdf/1706.02413.pdf)\] \[[code](https://github.com/charlesq34/pointnet2)] 38 | - sampling & grouping to learning local feature for fine-gaint objects 39 | - two PointNet 40 | - Graph/tree-based 41 | - Kd-Net, scape from cells: Deep kd- networks for the recognition of 3d point cloud models, arxiv2017 \[[paper](http://arxiv.org/abs/1704.01222)\] 42 | - kd-tree 43 | - Octnet: Learning deep 3d representations at high resolutions, CVPR2017 44 | - octree 45 | - O-cnn: Octree-based convolutional neural networks for 3d shape analysis, TOG2017 46 | - octree 47 | - SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 \[[paper](https://arxiv.org/abs/1803.04249)\] \[[code](https://github.com/lijx10/SO-Net)\] 48 | - point-to-node kNN search Self-Organizing Map \(SOM\) 49 | - KCNet, Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling, CVPR2018 \[[project](http://vis-www.cs.umass.edu/mvcnn/)\] \[[code](https://github.com/suhangpro/mvcnn)\] \[[paper](http://arxiv.org/abs/1505.00880)\]\[[data](http://maxwell.cs.umass.edu/mvcnn-data/)\] \[[video](http://vis-www.cs.umass.edu/mvcnn/docs/1694_video.mp4)\] 50 | - Kernel Correlation 51 | - Graph Pooling 52 | - 53 | 54 | ### 3D Segmentation 55 | 56 | #### Datasets 57 | 58 | - [HVSMR](http://segchd.csail.mit.edu/data.html) 59 | - [BRATS Data](https://sites.google.com/site/braintumorsegmentation/home/brats2015) 60 | - [ShapeNet]() 61 | 62 | #### Networks 63 | 64 | - HeartSeg, 3D-FC-Densenet, Automatic 3D Cardiovascular MR Segmentation with 65 | Densely-Connected Volumetric ConvNets - MICCAI 2017 - [[code](https://github.com/yulequan/HeartSeg)] 66 | - 3D-Unet \[[paper](http://lmb.informatik.uni-freiburg.de/Publications/2016/CABR16/cicek16miccai.pdf)] 67 | - ClusterNet: 3D Instance Segmentation in RGB-D Images \[[paper](https://arxiv.org/pdf/1807.08894.pdf)\] 68 | - PointNet \[[project](http://stanford.edu/~rqi/pointnet/)]\[[paper](http://arxiv.org/abs/1612.00593)]\[[code](https://github.com/charlesq34/pointnet)]\[[video](https://www.youtube.com/watch?v=Cge-hot0Oc0)][[slides](http://stanford.edu/~rqi/pointnet/docs/cvpr17_pointnet_slides.pdf)] 69 | - PointNet++ \[[paper](https://arxiv.org/pdf/1706.02413.pdf)\] \[[code](https://github.com/charlesq34/pointnet2)] 70 | - KCNet, Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling, CVPR2018 \[[project](http://vis-www.cs.umass.edu/mvcnn/)\] \[[code](https://github.com/suhangpro/mvcnn)\] \[[paper](http://arxiv.org/abs/1505.00880)\]\[[data](http://maxwell.cs.umass.edu/mvcnn-data/)\] \[[video](http://vis-www.cs.umass.edu/mvcnn/docs/1694_video.mp4)\] 71 | - SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 \[[paper](https://arxiv.org/abs/1803.04249)\] \[[code](https://github.com/lijx10/SO-Net)\] 72 | - 3D Shape Segmentation with Projective Convolutional Networks. CVPR2017. [`Project`](http://people.cs.umass.edu/~kalo/papers/shapepfcn/) [`Poster`](http://people.cs.umass.edu/~kalo/papers/shapepfcn/ShapePFCN_poster.pdf) [`Presentation`](http://people.cs.umass.edu/~kalo/papers/shapepfcn/ShapePFCN_poster.pdf) 73 | - nnU-Net: Breaking the Spell on Successful Medical Image Segmentation \[[paper](https://arxiv.org/pdf/1904.08128.pdf)\] \[[code](https://github.com/MIC-DKFZ/nnunet)] 74 | 75 | ### 3D Object Detection 76 | 77 | #### Datasets 78 | 79 | Data types: RGBD, Flow, Laser 80 | - [KITTI](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) 81 | - [KITTI Object Visualization Tool](https://github.com/barrykui/kitti_object_vis) 82 | 83 | #### Networks 84 | 85 | - MV3D, Multi-View 3D Object Detection Network for Autonomous Driving \[[paper](https://arxiv.org/pdf/1611.07759)\] [[code](https://github.com/bostondiditeam/MV3D)] 86 | - Avod, Joint 3D Proposal Generation and Object Detection from View Aggregation \[[paper](https://arxiv.org/abs/1712.02294)\] [[code](https://github.com/kujason/avod)] 87 | - F-PointNet, Frustum PointNets for 3D Object Detection from RGB-D Data \[[paper](https://arxiv.org/abs/1711.08488)\] \[[code](https://github.com/charlesq34/frustum-pointnets)\] 88 | - VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection \[[paper](https://arxiv.org/abs/1711.06396)\] 89 | - PIXOR: Real-time 3D Object Detection from Point Clouds - CVPR 2018 - \[[paper](http://openaccess.thecvf.com/content_cvpr_2018/papers/Yang_PIXOR_Real-Time_3D_CVPR_2018_paper.pdf)\] \[[code](https://github.com/charlesq34/frustum-pointnets)\] 90 | - 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection \[[paper](https://arxiv.org/abs/1608.07711)\] 91 | - 3D Bounding Box Estimation Using Deep Learning and Geometry \[[paper](https://arxiv.org/abs/1612.00496)\] \[[code](https://github.com/smallcorgi/3D-Deepbox)\] 92 | - [Learning 3D Object Orientations From Synthetic Images](http://cs231n.stanford.edu/reports/rqi_final_report.pdf) 93 | 94 | ### 3D Reconstruction & Generation 95 | 96 | #### Datasets 97 | 98 | Data types: RGBD, Flow, Laser 99 | - ShapeNet 100 | 101 | #### Networks 102 | 103 | - SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 \[[paper](https://arxiv.org/abs/1803.04249)\] \[[code](https://github.com/lijx10/SO-Net)\] 104 | - 3D-GAN \[[paper](https://arxiv.org/abs/1612.00496)\] \[[code](https://github.com/zck119/3dgan-release)\] 105 | - Generating 3D Adversarial Point Clouds \[[paper](https://arxiv.org/pdf/1809.07016.pdf)\] 106 | 107 | ### 3D Human Pose Estimation 108 | 109 | #### Datasets 110 | 111 | Data types: RGBD, Flow, Laser 112 | - ShapeNet 113 | 114 | #### Networks 115 | 116 | - Synthetic Occlusion Data Augmentation -2018 ECCV PoseTrack Challenge - \[[paper](https://arxiv.org/abs/1809.04987)\] \[[code](https://github.com/isarandi/synthetic-occlusion)\] 117 | - Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach - ICCV 2017 - \[[paper](https://arxiv.org/abs/1809.04987)\] \[[code](https://github.com/xingyizhou/pose-hg-3d)\] \[[code-pytorch](https://github.com/xingyizhou/Pytorch-pose-hg-3d) 118 | - 3D human pose estimation from depth maps using a deep combination of poses \[[paper](https://arxiv.org/pdf/1807.05389.pdf)\] 119 | 120 | 121 | ## CVPR2016 Tutorial: 3D Deep Learning with Marvin 122 | - [CVPR2016 Tutorial: 3D Deep Learning with Marvin](http://vision.princeton.edu/event/cvpr16/3DDeepLearning/) 123 | - [3D Shape Retrieval](https://shapenet.cs.stanford.edu/shrec16/) 124 | - [C3D](https://github.com/facebook/C3D), [website](http://www.cs.dartmouth.edu/~dutran/c3d/) 125 | - [Video Caffe(C3D)] [[code](https://github.com/chuckcho/video-caffe)] 126 | - [DeepMedic, Brain Lesion Segmentation] [[code(https://github.com/Kamnitsask/deepmedic)] 127 | - [3D Keypoint Detection and Feature Description](http://staffhome.ecm.uwa.edu.au/~00051632/page100.html) 128 | 129 | ## Codes and libs for 3D 130 | - [util3d](https://github.com/fyu/util3d) 131 | - [spectral-lib](https://github.com/mbhenaff/spectral-lib) 132 | - [3D-Caffe](https://github.com/yulequan/3D-Caffe#installation) 133 | - [3D-ResNets-PyTorch](https://github.com/kenshohara/3D-ResNets-PyTorch) 134 | 135 | 136 | ## DL on Medical Image 137 | - [Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5027738/) 138 | 139 | ## More 3D Papers 140 | 141 | see [ [3D-Machine-Learning](https://github.com/timzhang642/3D-Machine-Learning)] 142 | --------------------------------------------------------------------------------