└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # awesome-dynamic-slam 2 | 3 | The following papers are the papers that fits my PhD research interest. They are mainly tackling the real-time reconstruction of multiplle dynamic objects or non-rigid objects. Some papers are off-line work yet also very interesting. 4 | 5 | Due to my personal interests, SFM-based or factorization-based work are not collected here. Those papers can be found [here](https://github.com/openMVG/awesome_3DReconstruction_list). 6 | 7 | Another related paper list is about the semantic segmentation, which can be found [here](https://github.com/mrgloom/awesome-semantic-segmentation). 8 | 9 | 10 | 11 | ------ 12 | 13 | ## Related survey papers: 14 | 15 | - State of the Art in Real-time Registration of RGB-D Images ([link](http://cg.cs.uni-bonn.de/aigaion2root/attachments/StateoftheArtinReal-timeRegistrationofRGB-DImages.pdf)) 16 | - Visual SLAM and Structure from Motion in Dynamic Environments: A Survey ([link](https://dl.acm.org/citation.cfm?id=3177853)) 17 | - State of the Art on 3D Reconstruction with RGB-D Cameras ([link](https://web.stanford.edu/~zollhoef/papers/EG18_RecoSTAR/paper.pdf)) 18 | 19 | 20 | 21 | ## Multiple Rigid Object Tracking and Reconstruction (Real-time) 22 | 23 | - Joint 3D Reconstruction of a Static Scene and Moving Objects (3DV 2017) 24 | - http://www.merl.com/publications/docs/TR2017-147.pdf (paper) 25 | - Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects (ICRA 2017) 26 | - http://visual.cs.ucl.ac.uk/pubs/cofusion/ (website) 27 | - https://github.com/martinruenz/co-fusion (code) 28 | - http://visual.cs.ucl.ac.uk/pubs/cofusion/icra2017_co-fusion_print.pdf (paper) 29 | - Semantic segmentation is pre-computed 30 | 31 | 32 | - MaskFusion (submitted to ISMAR 2018) 33 | 34 | - https://arxiv.org/pdf/1804.09194.pdf (paper) 35 | - http://visual.cs.ucl.ac.uk/pubs/maskfusion/ (website) 36 | - Relevant: SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks (ICRA 2017) 37 | 38 | - https://arxiv.org/pdf/1606.02378.pdf (paper) 39 | - Video: https://www.youtube.com/watch?v=UA_MKLHCWSo 40 | - Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation (ECCV 2018) 41 | 42 | - paper: https://arxiv.org/abs/1804.04259 43 | - slides: http://on-demand.gputechconf.com/gtc/2018/presentation/s8798-learning-rigidity-in-dynamic-scenes-for-scene-flow-estimation-v2.pdf 44 | - video: https://www.youtube.com/watch?v=MnTHkOCY790&feature=youtu.be 45 | - web: http://research.nvidia.com/publication/2018-09_Learning-Rigidity-in 46 | - SfM-Net: Learning of Structure and Motion from Video 47 | 48 | - https://arxiv.org/pdf/1704.07804.pdf 49 | - Stereo Vision-based Semantic 3D Object and Ego-motion Tracking for Autonomous Driving (ECCV2018) 50 | 51 | 52 | - https://arxiv.org/pdf/1807.02062.pdf (paper) 53 | - Estimating metric poses of dynamic objects using monocular visual-inertial fusion (IROS 2018) 54 | 55 | 56 | - https://arxiv.org/pdf/1807.02062.pdf (paper) 57 | - Real-Time Object Pose Estimation with Pose Interpreter Networks (IROS2018) 58 | 59 | - https://arxiv.org/pdf/1808.01099.pdf (paper) 60 | - https://github.com/jimmyyhwu/pose-interpreter-networks (code) 61 | 62 | 63 | 64 | ## Non-rigid body Tracking and Reconstruction (Real-time) 65 | 66 | - DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time (CVPR 2015) 67 | - http://grail.cs.washington.edu/projects/dynamicfusion/papers/DynamicFusion.pdf (paper) 68 | - 3D Scanning Deformable Objects with a Single RGBD Sensor (CVPR 2015) 69 | - http://www.cs.unc.edu/~doums/pdfs/dkf.pdf (paper) 70 | - VolumeDeform: Real-time Volumetric Non-rigid Reconstruction (ECCV2016) 71 | - https://graphics.stanford.edu/~niessner/papers/2016/5volumeDeform/innmann2016deform.pdf (paper) 72 | - Fusion4D: Real-time Performance Capture of Challenging Scenes (Siggraph 2016) 73 | - http://www.samehkhamis.com/dou-siggraph2016.pdf (paper) 74 | - Real-time Geometry, Albedo and Motion Reconstruction Using a Single RGBD Camera (Siggraph 2017) 75 | - http://www.liuyebin.com/monofvv/monofvv_files/MonoFVV.pdf (paper) 76 | - http://www.liuyebin.com/monofvv/monofvv.html (website) 77 | - KillingFusion: Non-rigid 3D Reconstruction without Correspondences (CVPR 2017) 78 | - http://campar.in.tum.de/pub/slavcheva2017cvpr/slavcheva2017cvpr.pdf (paper) 79 | - SobolevFusion: 3D Reconstruction of Scenes Undergoing Free Non-rigid Motion (CVPR 2018) 80 | - http://campar.in.tum.de/pub/slavcheva2018cvpr/slavcheva2018cvpr.pdf (paper) 81 | - MixedFusion: Real-Time Reconstruction of an Indoor Scene with Dynamic Objects (TVCG 2017) 82 | - http://feng-xu.com/papers/MixedFusion-tvcg.pdf (paper) 83 | 84 | 85 | ## Background only, i.e. removing dynamic objects 86 | 87 | - Fast Odometry and Scene Flow from RGB-D Cameras based on Geometric Clustering (ICRA 2017) 88 | - http://mapir.isa.uma.es/mjaimez/Papers/Jaimez_et_al_VOSF_2017.pdf (paper) 89 | - https://github.com/MarianoJT88/Joint-VO-SF (code) 90 | - http://mapir.isa.uma.es/mapirwebsite/index.php?option=com_content&view=article&layout=edit&id=241 (website) 91 | - StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments (ICRA 2018) 92 | - http://www.robots.ox.ac.uk/~mobile/Papers/2018ICRA_scona.pdf (paper) 93 | - https://www.youtube.com/watch?v=UVsqIgxHoBM (video) 94 | - Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments (ICRA 2018) 95 | - https://arxiv.org/pdf/1711.06623.pdf (paper) 96 | - Detecting, Tracking and Eliminating Dynamic Objects in 3D Mapping using Deep Learning and Inpainting (ICRA 2018) 97 | - https://natanaso.github.io/rcw-icra18/assets/ref/ICRA-MRP18_paper_3.pdf (paper) 98 | 99 | 100 | ## Non-realtime: non-rigid reconstruction 101 | 102 | - Video Pop-up : Monocular 3D Reconstruction of Dynamic Scenes SCENE RECONSTRUCTION WITH AN ADAPTIVE NEIGHBOURHOOD (ECCV 2014) 103 | - http://discovery.ucl.ac.uk/1454051/1/Russell_new_paper.pdf (paper) 104 | - https://github.com/cvfish/VideoPopup (code) 105 | - http://www0.cs.ucl.ac.uk/staff/R.Yu/video_popup/VideoPopup2.html (website) 106 | - Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization (ICCV 2015) 107 | - http://media.au.tsinghua.edu.cn/liuyebin_files/nonrigid/nonrigid.pdf (paper) 108 | - Code available by sending mail to the author 109 | - http://media.au.tsinghua.edu.cn/nonrigid.html (website) 110 | 111 | 112 | 113 | 114 | 115 | ## Non-realtime: multiple object segmentation-tracking-reconstruction 116 | 117 | - Dense Multibody Motion Estimation and Reconstruction from a Handheld Camera (ISMAR 2012) 118 | - https://www.doc.ic.ac.uk/~aroussos/RoussosRussellGargAgapito_DenseMultibody_ISMAR12.pdf (paper) 119 | - Robust Dense Mapping for Large-Scale Dynamic Environments (ICRA 2018) 120 | - http://siegedog.com/dynslam/ (website) 121 | - http://siegedog.com/assets/dynslam/robust-dense-mapping-paper-submission.pdf (paper) 122 | - https://github.com/AndreiBarsan/DynSLAM (code) 123 | - pre-compute the semantic segmentation (cars) 124 | - Multimotion Visual Odometry (MVO): Simultaneous Estimation of Camera and Third-Party Motions (IROS 2018) 125 | - https://arxiv.org/pdf/1808.00274.pdf (paper) 126 | 127 | 128 | 129 | 130 | 131 | ## Learn Depth and camera motion 132 | 133 | - DeMoN: Depth and Motion Network for Learning Monocular Stereo 134 | 135 | - Unsupervised Learning of Depth and Ego-Motion from Video 136 | 137 | - Multi-view Supervision for Single-view Reconstructionvia Differentiable Ray Consistency 138 | 139 | 140 | 141 | 142 | 143 | ## Learn 3D map 144 | 145 | - OctNet: Learning Deep 3D Representations at High Resolutions --------------------------------------------------------------------------------