└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # SLAM_Resources 2 | Personal page of SLAM Resources to follow up current SLAM trends and papers. 3 | 4 | Inspired by [Event-based vision resources](https://github.com/uzh-rpg/event-based_vision_resources) 5 | 6 | Also reference pages are listed on [Pages collect resources for SLAM](#slamlist) 7 | 8 | **What I cannot create, I do not understand.** - richard feynman 9 | 10 | **Do the simplest thing that could possibly work** 11 | 12 | ## Table of Contents: 13 | - [Algorithms](#algorithms) 14 | - [Sensor Model](#models) 15 | - [Datasets and Simulators](#datasets) 16 | - [Calibration](#calibration) 17 | - [Evaluation](#evaluation) 18 | - [Workshops&Tutorials](#workshops) 19 | - [Survey](#survey) 20 | - [Papers](#papers) 21 | - [Deep Learning Related SLAM](#deepslam) 22 | - [Semantic SLAM - Object level SLAM](#semanticslam) 23 | - [Books](#books) 24 | - [Pages : collect resources for SLAM](#slamlist) 25 | - [Toolkit](#toolkit) 26 | - [Videos, Lectures](#lecture) 27 | - [Visualization](#visualization) 28 | ___ 29 |
30 | 31 | 32 | ## Algorithms 33 | ### Initialization 34 | - #### Homography 35 | - #### Fundemental 36 | - #### SFM 37 | - #### Visual-Inertial Alignment 38 | ### Tracking 39 | - #### Data Association : How to Define Data Selection, Match, Define Error 40 | - ##### Direct Dense 41 | - ##### Direct Sparse 42 | - ##### Feature (Sparse) 43 | - ###### Corner Selection 44 | - ###### Descriptors 45 | - ##### Feature Match 46 | - #### Motion Prior 47 | - ##### Constant Velocity Model 48 | - ##### Decaying Velocity Model 49 | - ##### [IMU Propagation](#imu) 50 | - ##### Using Prev Pose 51 | - #### Pose Estimation : How to minimize Error 52 | - ##### PnP : Perspective N Points 53 | - ##### Motion Only BA : **Coarse**-Fine 54 | - ##### Local BA : Coarse-**Fine** 55 | - ###### Sliding Window : Continous N Frame window 56 | - ###### Topological : Releated Keyframes 57 | 58 | ### Mapping 59 | - #### Map Type 60 | - #### Map Generation 61 | 62 | ### Global Consistency 63 | - #### Relocalization 64 | - #### Pose Graph Optimization : Loop Closure 65 | - #### Place Recognition 66 | 67 | ### Probabilistic Graphical Models 68 | - #### Factor Graph 69 | 70 | 71 | ## Sensor Models 72 | ### Camera Models & Undistorttion Models 73 | - [Image_Undistorter](https://github.com/ethz-asl/image_undistort) 74 | - [Camera Models](https://github.com/gaowenliang/-camera_model) - modified version of CamOdoCal 75 | - ROS Image Proc => Wiki Documentation of ROS [image pipeline](http://wiki.ros.org/image_pipeline/CameraInfo) 76 | 77 | ### IMU Models 78 | - #### Noise Model 79 | - #### IMU Propagation 80 | - #### IMU Preintegration 81 | 82 | 83 | ## Calibration 84 | ### Geometric Calibration : Reprojection Error 85 | - [GML: C++ Calibration Toolbox](https://graphics.cs.msu.ru/en/node/909) 86 | - [ROS camera calibration](http://wiki.ros.org/camera_calibration) 87 | - [Camera Calibration Toolbox for Matlab](http://www.vision.caltech.edu/bouguetj/calib_doc/) 88 | - [CamOdoCal](https://github.com/hengli/camodocal) 89 | - [OCamCalib: Omni-Camera Calibration](https://sites.google.com/site/scarabotix/ocamcalib-toolbox) 90 | ### Photometric Calibration 91 | - [TUM, Online Photometric calibration](https://github.com/tum-vision/online_photometric_calibration) 92 | ### Visual-Inertial Calibration : Reprojection Error + Extrinsic of Camera-IMU 93 | - [Kalibr](https://github.com/ethz-asl/kalibr) 94 | - [Vicalib](https://github.com/arpg/vicalib) 95 | - [Inervis Toolbox-Matlab](http://home.deec.uc.pt/~jlobo/InerVis_WebIndex/InerVis_Toolbox.html) 96 | ### Visual-Lidar Calibration : 97 | ### Lidar-IMU Calibration 98 | ### IMU Calibration - Not sure... 99 | - [IMUSensorModels-Data_Analysis_Tools](https://github.com/hanley6/IMUSensorModels) 100 | - [Kalibr_allan](https://github.com/rpng/kalibr_allan) 101 | - [NaveGO: an open-source MATLAB/GNU Octave toolbox for processing INS and performing IMU analysis](https://github.com/rodralez/NaveGo) 102 | - [imu_utils : ROS package tool to analyze the IMU performance](https://github.com/gaowenliang/imu_utils) 103 | 104 | 105 | ## Survey or Tutorial papers for slam users 106 | - [Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age](https://arxiv.org/pdf/1606.05830.pdf) 107 | - [Keyframe-based monocular SLAM: design, survey, and future directions](https://arxiv.org/pdf/1607.00470.pdf) 108 | - [Local Invariant Feature Detectors: A Survey](http://homes.esat.kuleuven.be/~tuytelaa/FT_survey_interestpoints08.pdf) 109 | - [Visual Odometry Part I: The First 30 Years and Fundamentals](https://www.ifi.uzh.ch/dam/jcr:5759a719-55db-4930-8051-4cc534f812b1/VO_Part_I_Scaramuzza.pdf) 110 | - [Visual odometry: Part II: Matching, robustness, optimization, and applications](http://www.zora.uzh.ch/71030/1/Fraundorfer_Scaramuzza_Visual_odometry.pdf) 111 | - [Visual simultaneous localization and mapping : a survey](https://www.researchgate.net/publication/234081012_Visual_Simultaneous_Localization_and_Mapping_A_Survey) 112 | - [Simultaneous Localization and mapping : a survey of current trends in Autonomous Driving](https://hal.archives-ouvertes.fr/hal-01615897/file/2017-simultaneous_localization_and_mapping_a_survey_of_current_trends_in_autonomous_driving.pdf) 113 | - [Visual SLAM Algorithms : a survey from 2010 to 2016](https://ipsjcva.springeropen.com/articles/10.1186/s41074-017-0027-2) 114 | 115 | 116 | ## Papers : ordered by year but not strictly ordered, not fully collected. 117 | - [Visual Odometry, Nister, CVPR 04](https://www.computer.org/csdl/proceedings/cvpr/2004/2158/01/01315094.pdf) 118 | - [Scalable monocular SLAM, E. Eade,T. Drummond, CVPR 06](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.141.7753&rep=rep1&type=pdf) 119 | - [Parallel Tracking and Mapping(PTAM) for Small AR Workspaces, Georg Klein, David Murray, ISMAR 07](https://www.robots.ox.ac.uk/~vgg/rg/papers/klein_murray__2007__ptam.pdf) 120 | - [MonoSLAM, AJ Davison, Reid, Molton, Stasse, PAMI 07](https://www.doc.ic.ac.uk/~ajd/Publications/davison_etal_pami2007.pdf) 121 | - [DTAM: Dense Tracking and Mapping in Real-Time, RA Newcombe, Steven J. Lovegrove, AJ Davison, ICCV 11](https://www.robots.ox.ac.uk/~vgg/rg/papers/newcombe_davison__2011__dtam.pdf) 122 | - Dense Visual SLAM for RGB-D Camera 123 | - [Semi-Dense Visual Odometry, J. Engel, J. Sturm, AJ Davision, ICCV 13](https://vision.in.tum.de/_media/spezial/bib/engel2013iccv.pdf) 124 | - [SVO: Fast Semi-Direct Monocular Visual Odometry, C Forster, M. Pizzoli, D. Scarammuzza, ICRA 14](https://www.ifi.uzh.ch/dam/jcr:e9b12a61-5dc8-48d2-a5f6-bd8ab49d1986/ICRA14_Forster.pdf) 125 | - [LSD-SLAM: Large-Scale Direct Monocular SLAM, J. Engel, T.Schoeps, AJ Davision, ECCV 14](https://vision.in.tum.de/_media/spezial/bib/engel14eccv.pdf) 126 | - [REMODE, M. Pizzoli, C. Forster, D. Scrammuza, ICRA 14](http://rpg.ifi.uzh.ch/docs/ICRA14_Pizzoli.pdf) 127 | - Dense Visual-Inertial Odometry for Tracking of Aggressive Motions 128 | - [ORB_SLAM, R. Mur-Artal, J. Montiel, JD Tardós, IEEE TRO 15](https://arxiv.org/pdf/1502.00956) 129 | - [OKVIS, S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, P.Furgale, IJRR 15](http://www.roboticsproceedings.org/rss09/p37.pdf) 130 | - [DPPTAM, Concha, Alejo and Civera, Javier, IROS 15](http://webdiis.unizar.es/~jcivera/papers/concha_civera_iros15.pdf) 131 | - [SOFT2 : Stereo odometry based on careful feature selection and tracking, I Cvišić, I Petrović, ECCV 15](http://www.cvlibs.net/projects/autonomous_vision_survey/literature/Cvisic2015ECMR.pdf) 132 | - [EVO: A Geometric Approach to Event-Based 6-DOF Parallal Tracking and Mapping in Real-time, H. Rebecq, T. Horstschaefer, G. Gallego, D. Scaramuzza, IEEE RA-L 16](http://rpg.ifi.uzh.ch/docs/RAL16_EVO.pdf) 133 | - [On-Manifold Preintegration for Real-Time VIO, C. Forster, L. Carlone, F. Dellaert, D. Scaramuzza, IEEE RA-L 17](http://rpg.ifi.uzh.ch/docs/TRO16_forster.pdf) 134 | - [ORB_SLAM2, R Mur-Artal, JD Tardós, IEEE TRO 17](https://arxiv.org/pdf/1610.06475.pdf) 135 | - [Direct Sparse Odometry, J. Engel, V. Kltun, AJ Davison, PAMI 17](https://vision.in.tum.de/research/vslam/dso) 136 | - [Real-time VIO for Event Cameras using Keyframe-based Nonlinear Optimization, H.Rebecq, T. Horstschaefer, D. Scaramuzza, BMVC 17](http://rpg.ifi.uzh.ch/docs/BMVC17_Rebecq.pdf) 137 | - ElasticFusion: Dense SLAM Without A Pose Graph 138 | - Dense RGB-D-Inertial SLAM with Map Deformations 139 | - [SVO for Monocular and Multi-Camera Systems, C. Forster, Z. Zhang, M. Gassner, M. Werlberger, D. Scaramuzza, IEEE TRO 17](http://rpg.ifi.uzh.ch/docs/TRO16_Forster-SVO.pdf) 140 | - [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, T. Qin, Tong and Li, Peiliang, Shen, Shaojie, IEEE TRO 18](https://arxiv.org/pdf/1708.03852) 141 | - [Ultimate SLAM? 142 | Combining Events, Images, and IMU for Robust 143 | Visual SLAM in HDR and High Speed Scenarios, T. Rosinol Vidal, H.Rebecq, T. Horstschaefer, D. Scaramuzza, IEEE RA-L 18](https://arxiv.org/pdf/1709.06310.pdf) 144 | - [Event-based, 6-DOF Camera Tracking from Photometric Depth Maps, Gallego, Jon E. A. Lund, E. Mueggler, H.Rebecq, T. Delbruck, D. Scaramuzza, PAMI 18](http://rpg.ifi.uzh.ch/docs/PAMI17_Gallego.pdf) 145 | - [Loosely-Coupled Semi-Direct Monocular SLAM, Seong Hun Lee and Javier Civera, IEEE Robotics and Automation Letters] 146 | (https://arxiv.org/pdf/1807.10073.pdf) 147 | 148 | 149 | ## Deep SLAM : Depth Estimation, Pose Estimation, Feature Matching, Backend etc... What ever use Deep Neural Network 150 | - [DeepVO: A Deep Learning approach for Monocular Visual Odometry, Vikram Mohanty, Shubh Agrawal, Shaswat Datta, Arna Ghosh, Vishnu D. Sharma, Debashish Chakravarty](https://arxiv.org/pdf/1611.06069.pdf) 151 | - [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction, CVPR, 2017, Keisuke Tateno, Federico Tombari, Iro Laina, Nassir Navab](http://openaccess.thecvf.com/content_cvpr_2017/papers/Tateno_CNN-SLAM_Real-Time_Dense_CVPR_2017_paper.pdf) 152 | - [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry, Nan Yang, Rui Wang, J¨org St¨uckler, Daniel Cremers](http://openaccess.thecvf.com/content_ECCV_2018/papers/Nan_Yang_Deep_Virtual_Stereo_ECCV_2018_paper.pdf) 153 | - [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://arxiv.org/pdf/1709.06841.pdf) 154 | - [SfMLearner++: Learning Monocular Depth & Ego-Motion using Meaningful Geometric Constraints, Vignesh Prasad, Brojeshwar Bhowmick](https://arxiv.org/pdf/1812.08370.pdf) 155 | - [CNN-SVO: Improving the Mapping in Semi-Direct Visual OdometryUsing Single-Image Depth Prediction, Shing Yan Loo, Ali Jahan, Amiri, Syamsiah Mashohor, Sai Hong Tang and Hong Zhang1](https://arxiv.org/pdf/1810.01011.pdf) 156 | - [Learning monocular visual odometry with dense 3D mapping from dense 3D flow, Cheng Zhao, Li Sun, Pulak Purkait, Tom Duckett and Rustam Stolkin1](https://arxiv.org/pdf/1803.02286.pdf) 157 | - [Learning to Prevent Monocular SLAM Failure using Reinforcement Learning, Vignesh Prasad, Karmesh Yadav, Rohitashva Singh Saurabh, Swapnil Daga, Nahas Pareekutty, K. Madhava Krishna. Balaraman Ravindran, Brojeshwar Bhowmick](https://arxiv.org/pdf/1812.09647.pdf) 158 | - CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM, Michael Bloesch, Jan Czarnowski, Ronald Clark, Stefan Leutenegger, Andrew J. Davison. 159 | - LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo. ECCV, 2018, Ronald Clark, Michael Bloesch, Jan Czarnowski, Stefan Leutenegger, Andrew J. Davison. 160 | - [DeepTAM: Deep Tracking and Mapping, Huizhong Zhou, Benjamin Ummenhofer, Thomas Brox](https://arxiv.org/pdf/1808.01900.pdf) 161 | - [Deep Auxiliary Learning for Visual Localization and Odometry, Abhinav Valada, Noha Radwan, Wolfram Burgard](http://ais.informatik.uni-freiburg.de/publications/papers/valada18icra.pdf) 162 | - [Mask-SLAM: Robust feature-based monocular SLAM by masking using semantic segmentation, CVPR 2018, Masaya Kaneko Kazuya Iwami Toru Ogawa Toshihiko Yamasaki Kiyoharu Aiza](http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w9/Kaneko_Mask-SLAM_Robust_Feature-Based_CVPR_2018_paper.pdf) 163 | - [MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network, Jian Jiao, Jichao Jiao, Yaokai Mo, Weilun Liu, Zhongliang Deng](https://arxiv.org/ftp/arxiv/papers/1811/1811.10964.pdf) 164 | - [Global Pose Estimation with an Attention-based Recurrent Network](https://arxiv.org/pdf/1802.06857.pdf) 165 | - [Geometric Consistency for Self-Supervised End-to-End Visual Odometry, CVPR 2018, Ganesh Iyer, J. Krishna Murthy, Gunshi Gupta1, K. Madhava Krishna1, Liam Paull](http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w9/Iyer_Geometric_Consistency_for_CVPR_2018_paper.pdf) 166 | - [DepthNet: A Recurrent Neural Network Architecture for Monocular Depth Prediction, CVPR 2018, Arun CS Kumar Suchendra M. Bhandarkar, Mukta Prasad](http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w9/Kumar_DepthNet_A_Recurrent_CVPR_2018_paper.pdf) 167 | - DeepFusion: Real-Time Dense 3D Reconstruction for Monocular SLAM using Single-View Depth and Gradient Predictions. ICRA, 2019, Tristan Laidlow, Jan Czarnowski, Stefan Leutenegger. 168 | - KO-Fusion: Dense Visual SLAM with Tightly-Coupled Kinematic and Odometric Tracking. ICRA, 2019, Charlie Houseago, Michael Bloesch, Stefan Leutenegger. 169 | - DF-SLAM: A Deep-Learning Enhanced Visual SLAM System based on Deep Local Features, Rong Kang, Xueming Li, Yang Liu, Xiao Liu, Jieqi Shi 170 | - 171 | 172 | 173 | 174 | ## Semantic SLAM, Object-level, Using Semantic Information 175 | - [Probabilistic Data Association for Semantic SLAM, Sean L. Bowman Nikolay Atanasov Kostas Daniilidis George J. Pappas](https://www.cis.upenn.edu/~kostas/mypub.dir/bowman17icra.pdf) 176 | - Fusion++: Volumetric Object-Level SLAM. 3DV, 2018, John McCormac, Ronald Clark, Michael Bloesch, Stefan Leutenegger, Andrew J. Davison. 177 | - [DynSLAM: Simultaneous Localization and Mapping in Dynamic Environments,Ioan Andrei Brsan and Peidong Liu and Marc Pollefeys and Andreas Geiger](https://arxiv.org/pdf/1806.05620.pdf) 178 | 179 | 180 | 181 | ## Evaluation 182 | - [Python package for evaluation of odometry and SLAM](https://github.com/MichaelGrupp/evo) 183 | - [uzh-rpg : rpg_trajectory_evaluation](https://github.com/uzh-rpg/rpg_trajectory_evaluation), [papers](http://rpg.ifi.uzh.ch/docs/IROS18_Zhang.pdf) 184 | - [TUM, useful tools for the RGBD benchmark](https://vision.in.tum.de/data/datasets/rgbd-dataset/tools) 185 | - [TUM, Matlab tools for evaluation](vision.in.tum.de/mono/evaluation_code_v2.zip), provided by [TUM, DSO : Direct Sparse Odometry](https://vision.in.tum.de/research/vslam/dso) 186 | 187 | 188 | ## Datasets and Simulators 189 | - [Awesome SLAM Dataset](https://sites.google.com/view/awesome-slam-datasets/) : 190 | - [2018 TUM Visual Inertial Dataset](https://vision.in.tum.de/data/datasets/visual-inertial-dataset) : Stereo,IMU,Calibrated(+Photometric) 191 | - [2018 MVSEC : Multi Vehicle Stereo Event Dataset](https://daniilidis-group.github.io/mvsec/) : Stereo, Event, IMU 192 | - [2016 TUM Mono Dataset](https://vision.in.tum.de/data/datasets/mono-dataset) : Mono,IMU,Photometric Calibration 193 | - [2016 RPG Event Dataset](http://rpg.ifi.uzh.ch/davis_data.html) : Mono,Event,IMU 194 | - [2016 EuRoC Dataset](http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) : Stereo,IMU 195 | - [2015 TUM Omni Dataset](https://vision.in.tum.de/data/datasets/omni-lsdslam) : Mono,Omni,IMU 196 | - [2014 ICL-NUIM Dataset](https://www.doc.ic.ac.uk/~ahanda/VaFRIC/iclnuim.html) : Mono,RGB-D 197 | - [2014 MRPT-MALAGA Dataset](https://www.mrpt.org/robotics_datasets) 198 | - [2013 KITTI Dataset](http://www.cvlibs.net/datasets/kitti/index.php) 199 | 200 | 201 | ## Workshops & Tutorials 202 | - [2014 CVPR Workshop and Tutorials](http://frc.ri.cmu.edu/~kaess/vslam_cvpr14/) 203 | - [2015 ICCV Imperial college Workshop](http://wp.doc.ic.ac.uk/thefutureofslam/) 204 | - [2016 ICRA SLAM Tutorials](http://www.dis.uniroma1.it/~labrococo/tutorial_icra_2016/) 205 | - 2017 CVPR Tutorials - pages removed 206 | - [2018 ECCV Visual Localization Workshop](https://sites.google.com/view/visual-localization-eccv-2018/home) 207 | - [2018 ECCV Workshop](https://eccv2018.org/program/workshops_tutorials/) 208 | - [2018 IROS Workshop - Unconventional Sensing and Processing 209 | for Robotic Visual Perception, No Material..](http://iros2018-uvsp.org/) 210 | - [2018 ECCV 3D Reconstruction meets Semantics](http://trimbot2020.webhosting.rug.nl/events/3drms/date-schedule/) 211 | - [2018 CVPR Tutorials - First Deep SLAM Workshop](http://visualslam.ai/) 212 | - 2018: http://cvpr2018.thecvf.com/program/tutorials 213 | - 2017: http://cvpr2017.thecvf.com/program/tutorials 214 | - 2016: http://cvpr2016.thecvf.com/program/tutorials 215 | - 2015: http://www.pamitc.org/cvpr15/tutorials.php 216 | - 2014: http://www.pamitc.org/cvpr14/tutorials.php 217 | - 2013: http://www.pamitc.org/cvpr13/tutorials.php 218 | 219 | 220 | ## Books 221 | - [slambook-no en,kr translation](), [source](https://github.com/gaoxiang12/slambook) 222 | 223 | 224 | ## resource pages that I refer to create this slam list pages 225 | - [awesome-SLAM-list](https://github.com/OpenSLAM/awesome-SLAM-list) 226 | - [SFM-Visual-SLAM](https://github.com/marknabil/SFM-Visual-SLAM) 227 | - [Event Vision Realted Resources - ETH Zurich](https://github.com/uzh-rpg/event-based_vision_resources) 228 | 229 | 230 | ## Toolkits and Libraries for SLAM 231 | ### Computer Vision 232 | - [OpenCV-Computer Vision](https://opencv.org/) 233 | - [MexOpenCV-Matlab mex functions for OpenCV](https://github.com/kyamagu/mexopencv) 234 | ### Mathmatics 235 | - [Eigen-Linear Algebra](http://eigen.tuxfamily.org/index.php?title=Main_Page) 236 | - [Sophus-Lie Groups using Eigen](https://github.com/strasdat/Sophus) 237 | ### Optimization Solver 238 | - [Ceres-NLLS Solver library](https://github.com/ceres-solver/ceres-solver) 239 | - [g2o: A General Framework for Graph Optimization](https://github.com/RainerKuemmerle/g2o) 240 | ### 3D Data Processing 241 | - [Open3D](http://www.open3d.org/) 242 | - [Point Cloud Library](http://pointclouds.org/) 243 | 244 | 245 | ## Lectures 246 | - [Multiple View Geometry, TUM, 2014](https://www.youtube.com/watch?v=RDkwklFGMfo&list=PLTBdjV_4f-EJn6udZ34tht9EVIW7lbeo4) 247 | - [Roboot Mapping, University Freiburg, 2013](https://www.youtube.com/watch?v=U6vr3iNrwRA&list=PLgnQpQtFTOGQrZ4O5QzbIHgl3b1JHimN_) 248 | ## Videos 249 | - [ARKit: Understanding ARKit Tracking and Dtection](https://developer.apple.com/videos/play/wwdc2018/610/) 250 | 251 | 252 | ## Visualization 253 | ### Visualize GN-Optimization 254 | ### Visualize Pose & 3D Map 255 | ### Visualize Tracking 256 | --------------------------------------------------------------------------------