├── README.md ├── interesting.md └── papers.md /README.md: -------------------------------------------------------------------------------- 1 | # Awesome Robotics 2 | 3 | [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) 4 | 5 | Awesome links, software libraries, [papers](papers.md), and other [intersting links](interesting.md) that are useful for robots. 6 | 7 | 8 | Relevant Awesome Lists 9 | ---------------------- 10 | 11 | - [Kiloreaux/awesome-robotics](https://github.com/Kiloreux/awesome-robotics) - Learn about Robotics. 12 | - [Robotics Libraries](https://github.com/jslee02/awesome-robotics-libraries) - Another list of awesome robotics libraries. 13 | - [Robotics Coursework](https://github.com/mithi/robotics-coursework) - A list of robotics courses you can take online 14 | - [Computer Vision](https://github.com/jbhuang0604/awesome-computer-vision) 15 | - [Deep Learning](https://github.com/ChristosChristofidis/awesome-deep-learning) - Neural networks. 16 | - [TensorFlow](https://github.com/jtoy/awesome-tensorflow) - Library for machine intelligence. 17 | - [Papers](https://github.com/terryum/awesome-deep-learning-papers) - The most cited deep learning papers. 18 | - [Deep Vision](https://github.com/kjw0612/awesome-deep-vision) - Deep learning for computer vision 19 | - [Data Visualization](https://github.com/fasouto/awesome-dataviz) - See what your robot is doing with any programming language. 20 | - [paperswithcode state of the art](https://paperswithcode.com/sota) - List of state of the art results on various machine learning benchmarks. 21 | 22 | Simulators 23 | ---------- 24 | 25 | - [CoppeliaSim](coppeliarobotics.com/index.html) - Create, Simulate, any Robot. (formerly named V-REP) 26 | - [Microsoft Airsim](https://github.com/Microsoft/AirSim) - Open source simulator based on Unreal Engine for autonomous vehicles from Microsoft AI & Research. 27 | - [Bullet Physics SDK](https://github.com/bulletphysics/bullet3) - Real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc. Also see [pybullet](https://pybullet.org). 28 | 29 | Visualization, Video, Display, and Rendering 30 | ----------------------- 31 | 32 | - [Pangolin](https://github.com/stevenlovegrove/Pangolin) - A lightweight portable rapid development library for managing OpenGL display / interaction and abstracting video input. 33 | - [PlotJuggler](https://github.com/facontidavide/PlotJuggler) - Quickly plot and re-plot data on the fly! Includes optional ROS integration. 34 | - [Data Visualization](https://github.com/fasouto/awesome-dataviz) - A list of awesome data visualization tools. 35 | 36 | Machine Learning 37 | ---------------- 38 | 39 | ### TensorFlow related 40 | 41 | - [Keras](https://keras.io) - Deep Learning library for Python. Convnets, recurrent neural networks, and more. Runs on TensorFlow or Theano. 42 | - [keras-contrib](https://github.com/farizrahman4u/keras-contrib) - Keras community contributions. 43 | - [TensorFlow](tensorflow.org) - An open-source software library for Machine Intelligence. 44 | - [recurrentshop](https://github.com/datalogai/recurrentshop) - Framework for building complex recurrent neural networks with Keras. 45 | - [tensorpack](https://github.com/ppwwyyxx/tensorpack) - Neural Network Toolbox on TensorFlow. 46 | - [tensorlayer](https://github.com/zsdonghao/tensorlayer) - Deep Learning and Reinforcement Learning Library for Researchers and Engineers. 47 | - [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) - TensorFlow Tutorial and Examples for beginners. 48 | - [hyperas](https://github.com/maxpumperla/hyperas) - Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization. 49 | - [elephas](https://github.com/maxpumperla/elephas) - Distributed Deep learning with Keras & Spark 50 | - [PipelineAI](https://github.com/fluxcapacitor/pipeline) - End-to-End ML and AI Platform for Real-time Spark and Tensorflow Data Pipelines. 51 | - [sonnet](https://github.com/deepmind/sonnet) - Google Deepmind APIs on top of TensorFlow. 52 | - [visipedia/tfrecords](https://github.com/visipedia/tfrecords) - Demonstrates the use of TensorFlow's TFRecord data format. 53 | 54 | #### Image Segmentation 55 | 56 | - [tf-image-segmentation](https://github.com/warmspringwinds/tf-image-segmentation) - Image Segmentation framework based on Tensorflow and TF-Slim library. 57 | - [Keras-FCN](https://github.com/aurora95/Keras-FCN) 58 | 59 | 60 | Logging and Messaging 61 | --------------------- 62 | 63 | - [spdlog](https://github.com/gabime/spdlog) - Super fast C++ logging library. 64 | - [lcm](https://github.com/lcm-proj/lcm) - Lightweight Communications and Marshalling, message passing and data marshalling for real-time systems where high-bandwidth and low latency are critical. 65 | 66 | Tracking 67 | -------- 68 | 69 | - [simtrack](https://github.com/karlpauwels/simtrack) - A simulation-based framework for tracking. 70 | - [ar_track_alvar](https://github.com/sniekum/ar_track_alvar) - AR tag tracking library for ROS. 71 | - [artoolkit5](https://github.com/artoolkit/artoolkit5) - Augmented Reality Toolkit, which has excellent AR tag tracking software. 72 | 73 | Robot Operating System (ROS) 74 | ---------------------------- 75 | 76 | - [ROS](ros.org) - Main ROS website. 77 | - [ros2/design](https://github.com/ros2/design) - Design documentation for ROS 2.0 effort. 78 | 79 | 80 | Kinematics, Dynamics, Constrained Optimization 81 | ---------------------------------------------- 82 | 83 | - [jrl-umi3218/Tasks](https://github.com/jrl-umi3218/Tasks) - Tasks is library for real time control of robots and kinematic trees using constrained optimization. 84 | - [jrl-umi3218/RBDyn](https://github.com/jrl-umi3218/RBDyn) - RBDyn provides a set of classes and functions to model the dynamics of rigid body systems. 85 | - [ceres-solver](https://github.com/ceres-solver/ceres-solver) - Solve Non-linear Least Squares problems with bounds constraints and general unconstrained optimization problems. Used in production at Google since 2010. 86 | - [orocos_kinematics_dynamics](https://github.com/orocos/orocos_kinematics_dynamics) - Orocos Kinematics and Dynamics C++ library. 87 | - [flexible-collsion-library](https://github.com/flexible-collision-library/fcl) - Performs three types of proximity queries on a pair of geometric models composed of triangles, integrated with ROS. 88 | - [robot_calibration](https://github.com/mikeferguson/robot_calibration) - generic robot kinematics calibration for ROS 89 | - [ruckig](https://github.com/pantor/ruckig) - Real-time, time-optimal and jerk-constrained online trajectory generation. 90 | 91 | Calibration 92 | ----------- 93 | 94 | - [handeye-calib-camodocal](https://github.com/jhu-lcsr/handeye_calib_camodocal) - generic robot hand-eye calibration. 95 | - [robot_calibration](https://github.com/mikeferguson/robot_calibration) - generic robot kinematics calibration for ROS 96 | - [kalibr](https://github.com/ethz-asl/kalibr) - camera and imu calibration for ROS 97 | 98 | Reinforcement Learning 99 | ---------------------- 100 | 101 | - ["Good Robot!": Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer](https://github.com/jhu-lcsr/good_robot) - A real robot completes multi-step tasks after <20k simulated actions. [Good Robot on ArXiV](https://arxiv.org/abs/1909.11730) (disclaimer: @ahundt is first author) 102 | - [TensorForce](https://github.com/reinforceio/tensorforce) - A TensorFlow library for applied reinforcement learning 103 | - [gqcnn](https://github.com/BerkeleyAutomation/gqcnn) - [Grasp Quality Convolutional Neural Networks (GQ-CNNs)](https://berkeleyautomation.github.io/gqcnn/info/info.html) for grasp planning using training datasets from the [Dexterity Network (Dex-Net)](https://berkeleyautomation.github.io/dex-net) 104 | - [Guided Policy Search](https://github.com/cbfinn/gps) - Guided policy search (gps) algorithm and LQG-based trajectory optimization, meant to help others understand, reuse, and build upon existing work. 105 | 106 | Drivers for Sensors, Devices and Arms 107 | ------------------------------------- 108 | 109 | - [libfreenect2](https://github.com/OpenKinect/libfreenect2) - Open source drivers for the Kinect for Windows v2 and Xbox One devices. 110 | - [iai_kinect2](https://github.com/code-iai/iai_kinect2) - Tools for using the Kinect One (Kinect v2) in ROS. 111 | - [grl](https://github.com/ahundt/grl) - Generic Robotics Library: Cross platform drivers for Kuka iiwa and Atracsys FusionTrack with optional v-rep and ros drivers. Also has cross platform Hand Eye Calibration and Tool Tip Calibration. 112 | 113 | Datasets 114 | -------- 115 | 116 | - [CoSTAR Block Stacking Dataset](https://sites.google.com/site/costardataset) - Robot stacking colored children's blocks (disclaimer: created by @ahundt) 117 | - [shapestacks](http://shapestacks.robots.ox.ac.uk/#paper) - simulated stacks of colored children's objects 118 | - [pascal voc 2012](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/) - The classic reference image segmentation dataset. 119 | - [openimages](https://github.com/openimages/dataset/) - Huge imagenet style dataset by Google. 120 | - [COCO](mscoco.org) - Objects with segmentation, keypoints, and links to many other external datasets. 121 | - [cocostuff](https://github.com/nightrome/cocostuff) - COCO additional full scene segmentation including backgrounds and annotator. 122 | - [Google Brain Robot Data](https://sites.google.com/site/brainrobotdata/home) - Robotics datasets including grasping, pushing, and pouring. 123 | - [Materials in Context](http://opensurfaces.cs.cornell.edu/publications/minc/) - Materials Dataset with real world images in 23 categories. 124 | - [Dex-Net 2.0](http://bair.berkeley.edu/blog/2017/06/27/dexnet-2.0/) - 6.7 million pairs of synthetic point clouds and grasps with robustness labels. 125 | 126 | #### Dataset Collection 127 | - [LabelFusion](labelfusion.csail.mit.edu) - "A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes" [code](https://github.com/RobotLocomotion/LabelFusion) 128 | - [cocostuff](https://github.com/nightrome/cocostuff) - COCO additional full scene segmentation including backgrounds and annotator. 129 | 130 | Linear Algebra & Geometry 131 | ------------------------- 132 | 133 | - [Eigen](http://eigen.tuxfamily.org) - Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. 134 | - [Boost.QVM](https://github.com/boostorg/qvm) - Quaternions, Vectors, Matrices library for Boost. 135 | - [Boost.Geometry](https://github.com/boostorg/geometry/) - Boost.Geometry contains instantiable geometry classes, but library users can also use their own. 136 | - [SpaceVecAlg](https://github.com/jrl-umi3218/SpaceVecAlg) - Implementation of spatial vector algebra for 3D geometry with the Eigen3 linear algebra library. 137 | - [Sophus](https://github.com/strasdat/Sophus) - C++ implementation of Lie Groups which are for 3D Geometry, using Eigen. 138 | 139 | 140 | Point Clouds 141 | ------------ 142 | 143 | - [libpointmatcher](https://github.com/ethz-asl/libpointmatcher) - An "Iterative Closest Point" library robotics and 2-D/3-D mapping. 144 | - [Point Cloud Library (pcl)](https://github.com/PointCloudLibrary/pcl) - The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing. 145 | 146 | 147 | 148 | Simultaneous Localization and Mapping (SLAM) 149 | -------------------------------------------- 150 | 151 | - [ElasticFusion](https://github.com/mp3guy/ElasticFusion) - Real-time dense visual SLAM system. 152 | - [co-fusion](https://github.com/martinruenz/co-fusion) - Real-time Segmentation, Tracking and Fusion of Multiple Objects. Extends ElasticFusion. 153 | - [Google Cartographer](https://github.com/googlecartographer/cartographer/) - Cartographer is a system that provides real-time simultaneous localization and mapping (SLAM) in 2D and 3D across multiple platforms and sensor configurations. 154 | - [OctoMap](https://github.com/OctoMap/octomap) - An Efficient Probabilistic 3D Mapping Framework Based on Octrees. Contains the main OctoMap library, the viewer octovis, and dynamicEDT3D. 155 | - [ORB_SLAM2](https://github.com/raulmur/ORB_SLAM2) - Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities. 156 | 157 | 158 | # License 159 | 160 | Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License. 161 | -------------------------------------------------------------------------------- /interesting.md: -------------------------------------------------------------------------------- 1 | Interesting Robotics 2 | -------------------- 3 | 4 | Interesting things not yet on the awesome list or no longer actively developed. 5 | 6 | - [Pytorch3D](https://github.com/facebookresearch/pytorch3d) - efficient, reusable components for 3D Computer Vision research with PyTorch. 7 | - [grandslam](https://github.com/gradslam/gradslam) - gradslam is a fully differentiable dense SLAM framework with PyTorch. No live image streams as of 2020-11-22. 8 | - [keras-finetuning](https://github.com/danielvarga/keras-finetuning) - Fine-tune InceptionV3 on your own data and try out on a webcam, plus use apple photos for labeling. 9 | - [Teaching Robotics with a Simulator](https://github.com/ULgRobotics/trs) - TRS: An Open-source Recipe for Teaching/Learning Robotics with the V-REP Simulator. 10 | - [dataset_loaders](https://github.com/fvisin/dataset_loaders) - Load a variety of video and segmentation datasets in python for machine learning. 11 | - [ros-tag-tracking](https://github.com/ablarry91/ros-tag-tracking) - Evaluates three different ROS tag tracking packages. 12 | - [gym-gazebo](https://github.com/erlerobot/gym-gazebo) - Simulator for reinforcement learning algorithms using ROS and Gazebo. 13 | - [neuralnets](https://github.com/mzaradzki/neuralnets) - Deep Learning libraries tested on images and time series, keras, theano, tensorflow. 14 | - [OpenSurfaces](https://github.com/seanbell/opensurfaces) - Crowdsourcing pipeline and website for OpenSurfaces [SIG '13] and Intrinsic Images in the Wild [SIG '14]. 15 | - [G3DB](https://uwaterloo.ca/neurorobotics-lab/g3db) - Simulated grasping dataset. 16 | - [MIME dataset](https://sites.google.com/view/mimedataset/people?authuser=0) - 8k+ human and robot tasks recorded, but no license saying what you can do with it and only an overview readme, no detailed documentation... maybe it will improve eventually? 17 | -------------------------------------------------------------------------------- /papers.md: -------------------------------------------------------------------------------- 1 | Awesome Papers 2 | -------------- 3 | 4 | Papers and implementations of papers that could have use in robotics. Implementations here may not be actively developed. While implementations may often be the author's original implementation, that isn't always the case. 5 | - ["Good Robot!": Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer](https://github.com/jhu-lcsr/good_robot) - 2020 - Real robot learns to complete multi-step tasks like table clearing, making stacks, and making rows in <20k simulated actions. [paper](https://arxiv.org/abs/1909.11730) (disclaimer: @ahundt is first author) [!["Good Robot!": Efficient Reinforcement Learning for Multi Step Visual Tasks via Reward Shaping](https://img.youtube.com/vi/MbCuEZadkIw/0.jpg)](https://youtu.be/MbCuEZadkIw) 6 | 7 | - [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://transporternets.github.io/) - [Ravens Simulator code](https://github.com/google-research/google-research/tree/master/ravens) - 2020 - Ravens is a collection of simulated tasks in PyBullet for learning vision-based robotic manipulation, with emphasis on pick and place. It features a Gym-like API with 10 tabletop rearrangement tasks, each with (i) a scripted oracle that provides expert demonstrations (for imitation learning), and (ii) reward functions that provide partial credit (for reinforcement learning). 8 | - [Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations](https://sites.google.com/view/concept2robot) - 2020 - Language + BERT to robot actions, code TBD, pybullet sim 9 | - [CURL: Contrastive Unsupervised Representations for RL](https://arxiv.org/abs/2004.04136) - 2020 - We use the simplest form of contrastive learning (instance-based) as an auxiliary task in model-free RL. SoTA by significant margin on DMControl and Atari for data-efficiency. 10 | - [Grasp Proposal Networks: An End-to-End Solution for Visual Learning of Robotic Grasps](https://github.com/CZ-Wu/GPNet) - 2020 - useful pybullet code robotiq gripper 11 | - [Self-Supervised Correspondence in Visuomotor Policy Learning](https://arxiv.org/abs/1909.06933) - 2019 - [video](https://youtu.be/nDRBKb4AGmA) 12 | - [Grasp2Vec: Learning Object Representations from Self-Supervised Grasping](https://sites.google.com/site/grasp2vec/) - 2018 - ![poster](https://pbs.twimg.com/media/Dqk8oPfWsAA96eM.jpg) no implementation available 13 | - [Dense Object Nets: Learning Dense Visual Descriptors by and for Robotic Manipulation](https://github.com/RobotLocomotion/pytorch-dense-correspondence) - 2018 ![object feature gif](https://github.com/RobotLocomotion/pytorch-dense-correspondence/blob/master/doc/shoes_trim.gif) 14 | - [Bounding Box Detection Accuracy Tradeoffs](https://arxiv.org/pdf/1611.10012.pdf) - Speed/accuracy trade-offs for modern convolutional object detectors 15 | - [pointnet++](https://github.com/charlesq34/pointnet2) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](http://stanford.edu/~rqi/pointnet2/) 16 | - [DA-RNN](https://github.com/yuxng/DA-RNN) - Semantic Mapping with Data Associated Recurrent Neural Networks 17 | - [rpg_open_remode](https://github.com/uzh-rpg/rpg_open_remode) - This repository contains an implementation of REMODE ([REgularized MOnocular Depth Estimation](http://rpg.ifi.uzh.ch/docs/ICRA14_Pizzoli.pdf)). 18 | - [shelhamer/fcn.berkeleyvision.org](https://github.com/shelhamer/fcn.berkeleyvision.org) - Fully Convolutional Networks for Semantic Segmentation, [PAMI FCN](https://arxiv.org/abs/1605.06211) and [CVPR FCN](http://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html) 19 | - [train-CRF-RNN](https://github.com/martinkersner/train-CRF-RNN) - Training scripts for [CRF-RNN for Semantic Image Segmentation](https://github.com/torrvision/crfasrnn). 20 | - [train-DeepLab](https://github.com/martinkersner/train-DeepLab) - Scripts for training [DeepLab for Semantic Image Segmentation](https://bitbucket.org/deeplab/deeplab-public) using [strongly](https://github.com/martinkersner/train-DeepLab#strong-annotations) and [weakly annotated data](https://github.com/martinkersner/train-DeepLab#weak-annotations). [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](http://arxiv.org/abs/1412.7062) and [Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation](http://arxiv.org/abs/1502.02734) papers describe training procedure using strongly and weakly annotated data, respectively. 21 | - [text_objseg](https://github.com/ronghanghu/text_objseg) Segmentation from Natural Language Expressions 22 | - [Asynchronous Methods for Deep Reinforcement Learning](http://arxiv.org/abs/1602.01783), Asynchronous Advantage Actor Critic (A3C) 23 | - [tensorpack/examples/A3C-Gym](https://github.com/ppwwyyxx/tensorpack/tree/master/examples/A3C-Gym) - Multi-GPU version of the A3C algorithm. 24 | - [ga3c](https://github.com/NVlabs/GA3C) - Hybrid CPU/GPU implementation of the A3C algorithm for deep reinforcement learning. 25 | 26 | ### Robotic Grasping 27 | 28 | - [Real-Time Grasp Detection Using Convolutional Neural Networks](https://arxiv.org/pdf/1412.3128.pdf) - (2015) 29 | - [Dex-Net 2.0](https://arxiv.org/pdf/1703.09312.pdf) - Dex-Net 2.0: Deep Learning to Plan Robust 30 | Grasps with Synthetic Point Clouds and Analytic Grasp Metrics 31 | - [Multi-task Domain Adaptation for Deep Learning of Instance Grasping from Simulation](https://arxiv.org/pdf/1710.06422.pdf) - (2017) 32 | - [End-to-End Learning of Semantic Grasping](https://arxiv.org/pdf/1707.01932.pdf) - Paper from google, uses hand eye coordination methods + classification of what object would be grasped 33 | - [Robotic Grasp Detection using Deep Convolutional Neural Networks](https://arxiv.org/pdf/1611.08036.pdf) - (2017) uses 2 resnets 34 | - [Convolutional Residual Network for Grasp Localization](http://www2.ift.ulaval.ca/~pgiguere/papers/ResNetGraspCRV2017.pdf) - (2017) uses 1 resnet 35 | - [Supervision via Competition: Robot Adversaries for Learning Tasks](https://arxiv.org/pdf/1610.01685v1.pdf) - (2017 with [Code](https://github.com/lerrel/Grasp-Detector)) One robot holds an object and tries to make the object difficult to grasp. 36 | 37 | #### Older papers useful as references for the above 38 | 39 | - [Deep Learning for Detecting Robotic Grasps](http://pr.cs.cornell.edu/papers/lenz_ijrr2014_deepgrasping.pdf) - (2014) Created the [Cornell Graping Dataset](http://pr.cs.cornell.edu/grasping/rect_data/data.php) 40 | - [Efficient Grasping from RGBD Images: Learning using a new 41 | Rectangle Representation](http://pr.cs.cornell.edu/grasping/jiang_rectanglerepresentation_fastgrasping.pdf) - (2011) Defines the rectangle representation utilized in many of the above grasping papers. 42 | 43 | 44 | ## Reinforcement learning (including non-robotics) 45 | 46 | - [Acme: A Research Framework for Distributed Reinforcement Learning](https://arxiv.org/abs/2006.00979) - 2020 - Deepmind 47 | - [RECURRENT EXPERIENCE REPLAY IN DISTRIBUTED REINFORCEMENT LEARNING](https://openreview.net/pdf?id=r1lyTjAqYX) - 2019 - Recurrent Replay Distributed DQN (R2D2) - Deepmind 48 | - [RL Unplugged: Benchmarks for Offline Reinforcement Learning](https://arxiv.org/pdf/2006.13888.pdf) - 2020 - Offline RL benchmark - Deepmind 49 | - [Making Efficient Use of Demonstrations to Solve Hard Exploration Problems](https://arxiv.org/pdf/1909.01387.pdf) - 2019 - R2D3 - Deepmind 50 | 51 | ## Language and Robots 52 | 53 | - [Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations](http://www.roboticsproceedings.org/rss16/p082.pdf) - RSS2020 54 | --------------------------------------------------------------------------------