└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Awesome-Human-Pose-Prediction 2 | ![Version](https://img.shields.io/badge/Version-0.5-ff69b4.svg) [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) ![LastUpdated](https://img.shields.io/badge/LastUpdated-2020.06.04-lightgrey.svg) 3 | [![HitCount](http://hits.dwyl.io/karttikeya/Awesome-Human-Pose-Prediction.svg)](http://hits.dwyl.io/karttikeya/Awesome-Human-Pose-Prediction) 4 | 5 |

6 | 7 |

8 | 9 | A selection of State Of The Art research papers (and code) on human trajectory prediction (forecasting). Papers marked with [W] are workshop papers. 10 | 11 | **Maintainers**: [Karttikeya Mangalam](http://karttikeya.github.io/) 12 | 13 | **Contributing**: Please feel free to [pull requests]() to add new resources or suggest addditions or changes to the list. While proposing a new addition, please keep in mind the following principles: 14 | 15 | - The work has been accepted in a reputable peer reviewed publication venue. 16 | - An opensource link to the paper pdf is attached (as far as possible). 17 | - Code for the paper is linked (if made opensource by the **authors**). 18 | 19 | Email: mangalam@cs.{berkeley,stanford).edu 20 | 21 | ## Datasets 22 | - Human3.6M: 23 | Large Scale Datasets and Predictive Methods 24 | for 3D Human Sensing in Natural Environments [[Paper]](http://vision.imar.ro/human3.6m/pami-h36m.pdf) 25 | - Stanford Drone Dataset (SDD): Learning Social Etiquette: Human Trajectory 26 | Understanding in Crowded Scenes [[Paper]](http://svl.stanford.edu/assets/papers/ECCV16social.pdf) [[Leaderboard]](https://paperswithcode.com/sota/trajectory-prediction-on-stanford-drone) 27 | 28 | ## Papers 29 | 30 | ### As End in Itself 31 | - From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [[Paper]](https://arxiv.org/pdf/2012.01526) 32 | - It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction [[Paper]](https://arxiv.org/pdf/2004.02025.pdf) 33 | - Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data [[Paper]](https://arxiv.org/pdf/2001.03093) 34 | - Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph [[paper]](https://arxiv.org/pdf/2009.12916.pdf) 35 | - Map-Adaptive Goal-Based Trajectory Prediction [[paper]](https://arxiv.org/pdf/2009.04450.pdf) 36 | - Interaction-Aware Trajectory Prediction based on a 3D Spatio-Temporal Tensor Representation using Convolutional–Recurrent Neural Networks [[paper]](https://ieeexplore.ieee.org/abstract/document/9304846) 37 | - DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning [[Paper]](https://arxiv.org/pdf/1908.00024) 38 | - Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction [[Paper]](http://proceedings.mlr.press/v100/jain20a.html) 39 | - Social-VRNN: One-Shot Multi-modal Trajectory Prediction for Interacting Pedestrians [[Paper]](https://arxiv.org/pdf/2010.09056) 40 | - Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions [[Paper]](https://arxiv.org/pdf/2012.01027) 41 | - Social NCE: Contrastive Learning of Socially-aware Motion Representations [[Paper]](https://arxiv.org/pdf/2012.11717.pdf) 42 | - Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach [[Paper]](https://ieeexplore.ieee.org/abstract/document/9286482) 43 | - Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction [[Paper]](https://arxiv.org/pdf/2009.05702.pdf) 44 | - Deep Learning for Vision-based Prediction: A Survey [[Paper]](https://arxiv.org/pdf/2007.00095) 45 | - Probabilistic Crowd GAN: Multimodal Pedestrian Trajectory Prediction Using a Graph Vehicle-Pedestrian Attention Network [[Paper]](https://ieeexplore.ieee.org/abstract/document/9123560) 46 | - Semantics for Robotic Mapping, Perception and Interaction: A Survey [[Paper]](https://arxiv.org/pdf/2101.00443.pdf) 47 | - Benchmark for Evaluating Pedestrian Action Prediction[[Paper]](https://openaccess.thecvf.com/content/WACV2021/papers/Kotseruba_Benchmark_for_Evaluating_Pedestrian_Action_Prediction_WACV_2021_paper.pdf) 48 | - Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking [[Paper]](https://arxiv.org/pdf/2012.02337.pdf) 49 | - Pedestrian Behavior Prediction via Multitask Learning and Categorical Interaction Modeling [[Paper]](https://arxiv.org/pdf/2012.03298) 50 | - Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction [[Paper]](https://arxiv.org/pdf/2012.02148.pdf) 51 | - Haar Wavelet based Block Autoregressive Flows for Trajectories [[Paper]](https://pure.mpg.de/pubman/faces/ViewItemOverviewPage.jsp?itemId=item_3267310) 52 | - Imitative Planning using Conditional Normalizing Flow [[Paper]](https://arxiv.org/pdf/2007.16162.pdf) 53 | - TNT: Target-driveN Trajectory Prediction [[Paper]](https://arxiv.org/pdf/2008.08294.pdf) 54 | - SimAug: Learning Robust Representations from Simulation for Trajectory Prediction [[Paper]](https://link.springer.com/chapter/10.1007/978-3-030-58601-0_17) 55 | - SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints 56 | [[Paper]](https://openaccess.thecvf.com/content_CVPR_2019/html/Sadeghian_SoPhie_An_Attentive_GAN_for_Predicting_Paths_Compliant_to_Social_CVPR_2019_paper.html) 57 | - Social GAN: Socially Acceptable Trajectories With Generative Adversarial Networks [[Paper]](https://openaccess.thecvf.com/content_cvpr_2018/html/Gupta_Social_GAN_Socially_CVPR_2018_paper.html) 58 | - DESIRE: Distant Future Prediction in Dynamic Scenes With Interacting Agents [[Paper]](https://openaccess.thecvf.com/content_cvpr_2017/html/Lee_DESIRE_Distant_Future_CVPR_2017_paper.html) 59 | 60 | 61 | - Predicting Whole Body Motion Trajectories using Conditional Neural 62 | Movement Primitives [[Paper]](https://motionpredictionicra2020.github.io/posters/lhmp2020_kurtoglu_paper.pdf) [W] 63 | - Anticipating Human Intention for Full-Body Motion Prediction [[Paper]](https://motionpredictionicra2020.github.io/posters/lhmp2020_kratzer_paper.pdf) [W] 64 | - Human Motion Prediction With Graph Neural Networks [[Paper]](https://motionpredictionicra2020.github.io/posters/lhmp2020_guzey_paper.pdf) [W] 65 | - Action-Agnostic Human Pose Forecasting [[Paper]](https://arxiv.org/pdf/1810.09676.pdf) 66 | - Human Torso Pose Forecasting in the Real World [[Paper]](http://harp.ri.cmu.edu/assets/pubs/mmpc_rss2018_biswas.pdf) 67 | - Imitation Learning for Human Pose Prediction [[Paper]](http://openaccess.thecvf.com/content_ICCV_2019/papers/Wang_Imitation_Learning_for_Human_Pose_Prediction_ICCV_2019_paper.pdf) 68 | - Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision [[Paper]](https://arxiv.org/pdf/1911.01138.pdf) 69 | - Predicting 3D Human Dynamics from Video [[Paper]](https://arxiv.org/pdf/1812.01601.pdf) 70 | - Recurrent Network Models for Human Dynamics [[Paper]](https://arxiv.org/pdf/1508.00271.pdf) 71 | - Structural-RNN: Deep Learning on Spatio-Temporal Graphs [[Paper]](https://arxiv.org/pdf/1511.05298.pdf) 72 | - Learning Trajectory Dependencies for Human Motion Prediction [[Paper]](https://arxiv.org/pdf/1908.05436.pdf) 73 | - Anticipating many futures: Online human motion prediction and generation 74 | for human-robot interaction [[Paper]](http://www.nada.kth.se/~hedvig/publications/icra_18.pdf) 75 | - Teaching Robots to Predict Human Motion [[Paper]](https://www.ri.cmu.edu/wp-content/uploads/2018/12/yuxiongw_iros18_teaching.pdf) 76 | - Deep representation learning for human motion prediction and classification [[Paper]](https://arxiv.org/pdf/1702.07486.pdf) 77 | - On human motion prediction using recurrent neural networks [[Paper]](https://arxiv.org/pdf/1705.02445.pdf) 78 | - Few-Shot Human Motion Prediction via Meta-learning [[Paper]](https://www.ri.cmu.edu/wp-content/uploads/2018/12/yuxiongw_eccv18_fewshot.pdf) 79 | - Efficient convolutional hierarchical autoencoder for human motion 80 | prediction [[Paper]](https://link.springer.com/content/pdf/10.1007%2Fs00371-019-01692-9.pdf) 81 | - Learning Human Motion Models for Long-term Predictions [[Paper]](https://arxiv.org/pdf/1704.02827.pdf) 82 | - Long-Term Human Motion Prediction by Modeling Motion Context and 83 | Enhancing Motion Dynamic [[Paper]](https://arxiv.org/pdf/1805.02513.pdf) 84 | - Context-aware Human Motion Prediction [[Paper]](https://arxiv.org/pdf/1904.03419.pdf) 85 | - Adversarial Geometry-Aware Human Motion Prediction [[Paper]](http://openaccess.thecvf.com/content_ECCV_2018/papers/Liangyan_Gui_Adversarial_Geometry-Aware_Human_ECCV_2018_paper.pdf) 86 | - Convolutional Sequence to Sequence Model for Human Dynamics [[Paper]](http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_Convolutional_Sequence_to_CVPR_2018_paper.pdf) 87 | - QuaterNet: A Quaternion-based Recurrent Model for Human Motion [[Paper]](https://arxiv.org/pdf/1805.06485.pdf) 88 | - BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN [[Paper]](https://arxiv.org/pdf/1812.02591.pdf) 89 | - Human Motion Modeling using DVGANs [[Paper]](https://arxiv.org/pdf/1804.10652.pdf) 90 | - Human Motion Prediction using Semi-adaptable Neural Networks [[Paper]](https://arxiv.org/pdf/1810.00781.pdf) 91 | - A Neural Temporal Model for Human Motion Prediction [[Paper]](http://openaccess.thecvf.com/content_CVPR_2019/papers/Gopalakrishnan_A_Neural_Temporal_Model_for_Human_Motion_Prediction_CVPR_2019_paper.pdf) 92 | - Modeling Human Motion with Quaternion-based Neural Networks [[Paper]](https://arxiv.org/pdf/1901.07677.pdf) 93 | - Human Motion Prediction via Learning Local Structure Representations and 94 | Temporal Dependencies [[Paper]](https://arxiv.org/pdf/1902.07367.pdf) 95 | - VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction [[Paper]](https://arxiv.org/pdf/1906.06514.pdf) 96 | - EAN: Error Attenuation Network for Long-term Human Motion Prediction [[Paper]](https://ieeexplore.ieee.org/abstract/document/8901951) 97 | - Structured Prediction Helps 3D Human Motion Modelling [[Paper]](http://openaccess.thecvf.com/content_ICCV_2019/papers/Aksan_Structured_Prediction_Helps_3D_Human_Motion_Modelling_ICCV_2019_paper.pdf) 98 | - Forecasting Human Dynamics from Static Images [[Paper]](https://arxiv.org/pdf/1704.03432.pdf) 99 | - HP-GAN: Probabilistic 3D human motion prediction via GAN [[Paper]](http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w29/Barsoum_HP-GAN_Probabilistic_3D_CVPR_2018_paper.pdf) 100 | - Learning Latent Representations of 3D Human Pose with Deep Neural Networks [[Paper]](https://infoscience.epfl.ch/record/252823/files/main_paper.pdf) 101 | - A Recurrent Variational Autoencoder for 102 | Human Motion Synthesis [[Paper]](http://www.ipab.inf.ed.ac.uk/cgvu/0414.pdf) 103 | - Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling [[Paper]](https://arxiv.org/abs/1908.07214) 104 | - Combining Recurrent Neural Networks and 105 | Adversarial Training for Human Motion Synthesis and Control [[Paper]](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8826012) 106 | - PISEP2: Pseudo Image Sequence Evolution based 3D Pose Prediction [[Paper]](https://arxiv.org/pdf/1909.01818.pdf) 107 | - Human Motion Prediction via Spatio-Temporal Inpainting [[Paper]](https://arxiv.org/pdf/1812.05478.pdf) 108 | - Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction [[Paper]](https://arxiv.org/pdf/1909.13245.pdf) 109 | - Human Pose Forecasting via Deep Markov Models [[Paper]](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8227441) 110 | - Auto-Conditioned Recurrent Networks For Extended Complex Human Motion Synthesis [[Paper]](https://openreview.net/pdf?id=r11Q2SlRW) 111 | - Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network [[Paper]](https://arxiv.org/pdf/1911.02404.pdf) 112 | 113 | ### As a Subtask 114 | - The Pose Knows: Video Forecasting by Generating Pose Futures [[Paper]](https://arxiv.org/pdf/1705.00053.pdf) 115 | - I-Planner: Intention-Aware Motion 116 | Planning Using Learning Based Human 117 | Motion Prediction [[Paper]](http://gamma.cs.unc.edu/SafeMP/papers/ijrr18.pdf) 118 | - Language2Pose: Natural Language Grounded Pose Forecasting [[Paper]](https://arxiv.org/pdf/1907.01108.pdf) 119 | - Long-Term Video Generation of Multiple Futures Using Human Poses [[Paper]](https://arxiv.org/pdf/1904.07538.pdf) 120 | - Predicting body movements for person identification under different walking conditions [[Paper]](https://reader.elsevier.com/reader/sd/pii/S0379073818304109?token=76C0C8F3C9DCB97DD1D9C1EB6985414D0E842BCDBB9F3E35389B3CA504BED12F86181D8AACBFB4572674B83842C5A31A) 121 | 122 | 123 | --------------------------------------------------------------------------------