├── src ├── template.md ├── end.md ├── language_prompt.md ├── head.md ├── point_cloud.md ├── 3D_robotics.md ├── 6D_pose_detection.md ├── 3D_content_animation_rendering.md ├── SLAM_reconstruct.md ├── dynamic_video.md ├── ssl3d.md └── neural_implicit.md ├── trends_2018-2019.md └── README.md /src/template.md: -------------------------------------------------------------------------------- 1 | [[xxxx et al. ( '22)]( )] xxx 2 | [[Project]( )] 3 | [[Code]( )] 4 | [[Video]( )] 5 | -------------------------------------------------------------------------------- /src/end.md: -------------------------------------------------------------------------------- 1 | #### Volume Rendering 2 | [[Sawhey et al. (SIGGRAPH '22)]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree-small.pdf )] Grid-free Monte Carlo for PDEs with spatially varying coefficients 3 | [[Project]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree.html )] 4 | [[Code]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree-reference-implementation.zip )] 5 | 6 | ### More resources 7 | - [Amesome 3D machine learning collection](https://github.com/timzhang642/3D-Machine-Learning) 8 | 9 | - [NeRF Fields in Visual Computing](https://neuralfields.cs.brown.edu/index.html) 10 | 11 | - [Awesome-point-cloud-registration](https://github.com/wsunid/awesome-point-clouds-registration) 12 | 13 | - [Awesome-equivariant-network](https://github.com/Chen-Cai-OSU/awesome-equivariant-network/blob/main/README.md#content) 14 | -------------------------------------------------------------------------------- /src/language_prompt.md: -------------------------------------------------------------------------------- 1 | 2 | ### [9. Prompt Learning to 3D](#content) 3 | [[Tevet et al. (ARXIV '22)]( https://guytevet.github.io/motionclip-page/static/source/MotionCLIP.pdf )] MotionCLIP: Exposing Human Motion Generation to CLIP Space 4 | [[Project]( https://guytevet.github.io/motionclip-page/ )] 5 | [[Code]( https://github.com/GuyTevet/MotionCLIP )] 6 | 7 | [[Wang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/html/Wang_CLIP-NeRF_Text-and-Image_Driven_Manipulation_of_Neural_Radiance_Fields_CVPR_2022_paper.html )] CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields 8 | [[Project]( https://cassiepython.github.io/clipnerf/ )] 9 | [[Code]( https://github.com/cassiePython/CLIPNeRF )] 10 | [[Video]( https://cassiepython.github.io/clipnerf/images/video.mp4 )] 11 | 12 | [[Michel et al. (ARXIV '21)]( https://arxiv.org/abs/2112.03221 )] Text2Mesh: Text-Driven Neural Stylization for Meshes 13 | [[Project]( https://threedle.github.io/text2mesh/ )] 14 | [[Code]( https://github.com/threedle/text2mesh )] 15 | -------------------------------------------------------------------------------- /src/head.md: -------------------------------------------------------------------------------- 1 | ## Trending in 3D Vision 2 | I first got fascinated by the beauty of 3D vision since 2015. After that, so many new and wonderful ideas, works have been brought into this field, and it seems so hard to catch up with this fast-evolving area today. This leads to the major motivation behind this paper reading list: to get a sense of current SOTA methods, and an overview of the research trending in the field of 3D vision, mainly with deep learning. 3 | 4 | From this list, you may say, various applications, multiple modalities of data, powerful neural backbones are the major working horses, or the boom of neural radiance field and differentiable rendering inspire a lot of new methods and tasks, or you want to point out that self-supervision, data-efficient learning are the critical keys. Different people may have different opinions, but this list is about existing possibilities in 3D vision, to which you may say 'wow, this is even possible', or 'aha, I never imagined such a method'. 5 | 6 | Note that this repo started as a self-collected paper list based on my own appetite, which may reflect some bias. Some may not be precisely categorized, for which you can raise an issue, or send a pull request. 7 | -------------------------------------------------------------------------------- /src/point_cloud.md: -------------------------------------------------------------------------------- 1 | ### [5. Frontiers on 3D Point Cloud Learning](#content) 2 | 3 | [[Wiersma et al. (SIGGRAPH '22)]( https://rubenwiersma.nl/assets/pdf/DeltaConv.pdf )] DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds 4 | [[Project]( https://rubenwiersma.nl/deltaconv )] 5 | [[Code]( https://github.com/rubenwiersma/deltaconv )] 6 | [[Supp.]( https://rubenwiersma.nl/assets/pdf/DeltaConv_supplement.pdf )] 7 | 8 | [[Ran et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.05740.pdf )] Surface Representation for Point Clouds 9 | [[Code]( https://github.com/hancyran/RepSurf )] 10 | 11 | [[Mittal et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.09516.pdf )] AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation 12 | [[Project]( https://yccyenchicheng.github.io/AutoSDF/ )] 13 | [[Code]( https://github.com/yccyenchicheng/AutoSDF/ )] 14 | 15 | [[Chen et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Chen_The_Devil_Is_in_the_Pose_Ambiguity-Free_3D_Rotation-Invariant_Learning_CVPR_2022_paper.pdf)] The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant 16 | Learning via Pose-aware Convolution 17 | 18 | [[Jakab et al. (CVPR '21)]( https://arxiv.org/abs/2104.11224 )] KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control 19 | [[Project]( https://tomasjakab.github.io/KeypointDeformer/ )] 20 | [[Code]( https://github.com/tomasjakab/keypoint_deformer/ )] 21 | [[Video]( https://youtu.be/GdDX1ZFh1k0 )] 22 | -------------------------------------------------------------------------------- /src/3D_robotics.md: -------------------------------------------------------------------------------- 1 | ### [8. 3D Representations Learning for Robotics](#content) 2 | [[Driess et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.01634.pdf )] Reinforcement Learning with Neural Radiance Fields 3 | [[Project]( https://dannydriess.github.io/nerf-rl/ )] 4 | [[Video]( https://dannydriess.github.io/nerf-rl/video.mp4 )] 5 | 6 | [[Gao et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/html/Gao_ObjectFolder_2.0_A_Multisensory_Object_Dataset_for_Sim2Real_Transfer_CVPR_2022_paper.html )] ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer 7 | [[Project]( https://ai.stanford.edu/~rhgao/objectfolder2.0/ )] 8 | [[Code]( https://github.com/rhgao/ObjectFolder )] 9 | [[Video]( https://youtu.be/e5aToT3LkRA )] 10 | 11 | [[Ortiz et al. (RSS '22)]( https://arxiv.org/abs/2204.02296 )] iSDF: Real-Time Neural Signed Distance Fields for Robot Perception 12 | [[Project]( https://joeaortiz.github.io/iSDF/ )] 13 | [[Code]( https://github.com/facebookresearch/iSDF )] 14 | [[Video]( https://youtu.be/mAKGl1wBSic )] 15 | 16 | [[Wi et al. (ICRA '22)]( https://arxiv.org/abs/2202.00868)] VIRDO: Visio-tactile Implicit Representations of Deformable Objects 17 | [[Project]( https://www.mmintlab.com/research/virdo-visio-tactile-implicit-representations-of-deformable-objects/ )] 18 | [[Code]( https://github.com/MMintLab/VIRDO )] 19 | 20 | [[Adamkiewicz et al. (ICRA '22)]( https://arxiv.org/pdf/2110.00168.pdf )] Vision-Only Robot Navigation in a Neural Radiance World 21 | [[Project]( https://mikh3x4.github.io/nerf-navigation/ )] 22 | [[Code]( https://github.com/mikh3x4/nerf-navigation )] 23 | [[Video]( https://youtu.be/5JjWpv9BaaE )] 24 | [[Data](https://drive.google.com/drive/folders/10_DWHIIetzeM2-1ziyZHujycWh-fNs29?usp=sharing)] 25 | 26 | [[Li et al. (CoRL '21)]( https://arxiv.org/abs/2107.04004 )] 3D Neural Scene Representations for Visuomotor Control 27 | [[Project]( https://3d-representation-learning.github.io/nerf-dy/)] 28 | [[Video]( https://youtu.be/ELPMiifELGc )] 29 | 30 | [[Ichnowski et al. (CoRL '21)]( https://arxiv.org/pdf/2110.14217.pdf )] Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects 31 | [[Project]( https://sites.google.com/view/dex-nerf )] 32 | [[Dataset]( https://github.com/BerkeleyAutomation/dex-nerf-datasets )] 33 | [[Video]( https://youtu.be/F9R6Nf1d7P4 )] 34 | -------------------------------------------------------------------------------- /src/6D_pose_detection.md: -------------------------------------------------------------------------------- 1 | 2 | ### [6. 3D Object Detection, Pose Estimation](#content) 3 | [[Yang et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Yang_ArtiBoost_Boosting_Articulated_3D_Hand-Object_Pose_Estimation_via_Online_Exploration_CVPR_2022_paper.pdf)] ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via 4 | Online Exploration and Synthesis 5 | 6 | [[Yin et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.15765.pdf )] FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering 7 | [[Project]( https://yd-yin.github.io/FisherMatch/ )] 8 | [[Code]( https://github.com/yd-yin/FisherMatch )] 9 | 10 | [[Sun et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.12257.pdf )] OnePose: One-Shot Object Pose Estimation without CAD Models 11 | [[Project]( https://zju3dv.github.io/onepose/ )] 12 | [[CodeSoon]( https://github.com/zju3dv/OnePose)] 13 | [[Supp]( https://zju3dv.github.io/onepose/files/onepose_supp.pdf )] 14 | 15 | [[Deng et al. (NeurIPS '21)]( https://arxiv.org/pdf/2112.07787.pdf )] Revisiting 3D Object Detection From an Egocentric Perspective 16 | 17 | [[Li et al. (NeurIPS '21)]( https://arxiv.org/abs/2111.00190 )] Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation 18 | [[Project]( https://dragonlong.github.io/equi-pose/ )] 19 | [[Code](https://github.com/dragonlong/equi-pose)] 20 | 21 | [[Lu et al. (ICCV '21)]( https://openaccess.thecvf.com/content/ICCV2021/papers/Lu_Geometry_Uncertainty_Projection_Network_for_Monocular_3D_Object_Detection_ICCV_2021_paper.pdf )] Geometry Uncertainty Projection Network for Monocular 3D Object Detection 22 | [[Code]( https://github.com/SuperMHP/GUPNet )] 23 | 24 | [[Ahmadyan et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Ahmadyan_Objectron_A_Large_Scale_Dataset_of_Object-Centric_Videos_in_the_CVPR_2021_paper.pdf )] Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild With Pose Annotations 25 | [[Project]( https://github.com/google-research-datasets/Objectron/#tutorials )] 26 | [[Code]( https://github.com/google-research-datasets/Objectron )] 27 | 28 | [[Murphy et al. (ICML '21)]( https://arxiv.org/abs/2106.05965 )] Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold 29 | [[Project]( https://implicit-pdf.github.io/ )] 30 | [[Code]( https://github.com/google-research/google-research/tree/master/implicit_pdf )] 31 | [[Video]( https://youtu.be/Y-MlRRy0xJA )] 32 | [[Data](https://www.tensorflow.org/datasets/catalog/symmetric_solids)] 33 | -------------------------------------------------------------------------------- /src/3D_content_animation_rendering.md: -------------------------------------------------------------------------------- 1 | 2 | ### [7. Neural Motion, Deformation Generation](#content) 3 | [[Kim et al. (ARXIV '22)]( https://arxiv.org/pdf/2202.04307.pdf )] Conditional Motion In-betweening 4 | [[Project]( https://jihoonerd.github.io/Conditional-Motion-In-Betweening/ )] 5 | 6 | [[He et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.03287.pdf )] NeMF: Neural Motion Fields for Kinematic Animation 7 | 8 | [[Ianina et al. (**CVPR '22**)]( https://nsarafianos.github.io/assets/bodymap/bodymap.pdf )] BodyMap: Learning Full-Body Dense Correspondence Map 9 | [[Project]( https://nsarafianos.github.io/bodymap )] 10 | [[Supp]( https://nsarafianos.github.io/assets/bodymap/bodymap_suppl.pdf )] 11 | 12 | [[Muralikrishnan et al. (**CVPR '22**)]( https://sanjeevmk.github.io/glass_webpage/resources/glass_fullRes.pdf )] GLASS: Geometric Latent Augmentation for Shape Spaces 13 | [[Project]( https://sanjeevmk.github.io/glass_webpage/ )] 14 | [[Code]( https://github.com/sanjeevmk/glass/ )] 15 | [[Video]( https://sanjeevmk.github.io/glass_webpage/video/glass_dist.mp4 )] 16 | 17 | [[Taheri et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.11454 )] GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping 18 | [[Project]( https://goal.is.tue.mpg.de/ )] 19 | [[Video]( https://youtu.be/A7b8DYovDZY )] 20 | [[Code]( https://github.com/otaheri/GOAL )] 21 | 22 | [[AIGERMAN et al. (SIGGRAPH '22)]( https://arxiv.org/pdf/2205.02904.pdf )] Neural Jacobian Fields: Learning Intrinsic Mappings of Arbitrary　Meshes 23 | 24 | [[Raab et al. (Siggraph '22)]( https://arxiv.org/abs/2206.08010 )] MoDi: Unconditional Motion Synthesis from Diverse Data 25 | [[Project(need fix)]( https://sigal-raab.github.io/MoDi )] 26 | [[Video]( https://youtu.be/lRkdF8y3Du4)] 27 | 28 | [[Li et al. (SIGGRAPH '22)]( https://arxiv.org/abs/2205.02625 )] GANimator: Neural Motion Synthesis from a Single Sequence 29 | [[Project]( https://peizhuoli.github.io/ganimator/ )] 30 | [[Code](https://github.com/PeizhuoLi/ganimator )] 31 | [[Video]( https://youtu.be/OV9VoHMEeyI )] 32 | 33 | [[Wang et al. (NeurIPS '21)]( https://arxiv.org/abs/2106.11944 )] MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images 34 | [[Project]( https://neuralbodies.github.io/metavatar/ )] 35 | [[Code](https://github.com/taconite/MetaAvatar-release )] 36 | [[Video](https://youtu.be/AwOwdKxuBXE )] 37 | 38 | [[Henter et al. (Siggraph Asia '20)]( http://kth.diva-portal.org/smash/get/diva2:1471598/FULLTEXT01.pdf )] MoGlow: Probabilistic and controllable motion synthesis using normalising flows 39 | [[Project]( https://simonalexanderson.github.io/MoGlow/ )] 40 | [[Code](https://github.com/simonalexanderson/MoGlow )] 41 | [[Video](https://youtu.be/pe-YTvavbtA )] 42 | -------------------------------------------------------------------------------- /src/SLAM_reconstruct.md: -------------------------------------------------------------------------------- 1 | ### [1. SLAM with Deep Learning](#content) 2 | [[Chen et al. (ARXIV '22)]( https://arxiv.org/pdf/2203.01087.pdf )] Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications 3 | 4 | [[Avraham et al. (ARXIV '22)]( https://arxiv.org/abs/2206.01916 )] Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation 5 | 6 | [[Hughe et al. (ARXIV '22)]( https://arxiv.org/pdf/2201.13360.pdf )] Hydra: A Real-time Spatial Perception Engine for 3D Scene Graph Construction and Optimization 7 | [[Video]( https://youtu.be/qZg2lSeTuvM )] 8 | 9 | [[Zhu at al. (**CVPR '22**)]( https://arxiv.org/pdf/2112.12130.pdf )] NICE-SLAM: Neural Implicit Scalable Encoding for SLAM 10 | [[Project]( https://pengsongyou.github.io/nice-slam )] 11 | [[Code]( https://github.com/cvg/nice-slam )] 12 | [[Video](https://youtu.be/V5hYTz5os0M )] 13 | 14 | [[Teed et al. (NeurIPS '21)]( https://arxiv.org/pdf/2108.10869.pdf )] DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras 15 | [[Code]( https://github.com/princeton-vl/DROID-SLAM )] 16 | [[Video]( https://www.youtube.com/watch?v=GG78CSlSHSA )] 17 | 18 | [[Yang et al. (3DV '21)]( https://arxiv.org/abs/2111.07418 )] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo 19 | [[Project]( https://vision.in.tum.de/research/vslam/tandem )] 20 | [[Code]( https://github.com/tum-vision/tandem )] 21 | [[Video](https://youtu.be/L4C8Q6Gvl1w)] 22 | 23 | [[Lin et al. (ARXIV '21)]( https://arxiv.org/pdf/2109.07982.pdf )] R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package 24 | [[Code]( https://github.com/hku-mars/r3live )] 25 | [[Video]( https://youtu.be/j5fT8NE5fdg )] 26 | 27 | [[Duzceker et al. (CVPR '21)](https://openaccess.thecvf.com/content/CVPR2021/papers/Duzceker_DeepVideoMVS_Multi-View_Stereo_on_Video_With_Recurrent_Spatio-Temporal_Fusion_CVPR_2021_paper.pdf)] DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion 28 | [[Code]( https://github.com/ardaduz/deep-video-mvs )] 29 | [[Video]( https://www.youtube.com/watch?v=ikpotjxwcp4 )] 30 | 31 | [[Teed at al. (CVPR '21)]( https://arxiv.org/pdf/2103.12032.pdf )] Tangent Space Backpropagation for 3D Transformation Groups 32 | [[Code]( https://github.com/princeton-vl/lietorch )] 33 | 34 | [[Sun et al. (CVPR '21)]( https://arxiv.org/pdf/2104.00681.pdf )] NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video 35 | [[Project]( http://zju3dv.github.io/neuralrecon/ )] 36 | [[Code]( https://github.com/zju3dv/NeuralRecon/ )] 37 | 38 | [[Murthy J. et al. (ICRA '20)](https://arxiv.org/pdf/1910.10672.pdf)] ∇SLAM: Automagically differentiable SLAM 39 | [[Project]( https://github.com/gradslam/gradslam )] 40 | [[Code]( https://gradslam.github.io/ )] 41 | 42 | [[Schops et al. (CVPR '19)]( https://openaccess.thecvf.com/content_CVPR_2019/papers/Schops_BAD_SLAM_Bundle_Adjusted_Direct_RGB-D_SLAM_CVPR_2019_paper.pdf )] BAD SLAM: Bundle Adjusted Direct RGB-D SLAM 43 | [[Project]( https://www.eth3d.net/ )] 44 | [[Code]( https://github.com/ETH3D/badslam )] 45 | -------------------------------------------------------------------------------- /src/dynamic_video.md: -------------------------------------------------------------------------------- 1 | ### [2. Dynamic Human, Animals and Objects Reconstruction](#content) 2 | #### Human avatars 3 | [[Su et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.01666.pdf )] DANBO: Disentangled Articulated Neural Body 4 | Representations via Graph Neural Networks 5 | [[Project]( https://lemonatsu.github.io/danbo/ )] 6 | 7 | [[Jiang et al. (**CVPR '22**)]( https://arxiv.org/pdf/2201.12792.pdf )] SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video 8 | [[Project]( https://jby1993.github.io/SelfRecon/ )] 9 | [[Code]( https://github.com/jby1993/SelfReconCode )] 10 | 11 | [[Weng et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.04127 )] HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video 12 | [[Project]( https://grail.cs.washington.edu/projects/humannerf/ )] 13 | [[Code]( https://github.com/chungyiweng/humannerf )] 14 | [[Video]( https://youtu.be/GM-RoZEymmw )] 15 | 16 | [[Yu et al. (CVPR '21)](https://arxiv.org/pdf/2105.01859.pdf )] Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors 17 | [[Project]( http://www.liuyebin.com/Function4D/Function4D.html )] 18 | [[Data]( https://github.com/ytrock/THuman2.0-Dataset )] 19 | [[Video]( http://www.liuyebin.com/Function4D/assets/supp_video.mp4 )] 20 | 21 | #### Animals capture 22 | [[Yang et al. (**CVPR '22**)]( https://banmo-www.github.io/banmo-2-14.pdf )] BANMo: Building Animatable 3D Neural Models from Many Casual Videos 23 | [[Project]( https://banmo-www.github.io/ )] 24 | [[Code]( https://github.com/facebookresearch/banmo )] 25 | [[Video]( https://youtu.be/1NUa-yvFGA0 )] 26 | 27 | [[Wu et al. (NeurIPS '21)]( https://arxiv.org/abs/2107.10844 )] DOVE: Learning Deformable 3D Objects by Watching Videos 28 | [[Project]( https://dove3d.github.io/ )] 29 | [[Video]( https://youtu.be/_FsADb0XmpY )] 30 | 31 | #### Human-object interaction 32 | [[Jiang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Jiang_NeuralHOFusion_Neural_Volumetric_Rendering_Under_Human-Object_Interactions_CVPR_2022_paper.pdf)] NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions 33 | [[Project]( https://nowheretrix.github.io/neuralfusion/ )] 34 | [[Video]( https://youtu.be/Stvks4rZMF0 )] 35 | 36 | [[Hasson et al. (CVPR '21)]( https://arxiv.org/abs/2108.07044 )] Towards unconstrained joint hand-object reconstruction from RGB videos 37 | [[Project](https://hassony2.github.io/homan.html )] 38 | [[Code]( https://github.com/hassony2/homan )] 39 | 40 | #### Scene-level 3D dynamics 41 | [[Grauman et al. (**CVPR '22**)]( https://arxiv.org/abs/2110.07058 )] Ego4D: Around the World in 3,000 Hours of Egocentric Video 42 | [[Project]( https://ego4d-data.org/ )] 43 | [[Code]( https://github.com/EGO4D )] 44 | 45 | [[Li et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Li_Neural_3D_Video_Synthesis_From_Multi-View_Video_CVPR_2022_paper.pdf )] Neural 3D Video Synthesis From Multi-View Video 46 | [[Project]( https://neural-3d-video.github.io/ )] 47 | [[Video]( https://neural-3d-video.github.io/resources/video.mp4 )] 48 | [[Data]( https://github.com/facebookresearch/Neural_3D_Video )] 49 | 50 | [[Zhang et al. (SIGGRAPH '21)]( https://arxiv.org/abs/2108.01166 )] Consistent Depth of Moving Objects in Video 51 | [[Project]( https://dynamic-video-depth.github.io/ )] 52 | [[Code]( https://github.com/google/dynamic-video-depth )] 53 | [[Video]( https://dynamic-video-depth.github.io/#video )] 54 | 55 | [[Zeed et al. (CVPR '21)]( https://arxiv.org/abs/2012.00726 )] RAFT-3D: Scene Flow using Rigid-Motion Embeddings 56 | [[Code]( https://github.com/princeton-vl/RAFT-3D )] 57 | 58 | [[Lu et al. (CVPR '21)]( https://arxiv.org/pdf/2105.06993.pdf )] Omnimatte: Associating Objects and Their Effects in Video 59 | [[Project]( https://omnimatte.github.io/ )] 60 | [[Code]( https://omnimatte.github.io/#code )] 61 | [[Video]( https://omnimatte.github.io/#video )] 62 | -------------------------------------------------------------------------------- /src/ssl3d.md: -------------------------------------------------------------------------------- 1 | ### [3. Self-Supervised 3D Representations Learning](#content) 2 | [[Hasselgren et al. (ARXIV '22)]( https://arxiv.org/abs/2206.03380 )] Shape, Light & Material Decomposition from Images using Monte Carlo Rendering and Denoising 3 | 4 | [[Gkioxari et al. (ARXIV '22)]( https://drive.google.com/file/d/1E6xSbUzuu6soAA-jkaGCFl97LZ8SVRvr/view )] Learning 3D Object Shape and Layout without 3D Supervision 5 | [[Project]( https://gkioxari.github.io/usl/ )] 6 | [[Video]( https://youtu.be/PKhGIiMuRJU )] 7 | 8 | [[Boss et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.15768.pdf)] SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections 9 | [[Project]( https://markboss.me/publication/2022-samurai/ )] 10 | [[Video]( https://youtu.be/LlYuGDjXp-8 )] 11 | 12 | [[Wei et al. (SIGGRAPH '22)]( https://arxiv.org/pdf/2205.02961.pdf )] Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search 13 | [[Project]( https://colin97.github.io/CoACD/ )] 14 | [[Code]( https://github.com/SarahWeiii/CoACD )] 15 | [[Video]( https://www.youtube.com/watch?v=r12O0z0723s )] 16 | 17 | [[Vicini et al. (SIGGRAPH '22)]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Vicini2022sdf_1.pdf )] Differentiable Signed Distance Function Rendering 18 | [[Project]( http://rgl.epfl.ch/publications/Vicini2022SDF )] 19 | [[Video]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Vicini2022sdf.mp4 )] 20 | 21 | [[Or-El et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.11427 )] StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation 22 | [[Project]( https://stylesdf.github.io/ )] 23 | [[Code](https://github.com/royorel/StyleSDF)] 24 | [[Demo]( https://colab.research.google.com/github/royorel/StyleSDF/blob/main/StyleSDF_demo.ipynb )] 25 | 26 | [[Girdhar et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.08377 )] Omnivore: A Single Model for Many Visual Modalities 27 | [[Project]( https://facebookresearch.github.io/omnivore )] 28 | [[Code](https://github.com/facebookresearch/omnivore )] 29 | 30 | 31 | 32 | [[Noguchi et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Noguchi_Watch_It_Move_Unsupervised_Discovery_of_3D_Joints_for_Re-Posing_CVPR_2022_paper.pdf )] Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects 33 | [[Project]( https://nvlabs.github.io/watch-it-move/ )] 34 | [[Code]( https://github.com/NVlabs/watch-it-move )] 35 | [[Video]( https://youtu.be/oRnnuCVV89o )] 36 | 37 | [[Gong et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.15625.pdf )] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision 38 | [[Code]( https://github.com/garfield-kh/posetriplet )] 39 | 40 | [[Wu et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.02306 )] Toward Practical Monocular Indoor Depth Estimation 41 | [[Project]( https://distdepth.github.io/ )] 42 | [[Code]( https://github.com/facebookresearch/DistDepth )] 43 | [[Video]( https://youtu.be/s9JdoR1xbz8 )] 44 | [[Data](https://drive.google.com/file/d/1KfDFyTg9-1w1oJB4oT-DUjKC6LG0enwb/view?usp=sharing)] 45 | 46 | 47 | [[Wei et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Wei_Self-Supervised_Neural_Articulated_Shape_and_Appearance_Models_CVPR_2022_paper.pdf )] Self-supervised Neural Articulated Shape and Appearance Models 48 | [[Project]( https://weify627.github.io/nasam/ )] 49 | [[Video]( https://youtu.be/0YbhTxALi8M )] 50 | 51 | [[Chan et al. (**CVPR '22**)](https://arxiv.org/pdf/2112.07945.pdf)] EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks 52 | [[Project]( https://nvlabs.github.io/eg3d/ )] 53 | [[Code]( https://github.com/NVlabs/eg3d )] 54 | [[Video]( https://www.youtube.com/watch?v=cXxEwI7QbKg )] 55 | 56 | [[Rombach et al. (ICCV '21)]( https://arxiv.org/pdf/2104.07652.pdf)] 57 | Geometry-Free View Synthesis: Transformers and no 3D Priors 58 | Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations 59 | [[Project]( https://compvis.github.io/geometry-free-view-synthesis/ )] 60 | [[Code]( https://github.com/CompVis/geometry-free-view-synthesis )] 61 | [[Video]( https://github.com/CompVis/geometry-free-view-synthesis/blob/master/assets/acid_long.mp4 )] 62 | 63 | [[Harley et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Harley_Track_Check_Repeat_An_EM_Approach_to_Unsupervised_Tracking_CVPR_2021_paper.pdf )] Track, Check, Repeat: An EM Approach to Unsupervised Tracking 64 | [[Project]( http://www.cs.cmu.edu/~aharley/em_cvpr21/ )] 65 | [[Code]( https://github.com/aharley/track_check_repeat )] 66 | 67 | [[Watson et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Watson_The_Temporal_Opportunist_Self-Supervised_Multi-Frame_Monocular_Depth_CVPR_2021_paper.pdf )] The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth 68 | [[Code]( https://github.com/nianticlabs/manydepth )] 69 | [[Video]( https://storage.googleapis.com/niantic-lon-static/research/manydepth/manydepth_cvpr_cc.mp4 )] 70 | 71 | [[Nicolet et al. (SIGGRAPH Asia '21)]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2021Large.pdf )] Large Steps in Inverse Rendering of Geometry 72 | [[Project]( https://rgl.epfl.ch/publications/Nicolet2021Large )] 73 | [[Code]( https://github.com/rgl-epfl/cholespy )] 74 | [[Video]( https://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2021Large_1.mp4 )] 75 | 76 | [[Wu et al. (CVPR '20)](https://openaccess.thecvf.com/content_CVPR_2020/papers/Wu_Unsupervised_Learning_of_Probably_Symmetric_Deformable_3D_Objects_From_Images_CVPR_2020_paper.pdf )] Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild 77 | [[Project]( https://elliottwu.com/projects/20_unsup3d/ )] 78 | [[Code]( https://github.com/elliottwu/unsup3d )] 79 | [[Video]( https://youtu.be/p3KB3eIQw24 )] 80 | -------------------------------------------------------------------------------- /src/neural_implicit.md: -------------------------------------------------------------------------------- 1 | 2 | ### [4. Breakthroughs in Deep Implicit Functions](#content) 3 | 4 | #### Topology-aware 5 | [[Palafox et al. (**CVPR '22**)]( https://arxiv.org/pdf/2201.08141.pdf )] SPAMs: Structured Implicit Parametric Models 6 | [[Project]( https://pablopalafox.github.io/spams/ )] 7 | [[Video]( https://www.youtube.com/watch?v=ChdjHNGgrzI )] 8 | 9 | [[Park et al. (SIGGRAPH Asia '21)]( https://arxiv.org/pdf/2106.13228.pdf )] A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields 10 | [[Project]( https://hypernerf.github.io/ )] 11 | [[Code]( https://github.com/google/hypernerf )] 12 | [[Video]( https://youtu.be/qzgdE_ghkaI )] 13 | 14 | #### Additional priors 15 | [[Guo et al. (**CVPR '22**)]( https://arxiv.org/abs/2205.02836 )] Neural 3D Scene Reconstruction with the Manhattan-world Assumption 16 | [[Project]( https://zju3dv.github.io/manhattan_sdf/)] 17 | [[Code]( https://github.com/zju3dv/manhattan_sdf )] 18 | [[Video]( https://www.youtube.com/watch?v=oEE7mK0YQtc )] 19 | 20 | #### Faster, memory-efficient 21 | [[Chen et al. (ARXIV '22)]( https://arxiv.org/pdf/2203.09517.pdf )] TensoRF: Tensorial Radiance Fields 22 | [[Project]( https://apchenstu.github.io/TensoRF/ )] 23 | [[Code]( https://github.com/apchenstu/TensoRF )] 24 | 25 | [[Müller et al. (ARXIV '22)]( https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.pdf )] Instant Neural Graphics Primitives with a Multiresolution Hash Encoding 26 | [[Project]( https://nvlabs.github.io/instant-ngp/)] 27 | [[Code]( https://nvlabs.github.io/instant-ngp/ )] 28 | [[Video]( https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.mp4 )] 29 | 30 | [[Schwarz et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.07695.pdf )] VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids 31 | 32 | [[Takikawa et al. (Siggraph '22)]( https://drive.google.com/file/d/1GTFPwQ3oe0etRJKP35oyRhHpsydTE_AR/view )] Variable Bitrate Neural Fields 33 | [[Project]( https://nv-tlabs.github.io/vqad/ )] 34 | [[Video]( https://www.youtube.com/watch?v=Lh0CoTRNFBA )] 35 | 36 | [[Sun et al. (**CVPR '22**)]( https://arxiv.org/pdf/2111.11215.pdf )] Direct Voxel Grid Optimization Super-fast Convergence for Radiance Fields Reconstruction 37 | [[Project]( https://sunset1995.github.io/dvgo/ )] 38 | [[Code]( https://github.com/sunset1995/DirectVoxGO )] 39 | [[Video]( https://youtu.be/gLmujfjRVGw )] 40 | [[DVGOv2](https://arxiv.org/abs/2206.05085)] 41 | 42 | [[Yu et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.05131 )] Plenoxels: Radiance Fields without Neural Networks 43 | [[Project]( https://alexyu.net/plenoxels/ )] 44 | [[Code]( https://github.com/sxyu/svox2 )] 45 | [[Video]( https://www.youtube.com/watch?v=KCDd7UFO1d0&t=6s )] 46 | 47 | [[Xu et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.08845 )] Point-NeRF: Point-based Neural Radiance Fields 48 | [[Project]( https://xharlie.github.io/projects/project_sites/pointnerf/index.html )] 49 | [[Code]( https://github.com/Xharlie/pointnerf )] 50 | 51 | [[Deng et al. (**CVPR '22**)]( https://arxiv.org/abs/2107.02791 )] Depth-Supervised NeRF: Fewer Views and Faster Training for Free 52 | [[Project]( https://www.cs.cmu.edu/~dsnerf/ )] 53 | [[Code](https://github.com/dunbar12138/DSNeRF )] 54 | [[Video]( https://youtu.be/84LFxCo7ogk )] 55 | 56 | [[Takikawa et al. (CVPR '21)]( https://arxiv.org/pdf/2101.10994.pdf )] Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes 57 | [[Project]( https://nv-tlabs.github.io/nglod/ )] 58 | [[Code]( https://github.com/nv-tlabs/nglod )] 59 | [[Video]( https://youtu.be/0cJZn_hV2Ms )] 60 | 61 | [[Garbin et al. (CVPR '21)]( https://arxiv.org/abs/2103.10380 )] FastNeRF: High-Fidelity Neural Rendering at 200FPS 62 | [[Project]( https://microsoft.github.io/FastNeRF/ )] 63 | [[Video]( https://youtu.be/JS5H-Usiphg )] 64 | 65 | #### Dynamic 66 | [[Fang et al. (ARXIV '22)]( https://arxiv.org/abs/2205.15285 )] TiNeuVox: Fast Dynamic Radiance Fields with Time-Aware Neural Voxels 67 | [[Project]( https://jaminfong.cn/tineuvox/ )] 68 | [[Code]( https://github.com/hustvl/TiNeuVox )] 69 | [[Video]( https://youtu.be/sROLfK_VkCk )] 70 | 71 | [[Wang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Fourier_PlenOctrees_for_Dynamic_Radiance_Field_Rendering_in_Real-Time_CVPR_2022_paper.pdf )] Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time 72 | [[Project]( https://aoliao12138.github.io/FPO/ )] 73 | [[Video]( https://youtu.be/XZSuQQOY6Ls )] 74 | 75 | [[Gao et al. (ICCV '21)]( https://arxiv.org/pdf/2105.06468.pdf )] Dynamic View Synthesis from Dynamic Monocular Video 76 | [[Project]( https://free-view-video.github.io/ )] 77 | [[Code](https://github.com/gaochen315/DynamicNeRF )] 78 | [[Video]( https://youtu.be/j8CUzIR0f8M )] 79 | 80 | [[Li et al. (CVPR '21)]( https://arxiv.org/abs/2011.13084 )] Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes 81 | [[Project]( https://www.cs.cornell.edu/~zl548/NSFF// )] 82 | [[Code](https://github.com/zhengqili/Neural-Scene-Flow-Fields )] 83 | [[Video](https://www.cs.cornell.edu/~zl548/NSFF//overview.mp4 )] 84 | 85 | #### Editable 86 | [[Zhang et al. (ARXIV '22)]( https://arxiv.org/abs/2206.06360 )] ARF: Artistic Radiance Fields 87 | [[Project]( https://www.cs.cornell.edu/projects/arf/ )] 88 | [[Code & Data]( https://github.com/Kai-46/ARF-svox2 )] 89 | 90 | [[Kobayashi et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.15585.pdf )] Decomposing NeRF for Editing via Feature Field Distillation 91 | [[Project]( https://pfnet-research.github.io/distilled-feature-fields/ )] 92 | 93 | [[Benaim et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.02776.pdf )] Volumetric Disentanglement for 3D Scene Manipulation 94 | [[Project]( https://sagiebenaim.github.io/volumetric-disentanglement/ )] 95 | 96 | [[Lazova et al. (**CVPR '22**)](https://arxiv.org/abs/2204.10850 )] Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation 97 | 98 | [[Yuan et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.04978.pdf )] NeRF-Editing: Geometry Editing of Neural Radiance Fields 99 | 100 | #### Generalizable 101 | [[Yu et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.00665.pdf )] MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction 102 | [[Project]( https://niujinshuchong.github.io/monosdf )] 103 | 104 | [[Rebain et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Rebain_LOLNerf_Learn_From_One_Look_CVPR_2022_paper.pdf )] LOLNeRF: Learn from One Look 105 | [[Project]( https://ubc-vision.github.io/lolnerf/ )] 106 | 107 | [[Chen et al. (ICCV '21)]( https://arxiv.org/abs/2103.15595 )] MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo 108 | [[Project]( https://apchenstu.github.io/mvsnerf/ )] 109 | [[Code]( https://github.com/apchenstu/mvsnerf )] 110 | [[Video]( https://youtu.be/3M3edNiaGsA )] 111 | 112 | [[Yu et al. (CVPR '21)]( https://arxiv.org/pdf/2012.02190.pdf )] Neural Radiance Fields from One or Few Images 113 | [[Project]( https://alexyu.net/pixelnerf/)] 114 | [[Code]( https://github.com/sxyu/pixel-nerf )] 115 | [[Video]( https://youtu.be/voebZx7f32g )] 116 | 117 | #### Large-scale 118 | [[Tancik et al. (ARXIV '22)](https://arxiv.org/abs/2202.05263)] Block-NeRF: Scalable Large Scene Neural View Synthesis 119 | [[Project]( https://waymo.com/research/block-nerf/)] 120 | [[Video]( https://youtu.be/6lGMCAzBzOQ )] 121 | 122 | [[Zhang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Zhang_NeRFusion_Fusing_Radiance_Fields_for_Large-Scale_Scene_Reconstruction_CVPR_2022_paper.pdf )] NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction 123 | [[Project]( https://jetd1.github.io/NeRFusion-Web/ )] 124 | 125 | #### Sparse input 126 | [[Long et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.05737.pdf)] SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse views 127 | [[Project]( https://www.xxlong.site/SparseNeuS/ )] 128 | 129 | [[Suhail et al. (**CVPR '22**)]( https://arxiv.org/pdf/2112.09687.pdf )] Light Field Neural Rendering 130 | [[Project]( https://light-field-neural-rendering.github.io/ )] 131 | [[Code]( https://github.com/google-research/google-research/tree/master/light_field_neural_rendering )] 132 | 133 | [[Niemeyer et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.00724 )] RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs 134 | [[Project]( https://m-niemeyer.github.io/regnerf )] 135 | [[Code]( https://github.com/google-research/google-research/tree/master/regnerf )] 136 | [[Video](https://www.youtube.com/watch?v=QyyyvA4-Kwc )] 137 | 138 | #### Datasets 139 | [[Downs et al. (ARXIV '22)]( https://arxiv.org/pdf/2204.11918.pdf )] Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Item 140 | [[Blog]( https://ai.googleblog.com/2022/06/scanned-objects-by-google-research.html )] 141 | [[Data]( https://app.gazebosim.org/GoogleResearch/fuel/collections/Scanned%20Objects%20by%20Google%20Research )] 142 | -------------------------------------------------------------------------------- /trends_2018-2019.md: -------------------------------------------------------------------------------- 1 | 2 | ## Trending in 3D Vision 3 | Deep learning not only provides good optimization techniques on feaure engineering, but also gives us unimaginable possibility to combine our intutions into research that could explore this 3D world better. In the research field of 3D Vision, a lot of excellent research work is happening. It's not enough to simply divide them into stereo vision, multi-view, monucular based 3D, we actually want to sort out research topics that could both connect with and distinguish from 2D vision. People have done paper collections on 3D as [link1](https://github.com/flamato/3D-computer-vision-resources) [link2](https://github.com/imkaywu/awesome-3d-vision-list), but they don't usually give us a good bird view, or most-updated infos. Based on above reasons, we create this special collection by summerizing the general ideas behind some most recent papers in 3D vision. 4 | 5 | ### 1. Single Image 3D & Unsupervised Monucular Video Depth 6 | [[Zhang et al. NIPS 2018](http://genre.csail.mit.edu/papers/genre_nips.pdf)] Learning to Reconstruct Shapes from Unseen Classes [[Project](http://genre.csail.mit.edu/)] [[Code]()] 7 | 8 | [[Wu et al. ECCV 2018](http://shapehd.csail.mit.edu/papers/shapehd_eccv.pdf)] Learning Shape Priors for 9 | Single-View 3D Completion and Reconstruction [[Project](http://shapehd.csail.mit.edu/)] [[Code]()] 10 | 11 | [[Niu et al. CVPR 2018](https://kevinkaixu.net/papers/niu_cvpr18_im2struct.pdf)] Im2Struct: Recovering 3D Shape Structure from a Single RGB Image [[Code](https://github.com/chengjieniu/Im2Struct)] 12 | 13 | [[Zou et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Zou_LayoutNet_Reconstructing_the_CVPR_2018_paper.pdf)] LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image [[Code](https://github.com/zouchuhang/LayoutNet)] [[Video](https://www.youtube.com/watch?v=WDzYXRP6XDs&feature=youtu.be)] 14 | 15 | ------------------------- 16 | [[Mahj. et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Mahjourian_Unsupervised_Learning_of_CVPR_2018_paper.pdf)] Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints [[Project](https://sites.google.com/view/vid2depth)] [[Code](https://github.com/tensorflow/models/tree/master/research/vid2depth)] 17 | 18 | [[Yang et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Yang_LEGO_Learning_Edge_CVPR_2018_paper.pdf)] LEGO: Learning Edge with Geometry all at Once by Watching Videos [[Demo](https://www.youtube.com/watch?v=40-GAgdUwI0)] [[Code](https://github.com/zhenheny/LEGO)] 19 | 20 | [[Zou et al. ECCV 2018](https://arxiv.org/abs/1809.01649)] DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency [[Project](http://yuliang.vision/DF-Net/)] [[Code](https://github.com/vt-vl-lab/DF-Net)] 21 | 22 | (`Note: more papers on 'Monucular Video Depth' in CVPR 2017, 2018, ECCV 2018, along with the Robust Vision Challenge`) 23 | 24 | 25 | 26 | 27 | ### 2. 3D Generative Model for Vision beyond Visiable 28 | [[Eslami et al.](Science-Machine Learning)]Neural scene representation and rendering 29 | [[Project](https://deepmind.com/blog/neural-scene-representation-and-rendering/)] Code[[tf-gqn](https://github.com/ogroth/tf-gqn)][[gqn-datasets](https://github.com/deepmind/gqn-datasets)] 30 | [[Pytorch-qgn](https://github.com/iShohei220/torch-gqn)]'seems to run quite slow' 31 | 32 | [[Tulsiani et al. ECCV 2018](https://arxiv.org/pdf/1807.10264.pdf)] Layer-structured 3D Scene Inference 33 | via View Synthesis [[Project](https://shubhtuls.github.io/lsi/)] [[Code](https://github.com/google/layered-scene-inference)] 34 | 35 | [[Rama. et al. ECCV 2018](https://arxiv.org/abs/1807.11010)] Sidekick Policy Learning for Active Visual Exploration [[Project](http://vision.cs.utexas.edu/projects/sidekicks/)] [[Code](https://github.com/srama2512/sidekicks)] 36 | 37 | [[Song et al. CVPR 2018](https://arxiv.org/abs/1712.04569)] Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View [[Project](http://im2pano3d.cs.princeton.edu/)] [[Code](https://github.com/shurans/im2pano3d/)] 38 | 39 | 40 | 41 | ### 3. Pose Estimation 42 | 43 | #### 3.1 Scene Layout and Object Pose 44 | [[Wang et al. ARXIV pre-print 2019](https://arxiv.org/abs/1901.04780)]DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion 45 | [[Code](https://github.com/j96w/DenseFusion)] 46 | [[Project](https://sites.google.com/view/densefusion)] 47 | [[Video](https://www.youtube.com/watch?v=SsE5-FuK5jo)] 48 | 49 | [[Zhao et al. ARXIV pre-print 2019](https://arxiv.org/abs/1812.01387)]Estimating 6D Pose From Localizing Designated Surface Keypoints 50 | [[Code](https://github.com/sjtuytc/betapose)] 51 | 52 | [[Huang et al. NIPS 2018](https://arxiv.org/pdf/1810.13049.pdf)] Cooperative Holistic Scene Understanding: Unifying3D Object, Layout, and Camera Pose Estimation [[Video](https://www.youtube.com/watch?v=kXCugGwnr68)] 53 | 54 | 55 | [[Trem. et al. CoRL 2018](https://arxiv.org/pdf/1809.10790.pdf)] Deep Object Pose Estimation for Semantic Robotic 56 | Grasping of Household Objects [[Project](https://research.nvidia.com/publication/2018-09_Deep-Object-Pose)] [[Code](https://github.com/NVlabs/Deep_Object_Pose)] 57 | 58 | [[Sundermeyer et al. ECCV 2018](http://openaccess.thecvf.com/content_ECCV_2018/papers/Martin_Sundermeyer_Implicit_3D_Orientation_ECCV_2018_paper.pdf)]Implicit 3D Orientation Learning for 59 | 6D Object Detection from RGB Images(**Best paper Award**) 60 | [[Code](https://github.com/DLR-RM/AugmentedAutoencoder)] 61 | [[Supplement](https://static-content.springer.com/esm/chp%3A10.1007%2F978-3-030-01231-1_43/MediaObjects/474211_1_En_43_MOESM1_ESM.pdf)] 62 | [[Video](https://www.youtube.com/watch?v=jgb2eNNlPq4)] 63 | 64 | [[Tulsiani et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Tulsiani_Factoring_Shape_Pose_CVPR_2018_paper.pdf)] Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene [[Project](https://shubhtuls.github.io/factored3d/)] [[Code](https://github.com/shubhtuls/factored3d)] 65 | 66 | [[Tulsiani et al. CVPR 2018](https://arxiv.org/pdf/1801.03910.pdf)]Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction 67 | [[Project](https://shubhtuls.github.io/mvcSnP/)] 68 | [[Code](https://github.com/shubhtuls/mvcSnP)] 69 | 70 | [[Tekin et al. CVPR 2018](https://arxiv.org/pdf/1711.08848.pdf)] Real-Time Seamless Single Shot 6D Object Pose Prediction [[Code](https://github.com/Microsoft/singleshotpose)] [[Supp.](http://openaccess.thecvf.com/content_cvpr_2018/Supplemental/3117-supp.pdf)] 71 | 72 | [[Li et. al. ECCV 2018](https://arxiv.org/abs/1804.00175)] DeepIM: Deep Iterative Matching for 6D Pose Estimation [[Code](https://github.com/liyi14/mx-DeepIM0)] 73 | 74 | [[Qi et al. CVPR 2018](https://arxiv.org/pdf/1711.08488.pdf)] Frustum PointNets for 3D Object Detection from RGB-D Data [[Project](http://stanford.edu/~rqi/frustum-pointnets/)] [[Code](https://github.com/charlesq34/frustum-pointnets)] 75 | 76 | #### 3.2 Body Pose 77 | [[Guo et al. ECCV 2018](http://openaccess.thecvf.com/content_ECCV_2018/papers/Michelle_Guo_Neural_Graph_Matching_ECCV_2018_paper.pdf)] Neural Graph Matching Networks for Fewshot 3D Action Recognition (`Pose for action recognition`) 78 | 79 | [[Groueix et al. ECCV 2018](https://arxiv.org/pdf/1806.05228.pdf)] 3D-CODED : 3D Correspondences by Deep Deformation [[Project](http://imagine.enpc.fr/~groueixt/3D-CODED/)] [[Code](https://github.com/ThibaultGROUEIX/3D-CODED)] 80 | 81 | [[Joo et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Joo_Total_Capture_A_CVPR_2018_paper.pdf)] Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies [[Project](http://www.cs.cmu.edu/~hanbyulj/totalcapture/)] [[Supp.](http://www.cs.cmu.edu/~hanbyulj/totalcapture/totalBody_camready_supp.pdf)] 82 | 83 | [[Riza et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Guler_DensePose_Dense_Human_CVPR_2018_paper.pdf)] 84 | Dense Human Pose Estimation In The Wild [[Project](http://densepose.org/)] [[Code](https://github.com/facebookresearch/DensePose)] 85 | 86 | [[Pavl. et al. CVPR 2018](https://arxiv.org/pdf/1805.04092.pdf)] Learning to Estimate 3D Human Pose and Shape from a Single Color Image [[Project](https://www.seas.upenn.edu/~pavlakos/projects/humanshape/)] 87 | 88 | #### 3.3 Face Pose 89 | [[Moniz et al. NIPS 2018](https://papers.nips.cc/paper/8181-unsupervised-depth-estimation-3d-face-rotation-and-replacement.pdf)] Unsupervised Depth Estimation, 90 | 3D Face Rotation and Replacement [[Code releasing soon](https://github.com/joelmoniz/DepthNets/)] 91 | 92 | [[Feng et al. ECCV 2018](https://arxiv.org/pdf/1803.07835v1.pdf)] Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network [[Code](https://github.com/YadiraF/PRNet)] [[Video](https://www.youtube.com/watch?v=tXTgLSyIha8&feature=youtu.be)] 93 | 94 | [[Genova et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Genova_Unsupervised_Training_for_CVPR_2018_paper.pdf)] Unsupervised Training for 3D Morphable Model Regression [[Project]()] [[Code](https://github.com/google/tf_mesh_renderer)] 95 | 96 | [[Deng et al. CVPR 2018](https://arxiv.org/pdf/1712.04695.pdf)] UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition 97 | 98 | #### 3.4 Hand Pose 99 | [[Tsoli et al. ECCV 2018](http://openaccess.thecvf.com/content_ECCV_2018/papers/Aggeliki_Tsoli_Joint_3D_tracking_ECCV_2018_paper.pdf)] Joint 3D Tracking of a Deformable Object in Interaction with a Hand 100 | 101 | [[Ge et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/papers/Ge_Hand_PointNet_3D_CVPR_2018_paper.pdf)] Hand PointNet: 3D Hand Pose Estimation using Point Sets [[Project](https://sites.google.com/site/geliuhaontu/home/cvpr2018)] [[Code](https://sites.google.com/site/geliuhaontu/HandPointNet.zip?attredirects=0&d=1)] [[VIdeo](https://youtu.be/-eiZYOo8cWc)] 102 | 103 | [[Baek et al. CVPR 2018](https://arxiv.org/abs/1805.04497)] Augmented skeleton space transfer for depth-based hand pose estimation 104 | 105 | (`Note: more papers on 'Hand Pose Estimation' in CVPR 2018, ECCV 2018`) 106 | 107 | 108 | ### 4. Disentangle Representations in 3D 109 | [[Yao et al. NIPS 2018](https://arxiv.org/pdf/1808.09351.pdf)] 3D-Aware Scene Manipulation via Inverse Graphics [[Project](http://3dsdn.csail.mit.edu/)] 110 | 111 | [[Smith et al. NIPS 2018](https://papers.nips.cc/paper/7883-multi-view-silhouette-and-depth-decomposition-for-high-resolution-3d-object-representation.pdf)] Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation [[Project](https://sites.google.com/site/mvdnips2018)] 112 | 113 | [[Zhu et al. NIPS 2018](https://papers.nips.cc/paper/7297-visual-object-networks-image-generation-with-disentangled-3d-representations.pdf)] Visual Object Networks: Image Generation with 114 | Disentangled 3D Representation 115 | 116 | 117 | 118 | ### 5. Unsupervised Key Points Detection 119 | [[Suwa. et al. NIPS 2018](https://arxiv.org/pdf/1807.03146.pdf)] Discovery of Latent 3D Keypoints via 120 | End-to-end Geometric Reasoning [[Project](https://keypointnet.github.io/)] [[Code](https://github.com/tensorflow/models/tree/master/research/keypointnet)] 121 | 122 | [[Zhou et al. ECCV 2018](https://arxiv.org/pdf/1712.05765.pdf)] Unsupervised Domain Adaptation for 3D 123 | Keypoint Estimation via View Consistency [[Code](https://github.com/xingyizhou/3DKeypoints-DA)] [[Results](https://drive.google.com/file/d/1UtlL7moKtNoVGyqWGRn8_c_57dwiqlVm/view)] 124 | 125 | 126 | 127 | ### 6. Point Cloud(PCL) Processing 128 | #### 6.1 Neural Network for 3D data 129 | [[Tata. et al. CVPR 2018](http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/0144.pdf)] Tangent Convolutions for Dense Prediction in 3D [[Code](https://github.com/tatarchm/tangent_conv)] 130 | 131 | [[Su et al. CVPR 2018](https://arxiv.org/abs/1802.08275)] SPLATNet: Sparse Lattice Networks for Point Cloud Processing 132 | [[Project](http://siyuanhuang.com/cooperative_parsing/main.html)] [[Code](https://github.com/NVlabs/splatnet)] 133 | 134 | [[Qi et al. NIPS 2017](https://arxiv.org/abs/1706.02413)] PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space [[Project](http://stanford.edu/~rqi/pointnet2/)] [[Code](https://github.com/charlesq34/pointnet2)] 135 | 136 | [[Weiler et al. NIPS 2018](https://papers.nips.cc/paper/8239-3d-steerable-cnns-learning-rotationally-equivariant-features-in-volumetric-data.pdf)] 3D Steerable CNNs: Learning Rotationally 137 | Equivariant Features in Volumetric Data [[Project]()] [[Code](https://github.com/mariogeiger/se3cnn)] 138 | 139 | [[Sung et al. NIPS 2018](https://papers.nips.cc/paper/7330-deep-functional-dictionaries-learning-consistent-semantic-structures-on-3d-models-from-functions.pdf)] Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions [[Project]()] [[Code](https://github.com/mhsung/deep-functional-dictionaries)] 140 | 141 | 142 | #### 6.2 3D Registration & Rendering 143 | [[Nguy. et al. NIPS 2018](https://papers.nips.cc/paper/8014-rendernet-a-deep-convolutional-network-for-differentiable-rendering-from-3d-shapes.pdf)] RenderNet: A deep convolutional network for 144 | differentiable rendering from 3D shapes 145 | 146 | [[Kim et al. ECCV 2018](https://arxiv.org/pdf/1807.02587.pdf)] Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures [[Project](https://research.nvidia.com/publication/2018-09_HGMM-Registration)] [[Video](https://www.youtube.com/watch?v=Bczht9CspiY)] 147 | 148 | [[Sung et al. Siggraph Asia 2017](https://arxiv.org/abs/1708.01841)] ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling [[Project](https://mhsung.github.io/complement-me.html)] [[Code](https://github.com/mhsung/complement-me)] 149 | 150 | [[Zhu et al. SIGGRAPH Asia 2018]()] SCORES: Shape Composition with Recursive Substructure Priors [[Project](https://kevinkaixu.net/projects/scores.html)] [[Code](https://kevinkaixu.net/projects/scores.html#code)] 151 | 152 | [[Este. et al. 3DV 2018](https://vision.in.tum.de/_media/spezial/bib/estellers2018.pdf)] Robust Fitting of Subdivision Surfaces for Smooth Shape Analysis [[Project]()] [[Code](https://bitbucket.org/ginie/subdivision_surfaces_3dv2018/src/master/)] 153 | 154 | [[Kato et al. CVPR 2018](https://arxiv.org/abs/1711.07566)] Neural 3D Mesh Renderer [[Project](http://hiroharu-kato.com/projects_en/neural_renderer.html)] [[Code](https://github.com/hiroharu-kato/neural_renderer)] 155 | 156 | 157 | 158 | 159 | ### 7. SLAM today 160 | #### 7.1 3D Reconstruction & SLAM 161 | [[Shi et al. ECCV 2018](https://arxiv.org/pdf/1803.08407)] PlaneMatch: Patch Coplanarity Prediction for 162 | Robust RGB-D Reconstruction [[Project](http://www.yifeishi.net/planematch.html)] [[Code](https://github.com/yifeishi/PlaneMatch)] 163 | 164 | [[Bloesch et al. CVPR 2018](https://arxiv.org/abs/1804.00874)] CodeSLAM — Learning a Compact, Optimisable Representation for Dense Visual SLAM [[Project](http://www.imperial.ac.uk/dyson-robotics-lab/projects/codeslam/)] [[Video](https://www.youtube.com/watch?v=PbSggzaZWAQ&t=1s)] 165 | 166 | [[Berg. et al. ICRA 2018](https://arxiv.org/abs/1710.02081)] Online Photometric Calibration of Auto Exposure Video for Realtime Visual Odometry and SLAM [[Project](https://vision.in.tum.de/research/vslam/photometric-calibration)] [[Code](https://vision.in.tum.de/research/vslam/photometric-calibration)] 167 | 168 | #### 7.2 Object-aware SLAM 169 | [[Rünz et al. ISMAR 2018](https://arxiv.org/pdf/1804.09194.pdf)] MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects [[Project](http://visual.cs.ucl.ac.uk/pubs/maskfusion/index.html)] [[Code](https://github.com/martinruenz/maskfusion)] [[Demo](http://visual.cs.ucl.ac.uk/pubs/maskfusion/MaskFusion.webm)] 170 | 171 | [[McCo. et al. 3DV 2018](https://www.doc.ic.ac.uk/~sleutene/publications/fusion_plusplus_3dv_camera_ready.pdf)] Fusion++: Volumetric Object-Level SLAM [[Video](https://www.youtube.com/watch?v=2luKNC03x4k&feature=youtu.be)] 172 | 173 | [[Zhou et al. ECCV 2018](https://arxiv.org/pdf/1808.01900.pdf)] DeepTAM: Deep Tracking and Mapping [[Project](https://lmb.informatik.uni-freiburg.de/people/zhouh/deeptam/)] [[Code](https://github.com/lmb-freiburg/deeptam)] 174 | 175 | #### 7.3 3D Photography 176 | [[Hedman et al. SIGGRAPH](http://visual.cs.ucl.ac.uk/pubs/instant3d/instant3d_siggraph_2018.pdf)] Instant 3D Photography [[Project](http://visual.cs.ucl.ac.uk/pubs/instant3d/)] [[Code](http://visual.cs.ucl.ac.uk/pubs/instant3d/implementation_details.pdf)] 177 | 178 | [[Chen et al. ECCV 2018](http://gychen.org/PS-FCN/)] PS-FCN: A Flexible Learning Framework for Photometric Stereo [[Project](http://gychen.org/PS-FCN/)] [[Code](https://github.com/guanyingc/PS-FCN)] 179 | 180 | ## Misc: 181 | ### Workshops 182 | [Bridge to CVPR 3D workshop](https://bridgesto3d.github.io/#schedule) 183 | 184 | ### Dataset 2018 185 | [[Trem. et al. CVPR 2018 Workshop](https://research.nvidia.com/publication/2018-06_Falling-Things)] Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation [[Dataset](https://drive.google.com/open?id=1y4h9T6D9rf6dAmsRwEtfzJdcghCnI_01)] 186 | 187 | [[Sun et al. CVPR 2018](https://arxiv.org/pdf/1804.04610.pdf)] Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling [[Dataset](https://github.com/xingyuansun/pix3d)] 188 | 189 | [[Muel. et al. CVPR 2018](https://arxiv.org/abs/1712.01057)] GANerated Hands Dataset [[Dataset](http://handtracker.mpi-inf.mpg.de/projects/GANeratedHands/GANeratedDataset.htm)] 190 | 191 | SUMO challenge dataset [[Dataset](https://sumochallenge.org/)] [[Code](https://github.com/facebookresearch/sumo-challenge)] 192 | ### More resources 193 | Amesome 3D machine learning collection [here](https://github.com/timzhang642/3D-Machine-Learning) 194 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Trending in 3D Vision 2 | I first got fascinated by the beauty of 3D vision since 2015. After that, so many new and wonderful ideas, works have been brought into this field, and it seems so hard to catch up with this fast-evolving area today. This leads to the major motivation behind this paper reading list: to get a sense of current SOTA methods, and an overview of the research trending in the field of 3D vision, mainly with deep learning. 3 | 4 | From this list, you may say, various applications, multiple modalities of data, powerful neural backbones are the major working horses, or the boom of neural radiance field and differentiable rendering inspire a lot of new methods and tasks, or you want to point out that self-supervision, data-efficient learning are the critical keys. Different people may have different opinions, but this list is about existing possibilities in 3D vision, to which you may say 'wow, this is even possible', or 'aha, I never imagined such a method'. 5 | 6 | Note that this repo started as a self-collected paper list based on my own appetite, which may reflect some bias. Some may not be precisely categorized, for which you can raise an issue, or send a pull request. 7 | 8 | ### [1. SLAM with Deep Learning](#content) 9 | [[Chen et al. (ARXIV '22)]( https://arxiv.org/pdf/2203.01087.pdf )] Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications 10 | 11 | [[Avraham et al. (ARXIV '22)]( https://arxiv.org/abs/2206.01916 )] Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation 12 | 13 | [[Hughe et al. (ARXIV '22)]( https://arxiv.org/pdf/2201.13360.pdf )] Hydra: A Real-time Spatial Perception Engine for 3D Scene Graph Construction and Optimization 14 | [[Video]( https://youtu.be/qZg2lSeTuvM )] 15 | 16 | [[Zhu at al. (**CVPR '22**)]( https://arxiv.org/pdf/2112.12130.pdf )] NICE-SLAM: Neural Implicit Scalable Encoding for SLAM 17 | [[Project]( https://pengsongyou.github.io/nice-slam )] 18 | [[Code]( https://github.com/cvg/nice-slam )] 19 | [[Video](https://youtu.be/V5hYTz5os0M )] 20 | 21 | [[Teed et al. (NeurIPS '21)]( https://arxiv.org/pdf/2108.10869.pdf )] DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras 22 | [[Code]( https://github.com/princeton-vl/DROID-SLAM )] 23 | [[Video]( https://www.youtube.com/watch?v=GG78CSlSHSA )] 24 | 25 | [[Yang et al. (3DV '21)]( https://arxiv.org/abs/2111.07418 )] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo 26 | [[Project]( https://vision.in.tum.de/research/vslam/tandem )] 27 | [[Code]( https://github.com/tum-vision/tandem )] 28 | [[Video](https://youtu.be/L4C8Q6Gvl1w)] 29 | 30 | [[Lin et al. (ARXIV '21)]( https://arxiv.org/pdf/2109.07982.pdf )] R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package 31 | [[Code]( https://github.com/hku-mars/r3live )] 32 | [[Video]( https://youtu.be/j5fT8NE5fdg )] 33 | 34 | [[Duzceker et al. (CVPR '21)](https://openaccess.thecvf.com/content/CVPR2021/papers/Duzceker_DeepVideoMVS_Multi-View_Stereo_on_Video_With_Recurrent_Spatio-Temporal_Fusion_CVPR_2021_paper.pdf)] DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion 35 | [[Code]( https://github.com/ardaduz/deep-video-mvs )] 36 | [[Video]( https://www.youtube.com/watch?v=ikpotjxwcp4 )] 37 | 38 | [[Teed at al. (CVPR '21)]( https://arxiv.org/pdf/2103.12032.pdf )] Tangent Space Backpropagation for 3D Transformation Groups 39 | [[Code]( https://github.com/princeton-vl/lietorch )] 40 | 41 | [[Sun et al. (CVPR '21)]( https://arxiv.org/pdf/2104.00681.pdf )] NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video 42 | [[Project]( http://zju3dv.github.io/neuralrecon/ )] 43 | [[Code]( https://github.com/zju3dv/NeuralRecon/ )] 44 | 45 | [[Murthy J. et al. (ICRA '20)](https://arxiv.org/pdf/1910.10672.pdf)] ∇SLAM: Automagically differentiable SLAM 46 | [[Project]( https://github.com/gradslam/gradslam )] 47 | [[Code]( https://gradslam.github.io/ )] 48 | 49 | [[Schops et al. (CVPR '19)]( https://openaccess.thecvf.com/content_CVPR_2019/papers/Schops_BAD_SLAM_Bundle_Adjusted_Direct_RGB-D_SLAM_CVPR_2019_paper.pdf )] BAD SLAM: Bundle Adjusted Direct RGB-D SLAM 50 | [[Project]( https://www.eth3d.net/ )] 51 | [[Code]( https://github.com/ETH3D/badslam )] 52 | 53 | ### [2. Dynamic Human, Animals and Objects Reconstruction](#content) 54 | #### Human avatars 55 | [[Su et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.01666.pdf )] DANBO: Disentangled Articulated Neural Body 56 | Representations via Graph Neural Networks 57 | [[Project]( https://lemonatsu.github.io/danbo/ )] 58 | 59 | [[Jiang et al. (**CVPR '22**)]( https://arxiv.org/pdf/2201.12792.pdf )] SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video 60 | [[Project]( https://jby1993.github.io/SelfRecon/ )] 61 | [[Code]( https://github.com/jby1993/SelfReconCode )] 62 | 63 | [[Weng et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.04127 )] HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video 64 | [[Project]( https://grail.cs.washington.edu/projects/humannerf/ )] 65 | [[Code]( https://github.com/chungyiweng/humannerf )] 66 | [[Video]( https://youtu.be/GM-RoZEymmw )] 67 | 68 | [[Yu et al. (CVPR '21)](https://arxiv.org/pdf/2105.01859.pdf )] Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors 69 | [[Project]( http://www.liuyebin.com/Function4D/Function4D.html )] 70 | [[Data]( https://github.com/ytrock/THuman2.0-Dataset )] 71 | [[Video]( http://www.liuyebin.com/Function4D/assets/supp_video.mp4 )] 72 | 73 | #### Animals capture 74 | [[Yang et al. (**CVPR '22**)]( https://banmo-www.github.io/banmo-2-14.pdf )] BANMo: Building Animatable 3D Neural Models from Many Casual Videos 75 | [[Project]( https://banmo-www.github.io/ )] 76 | [[Code]( https://github.com/facebookresearch/banmo )] 77 | [[Video]( https://youtu.be/1NUa-yvFGA0 )] 78 | 79 | [[Wu et al. (NeurIPS '21)]( https://arxiv.org/abs/2107.10844 )] DOVE: Learning Deformable 3D Objects by Watching Videos 80 | [[Project]( https://dove3d.github.io/ )] 81 | [[Video]( https://youtu.be/_FsADb0XmpY )] 82 | 83 | #### Human-object interaction 84 | [[Jiang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Jiang_NeuralHOFusion_Neural_Volumetric_Rendering_Under_Human-Object_Interactions_CVPR_2022_paper.pdf)] NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions 85 | [[Project]( https://nowheretrix.github.io/neuralfusion/ )] 86 | [[Video]( https://youtu.be/Stvks4rZMF0 )] 87 | 88 | [[Hasson et al. (CVPR '21)]( https://arxiv.org/abs/2108.07044 )] Towards unconstrained joint hand-object reconstruction from RGB videos 89 | [[Project](https://hassony2.github.io/homan.html )] 90 | [[Code]( https://github.com/hassony2/homan )] 91 | 92 | #### Scene-level 3D dynamics 93 | [[Grauman et al. (**CVPR '22**)]( https://arxiv.org/abs/2110.07058 )] Ego4D: Around the World in 3,000 Hours of Egocentric Video 94 | [[Project]( https://ego4d-data.org/ )] 95 | [[Code]( https://github.com/EGO4D )] 96 | 97 | [[Li et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Li_Neural_3D_Video_Synthesis_From_Multi-View_Video_CVPR_2022_paper.pdf )] Neural 3D Video Synthesis From Multi-View Video 98 | [[Project]( https://neural-3d-video.github.io/ )] 99 | [[Video]( https://neural-3d-video.github.io/resources/video.mp4 )] 100 | [[Data]( https://github.com/facebookresearch/Neural_3D_Video )] 101 | 102 | [[Zhang et al. (SIGGRAPH '21)]( https://arxiv.org/abs/2108.01166 )] Consistent Depth of Moving Objects in Video 103 | [[Project]( https://dynamic-video-depth.github.io/ )] 104 | [[Code]( https://github.com/google/dynamic-video-depth )] 105 | [[Video]( https://dynamic-video-depth.github.io/#video )] 106 | 107 | [[Zeed et al. (CVPR '21)]( https://arxiv.org/abs/2012.00726 )] RAFT-3D: Scene Flow using Rigid-Motion Embeddings 108 | [[Code]( https://github.com/princeton-vl/RAFT-3D )] 109 | 110 | [[Lu et al. (CVPR '21)]( https://arxiv.org/pdf/2105.06993.pdf )] Omnimatte: Associating Objects and Their Effects in Video 111 | [[Project]( https://omnimatte.github.io/ )] 112 | [[Code]( https://omnimatte.github.io/#code )] 113 | [[Video]( https://omnimatte.github.io/#video )] 114 | 115 | ### [3. Self-Supervised 3D Representations Learning](#content) 116 | [[Hasselgren et al. (ARXIV '22)]( https://arxiv.org/abs/2206.03380 )] Shape, Light & Material Decomposition from Images using Monte Carlo Rendering and Denoising 117 | 118 | [[Gkioxari et al. (ARXIV '22)]( https://drive.google.com/file/d/1E6xSbUzuu6soAA-jkaGCFl97LZ8SVRvr/view )] Learning 3D Object Shape and Layout without 3D Supervision 119 | [[Project]( https://gkioxari.github.io/usl/ )] 120 | [[Video]( https://youtu.be/PKhGIiMuRJU )] 121 | 122 | [[Boss et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.15768.pdf)] SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections 123 | [[Project]( https://markboss.me/publication/2022-samurai/ )] 124 | [[Video]( https://youtu.be/LlYuGDjXp-8 )] 125 | 126 | [[Wei et al. (SIGGRAPH '22)]( https://arxiv.org/pdf/2205.02961.pdf )] Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search 127 | [[Project]( https://colin97.github.io/CoACD/ )] 128 | [[Code]( https://github.com/SarahWeiii/CoACD )] 129 | [[Video]( https://www.youtube.com/watch?v=r12O0z0723s )] 130 | 131 | [[Vicini et al. (SIGGRAPH '22)]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Vicini2022sdf_1.pdf )] Differentiable Signed Distance Function Rendering 132 | [[Project]( http://rgl.epfl.ch/publications/Vicini2022SDF )] 133 | [[Video]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Vicini2022sdf.mp4 )] 134 | 135 | [[Or-El et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.11427 )] StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation 136 | [[Project]( https://stylesdf.github.io/ )] 137 | [[Code](https://github.com/royorel/StyleSDF)] 138 | [[Demo]( https://colab.research.google.com/github/royorel/StyleSDF/blob/main/StyleSDF_demo.ipynb )] 139 | 140 | [[Girdhar et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.08377 )] Omnivore: A Single Model for Many Visual Modalities 141 | [[Project]( https://facebookresearch.github.io/omnivore )] 142 | [[Code](https://github.com/facebookresearch/omnivore )] 143 | 144 | 145 | 146 | [[Noguchi et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Noguchi_Watch_It_Move_Unsupervised_Discovery_of_3D_Joints_for_Re-Posing_CVPR_2022_paper.pdf )] Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects 147 | [[Project]( https://nvlabs.github.io/watch-it-move/ )] 148 | [[Code]( https://github.com/NVlabs/watch-it-move )] 149 | [[Video]( https://youtu.be/oRnnuCVV89o )] 150 | 151 | [[Gong et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.15625.pdf )] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision 152 | [[Code]( https://github.com/garfield-kh/posetriplet )] 153 | 154 | [[Wu et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.02306 )] Toward Practical Monocular Indoor Depth Estimation 155 | [[Project]( https://distdepth.github.io/ )] 156 | [[Code]( https://github.com/facebookresearch/DistDepth )] 157 | [[Video]( https://youtu.be/s9JdoR1xbz8 )] 158 | [[Data](https://drive.google.com/file/d/1KfDFyTg9-1w1oJB4oT-DUjKC6LG0enwb/view?usp=sharing)] 159 | 160 | 161 | [[Wei et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Wei_Self-Supervised_Neural_Articulated_Shape_and_Appearance_Models_CVPR_2022_paper.pdf )] Self-supervised Neural Articulated Shape and Appearance Models 162 | [[Project]( https://weify627.github.io/nasam/ )] 163 | [[Video]( https://youtu.be/0YbhTxALi8M )] 164 | 165 | [[Chan et al. (**CVPR '22**)](https://arxiv.org/pdf/2112.07945.pdf)] EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks 166 | [[Project]( https://nvlabs.github.io/eg3d/ )] 167 | [[Code]( https://github.com/NVlabs/eg3d )] 168 | [[Video]( https://www.youtube.com/watch?v=cXxEwI7QbKg )] 169 | 170 | [[Rombach et al. (ICCV '21)]( https://arxiv.org/pdf/2104.07652.pdf)] 171 | Geometry-Free View Synthesis: Transformers and no 3D Priors 172 | Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations 173 | [[Project]( https://compvis.github.io/geometry-free-view-synthesis/ )] 174 | [[Code]( https://github.com/CompVis/geometry-free-view-synthesis )] 175 | [[Video]( https://github.com/CompVis/geometry-free-view-synthesis/blob/master/assets/acid_long.mp4 )] 176 | 177 | [[Harley et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Harley_Track_Check_Repeat_An_EM_Approach_to_Unsupervised_Tracking_CVPR_2021_paper.pdf )] Track, Check, Repeat: An EM Approach to Unsupervised Tracking 178 | [[Project]( http://www.cs.cmu.edu/~aharley/em_cvpr21/ )] 179 | [[Code]( https://github.com/aharley/track_check_repeat )] 180 | 181 | [[Watson et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Watson_The_Temporal_Opportunist_Self-Supervised_Multi-Frame_Monocular_Depth_CVPR_2021_paper.pdf )] The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth 182 | [[Code]( https://github.com/nianticlabs/manydepth )] 183 | [[Video]( https://storage.googleapis.com/niantic-lon-static/research/manydepth/manydepth_cvpr_cc.mp4 )] 184 | 185 | [[Nicolet et al. (SIGGRAPH Asia '21)]( http://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2021Large.pdf )] Large Steps in Inverse Rendering of Geometry 186 | [[Project]( https://rgl.epfl.ch/publications/Nicolet2021Large )] 187 | [[Code]( https://github.com/rgl-epfl/cholespy )] 188 | [[Video]( https://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2021Large_1.mp4 )] 189 | 190 | [[Wu et al. (CVPR '20)](https://openaccess.thecvf.com/content_CVPR_2020/papers/Wu_Unsupervised_Learning_of_Probably_Symmetric_Deformable_3D_Objects_From_Images_CVPR_2020_paper.pdf )] Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild 191 | [[Project]( https://elliottwu.com/projects/20_unsup3d/ )] 192 | [[Code]( https://github.com/elliottwu/unsup3d )] 193 | [[Video]( https://youtu.be/p3KB3eIQw24 )] 194 | 195 | 196 | ### [4. Breakthroughs in Deep Implicit Functions](#content) 197 | 198 | #### Topology-aware 199 | [[Palafox et al. (**CVPR '22**)]( https://arxiv.org/pdf/2201.08141.pdf )] SPAMs: Structured Implicit Parametric Models 200 | [[Project]( https://pablopalafox.github.io/spams/ )] 201 | [[Video]( https://www.youtube.com/watch?v=ChdjHNGgrzI )] 202 | 203 | [[Park et al. (SIGGRAPH Asia '21)]( https://arxiv.org/pdf/2106.13228.pdf )] A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields 204 | [[Project]( https://hypernerf.github.io/ )] 205 | [[Code]( https://github.com/google/hypernerf )] 206 | [[Video]( https://youtu.be/qzgdE_ghkaI )] 207 | 208 | #### Additional priors 209 | [[Guo et al. (**CVPR '22**)]( https://arxiv.org/abs/2205.02836 )] Neural 3D Scene Reconstruction with the Manhattan-world Assumption 210 | [[Project]( https://zju3dv.github.io/manhattan_sdf/)] 211 | [[Code]( https://github.com/zju3dv/manhattan_sdf )] 212 | [[Video]( https://www.youtube.com/watch?v=oEE7mK0YQtc )] 213 | 214 | #### Faster, memory-efficient 215 | [[Chen et al. (ARXIV '22)]( https://arxiv.org/pdf/2203.09517.pdf )] TensoRF: Tensorial Radiance Fields 216 | [[Project]( https://apchenstu.github.io/TensoRF/ )] 217 | [[Code]( https://github.com/apchenstu/TensoRF )] 218 | 219 | [[Müller et al. (ARXIV '22)]( https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.pdf )] Instant Neural Graphics Primitives with a Multiresolution Hash Encoding 220 | [[Project]( https://nvlabs.github.io/instant-ngp/)] 221 | [[Code]( https://nvlabs.github.io/instant-ngp/ )] 222 | [[Video]( https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.mp4 )] 223 | 224 | [[Schwarz et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.07695.pdf )] VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids 225 | 226 | [[Takikawa et al. (Siggraph '22)]( https://drive.google.com/file/d/1GTFPwQ3oe0etRJKP35oyRhHpsydTE_AR/view )] Variable Bitrate Neural Fields 227 | [[Project]( https://nv-tlabs.github.io/vqad/ )] 228 | [[Video]( https://www.youtube.com/watch?v=Lh0CoTRNFBA )] 229 | 230 | [[Sun et al. (**CVPR '22**)]( https://arxiv.org/pdf/2111.11215.pdf )] Direct Voxel Grid Optimization Super-fast Convergence for Radiance Fields Reconstruction 231 | [[Project]( https://sunset1995.github.io/dvgo/ )] 232 | [[Code]( https://github.com/sunset1995/DirectVoxGO )] 233 | [[Video]( https://youtu.be/gLmujfjRVGw )] 234 | [[DVGOv2](https://arxiv.org/abs/2206.05085)] 235 | 236 | [[Yu et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.05131 )] Plenoxels: Radiance Fields without Neural Networks 237 | [[Project]( https://alexyu.net/plenoxels/ )] 238 | [[Code]( https://github.com/sxyu/svox2 )] 239 | [[Video]( https://www.youtube.com/watch?v=KCDd7UFO1d0&t=6s )] 240 | 241 | [[Xu et al. (**CVPR '22**)]( https://arxiv.org/abs/2201.08845 )] Point-NeRF: Point-based Neural Radiance Fields 242 | [[Project]( https://xharlie.github.io/projects/project_sites/pointnerf/index.html )] 243 | [[Code]( https://github.com/Xharlie/pointnerf )] 244 | 245 | [[Deng et al. (**CVPR '22**)]( https://arxiv.org/abs/2107.02791 )] Depth-Supervised NeRF: Fewer Views and Faster Training for Free 246 | [[Project]( https://www.cs.cmu.edu/~dsnerf/ )] 247 | [[Code](https://github.com/dunbar12138/DSNeRF )] 248 | [[Video]( https://youtu.be/84LFxCo7ogk )] 249 | 250 | [[Takikawa et al. (CVPR '21)]( https://arxiv.org/pdf/2101.10994.pdf )] Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes 251 | [[Project]( https://nv-tlabs.github.io/nglod/ )] 252 | [[Code]( https://github.com/nv-tlabs/nglod )] 253 | [[Video]( https://youtu.be/0cJZn_hV2Ms )] 254 | 255 | [[Garbin et al. (CVPR '21)]( https://arxiv.org/abs/2103.10380 )] FastNeRF: High-Fidelity Neural Rendering at 200FPS 256 | [[Project]( https://microsoft.github.io/FastNeRF/ )] 257 | [[Video]( https://youtu.be/JS5H-Usiphg )] 258 | 259 | #### Dynamic 260 | [[Fang et al. (ARXIV '22)]( https://arxiv.org/abs/2205.15285 )] TiNeuVox: Fast Dynamic Radiance Fields with Time-Aware Neural Voxels 261 | [[Project]( https://jaminfong.cn/tineuvox/ )] 262 | [[Code]( https://github.com/hustvl/TiNeuVox )] 263 | [[Video]( https://youtu.be/sROLfK_VkCk )] 264 | 265 | [[Wang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Fourier_PlenOctrees_for_Dynamic_Radiance_Field_Rendering_in_Real-Time_CVPR_2022_paper.pdf )] Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time 266 | [[Project]( https://aoliao12138.github.io/FPO/ )] 267 | [[Video]( https://youtu.be/XZSuQQOY6Ls )] 268 | 269 | [[Gao et al. (ICCV '21)]( https://arxiv.org/pdf/2105.06468.pdf )] Dynamic View Synthesis from Dynamic Monocular Video 270 | [[Project]( https://free-view-video.github.io/ )] 271 | [[Code](https://github.com/gaochen315/DynamicNeRF )] 272 | [[Video]( https://youtu.be/j8CUzIR0f8M )] 273 | 274 | [[Li et al. (CVPR '21)]( https://arxiv.org/abs/2011.13084 )] Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes 275 | [[Project]( https://www.cs.cornell.edu/~zl548/NSFF// )] 276 | [[Code](https://github.com/zhengqili/Neural-Scene-Flow-Fields )] 277 | [[Video](https://www.cs.cornell.edu/~zl548/NSFF//overview.mp4 )] 278 | 279 | #### Editable 280 | [[Zhang et al. (ARXIV '22)]( https://arxiv.org/abs/2206.06360 )] ARF: Artistic Radiance Fields 281 | [[Project]( https://www.cs.cornell.edu/projects/arf/ )] 282 | [[Code & Data]( https://github.com/Kai-46/ARF-svox2 )] 283 | 284 | [[Kobayashi et al. (ARXIV '22)]( https://arxiv.org/pdf/2205.15585.pdf )] Decomposing NeRF for Editing via Feature Field Distillation 285 | [[Project]( https://pfnet-research.github.io/distilled-feature-fields/ )] 286 | 287 | [[Benaim et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.02776.pdf )] Volumetric Disentanglement for 3D Scene Manipulation 288 | [[Project]( https://sagiebenaim.github.io/volumetric-disentanglement/ )] 289 | 290 | [[Lazova et al. (**CVPR '22**)](https://arxiv.org/abs/2204.10850 )] Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation 291 | 292 | [[Yuan et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.04978.pdf )] NeRF-Editing: Geometry Editing of Neural Radiance Fields 293 | 294 | #### Generalizable 295 | [[Yu et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.00665.pdf )] MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction 296 | [[Project]( https://niujinshuchong.github.io/monosdf )] 297 | 298 | [[Rebain et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Rebain_LOLNerf_Learn_From_One_Look_CVPR_2022_paper.pdf )] LOLNeRF: Learn from One Look 299 | [[Project]( https://ubc-vision.github.io/lolnerf/ )] 300 | 301 | [[Chen et al. (ICCV '21)]( https://arxiv.org/abs/2103.15595 )] MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo 302 | [[Project]( https://apchenstu.github.io/mvsnerf/ )] 303 | [[Code]( https://github.com/apchenstu/mvsnerf )] 304 | [[Video]( https://youtu.be/3M3edNiaGsA )] 305 | 306 | [[Yu et al. (CVPR '21)]( https://arxiv.org/pdf/2012.02190.pdf )] Neural Radiance Fields from One or Few Images 307 | [[Project]( https://alexyu.net/pixelnerf/)] 308 | [[Code]( https://github.com/sxyu/pixel-nerf )] 309 | [[Video]( https://youtu.be/voebZx7f32g )] 310 | 311 | #### Large-scale 312 | [[Tancik et al. (ARXIV '22)](https://arxiv.org/abs/2202.05263)] Block-NeRF: Scalable Large Scene Neural View Synthesis 313 | [[Project]( https://waymo.com/research/block-nerf/)] 314 | [[Video]( https://youtu.be/6lGMCAzBzOQ )] 315 | 316 | [[Zhang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/papers/Zhang_NeRFusion_Fusing_Radiance_Fields_for_Large-Scale_Scene_Reconstruction_CVPR_2022_paper.pdf )] NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction 317 | [[Project]( https://jetd1.github.io/NeRFusion-Web/ )] 318 | 319 | #### Sparse input 320 | [[Long et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.05737.pdf)] SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse views 321 | [[Project]( https://www.xxlong.site/SparseNeuS/ )] 322 | 323 | [[Suhail et al. (**CVPR '22**)]( https://arxiv.org/pdf/2112.09687.pdf )] Light Field Neural Rendering 324 | [[Project]( https://light-field-neural-rendering.github.io/ )] 325 | [[Code]( https://github.com/google-research/google-research/tree/master/light_field_neural_rendering )] 326 | 327 | [[Niemeyer et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.00724 )] RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs 328 | [[Project]( https://m-niemeyer.github.io/regnerf )] 329 | [[Code]( https://github.com/google-research/google-research/tree/master/regnerf )] 330 | [[Video](https://www.youtube.com/watch?v=QyyyvA4-Kwc )] 331 | 332 | #### Datasets 333 | [[Downs et al. (ARXIV '22)]( https://arxiv.org/pdf/2204.11918.pdf )] Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Item 334 | [[Blog]( https://ai.googleblog.com/2022/06/scanned-objects-by-google-research.html )] 335 | [[Data]( https://app.gazebosim.org/GoogleResearch/fuel/collections/Scanned%20Objects%20by%20Google%20Research )] 336 | 337 | ### [5. Frontiers on 3D Point Cloud Learning](#content) 338 | 339 | [[Wiersma et al. (SIGGRAPH '22)]( https://rubenwiersma.nl/assets/pdf/DeltaConv.pdf )] DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds 340 | [[Project]( https://rubenwiersma.nl/deltaconv )] 341 | [[Code]( https://github.com/rubenwiersma/deltaconv )] 342 | [[Supp.]( https://rubenwiersma.nl/assets/pdf/DeltaConv_supplement.pdf )] 343 | 344 | [[Ran et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.05740.pdf )] Surface Representation for Point Clouds 345 | [[Code]( https://github.com/hancyran/RepSurf )] 346 | 347 | [[Mittal et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.09516.pdf )] AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation 348 | [[Project]( https://yccyenchicheng.github.io/AutoSDF/ )] 349 | [[Code]( https://github.com/yccyenchicheng/AutoSDF/ )] 350 | 351 | [[Chen et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Chen_The_Devil_Is_in_the_Pose_Ambiguity-Free_3D_Rotation-Invariant_Learning_CVPR_2022_paper.pdf)] The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant 352 | Learning via Pose-aware Convolution 353 | 354 | [[Jakab et al. (CVPR '21)]( https://arxiv.org/abs/2104.11224 )] KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control 355 | [[Project]( https://tomasjakab.github.io/KeypointDeformer/ )] 356 | [[Code]( https://github.com/tomasjakab/keypoint_deformer/ )] 357 | [[Video]( https://youtu.be/GdDX1ZFh1k0 )] 358 | 359 | 360 | ### [6. 3D Object Detection, Pose Estimation](#content) 361 | [[Yang et al. (**CVPR '22**)](https://openaccess.thecvf.com/content/CVPR2022/papers/Yang_ArtiBoost_Boosting_Articulated_3D_Hand-Object_Pose_Estimation_via_Online_Exploration_CVPR_2022_paper.pdf)] ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via 362 | Online Exploration and Synthesis 363 | 364 | [[Yin et al. (**CVPR '22**)]( https://arxiv.org/pdf/2203.15765.pdf )] FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering 365 | [[Project]( https://yd-yin.github.io/FisherMatch/ )] 366 | [[Code]( https://github.com/yd-yin/FisherMatch )] 367 | 368 | [[Sun et al. (**CVPR '22**)]( https://arxiv.org/pdf/2205.12257.pdf )] OnePose: One-Shot Object Pose Estimation without CAD Models 369 | [[Project]( https://zju3dv.github.io/onepose/ )] 370 | [[CodeSoon]( https://github.com/zju3dv/OnePose)] 371 | [[Supp]( https://zju3dv.github.io/onepose/files/onepose_supp.pdf )] 372 | 373 | [[Deng et al. (NeurIPS '21)]( https://arxiv.org/pdf/2112.07787.pdf )] Revisiting 3D Object Detection From an Egocentric Perspective 374 | 375 | [[Li et al. (NeurIPS '21)]( https://arxiv.org/abs/2111.00190 )] Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation 376 | [[Project]( https://dragonlong.github.io/equi-pose/ )] 377 | [[Code](https://github.com/dragonlong/equi-pose)] 378 | 379 | [[Lu et al. (ICCV '21)]( https://openaccess.thecvf.com/content/ICCV2021/papers/Lu_Geometry_Uncertainty_Projection_Network_for_Monocular_3D_Object_Detection_ICCV_2021_paper.pdf )] Geometry Uncertainty Projection Network for Monocular 3D Object Detection 380 | [[Code]( https://github.com/SuperMHP/GUPNet )] 381 | 382 | [[Ahmadyan et al. (CVPR '21)]( https://openaccess.thecvf.com/content/CVPR2021/papers/Ahmadyan_Objectron_A_Large_Scale_Dataset_of_Object-Centric_Videos_in_the_CVPR_2021_paper.pdf )] Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild With Pose Annotations 383 | [[Project]( https://github.com/google-research-datasets/Objectron/#tutorials )] 384 | [[Code]( https://github.com/google-research-datasets/Objectron )] 385 | 386 | [[Murphy et al. (ICML '21)]( https://arxiv.org/abs/2106.05965 )] Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold 387 | [[Project]( https://implicit-pdf.github.io/ )] 388 | [[Code]( https://github.com/google-research/google-research/tree/master/implicit_pdf )] 389 | [[Video]( https://youtu.be/Y-MlRRy0xJA )] 390 | [[Data](https://www.tensorflow.org/datasets/catalog/symmetric_solids)] 391 | 392 | 393 | ### [7. Neural Motion, Deformation Generation](#content) 394 | [[Kim et al. (ARXIV '22)]( https://arxiv.org/pdf/2202.04307.pdf )] Conditional Motion In-betweening 395 | [[Project]( https://jihoonerd.github.io/Conditional-Motion-In-Betweening/ )] 396 | 397 | [[He et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.03287.pdf )] NeMF: Neural Motion Fields for Kinematic Animation 398 | 399 | [[Ianina et al. (**CVPR '22**)]( https://nsarafianos.github.io/assets/bodymap/bodymap.pdf )] BodyMap: Learning Full-Body Dense Correspondence Map 400 | [[Project]( https://nsarafianos.github.io/bodymap )] 401 | [[Supp]( https://nsarafianos.github.io/assets/bodymap/bodymap_suppl.pdf )] 402 | 403 | [[Muralikrishnan et al. (**CVPR '22**)]( https://sanjeevmk.github.io/glass_webpage/resources/glass_fullRes.pdf )] GLASS: Geometric Latent Augmentation for Shape Spaces 404 | [[Project]( https://sanjeevmk.github.io/glass_webpage/ )] 405 | [[Code]( https://github.com/sanjeevmk/glass/ )] 406 | [[Video]( https://sanjeevmk.github.io/glass_webpage/video/glass_dist.mp4 )] 407 | 408 | [[Taheri et al. (**CVPR '22**)]( https://arxiv.org/abs/2112.11454 )] GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping 409 | [[Project]( https://goal.is.tue.mpg.de/ )] 410 | [[Video]( https://youtu.be/A7b8DYovDZY )] 411 | [[Code]( https://github.com/otaheri/GOAL )] 412 | 413 | [[AIGERMAN et al. (SIGGRAPH '22)]( https://arxiv.org/pdf/2205.02904.pdf )] Neural Jacobian Fields: Learning Intrinsic Mappings of Arbitrary　Meshes 414 | 415 | [[Raab et al. (Siggraph '22)]( https://arxiv.org/abs/2206.08010 )] MoDi: Unconditional Motion Synthesis from Diverse Data 416 | [[Project(need fix)]( https://sigal-raab.github.io/MoDi )] 417 | [[Video]( https://youtu.be/lRkdF8y3Du4)] 418 | 419 | [[Li et al. (SIGGRAPH '22)]( https://arxiv.org/abs/2205.02625 )] GANimator: Neural Motion Synthesis from a Single Sequence 420 | [[Project]( https://peizhuoli.github.io/ganimator/ )] 421 | [[Code](https://github.com/PeizhuoLi/ganimator )] 422 | [[Video]( https://youtu.be/OV9VoHMEeyI )] 423 | 424 | [[Wang et al. (NeurIPS '21)]( https://arxiv.org/abs/2106.11944 )] MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images 425 | [[Project]( https://neuralbodies.github.io/metavatar/ )] 426 | [[Code](https://github.com/taconite/MetaAvatar-release )] 427 | [[Video](https://youtu.be/AwOwdKxuBXE )] 428 | 429 | [[Henter et al. (Siggraph Asia '20)]( http://kth.diva-portal.org/smash/get/diva2:1471598/FULLTEXT01.pdf )] MoGlow: Probabilistic and controllable motion synthesis using normalising flows 430 | [[Project]( https://simonalexanderson.github.io/MoGlow/ )] 431 | [[Code](https://github.com/simonalexanderson/MoGlow )] 432 | [[Video](https://youtu.be/pe-YTvavbtA )] 433 | 434 | ### [8. 3D Representations Learning for Robotics](#content) 435 | [[Driess et al. (ARXIV '22)]( https://arxiv.org/pdf/2206.01634.pdf )] Reinforcement Learning with Neural Radiance Fields 436 | [[Project]( https://dannydriess.github.io/nerf-rl/ )] 437 | [[Video]( https://dannydriess.github.io/nerf-rl/video.mp4 )] 438 | 439 | [[Gao et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/html/Gao_ObjectFolder_2.0_A_Multisensory_Object_Dataset_for_Sim2Real_Transfer_CVPR_2022_paper.html )] ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer 440 | [[Project]( https://ai.stanford.edu/~rhgao/objectfolder2.0/ )] 441 | [[Code]( https://github.com/rhgao/ObjectFolder )] 442 | [[Video]( https://youtu.be/e5aToT3LkRA )] 443 | 444 | [[Ortiz et al. (RSS '22)]( https://arxiv.org/abs/2204.02296 )] iSDF: Real-Time Neural Signed Distance Fields for Robot Perception 445 | [[Project]( https://joeaortiz.github.io/iSDF/ )] 446 | [[Code]( https://github.com/facebookresearch/iSDF )] 447 | [[Video]( https://youtu.be/mAKGl1wBSic )] 448 | 449 | [[Wi et al. (ICRA '22)]( https://arxiv.org/abs/2202.00868)] VIRDO: Visio-tactile Implicit Representations of Deformable Objects 450 | [[Project]( https://www.mmintlab.com/research/virdo-visio-tactile-implicit-representations-of-deformable-objects/ )] 451 | [[Code]( https://github.com/MMintLab/VIRDO )] 452 | 453 | [[Adamkiewicz et al. (ICRA '22)]( https://arxiv.org/pdf/2110.00168.pdf )] Vision-Only Robot Navigation in a Neural Radiance World 454 | [[Project]( https://mikh3x4.github.io/nerf-navigation/ )] 455 | [[Code]( https://github.com/mikh3x4/nerf-navigation )] 456 | [[Video]( https://youtu.be/5JjWpv9BaaE )] 457 | [[Data](https://drive.google.com/drive/folders/10_DWHIIetzeM2-1ziyZHujycWh-fNs29?usp=sharing)] 458 | 459 | [[Li et al. (CoRL '21)]( https://arxiv.org/abs/2107.04004 )] 3D Neural Scene Representations for Visuomotor Control 460 | [[Project]( https://3d-representation-learning.github.io/nerf-dy/)] 461 | [[Video]( https://youtu.be/ELPMiifELGc )] 462 | 463 | [[Ichnowski et al. (CoRL '21)]( https://arxiv.org/pdf/2110.14217.pdf )] Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects 464 | [[Project]( https://sites.google.com/view/dex-nerf )] 465 | [[Dataset]( https://github.com/BerkeleyAutomation/dex-nerf-datasets )] 466 | [[Video]( https://youtu.be/F9R6Nf1d7P4 )] 467 | 468 | 469 | ### [9. Prompt Learning to 3D](#content) 470 | [[Tevet et al. (ARXIV '22)]( https://guytevet.github.io/motionclip-page/static/source/MotionCLIP.pdf )] MotionCLIP: Exposing Human Motion Generation to CLIP Space 471 | [[Project]( https://guytevet.github.io/motionclip-page/ )] 472 | [[Code]( https://github.com/GuyTevet/MotionCLIP )] 473 | 474 | [[Wang et al. (**CVPR '22**)]( https://openaccess.thecvf.com/content/CVPR2022/html/Wang_CLIP-NeRF_Text-and-Image_Driven_Manipulation_of_Neural_Radiance_Fields_CVPR_2022_paper.html )] CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields 475 | [[Project]( https://cassiepython.github.io/clipnerf/ )] 476 | [[Code]( https://github.com/cassiePython/CLIPNeRF )] 477 | [[Video]( https://cassiepython.github.io/clipnerf/images/video.mp4 )] 478 | 479 | [[Michel et al. (ARXIV '21)]( https://arxiv.org/abs/2112.03221 )] Text2Mesh: Text-Driven Neural Stylization for Meshes 480 | [[Project]( https://threedle.github.io/text2mesh/ )] 481 | [[Code]( https://github.com/threedle/text2mesh )] 482 | 483 | #### Volume Rendering 484 | [[Sawhey et al. (SIGGRAPH '22)]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree-small.pdf )] Grid-free Monte Carlo for PDEs with spatially varying coefficients 485 | [[Project]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree.html )] 486 | [[Code]( https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22gridfree-reference-implementation.zip )] 487 | 488 | ### More resources 489 | - [Amesome 3D machine learning collection](https://github.com/timzhang642/3D-Machine-Learning) 490 | 491 | - [NeRF Fields in Visual Computing](https://neuralfields.cs.brown.edu/index.html) 492 | 493 | - [Awesome-point-cloud-registration](https://github.com/wsunid/awesome-point-clouds-registration) 494 | 495 | - [Awesome-equivariant-network](https://github.com/Chen-Cai-OSU/awesome-equivariant-network/blob/main/README.md#content) 496 | --------------------------------------------------------------------------------