├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Chen Wang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Awesome 4D Generation 2 | This repo collects papers for 4D generation. 3 | 4 | ## Table of Contents 5 | - [Camera Control for Video Diffusion](#camera-control-for-video-diffusion) 6 | - [Multi-view for Video Diffusion](#multi-view-for-video-diffusion) 7 | - [Distillation from Video Diffusion](#distillation-from-video-diffusion) 8 | - [Generation by Reconstruction](#generation-by-reconstruction) 9 | - [4D Editing](#4d-editing) 10 | - [Physics](#physics) 11 | 12 | ## Camera Control for Video Diffusion 13 | 14 | VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control 15 | 16 | [📄 Paper](https://arxiv.org/abs/2407.12781) | [🌐 Project Page](https://snap-research.github.io/vd3d/) 17 | 18 | Controlling Space and Time with Diffusion Models 19 | 20 | [📄 Paper](https://arxiv.org/pdf/2407.07860) | [🌐 Project Page](https://4d-diffusion.github.io/) 21 | 22 | CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation 23 | 24 | [📄 Paper](https://arxiv.org/abs/2406.02509) | [🌐 Project Page](https://ir1d.github.io/CamCo/) 25 | 26 | Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control 27 | 28 | [📄 Paper](https://arxiv.org/pdf/2405.17414) | [🌐 Project Page](https://collaborativevideodiffusion.github.io/) 29 | 30 | ## Multi-view for Video Diffusion 31 | 32 | SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency, Xie et al., Arxiv 2024 33 | 34 | [📄 Paper](https://arxiv.org/pdf/2407.17470) | [🌐 Project Page](https://sv4d.github.io/) | [💻 Code](https://github.com/Stability-AI/generative-models) 35 | 36 | L4GM: Large 4D Gaussian Reconstruction Model, Ren et al., Arxiv 2024 37 | 38 | [📄 Paper](https://arxiv.org/abs/2406.10324) | [🌐 Project Page](https://research.nvidia.com/labs/toronto-ai/l4gm/) 39 | 40 | 4Diffusion: Multi-view Video Diffusion Model for 4D Generation, Zhang et al., Arxiv 2024 41 | 42 | [📄 Paper](https://arxiv.org/pdf/2405.20674) | [🌐 Project Page](https://aejion.github.io/4diffusion) | [💻 Code](https://github.com/aejion/4Diffusion) 43 | 44 | Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models, Liang et al., Arxiv 2024 45 | 46 | [📄 Paper](https://arxiv.org/abs/2405.16645) | [🌐 Project Page](https://vita-group.github.io/Diffusion4D/) | [💻 Code](https://github.com/VITA-Group/Diffusion4D) | [🎥 Video](https://www.youtube.com/watch?v=XJT-cMt_xVo) 47 | 48 | ## Distillation from Video Diffusion 49 | Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis, Zeng et al., Arxiv 2024 50 | 51 | [📄 Paper](https://arxiv.org/abs/2410.07155) | [💻 Code](https://github.com/YangLing0818/Trans4D) 52 | 53 | CT4D: Consistent Text-to-4D Generation with Animatable Meshes, Ce et al., Arxiv 2024 54 | 55 | [📄 Paper](https://arxiv.org/pdf/2408.08342) 56 | 57 | Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion, Deng et al., SIGGRAPH 2024 58 | 59 | [📄 Paper](https://arxiv.org/pdf/2407.13759) | [🌐 Project Page](https://boyangdeng.com/streetscapes/) | [🎥 Video](https://www.youtube.com/watch?v=13hLTnrVVKk) 60 | 61 | 4Dynamic: Text-to-4D Generation with Hybrid Priors, Yuan et al., Arxiv 2024 62 | 63 | [📄 Paper](https://arxiv.org/abs/2407.12684) 64 | 65 | Animate3D: Animating Any 3D Model with Multi-view Video Diffusion, Jiang et al., Arxiv 2024 66 | 67 | [📄 Paper](https://arxiv.org/pdf/2407.11398) | [🌐 Project Page](https://animate3d.github.io/) 68 | 69 | STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting, Chai et al., Arxiv 2024 70 | 71 | [📄 Paper](https://arxiv.org/abs/2406.04629) | [🌐 Project Page](https://star-avatar.github.io/) | [💻 Code](https://github.com/czh-98/STAR) 72 | 73 | MotionDreamer: Zero-Shot 3D Mesh Animation from Video Diffusion Models, Uzolas et al., Arxiv 2024 74 | 75 | [📄 Paper](https://arxiv.org/pdf/2405.20155) | [💻 Code](https://github.com/lukasuz/MotionDreamer) 76 | 77 | PLA4D: Pixel-Level Alignments for Text-to-4D Gaussian Splatting, Miao et al., Arxiv 2024 78 | 79 | [📄 Paper](https://arxiv.org/pdf/2405.19957) | [🌐 Project Page](https://github.com/MiaoQiaowei/PLA4D.github.io) 80 | 81 | MagicPose4D: Crafting Articulated Models with Appearance and Motion Control, Zhang et al., Arxiv 2024 82 | 83 | [📄 Paper](https://arxiv.org/pdf/2405.14017) | [🌐 Project Page](https://boese0601.github.io/magicpose4d/) | [💻 Code](https://github.com/haoz19/MagicPose4D) 84 | 85 | SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer, Wu et al., Arxiv 2024 86 | 87 | [📄 Paper](https://arxiv.org/abs/2404.03736) | [🌐 Project Page](https://sc4d.github.io/) | [💻 Code](https://github.com/JarrentWu1031/SC4D) | [🎥 Video](https://www.youtube.com/watch?v=SkpTEuX4B5c) 88 | 89 | TC4D: Trajectory-Conditioned Text-to-4D Generation, Bahmani et al., Arxiv 2024 90 | 91 | [📄 Paper](https://arxiv.org/pdf/2403.17920) | [🌐 Project Page](https://sherwinbahmani.github.io/tc4d) | [💻 Code](https://github.com/sherwinbahmani/tc4d) 92 | 93 | Comp4D: LLM-Guided Compositional 4D Scene Generation, Xu et al., Arxiv 2024 94 | 95 | [📄 Paper](https://arxiv.org/abs/2403.16993) | [🌐 Project Page](https://vita-group.github.io/Comp4D/) | [💻 Code](https://github.com/VITA-Group/Comp4D) | [🎥 Video](https://www.youtube.com/watch?v=9q8SV1Xf_Xw) 96 | 97 | STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians, Zetn et al., Arxiv 2024 98 | 99 | [📄 Paper](https://arxiv.org/pdf/2403.14939.pdf) | [🌐 Project Page](https://nju-3dv.github.io/projects/STAG4D/) | [💻 Code](https://github.com/zeng-yifei/STAG4D) 100 | 101 | GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation, Gao et al., Arxiv 2024 102 | 103 | [📄 Paper](https://arxiv.org/abs/2403.12365) | [🌐 Project Page](https://zerg-overmind.github.io/GaussianFlow.github.io/) | [💻 Code](https://github.com/Zerg-Overmind/GaussianFlow) 104 | 105 | 4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency, Yin et al., Arxiv 2023 106 | 107 | [📄 Paper](https://arxiv.org/pdf/2312.17225) | [🌐 Project Page](https://vita-group.github.io/4DGen/) | [💻 Code](https://github.com/VITA-Group/4DGen) | [🎥 Video](https://www.youtube.com/watch?v=-bXyBKdpQ1o) 108 | 109 | DreamGaussian4D: Generative 4D Gaussian Splatting, Ren et al., CVPR 2024 110 | 111 | [📄 Paper](https://arxiv.org/pdf/2312.13763) | [🌐 Project Page](https://jiawei-ren.github.io/projects/dreamgaussian4d/) 112 | 113 | Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models, Ling et al., Arxiv 2023 114 | 115 | [📄 Paper](https://arxiv.org/pdf/2312.03795) | [🌐 Project Page](https://animatabledreamer.github.io/) | [💻 Code](https://github.com/AnimatableDreamer/AnimatableDreamer) 116 | 117 | AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation, Wang et al., Arxiv 2023 118 | 119 | [📄 Paper](https://arxiv.org/pdf/2312.03795) | [🌐 Project Page](https://animatabledreamer.github.io/) | [💻 Code](https://github.com/AnimatableDreamer/AnimatableDreamer) 120 | 121 | 4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling, Bahmani et al., CVPR 2024 122 | 123 | [📄 Paper](https://arxiv.org/pdf/2311.17984) | [🌐 Project Page](https://research.nvidia.com/labs/nxp/dream-in-4d/) | [💻 Code](https://github.com/sherwinbahmani/4dfy) 124 | 125 | A Unified Approach for Text- and Image-guided 4D Scene Generation, Zheng et al., CVPR 2024 126 | 127 | [📄 Paper](https://arxiv.org/pdf/2311.17984) | [🌐 Project Page](https://sherwinbahmani.github.io/4dfy) | [💻 Code](https://github.com/NVlabs/dream-in-4d) 128 | 129 | Animate124: Animating One Image to 4D Dynamic Scene, Zhao et al., Arxiv 2023 130 | 131 | [📄 Paper](https://arxiv.org/pdf/2311.14603) | [🌐 Project Page](https://animate124.github.io/) | [💻 Code](https://github.com/HeliosZhao/Animate124) 132 | 133 | Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video, Jiang et al., Arxiv 2024 134 | 135 | [📄 Paper](https://arxiv.org/pdf/2311.02848) | [🌐 Project Page](https://consistent4d.github.io/) | [💻 Code](https://github.com/yanqinJiang/Consistent4D) 136 | 137 | Text-To-4D Dynamic Scene Generation, Singer et al., Arxiv 2023 138 | 139 | [📄 Paper](https://arxiv.org/pdf/2301.11280) | [🌐 Project Page](https://make-a-video3d.github.io) 140 | 141 | ## Generation by Reconstruction 142 | 143 | 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models, Yu et al., Arxiv 2024 144 | 145 | [📄 Paper](https://arxiv.org/abs/2406.07472) | [🌐 Project Page](https://snap-research.github.io/4Real/) 146 | 147 | Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels, Wang et al., Arxiv 2024 148 | 149 | [📄 Paper](https://arxiv.org/abs/2405.16822) | [🌐 Project Page](https://vidu4d-dgs.github.io/) 150 | 151 | ## 4D Editing 152 | 153 | Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D Diffusion, Mou et al., CVPR 2024 154 | 155 | [📄 Paper](https://arxiv.org/abs/2406.09402) | [🌐 Project Page](https://immortalco.github.io/Instruct-4D-to-4D/) 156 | 157 | ## Physics 158 | 159 | Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generation, Fu et al., Arxiv 2024 160 | 161 | [📄 Paper](https://arxiv.org/abs/2405.16849) | [🌐 Project Page](https://sync4dphys.github.io/) 162 | --------------------------------------------------------------------------------