├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── index.html ├── jemdoc.css ├── reference.bib └── save_cited_bib.py /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # CONTRIBUTING 2 | 3 | If you think I have missed out on something (or) have any suggestions (papers, implementations and other resources), feel free to [pull a request](https://github.com/weihaox/awesome-3D-aware-synthesis/pulls). 4 | 5 | All kinds of contributions are welcome, including but not limited to: 6 | 7 | - fixing typo or bugs 8 | - adding new categories and methods 9 | 10 | 11 | ## Edit README.md 12 | 13 | markdown format: 14 | ``` markdown 15 | * **Here is the Paper Name.**
16 | *[Author 1](homepage), Author 2, and Author 3.*
17 | Conference or Journal Year. [[PDF](link)] [[Project](link)] [[Github](link)] [[Video](link)] [[Data](link)] 18 | ``` 19 | 20 | ## Convert to HTML 21 | 22 | Use the following command to convert markdown to html: 23 | 24 | ```shell 25 | pandoc -f markdown -t html README.md -o index.html 26 | ``` 27 | Installation instructions can be found [here](https://pandoc.org/installing.html). 28 | 29 | Project page will be employed at [this page](https://weihaox.github.io/awesome-3D-aware-synthesis/). 30 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 weihao 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 12 | 13 |
14 | 15 |

16 |

A Survey on Deep Generative 3D-aware Image Synthesis

17 |

18 | ACM Computing Surveys, 2023 19 |
20 | Weihao Xia 21 | · 22 | Jing-Hao Xue 23 |

24 | 25 |

26 | 27 | arxiv Paper 28 | 29 | 30 | Project Page 31 | 32 | 33 | CSUR Paper 34 | 35 |

36 |

37 | 38 | 39 | Introduction 40 | ------------ 41 | 42 | This project lists representative papers/codes/datasets about deep **[3D-aware image synthesis](https://weihaox.github.io/3D-aware-Gen)**. Besides **3D-aware Generative Models** (GANs and Diffusion Models) discussed in this [survey](https://arxiv.org/abs/2210.14267), this project additionally covers novel view synthesis studies, especially those based on [implicit neural representations](https://github.com/weihaox/awesome-neural-rendering#implicit-neural-representation-and-rendering) such as NeRF. 43 | 44 | We aim to constantly update the latest relevant papers and help the community track this topic. Please feel free to join us and [contribute](https://github.com/weihaox/3D-aware-Gen/blob/main/CONTRIBUTING.md) to the project. Please do not hesitate to reach out if you have any questions or suggestions. 45 | 46 | Survey paper 47 | ------------ 48 | 49 | * [A Survey on Deep Generative 3D-aware Image Synthesis](https://arxiv.org/abs/2210.14267) 50 | Weihao Xia and Jing-Hao Xue. _ACM Computing Surveys_, 2023. 51 | 52 | ## 3D Control of 2D GANs 53 | 54 | ### 3D Control Latent Directions 55 | 56 | For 3D control over diffusion models simiar to [GAN](https://github.com/weihaox/GAN-Inversion#gan-latent-space-editing), please refer to [diffusion latent editing](https://github.com/weihaox/GAN-Inversion#semantic-editing-in-diffusion-latent-spaces). 57 | 58 | * **SeFa: Closed-Form Factorization of Latent Semantics in GANs.**
59 | *Yujun Shen, Bolei Zhou.*
60 | CVPR 2021. [[Paper](https://arxiv.org/abs/2007.06600)] [[Project](https://genforce.github.io/sefa/)] [[Code](https://github.com/genforce/sefa)] 61 | 62 | * **GANSpace: Discovering Interpretable GAN Controls.**
63 | *Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris.*
64 | NeurIPS 2020. [[Paper](https://arxiv.org/abs/2004.02546)] [[Code](https://github.com/harskish/ganspace)] 65 | 66 | * **Interpreting the Latent Space of GANs for Semantic Face Editing.**
67 | *[Yujun Shen](http://shenyujun.github.io/), [Jinjin Gu](http://www.jasongt.com/), [Xiaoou Tang](http://www.ie.cuhk.edu.hk/people/xotang.shtml), [Bolei Zhou](http://bzhou.ie.cuhk.edu.hk/).*
68 | CVPR 2020. [[Paper](https://arxiv.org/abs/1907.10786)] [[Project](https://genforce.github.io/interfacegan/)] [[Code](https://github.com/genforce/interfacegan)] 69 | 70 | * **Unsupervised Discovery of Interpretable Directions in the GAN Latent Space.**
71 | *Andrey Voynov, Artem Babenko.*
72 | ICML 2020. [[Paper](https://arxiv.org/abs/2002.03754)] [[Code](https://github.com/anvoynov/GANLatentDiscovery)] 73 | 74 | * **On the "steerability" of generative adversarial networks.**
75 | *Ali Jahanian, Lucy Chai, Phillip Isola.*
76 | ICLR 2020. [[Paper](https://arxiv.org/abs/1907.07171)] [[Project](https://ali-design.github.io/gan_steerability/)] [[Code](https://github.com/ali-design/gan_steerability)] 77 | 78 | ### 3D Parameters as Controls 79 | 80 | * **3D-FM GAN: Towards 3D-Controllable Face Manipulation.**
81 | *[Yuchen Liu](https://lychenyoko.github.io/), Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, and Sun-Yuan Kung.*
82 | ECCV 2022. [[Paper](https://arxiv.org/abs/2208.11257)] [[Project](https://lychenyoko.github.io/3D-FM-GAN-Webpage/)] 83 | 84 | * **GAN-Control: Explicitly Controllable GANs.**
85 | *Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni.*
86 | ICCV 2021. [[Paper](https://arxiv.org/abs/2101.02477)] [[Project](https://alonshoshan10.github.io/gan_control/)] [[Code](https://github.com/amazon-science/gan-control)] 87 | 88 | * **CONFIG: Controllable Neural Face Image Generation.**
89 | *Marek Kowalski, Stephan J. Garbin, Virginia Estellers, Tadas Baltrušaitis, Matthew Johnson, Jamie Shotton.*
90 | ECCV 2020. [[Paper](https://arxiv.org/abs/2005.02671)] [[Code](https://github.com/microsoft/ConfigNet)] 91 | 92 | * **DiscoFaceGAN: Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning.**
93 | *Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, Xin Tong.*
94 | CVPR 2020. [[Paper](https://arxiv.org/Paper/2004.11660.Paper)] [[Code](https://github.com/microsoft/DiscoFaceGAN)] 95 | 96 | * **StyleRig: Rigging StyleGAN for 3D Control over Portrait Images.**
97 | *Ayush Tewari, Mohamed Elgharib, Gaurav Bharaj, Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt.*
98 | CVPR 2020 (oral). [[Paper](https://arxiv.org/abs/2004.00121)] [[Project](https://gvv.mpi-inf.mpg.de/projects/StyleRig/)] 99 | 100 | * **PIE: Portrait Image Embedding for Semantic Control.**
101 | *[Ayush Tewari](http://people.mpi-inf.mpg.de/~atewari/), Mohamed Elgharib, Mallikarjun B R., Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt.*
102 | TOG (SIGGRAPH Asia) 2020. [[Paper](http://gvv.mpi-inf.mpg.de/projects/PIE/data/paper.Paper)] [[Project](http://gvv.mpi-inf.mpg.de/projects/PIE/)] 103 | 104 | ### 3D Prior Knowledge as Constraints 105 | 106 | * **3D-Aware Indoor Scene Synthesis with Depth Priors.**
107 | *Zifan Shi, Yujun Shen, Jiapeng Zhu, Dit-Yan Yeung, Qifeng Chen.*
108 | ECCV 2022 (oral). [[Paper](https://arxiv.org/abs/2202.08553)] [[Project](https://vivianszf.github.io/depthgan/)] [[Code](https://github.com/vivianszf/depthgan)] 109 | 110 | * **NGP: Towards a Neural Graphics Pipeline for Controllable Image Generation.**
111 | *Xuelin Chen, Daniel Cohen-Or, Baoquan Chen, Niloy J. Mitra.*
112 | Eurographics 2021. [[Paper](https://arxiv.org/abs/2006.10569)] [[Code](http://geometry.cs.ucl.ac.uk/projects/2021/ngp)] 113 | 114 | * **Lifting 2D StyleGAN for 3D-Aware Face Generation.**
115 | *[Yichun Shi](https://seasonsh.github.io/), Divyansh Aggarwal, [Anil K. Jain](http://www.cse.msu.edu/~jain/).*
116 | CVPR 2021. [[Paper](https://arxiv.org/abs/2011.13126)] [[Code](https://github.com/seasonSH/LiftedGAN)] 117 | 118 | * **RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis.**
119 | *Atsuhiro Noguchi, Tatsuya Harada.*
120 | ICLR 2020. [[Paper](https://arxiv.org/abs/1909.12573)] [[Code](https://github.com/nogu-atsu/RGBD-GAN)] 121 | 122 | * **Visual Object Networks: Image Generation with Disentangled 3D Representation.**
123 | *Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman.*
124 | NeurIPS 2018. [[Paper](https://arxiv.org/abs/1812.02725)] [[Project](http://von.csail.mit.edu/)] [[Code](https://github.com/junyanz/VON)] 125 | 126 | * **3D Shape Induction from 2D Views of Multiple Objects.**
127 | *Matheus Gadelha, Subhransu Maji, Rui Wang.*
128 | 3DV 2017. [[Paper](https://arxiv.org/abs/1612.05872)] [[Project](http://mgadelha.me/prgan/index.html)] [[Code](https://github.com/matheusgadelha/PrGAN)] 129 | 130 | * **Generative Image Modeling using Style and Structure Adversarial Networks.**
131 | *Xiaolong Wang, Abhinav Gupta.*
132 | ECCV 2016. [[Paper](https://arxiv.org/abs/1603.05631)] [[Project](https://github.com/facebook/eyescream)] [[Code](https://github.com/xiaolonw/ss-gan)] 133 | 134 | ## 3D-aware GANs for a Single Image Category 135 | 136 | ### Unconditional 3D Generative Models 137 | 138 | * **BallGAN: 3D-aware Image Synthesis with a Spherical Background.**
139 | *Minjung Shin, Yunji Seo, Jeongmin Bae, Young Sun Choi, Hyunsu Kim, Hyeran Byun, Youngjung Uh.*
140 | ICCV 2023. [[Paper](https://arxiv.org/abs/2301.09091)] [[Project](https://minjung-s.github.io/ballgan/)] [[Code](https://github.com/minjung-s/BallGAN)] 141 | 142 | * **Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitation.**
143 | *Xingyu Chen, Yu Deng, Baoyuan Wang.*
144 | ICCV 2023. [[Paper](https://arxiv.org/abs/2303.09036)] [[Project](https://seanchenxy.github.io/Mimic3DWeb/)] 145 | 146 | * **GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifolds.**
147 | *Jianfeng Xiang, Jiaolong Yang, Yu Deng, Xin Tong.*
148 | ICCV 2023. [[Paper](https://arxiv.org/abs/2206.07255)] [[Project](https://jeffreyxiang.github.io/GRAM-HD/)] 149 | 150 | * **Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis.**
151 | *Alex Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano.*
152 | TOG (SIGGRAPH) 2023. [[Paper](https://research.nvidia.com/labs/nxp/lp3d//media/paper.Paper)] [[Project](https://research.nvidia.com/labs/nxp/lp3d//)] 153 | 154 | * **VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids.**
155 | *Katja Schwarz, Axel Sauer, Michael Niemeyer, Yiyi Liao, Andreas Geiger.*
156 | NeurIPS 2022. [[Paper](https://arxiv.org/Paper/2206.07695.Paper)] [[Code](https://github.com/autonomousvision/voxgraf)] 157 | 158 | * **GeoD: Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator.**
159 | *Zifan Shi, Yinghao Xu, Yujun Shen, Deli Zhao, Qifeng Chen, Dit-Yan Yeung.*
160 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2209.15637)] [[Project](https://vivianszf.github.io/geod)] 161 | 162 | * **EpiGRAF: Rethinking training of 3D GANs.**
163 | *[Ivan Skorokhodov](https://universome.github.io/), [Sergey Tulyakov](http://www.stulyakov.com/), [Yiqun Wang](https://sites.google.com/view/yiqun-wang/home), [Peter Wonka](https://peterwonka.net/).*
164 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2206.10535)] [[Project](https://universome.github.io/epigraf)] [[Code](https://github.com/universome/epigraf)] 165 | 166 | * **VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids.**
167 | *Schwarz, Katja, Sauer, Axel, Niemeyer, Michael, Liao, Yiyi, and Geiger, Andreas.*
168 | NeurIPS 2022. [[Paper](https://arxiv.org/Paper/2206.07695.Paper)] [[Project](https://katjaschwarz.github.io/voxgraf)] 169 | 170 | * **Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis.**
171 | *Jeong-gi Kwak, Yuanming Li, Dongsik Yoon, Donghyeon Kim, David Han, Hanseok Ko.*
172 | ECCV 2022. [[Paper](https://arxiv.org/abs/2207.10257)] [[Project](https://jgkwak95.github.io/surfgan/)] [[Code](https://github.com/jgkwak95/SURF-GAN)] 173 | 174 | * **Generative Multiplane Images: Making a 2D GAN 3D-Aware.**
175 | *[Xiaoming Zhao](https://xiaoming-zhao.com/), [Fangchang Ma](https://fangchangma.github.io/), [David Güera](https://scholar.google.com/citations?user=bckYvFkAAAAJ&hl=en), [Zhile Ren](https://jrenzhile.com/), [Alexander G. Schwing](https://www.alexander-schwing.de/), [Alex Colburn](https://www.colburn.org/).*
176 | ECCV 2022. [[Paper](https://arxiv.org/abs/2207.10642)] [[Project](https://xiaoming-zhao.github.io/projects/gmpi/)] [[Code](https://github.com/apple/ml-gmpi)] 177 | 178 | * **3D-FM GAN: Towards 3D-Controllable Face Manipulation.**
179 | *[Yuchen Liu](https://lychenyoko.github.io/), Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, and Sun-Yuan Kung.*
180 | ECCV 2022. [[Paper](https://arxiv.org/abs/2208.11257)] [[Project](https://lychenyoko.github.io/3D-FM-GAN-Webpage/)] 181 | 182 | * **EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks.**
183 | *[Eric R. Chan](https://ericryanchan.github.io/), [Connor Z. Lin](https://connorzlin.com/), [Matthew A. Chan](https://matthew-a-chan.github.io/), [Koki Nagano](https://luminohope.org/), [Boxiao Pan](https://cs.stanford.edu/~bxpan/), [Shalini De Mello](https://research.nvidia.com/person/shalini-gupta), [Orazio Gallo](https://oraziogallo.github.io/), [Leonidas Guibas](https://geometry.stanford.edu/member/guibas/), [Jonathan Tremblay](https://research.nvidia.com/person/jonathan-tremblay), [Sameh Khamis](https://www.samehkhamis.com/), [Tero Karras](https://research.nvidia.com/person/tero-karras), [Gordon Wetzstein](https://stanford.edu/~gordonwz/).*
184 | CVPR 2022. [[Paper](https://arxiv.org/abs/2112.07945)] [[Project](https://matthew-a-chan.github.io/EG3D)] [[Code](https://github.com/NVlabs/eg3d)] 185 | 186 | * **StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning.**
187 | *Yi-Hua Huang, Yue He, Yu-Jie Yuan, Yu-Kun Lai, Lin Gao.*
188 | CVPR 2022. [[Paper](https://arxiv.org/abs/2205.12183)] 189 | 190 | * **Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis.**
191 | *Xuanmeng Zhang, Zhedong Zheng, Daiheng Gao, Bang Zhang, Pan Pan, Yi Yang.*
192 | CVPR 2022. [[Paper](https://arxiv.org/abs/2204.06307)] [[Code](https://github.com/Xuanmeng-Zhang/MVCGAN)] 193 | 194 | * **Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images.**
195 | *[Ayush Tewari](https://ayushtewari.com/), Mallikarjun B R, Xingang Pan, Ohad Fried, Maneesh Agrawala, Christian Theobalt.*
196 | CVPR 2022. [[Paper](https://people.mpi-inf.mpg.de/~atewari/projects/D3D/data/paper.Paper)] [[Project](https://people.mpi-inf.mpg.de/~atewari/projects/D3D/)] 197 | 198 | * **GIRAFFE HD: A High-Resolution 3D-aware Generative Model.**
199 | *Yang Xue, Yuheng Li, Krishna Kumar Singh, Yong Jae Lee.*
200 | CVPR 2022. [[Paper](https://arxiv.org/abs/2203.14954)] [[Code](https://github.com/AustinXY/GIRAFFEHD)] 201 | 202 | * **StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation.**
203 | *[Roy Or-El](https://homes.cs.washington.edu/~royorel/), [Xuan Luo](https://roxanneluo.github.io/), Mengyi Shan, Eli Shechtman, Jeong Joon Park, Ira Kemelmacher-Shlizerman.*
204 | CVPR 2022. [[Paper](https://arxiv.org/abs/2112.11427)] [[Project](https://stylesdf.github.io/)] [[Code](https://github.com/royorel/StyleSDF)] 205 | 206 | * **FENeRF: Face Editing in Neural Radiance Fields.**
207 | *Jingxiang Sun, Xuan Wang, Yong Zhang, Xiaoyu Li, Qi Zhang, Yebin Liu, Jue Wang.*
208 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.15490)] [[Code](https://github.com/MrTornado24/FENeRF)] 209 | 210 | * **LOLNeRF: Learn from One Look.**
211 | *[Daniel Rebain](https://vision.cs.ubc.ca/team/), Mark Matthews, Kwang Moo Yi, Dmitry Lagun, Andrea Tagliasacchi.*
212 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.09996)] [[Project](https://ubc-vision.github.io/lolnerf/)] 213 | 214 | * **GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation.**
215 | *[Yu Deng](https://yudeng.github.io/), [Jiaolong Yang](https://jlyang.org/), [Jianfeng Xiang](http://www.xtong.info/), [Xin Tong]().*
216 | CVPR 2022. [[Paper](https://arxiv.org/abs/2112.08867)] [[Project](https://yudeng.github.io/GRAM/)] [[Code](https://yudeng.github.io/GRAM/)] 217 | 218 | * **VolumeGAN: 3D-aware Image Synthesis via Learning Structural and Textural Representations.**
219 | *Yinghao Xu, Sida Peng, Ceyuan Yang, Yujun Shen, Bolei Zhou.*
220 | CVPR 2022. [[Paper](https://arxiv.org/abs/2112.10759)] [[Project](https://genforce.github.io/volumegan/)] [[Code](https://github.com/genforce/VolumeGAN)] 221 | 222 | * **Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks.**
223 | *Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung-Woo Ha, Jinwoo Shin.*
224 | ICLR 2022. [[Paper](https://openreview.net/forum?id=Czsdv-S4-w9)] [[Project](https://sihyun-yu.github.io/digan/)] [[Code](https://github.com/sihyun-yu/digan)] 225 | 226 | * **StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis.**
227 | *[Jiatao Gu](http://jiataogu.me/), [Lingjie Liu](https://lingjie0206.github.io/), [Peng Wang](https://totoro97.github.io/about.html), [Christian Theobalt](http://people.mpi-inf.mpg.de/~theobalt/).*
228 | ICLR 2022. [[Paper](https://arxiv.org/abs/2110.08985)] [[Project](http://jiataogu.me/style_nerf/)] 229 | 230 | * **MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation.**
231 | *Safa C. Medin, Bernhard Egger, Anoop Cherian, Ye Wang, Joshua B. Tenenbaum, Xiaoming Liu, Tim K. Marks.*
232 | AAAI 2022. [[Paper](https://arxiv.org/abs/2111.01048)] 233 | 234 | * **A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis.**
235 | *Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai.*
236 | NeurIPS 2021. [[Paper](https://arxiv.org/abs/2110.15678)] 237 | 238 | * **pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis.**
239 | *[Eric R. Chan](https://ericryanchan.github.io/), [Marco Monteiro](https://marcoamonteiro.github.io/pi-GAN-website/), [Petr Kellnhofer](https://kellnhofer.xyz/), [Jiajun Wu](https://jiajunwu.com/), [Gordon Wetzstein](https://stanford.edu/~gordonwz/).*
240 | CVPR 2021. [[Paper](https://arxiv.org/abs/2012.00926)] [[Project](https://marcoamonteiro.github.io/pi-GAN-website/)] [[Code](https://github.com/lucidrains/pi-GAN-pytorch)] 241 | 242 | * **GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields.**
243 | *Michael Niemeyer, Andreas Geiger.*
244 | CVPR 2021 (Best Paper). [[Paper](https://arxiv.org/abs/2011.12100)] [[Project](https://m-niemeyer.github.io/project-pages/giraffe/index.html)] [[Code](https://github.com/autonomousvision/giraffe)] 245 | 246 | * **BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images.**
247 | *Thu Nguyen-Phuoc, Christian Richardt, Long Mai, Yong-Liang Yang, Niloy Mitra.*
248 | NeurIPS 2020. [[Paper](https://arxiv.org/abs/2002.08988)] [[Project](https://www.monkeyoverflow.com/#/blockgan/)] [[Code](https://github.com/thunguyenphuoc/BlockGAN)] 249 | 250 | * **GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis.**
251 | *[Katja Schwarz](https://katjaschwarz.github.io/), [Yiyi Liao](https://yiyiliao.github.io/), [Michael Niemeyer](https://m-niemeyer.github.io/), [Andreas Geiger](http://www.cvlibs.net/).*
252 | NeurIPS 2020. [[Paper](https://arxiv.org/abs/2007.02442)] [[Project](https://avg.is.tuebingen.mpg.de/publications/schwarz2020neurips)] [[Code](https://github.com/autonomousvision/graf)] 253 | 254 | * **HoloGAN: Unsupervised learning of 3D representations from natural images.**
255 | *[Thu Nguyen-Phuoc](https://monkeyoverflow.com/about/), [Chuan Li](https://lambdalabs.com/blog/author/chuan/), Lucas Theis, [Christian Richardt]( https://richardt.name/), [Yong-liang Yang](http://yongliangyang.net/).*
256 | ICCV 2019. [[Paper](https://arxiv.org/abs/1904.01326)] [[Project](https://www.monkeyoverflow.com/hologan-unsupervised-learning-of-3d-representations-from-natural-images/)] [[Code](https://github.com/thunguyenphuoc/HoloGAN)] 257 | 258 | ### Conditional 3D Generative Models 259 | 260 | * **3D-aware Conditional Image Synthesis.**
261 | *[Kangle Deng](https://dunbar12138.github.io/), [Gengshan Yang](https://gengshan-y.github.io/), [Deva Ramanan](https://www.cs.cmu.edu/~deva/), [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/).*
262 | CVPR 2023. [[Paper](https://arxiv.org/abs/2302.08509)] [[Project](https://www.cs.cmu.edu/~pix2pix3D/)] [[Code](https://github.com/dunbar12138/pix2pix3D)] 263 | 264 | * **Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields.**
265 | *[Yuedong Chen](https://donydchen.github.io/), [Qianyi Wu](https://wuqianyi.top/), [Chuanxia Zheng](https://www.chuanxiaz.com/), [Tat-Jen Cham](https://personal.ntu.edu.sg/astjcham/), [Jianfei Cai](https://jianfei-cai.github.io/).*
266 | ECCV 2022. [[Paper](https://arxiv.org/abs/2203.10821)] [[Project](https://donydchen.github.io/sem2nerf)] [[Code](https://github.com/donydchen/sem2nerf)] 267 | 268 | * **IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis.**
269 | *[Jingxiang Sun](https://github.com/MrTornado24), [Xuan Wang](https://mrtornado24.github.io/IDE-3D/), [Yichun Shi](https://seasonsh.github.io/), [Lizhen Wang](https://lizhenwangt.github.io/), [Jue Wang](https://juewang725.github.io/), [Yebin Liu](https://liuyebin.com/).*
270 | SIGGRAPH Asia 2022. [[Paper](https://arxiv.org/abs/2205.15517)] [[Project](https://mrtornado24.github.io/IDE-3D/)] [[Code](https://github.com/MrTornado24/IDE-3D)] 271 | 272 | * **NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields.**
273 | *Kaiwen Jiang, [Shu-Yu Chen](http://people.geometrylearning.com/csy/), [Feng-Lin Liu](http://people.geometrylearning.com/lfl/), [Hongbo Fu](http://sweb.cityu.edu.hk/hongbofu/), [Lin Gao](http://www.geometrylearning.com/cn/).*
274 | SIGGRAPH Asia 2022. [[Paper](https://arxiv.org/abs/2211.07968)] [[Project](http://geometrylearning.com/NeRFFaceEditing/)] 275 | 276 | * **GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds.**
277 | *Zekun Hao, Arun Mallya, Serge Belongie, Ming-Yu Liu.*
278 | ICCV 2021. [[Paper](https://arxiv.org/abs/2104.07659)] [[Project](https://nvlabs.github.io/GANcraft/)] [[Code](https://github.com/NVlabs/imaginaire)] 279 | 280 | ## 3D-aware Diffusion Models for a Single Image Category 281 | 282 | * **Generating Images with 3D Annotations Using Diffusion Models.**
283 | *[Wufei Ma](https://wufeim.github.io/), [Qihao Liu](https://qihao067.github.io/), [Jiahao Wang](https://jiahaoplus.github.io/), Angtian Wang, Xiaoding Yuan, Yi Zhang, Zihao Xiao, Guofeng Zhang, Beijia Lu, Ruxiao Duan, Yongrui Qi, [Adam Kortylewski](https://adamkortylewski.com/), [Yaoyao Liu](https://www.cs.jhu.edu/~yyliu/), [Alan Yuille](https://www.cs.jhu.edu/~ayuille/).*
284 | ICLR 2024. [[Paper](https://arxiv.org/abs/2306.08103)] [[Project](https://ccvl.jhu.edu/3D-DST/)] [[Code](https://github.com/wufeim/DST3D)] 285 | 286 | * **GeNVS: Generative Novel View Synthesis with 3D-Aware Diffusion Models.**
287 | *Eric R. Chan, Koki Nagano, Matthew A. Chan, Alexander W. Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, Gordon Wetzstein.*
288 | ICCV 2023. [[Paper](https://arxiv.org/abs/2304.02602)] [[Project](https://nvlabs.github.io/genvs/)] [[Code](https://github.com/NVlabs/genvs)] 289 | 290 | * **Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction.**
291 | *Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, Hao Su.*
292 | ICCV 2023. [[PDF](http://arxiv.org/abs/2304.06714)] [[Project](https://lakonik.github.io/ssdnerf)] [[Code](https://github.com/Lakonik/SSDNeRF)] 293 | 294 | * **3D-aware Image Generation using 2D Diffusion Models.**
295 | *[Jianfeng Xiang](https://jeffreyxiang.github.io/), Jiaolong Yang, Binbin Huang, Xin Tong.*
296 | ICCV 2023. [[Paper](https://arxiv.org/abs/2303.17905)] [[Project](https://jeffreyxiang.github.io/ivid/)] [[Code](https://github.com/JeffreyXiang/ivid)] 297 | 298 | * **HoloFusion: Towards Photo-realistic 3D Generative Modeling.**
299 | *Animesh Karnewar, Niloy J. Mitra, Andrea Vedaldi, David Novotny.*
300 | ICCV 2023. [[Paper](http://arxiv.org/abs/2308.14244)] [[Project](https://holodiffusion.github.io/holofusion)] 301 | 302 | * **HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion.**
303 | *[Ziya Erkoç](https://ziyaerkoc.com/), [Fangchang Ma](https://fangchangma.github.io/), [Qi Shan](http://shanqi.github.io/), [Matthias Nießner](https://niessnerlab.org/members/matthias_niessner/profile.html), [Angela Dai](https://www.3dunderstanding.org/team.html).*
304 | ICCV 2023. [[Paper](https://arxiv.org/abs/2303.17015)] [[Project](https://ziyaerkoc.com/hyperdiffusion/)] 305 | 306 | * **LatentSwap3D: Semantic Edits on 3D Image GANs.**
307 | *Enis Simsar, Alessio Tonioni, Evin Pınar Örnek, Federico Tombari.*
308 | ICCV 2023 Workshop on AI3DCC. [[Paper](https://arxiv.org/abs/2212.01381)] 309 | 310 | * **DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models.**
311 | *[Jamie Wynn](https://scholar.google.com/citations?user=ASP-uu4AAAAJ&hl=en&oi=ao) and [Daniyar Turmukhambetov](https://scholar.google.com/citations?user=ELFm0CgAAAAJ&hl=en&oi=ao).*
312 | CVPR 2023. [[Paper](https://arxiv.org/abs/2302.12231)] [[Supplementary material](https://storage.googleapis.com/niantic-lon-static/research/diffusionerf/diffusionerf_supplemental.Paper)] [[COde](https://github.com/nianticlabs/diffusionerf)] 313 | 314 | * **NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models.**
315 | *Seung Wook Kim, Bradley Brown, Kangxue Yin, Karsten Kreis, Katja Schwarz, Daiqing Li, Robin Rombach, Antonio Torralba, Sanja Fidler.*
316 | CVPR 2023. [[Paper](https://arxiv.org/abs/2304.09787)] [[Project](https://research.nvidia.com/labs/toronto-ai/NFLDM/)] 317 | 318 | * **Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion.**
319 | *Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, Baining Guo.*
320 | CVPR 2023. [[Paper](https://arxiv.org/abs/2212.06135)] [[Project](https://3d-avatar-diffusion.microsoft.com/)] 321 | 322 | * **DiffRF: Rendering-guided 3D Radiance Field Diffusion.**
323 | *[Norman Müller](https://niessnerlab.org/members/norman_mueller/profile.html), [Yawar Siddiqui](https://niessnerlab.org/members/yawar_siddiqui/profile.html), [Lorenzo Porzi](https://scholar.google.com/citations?user=vW1gaVEAAAAJ), [Samuel Rota Bulò](https://scholar.google.com/citations?hl=de&user=484sccEAAAAJ), [Peter Kontschieder](https://scholar.google.com/citations?user=CxbDDRMAAAAJ&hl=en), [Matthias Nießner](https://niessnerlab.org/members/matthias_niessner/profile.html).*
324 | CVPR 2023 (Highlight). [[Paper](https://arxiv.org/abs/2212.01206)] [[Project](https://sirwyver.github.io/DiffRF/)] 325 | 326 | * **RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation.**
327 | *Titas Anciukevičius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy J. Mitra, Paul Guerrero.*
328 | CVPR 2023. [[Paper](https://arxiv.org/abs/2211.09869)] [[Project](https://holodiffusion.github.io/)] [[Code](https://github.com/Anciukevicius/RenderDiffusion)] 329 | 330 | * **SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction.**
331 | *[Zhizhuo Zhou](https://www.zhiz.dev/), [Shubham Tulsiani](https://shubhtuls.github.io/).*
332 | CVPR 2023. [[Paper](https://arxiv.org/abs/2212.00792)] [[Project](https://sparsefusion.github.io/)] [[Code](https://github.com/zhizdev/sparsefusion)] 333 | 334 | * **HoloDiffusion: Training a 3D Diffusion Model using 2D Images.**
335 | *Animesh Karnewar, Andrea Vedaldi, David Novotny, Niloy Mitra.*
336 | CVPR 2023. [[Paper](https://arxiv.org/abs/2303.16509)] [[Project](https://3d-diffusion.github.io/)] 337 | 338 | * **3DiM: Novel View Synthesis with Diffusion Models.**
339 | *Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, Mohammad Norouzi.*
340 | ICLR 2023. [[Paper](https://arxiv.org/abs/2210.04628)] [[Project](https://3d-diffusion.github.io/)] 341 | 342 | * **3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models.**
343 | *[Biao Zhang](https://1zb.github.io/), [Jiapeng Tang](https://tangjiapeng.github.io/), [Matthias Niessner](https://www.niessnerlab.org/), [Peter Wonka](http://peterwonka.net/).*
344 | SIGGRAPH 2023. [[Paper](https://arxiv.org/abs/2301.11445)] [[Project](https://1zb.github.io/3DShape2VecSet/)] [[Code](https://github.com/1zb/3DShape2VecSet)] 345 | 346 | * **GAUDI: A Neural Architect for Immersive 3D Scene Generation.**
347 | *Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Walter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, Afshin Dehghan, Josh Susskind.*
348 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2212.01381)] [[Project](https://github.com/apple/ml-gaudi)] 349 | 350 | * **Learning a Diffusion Prior for NeRFs.**
351 | *Guandao Yang, Abhijit Kundu, Leonidas J. Guibas, Jonathan T. Barron, Ben Poole.*
352 | arxiv 2023. [[Paper](https://arxiv.org/abs/2304.14473)] 353 | 354 | * **3D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models.**
355 | *Gimin Nam, Mariem Khlifi, Andrew Rodriguez, Alberto Tono, Linqi Zhou, Paul Guerrero.*
356 | arxiv 2022. [[Paper](https://arxiv.org/abs/2212.00842)] 357 | 358 | ## 3D-Aware Generative Models on ImageNet 359 | 360 | * **VQ3D: Learning a 3D-Aware Generative Model on ImageNet.**
361 | *Kyle Sargent, Jing Yu Koh, Han Zhang, Huiwen Chang, Charles Herrmann, Pratul Srinivasan, Jiajun Wu, Deqing Sun.*
362 | ICCV 2023 (Oral). [[Paper](https://arxiv.org/abs/2302.06833)] [[Project](http://kylesargent.github.io/vq3d)] 363 | 364 | * **3D Generation on ImageNet.**
365 | *Ivan Skorokhodov, Aliaksandr Siarohin, Yinghao Xu, Jian Ren, Hsin-Ying Lee, Peter Wonka, Sergey Tulyakov.*
366 | ICLR 2023 (Oral). [[Paper](https://openreview.net/forum?id=U2WjB9xxZ9q)] [[Project](https://u2wjb9xxz9q.github.io/)] [[Code](https://justimyhxu.github.io/pub.html)] 367 | 368 | ## 3D-aware Video Synthesis 369 | 370 | * **3D-Aware Video Generation.**
371 | *[Sherwin Bahmani](https://sherwinbahmani.github.io/), [Jeong Joon Park](https://jjparkcv.github.io/), [Despoina Paschalidou](https://paschalidoud.github.io/), [Hao Tang](https://scholar.google.com/citations?user=9zJkeEMAAAAJ&hl=en/), [Gordon Wetzstein](https://stanford.edu/~gordonwz/), [Leonidas Guibas](https://geometry.stanford.edu/member/guibas/), [Luc Van Gool](https://ee.ethz.ch/the-department/faculty/professors/person-detail.OTAyMzM=.TGlzdC80MTEsMTA1ODA0MjU5.html/), [Radu Timofte](https://ee.ethz.ch/the-department/people-a-z/person-detail.MjAxNjc4.TGlzdC8zMjc5LC0xNjUwNTg5ODIw.html/).*
372 | TMLR 2023. [[Paper](https://arxiv.org/abs/2206.14797)] [[Project](https://sherwinbahmani.github.io/3dvidgen/)] [[Code](https://github.com/sherwinbahmani/3dvideogeneration/)] 373 | 374 | * **Streaming Radiance Fields for 3D Video Synthesis.**
375 | *Lingzhi Li, Zhen Shen, Zhongshu Wang, Li Shen, Ping Tan.*
376 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2210.14831)] [[Code](https://github.com/AlgoHunt/StreamRF)] 377 | 378 | ## INR-based 3D Novel View Synthesis 379 | 380 | ### Neural Scene Representations 381 | 382 | * **Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations.**
383 | *Mehdi S. M. Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani Vora, Mario Lucic, Daniel Duckworth, Alexey Dosovitskiy, Jakob Uszkoreit, Thomas Funkhouser, Andrea Tagliasacchi.*
384 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.13152)] [[Project](https://srt-paper.github.io/)] [[Code](https://github.com/stelzner/srt)] 385 | 386 | * **Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering.**
387 | *Vincent Sitzmann, Semon Rezchikov, William T. Freeman, Joshua B. Tenenbaum, Fredo Durand.*
388 | NeurIPS 2021 (Spotlight). [[Paper](https://arxiv.org/abs/2106.02634)] [[Project](https://vsitzmann.github.io/lfns/)] [[Code](https://github.com/vsitzmann/light-field-networks)] 389 | 390 | * **Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.**
391 | *[Jonathan T. Barron](https://jonbarron.info/), [Ben Mildenhall](https://bmild.github.io/), [Matthew Tancik](https://www.matthewtancik.com/), [Peter Hedman](https://phogzone.com/cv.html), [Ricardo Martin-Brualla](http://ricardomartinbrualla.com/), [Pratul P. Srinivasan](https://pratulsrinivasan.github.io/).*
392 | ICCV 2021. [[Paper](https://arxiv.org/abs/2103.13415)] [[Project](http://jonbarron.info/mipnerf)] [[Github](https://github.com/google/mipnerf)] 393 | 394 | * **NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.**
395 | *[Ben Mildenhall](http://people.eecs.berkeley.edu/~bmild/), [Pratul P. Srinivasan](https://people.eecs.berkeley.edu/~pratul/), [Matthew Tancik](http://www.matthewtancik.com/), [Jonathan T. Barron](https://jonbarron.info/), [Ravi Ramamoorthi](http://cseweb.ucsd.edu/~ravir/), [Ren Ng](https://www2.eecs.berkeley.edu/Faculty/Homepages/yirenng.html).*
396 | ECCV 2020. [[Paper](https://arxiv.org/abs/2003.08934)] [[Project](http://tancik.com/nerf)] [[Gtihub-Tensorflow](https://github.com/bmild/nerf)] [[krrish94-PyTorch](https://github.com/krrish94/nerf-pytorch)] [[yenchenlin-PyTorch](https://github.com/yenchenlin/nerf-pytorch)] 397 | 398 | * **Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision.**
399 | *Michael Niemeyer, Lars Mescheder, Michael Oechsle, Andreas Geiger.*
400 | CVPR 2020. [[Paper](http://www.cvlibs.net/publications/Niemeyer2020CVPR.Paper)] [[Code](https://github.com/autonomousvision/differentiable_volumetric_rendering)] 401 | 402 | * **Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations.**
403 | *[Vincent Sitzmann](https://vsitzmann.github.io/), Michael Zollhöfer, Gordon Wetzstein.*
404 | NeurIPS 2019 (Oral, Honorable Mention "Outstanding New Directions"). 405 | [[Paper](http://arxiv.org/abs/1906.01618)] [[Project](https://github.com/vsitzmann/scene-representation-networks)] [[Code](https://github.com/vsitzmann/scene-representation-networks)] [[Dataset](https://drive.google.com/drive/folders/1OkYgeRcIcLOFu1ft5mRODWNQaPJ0ps90?usp=sharing)] 406 | 407 | * **LLFF: Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines.**
408 | *[Ben Mildenhall](http://people.eecs.berkeley.edu/~bmild/), Pratul Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, Abhishek Kar.*
409 | SIGGRAPH 2019. [[Paper](https://arxiv.org/abs/1905.00889)] [[Project](https://people.eecs.berkeley.edu/~bmild/llff/)] [[Code](https://github.com/Fyusion/LLFF)] 410 | 411 | * **DeepVoxels: Learning Persistent 3D Feature Embeddings.**
412 | *Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer.*
413 | CVPR 2019 (Oral). [[Paper](https://arxiv.org/abs/1812.01024)] [[Project](http://vsitzmann.github.io/deepvoxels/)] [[Code](https://github.com/vsitzmann/deepvoxels)] 414 | 415 | ### Acceleration 416 | 417 | * **Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.**
418 | *[Thomas Müller](https://tom94.net/), [Alex Evans](https://research.nvidia.com/person/alex-evans), [Christoph Schied](https://research.nvidia.com/person/christoph-schied), [Alexander Keller](https://research.nvidia.com/person/alex-keller).*
419 | SIGGRAPH (TOG) 2022. [[Paper](https://nvlabs.github.io/instant-ngp/assets/mueller2022instant.Paper)] [[Project](https://nvlabs.github.io/instant-ngp)] [[Code](https://github.com/NVlabs/instant-ngp)] 420 | 421 | * **DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Rendering.**
422 | *[Liwen Wu](https://lwwu2.github.io/), [Jae Yong Lee](https://jyl.kr/), [Anand Bhattad](https://anandbhattad.github.io/), [Yuxiong Wang](https://yxw.web.illinois.edu/), [David A. Forsyth](http://luthuli.cs.uiuc.edu/~daf/).*
423 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.10427)] [[Project](https://lwwu2.github.io/diver/)] [[Code](https://github.com/lwwu2/diver)] 424 | 425 | * **KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs.**
426 | *Christian Reiser, Songyou Peng, Yiyi Liao, Andreas Geiger.*
427 | ICCV 2021. [[Paper](https://arxiv.org/abs/2103.13744)] [[Code](https://github.com/creiser/kilonerf)] 428 | 429 | * **FastNeRF: High-Fidelity Neural Rendering at 200FPS.**
430 | *Stephan J. Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, Julien Valentin.*
431 | ICCV 2021. [[Paper](https://arxiv.org/abs/2103.10380)] 432 | 433 | * **PlenOctrees for Real-time Rendering of Neural Radiance Fields.**
434 | *[Alex Yu](https://alexyu.net/), [Ruilong Li](https://www.liruilong.cn/), [Matthew Tancik](https://www.matthewtancik.com/), [Hao Li](https://www.hao-li.com/), [Ren Ng](https://www2.eecs.berkeley.edu/Faculty/Homepages/yirenng.html), [Angjoo Kanazawa](https://people.eecs.berkeley.edu/~kanazawa/).*
435 | ICCV 2021. [[Paper](https://arxiv.org/abs/2103.14024)] [[Project](https://alexyu.net/plenoctrees/)] [[Code](https://github.com/sxyu/plenoctree)] 436 | 437 | * **Baking Neural Radiance Fields for Real-Time View Synthesis.**
438 | *[Peter Hedman](https://phogzone.com/), [Pratul P. Srinivasan](https://pratulsrinivasan.github.io/), [Ben Mildenhall](https://bmild.github.io/), [Jonathan T. Barron](https://jonbarron.info/), [Paul Debevec](https://www.pauldebevec.com/).*
439 | ICCV 2021 (oral). [[Paper](https://arxiv.org/abs/2103.14645)] [[Project](https://nerf.live/)] [[Code](https://github.com/google-research/google-research/tree/master/snerg)] 440 | 441 | * **AutoInt: Automatic Integration for Fast Neural Volume Rendering.**
442 | *David B. Lindell, Julien N. P. Martel, Gordon Wetzstein.*
443 | CVPR 2021 (oral). [[Paper](https://arxiv.org/abs/2012.01714)] [[Project](http://www.computationalimaging.org/publications/automatic-integration/)] [[Code](https://github.com/computational-imaging/automatic-integration)] 444 | 445 | * **NSVF: Neural Sparse Voxel Fields.**
446 | *[Lingjie Liu](https://lingjie0206.github.io/), Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, Christian Theobalt.*
447 | NeurIPS 2020. [[Paper](https://arxiv.org/abs/2007.11571)] [[Project](https://lingjie0206.github.io/papers/NSVF/)] [[Code](https://github.com/facebookresearch/NSVF)] 448 | 449 | ### From Constrained to In-the-wild Conditions 450 | 451 | #### Few Images 452 | 453 | * **GRF: Learning a General Radiance Field for 3D Representation and Rendering.**
454 | *Alex Trevithick, Bo Yang.*
455 | ICCV 2021. [[Paper](https://openaccess.thecvf.com/content/ICCV2021/html/Trevithick_GRF_Learning_a_General_Radiance_Field_for_3D_Representation_and_ICCV_2021_paper.html)] [[Code](https://github.com/alextrevithick/GRF)] 456 | 457 | * **MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo.**
458 | *[Anpei Chen](https://apchenstu.github.io/), [Zexiang Xu](http://cseweb.ucsd.edu/~zex014/), Fuqiang Zhao, Xiaoshuai Zhang, [Fanbo Xiang](https://www.fbxiang.com/), [Jingyi Yu](http://vic.shanghaitech.edu.cn/vrvc/en/people/), [Hao Su](https://cseweb.ucsd.edu/~haosu/).*
459 | ICCV 2021. [[Paper](https://arxiv.org/abs/2103.15595)] [[Project](https://apchenstu.github.io/mvsnerf/)] [[Code](https://github.com/apchenstu/mvsnerf)] 460 | 461 | * **CodeNeRF: Disentangled Neural Radiance Fields for Object Categories.**
462 | *Wonbong Jang, Lourdes Agapito.*
463 | ICCV 2021. [[Paper](https://arxiv.org/abs/2109.01750)] [[Project](https://sites.google.com/view/wbjang/home/codenerf)] [[Code](https://github.com/wayne1123/code-nerf)] 464 | 465 | * **pixelNeRF: Neural Radiance Fields from One or Few Images.**
466 | *[Alex Yu](https://alexyu.net/), Vickie Ye, Matthew Tancik, Angjoo Kanazawa.*
467 | CVPR 2021. [[Paper](https://arxiv.org/abs/2012.02190)] [[Project](https://alexyu.net/pixelnerf)] [[Code](https://github.com/sxyu/pixel-nerf)] 468 | 469 | * **IBRNet: Learning Multi-View Image-Based Rendering.**
470 | *Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser.*
471 | CVPR 2021. [[Paper](https://arxiv.org/abs/2102.13090)] [[Project](https://ibrnet.github.io/)] [[Code](https://github.com/googleinterns/IBRNet)] 472 | 473 | * **NeRF-VAE: A Geometry Aware 3D Scene Generative Model.**
474 | *Adam R. Kosiorek, Heiko Strathmann, Daniel Zoran, Pol Moreno, Rosalia Schneider, Soňa Mokrá, Danilo J. Rezende.*
475 | ICML 2021. [[Paper](https://arxiv.org/abs/2104.00587)] 476 | 477 | #### Pose-free 478 | 479 | * **Self-Calibrating Neural Radiance Fields.**
480 | *Yoonwoo Jeong, Seokjun Ahn, Christopher Choy, Animashree Anandkumar, Minsu Cho, Jaesik Park.*
481 | ICCV 2021. [[Paper](https://arxiv.org/abs/2108.13826)] [[Project](https://postech-cvlab.github.io/SCNeRF/)] [[Code](https://github.com/POSTECH-CVLab/SCNeRF)] 482 | 483 | * **BARF: Bundle-Adjusting Neural Radiance Fields.**
484 | *[Chen-Hsuan Lin](https://chenhsuanlin.bitbucket.io/), [Wei-Chiu Ma](http://people.csail.mit.edu/weichium/), Antonio Torralba, Simon Lucey.*
485 | ICCV 2021. [[Paper](https://arxiv.org/abs/2104.06405)] [[Code](https://github.com/chenhsuanlin/bundle-adjusting-NeRF)] 486 | 487 | * **NeRF--: Neural Radiance Fields Without Known Camera Parameters.**
488 | *[Zirui Wang](https://scholar.google.com/citations?user=zCBKqa8AAAAJ&hl=en), [Shangzhe Wu](http://elliottwu.com), [Weidi Xie](https://weidixie.github.io/weidi-personal-webpage/), [Min Chen](https://sites.google.com/site/drminchen/home), [Victor Adrian Prisacariu](https://eng.ox.ac.uk/people/victor-prisacariu/).*
489 | arxiv 2021. [[Paper](https://arxiv.org/abs/2102.07064)] [[Project](http://nerfmm.active.vision/)] [[Code](https://github.com/ActiveVisionLab/nerfmm)] 490 | 491 | #### Varying Appearance 492 | 493 | * **NeRFReN: Neural Radiance Fields with Reflections.**
494 | *Yuan-Chen Guo, Di Kang, Linchao Bao, Yu He, Song-Hai Zhang.*
495 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.15234)] [[Project](https://bennyguo.github.io/nerfren/] 496 | 497 | * **NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections.**
498 | *[Ricardo Martin-Brualla](http://www.ricardomartinbrualla.com/), [Noha Radwan](https://scholar.google.com/citations?user=g98QcZUAAAAJ&hl=en), [Mehdi S. M. Sajjadi](https://research.google/people/105804/), [Jonathan T. Barron](https://jonbarron.info/), [Alexey Dosovitskiy](https://scholar.google.com/citations?user=FXNJRDoAAAAJ&hl=en), [Daniel Duckworth](http://www.stronglyconvex.com/about.html).*
499 | CVPR 2021 (oral). [[Paper](https://arxiv.org/abs/2008.02268)] [[Code](https://nerf-w.github.io/)] 500 | 501 | #### Large-scale Scene 502 | 503 | * **Grid-guided Neural Radiance Fields for Large Urban Scenes.**
504 | *Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin.*
505 | CVPR 2023. [[Paper](https://arxiv.org/abs/2303.14001)] [[Project](https://city-super.github.io/gridnerf/)] 506 | 507 | * **S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint.**
508 | *[Wenqi Yang](https://ywq.github.io/), [Guanying Chen](https://guanyingc.github.io/), [Chaofeng Chen](http://chaofengc.github.io/), [Zhenfang Chen](https://zfchenunique.github.io/), [Kwan-Yee K. Wong](http://i.cs.hku.hk/~kykwong/).*
509 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2210.08936)] [[Project](https://ywq.github.io/s3nerf)] 510 | 511 | * **BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering.**
512 | *Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Anyi Rao, Christian Theobalt, Bo Dai, Dahua Lin.*
513 | ECCV 2022. [[Paper](https://arxiv.org/abs/2112.05504)] [[Project](https://city-super.github.io/citynerf)] 514 | 515 | * **Block-NeRF: Scalable Large Scene Neural View Synthesis.**
516 | *Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P. Srinivasan, Jonathan T. Barron, Henrik Kretzschmar.*
517 | CVPR 2022. [[Paper](https://arxiv.org/abs/2202.05263)] [[Project](https://waymo.com/research/block-nerf/)] 518 | 519 | * **Urban Radiance Fields.**
520 | *[Konstantinos Rematas](http://www.krematas.com/), Andrew Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasacchi, Thomas Funkhouser, Vittorio Ferrari.*
521 | CVPR 2022. [[Paper](https://arxiv.org/abs/2111.14643)] [[Project](https://urban-radiance-fields.github.io/)] 522 | 523 | * **Mega-NERF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs.**
524 | *Haithem Turki, Deva Ramanan, Mahadev Satyanarayanan.*
525 | CVPR 2022. [[Paper](https://openaccess.thecvf.com/content/CVPR2022/html/Turki_Mega-NERF_Scalable_Construction_of_Large-Scale_NeRFs_for_Virtual_Fly-Throughs_CVPR_2022_paper.html)] [[Code](https://github.com/cmusatyalab/mega-nerf)] 526 | 527 | * **Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry.**
528 | *Dawa Derksen, Dario Izzo.*
529 | CVPR 2021. [[Paper](https://arxiv.org/abs/2104.09877)] [[Code](https://github.com/esa/snerf)] 530 | 531 | #### Dynamic Scene 532 | 533 | * **NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields.**
534 | *Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, Andreas Geiger.*
535 | TVCG 2023. [[Paper](https://arxiv.org/abs/2210.15947)] [[Project](https://lsongx.github.io/projects/nerfplayer.html)] 536 | 537 | * **Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects.**
538 | *Ziyu Wang, Yu Deng, Jiaolong Yang, Jingyi Yu, Xin Tong.*
539 | Pacific Graphics 2022. [[Paper](https://arxiv.org/abs/2209.04183)] [[Code](https://ziyuwang98.github.io/GDRF/)] 540 | 541 | * **Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera.**
542 | *[Hongrui Cai](https://rainbowrui.github.io/), [Wanquan Feng](https://github.com/WanquanF), [Xuetao Feng](https://scholar.google.com/citations?hl=en&user=5G-2EFcAAAAJ), [Yan Wang](), [Juyong Zhang](http://staff.ustc.edu.cn/~juyong/).*
543 | NeurIPS 2022. [[Paper](https://arxiv.org/abs/2206.15258)] [[Project](https://ustc3dv.github.io/ndr/)] [[Code](https://github.com/USTC3DV/NDR-code)] 544 | 545 | * **LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling.**
546 | *Boyan Jiang, Xinlin Ren, Mingsong Dou, Xiangyang Xue, Yanwei Fu, Yinda Zhang.*
547 | ECCV 2022. [[Paper](https://arxiv.org/abs/2208.08622)] [[Code](https://boyanjiang.github.io/LoRD/)] 548 | 549 | * **Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time.**
550 | *[Liao Wang](https://aoliao12138.github.io/), [Jiakai Zhang](https://jiakai-zhang.github.io/), Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu, Lan Xu, Jingyi Yu.*
551 | CVPR 2022 (Oral). [[Paper](https://arxiv.org/abs/2202.08614)] [[Project](https://aoliao12138.github.io/FPO/)] 552 | 553 | * **CoNeRF: Controllable Neural Radiance Fields.**
554 | *Kacper Kania, Kwang Moo Yi, Marek Kowalski, Tomasz Trzciński, Andrea Taliasacchi.*
555 | CVPR 2022. [[Paper](https://arxiv.org/abs/2112.01983)] [[Project](https://conerf.github.io/)] 556 | 557 | * **Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video.**
558 | *Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, Christian Theobalt.*
559 | ICCV 2021. [[Paper](https://arxiv.org/abs/2012.12247)] [[Project](https://gvv.mpi-inf.mpg.de/projects/nonrigid_nerf/)] [[Code](https://github.com/facebookresearch/nonrigid_nerf)] 560 | 561 | * **NeRFlow: Neural Radiance Flow for 4D View Synthesis and Video Processing.**
562 | *Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu.*
563 | ICCV 2021. [[Paper](https://arxiv.org/abs/2012.09790)] [[Project](https://yilundu.github.io/nerflow/)] 564 | 565 | * **Nerfies: Deformable Neural Radiance Fields.**
566 | *[Keunhong Park](https://keunhong.com/), [Utkarsh Sinha](https://utkarshsinha.com/), [Jonathan T. Barron](https://jonbarron.info/), [Sofien Bouaziz](http://sofienbouaziz.com/), [Dan B Goldman](https://www.danbgoldman.com/), [Steven M. Seitz](https://homes.cs.washington.edu/~seitz/), [Ricardo-Martin Brualla](http://www.ricardomartinbrualla.com/).*
567 | ICCV 2021. [[Paper](https://arxiv.org/abs/2011.12948)] [[Project](https://nerfies.github.io/)] [[Code](https://github.com/google/nerfies)] 568 | 569 | * **D-NeRF: Neural Radiance Fields for Dynamic Scenes.**
570 | *[Albert Pumarola](https://www.albertpumarola.com/), [Enric Corona](https://www.iri.upc.edu/people/ecorona/), [Gerard Pons-Moll](http://virtualhumans.mpi-inf.mpg.de/), [Francesc Moreno-Noguer](http://www.iri.upc.edu/people/fmoreno/).*
571 | CVPR 2021. [[Paper](https://arxiv.org/abs/2011.13961)] [[Project](https://www.albertpumarola.com/research/D-NeRF/index.html)] [[Code](https://github.com/albertpumarola/D-NeRF)] [[Data](https://www.dropbox.com/s/0bf6fl0ye2vz3vr/data.zip?dl=0)] 572 | 573 | * **Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction.**
574 | *Guy Gafni, Justus Thies, Michael Zollhöfer, Matthias Nießner.*
575 | CVPR 2021. [[Paper](https://arxiv.org/abs/2012.03065)] [[Project](https://gafniguy.github.io/4D-Facial-Avatars/)] [[Video](https://youtu.be/m7oROLdQnjk)] 576 | 577 | * **NSFF: Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes.**
578 | *[Zhengqi Li](https://www.cs.cornell.edu/~zl548/), [Simon Niklaus](https://sniklaus.com/welcome), [Noah Snavely](https://www.cs.cornell.edu/~snavely/), [Oliver Wang](https://research.adobe.com/person/oliver-wang/).*
579 | CVPR 2021. [[Paper](https://arxiv.org/abs/2011.13084)] [[Project](http://www.cs.cornell.edu/~zl548/NSFF)] [[Code](https://github.com/zhengqili/Neural-Scene-Flow-Fields)] 580 | 581 | * **Space-time Neural Irradiance Fields for Free-Viewpoint Video.**
582 | *[Wenqi Xian](https://www.cs.cornell.edu/~wenqixian/), [Jia-Bin Huang](https://filebox.ece.vt.edu/~jbhuang/), [Johannes Kopf](https://johanneskopf.de/), [Changil Kim](https://changilkim.com/).*
583 | CVPR 2021. [[Paper](https://arxiv.org/abs/2011.12950)] [[Project](https://video-nerf.github.io/)] 584 | 585 | The following papers are not directly related to 3D-aware image synthesis. But it would be beneficial to pay attention to those works. For example, in our survey, inverse rendering are not classified as 3D-aware image synthesis as they are not deliberately designed for this purpose. But with the inferred intrinsic components, photorealistic images can be rendered. 3D reconstruction models geometry only with no appearance information, meaning them not able to render images with photorealistic textures. But these representations have been introduced as the geometric representation along with a textural representation (e.g., Texture Field) for 3D-aware image synthesis. 586 | 587 | ## 3D Representations 588 | 589 | * **K-Planes: Explicit Radiance Fields in Space, Time, and Appearance.**
590 | *[Sara Fridovich-Keil](https://people.eecs.berkeley.edu/~sfk/), [Giacomo Meanti](https://www.iit.it/web/iit-mit-usa/people-details/-/people/giacomo-meanti), [Frederik Warburg](https://frederikwarburg.github.io/), [Benjamin Recht](https://people.eecs.berkeley.edu/~brecht/), [Angjoo Kanazawa](https://people.eecs.berkeley.edu/~kanazawa/).*
591 | CVPR 2023. [[Paper](https://arxiv.org/abs/2301.10241)] [[Project](https://sarafridov.github.io/K-Planes/)] [[Code](https://github.com/sarafridov/K-Planes)] 592 | 593 | * **HexPlane: A Fast Representation for Dynamic Scenes.**
594 | *[Ang Cao](https://caoang327.github.io/), [Justin Johnson](https://web.eecs.umich.edu/~justincj).*
595 | CVPR 2023. [[Paper](https://arxiv.org/abs/2301.09632)] [[Project](https://caoang327.github.io/HexPlane/)] [[Code](https://caoang327.github.io/HexPlane/)] 596 | 597 | * **GIFS: Neural Implicit Function for General Shape Representation.**
598 | *Jianglong Ye, Yuntao Chen, Naiyan Wang, Xiaolong Wang.*
599 | CVPR 2022. [[Paper](https://arxiv.org/abs/2204.07126)] [[Project](https://jianglongye.com/gifs)] [[Code](https://github.com/jianglongye/gifs)] 600 | 601 | * **Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields.**
602 | *[Wang Yifan](https://yifita.github.io/), Lukas Rahmann, [Olga Sorkine-Hornung](https://igl.ethz.ch/people/sorkine/).*
603 | ICLR 2022. [[Paper](https://arxiv.org/abs/2106.05187)] [[Project](https://yifita.github.io/publication/idf/)] [[Code](https://github.com/yifita/idf)] 604 | 605 | * **Neural Volumes: Learning Dynamic Renderable Volumes from Images.**
606 | *Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, Yaser Sheikh.*
607 | TOG 2019. [[Paper](https://arxiv.org/abs/1906.07751)] [[Code](https://github.com/facebookresearch/neuralvolumes)] 608 | 609 | * **DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation.**
610 | *eong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove.*
611 | CVPR 2019. [[Paper](http://openaccess.thecvf.com/content_CVPR_2019/html/Park_DeepSDF_Learning_Continuous_Signed_Distance_Functions_for_Shape_Representation_CVPR_2019_paper.html)] [[Code](https://github.com/facebookresearch/DeepSDF)] 612 | 613 | * **Occupancy Networks: Learning 3D Reconstruction in Function Space.**
614 | *Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, Andreas Geiger.*
615 | CVPR 2019. [[Paper](https://arxiv.org/abs/1812.03828)] [[Project](https://avg.is.mpg.de/publications/occupancy-networks)] [[Code](http://avg.is.tuebingen.mpg.de/publications/occupancy-networks)] 616 | 617 | ## Neural Inverse Rendering (Neural De-rendering) 618 | 619 | [Inverse rendering](https://github.com/weihaox/awesome-neural-rendering/blob/master/docs/NEURAL-INVERSE-RENDERING.md) is to infer underlying intrinsic components of a scene from rendered 2D images. These properties include shape (surface, depth, normal), material (albedo, reflectivity, shininess), and lighting (direction, intensity), which can be further used to render photorealistic images. 620 | 621 | * **NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination.**
622 | *[Xiuming Zhang](http://people.csail.mit.edu/xiuming/), [Pratul P. Srinivasan](https://pratulsrinivasan.github.io/), [Boyang Deng](https://boyangdeng.com/), [Paul Debevec](http://www.pauldebevec.com/), [William T. Freeman](http://billf.mit.edu/), [Jonathan T. Barron](https://jonbarron.info/).*
623 | SIGGRAPH Asia 2021. [[Paper](https://arxiv.org/abs/2106.01970)] [[Project](http://people.csail.mit.edu/xiuming/projects/nerfactor/)] [[Code](https://github.com/google/nerfactor)] 624 | 625 | * **Extracting Triangular 3D Models, Materials, and Lighting From Images.**
626 | *[Jacob Munkberg](https://research.nvidia.com/person/jacob-munkberg), [Jon Hasselgren](https://research.nvidia.com/person/jon-hasselgren), [Tianchang Shen](http://www.cs.toronto.edu/~shenti11/), [Jun Gao](http://www.cs.toronto.edu/~jungao/), [Wenzheng Chen](http://www.cs.toronto.edu/~wenzheng/), [Alex Evans](https://research.nvidia.com/person/alex-evans), [Thomas Müller](https://research.nvidia.com/person/thomas-mueller), [Sanja Fidler](https://www.cs.toronto.edu/~fidler/).*
627 | CVPR 2022. [[Paper](http://arxiv.org/abs/2111.12503)] [[Code](https://github.com/NVlabs/nvdiffrec)] [[Project](https://nvlabs.github.io/nvdiffrec/)] 628 | 629 | * **Modeling Indirect Illumination for Inverse Rendering.**
630 | *Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, Xiaowei Zhou.*
631 | CVPR 2022. [[Paper](https://arxiv.org/abs/2204.06837)] 632 | 633 | * **IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes.**
634 | *Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, Manmohan Chandraker.*
635 | CVPR 2022. [[Paper](https://openaccess.thecvf.com/content/CVPR2022/html/Zhu_IRISformer_Dense_Vision_Transformers_for_Single-Image_Inverse_Rendering_in_Indoor_CVPR_2022_paper.html)] 636 | 637 | * **De-rendering 3D Objects in the Wild.**
638 | *[Felix Wimbauer](https://www.linkedin.com/in/felixwimbauer/), [Shangzhe Wu](https://elliottwu.com/), [Christian Rupprecht](https://chrirupp.github.io/).*
639 | CVPR 2022. [[Paper](https://arxiv.org/abs/2201.02279)] [[Project](https://www.robots.ox.ac.uk/~vgg/research/derender3d/)] [[Code](https://github.com/Brummi/derender3d)] 640 | 641 | * **GAN2X: Non-Lambertian Inverse Rendering of Image GANs.**
642 | *[Xingang Pan](https://xingangpan.github.io/), [Ayush Tewari](https://ayushtewari.com/), [Lingjie Liu](https://lingjie0206.github.io/), [Christian Theobalt](http://www.mpi-inf.mpg.de/~theobalt/).*
643 | 3DV 2022. [[Paper](https://arxiv.org/abs/2206.09244)] [[Project](https://people.mpi-inf.mpg.de/~xpan/GAN2X/)] 644 | 645 | * **PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting.**
646 | *Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely.*
647 | CVPR 2021. [[Paper](https://arxiv.org/abs/2104.00674)] [[Project](https://kai-46.github.io/PhySG-website/)] 648 | 649 | * **Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering.**
650 | *[Fujun Luan](https://www.cs.cornell.edu/~fujun/), [Shuang Zhao](https://www.shuangz.com/), [Kavita Bala](https://www.cs.cornell.edu/~kb/), [Zhao Dong](http://flycooler.com/).*
651 | EGSR 2021. [[Paper](https://www.cs.cornell.edu/~fujun/files/egsr2021/paper.Paper)] [[Project](https://luanfujun.github.io/InverseMeshSVBRDF/)] [[Video](https://youtu.be/u9HqKGqvJhQ?t=8404)] 652 | 653 | * **Invertible Neural BRDF for Object Inverse Rendering.**
654 | *Zhe Chen, Shohei Nobuhara, Ko Nishino.*
655 | ECCV 2020. [[Paper](https://arxiv.org/abs/2008.04030)] [[Code](https://github.com/chenzhekl/iBRDF)] 656 | 657 | * **Polarimetric Multi-View Inverse Rendering.**
658 | *Jinyu Zhao, Yusuke Monno, Masatoshi Okutomi.*
659 | ECCV 2020. [[Paper](https://arxiv.org/abs/2007.08830)] 660 | 661 | * **Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image.**
662 | *[Zhengqin Li](http://sites.google.com/a/eng.ucsd.edu/zhengqinli/), [Mohammad Shafiei](https://www.linkedin.com/in/mohammadshafiei/), [Ravi Ramamoorthi](http://cseweb.ucsd.edu/~ravir/), [Kalyan Sunkavalli](http://www.kalyans.org/), [Manmohan Chandraker](http://cseweb.ucsd.edu/~mkchandraker/).*
663 | CVPR 2020.[[Paper](https://drive.google.com/file/d/18zG1kzVpL9XsEVBK95hbpnB-FMlChRXP/view?usp=sharing)] [[Project](http://cseweb.ucsd.edu/~viscomp/projects/CVPR20InverseIndoor/)] [[Code](https://github.com/lzqsd/InverseRenderingOfIndoorScene)] 664 | 665 | * **DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images.**
666 | *Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker.*
667 | ICML 2020. [[Paper](https://arxiv.org/abs/2007.06127)] 668 | 669 | * **Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer.**
670 | *Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler.*
671 | NeurIPS 2019. [[Paper](https://arxiv.org/abs/1908.01210)] [[Code](https://github.com/nv-tlabs/DIB-R)] 672 | 673 | * **InverseRenderNet: Learning Single Image Inverse Rendering.**
674 | *Ye Yu, William A. P. Smith.*
675 | CVPR 2019. [[Paper](http://openaccess.thecvf.com/content_CVPR_2019/html/Yu_InverseRenderNet_Learning_Single_Image_Inverse_Rendering_CVPR_2019_paper.html)] [[Code](https://github.com/YeeU/InverseRenderNet)] [[IIW Dataset](http://opensurfaces.cs.cornell.edu/publications/intrinsic/#download)] 676 | 677 | ## Neural Rerendering 678 | 679 | * **Hybrid Neural Fusion for Full-frame Video Stabilization.**
680 | *[Yu-Lun Liu](https://www.cmlab.csie.ntu.edu.tw/~nothinglo/), [Wei-Sheng Lai](https://www.wslai.net/), [Ming-Hsuan Yang](https://faculty.ucmerced.edu/mhyang/), [Yung-Yu Chuang](https://www.csie.ntu.edu.tw/~cyy/), [Jia-Bin Huang](https://filebox.ece.vt.edu/~jbhuang/).*
681 | ICCV 2021. [[Paper](https://arxiv.org/abs/2102.06205)] [[Code](https://alex04072000.github.io/NeRViS/)] 682 | 683 | * **Neural Lumigraph Rendering.**
684 | *Petr Kellnhofer, Lars Jebe, Andrew Jones, Ryan Spicer, Kari Pulli, Gordon Wetzstein.*
685 | CVPR 2021. [[Paper](https://arxiv.org/abs/2103.11571)] [[Project](http://www.computationalimaging.org/publications/nlr/)] [[Data](https://drive.google.com/file/d/1BBpIfrqwZNYmG1TiFljlCnwsmL2OUxNT/view?usp=sharing)] 686 | 687 | * **Neural Re-Rendering of Humans from a Single Image.**
688 | *Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt.*
689 | ECCV 2020. [[Paper](https://arxiv.org/abs/2101.04104)] 690 | 691 | * **Neural Rerendering in the Wild.**
692 | *Moustafa Meshry, Dan B Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla.*
693 | CVPR 2019. [[Paper](https://arxiv.org/abs/1904.04290)] 694 | 695 | * **Revealing Scenes by Inverting Structure from Motion Reconstructions.**
696 | *Francesco Pittaluga, Sanjeev J. Koppal, Sing Bing Kang, Sudipta N. Sinha.*
697 | CVPR 2019. [[Paper](https://arxiv.org/abs/1904.03303)] 698 | 699 | Datasets 700 | ------------------ 701 | 702 | Summary of popular 3D-aware image synthesis datasets. 703 | 704 | ### Multi-view image collections 705 | 706 | The images are rendered or collected according to different experimental settings, such as Synthetic-NeRF dataset, the DTU dataset, and the Tanks and Temples dataset for general purposes, the crowded Phototourism dataset for varying lighting conditions, the Blender Forward Facing (BLEFF) dataset to benchmark camera parameter estimation and novel view synthesis quality, and the San Francisco Alamo Square Dataset for large-scale scenes. 707 | 708 | Examples of multi-view image datasets. 709 | 710 | | **dataset** | **published in** | **# scene** | **# samples per scene** | **range (m × m)** | **resolution** | **keyword** | 711 | |:-----------------:|:--------------------:|:---------------------:|:---------------------------:|:----------------------:|:-------------------:|:-----------------------:| 712 | | DeepVoxels | CVPR 2019 | 4 simple objects | 479 / 1,000 | \ | 512 × 512 | synthetic, 360 degree | 713 | | NeRF Synthetics | ECCV 2020 | 8 complex objects | 100 / 200 | \ | 800 ×800 | synthetic, 360 degree | 714 | | NeRF Captured | ECCV 2020 | 8 complex scenes | 20-62 | a few | 1,008 × 756 | real, forward-facing | 715 | | DTU | CVPR 2014 | 124 scenes | 49 or 64 | a few to thousand | 1,600 × 1,200 | often used in few-views | 716 | | Tanks and Temples | CVPR 2015 | 14 objects and scenes | 4,395 - 21,871 | dozen to thousand | 8-megapixel | real, large-scale | 717 | | Phototourism | IJCV 2021 | 6 landmarks | 763-2,000 | dozen to thousand | 564-1,417 megapixel | varying illumination | 718 | | Alamo Square | CVPR 2022 | San Francisco | 2,818,745 | 570 × 960 | 1,200 × 900 | real, large-scale | 719 | 720 | ### Single-view image collections 721 | 722 | Summary of popular single-view image datasets organized by their major categories and sorted by their popularity. 723 | 724 | | **dataset** | **year** | **category** | **# samples** | **resolution** | **keyword** | 725 | |:-----------:|:------------:|:----------------------:|:-------------:|:------------------:|:----------------------:| 726 | | FFHQ | CVPR 2019 | Human Face | 70k | 1024 × 1024 | single simple-shape | 727 | | AFHQ | CVPR 2020 | Cat, Dog, and Wildlife | 15k | 512 × 512 | single simple-shape | 728 | | CompCars | CVPR 2015 | Real Car | 136K | 256 × 256 | single simple-shape | 729 | | CARLA | CoRL 2017 | Synthetic Car | 10k | 128 × 128 | single simple-shape | 730 | | CLEVR | CVPR 2017 | Objects | 100k | 256 × 256 | multiple, simple-shape | 731 | | LSUN | 2015 | Bedroom | 300K | 256 × 256 | single, simple-shape | 732 | | CelebA | ICCV 2015 | Human Face | 200k | 178 × 218 | single simple-shape | 733 | | CelebA-HQ | ICLR 2018 | Human Face | 30k | 1024 × 1024 | single, simple-shape | 734 | | MetFaces | NeurIPS 2020 | Art Face | 1336 | 1024 × 1024 | single, simple-shape | 735 | | M-Plants | NeurIPS 2022 | Variable-Shape | 141,824 | 256 × 256 | single, variable-shape | 736 | | M-Food | NeurIPS 2022 | Variable-Shape | 25,472 | 256 × 256 | single, variable-shape | 737 | 738 | Citation 739 | ------------------ 740 | 741 | If this repository benefits your research, please consider citing our paper. 742 | 743 | ```bibtex 744 | @inproceedings{xia2023survey, 745 | title={A Survey on Deep Generative 3D-aware Image Synthesis}, 746 | author={Xia, Weihao and Xue, Jing-Hao}, 747 | booktitle={ACM Computing Surveys (CSUR)}, 748 | year={2023} 749 | } 750 | ``` 751 | 752 | License 753 | ------------------ 754 | 755 | Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License. 756 | -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 11 |
12 | 13 |

14 |

15 | A Survey on Deep Generative 3D-aware Image Synthesis 16 |

17 |

18 | ACM Computing Surveys, 2023
19 | Weihao Xia · 20 | Jing-Hao 21 | Xue 22 |

23 |

24 | 25 | arxiv Paper 26 | 27 | 28 | Project Page 29 | 30 | 31 | CSUR Paper 32 | 33 |

34 |

35 | 36 |

Introduction

37 |

This project lists representative papers/codes/datasets about deep 38 | 3D-aware image 39 | synthesis. Besides 3D-aware Generative 40 | Models (GANs and Diffusion Models) discussed in this survey, this project 42 | additionally covers novel view synthesis studies, especially those based 43 | on implicit 45 | neural representations such as NeRF.

46 |

We aim to constantly update the latest relevant papers and help the 47 | community track this topic. Please feel free to join us and contribute 49 | to the project. Please do not hesitate to reach out if you have any 50 | questions or suggestions.

51 |

Survey paper

52 | 57 |

3D Control of 2D GANs

58 |

3D Control Latent Directions

59 |

For 3D control over diffusion models simiar to GAN, 61 | please refer to diffusion 63 | latent editing.

64 | 95 |

3D Parameters as Controls

96 | 134 |

3D Prior Knowledge as 135 | Constraints

136 | 179 |

3D-aware GANs for a 180 | Single Image Category

181 |

Unconditional 3D Generative 182 | Models

183 | 391 |

Conditional 3D Generative 392 | Models

393 | 439 |

3D-aware 440 | Diffusion Models for a Single Image Category

441 | 581 |

3D-Aware Generative 582 | Models on ImageNet

583 | 597 |

3D-aware Video Synthesis

598 | 621 |

INR-based 3D Novel View 622 | Synthesis

623 |

Neural Scene Representations

624 | 701 |

Acceleration

702 | 767 |

From Constrained to 768 | In-the-wild Conditions

769 |

Few Images

770 | 810 |

Pose-free

811 | 837 |

Varying Appearance

838 | 859 |

Large-scale Scene

860 | 903 |

Dynamic Scene

904 | 1006 |

The following papers are not directly related to 3D-aware image 1007 | synthesis. But it would be beneficial to pay attention to those works. 1008 | For example, in our survey, inverse rendering are not classified as 1009 | 3D-aware image synthesis as they are not deliberately designed for this 1010 | purpose. But with the inferred intrinsic components, photorealistic 1011 | images can be rendered. 3D reconstruction models geometry only with no 1012 | appearance information, meaning them not able to render images with 1013 | photorealistic textures. But these representations have been introduced 1014 | as the geometric representation along with a textural representation 1015 | (e.g., Texture Field) for 3D-aware image synthesis.

1016 |

3D Representations

1017 | 1072 |

Neural Inverse 1073 | Rendering (Neural De-rendering)

1074 |

Inverse 1076 | rendering is to infer underlying intrinsic components of a scene 1077 | from rendered 2D images. These properties include shape (surface, depth, 1078 | normal), material (albedo, reflectivity, shininess), and lighting 1079 | (direction, intensity), which can be further used to render 1080 | photorealistic images.

1081 | 1187 |

Neural Rerendering

1188 | 1218 |

Datasets

1219 |

Summary of popular 3D-aware image synthesis datasets.

1220 |

Multi-view image collections

1221 |

The images are rendered or collected according to different 1222 | experimental settings, such as Synthetic-NeRF dataset, the DTU dataset, 1223 | and the Tanks and Temples dataset for general purposes, the crowded 1224 | Phototourism dataset for varying lighting conditions, the Blender 1225 | Forward Facing (BLEFF) dataset to benchmark camera parameter estimation 1226 | and novel view synthesis quality, and the San Francisco Alamo Square 1227 | Dataset for large-scale scenes.

1228 |

Examples of multi-view image datasets.

1229 | 1230 | 1231 | 1232 | 1233 | 1234 | 1235 | 1236 | 1237 | 1238 | 1239 | 1240 | 1241 | 1242 | 1243 | 1244 | 1246 | 1247 | 1248 | 1249 | 1250 | 1251 | 1252 | 1253 | 1254 | 1255 | 1256 | 1257 | 1258 | 1259 | 1260 | 1261 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1271 | 1272 | 1273 | 1274 | 1275 | 1276 | 1277 | 1278 | 1279 | 1280 | 1281 | 1282 | 1283 | 1284 | 1285 | 1286 | 1287 | 1288 | 1289 | 1290 | 1291 | 1292 | 1293 | 1294 | 1295 | 1296 | 1297 | 1298 | 1299 | 1300 | 1301 | 1302 | 1303 | 1304 | 1305 | 1306 | 1307 | 1308 | 1309 | 1310 | 1311 | 1312 | 1313 | 1314 | 1315 | 1316 |
datasetpublished in# scene# samples per 1245 | scenerange (m × m)resolutionkeyword
DeepVoxelsCVPR 20194 simple objects479 / 1,000\512 × 512synthetic, 360 degree
NeRF SyntheticsECCV 20208 complex objects100 / 200\800 ×800synthetic, 360 degree
NeRF CapturedECCV 20208 complex scenes20-62a few1,008 × 756real, forward-facing
DTUCVPR 2014124 scenes49 or 64a few to thousand1,600 × 1,200often used in few-views
Tanks and TemplesCVPR 201514 objects and scenes4,395 - 21,871dozen to thousand8-megapixelreal, large-scale
PhototourismIJCV 20216 landmarks763-2,000dozen to thousand564-1,417 megapixelvarying illumination
Alamo SquareCVPR 2022San Francisco2,818,745570 × 9601,200 × 900real, large-scale
1317 |

Single-view image 1318 | collections

1319 |

Summary of popular single-view image datasets organized by their 1320 | major categories and sorted by their popularity.

1321 | 1322 | 1323 | 1324 | 1325 | 1326 | 1327 | 1328 | 1329 | 1330 | 1331 | 1332 | 1333 | 1334 | 1335 | 1336 | 1337 | 1338 | 1339 | 1340 | 1341 | 1342 | 1343 | 1344 | 1345 | 1346 | 1347 | 1348 | 1349 | 1350 | 1351 | 1352 | 1353 | 1354 | 1355 | 1356 | 1357 | 1358 | 1359 | 1360 | 1361 | 1362 | 1363 | 1364 | 1365 | 1366 | 1367 | 1368 | 1369 | 1370 | 1371 | 1372 | 1373 | 1374 | 1375 | 1376 | 1377 | 1378 | 1379 | 1380 | 1381 | 1382 | 1383 | 1384 | 1385 | 1386 | 1387 | 1388 | 1389 | 1390 | 1391 | 1392 | 1393 | 1394 | 1395 | 1396 | 1397 | 1398 | 1399 | 1400 | 1401 | 1402 | 1403 | 1404 | 1405 | 1406 | 1407 | 1408 | 1409 | 1410 | 1411 | 1412 | 1413 | 1414 | 1415 | 1416 | 1417 | 1418 | 1419 | 1420 | 1421 | 1422 | 1423 | 1424 | 1425 | 1426 | 1427 | 1428 | 1429 | 1430 |
datasetyearcategory# samplesresolutionkeyword
FFHQCVPR 2019Human Face70k1024 × 1024single simple-shape
AFHQCVPR 2020Cat, Dog, and Wildlife15k512 × 512single simple-shape
CompCarsCVPR 2015Real Car136K256 × 256single simple-shape
CARLACoRL 2017Synthetic Car10k128 × 128single simple-shape
CLEVRCVPR 2017Objects100k256 × 256multiple, simple-shape
LSUN2015Bedroom300K256 × 256single, simple-shape
CelebAICCV 2015Human Face200k178 × 218single simple-shape
CelebA-HQICLR 2018Human Face30k1024 × 1024single, simple-shape
MetFacesNeurIPS 2020Art Face13361024 × 1024single, simple-shape
M-PlantsNeurIPS 2022Variable-Shape141,824256 × 256single, variable-shape
M-FoodNeurIPS 2022Variable-Shape25,472256 × 256single, variable-shape
1431 |

Citation

1432 |

If this repository benefits your research, please consider citing our 1433 | paper.

1434 | 
  @inproceedings{xia2023survey,
1436 |     title={A Survey on Deep Generative 3D-aware Image Synthesis},
1437 |     author={Xia, Weihao and Xue, Jing-Hao},
1438 |     booktitle={ACM Computing Surveys (CSUR)},
1439 |     year={2023}
1440 |   }
1441 |

License

1442 |

Creative Commons License
This 1443 | work is licensed under a 1444 | Creative 1445 | Commons Attribution 4.0 International License.

1446 | -------------------------------------------------------------------------------- /jemdoc.css: -------------------------------------------------------------------------------- 1 | /* Default css file for jemdoc. */ 2 | 3 | table#tlayout { 4 | border: none; 5 | border-collapse: separate; 6 | background: white; 7 | } 8 | 9 | body { 10 | background: white; 11 | font-family: Georgia, serif; 12 | padding-bottom: 8px; 13 | margin: 0; 14 | text-align: center; 15 | } 16 | 17 | #layout-menu { 18 | background: #f6f6f6; 19 | border: 1px solid #dddddd; 20 | padding-top: 0.5em; 21 | padding-left: 8px; 22 | padding-right: 8px; 23 | font-size: 1.0em; 24 | width: auto; 25 | white-space: nowrap; 26 | text-align: left; 27 | vertical-align: top; 28 | } 29 | 30 | #layout-menu td { 31 | background: #f4f4f4; 32 | vertical-align: top; 33 | } 34 | 35 | #layout-content { 36 | padding-top: 0.0em; 37 | padding-left: 1.0em; 38 | padding-right: 1.0em; 39 | border: none; 40 | background: white; 41 | text-align: left; 42 | vertical-align: top; 43 | width:900px; 44 | margin: auto; 45 | } 46 | 47 | #layout-menu a { 48 | line-height: 1.5em; 49 | margin-left: 0.5em; 50 | } 51 | 52 | tt { 53 | background: #ffffdd; 54 | } 55 | 56 | pre, tt { 57 | font-size: 90%; 58 | font-family: monaco, monospace; 59 | } 60 | 61 | a, a > tt { 62 | color: #224b8d; 63 | text-decoration: none; 64 | } 65 | 66 | a:hover { 67 | border-bottom: 1px gray dotted; 68 | } 69 | 70 | #layout-menu a.current:link, #layout-menu a.current:visited { 71 | color: #022b6d; 72 | border-bottom: 1px gray solid; 73 | } 74 | #layout-menu a:link, #layout-menu a:visited, #layout-menu a:hover { 75 | color: #527bbd; 76 | text-decoration: none; 77 | } 78 | #layout-menu a:hover { 79 | text-decoration: none; 80 | } 81 | 82 | div.menu-category { 83 | border-bottom: 1px solid gray; 84 | margin-top: 0.8em; 85 | padding-top: 0.2em; 86 | padding-bottom: 0.1em; 87 | font-weight: bold; 88 | } 89 | 90 | div.menu-item { 91 | padding-left: 16px; 92 | text-indent: -16px; 93 | } 94 | 95 | div#toptitle { 96 | padding-bottom: 0.2em; 97 | margin-bottom: 1.5em; 98 | border-bottom: 3px double gray; 99 | } 100 | 101 | /* Reduce space if we begin the page with a title. */ 102 | div#toptitle + h2, div#toptitle + h3 { 103 | margin-top: -0.7em; 104 | } 105 | 106 | div#subtitle { 107 | margin-top: 0.0em; 108 | margin-bottom: 0.0em; 109 | padding-top: 0em; 110 | padding-bottom: 0.1em; 111 | } 112 | 113 | em { 114 | font-style: italic; 115 | } 116 | 117 | strong { 118 | font-weight: bold; 119 | } 120 | 121 | 122 | h1{ 123 | color: #b22222; 124 | margin-top: 0.7em; 125 | margin-bottom: 0.3em; 126 | padding-bottom: 0.2em; 127 | line-height: 1.0; 128 | padding-top: 0.5em; 129 | border-bottom: 1px solid #aaaaaa; 130 | } 131 | 132 | h2{ 133 | color: #b22222; 134 | margin-top: 0.7em; 135 | margin-bottom: 0.3em; 136 | padding-bottom: 0.2em; 137 | line-height: 1.0; 138 | padding-top: 0.5em; 139 | border-bottom: 1px solid #aaaaaa; 140 | } 141 | 142 | h3{ 143 | color: #000000; 144 | margin-top: 0.7em; 145 | margin-bottom: 0.3em; 146 | padding-bottom: 0.2em; 147 | line-height: 1.0; 148 | padding-top: 0.5em; 149 | border-bottom: 1px solid #aaaaaa; 150 | } 151 | 152 | h1 { 153 | font-size: 165%; 154 | } 155 | 156 | h2 { 157 | padding-top: 0.8em; 158 | font-size: 125%; 159 | } 160 | 161 | h2 + h3 { 162 | padding-top: 0.2em; 163 | } 164 | 165 | h3 { 166 | font-size: 110%; 167 | border-bottom: none; 168 | } 169 | 170 | p { 171 | margin-top: 0.0em; 172 | margin-bottom: 0.8em; 173 | padding: 0; 174 | line-height: 1.3; 175 | } 176 | 177 | pre { 178 | padding: 0; 179 | margin: 0; 180 | } 181 | 182 | div#footer { 183 | font-size: small; 184 | border-top: 1px solid #c0c0c0; 185 | padding-top: 0.1em; 186 | margin-top: 4.0em; 187 | 188 | } 189 | 190 | div#footer a { 191 | color: #80a0b0; 192 | } 193 | 194 | div#footer-text { 195 | float: left; 196 | padding-bottom: 8px; 197 | } 198 | 199 | ul, ol, dl { 200 | margin-top: 0.2em; 201 | padding-top: 0; 202 | margin-bottom: 0.8em; 203 | } 204 | 205 | dt { 206 | margin-top: 0.5em; 207 | margin-bottom: 0; 208 | } 209 | 210 | dl { 211 | margin-left: 20px; 212 | } 213 | 214 | dd { 215 | color: #222222; 216 | } 217 | 218 | dd > *:first-child { 219 | margin-top: 0; 220 | } 221 | 222 | ul { 223 | list-style-position: outside; 224 | list-style-type: square; 225 | } 226 | 227 | p + ul, p + ol { 228 | margin-top: -0.5em; 229 | } 230 | 231 | li ul, li ol { 232 | margin-top: -0.3em; 233 | } 234 | 235 | ol { 236 | list-style-position: outside; 237 | list-style-type: decimal; 238 | } 239 | 240 | li p, dd p { 241 | margin-bottom: 0.3em; 242 | } 243 | 244 | 245 | ol ol { 246 | list-style-type: lower-alpha; 247 | } 248 | 249 | ol ol ol { 250 | list-style-type: lower-roman; 251 | } 252 | 253 | p + div.codeblock { 254 | margin-top: -0.6em; 255 | } 256 | 257 | div.codeblock, div.infoblock { 258 | margin-right: 0%; 259 | margin-top: 1.2em; 260 | margin-bottom: 1.3em; 261 | } 262 | 263 | div.blocktitle { 264 | font-weight: bold; 265 | color: #cd7b62; 266 | margin-top: 1.2em; 267 | margin-bottom: 0.1em; 268 | } 269 | 270 | div.blockcontent { 271 | border: 1px solid silver; 272 | padding: 0.3em 0.5em; 273 | } 274 | 275 | div.infoblock > div.blockcontent { 276 | background: #ffffee; 277 | } 278 | 279 | div.blockcontent p + ul, div.blockcontent p + ol { 280 | margin-top: 0.4em; 281 | } 282 | 283 | div.infoblock p { 284 | margin-bottom: 0em; 285 | } 286 | 287 | div.infoblock li p, div.infoblock dd p { 288 | margin-bottom: 0.5em; 289 | } 290 | 291 | div.infoblock p + p { 292 | margin-top: 0.8em; 293 | } 294 | 295 | div.codeblock > div.blockcontent { 296 | background: #f6f6f6; 297 | } 298 | 299 | span.pycommand { 300 | color: #000070; 301 | } 302 | 303 | span.statement { 304 | color: #008800; 305 | } 306 | span.builtin { 307 | color: #000088; 308 | } 309 | span.special { 310 | color: #990000; 311 | } 312 | span.operator { 313 | color: #880000; 314 | } 315 | span.error { 316 | color: #aa0000; 317 | } 318 | span.comment, span.comment > *, span.string, span.string > * { 319 | color: #606060; 320 | } 321 | 322 | @media print { 323 | #layout-menu { display: none; } 324 | } 325 | 326 | #fwtitle { 327 | margin: 2px; 328 | } 329 | 330 | #fwtitle #toptitle { 331 | padding-left: 0.5em; 332 | margin-bottom: 0.5em; 333 | } 334 | 335 | #layout-content h1:first-child, #layout-content h2:first-child, #layout-content h3:first-child { 336 | margin-top: -0.7em; 337 | } 338 | 339 | div#toptitle h1, #layout-content div#toptitle h1 { 340 | margin-bottom: 0.0em; 341 | padding-bottom: 0.1em; 342 | padding-top: 0; 343 | margin-top: 0.5em; 344 | border-bottom: none; 345 | } 346 | 347 | img.eq { 348 | padding: 0; 349 | padding-left: 0.1em; 350 | padding-right: 0.1em; 351 | margin: 0; 352 | } 353 | 354 | img.eqwl { 355 | padding-left: 2em; 356 | padding-top: 0.6em; 357 | padding-bottom: 0.2em; 358 | margin: 0; 359 | } 360 | 361 | table { 362 | border: 2px solid black; 363 | border-collapse: collapse; 364 | } 365 | 366 | td { 367 | padding: 2px; 368 | padding-left: 0.5em; 369 | padding-right: 0.5em; 370 | text-align: center; 371 | border: 1px solid gray; 372 | } 373 | 374 | table + table { 375 | margin-top: 1em; 376 | } 377 | 378 | tr.heading { 379 | font-weight: bold; 380 | border-bottom: 2px solid black; 381 | } 382 | 383 | img { 384 | border: none; 385 | } 386 | 387 | table.imgtable, table.imgtable td { 388 | border: none; 389 | text-align: left; 390 | } 391 | -------------------------------------------------------------------------------- /reference.bib: -------------------------------------------------------------------------------- 1 | @article{xia2022gan, 2 | title={{GAN} Inversion: A Survey}, 3 | author={Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan}, 4 | journal={TPAMI}, 5 | volume={45}, 6 | number={3}, 7 | pages={3121--3138}, 8 | year={2022} 9 | } 10 | 11 | @inproceedings{sauer2022stylegan, 12 | title={{StyleGAN-XL}: Scaling {StyleGAN} to large diverse datasets}, 13 | author={Sauer, Axel and Schwarz, Katja and Geiger, Andreas}, 14 | booktitle={SIGGRAPH}, 15 | pages={1--10}, 16 | year={2022} 17 | } 18 | 19 | @inproceedings{xia2021tedigan, 20 | title={TediGAN: Text-Guided Diverse Face Image Generation and Manipulation}, 21 | author={Xia, Weihao and Yang, Yujiu and Xue, Jing-Hao and Wu, Baoyuan}, 22 | booktitle={CVPR}, 23 | pages = {2256--2265}, 24 | year={2021} 25 | } 26 | 27 | @article{xia2021open, 28 | title={Towards Open-World Text-Guided Face Image Generation and Manipulation}, 29 | author={Xia, Weihao and Yang, Yujiu and Xue, Jing-Hao and Wu, Baoyuan}, 30 | journal={arxiv preprint arxiv: 2104.08910}, 31 | year={2021} 32 | } 33 | 34 | @article{goodfellow2020generative, 35 | title={Generative adversarial networks}, 36 | author={Goodfellow, Ian and Pouget-Abadie, Jean and Mirza, Mehdi and Xu, Bing and Warde-Farley, David and Ozair, Sherjil and Courville, Aaron and Bengio, Yoshua}, 37 | journal={Communications of the ACM}, 38 | volume={63}, 39 | number={11}, 40 | pages={139--144}, 41 | year={2020} 42 | } 43 | 44 | @article{roich2021pivotal, 45 | title={Pivotal Tuning for Latent-based Editing of Real Images}, 46 | author={Roich, Daniel and Mokady, Ron and Bermano, Amit H and Cohen-Or, Daniel}, 47 | journal={TOG}, 48 | volume={42}, 49 | number={1}, 50 | pages={1--13}, 51 | year={2022} 52 | } 53 | 54 | @inproceedings{sitzmann2020implicit, 55 | title={Implicit neural representations with periodic activation functions}, 56 | author={Sitzmann, Vincent and Martel, Julien NP and Bergman, Alexander W and Lindell, David B and Wetzstein, Gordon}, 57 | booktitle={NeurIPS}, 58 | volume={33}, 59 | pages={7462--7473}, 60 | year={2020} 61 | } 62 | 63 | @inproceedings{shen2021closedform, 64 | title = {Closed-Form Factorization of Latent Semantics in {GANs}}, 65 | author = {Shen, Yujun and Zhou, Bolei}, 66 | booktitle = {CVPR}, 67 | pages = {1532--1540}, 68 | year = {2021} 69 | } 70 | 71 | @inproceedings{eric2020GANSpace, 72 | title={{GANSpace}: Discovering Interpretable {GAN} Controls}, 73 | author={Erik, Härkönen and Aaron, Hertzmann and Jaakko, Lehtinen and Sylvain, Paris}, 74 | booktitle={NeurIPS}, 75 | volume={33}, 76 | pages={9841--9850}, 77 | year={2020} 78 | } 79 | 80 | @article{yang20223dhumangan, 81 | title={3DHumanGAN: Towards photo-realistic 3D-aware human image generation}, 82 | author={Yang, Zhuoqian and Li, Shikai and Wu, Wayne and Dai, Bo}, 83 | journal={arXiv preprint arXiv:2212.07378}, 84 | year={2022} 85 | } 86 | 87 | @inproceedings{fu2022stylegan, 88 | title={Stylegan-human: A data-centric odyssey of human generation}, 89 | author={Fu, Jianglin and Li, Shikai and Jiang, Yuming and Lin, Kwan-Yee and Qian, Chen and Loy, Chen Change and Wu, Wayne and Liu, Ziwei}, 90 | booktitle={ECCV}, 91 | pages={1--19}, 92 | year={2022}, 93 | organization={Springer} 94 | } 95 | 96 | @inproceedings{skorokhodov20233d, 97 | title={3D generation on ImageNet}, 98 | author={Ivan Skorokhodov and Aliaksandr Siarohin and Yinghao Xu and Jian Ren and Hsin-Ying Lee and Peter Wonka and Sergey Tulyakov}, 99 | booktitle={ICLR}, 100 | year={2023} 101 | } 102 | 103 | @inproceedings{tewari2022advances, 104 | title={Advances in neural rendering}, 105 | author={Tewari, Ayush and Thies, Justus and Mildenhall, Ben and Srinivasan, Pratul and Tretschk, Edgar and Yifan, Wang and Lassner, Christoph and Sitzmann, Vincent and Martin-Brualla, Ricardo and Lombardi, Stephen and others}, 106 | booktitle={Computer Graphics Forum}, 107 | volume={41}, 108 | number={2}, 109 | pages={703--735}, 110 | year={2022} 111 | } 112 | 113 | @inproceedings{xu2023discoscene, 114 | title={DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis}, 115 | author={Xu, Yinghao and Chai, Menglei and Shi, Zifan and Peng, Sida and Skorokhodov, Ivan and Siarohin, Aliaksandr and Yang, Ceyuan and Shen, Yujun and Lee, Hsin-Ying and Zhou, Bolei and others}, 116 | booktitle={CVPR}, 117 | pages={4402--4412}, 118 | year={2023} 119 | } 120 | 121 | @inproceedings{nurit2021steerability, 122 | title={{GAN} Steerability without optimization}, 123 | author={Nurit, Spingarn and Ron, Banner and Tomer, Michaeli}, 124 | booktitle={ICLR}, 125 | year={2021} 126 | } 127 | 128 | @inproceedings{jahanian2020steerability, 129 | title={On the ''steerability" of generative adversarial networks}, 130 | author={Jahanian, Ali and Chai, Lucy and Isola, Phillip}, 131 | booktitle={ICLR}, 132 | year={2020} 133 | } 134 | 135 | @inproceedings{shen2020interpreting, 136 | title={Interpreting the latent space of {GANs} for semantic face editing}, 137 | author={Shen, Yujun and Gu, Jinjin and Tang, Xiaoou and Zhou, Bolei}, 138 | booktitle={CVPR}, 139 | pages = {9240--9249}, 140 | year={2020} 141 | } 142 | 143 | @article{yu2015lsun, 144 | Author = {Yu, Fisher and Zhang, Yinda and Song, Shuran and Seff, Ari and Xiao, Jianxiong}, 145 | Title = {LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop}, 146 | Journal = {arXiv preprint arXiv:1506.03365}, 147 | Year = {2015} 148 | } 149 | 150 | @inproceedings{dosovitskiy2017carla, 151 | title = { {CARLA}: An Open Urban Driving Simulator}, 152 | author = {Alexey Dosovitskiy and German Ros and Felipe Codevilla and Antonio Lopez and Vladlen Koltun}, 153 | booktitle = {CoRL}, 154 | pages={1--16}, 155 | year = {2017} 156 | } 157 | 158 | @inproceedings{yang2015large, 159 | title={A large-scale car dataset for fine-grained categorization and verification}, 160 | author={Yang, Linjie and Luo, Ping and Change Loy, Chen and Tang, Xiaoou}, 161 | booktitle={CVPR}, 162 | pages={3973--3981}, 163 | year={2015} 164 | } 165 | 166 | @inproceedings{johnson2017clevr, 167 | title={{CLEVR}: A diagnostic dataset for compositional language and elementary visual reasoning}, 168 | author={Johnson, Justin and Hariharan, Bharath and Van Der Maaten, Laurens and Fei-Fei, Li and Lawrence Zitnick, C and Girshick, Ross}, 169 | booktitle={CVPR}, 170 | pages={2901--2910}, 171 | year={2017} 172 | } 173 | 174 | @inproceedings{deng2019arcface, 175 | title={{ArcFace}: Additive angular margin loss for deep face recognition}, 176 | author={Deng, Jiankang and Guo, Jia and Xue, Niannan and Zafeiriou, Stefanos}, 177 | booktitle={CVPR}, 178 | pages = {4690--4699}, 179 | year={2019} 180 | } 181 | 182 | @inproceedings{zhang2022styleswin, 183 | title={StyleSwin: Transformer-based GAN for High-resolution Image Generation}, 184 | author={Bowen Zhang and Shuyang Gu and Bo Zhang and Jianmin Bao and Dong Chen and Fang Wen and Yong Wang and Baining Guo}, 185 | booktitle={CVPR}, 186 | pages={11304--11314}, 187 | year={2022} 188 | } 189 | 190 | @inproceedings{voynov2020latent, 191 | title = {Unsupervised Discovery of Interpretable Directions in the {GAN} Latent Space}, 192 | author = {Andrey Voynov and Artem Babenko}, 193 | booktitle = {ICML}, 194 | year = {2020}, 195 | pages = {9786--9796}, 196 | volume = {119}, 197 | series = {Proceedings of Machine Learning Research}, 198 | } 199 | 200 | @article{binkowski2018demystifying, 201 | title={Demystifying mmd gans}, 202 | author={Bi{\'n}kowski, Miko{\l}aj and Sutherland, Danica J and Arbel, Michael and Gretton, Arthur}, 203 | journal={arXiv preprint arXiv:1801.01401}, 204 | year={2018} 205 | } 206 | 207 | @inproceedings{zhang2018unreasonable, 208 | title={The unreasonable effectiveness of deep features as a perceptual metric}, 209 | author={Zhang, Richard and Isola, Phillip and Efros, Alexei A and Shechtman, Eli and Wang, Oliver}, 210 | booktitle={CVPR}, 211 | pages={586--595}, 212 | year={2018} 213 | } 214 | 215 | @inproceedings{szegedy2016rethinking, 216 | title={Rethinking the inception architecture for computer vision}, 217 | author={Szegedy, Christian and Vanhoucke, Vincent and Ioffe, Sergey and Shlens, Jon and Wojna, Zbigniew}, 218 | booktitle={CVPR}, 219 | pages={2818--2826}, 220 | year={2016} 221 | } 222 | 223 | @article{TIP2004ImageWang, 224 | title={Image quality assessment: From error visibility to structural similarity}, 225 | author={Wang, Zhou and Bovik, Alan C and Sheikh, Hamid R and Simoncelli, Eero P}, 226 | journal={TIP}, 227 | volume={13}, 228 | number={4}, 229 | pages={600--612}, 230 | year={2004} 231 | } 232 | 233 | @inproceedings{heusel2017gans, 234 | title={{GANs} trained by a two time-scale update rule converge to a local {Nash} equilibrium}, 235 | author={Heusel, Martin and Ramsauer, Hubert and Unterthiner, Thomas and Nessler, Bernhard and Hochreiter, Sepp}, 236 | booktitle={NeurIPS}, 237 | volume={30}, 238 | pages = {6626--6637}, 239 | year={2017} 240 | } 241 | 242 | @inproceedings{salimans2016improved, 243 | title={Improved techniques for training {GANs}}, 244 | author={Salimans, Tim and Goodfellow, Ian and Zaremba, Wojciech and Cheung, Vicki and Radford, Alec and Chen, Xi}, 245 | booktitle={NeurIPS}, 246 | pages = {2226--2234}, 247 | year={2016} 248 | } 249 | 250 | @inproceedings{deng2009imagenet, 251 | author = {Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li}, 252 | title = {Imagenet: A large-scale hierarchical image database}, 253 | booktitle = {CVPR}, 254 | pages = {248--255}, 255 | year = {2009} 256 | } 257 | 258 | @inproceedings{mokady2022self, 259 | title={Self-distilled {StyleGAN}: Towards generation from internet photos}, 260 | author={Mokady, Ron and Tov, Omer and Yarom, Michal and Lang, Oran and Mosseri, Inbar and Dekel, Tali and Cohen-Or, Daniel and Irani, Michal}, 261 | booktitle={SIGGRAPH}, 262 | pages={1--9}, 263 | year={2022} 264 | } 265 | 266 | @inproceedings{simonyan2014very, 267 | title={Very deep convolutional networks for large-scale image recognition}, 268 | author={Simonyan, Karen and Zisserman, Andrew}, 269 | booktitle={ICLR}, 270 | year={2015} 271 | } 272 | 273 | @inproceedings{liu2016deepfashion, 274 | title={Deepfashion: Powering robust clothes recognition and retrieval with rich annotations}, 275 | author={Liu, Ziwei and Luo, Ping and Qiu, Shi and Wang, Xiaogang and Tang, Xiaoou}, 276 | booktitle={CVPR}, 277 | pages={1096--1104}, 278 | year={2016} 279 | } 280 | 281 | @inproceedings{liu2015faceattributes, 282 | title = {Deep Learning Face Attributes in the Wild}, 283 | author = {Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou}, 284 | booktitle = {ICCV}, 285 | pages = {3730--3738}, 286 | year = {2015} 287 | } 288 | 289 | @inproceedings{hong2022eva3d, 290 | title={EVA3D: Compositional 3D Human Generation from 2D Image Collections}, 291 | author={Hong, Fangzhou and Chen, Zhaoxi and Lan, Yushi and Pan, Liang and Liu, Ziwei}, 292 | booktitle={ICLR}, 293 | year={2023} 294 | } 295 | 296 | @inproceedings{reizenstein2021common, 297 | title={Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction}, 298 | author={Reizenstein, Jeremy and Shapovalov, Roman and Henzler, Philipp and Sbordone, Luca and Labatut, Patrick and Novotny, David}, 299 | booktitle={ICCV}, 300 | pages={10901--10911}, 301 | year={2021} 302 | } 303 | 304 | @inproceedings{sun2023next3d, 305 | title={Next3d: Generative neural texture rasterization for 3d-aware head avatars}, 306 | author={Sun, Jingxiang and Wang, Xuan and Wang, Lizhen and Li, Xiaoyu and Zhang, Yong and Zhang, Hongwen and Liu, Yebin}, 307 | booktitle={CVPR}, 308 | pages={20991--21002}, 309 | year={2023} 310 | } 311 | 312 | @article{bergman2022generative, 313 | title={Generative neural articulated radiance fields}, 314 | author={Bergman, Alexander and Kellnhofer, Petr and Yifan, Wang and Chan, Eric and Lindell, David and Wetzstein, Gordon}, 315 | journal={NeurIPS}, 316 | volume={35}, 317 | pages={19900--19916}, 318 | year={2022} 319 | } 320 | 321 | @article{sargent2023vq3d, 322 | title={Vq3d: Learning a 3d-aware generative model on imagenet}, 323 | author={Sargent, Kyle and Koh, Jing Yu and Zhang, Han and Chang, Huiwen and Herrmann, Charles and Srinivasan, Pratul and Wu, Jiajun and Sun, Deqing}, 324 | journal={arXiv preprint arXiv:2302.06833}, 325 | year={2023} 326 | } 327 | 328 | @inproceedings{karras2017progressive, 329 | title={Progressive growing of {GANs} for improved quality, stability, and variation}, 330 | author={Karras, Tero and Aila, Timo and Laine, Samuli and Lehtinen, Jaakko}, 331 | booktitle={ICLR}, 332 | year={2018} 333 | } 334 | 335 | @InProceedings{karras2019style, 336 | author = {Karras, Tero and Laine, Samuli and Aila, Timo}, 337 | title = {A style-based generator architecture for generative adversarial networks}, 338 | booktitle = {CVPR}, 339 | pages = {4401--4410}, 340 | year = {2019} 341 | } 342 | 343 | @inproceedings{karras2020analyzing, 344 | title={Analyzing and improving the image quality of {StyleGAN}}, 345 | author={Karras, Tero and Laine, Samuli and Aittala, Miika and Hellsten, Janne and Lehtinen, Jaakko and Aila, Timo}, 346 | booktitle={CVPR}, 347 | pages = {8107--8116}, 348 | year={2020} 349 | } 350 | 351 | @article{ma2023adding, 352 | title={Adding 3D Geometry Control to Diffusion Models}, 353 | author={Ma, Wufei and Liu, Qihao and Wang, Jiahao and Wang, Angtian and Liu, Yaoyao and Kortylewski, Adam and Yuille, Alan}, 354 | journal={arXiv preprint arXiv:2306.08103}, 355 | year={2023} 356 | } 357 | 358 | @inproceedings{karras2020ada, 359 | title = {Training Generative Adversarial Networks with Limited Data}, 360 | author = {Tero Karras and Miika Aittala and Janne Hellsten and Samuli Laine and Jaakko Lehtinen and Timo Aila}, 361 | booktitle = {NeurIPS}, 362 | volume={33}, 363 | pages={12104--12114}, 364 | year={2020} 365 | } 366 | 367 | @inproceedings{Karras2021free, 368 | author = {Tero Karras and Miika Aittala and Samuli Laine and Erik H\"{a}rk\"{o}nen and Janne Hellsten and Jaakko Lehtinen and Timo Aila}, 369 | title = {Alias-Free Generative Adversarial Networks}, 370 | booktitle = {NeurIPS}, 371 | pages = {852--863}, 372 | year = {2021} 373 | } 374 | 375 | @inproceedings{kowalski2020config, 376 | title={Config: Controllable neural face image generation}, 377 | author={Kowalski, Marek and Garbin, Stephan J and Estellers, Virginia and Baltru{\v{s}}aitis, Tadas and Johnson, Matthew and Shotton, Jamie}, 378 | booktitle={ECCV}, 379 | pages={299--315}, 380 | year={2020} 381 | } 382 | 383 | @inproceedings{shoshan2021gan, 384 | title={{GAN}-control: Explicitly controllable {GANs}}, 385 | author={Shoshan, Alon and Bhonker, Nadav and Kviatkovsky, Igor and Medioni, Gerard}, 386 | booktitle={ICCV}, 387 | pages={14083--14093}, 388 | year={2021} 389 | } 390 | 391 | @article{abdal2020styleflow, 392 | title={{StyleFlow}: Attribute-conditioned Exploration of {StyleGAN}-Generated Images using Conditional Continuous Normalizing Flows}, 393 | author={Abdal, Rameen and Zhu, Peihao and Mitra, Niloy and Wonka, Peter}, 394 | journal={TOG}, 395 | volume={40}, 396 | number={3}, 397 | pages={1--21}, 398 | year={2021} 399 | } 400 | 401 | @inproceedings{deng2020disentangled, 402 | title={Disentangled and controllable face image generation via {3D} imitative-contrastive learning}, 403 | author={Deng, Yu and Yang, Jiaolong and Chen, Dong and Wen, Fang and Tong, Xin}, 404 | booktitle={CVPR}, 405 | pages={5154--5163}, 406 | year={2020} 407 | } 408 | 409 | @inproceedings{tewari2020stylerig, 410 | title = {{StyleRig}: Rigging {StyleGAN} for {3D} Control over Portrait Images}, 411 | author={Tewari, Ayush and Elgharib, Mohamed and Bharaj, Gaurav and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Z{\"o}llhofer, Michael and Theobalt, Christian}, 412 | booktitle = {CVPR}, 413 | pages = {6141--6150}, 414 | year = {2020}, 415 | } 416 | 417 | @article{tewari2020pie, 418 | title = {PIE: Portrait Image Embedding for Semantic Control}, 419 | author={Tewari, Ayush and Elgharib, Mohamed and BR, Mallikarjun and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Z{\"o}llhofer, Michael and Theobalt, Christian}, 420 | journal = {TOG}, 421 | volume={39}, 422 | number={6}, 423 | pages={1--14}, 424 | year = {2020} 425 | } 426 | 427 | @inproceedings{ho2020denoising, 428 | title={Denoising diffusion probabilistic models}, 429 | author={Ho, Jonathan and Jain, Ajay and Abbeel, Pieter}, 430 | booktitle={NeurIPS}, 431 | volume={33}, 432 | pages={6840--6851}, 433 | year={2020} 434 | } 435 | 436 | @inproceedings{song2019generative, 437 | title={Generative modeling by estimating gradients of the data distribution}, 438 | author={Song, Yang and Ermon, Stefano}, 439 | booktitle={NeurIPS}, 440 | volume={32}, 441 | pages = {11895--11907}, 442 | year={2019} 443 | } 444 | 445 | @inproceedings{song2020denoising, 446 | title={Denoising diffusion implicit models}, 447 | author={Song, Jiaming and Meng, Chenlin and Ermon, Stefano}, 448 | booktitle={ICLR}, 449 | year={2021} 450 | } 451 | 452 | @inproceedings{song2020score, 453 | title={Score-based generative modeling through stochastic differential equations}, 454 | author={Song, Yang and Sohl-Dickstein, Jascha and Kingma, Diederik P and Kumar, Abhishek and Ermon, Stefano and Poole, Ben}, 455 | booktitle={ICLR}, 456 | year={2020} 457 | } 458 | 459 | @article{zhang20233dshape2vecset, 460 | title={3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models}, 461 | author={Zhang, Biao and Tang, Jiapeng and Niessner, Matthias and Wonka, Peter}, 462 | journal={arXiv preprint arXiv:2301.11445}, 463 | year={2023} 464 | } 465 | 466 | @inproceedings{shi2023learning, 467 | title={Learning 3d-aware image synthesis with unknown pose distribution}, 468 | author={Shi, Zifan and Shen, Yujun and Xu, Yinghao and Peng, Sida and Liao, Yiyi and Guo, Sheng and Chen, Qifeng and Yeung, Dit-Yan}, 469 | booktitle={CVPR}, 470 | pages={13062--13071}, 471 | year={2023} 472 | } 473 | 474 | @inproceedings{kim2023neuralfield, 475 | title={NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models}, 476 | author={Kim, Seung Wook and Brown, Bradley and Yin, Kangxue and Kreis, Karsten and Schwarz, Katja and Li, Daiqing and Rombach, Robin and Torralba, Antonio and Fidler, Sanja}, 477 | booktitle={CVPR}, 478 | pages={8496--8506}, 479 | year={2023} 480 | } 481 | 482 | @inproceedings{muller2023diffrf, 483 | title={Diffrf: Rendering-guided 3d radiance field diffusion}, 484 | author={M{\"u}ller, Norman and Siddiqui, Yawar and Porzi, Lorenzo and Bulo, Samuel Rota and Kontschieder, Peter and Nie{\ss}ner, Matthias}, 485 | booktitle={CVPR}, 486 | pages={4328--4338}, 487 | year={2023} 488 | } 489 | 490 | @inproceedings{karnewar2023holodiffusion, 491 | title={Holodiffusion: Training a 3D diffusion model using 2D images}, 492 | author={Karnewar, Animesh and Vedaldi, Andrea and Novotny, David and Mitra, Niloy J}, 493 | booktitle={CVPR}, 494 | pages={18423--18433}, 495 | year={2023} 496 | } 497 | 498 | @inproceedings{ko20233d, 499 | title={3d gan inversion with pose optimization}, 500 | author={Ko, Jaehoon and Cho, Kyusun and Choi, Daewon and Ryoo, Kwangrok and Kim, Seungryong}, 501 | booktitle={WACV}, 502 | pages={2967--2976}, 503 | year={2023} 504 | } 505 | 506 | @inproceedings{lan2023self, 507 | title={Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion}, 508 | author={Lan, Yushi and Meng, Xuyi and Yang, Shuai and Loy, Chen Change and Dai, Bo}, 509 | booktitle={CVPR}, 510 | pages={20940--20949}, 511 | year={2023} 512 | } 513 | 514 | @inproceedings{xie2023high, 515 | title={High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization}, 516 | author={Xie, Jiaxin and Ouyang, Hao and Piao, Jingtan and Lei, Chenyang and Chen, Qifeng}, 517 | booktitle={CVPR}, 518 | pages={321--331}, 519 | year={2023} 520 | } 521 | 522 | @inproceedings{kwon2022diffusion, 523 | title={Diffusion models already have a semantic latent space}, 524 | author={Kwon, Mingi and Jeong, Jaeseok and Uh, Youngjung}, 525 | booktitle={ICLR}, 526 | year={2023} 527 | } 528 | 529 | @article{wang2022rewriting, 530 | author = {Wang, Sheng-Yu and Bau, David and Zhu, Jun-Yan}, 531 | title = {Rewriting Geometric Rules of a GAN}, 532 | year = {2022}, 533 | journal = {TOG}, 534 | volume={41}, 535 | number={4}, 536 | pages={1--16}, 537 | } 538 | 539 | @inproceedings{endo2022user, 540 | title={User-Controllable Latent Transformer for StyleGAN Image Layout Editing}, 541 | author={Endo, Yuki}, 542 | booktitle={Computer Graphics Forum}, 543 | volume={41}, 544 | number={7}, 545 | pages={395--406}, 546 | year={2022} 547 | } 548 | 549 | @inproceedings{wang2022improving, 550 | title={Improving gan equilibrium by raising spatial awareness}, 551 | author={Wang, Jianyuan and Yang, Ceyuan and Xu, Yinghao and Shen, Yujun and Li, Hongdong and Zhou, Bolei}, 552 | booktitle={CVPR}, 553 | pages={11285--11293}, 554 | year={2022} 555 | } 556 | 557 | @inproceedings{epstein2022blobgan, 558 | title={Blobgan: Spatially disentangled scene representations}, 559 | author={Epstein, Dave and Park, Taesung and Zhang, Richard and Shechtman, Eli and Efros, Alexei A}, 560 | booktitle={ECCV}, 561 | pages={616--635}, 562 | year={2022} 563 | } 564 | 565 | @inproceedings{pan2023drag, 566 | title={Drag Your {GAN}: Interactive Point-based Manipulation on the Generative Image Manifold}, 567 | author={Pan, Xingang and Tewari, Ayush and Leimk{\"u}hler, Thomas and Liu, Lingjie and Meka, Abhimitra and Theobalt, Christian}, 568 | booktitle = {SIGGRAPH}, 569 | year={2023} 570 | } 571 | 572 | @article{shi2023dragdiffusion, 573 | title={DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing}, 574 | author={Shi, Yujun and Xue, Chuhui and Pan, Jiachun and Zhang, Wenqing and Tan, Vincent YF and Bai, Song}, 575 | journal={arXiv preprint arXiv:2306.14435}, 576 | year={2023} 577 | } 578 | 579 | @article{mou2023dragondiffusion, 580 | title={DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models}, 581 | author={Mou, Chong and Wang, Xintao and Song, Jiechong and Shan, Ying and Zhang, Jian}, 582 | journal={arXiv preprint arXiv:2307.02421}, 583 | year={2023} 584 | } 585 | 586 | @article{brack2022stable, 587 | title={The stable artist: Steering semantics in diffusion latent space}, 588 | author={Brack, Manuel and Schramowski, Patrick and Friedrich, Felix and Hintersdorf, Dominik and Kersting, Kristian}, 589 | journal={arXiv preprint arXiv:2212.06013}, 590 | year={2022} 591 | } 592 | 593 | @article{haas2023discovering, 594 | title={Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models}, 595 | author={Haas, Ren{\'e} and Huberman-Spiegelglas, Inbar and Mulayoff, Rotem and Michaeli, Tomer}, 596 | journal={arXiv preprint arXiv:2303.11073}, 597 | year={2023} 598 | } 599 | 600 | @article{chan2023generative, 601 | title={Generative novel view synthesis with 3d-aware diffusion models}, 602 | author={Chan, Eric R and Nagano, Koki and Chan, Matthew A and Bergman, Alexander W and Park, Jeong Joon and Levy, Axel and Aittala, Miika and De Mello, Shalini and Karras, Tero and Wetzstein, Gordon}, 603 | journal={arXiv preprint arXiv:2304.02602}, 604 | year={2023} 605 | } 606 | 607 | @article{xiang20233d, 608 | title={3D-aware Image Generation using 2D Diffusion Models}, 609 | author={Xiang, Jianfeng and Yang, Jiaolong and Huang, Binbin and Tong, Xin}, 610 | journal={arXiv preprint arXiv:2303.17905}, 611 | year={2023} 612 | } 613 | 614 | @inproceedings{zhu2020domain, 615 | title={In-domain gan inversion for real image editing}, 616 | author={Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei}, 617 | booktitle={ECCV}, 618 | pages={592--608}, 619 | year={2020}, 620 | organization={Springer} 621 | } 622 | 623 | @inproceedings{anciukevivcius2023renderdiffusion, 624 | title={Renderdiffusion: Image diffusion for 3d reconstruction, inpainting and generation}, 625 | author={Anciukevi{\v{c}}ius, Titas and Xu, Zexiang and Fisher, Matthew and Henderson, Paul and Bilen, Hakan and Mitra, Niloy J and Guerrero, Paul}, 626 | booktitle={CVPR}, 627 | pages={12608--12618}, 628 | year={2023} 629 | } 630 | 631 | @inproceedings{dhariwal2021diffusion, 632 | title={Diffusion models beat gans on image synthesis}, 633 | author={Dhariwal, Prafulla and Nichol, Alexander}, 634 | booktitle={NeurIPS}, 635 | volume={34}, 636 | pages={8780--8794}, 637 | year={2021} 638 | } 639 | 640 | @inproceedings{rombach2022high, 641 | title={High-resolution image synthesis with latent diffusion models}, 642 | author={Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj{\"o}rn}, 643 | booktitle={CVPR}, 644 | pages={10684--10695}, 645 | year={2022} 646 | } 647 | 648 | @article{bojanowski2017optimizing, 649 | title={Optimizing the latent space of generative networks}, 650 | author={Bojanowski, Piotr and Joulin, Armand and Lopez-Paz, David and Szlam, Arthur}, 651 | journal={arXiv preprint arXiv:1707.05776}, 652 | year={2017} 653 | } 654 | 655 | @inproceedings{bojanowski2017optimizing, 656 | author={Bojanowski, Piotr and Joulin, Armand and Lopez-Paz, David and Szlam, Arthur}, 657 | title = {Optimizing the Latent Space of Generative Networks}, 658 | booktitle = {ICML}, 659 | series = {Proceedings of Machine Learning Research}, 660 | pages = {599--608}, 661 | volume = {80}, 662 | year = {2018} 663 | } 664 | 665 | @inproceedings{shi2022improving, 666 | title={Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator}, 667 | author={Shi, Zifan and Xu, Yinghao and Shen, Yujun and Zhao, Deli and Chen, Qifeng and Yeung, Dit-Yan}, 668 | booktitle={NeurIPS}, 669 | volume={35}, 670 | pages={7921--7932}, 671 | year={2022} 672 | } 673 | 674 | @inproceedings{zhang20213d, 675 | title={3D-Aware Semantic-Guided Generative Model for Human Synthesis}, 676 | author={Zhang, Jichao and Sangineto, Enver and Tang, Hao and Siarohin, Aliaksandr and Zhong, Zhun and Sebe, Nicu and Wang, Wei}, 677 | booktitle={ECCV}, 678 | pages={339--356}, 679 | year={2022} 680 | } 681 | 682 | @inproceedings{skorokhodov2022epigraf, 683 | title={{EpiGRAF}: Rethinking training of {3D} GANs}, 684 | author={Skorokhodov, Ivan and Tulyakov, Sergey and Wang, Yiqun and Wonka, Peter}, 685 | booktitle={NeurIPS}, 686 | volume={35}, 687 | pages={24487--24501}, 688 | year={2022} 689 | } 690 | 691 | @inproceedings{chen2022sem2nerf, 692 | title={{Sem2NeRF}: Converting Single-View Semantic Masks to Neural Radiance Fields}, 693 | author={Chen, Yuedong and Wu, Qianyi and Zheng, Chuanxia and Cham, Tat-Jen and Cai, Jianfei}, 694 | booktitle={ECCV}, 695 | pages={730--748}, 696 | year={2022} 697 | } 698 | 699 | @inproceedings{zhang2022multi, 700 | title={Multi-View Consistent Generative Adversarial Networks for {3D}-aware Image Synthesis}, 701 | author={Zhang, Xuanmeng and Zheng, Zhedong and Gao, Daiheng and Zhang, Bang and Pan, Pan and Yang, Yi}, 702 | booktitle={CVPR}, 703 | pages={18450--18459}, 704 | year={2022} 705 | } 706 | 707 | @inproceedings{cai2022pix2nerf, 708 | title={{Pix2NeRF}: Unsupervised Conditional pi-{GAN} for Single Image to Neural Radiance Fields Translation}, 709 | author={Cai, Shengqu and Obukhov, Anton and Dai, Dengxin and Van Gool, Luc}, 710 | booktitle={CVPR}, 711 | pages={3981--3990}, 712 | year={2022} 713 | } 714 | 715 | @inproceedings{lin20223d, 716 | title={{3D} {GAN} Inversion for Controllable Portrait Image Animation}, 717 | author={Lin, Connor Z and Lindell, David B and Chan, Eric R and Wetzstein, Gordon}, 718 | booktitle={ECCV Workshop}, 719 | year={2022} 720 | } 721 | 722 | @inproceedings{liao2020towards, 723 | title={Towards unsupervised learning of generative models for {3D} controllable image synthesis}, 724 | author={Liao, Yiyi and Schwarz, Katja and Mescheder, Lars and Geiger, Andreas}, 725 | booktitle={CVPR}, 726 | pages={5871--5880}, 727 | year={2020} 728 | } 729 | 730 | @inproceedings{gadelha20173d, 731 | title={3d shape induction from 2d views of multiple objects}, 732 | author={Gadelha, Matheus and Maji, Subhransu and Wang, Rui}, 733 | booktitle={3DV}, 734 | pages={402--411}, 735 | year={2017} 736 | } 737 | 738 | @article{kajiya1984ray, 739 | title={Ray tracing volume densities}, 740 | author={Kajiya, James T and Von Herzen, Brian P}, 741 | journal={SIGGRAPH}, 742 | volume={18}, 743 | number={3}, 744 | pages={165--174}, 745 | year={1984}, 746 | 747 | } 748 | 749 | @article{lorensen1987marching, 750 | title={Marching cubes: A high resolution 3D surface construction algorithm}, 751 | author={Lorensen, William E and Cline, Harvey E}, 752 | journal={SIGGRAPH}, 753 | pages={347--353}, 754 | year={1987}, 755 | 756 | } 757 | 758 | @inproceedings{tewari2017mofa, 759 | title={{MoFA}: Model-based deep convolutional face autoencoder for unsupervised monocular reconstruction}, 760 | author={Tewari, Ayush and Zollhofer, Michael and Kim, Hyeongwoo and Garrido, Pablo and Bernard, Florian and Perez, Patrick and Theobalt, Christian}, 761 | booktitle={ICCV workshops}, 762 | pages={1274--1283}, 763 | year={2017} 764 | } 765 | 766 | @inproceedings{ruiz2018fine, 767 | title={Fine-grained head pose estimation without keypoints}, 768 | author={Ruiz, Nataniel and Chong, Eunji and Rehg, James M}, 769 | booktitle={CVPR workshops}, 770 | year={2018} 771 | } 772 | 773 | @inproceedings{sun2022controllable, 774 | title={Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields}, 775 | author={Sun, Keqiang and Wu, Shangzhe and Huang, Zhaoyang and Zhang, Ning and Wang, Quan and Li, HongSheng}, 776 | booktitle={NeurIPS}, 777 | volume={35}, 778 | pages={16331--16343}, 779 | year={2022} 780 | } 781 | 782 | @inproceedings{henzler2019escaping, 783 | title = {{Escaping Plato's cave: 3D shape from adversarial rendering}}, 784 | author = {Philipp Henzler and Niloy~J. Mitra and Tobias Ritschel}, 785 | booktitle = {ICCV}, 786 | year = {2019}, 787 | pages = {9983--9992} 788 | } 789 | 790 | @inproceedings{pan2021shadegan, 791 | title={A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis}, 792 | author={Xingang Pan and Xudong Xu and Chen Change Loy and Christian Theobalt and Bo Dai}, 793 | booktitle={NeurIPS}, 794 | pages = {20002--20013}, 795 | year={2021}, 796 | } 797 | 798 | @article{lombardi2019neural, 799 | title={Neural volumes: Learning dynamic renderable volumes from images}, 800 | author={Lombardi, Stephen and Simon, Tomas and Saragih, Jason and Schwartz, Gabriel and Lehrmann, Andreas and Sheikh, Yaser}, 801 | journal={TOG}, 802 | volume = {38}, 803 | number = {4}, 804 | articleno = {65}, 805 | numpages = {14}, 806 | year={2019} 807 | } 808 | 809 | @inproceedings{pan2022gan2x, 810 | title={{GAN2X}: Non-Lambertian Inverse Rendering of Image {GANs}}, 811 | author={Pan, Xingang and Tewari, Ayush and Liu, Lingjie and Theobalt, Christian}, 812 | booktitle={3DV}, 813 | pages={711--721}, 814 | year={2022} 815 | } 816 | 817 | @inproceedings{pan2020gan2shape, 818 | title = {Do {2D} {GANs} Know {3D} Shape? Unsupervised {3D} Shape Reconstruction from {2D} Image {GANs}}, 819 | author = {Pan, Xingang and Dai, Bo and Liu, Ziwei and Loy, Chen Change and Luo, Ping}, 820 | booktitle = {ICLR}, 821 | year = {2021} 822 | } 823 | 824 | @inproceedings{schwarz2022voxgraf, 825 | title={{VoxGRAF}: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids}, 826 | author={Schwarz, Katja and Sauer, Axel and Niemeyer, Michael and Liao, Yiyi and Geiger, Andreas}, 827 | booktitle={NeurIPS}, 828 | volume={35}, 829 | pages={33999--34011}, 830 | year={2022} 831 | } 832 | 833 | @inproceedings{liu2020neural, 834 | title={Neural sparse Voxel Fields}, 835 | author={Liu, Lingjie and Gu, Jiatao and Lin, Kyaw Zaw and Chua, Tat-Seng and Theobalt, Christian}, 836 | booktitle={NeurIPS}, 837 | volume={33}, 838 | pages={15651--15663}, 839 | year={2020} 840 | } 841 | 842 | @inproceedings{nguyen2019hologan, 843 | title={{HoloGAN}: Unsupervised learning of {3D} representations from natural images}, 844 | author={Nguyen-Phuoc, Thu and Li, Chuan and Theis, Lucas and Richardt, Christian and Yang, Yong-Liang}, 845 | booktitle={ICCV}, 846 | pages = {7587--7596}, 847 | year={2019} 848 | } 849 | 850 | @inproceedings{nguyen2020blockgan, 851 | title={{BlockGAN}: Learning {3D} Object-aware Scene Representations from Unlabelled Images}, 852 | author={Nguyen-Phuoc, Thu and Richardt, Christian and Mai, Long and Yang, Yong-Liang and Mitra, Niloy}, 853 | booktitle={NeurIPS}, 854 | volume={33}, 855 | pages={6767--6778}, 856 | year={2020} 857 | } 858 | 859 | @article{zhu2018visual, 860 | title={Visual object networks: Image generation with disentangled 3D representations}, 861 | author={Zhu, Jun-Yan and Zhang, Zhoutong and Zhang, Chengkai and Wu, Jiajun and Torralba, Antonio and Tenenbaum, Josh and Freeman, Bill}, 862 | journal={NeurIPS}, 863 | pages = {118--129}, 864 | year={2018} 865 | } 866 | 867 | @article{wu2016learning, 868 | title={Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling}, 869 | author={Wu, Jiajun and Zhang, Chengkai and Xue, Tianfan and Freeman, Bill and Tenenbaum, Josh}, 870 | journal={NeurIPS}, 871 | volume={29}, 872 | pages = {82--90}, 873 | year={2016} 874 | } 875 | 876 | @inproceedings{wang2016generative, 877 | title={Generative image modeling using style and structure adversarial networks}, 878 | author={Wang, Xiaolong and Gupta, Abhinav}, 879 | booktitle={ECCV}, 880 | pages={318--335}, 881 | year={2016} 882 | } 883 | 884 | @inproceedings{chen2021towards, 885 | title={Towards a neural graphics pipeline for controllable image generation}, 886 | author={Chen, Xuelin and Cohen-Or, Daniel and Chen, Baoquan and Mitra, Niloy J}, 887 | booktitle={CGF}, 888 | volume={40}, 889 | number={2}, 890 | pages={127--140}, 891 | year={2021} 892 | } 893 | 894 | @inproceedings{abu2018geometric, 895 | title={Geometric image synthesis}, 896 | author={Abu Alhaija, Hassan and Mustikovela, Siva Karthik and Geiger, Andreas and Rother, Carsten}, 897 | booktitle={ACCV}, 898 | pages={85--100}, 899 | year={2018} 900 | } 901 | 902 | @inproceedings{noguchi2020rgbd, 903 | title={{RGBD-GAN}: Unsupervised 3d representation learning from natural image datasets via rgbd image synthesis}, 904 | author={Noguchi, Atsuhiro and Harada, Tatsuya}, 905 | booktitle={ICLR}, 906 | year={2020} 907 | } 908 | 909 | @inproceedings{sun2022ide3d, 910 | title={{IDE-3D}: Interactive Disentangled Editing for High-Resolution {3D}-aware Portrait Synthesis}, 911 | author={Sun, Jingxiang and Wang, Xuan and Shi, Yichun and Wang, Lizhen and Wang, Jue and Liu, Yebin}, 912 | booktitle={SIGGRAPH Asia}, 913 | year={2022} 914 | } 915 | 916 | @inproceedings{shi2021lifting, 917 | title={Lifting 2d stylegan for 3d-aware face generation}, 918 | author={Shi, Yichun and Aggarwal, Divyansh and Jain, Anil K}, 919 | booktitle={CVPR}, 920 | pages={6258--6266}, 921 | year={2021} 922 | } 923 | 924 | @inproceedings{shi20223d, 925 | title={3D-Aware Indoor Scene Synthesis with Depth Priors}, 926 | author={Shi, Zifan and Shen, Yujun and Zhu, Jiapeng and Yeung, Dit-Yan and Chen, Qifeng}, 927 | booktitle={ECCV}, 928 | pages={406--422}, 929 | year={2022} 930 | } 931 | 932 | @inproceedings{niemeyer2020differentiable, 933 | title={Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision}, 934 | author={Niemeyer, Michael and Mescheder, Lars and Oechsle, Michael and Geiger, Andreas}, 935 | booktitle={CVPR}, 936 | pages = {3501--3512}, 937 | year={2020} 938 | } 939 | 940 | @inproceedings{park2019deepsdf, 941 | title={{DeepSDF}: Learning continuous signed distance functions for shape representation}, 942 | author={Park, Jeong Joon and Florence, Peter and Straub, Julian and Newcombe, Richard and Lovegrove, Steven}, 943 | booktitle={CVPR}, 944 | pages={165--174}, 945 | year={2019} 946 | } 947 | 948 | @inproceedings{sun2021fenerf, 949 | title={{FENeRF}: Face Editing in Neural Radiance Fields}, 950 | author={Sun, Jingxiang and Wang, Xuan and Zhang, Yong and Li, Xiaoyu and Zhang, Qi and Liu, Yebin and Wang, Jue}, 951 | booktitle = {CVPR}, 952 | pages={7672--7682}, 953 | year={2022} 954 | } 955 | 956 | @inproceedings{chan2021pigan, 957 | author = {Eric~R. Chan and Marco Monteiro and Petr Kellnhofer and Jiajun Wu and Gordon Wetzstein}, 958 | title = {{pi-GAN: Periodic implicit generative adversarial networks for 3D-aware image synthesis}}, 959 | booktitle = {CVPR}, 960 | pages = {5799--5809}, 961 | year = {2021} 962 | } 963 | 964 | @article{zhou2021cips3d, 965 | title={{CIPS-3D}: A 3d-aware generator of gans based on conditionally-independent pixel synthesis}, 966 | author={Zhou, Peng and Xie, Lingxi and Ni, Bingbing and Tian, Qi}, 967 | journal={arXiv preprint arXiv:2110.09788}, 968 | year={2021} 969 | } 970 | 971 | @inproceedings{schwarz2020graf, 972 | title={{GRAF}: Generative Radiance Fields for {3D}-Aware Image Synthesis}, 973 | author={Schwarz, Katja and Liao, Yiyi and Niemeyer, Michael and Geiger, Andreas}, 974 | booktitle={NeurIPS}, 975 | volume={33}, 976 | pages={20154--20166}, 977 | year={2020} 978 | } 979 | 980 | @inproceedings{zhao2022generative, 981 | title={Generative Multiplane Images: Making a 2D GAN 3D-Aware}, 982 | author={Zhao, Xiaoming and Ma, Fangchang and G{\"u}era, David and Ren, Zhile and Schwing, Alexander G and Colburn, Alex}, 983 | booktitle={ECCV}, 984 | pages={18--35}, 985 | year={2022} 986 | } 987 | 988 | @article{orel2022stylesdf, 989 | title={Style{SDF}: High-Resolution 3D-Consistent Image and Geometry Generation}, 990 | author={Roy Or-El and Xuan Luo and Mengyi Shan and Eli Shechtman and Jeong Joon Park and Ira Kemelmacher-Shlizerman}, 991 | journal={CVPR}, 992 | pages={13503--13513}, 993 | year={2022}, 994 | } 995 | 996 | @inproceedings{xu2022volumegan, 997 | title = {{3D-aware Image Synthesis via Learning Structural and Textural Representations}}, 998 | author = {Xu, Yinghao and Peng, Sida and Yang, Ceyuan and Shen, Yujun and Zhou, Bolei}, 999 | booktitle = {CVPR}, 1000 | pages={18430--18439}, 1001 | year = {2022} 1002 | } 1003 | 1004 | @inproceedings{gu2022stylenerf, 1005 | title={{StyleNeRF}: A Style-based {3D} Aware Generator for High-resolution Image Synthesis}, 1006 | author={Jiatao Gu and Lingjie Liu and Peng Wang and Christian Theobalt}, 1007 | booktitle={ICLR}, 1008 | year={2022} 1009 | } 1010 | 1011 | @inproceedings{deng2022gram, 1012 | title={{GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation}}, 1013 | author={Yu Deng and Jiaolong Yang and Jianfeng Xiang and Xin Tong}, 1014 | booktitle={CVPR}, 1015 | pages = {10663--10673}, 1016 | year={2022}, 1017 | } 1018 | 1019 | @inproceedings{chan2022efficient, 1020 | title = {{Efficient Geometry-aware {3D} Generative Adversarial Networks}}, 1021 | author = {Eric~R. Chan and Connor~Z. Lin and Matthew~A. Chan and Koki Nagano and Boxiao Pan and Shalini De~Mello and Orazio Gallo and Leonidas Guibas and Jonathan Tremblay and Sameh Khamis and Tero Karras and Gordon Wetzstein}, 1022 | booktitle={CVPR}, 1023 | pages = {16102--16112}, 1024 | year={2022} 1025 | } 1026 | 1027 | @inproceedings{liu20223d, 1028 | title={{3D-FM GAN}: Towards 3D-Controllable Face Manipulation}, 1029 | author={Liu, Yuchen and Shu, Zhixin and Li, Yijun and Lin, Zhe and Zhang, Richard and Kung, SY}, 1030 | booktitle={ECCV}, 1031 | pages={107--125}, 1032 | year={2022} 1033 | } 1034 | 1035 | @inproceedings{kwak2022injecting, 1036 | title={Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis}, 1037 | author={Kwak, Jeong-gi and Li, Yuanming and Yoon, Dongsik and Kim, Donghyeon and Han, David and Ko, Hanseok}, 1038 | booktitle={ECCV}, 1039 | pages={236--253}, 1040 | year={2022} 1041 | } 1042 | 1043 | @inproceedings{hong2022headnerf, 1044 | title={{HeadNeRF}: A real-time nerf-based parametric head model}, 1045 | author={Hong, Yang and Peng, Bo and Xiao, Haiyao and Liu, Ligang and Zhang, Juyong}, 1046 | booktitle={CVPR}, 1047 | pages = {20342--20352}, 1048 | year={2022} 1049 | } 1050 | 1051 | @inproceedings{mildenhall2020nerf, 1052 | title={Nerf: Representing scenes as neural radiance fields for view synthesis}, 1053 | author={Mildenhall, Ben and Srinivasan, Pratul P and Tancik, Matthew and Barron, Jonathan T and Ramamoorthi, Ravi and Ng, Ren}, 1054 | booktitle={ECCV}, 1055 | pages={405--421}, 1056 | year={2020} 1057 | } 1058 | 1059 | @inproceedings{martin2020nerf, 1060 | title={NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections}, 1061 | author={Martin-Brualla, Ricardo and Radwan, Noha and Sajjadi, Mehdi SM and Barron, Jonathan T and Dosovitskiy, Alexey and Duckworth, Daniel}, 1062 | booktitle={CVPR}, 1063 | pages = {7210--7219}, 1064 | year={2021} 1065 | } 1066 | 1067 | @inproceedings{muller2022autorf, 1068 | title={{AutoRF}: Learning 3D Object Radiance Fields from Single View Observations}, 1069 | author={M{\"u}ller, Norman and Simonelli, Andrea and Porzi, Lorenzo and Bul{\`o}, Samuel Rota and Nie{\ss}ner, Matthias and Kontschieder, Peter}, 1070 | booktitle={CVPR}, 1071 | pages={3971--3980}, 1072 | year={2022} 1073 | } 1074 | 1075 | @inproceedings{rebain2022lolnerf, 1076 | title={{LOLNeRF}: Learn from One Look}, 1077 | author={Rebain, Daniel and Matthews, Mark and Yi, Kwang Moo and Lagun, Dmitry and Tagliasacchi, Andrea}, 1078 | booktitle={CVPR}, 1079 | pages={1558--1567}, 1080 | year={2022} 1081 | } 1082 | 1083 | @inproceedings{niemeyer2021giraffe, 1084 | author = {Michael Niemeyer and Andreas Geiger}, 1085 | title = {{GIRAFFE: Representing scenes as compositional generative neural feature fields}}, 1086 | booktitle = {CVPR}, 1087 | pages = {11453--11464}, 1088 | year = {2021} 1089 | } 1090 | 1091 | @inproceedings{xue2022giraffehd, 1092 | title={{GIRAFFE-HD}: A High-Resolution 3D-aware Generative Model}, 1093 | author={Xue, Yang and Li, Yuheng and Singh, Krishna Kumar and Lee, Yong Jae}, 1094 | booktitle = {CVPR}, 1095 | year={2022} 1096 | } 1097 | 1098 | @inproceedings{mescheder2019occupancy, 1099 | title={Occupancy networks: Learning 3d reconstruction in function space}, 1100 | author={Mescheder, Lars and Oechsle, Michael and Niemeyer, Michael and Nowozin, Sebastian and Geiger, Andreas}, 1101 | booktitle={CVPR}, 1102 | pages={4460--4470}, 1103 | year={2019} 1104 | } 1105 | 1106 | @inproceedings{peng2020convolutional, 1107 | title={Convolutional occupancy networks}, 1108 | author={Peng, Songyou and Niemeyer, Michael and Mescheder, Lars and Pollefeys, Marc and Geiger, Andreas}, 1109 | booktitle={ECCV}, 1110 | pages={523--540}, 1111 | year={2020} 1112 | } 1113 | 1114 | @inproceedings{xu2021generative, 1115 | title={Generative occupancy fields for 3d surface-aware image synthesis}, 1116 | author={Xu, Xudong and Pan, Xingang and Lin, Dahua and Dai, Bo}, 1117 | booktitle={NeurIPS}, 1118 | pages = {20683--20695}, 1119 | year={2021} 1120 | } 1121 | 1122 | @article{sitzmann2019scene, 1123 | title={Scene representation networks: Continuous 3d-structure-aware neural scene representations}, 1124 | author={Sitzmann, Vincent and Zollh{\"o}fer, Michael and Wetzstein, Gordon}, 1125 | journal={NeurIPS}, 1126 | pages = {1119--1130}, 1127 | volume={32}, 1128 | year={2019} 1129 | } 1130 | 1131 | @inproceedings{yu2021pixelnerf, 1132 | title={{pixelNeRF}: Neural radiance fields from one or few images}, 1133 | author={Yu, Alex and Ye, Vickie and Tancik, Matthew and Kanazawa, Angjoo}, 1134 | booktitle={CVPR}, 1135 | pages={4578--4587}, 1136 | year={2021} 1137 | } 1138 | 1139 | @inproceedings{yu2021plenoctrees, 1140 | title={Plenoctrees for real-time rendering of neural radiance fields}, 1141 | author={Yu, Alex and Li, Ruilong and Tancik, Matthew and Li, Hao and Ng, Ren and Kanazawa, Angjoo}, 1142 | booktitle={ICCV}, 1143 | pages={5752--5761}, 1144 | year={2021} 1145 | } 1146 | 1147 | @inproceedings{reiser2021kilonerf, 1148 | title={Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps}, 1149 | author={Reiser, Christian and Peng, Songyou and Liao, Yiyi and Geiger, Andreas}, 1150 | booktitle={ICCV}, 1151 | pages={14335--14345}, 1152 | year={2021} 1153 | } 1154 | 1155 | @article{muller2022instant, 1156 | title={Instant neural graphics primitives with a multiresolution hash encoding}, 1157 | author={M{\"u}ller, Thomas and Evans, Alex and Schied, Christoph and Keller, Alexander}, 1158 | journal={TOG}, 1159 | volume={41}, 1160 | number={4}, 1161 | pages={1--15}, 1162 | year={2022} 1163 | } 1164 | 1165 | @inproceedings{garbin2021fastnerf, 1166 | title={{FastNeRF}: High-fidelity neural rendering at 200fps}, 1167 | author={Garbin, Stephan J and Kowalski, Marek and Johnson, Matthew and Shotton, Jamie and Valentin, Julien}, 1168 | booktitle={ICCV}, 1169 | pages={14346--14355}, 1170 | year={2021} 1171 | } 1172 | 1173 | @inproceedings{hu2022efficientnerf, 1174 | title={{EfficientNeRF}: Efficient Neural Radiance Fields}, 1175 | author={Hu, Tao and Liu, Shu and Chen, Yilun and Shen, Tiancheng and Jia, Jiaya}, 1176 | booktitle={CVPR}, 1177 | pages={12902--12911}, 1178 | year={2022} 1179 | } 1180 | 1181 | @inproceedings{hedman2021snerf, 1182 | title={Baking neural radiance fields for real-time view synthesis}, 1183 | author={Hedman, Peter and Srinivasan, Pratul P and Mildenhall, Ben and Barron, Jonathan T and Debevec, Paul}, 1184 | booktitle={ICCV}, 1185 | pages={5875--5884}, 1186 | year={2021} 1187 | } 1188 | 1189 | @inproceedings{rematas2022urbannerf, 1190 | title={Urban radiance fields}, 1191 | author={Rematas, Konstantinos and Liu, Andrew and Srinivasan, Pratul P and Barron, Jonathan T and Tagliasacchi, Andrea and Funkhouser, Thomas and Ferrari, Vittorio}, 1192 | booktitle={CVPR}, 1193 | pages={12932--12942}, 1194 | year={2022} 1195 | } 1196 | 1197 | @inproceedings{wu2022diver, 1198 | title={{DIVeR}: Real-time and accurate neural radiance fields with deterministic integration for volume rendering}, 1199 | author={Wu, Liwen and Lee, Jae Yong and Bhattad, Anand and Wang, Yu-Xiong and Forsyth, David}, 1200 | booktitle={CVPR}, 1201 | year={2022} 1202 | } 1203 | 1204 | @inproceedings{lindell2021autoint, 1205 | title={Autoint: Automatic integration for fast neural volume rendering}, 1206 | author={Lindell, David B and Martel, Julien NP and Wetzstein, Gordon}, 1207 | booktitle={CVPR}, 1208 | pages = {14556--14565}, 1209 | year={2021} 1210 | } 1211 | 1212 | @inproceedings{barron2021mipnerf, 1213 | title={{Mip-NeRF}: A multiscale representation for anti-aliasing neural radiance fields}, 1214 | author={Barron, Jonathan T and Mildenhall, Ben and Tancik, Matthew and Hedman, Peter and Martin-Brualla, Ricardo and Srinivasan, Pratul P}, 1215 | booktitle={ICCV}, 1216 | pages={5855--5864}, 1217 | year={2021} 1218 | } 1219 | 1220 | @inproceedings{sitzmann2019deepvoxels, 1221 | title={Deepvoxels: Learning persistent 3d feature embeddings}, 1222 | author={Sitzmann, Vincent and Thies, Justus and Heide, Felix and Nie{\ss}ner, Matthias and Wetzstein, Gordon and Zollhofer, Michael}, 1223 | booktitle={CVPR}, 1224 | pages={2437--2446}, 1225 | year={2019} 1226 | } 1227 | 1228 | @article{chang2015shapenet, 1229 | title={Shapenet: An information-rich 3d model repository}, 1230 | author={Chang, Angel X and Funkhouser, Thomas and Guibas, Leonidas and Hanrahan, Pat and Huang, Qixing and Li, Zimo and Savarese, Silvio and Savva, Manolis and Song, Shuran and Su, Hao and others}, 1231 | journal={arXiv preprint arXiv:1512.03012}, 1232 | year={2015} 1233 | } 1234 | 1235 | @inproceedings{dai2017scannet, 1236 | title={Scannet: Richly-annotated 3d reconstructions of indoor scenes}, 1237 | author={Dai, Angela and Chang, Angel X and Savva, Manolis and Halber, Maciej and Funkhouser, Thomas and Nie{\ss}ner, Matthias}, 1238 | booktitle={CVPR}, 1239 | pages={5828--5839}, 1240 | year={2017} 1241 | } 1242 | 1243 | @inproceedings{gafni2021dynamic, 1244 | title={Dynamic neural radiance fields for monocular 4d facial avatar reconstruction}, 1245 | author={Gafni, Guy and Thies, Justus and Zollhofer, Michael and Nie{\ss}ner, Matthias}, 1246 | booktitle={CVPR}, 1247 | pages={8649--8658}, 1248 | year={2021} 1249 | } 1250 | 1251 | @inproceedings{schonberger2016structure, 1252 | title={Structure-from-motion revisited}, 1253 | author={Schonberger, Johannes L and Frahm, Jan-Michael}, 1254 | booktitle={CVPR}, 1255 | pages={4104--4113}, 1256 | year={2016} 1257 | } 1258 | 1259 | @inproceedings{blanz1999morphable, 1260 | title={A morphable model for the synthesis of 3D faces}, 1261 | author={Blanz, Volker and Vetter, Thomas}, 1262 | booktitle={SIGGRAPH}, 1263 | pages={187--194}, 1264 | year={1999} 1265 | } 1266 | 1267 | @inproceedings{ramamoorthi2001efficient, 1268 | title={An efficient representation for irradiance environment maps}, 1269 | author={Ramamoorthi, Ravi and Hanrahan, Pat}, 1270 | booktitle={SIGGRAPH}, 1271 | pages={497--500}, 1272 | year={2001} 1273 | } 1274 | 1275 | @inproceedings{sajjadi2022scene, 1276 | title={Scene representation transformer: Geometry-free novel view synthesis through set-latent scene representations}, 1277 | author={Sajjadi, Mehdi SM and Meyer, Henning and others}, 1278 | booktitle={CVPR}, 1279 | pages={6229--6238}, 1280 | year={2022} 1281 | } 1282 | 1283 | @article{sitzmann2021light, 1284 | title={Light field networks: Neural scene representations with single-evaluation rendering}, 1285 | author={Sitzmann, Vincent and Rezchikov, Semon and Freeman, Bill and Tenenbaum, Josh and Durand, Fredo}, 1286 | journal={NeurIPS}, 1287 | volume={34}, 1288 | pages = {19313--19325}, 1289 | year={2021} 1290 | } 1291 | 1292 | @article{watson2022novel, 1293 | title={Novel View Synthesis with Diffusion Models}, 1294 | author={Watson, Daniel and Chan, William and Martin-Brualla, Ricardo and Ho, Jonathan and Tagliasacchi, Andrea and Norouzi, Mohammad}, 1295 | journal={arXiv preprint arXiv:2210.04628}, 1296 | year={2022} 1297 | } 1298 | 1299 | @inproceedings{kosiorek2021nerf, 1300 | title={{NeRF-VAE}: A geometry aware {3D} scene generative model}, 1301 | author={Kosiorek, Adam R and Strathmann, Heiko and Zoran, Daniel and Moreno, Pol and Schneider, Rosalia and Mokr{\'a}, Sona and Rezende, Danilo Jimenez}, 1302 | booktitle={ICML}, 1303 | volume = {139}, 1304 | pages = {5742--5752}, 1305 | year={2021} 1306 | } 1307 | 1308 | -------------------------------------------------------------------------------- /save_cited_bib.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | ''' 4 | @File : save_cited_bib.py 5 | @Time : 2023/09/19 00:04:41 6 | @Author : Weihao Xia (xiawh3@outlook.com) 7 | @Version : 1.0 8 | @Desc : This script is to create a .bib file containing only the references used in the present paper 9 | This can serve as an alternative to converting from .bbl to .bib. 10 | ''' 11 | 12 | import re 13 | import argparse 14 | 15 | def extract_cite_content_from_file(filename): 16 | """Extract all citation keys from a .tex file.""" 17 | with open(filename, 'r', encoding='utf-8') as file: 18 | content = file.read() 19 | 20 | pattern = r'\\cite\{(.*?)\}' 21 | raw_matches = re.findall(pattern, content) 22 | matches = [ref.strip() for match in raw_matches for ref in match.split(',')] 23 | return set(matches) 24 | 25 | def extract_bib_entries(bib_filename, cite_keys): 26 | """Extract relevant bib entries based on citation keys.""" 27 | with open(bib_filename, 'r', encoding='utf-8') as file: 28 | content = file.readlines() 29 | 30 | inside_entry = False 31 | current_key = None 32 | saved_entries = [] 33 | temp_entries = [] 34 | 35 | for line in content: 36 | # Check if a new bib entry starts 37 | if line.startswith('@'): 38 | inside_entry = True 39 | current_key = line.split("{", 1)[1].split(",", 1)[0].strip() 40 | temp_entries = [line] 41 | elif inside_entry: 42 | temp_entries.append(line) 43 | 44 | # Check the end of the bib entry 45 | if inside_entry and line.strip() == "}": 46 | inside_entry = False 47 | if current_key in cite_keys: 48 | saved_entries.extend(temp_entries) 49 | saved_entries.append('\n') # Add a newline after each entry 50 | 51 | return saved_entries 52 | 53 | def main(): 54 | parser = argparse.ArgumentParser(description="Extract cited bib entries from a larger bib file.") 55 | parser.add_argument("--tex_filename", default='main_csur.tex', help="Path to the .tex file") 56 | parser.add_argument("--bib_filename", default='reference_full.bib', help="Path to the full .bib file") 57 | parser.add_argument("--out_filename", default='reference.bib', help="Path for the output .bib file with only cited entries") 58 | args = parser.parse_args() 59 | 60 | cite_keys = extract_cite_content_from_file(args.tex_filename) 61 | filtered_bib_entries = extract_bib_entries(args.bib_filename, cite_keys) 62 | 63 | with open(args.out_filename, 'w', encoding='utf-8') as out_file: 64 | out_file.writelines(filtered_bib_entries) 65 | 66 | if __name__ == "__main__": 67 | main() --------------------------------------------------------------------------------