├── README.md └── img ├── abo.gif ├── art_gallery.gif ├── cliff.gif ├── compare_figure.png ├── compare_table.png ├── flux_1.gif ├── green_man.gif ├── gso.gif ├── pipeline.png ├── plaza.gif ├── real_img.gif ├── sd_1.gif ├── sd_2.gif ├── town.gif └── wild.gif /README.md: -------------------------------------------------------------------------------- 1 |   2 | 3 |
4 | 5 |

Baking Gaussian Splatting into Diffusion Denoiser for Fast
and Scalable Single-stage Image-to-3D Generation and Reconstruction

6 | 7 | [![arXiv](https://img.shields.io/badge/paper-arxiv-179bd3)](https://arxiv.org/abs/2411.14384) 8 | [![project](https://img.shields.io/badge/project-page-green)](https://caiyuanhao1998.github.io/project/DiffusionGS/) 9 | [![hf](https://img.shields.io/badge/hugging-face-green)](https://huggingface.co/datasets/CaiYuanhao/DiffusionGS) 10 | [![MrNeRF](https://img.shields.io/badge/media-MrNeRF-yellow)](https://x.com/janusch_patas/status/1859867424859856997?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet) 11 | 12 |

13 | abo 14 | gso 15 | real_img 16 | wild 17 |

18 |

19 | sd_2 20 | sd_1 21 | flux_1 22 | green_man 23 |

24 |

25 | plaza 26 | town 27 |

28 |

29 | cliff 30 | art_gallery 31 |

32 | 33 | 34 |   35 | 36 |
37 | 38 | 39 | 40 | ### Introduction 41 | This is an implementation of our work "Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction 42 | ". Our DiffusionGS is single-stage and does not rely on 2D multi-view diffusion model. DiffusionGS can be applied to single-view 3D object generation and scene reconstruction without using depth estimator in ~6 seconds. If you find our repo useful, please give it a star ⭐ and consider citing our paper. Thank you :) 43 | 44 | ![pipeline](/img/pipeline.png) 45 | 46 | 47 | ### News 48 | - **2025.06.25 :** Our paper has been accepted by ICCV 2025. Code wil be released. Stay tuned! 🎉 🎊 49 | - **2024.11.22 :** Our [project page](https://caiyuanhao1998.github.io/project/DiffusionGS/) has been built up. Feel free to check the video and interactive generation results on the project page. 50 | - **2024.11.21 :** We upload the prompt image and our generation results to our [hugging face dataset](https://huggingface.co/datasets/CaiYuanhao/DiffusionGS). Feel free to download and make a comparison with your method. 🤗 51 | - **2024.11.20 :** Our paper is on [arxiv](https://arxiv.org/abs/2411.14384) now. 🚀 52 | 53 | ### Comparison with State-of-the-Art Methods 54 | 55 |
56 | Qualitative Comparison 57 | 58 | ![visual_results](/img/compare_figure.png) 59 | 60 |
61 | 62 | 63 |
64 | Quantitative Comparison 65 | 66 | ![results1](/img/compare_table.png) 67 | 68 |
69 | 70 | 71 | 72 | 73 | 74 | 75 |   76 | 77 | ## Citation 78 | ```sh 79 | @article{cai2024baking, 80 | title={Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction}, 81 | author={Yuanhao Cai and He Zhang and Kai Zhang and Yixun Liang and Mengwei Ren and Fujun Luan and Qing Liu and Soo Ye Kim and Jianming Zhang and Zhifei Zhang and Yuqian Zhou and Yulun Zhang and Xiaokang Yang and Zhe Lin and Alan Yuille}, 82 | journal={arXiv preprint arXiv:2411.14384}, 83 | year={2024} 84 | } 85 | ``` 86 | -------------------------------------------------------------------------------- /img/abo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/abo.gif -------------------------------------------------------------------------------- /img/art_gallery.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/art_gallery.gif -------------------------------------------------------------------------------- /img/cliff.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/cliff.gif -------------------------------------------------------------------------------- /img/compare_figure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/compare_figure.png -------------------------------------------------------------------------------- /img/compare_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/compare_table.png -------------------------------------------------------------------------------- /img/flux_1.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/flux_1.gif -------------------------------------------------------------------------------- /img/green_man.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/green_man.gif -------------------------------------------------------------------------------- /img/gso.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/gso.gif -------------------------------------------------------------------------------- /img/pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/pipeline.png -------------------------------------------------------------------------------- /img/plaza.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/plaza.gif -------------------------------------------------------------------------------- /img/real_img.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/real_img.gif -------------------------------------------------------------------------------- /img/sd_1.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/sd_1.gif -------------------------------------------------------------------------------- /img/sd_2.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/sd_2.gif -------------------------------------------------------------------------------- /img/town.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/town.gif -------------------------------------------------------------------------------- /img/wild.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caiyuanhao1998/Open-DiffusionGS/2d28a568e9308705caaf99f88e8164e25f680ceb/img/wild.gif --------------------------------------------------------------------------------