├── fig ├── D.png ├── M.png ├── R1.png └── R2.png └── README.md /fig/D.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KD-TAO/MGFR/HEAD/fig/D.png -------------------------------------------------------------------------------- /fig/M.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KD-TAO/MGFR/HEAD/fig/M.png -------------------------------------------------------------------------------- /fig/R1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KD-TAO/MGFR/HEAD/fig/R1.png -------------------------------------------------------------------------------- /fig/R2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KD-TAO/MGFR/HEAD/fig/R2.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # MGFR: **Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model** 3 | 4 | Keda Tao, [Jinjin Gu*](https://www.jasongt.com/), [Yulun Zhang](http://yulunzhang.com/), Xiucheng Wang and Nan Cheng, "Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model" 5 | 6 |

7 | 8 | [arXiv](https://arxiv.org/abs/2410.04161) [Reface-HQ](https://drive.google.com/file/d/1EUQlRCZaoG9eKMoJ8mWaHW5kqvhL0elY/view?usp=sharing) 9 | 10 | #### 🔥🔥🔥 News 11 | 12 | - **2025-1-22:** Our paper has been accepted by ICLR 2025 (Spotlight)!. 13 | - **2025-4-22:** The new dataset we proposed-Reface-HQ-has been released. 14 | 15 | --- 16 | 17 | > **Abstract:** We introduce a novel Multi-modal Guided Real-World Face Restoration (MGFR) technique designed to improve the quality of facial image restoration from low-quality inputs. Leveraging a blend of attribute text prompts, high-quality reference images, and identity information, MGFR can mitigate the generation of false facial attributes and identities often associated with generative face restoration methods. By incorporating a dual-control adapter and a two-stage training strategy, our method effectively utilizes multi-modal prior information for targeted restoration tasks. We also present the Reface-HQ dataset, comprising over 21,000 high-resolution facial images across 4800 identities, to address the need for reference face training images. Our approach achieves superior visual quality in restoring facial details under severe degradation and allows for controlled restoration processes, enhancing the accuracy of identity preservation and attribute correction. Including negative quality samples and attribute prompts in the training further refines the model's ability to generate detailed and perceptually accurate images. 18 | 19 | ## ⚒️ TODO 20 | 21 | * [x] Release Reface-HQ dataset. 22 | * [ ] Release code and pretrained models. 23 | 24 | ## :sparkles: Reface-HQ Dataset 25 | We introduce a novel, high-quality dataset tailored for reference-based face image restoration tasks, which is now publicly accessible. The dataset comprises face images from over 4,800 unique identities, offering an expanded pool of training samples at higher image resolutions. 26 | ### Download 27 | - [Reface-HQ](https://drive.google.com/file/d/1EUQlRCZaoG9eKMoJ8mWaHW5kqvhL0elY/view?usp=sharing) [Reface-Test](https://drive.google.com/file/d/196UvnatfexBw97SZ7OXXFKQ_XIJ4jl7O/view?usp=sharing) 28 |

29 | 30 | ## :eyes: Visual Results 31 | ### Face Restoration 32 | 33 |

34 | 35 |

36 | 37 | ## Acknowledgement 38 | 39 | This project is based on [ControlNet](https://github.com/lllyasviel/ControlNet). Thanks for their awesome work. 40 | 41 | ## Contact 42 | 43 | If you have any questions, please feel free to contact with me at KD.TAO@outlook.com 44 | --------------------------------------------------------------------------------