126 |
134 |
127 | 
128 |
129 |
133 |
128 | 129 |
130 | PEChead generates high-fidelity head pose and expression editing results. 131 |
132 |144 | Talking head generation is to generate video based on a given source identity and target motion. However, current methods face several 145 | challenges that limit the quality and controllability of the generated videos. First, the generated face often has unexpected 146 | deformation and severe distortions. Second, the driving image does not explicitly disentangle movement-relevant information, such as 147 | poses and expressions, which restricts the manipulation of different attributes during generation. Third, the generated videos tend to 148 | have flickering artifacts due to the inconsistency of the extracted landmarks between adjacent frames. 149 |
150 |151 | In this paper, we propose a novel model that produces high-fidelity talking head videos with free control over head pose and 152 | expression. Our method leverages both self-supervised learned landmarks and 3D face model-based landmarks to model the motion. We also 153 | introduce a novel motion-aware multi-scale feature alignment module to effectively transfer the motion without face distortion. 154 | Furthermore, we enhance the smoothness of the synthesized talking head videos with a feature context adaptation and propagation 155 | module. We evaluate our model on challenging datasets and demonstrate its state-of-the-art performance. 156 |
157 |188 | Using PECHead you can create great visual results for head pose and expression editing. 189 |
190 |233 | Using PECHead you can create great visual results for portrait frontalization. 234 |
235 |283 | Using PECHead you can create great visual results for portrait reconstruction. 284 |
285 |339 | Using PECHead you can create great visual results for portrait reenactment. 340 |
341 |374 | Using PECHead you can create great visual results for expression editing. 375 |
376 |415 | Using PECHead you can create great visual results for portrait reconstruction on wild identities. 416 |
417 |@inproceedings{gao2023high,
440 | title={High-fidelity and freely controllable talking head video generation},
441 | author={Gao, Yue and Zhou, Yuan and Wang, Jinglu and Li, Xiao and Ming, Xiang and Lu, Yan},
442 | booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
443 | pages={5609--5619},
444 | year={2023}
445 | }