├── assets └── teaser.png └── README.md /assets/teaser.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wsj-sjtu/SingingHead/HEAD/assets/teaser.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SingingHead: A Large-scale 4D Dataset for Singing Head Animation 2 | ## [arXiv](https://arxiv.org/pdf/2312.04369.pdf) | [Project Page](https://wsj-sjtu.github.io/SingingHead/) | [Dataset](https://huggingface.co/datasets/Human-X/SingingHead) 3 | 4 | 5 | 6 | ## TODO 7 | - [ ] Release the codes for calculating the metrics of two benchmarks. 8 | - [ ] Release the scripts for visualizing the 3D facial motion. 9 | - [x] Release the SingingHead dataset. 10 | 11 | ## SingingHead Dataset 12 | ### Download 13 | The dataset can be downloaded from [Hugging Face](https://huggingface.co/datasets/Human-X/SingingHead). 14 | 15 | If you are unable to download from Hugging Face, please first fill out the required information on [Hugging Face](https://huggingface.co/datasets/Human-X/SingingHead) to obtain authorization, and then contact us [(wusijing@sjtu.edu.cn)](wusijing@sjtu.edu.cn) using the same email address to get the download link of Baidu (百度网盘). 16 | 17 | Please note that by requesting the dataset, you confirm that you have read, understood, and agree to be bound by the terms of the agreement. 18 | 19 | **Agreement** 20 | 21 | 1. The SingingHead dataset is available for **non-commercial** research purposes only. 22 | 23 | 2. You agree **not to** reproduce, modified, duplicate, copy, sell, trade, resell or exploit any portion of the images and any portion of the derived data for commercial purposes. 24 | 25 | 3. You agree **not to** further copy, publish or distribute any portion of the SingingHead dataset to any third party for any purpose. Except, for internal use at a single site within the same organization it is allowed to make copies of the dataset. 26 | 27 | 4. Shanghai Jiao Tong University reserves the right to terminate your access to the SingingHead dataset at any time. 28 | 29 | 30 | ### Overview 31 | The SingingHead dataset is a large-scale 4D dataset for singing head animation. It contains more than 27 hours of synchronized singing video, 3D facial motion, singing 32 | audio, and background music collected from 76 subjects. 33 | The video is captured in 30fps and cropped into a resolution of 1024×1024. 34 | The 3D facial motion is represented by 59-dimensional [FLAME](https://flame.is.tue.mpg.de/) parameters (50 expression + 3 global pose + 3 neck pose + 3 jaw pose). 35 | All the data sequences are cut into equal-length 8s segments, resulting in a total of 12196 sequences. 36 | 37 | ### Data Structure 38 | ``` 39 | SingingHead 40 | ├── train.txt 41 | ├── val.txt 42 | ├── test.txt 43 | ├── video_seqs.zip 44 | │   ├── id0_10_0_0.mp4 45 | │   └── ... 46 | ├── flame_seqs.zip 47 | │   ├── id0_10_0_0.pkl 48 | │   └── ... 49 | ├── audio_seqs.zip 50 | │   ├── id0_10_0_0.wav 51 | │   └── ... 52 | └── bgm_seqs.zip 53 | ├── id0_10_0_0_bgm.wav 54 | └── ... 55 | ``` 56 | 57 | ## Citation 58 | If you use this dataset, please consider citing 59 | ``` 60 | @article{wu2025singinghead, 61 | title={Singinghead: A large-scale 4d dataset for singing head animation}, 62 | author={Wu, Sijing and Li, Yunhao and Zhang, Weitian and Jia, Jun and Zhu, Yucheng and Yan, Yichao and Zhai, Guangtao and Yang, Xiaokang}, 63 | journal={IEEE Transactions on Multimedia}, 64 | year={2025}, 65 | publisher={IEEE} 66 | } 67 | ``` 68 | 69 | ## Contact 70 | - Sijing Wu [(wusijing@sjtu.edu.cn)](wusijing@sjtu.edu.cn) 71 | --------------------------------------------------------------------------------