├── README.md
├── _config.yml
├── images
├── DurIAN_4S.png
├── DurIAN_SC.png
└── logo.png
├── index.md
└── wavs
├── 025_000009.wav
├── 025_r_1.wav
├── 025_r_2.wav
├── 025_r_3.wav
├── 025_r_4.wav
├── 025_r_5.wav
├── daj_000023.wav
├── daj_r_1.wav
├── daj_r_2.wav
├── daj_r_3.wav
├── daj_r_4.wav
├── daj_r_5.wav
├── dx_r_1.wav
├── dx_r_2.wav
├── dx_r_3.wav
├── dx_r_4.wav
├── dx_r_5.wav
├── gui_10000224.wav
├── gui_r_1.wav
├── gui_r_2.wav
├── gui_r_3.wav
├── gui_r_4.wav
├── gui_r_5.wav
├── inset_dvec
├── dvec_101.wav
├── dvec_1161.wav
├── dvec_1189.wav
├── dvec_1512.wav
├── dvec_238.wav
└── dvec_658.wav
├── inset_emb
├── lut_101.wav
├── lut_1161.wav
├── lut_1189.wav
├── lut_1512.wav
├── lut_238.wav
└── lut_658.wav
├── inset_ref
├── 101.wav
├── 1161.wav
├── 1189.wav
├── 1512.wav
├── 238.wav
└── 658.wav
├── liu_88010505.wav
├── liu_r_1.wav
├── liu_r_2.wav
├── liu_r_3.wav
├── liu_r_4.wav
├── liu_r_5.wav
├── lpl_r_1.wav
├── lpl_r_2.wav
├── lpl_r_3.wav
├── lpl_r_4.wav
├── lpl_r_5.wav
├── lpl_sample.wav
├── outset_dvec
├── dvec_10001.wav
├── dvec_10003.wav
├── dvec_10004.wav
└── dvec_10005.wav
├── song_dx_801000013.wav
├── speech_and_singing
├── speech_and_sing_10001.wav
├── speech_and_sing_10002.wav
├── speech_and_sing_10003.wav
├── speech_and_sing_10004.wav
├── speech_and_sing_10005.wav
└── speech_and_sing_10006.wav
├── speech_only
├── speech_10001.wav
├── speech_10002.wav
├── speech_10003.wav
├── speech_10004.wav
├── speech_10005.wav
└── speech_10006.wav
├── speech_ref
├── 10001.wav
├── 10002.wav
├── 10003.wav
├── 10004.wav
├── 10005.wav
└── 10006.wav
├── ssx_08010642.wav
├── ssx_r_1.wav
├── ssx_r_2.wav
├── ssx_r_3.wav
├── ssx_r_4.wav
└── ssx_r_5.wav
/README.md:
--------------------------------------------------------------------------------
1 | # DurIAN : DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System
2 |
3 | Project page for our paper "DurIAN : DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System". Link is: https://tencent-ailab.github.io/learning_singing_from_speech/
4 |
--------------------------------------------------------------------------------
/_config.yml:
--------------------------------------------------------------------------------
1 | theme: jekyll-theme-cayman
2 | github:
3 | is_project_page: false
4 | title: [DurIAN_SC]
5 |
--------------------------------------------------------------------------------
/images/DurIAN_4S.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tencent-ailab/learning_singing_from_speech/f17488de7849c50b779839b5e6fc05674a3eabd6/images/DurIAN_4S.png
--------------------------------------------------------------------------------
/images/DurIAN_SC.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tencent-ailab/learning_singing_from_speech/f17488de7849c50b779839b5e6fc05674a3eabd6/images/DurIAN_SC.png
--------------------------------------------------------------------------------
/images/logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tencent-ailab/learning_singing_from_speech/f17488de7849c50b779839b5e6fc05674a3eabd6/images/logo.png
--------------------------------------------------------------------------------
/index.md:
--------------------------------------------------------------------------------
1 | # DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System
2 |
3 | Liqiang Zhang, Chengzhu Yu, Heng Lu, Chao Weng, Chunlei Zhang, Yusong Wu, Xiang Xie, Zijin Li, Dong Yu
4 | Tencent AI Lab
5 |
6 |
7 | ## Abstract
8 |
9 | Singing voice conversion is converting the timbre in the source singing to the target speaker's voice while keeping singing content the same. However, singing data for target speaker is much more difficult to collect compared with normal speech data. In this paper, we introduce a singing voice conversion algorithm that is capable of generating high quality target speaker's singing using only his/her normal speech data. First, we manage to integrate the training and conversion process of speech and singing into one framework by unifying the features used in standard speech synthesis system and singing synthesis system. In this way, normal speech data can also contribute to singing voice conversion training, making the singing voice conversion system more robust especially when the singing database is small. Moreover, in order to achieve one-shot singing voice conversion, a speaker embedding module is developed using both speech and singing data, which provides target speaker identify information during conversion. Experiments indicate proposed sing conversion system can convert source singing to target speaker's high-quality singing with only 20 seconds of target speaker's enrollment speech data.
10 |
11 | Note: All samples are in Mandrin Chinese.
21 | There are 6 in-set singers---3 male singers and 3 female singers, shown here.
23 | The "Reference Voice" is shown here for timbre similarity test, which is the target singer's singing.
25 |
26 |
27 |
31 |
32 |
33 |
34 | Reference Voice
35 | LUT Sample
36 | D-vector Sample
37 |
38 |
39 |
40 |
41 | Female Singer1
42 |
43 |
45 |
47 |
49 |
50 | Female Singer2
51 |
52 |
54 |
56 |
58 |
59 | Female Singer3
60 |
61 |
63 |
65 |
67 |
68 | Male Singer1
69 |
70 |
72 |
74 |
76 |
77 | Male Singer2
78 |
79 |
81 |
83 |
85 |
86 | Male Singer3
87 |
88 |
90 |
92 |
94 |
95 |
96 |
97 | Note: All samples are in Mandrin Chinese.
103 | There are 4 out-of-set speakers---2 male speakers and 2 female speakers, shown here.
105 | The "Register Voice" is shown here for similarity test, which is the target speaker's speech.
107 |
108 |
109 |
112 |
113 |
114 |
115 | Register Voice
116 | D-vector Sample
117 |
118 |
119 |
120 |
121 | Female Speaker1
122 |
123 |
125 |
127 |
128 | Female Speaker2
129 |
130 |
132 |
134 |
135 | Male Speaker1
136 |
137 |
139 |
141 |
142 | Male Speaker2
143 |
144 |
146 |
148 |
149 |
150 |
151 | Note: All samples are in Mandrin Chinese.
158 | There are 6 in-set speakers---3 male speakers and 3 female speakers, shown here.
160 | The "Reference Voice" is shown here for timbre similarity test, which is the target speaker's speech.
162 | "Speech only" means training only with speech data, while "Speech & Singing" means training with speech data and other singers' singing data.
164 |
165 |
169 |
170 |
171 |
172 | Reference Voice
173 | Speech Only
174 | Speechh & Singing
175 |
176 |
177 |
178 |
179 | Male Speaker1
180 |
181 |
183 |
185 |
187 |
188 | Male Speaker2
189 |
190 |
192 |
194 |
196 |
197 | Male Speaker3
198 |
199 |
201 |
203 |
205 |
206 | Female Speaker1
207 |
208 |
210 |
212 |
214 |
215 | Female Speaker2
216 |
217 |
219 |
221 |
223 |
224 | Female Speaker3
225 |
226 |
228 |
230 |
232 |
233 |
234 |
235 |