├── .gitignore
├── LICENSE
├── README.md
├── audio.py
├── dataset.py
├── distributions.py
├── hparams.py
├── inputs
    └── sample.wav
├── loss_function.py
├── lrschedule.py
├── model.py
├── preprocess.py
├── requirements.txt
├── train.py
└── utils.py


/.gitignore:
--------------------------------------------------------------------------------
  1 | # Byte-compiled / optimized / DLL files
  2 | __pycache__/
  3 | *.py[cod]
  4 | *$py.class
  5 | 
  6 | # C extensions
  7 | *.so
  8 | 
  9 | # Distribution / packaging
 10 | .Python
 11 | build/
 12 | develop-eggs/
 13 | dist/
 14 | downloads/
 15 | eggs/
 16 | .eggs/
 17 | lib/
 18 | lib64/
 19 | parts/
 20 | sdist/
 21 | var/
 22 | wheels/
 23 | *.egg-info/
 24 | .installed.cfg
 25 | *.egg
 26 | MANIFEST
 27 | 
 28 | # PyInstaller
 29 | #  Usually these files are written by a python script from a template
 30 | #  before PyInstaller builds the exe, so as to inject date/other infos into it.
 31 | *.manifest
 32 | *.spec
 33 | 
 34 | # Installer logs
 35 | pip-log.txt
 36 | pip-delete-this-directory.txt
 37 | 
 38 | # Unit test / coverage reports
 39 | htmlcov/
 40 | .tox/
 41 | .coverage
 42 | .coverage.*
 43 | .cache
 44 | nosetests.xml
 45 | coverage.xml
 46 | *.cover
 47 | .hypothesis/
 48 | .pytest_cache/
 49 | 
 50 | # Translations
 51 | *.mo
 52 | *.pot
 53 | 
 54 | # Django stuff:
 55 | *.log
 56 | local_settings.py
 57 | db.sqlite3
 58 | 
 59 | # Flask stuff:
 60 | instance/
 61 | .webassets-cache
 62 | 
 63 | # Scrapy stuff:
 64 | .scrapy
 65 | 
 66 | # Sphinx documentation
 67 | docs/_build/
 68 | 
 69 | # PyBuilder
 70 | target/
 71 | 
 72 | # Jupyter Notebook
 73 | .ipynb_checkpoints
 74 | 
 75 | # pyenv
 76 | .python-version
 77 | 
 78 | # celery beat schedule file
 79 | celerybeat-schedule
 80 | 
 81 | # SageMath parsed files
 82 | *.sage.py
 83 | 
 84 | # Environments
 85 | .env
 86 | .venv
 87 | env/
 88 | venv/
 89 | ENV/
 90 | env.bak/
 91 | venv.bak/
 92 | 
 93 | # Spyder project settings
 94 | .spyderproject
 95 | .spyproject
 96 | 
 97 | # Rope project settings
 98 | .ropeproject
 99 | 
100 | # mkdocs documentation
101 | /site
102 | 
103 | # mypy
104 | .mypy_cache/
105 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2018 Gary Wang
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # WaveRNN-Pytorch
 2 | This repository contains Fatcord's [Alternative](https://github.com/fatchord/WaveRNN) WaveRNN (Faster training), which contains a fast-training, small GPU memory implementation of WaveRNN vocoder.
 3 | 
 4 | # Model Pruning and Real Time CPU Inference
 5 | See geneing's awesome fork that has model pruning, export to C++ and real time inference on CPU: https://github.com/geneing/WaveRNN-Pytorch.
 6 | 
 7 | 
 8 | # Highlights
 9 | * support raw audio wav modelling (via a single Beta Distribution)
10 | * relatively fast synthesis speed without much optimization yet (around 2000 samples/sec on GTX 1060 Ti, 16 GB ram, i5 processor)
11 | * support Fatcord's original quantized (9-bit) wav modelling
12 | 
13 | # Audio Samples
14 | 1. [Obama & Bernie Sanders](https://soundcloud.com/gary-wang-23/sets/obama_bernie_fun) See this repo in action!
15 | 
16 | 2. [10-bit audio](https://soundcloud.com/gary-wang-23/sets/wavernn-pytorch-10-bit-raw-audio-200k) on held-out testing data from LJSpeech. This model sounds and trains pretty close to 9 bit. We want the higher bit the better.
17 | 
18 | 3. [9-bit audio](https://soundcloud.com/gary-wang-23/sets/wave_rnn_9_bit_11k_step) on held-out testing data from LJSpeech. This model trains the fastest (this is around 130 epochs)
19 | 
20 | 4. [Single beta distribution](https://soundcloud.com/gary-wang-23/sets/wavernn-samples) on held-out testing data from LjSpeech. This is trained with the single Beta distribution.
21 | 
22 | # Pretrained Checkpoints
23 | 1. [Single Beta Distribution](https://drive.google.com/open?id=138i0MtEkDqLM6fmBniQloEMtMlCHgJha) trained for 112k. Make sure to change `hparams.input_type` to `raw`.
24 | 2. [9-bit quantized audio](https://drive.google.com/open?id=114Xk3P9dD-_e2W8jmiKSpOX1UGb7qem3) trained for 11k, or around 130 epochs, can be trained further. Make sure to change `hparams.input_type` to `bits`.
25 | 3. [10-bit quantized audio](https://drive.google.com/open?id=1djWm62tHIndopyS5spkHf68lI6-h5a3H). To ensure your model is built properly, download the `hparams.py` [here](https://drive.google.com/open?id=1nXSW4u01bEbUkRW4Vd3IQ6soBAXPg6aw), either replace this with your local `hparams.py` file or note and update any changes.
26 | 
27 | 
28 | 
29 | 
30 | # Requirements
31 | * Python 3
32 | * CUDA >=8.0
33 | * PyTorch >= v0.4.1
34 | 
35 | # Installation
36 | Ensure above requirements are met.
37 | 
38 | ```
39 | git clone https://github.com/G-Wang/WaveRNN-Pytorch.git
40 | cd WaveRNN-Pytorch
41 | pip install -r requirements.txt
42 | ```
43 | 
44 | # Usage
45 | ## 1. Adjusting Hyperparameters
46 | Before running scripts, one can adjust hyperparameters in `hparams.py`.
47 | 
48 | Some hyperparameters that you might want to adjust:
49 | * `fix_learning_rate` The model is robust enough to learn well with a fix learning rate of `1e-4`, I suggest you try this setting for fastest training, you can decrease this down to `5e-6` for final step refinement. Set this to `None` to train with learning rate schedule instead
50 | * `input_type` (best performing ones are currently `bits` and `raw`, see `hparams.py` for more details)
51 | * `batch_size`
52 | * `save_every_step` (checkpoint saving frequency)
53 | * `evaluate_every_step` (evaluation frequency)
54 | * `seq_len_factor` (sequence length of training audio, the longer the more GPU it takes)
55 | ## 2. Preprocessing
56 | This function processes raw wav files into corresponding mel-spectrogram and wav files according to the audio processing hyperparameters.
57 | 
58 | Example usage:
59 | ```
60 | python preprocess.py /path/to/my/wav/files
61 | ```
62 | This will process all the `.wav` files in the folder `/path/to/my/wav/files` and save them in the default local directory called `data_dir`.
63 | 
64 | Can include `--output_dir` to specify a specific directory to store the processed outputs.
65 | 
66 | ## 3. Training
67 | Start training process. checkpoints are by default stored in the local directory `checkpoints`.
68 | The script will automatically save a checkpoint when terminated by `crtl + c`.
69 | 
70 | 
71 | Example 1: starting a new model for training
72 | ```
73 | python train.py data_dir
74 | ```
75 | `data_dir` is the directory containing the processed files.
76 | 
77 | Example 2: Restoring training from checkpoint
78 | ```
79 | python train.py data_dir --checkpoint=checkpoints/checkpoint0010000.pth
80 | ```
81 | Evaluation `.wav` files and plots are saved in `checkpoints/eval`.
82 | 
83 | # WIP
84 | - [ ] optimize learning rate schedule
85 | - [ ] optimize training hyperparameters (seq_len and batch_size)
86 | - [ ] batch generation for synthesis speedup
87 | - [ ] model pruning
88 | 
89 | 
90 | 
91 | 
92 | 
93 | 
94 | 
95 | 
96 | 


--------------------------------------------------------------------------------
/audio.py:
--------------------------------------------------------------------------------
  1 | import librosa
  2 | import librosa.filters
  3 | import math
  4 | import numpy as np
  5 | from scipy import signal
  6 | from hparams import hparams
  7 | from scipy.io import wavfile
  8 | 
  9 | # r9r9 preprocessing
 10 | import lws
 11 | 
 12 | 
 13 | def load_wav(path):
 14 |     return librosa.load(path, sr=hparams.sample_rate)[0]
 15 | 
 16 | def save_wav(wav, path):
 17 |     wav = wav * 32767 / max(0.01, np.max(np.abs(wav)))
 18 |     wavfile.write(path, hparams.sample_rate, wav.astype(np.int16))
 19 | 
 20 | 
 21 | def preemphasis(x):
 22 |     from nnmnkwii.preprocessing import preemphasis
 23 |     return preemphasis(x, hparams.preemphasis)
 24 | 
 25 | 
 26 | def inv_preemphasis(x):
 27 |     from nnmnkwii.preprocessing import inv_preemphasis
 28 |     return inv_preemphasis(x, hparams.preemphasis)
 29 | 
 30 | 
 31 | def spectrogram(y):
 32 |     D = _lws_processor().stft(preemphasis(y)).T
 33 |     S = _amp_to_db(np.abs(D)) - hparams.ref_level_db
 34 |     return _normalize(S)
 35 | 
 36 | 
 37 | def inv_spectrogram(spectrogram):
 38 |     '''Converts spectrogram to waveform using librosa'''
 39 |     S = _db_to_amp(_denormalize(spectrogram) + hparams.ref_level_db)  # Convert back to linear
 40 |     processor = _lws_processor()
 41 |     D = processor.run_lws(S.astype(np.float64).T ** hparams.power)
 42 |     y = processor.istft(D).astype(np.float32)
 43 |     return inv_preemphasis(y)
 44 | 
 45 | 
 46 | def melspectrogram(y):
 47 |     D = _lws_processor().stft(preemphasis(y)).T
 48 |     S = _amp_to_db(_linear_to_mel(np.abs(D))) - hparams.ref_level_db
 49 |     if not hparams.allow_clipping_in_normalization:
 50 |         assert S.max() <= 0 and S.min() - hparams.min_level_db >= 0
 51 |     return _normalize(S)
 52 | 
 53 | 
 54 | def _lws_processor():
 55 |     return lws.lws(hparams.fft_size, hparams.hop_size, mode="speech")
 56 | 
 57 | 
 58 | # Conversions:
 59 | 
 60 | 
 61 | _mel_basis = None
 62 | 
 63 | 
 64 | def _linear_to_mel(spectrogram):
 65 |     global _mel_basis
 66 |     if _mel_basis is None:
 67 |         _mel_basis = _build_mel_basis()
 68 |     return np.dot(_mel_basis, spectrogram)
 69 | 
 70 | 
 71 | def _build_mel_basis():
 72 |     if hparams.fmax is not None:
 73 |         assert hparams.fmax <= hparams.sample_rate // 2
 74 |     return librosa.filters.mel(hparams.sample_rate, hparams.fft_size,
 75 |                                fmin=hparams.fmin, fmax=hparams.fmax,
 76 |                                n_mels=hparams.num_mels)
 77 | 
 78 | 
 79 | def _amp_to_db(x):
 80 |     min_level = np.exp(hparams.min_level_db / 20 * np.log(10))
 81 |     return 20 * np.log10(np.maximum(min_level, x))
 82 | 
 83 | 
 84 | def _db_to_amp(x):
 85 |     return np.power(10.0, x * 0.05)
 86 | 
 87 | 
 88 | def _normalize(S):
 89 |     return np.clip((S - hparams.min_level_db) / -hparams.min_level_db, 0, 1)
 90 | 
 91 | 
 92 | def _denormalize(S):
 93 |     return (np.clip(S, 0, 1) * -hparams.min_level_db) + hparams.min_level_db
 94 | 
 95 | 
 96 | # Fatcord's preprocessing
 97 | def quantize(x):
 98 |     """quantize audio signal
 99 | 
100 |     """
101 |     quant = (x + 1.) * (2**hparams.bits - 1) / 2
102 |     return quant.astype(np.int)
103 | 
104 | 
105 | # testing
106 | def test_everything():
107 |     wav = np.random.randn(12000,)
108 |     mel = melspectrogram(wav)
109 |     spec = spectrogram(wav)
110 |     quant = quantize(wav)
111 |     print(wav.shape, mel.shape, spec.shape, quant.shape)
112 |     print(quant.max(), quant.min(), mel.max(), mel.min(), spec.max(), spec.min())


--------------------------------------------------------------------------------
/dataset.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | 
  3 | import os
  4 | 
  5 | import torch
  6 | from torch.utils.data import DataLoader, Dataset
  7 | from hparams import hparams as hp
  8 | from utils import mulaw_quantize, inv_mulaw_quantize
  9 | import pickle
 10 | 
 11 | 
 12 | class AudiobookDataset(Dataset):
 13 |     def __init__(self, data_path):
 14 |         self.path = os.path.join(data_path, "")
 15 |         with open(os.path.join(self.path,'dataset_ids.pkl'), 'rb') as f:
 16 |             self.metadata = pickle.load(f)
 17 |         self.mel_path = os.path.join(data_path, "mel")
 18 |         self.wav_path = os.path.join(data_path, "wav")
 19 |         self.test_path = os.path.join(data_path, "test")
 20 |         
 21 |     def __getitem__(self, index):
 22 |         file = self.metadata[index]
 23 |         m = np.load(os.path.join(self.mel_path,'{}.npy'.format(file)))
 24 |         x = np.load(os.path.join(self.wav_path,'{}.npy'.format(file)))
 25 |         return m, x
 26 | 
 27 |     def __len__(self):
 28 |         return len(self.metadata)
 29 | 
 30 | 
 31 | def raw_collate(batch) :
 32 |     """collate function used for raw wav forms, such as using beta/guassian/mixture of logistic
 33 |     """
 34 |     
 35 |     pad = 2
 36 |     mel_win = hp.seq_len // hp.hop_size + 2 * pad
 37 |     max_offsets = [x[0].shape[-1] - (mel_win + 2 * pad) for x in batch]
 38 |     mel_offsets = [np.random.randint(0, offset) for offset in max_offsets]
 39 |     sig_offsets = [(offset + pad) * hp.hop_size for offset in mel_offsets]
 40 |     
 41 |     mels = [x[0][:, mel_offsets[i]:mel_offsets[i] + mel_win] \
 42 |             for i, x in enumerate(batch)]
 43 |     
 44 |     coarse = [x[1][sig_offsets[i]:sig_offsets[i] + hp.seq_len + 1] \
 45 |               for i, x in enumerate(batch)]
 46 |     
 47 |     mels = np.stack(mels).astype(np.float32)
 48 |     coarse = np.stack(coarse).astype(np.float32)
 49 |     
 50 |     mels = torch.FloatTensor(mels)
 51 |     coarse = torch.FloatTensor(coarse)
 52 |     
 53 |     x_input = coarse[:,:hp.seq_len]
 54 |     
 55 |     y_coarse = coarse[:, 1:]
 56 |     
 57 |     return x_input, mels, y_coarse
 58 | 
 59 | 
 60 | 
 61 | def discrete_collate(batch) :
 62 |     """collate function used for discrete wav output, such as 9-bit, mulaw-discrete, etc.
 63 |     """
 64 |     
 65 |     pad = 2
 66 |     mel_win = hp.seq_len // hp.hop_size + 2 * pad
 67 |     max_offsets = [x[0].shape[-1] - (mel_win + 2 * pad) for x in batch]
 68 |     mel_offsets = [np.random.randint(0, offset) for offset in max_offsets]
 69 |     sig_offsets = [(offset + pad) * hp.hop_size for offset in mel_offsets]
 70 |     
 71 |     mels = [x[0][:, mel_offsets[i]:mel_offsets[i] + mel_win] \
 72 |             for i, x in enumerate(batch)]
 73 |     
 74 |     coarse = [x[1][sig_offsets[i]:sig_offsets[i] + hp.seq_len + 1] \
 75 |               for i, x in enumerate(batch)]
 76 |     
 77 |     mels = np.stack(mels).astype(np.float32)
 78 |     coarse = np.stack(coarse).astype(np.int64)
 79 |     
 80 |     mels = torch.FloatTensor(mels)
 81 |     coarse = torch.LongTensor(coarse)
 82 |     if hp.input_type == 'bits':
 83 |         x_input = 2 * coarse[:, :hp.seq_len].float() / (2**hp.bits - 1.) - 1.
 84 |     elif hp.input_type == 'mulaw':
 85 |         x_input = inv_mulaw_quantize(coarse[:, :hp.seq_len], hp.mulaw_quantize_channels)
 86 |     
 87 |     y_coarse = coarse[:, 1:]
 88 |     
 89 |     return x_input, mels, y_coarse
 90 | 
 91 | 
 92 | def no_test_raw_collate():
 93 |     import matplotlib.pyplot as plt
 94 |     from test_utils import plot, plot_spec
 95 |     data_id_path = "data_dir/"
 96 |     data_path = "data_dir/"
 97 |     print(hp.seq_len)
 98 |     
 99 |     with open('{}dataset_ids.pkl'.format(data_id_path), 'rb') as f:
100 |         dataset_ids = pickle.load(f)
101 |     dataset = AudiobookDataset(data_path)
102 |     print(len(dataset))
103 | 
104 |     data_loader = DataLoader(dataset, collate_fn=raw_collate, batch_size=32, 
105 |                          num_workers=0, shuffle=True)
106 | 
107 |     x, m, y = next(iter(data_loader))
108 |     print(x.shape, m.shape, y.shape)
109 |     plot(x.numpy()[0]) 
110 |     plot(y.numpy()[0])
111 |     plot_spec(m.numpy()[0])
112 | 
113 | 
114 | def test_discrete_collate():
115 |     import matplotlib.pyplot as plt
116 |     from test_utils import plot, plot_spec
117 |     data_id_path = "data_dir/"
118 |     data_path = "data_dir/"
119 |     print(hp.seq_len)
120 |     
121 |     with open('{}dataset_ids.pkl'.format(data_id_path), 'rb') as f:
122 |         dataset_ids = pickle.load(f)
123 |     dataset = AudiobookDataset(data_path)
124 |     print(len(dataset))
125 | 
126 |     data_loader = DataLoader(dataset, collate_fn=discrete_collate, batch_size=32, 
127 |                          num_workers=0, shuffle=True)
128 | 
129 |     x, m, y = next(iter(data_loader))
130 |     print(x.shape, m.shape, y.shape)
131 |     plot(x.numpy()[0]) 
132 |     plot(y.numpy()[0])
133 |     plot_spec(m.numpy()[0])
134 | 
135 | 
136 | 
137 | def no_test_dataset():
138 |     data_id_path = "data_dir/"
139 |     data_path = "data_dir/"
140 |     print(hp.seq_len)
141 |     dataset = AudiobookDataset(data_path)
142 | 


--------------------------------------------------------------------------------
/distributions.py:
--------------------------------------------------------------------------------
  1 | import math
  2 | import numpy as np
  3 | 
  4 | import torch
  5 | from torch import nn
  6 | from torch.nn import functional as F
  7 | from torch.distributions import Beta, Normal
  8 | from hparams import hparams as hp
  9 | 
 10 | def sample_from_beta_dist(y_hat):
 11 |     """
 12 |     y_hat (batch_size x seq_len x 2):
 13 |     
 14 |     """
 15 |     # take exponentional to ensure positive
 16 |     loc_y = y_hat.exp()
 17 |     alpha = loc_y[:,:,0].unsqueeze(-1)
 18 |     beta = loc_y[:,:,1].unsqueeze(-1)
 19 |     dist = Beta(alpha, beta)
 20 |     sample = dist.sample()
 21 |     # rescale sample from [0,1] to [-1, 1]
 22 |     sample = 2.0*sample-1.0
 23 |     return sample
 24 | 
 25 | 
 26 | def beta_mle_loss(y_hat, y, reduce=True):
 27 |     """y_hat (batch_size x seq_len x 2)
 28 |         y (batch_size x seq_len x 1)
 29 |         
 30 |     """
 31 |     # take exponentional to ensure positive
 32 |     loc_y = y_hat.exp()
 33 |     alpha = loc_y[:,:,0].unsqueeze(-1)
 34 |     beta = loc_y[:,:,1].unsqueeze(-1)
 35 |     dist = Beta(alpha, beta)
 36 |     # rescale y to be between 
 37 |     y = (y + 1.0)/2.0
 38 |     # note that we will get inf loss if y == 0 or 1.0 exactly, so we will clip it slightly just in case
 39 |     y = torch.clamp(y, 1e-5, 0.99999)
 40 |     # compute logprob
 41 |     loss = -dist.log_prob(y).squeeze(-1)
 42 |     if reduce:
 43 |         return loss.mean()
 44 |     else:
 45 |         return loss
 46 | 
 47 | 
 48 | def log_sum_exp(x):
 49 |     """ numerically stable log_sum_exp implementation that prevents overflow """
 50 |     # TF ordering
 51 |     axis = len(x.size()) - 1
 52 |     m, _ = torch.max(x, dim=axis)
 53 |     m2, _ = torch.max(x, dim=axis, keepdim=True)
 54 |     return m + torch.log(torch.sum(torch.exp(x - m2), dim=axis))
 55 | 
 56 | 
 57 | def discretized_mix_logistic_loss(y_hat, y, num_classes=256,
 58 |                                   log_scale_min=hp.log_scale_min, reduce=True):
 59 |     """Discretized mixture of logistic distributions loss
 60 | 
 61 |     Note that it is assumed that input is scaled to [-1, 1].
 62 | 
 63 |     Args:
 64 |         y_hat (Tensor): Predicted output (B x T x C)
 65 |         y (Tensor): Target (B x T x 1).
 66 |         num_classes (int): Number of classes
 67 |         log_scale_min (float): Log scale minimum value
 68 |         reduce (bool): If True, the losses are averaged or summed for each
 69 |           minibatch.
 70 | 
 71 |     Returns
 72 |         Tensor: loss
 73 |     """
 74 |     y_hat = y_hat.permute(0,2,1)
 75 |     assert y_hat.dim() == 3
 76 |     assert y_hat.size(1) % 3 == 0
 77 |     nr_mix = y_hat.size(1) // 3
 78 | 
 79 |     # (B x T x C)
 80 |     y_hat = y_hat.transpose(1, 2)
 81 | 
 82 |     # unpack parameters. (B, T, num_mixtures) x 3
 83 |     logit_probs = y_hat[:, :, :nr_mix]
 84 |     means = y_hat[:, :, nr_mix:2 * nr_mix]
 85 |     log_scales = torch.clamp(y_hat[:, :, 2 * nr_mix:3 * nr_mix], min=log_scale_min)
 86 | 
 87 |     # B x T x 1 -> B x T x num_mixtures
 88 |     y = y.expand_as(means)
 89 | 
 90 |     centered_y = y - means
 91 |     inv_stdv = torch.exp(-log_scales)
 92 |     plus_in = inv_stdv * (centered_y + 1. / (num_classes - 1))
 93 |     cdf_plus = torch.sigmoid(plus_in)
 94 |     min_in = inv_stdv * (centered_y - 1. / (num_classes - 1))
 95 |     cdf_min = torch.sigmoid(min_in)
 96 | 
 97 |     # log probability for edge case of 0 (before scaling)
 98 |     # equivalent: torch.log(F.sigmoid(plus_in))
 99 |     log_cdf_plus = plus_in - F.softplus(plus_in)
100 | 
101 |     # log probability for edge case of 255 (before scaling)
102 |     # equivalent: (1 - F.sigmoid(min_in)).log()
103 |     log_one_minus_cdf_min = -F.softplus(min_in)
104 | 
105 |     # probability for all other cases
106 |     cdf_delta = cdf_plus - cdf_min
107 | 
108 |     mid_in = inv_stdv * centered_y
109 |     # log probability in the center of the bin, to be used in extreme cases
110 |     # (not actually used in our code)
111 |     log_pdf_mid = mid_in - log_scales - 2. * F.softplus(mid_in)
112 | 
113 |     # tf equivalent
114 |     """
115 |     log_probs = tf.where(x < -0.999, log_cdf_plus,
116 |                          tf.where(x > 0.999, log_one_minus_cdf_min,
117 |                                   tf.where(cdf_delta > 1e-5,
118 |                                            tf.log(tf.maximum(cdf_delta, 1e-12)),
119 |                                            log_pdf_mid - np.log(127.5))))
120 |     """
121 |     # TODO: cdf_delta <= 1e-5 actually can happen. How can we choose the value
122 |     # for num_classes=65536 case? 1e-7? not sure..
123 |     inner_inner_cond = (cdf_delta > 1e-5).float()
124 | 
125 |     inner_inner_out = inner_inner_cond * \
126 |         torch.log(torch.clamp(cdf_delta, min=1e-12)) + \
127 |         (1. - inner_inner_cond) * (log_pdf_mid - np.log((num_classes - 1) / 2))
128 |     inner_cond = (y > 0.999).float()
129 |     inner_out = inner_cond * log_one_minus_cdf_min + (1. - inner_cond) * inner_inner_out
130 |     cond = (y < -0.999).float()
131 |     log_probs = cond * log_cdf_plus + (1. - cond) * inner_out
132 | 
133 |     log_probs = log_probs + F.log_softmax(logit_probs, -1)
134 | 
135 |     if reduce:
136 |         return -torch.sum(log_sum_exp(log_probs))
137 |     else:
138 |         return -log_sum_exp(log_probs).unsqueeze(-1)
139 | 
140 | 
141 | def to_one_hot(tensor, n, fill_with=1.):
142 |     # we perform one hot encore with respect to the last axis
143 |     one_hot = torch.FloatTensor(tensor.size() + (n,)).zero_()
144 |     if tensor.is_cuda:
145 |         one_hot = one_hot.cuda()
146 |     one_hot.scatter_(len(tensor.size()), tensor.unsqueeze(-1), fill_with)
147 |     return one_hot
148 | 
149 | 
150 | def sample_from_discretized_mix_logistic(y, log_scale_min=hp.log_scale_min):
151 |     """
152 |     Sample from discretized mixture of logistic distributions
153 | 
154 |     Args:
155 |         y (Tensor): B x C x T
156 |         log_scale_min (float): Log scale minimum value
157 | 
158 |     Returns:
159 |         Tensor: sample in range of [-1, 1].
160 |     """
161 |     assert y.size(1) % 3 == 0
162 |     nr_mix = y.size(1) // 3
163 | 
164 |     # B x T x C
165 |     y = y.transpose(1, 2)
166 |     logit_probs = y[:, :, :nr_mix]
167 | 
168 |     # sample mixture indicator from softmax
169 |     temp = logit_probs.data.new(logit_probs.size()).uniform_(1e-5, 1.0 - 1e-5)
170 |     temp = logit_probs.data - torch.log(- torch.log(temp))
171 |     _, argmax = temp.max(dim=-1)
172 | 
173 |     # (B, T) -> (B, T, nr_mix)
174 |     one_hot = to_one_hot(argmax, nr_mix)
175 |     # select logistic parameters
176 |     means = torch.sum(y[:, :, nr_mix:2 * nr_mix] * one_hot, dim=-1)
177 |     log_scales = torch.clamp(torch.sum(
178 |         y[:, :, 2 * nr_mix:3 * nr_mix] * one_hot, dim=-1), min=log_scale_min)
179 |     # sample from logistic & clip to interval
180 |     # we don't actually round to the nearest 8bit value when sampling
181 |     u = means.data.new(means.size()).uniform_(1e-5, 1.0 - 1e-5)
182 |     x = means + torch.exp(log_scales) * (torch.log(u) - torch.log(1. - u))
183 | 
184 |     x = torch.clamp(torch.clamp(x, min=-1.), max=1.)
185 | 
186 |     return x
187 | 
188 | 
189 | # add gaussian from clarinet implementation:https://raw.githubusercontent.com/ksw0306/ClariNet/master/loss.py
190 | def gaussian_loss(y_hat, y, log_std_min=-7.0, reduce=True):
191 |     """y_hat (batch_size x seq_len x 2)
192 |         y (batch_size x seq_len x 1)
193 |     """
194 |     assert y_hat.dim() == 3
195 |     assert y_hat.size(2) == 2
196 | 
197 |     mean = y_hat[:, :, :1]
198 |     log_std = torch.clamp(y_hat[:, :, 1:], min=log_std_min)
199 | 
200 |     log_probs = -0.5 * (- math.log(2.0 * math.pi) - 2. * log_std - torch.pow(y - mean, 2) * torch.exp((-2.0 * log_std)))
201 |     
202 |     if reduce:
203 |         return log_probs.squeeze().mean()
204 |     else:
205 |         return log_probs.squeeze()
206 | 
207 | 
208 | def sample_from_gaussian(y_hat, log_std_min=-7.0, scale_factor=1.):
209 |     """y_hat (batch_size x seq_len x 2)
210 |         y (batch_size x seq_len x 1)
211 |     """
212 |     assert y_hat.size(2) == 2
213 | 
214 |     mean = y_hat[:, :, :1]
215 |     log_std = torch.clamp(y_hat[:, :, 1:], min=log_std_min)
216 |     dist = Normal(mean, torch.exp(log_std))
217 |     sample = dist.sample()
218 |     sample = torch.clamp(torch.clamp(sample, min=-scale_factor), max=scale_factor)
219 |     del dist
220 |     return sample
221 | 
222 | 
223 | 
224 | 
225 | 
226 | def test_gaussian():
227 | 
228 |     y_hat = torch.rand(16, 120, 2)
229 |     y_true = torch.rand(16, 120, 1)
230 |     out = sample_from_gaussian(y_hat)
231 |     loss = gaussian_loss(y_hat, y_true)
232 |     loss_mean = loss.mean()
233 |     print(out.shape, loss.shape, loss_mean.item())


--------------------------------------------------------------------------------
/hparams.py:
--------------------------------------------------------------------------------
 1 | class hparams:
 2 | 
 3 |     # option parameters
 4 | 
 5 |     # Input type:
 6 |     # 1. raw [-1, 1]
 7 |     # 2. mixture [-1, 1]
 8 |     # 3. bits [0, 512]
 9 |     # 4. mulaw[0, mulaw_quantize_channels]
10 |     #
11 |     input_type = 'raw'
12 |     #
13 |     # distribution type, currently supports only 'beta' and 'mixture'
14 |     distribution = 'gaussian' # or "mixture"
15 |     log_scale_min = -32.23619130191664 # = float(np.log(1e-7))
16 |     quantize_channels = 65536 # quantize channel used for compute loss for mixture of logistics
17 |     #
18 |     # for Fatcord's original 9 bit audio, specify the audio bit rate. Note this corresponds to network output
19 |     # of size 2**bits, so 9 bits would be 512 output, etc.
20 |     bits = 10
21 |     # for mu-law
22 |     mulaw_quantize_channels = 512
23 |     # note: r9r9's deepvoice3 preprocessing is used instead of Fatcord's original.
24 |     #--------------     
25 |     # audio processing parameters
26 |     num_mels = 80
27 |     fmin = 125
28 |     fmax = 7600
29 |     fft_size = 1024
30 |     hop_size = 256
31 |     win_length = 1024
32 |     sample_rate = 22050
33 |     preemphasis = 0.97
34 |     min_level_db = -100
35 |     ref_level_db = 20
36 |     rescaling = False
37 |     rescaling_max = 0.999
38 |     allow_clipping_in_normalization = True
39 |     #----------------
40 |     #
41 |     #----------------
42 |     # model parameters
43 |     rnn_dims = 600
44 |     fc_dims = 512
45 |     pad = 2
46 |     # note upsample factors must multiply out to be equal to hop_size, so adjust
47 |     # if necessary (i.e 4 x 4 x 16 = 256)
48 |     upsample_factors = (4, 4, 16)
49 |     compute_dims = 128
50 |     res_out_dims = 128
51 |     res_blocks = 10
52 |     #----------------
53 |     #
54 |     #----------------
55 |     # training parameters
56 |     batch_size = 32
57 |     nepochs = 5000
58 |     save_every_step = 10000
59 |     evaluate_every_step = 5000
60 |     # seq_len_factor can be adjusted to increase training sequence length (will increase GPU usage)
61 |     seq_len_factor = 5
62 |     seq_len = seq_len_factor * hop_size
63 |     grad_norm = 10
64 |     #learning rate parameters
65 |     initial_learning_rate=1e-3
66 |     lr_schedule_type = 'step' # or 'noam'
67 |     # for noam learning rate schedule
68 |     noam_warm_up_steps = 2000 * (batch_size // 16)
69 |     # for step learning rate schedule
70 |     step_gamma = 0.5
71 |     lr_step_interval = 15000
72 | 
73 |     adam_beta1=0.9
74 |     adam_beta2=0.999
75 |     adam_eps=1e-8
76 |     amsgrad=False
77 |     weight_decay = 0.0
78 |     fix_learning_rate = None # modify if one wants to use a fixed learning rate, else set to None to use noam learning rate
79 |     #-----------------
80 | 


--------------------------------------------------------------------------------
/inputs/sample.wav:
--------------------------------------------------------------------------------
  1 | 
  2 | 
  3 | 
  4 | 
  5 | 
  6 | 
  7 | <!DOCTYPE html>
  8 | <html lang="en">
  9 |   <head>
 10 |     <meta charset="utf-8">
 11 |   <link rel="dns-prefetch" href="https://assets-cdn.github.com">
 12 |   <link rel="dns-prefetch" href="https://avatars0.githubusercontent.com">
 13 |   <link rel="dns-prefetch" href="https://avatars1.githubusercontent.com">
 14 |   <link rel="dns-prefetch" href="https://avatars2.githubusercontent.com">
 15 |   <link rel="dns-prefetch" href="https://avatars3.githubusercontent.com">
 16 |   <link rel="dns-prefetch" href="https://github-cloud.s3.amazonaws.com">
 17 |   <link rel="dns-prefetch" href="https://user-images.githubusercontent.com/">
 18 | 
 19 | 
 20 | 
 21 |   <link crossorigin="anonymous" media="all" integrity="sha512-Z0JAar9+DkI1NjGVdZr3GivARUgJtA0o2eHlTv7Ou2gshR5awWVf8QGsq11Ns9ZxQLEs+G5/SuARmvpOLMzulw==" rel="stylesheet" href="https://assets-cdn.github.com/assets/frameworks-95aff0b550d3fe338b645a4deebdcb1b.css" />
 22 |   <link crossorigin="anonymous" media="all" integrity="sha512-h5cEqWTuBT7ANPGSQLt1mH+ozRnf2uZHIo5hzaBUEaFGGVZkq/aXrTxFNXPfCm9ir2ztHtlW4AAMl2IxBKc1pQ==" rel="stylesheet" href="https://assets-cdn.github.com/assets/github-e6bb18b320358b77abe040d2eb46b547.css" />
 23 |   
 24 |   
 25 |   <link crossorigin="anonymous" media="all" integrity="sha512-/YdlYmXBl0rVnTX/ANGx7YStBlWdlnfFUtRTlTr8eBpzeBkOUj1GjReYy1/BdJ4wMwOXzQEri+e91nqx5xrwGg==" rel="stylesheet" href="https://assets-cdn.github.com/assets/site-d3b91707396b15f2783f5f891bc6d36e.css" />
 26 |   
 27 | 
 28 |   <meta name="viewport" content="width=device-width">
 29 |   
 30 |   <title>some_files/20180510_mixture_lj_checkpoint_step000320000_ema.wav at master · G-Wang/some_files · GitHub</title>
 31 |     <meta name="description" content="GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects.">
 32 |     <link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="GitHub">
 33 |   <link rel="fluid-icon" href="https://github.com/fluidicon.png" title="GitHub">
 34 |   <meta property="fb:app_id" content="1401488693436528">
 35 | 
 36 |     
 37 |     <meta property="og:image" content="https://avatars0.githubusercontent.com/u/14854468?s=400&amp;v=4" /><meta property="og:site_name" content="GitHub" /><meta property="og:type" content="object" /><meta property="og:title" content="G-Wang/some_files" /><meta property="og:url" content="https://github.com/G-Wang/some_files" /><meta property="og:description" content="Contribute to some_files development by creating an account on Github." />
 38 | 
 39 |   <link rel="assets" href="https://assets-cdn.github.com/">
 40 |   
 41 |   <meta name="pjax-timeout" content="1000">
 42 |   
 43 |   <meta name="request-id" content="21FB:0709:20D7C93:3E15184:5B7336E4" data-pjax-transient>
 44 | 
 45 | 
 46 |   
 47 | 
 48 |   <meta name="selected-link" value="repo_source" data-pjax-transient>
 49 | 
 50 |     <meta name="google-site-verification" content="KT5gs8h0wvaagLKAVWq8bbeNwnZZK1r1XQysX3xurLU">
 51 |   <meta name="google-site-verification" content="ZzhVyEFwb7w3e0-uOTltm8Jsck2F5StVihD0exw2fsA">
 52 |   <meta name="google-site-verification" content="GXs5KoUUkNCoaAZn7wPN-t01Pywp9M3sEjnt_3_ZWPc">
 53 |     <meta name="google-analytics" content="UA-3769691-2">
 54 | 
 55 | <meta name="octolytics-host" content="collector.githubapp.com" /><meta name="octolytics-app-id" content="github" /><meta name="octolytics-event-url" content="https://collector.githubapp.com/github-external/browser_event" /><meta name="octolytics-dimension-request_id" content="21FB:0709:20D7C93:3E15184:5B7336E4" /><meta name="octolytics-dimension-region_edge" content="iad" /><meta name="octolytics-dimension-region_render" content="iad" />
 56 | <meta name="analytics-location" content="/&lt;user-name&gt;/&lt;repo-name&gt;/blob/show" data-pjax-transient="true" />
 57 | 
 58 | 
 59 | 
 60 | 
 61 | <meta class="js-ga-set" name="dimension1" content="Logged Out">
 62 | 
 63 | 
 64 |   
 65 | 
 66 |       <meta name="hostname" content="github.com">
 67 |     <meta name="user-login" content="">
 68 | 
 69 |       <meta name="expected-hostname" content="github.com">
 70 |     <meta name="js-proxy-site-detection-payload" content="Y2UwMThiOTY2NjQ4NzEwNjRkOWEwODA4OWUwZGVhYTBhODc2NmY3ZGYxNjYxYWNiYmI3ZGJkZjFmNDAxMDYxNnx7InJlbW90ZV9hZGRyZXNzIjoiMjQuMjQ0LjIzLjE5NCIsInJlcXVlc3RfaWQiOiIyMUZCOjA3MDk6MjBEN0M5MzozRTE1MTg0OjVCNzMzNkU0IiwidGltZXN0YW1wIjoxNTM0Mjc3MzQ4LCJob3N0IjoiZ2l0aHViLmNvbSJ9">
 71 | 
 72 |     <meta name="enabled-features" content="DASHBOARD_V2_LAYOUT_OPT_IN,EXPLORE_DISCOVER_REPOSITORIES,UNIVERSE_BANNER,FREE_TRIALS,MARKETPLACE_INSIGHTS,MARKETPLACE_PLAN_RESTRICTION_EDITOR,MARKETPLACE_SEARCH,MARKETPLACE_INSIGHTS_CONVERSION_PERCENTAGES">
 73 | 
 74 |   <meta name="html-safe-nonce" content="e32ab29a4e2c907fbfc3541eff514c8faafada31">
 75 | 
 76 |   <meta http-equiv="x-pjax-version" content="3d6f5e3970a8b885abb36c14c74e9b9c">
 77 |   
 78 | 
 79 |       <link href="https://github.com/G-Wang/some_files/commits/master.atom" rel="alternate" title="Recent Commits to some_files:master" type="application/atom+xml">
 80 | 
 81 |   <meta name="go-import" content="github.com/G-Wang/some_files git https://github.com/G-Wang/some_files.git">
 82 | 
 83 |   <meta name="octolytics-dimension-user_id" content="14854468" /><meta name="octolytics-dimension-user_login" content="G-Wang" /><meta name="octolytics-dimension-repository_id" content="142387092" /><meta name="octolytics-dimension-repository_nwo" content="G-Wang/some_files" /><meta name="octolytics-dimension-repository_public" content="true" /><meta name="octolytics-dimension-repository_is_fork" content="false" /><meta name="octolytics-dimension-repository_network_root_id" content="142387092" /><meta name="octolytics-dimension-repository_network_root_nwo" content="G-Wang/some_files" /><meta name="octolytics-dimension-repository_explore_github_marketplace_ci_cta_shown" content="false" />
 84 | 
 85 | 
 86 |     <link rel="canonical" href="https://github.com/G-Wang/some_files/blob/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav" data-pjax-transient>
 87 | 
 88 | 
 89 |   <meta name="browser-stats-url" content="https://api.github.com/_private/browser/stats">
 90 | 
 91 |   <meta name="browser-errors-url" content="https://api.github.com/_private/browser/errors">
 92 | 
 93 |   <link rel="mask-icon" href="https://assets-cdn.github.com/pinned-octocat.svg" color="#000000">
 94 |   <link rel="icon" type="image/x-icon" class="js-site-favicon" href="https://assets-cdn.github.com/favicon.ico">
 95 | 
 96 | <meta name="theme-color" content="#1e2327">
 97 | 
 98 | 
 99 | 
100 | <link rel="manifest" href="/manifest.json" crossOrigin="use-credentials">
101 | 
102 |   </head>
103 | 
104 |   <body class="logged-out env-production page-blob">
105 |     
106 | 
107 |   <div class="position-relative js-header-wrapper ">
108 |     <a href="#start-of-content" tabindex="1" class="px-2 py-4 bg-blue text-white show-on-focus js-skip-to-content">Skip to content</a>
109 |     <div id="js-pjax-loader-bar" class="pjax-loader-bar"><div class="progress"></div></div>
110 | 
111 |     
112 |     
113 |     
114 | 
115 | 
116 | 
117 |         
118 | 
119 |   <header class="Header header-logged-out  position-relative f4 py-3" role="banner" data-ga-load="(Logged out) Header, view, experiment:site_header_dropdowns; group:dropdowns">
120 |     <div class="container-lg d-flex px-3">
121 |       <div class="d-flex flex-justify-between flex-items-center">
122 |           <a class="mr-5" href="https://github.com/" aria-label="Homepage" data-ga-click="(Logged out) Header, go to homepage, icon:logo-wordmark; experiment:site_header_dropdowns; group:dropdowns">
123 |             <svg height="32" class="octicon octicon-mark-github text-white" viewBox="0 0 16 16" version="1.1" width="32" aria-hidden="true"><path fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0 0 16 8c0-4.42-3.58-8-8-8z"/></svg>
124 |           </a>
125 |       </div>
126 | 
127 |       <div class="HeaderMenu HeaderMenu--experiment d-flex flex-justify-between flex-items-center flex-auto">
128 |         <div class="d-none">
129 |           <button class="btn-link js-details-target" type="button" aria-label="Toggle navigation" aria-expanded="false">
130 |             <svg height="24" class="octicon octicon-x text-gray" viewBox="0 0 12 16" version="1.1" width="18" aria-hidden="true"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
131 |           </button>
132 |         </div>
133 | 
134 |           <nav class="">
135 |             <ul class="d-flex list-style-none">
136 |                 <li class="HeaderMenu-item dropdown mr-5">
137 |                   <details class="details-expanded details-reset js-dropdown-details ">
138 |                     <summary class="HeaderMenu-target text-white">
139 |                       <div class="d-flex flex-items-baseline flex-justify-between">
140 |                         <span class="d-inline-block mr-1">Features</span>
141 |                         <span class="dropdown-caret"></span>
142 |                       </div>
143 |                     </summary>
144 |                     <div class="dropdown-menu dropdown-menu-s p-4 ml-n4 mt-3 mt-lg-2">
145 |                       <a href="/features" class="d-block d-lg-flex flex-items-center flex-justify-between f5 link-gray-dark text-bold mb-2" data-ga-click="(Logged out) Header, go to Features, experiment:site_header_dropdowns; group:dropdowns"><span>Features overview</span> <svg height="16" class="octicon octicon-chevron-right text-gray-dark" viewBox="0 0 8 16" version="1.1" width="8" aria-hidden="true"><path fill-rule="evenodd" d="M7.5 8l-5 5L1 11.5 4.75 8 1 4.5 2.5 3l5 5z"/></svg></a>
146 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
147 |                       <ul class="list-style-none f5">
148 |                         <li class="mb-2"><a href="/features/code-review/" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Code review, experiment:site_header_dropdowns; group:dropdowns">Code review</a></li>
149 |                         <li class="mb-2"><a href="/features/project-management/" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Project management, experiment:site_header_dropdowns; group:dropdowns">Project management</a></li>
150 |                         <li class="mb-2"><a href="/features/integrations" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Integrations, experiment:site_header_dropdowns; group:dropdowns">Integrations</a></li>
151 |                         <li class="mb-2"><a href="/features#team-management" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Team management, experiment:site_header_dropdowns; group:dropdowns">Team management</a></li>
152 |                         <li class="mb-2"><a href="/features#social-coding" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Social coding, experiment:site_header_dropdowns; group:dropdowns">Social coding</a></li>
153 |                         <li class="mb-2"><a href="/features#documentation" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Documentation, experiment:site_header_dropdowns; group:dropdowns">Documentation</a></li>
154 |                         <li class="mb-2"><a href="/features#code-hosting" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Code hosting, experiment:site_header_dropdowns; group:dropdowns">Code hosting</a></li>
155 |                       </ul>
156 |                     </div>
157 |                   </details>
158 |                 </li>
159 |                 <li class="HeaderMenu-item dropdown platform-nav mr-5">
160 |                   <details class="details-expanded details-reset js-dropdown-details ">
161 |                     <summary class="HeaderMenu-target text-white">
162 |                       <div class="d-flex flex-items-baseline flex-justify-between">
163 |                         <span class="d-inline-block mr-1">Platform</span>
164 |                         <span class="dropdown-caret"></span>
165 |                       </div>
166 |                     </summary>
167 |                     <div class="dropdown-menu dropdown-menu-s p-4 ml-n4 mt-3 mt-lg-2">
168 |                       <div class="d-flex gutter-spacious ">
169 |                         <div class="position-relative col-8">
170 |                           <dl class="my-0">
171 |                             <a href="/marketplace" class="d-flex mb-3 link-gray-dark no-underline" data-ga-click="(Logged out) Header, go to Marketplace, experiment:site_header_dropdowns; group:dropdowns">
172 |                               <div class="mr-3">
173 |                                 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 70.92 56.98" class="d-block" width="34"><title>Asset 1</title><g data-name="Layer 2"><path d="M6.18 57H1a1 1 0 0 1 0-2h5.18a1 1 0 0 1 0 2zM69.92 57h-5.18a1 1 0 1 1 0-2h5.18a1 1 0 0 1 0 2z" fill="#2088ff"></path><path d="M29.67 56.47a1 1 0 0 1-1-1V34.84H16v20.23a1 1 0 0 1-2 0V33.84a1 1 0 0 1 1-1h14.67a1 1 0 0 1 1 1v21.63a1 1 0 0 1-1 1z" fill="#79b8ff"></path><path d="M64.74 57H6.18a1 1 0 0 1-1-1v-8.65a1 1 0 0 1 2 0V55h56.56V33.84a1 1 0 0 1 2 0V56a1 1 0 0 1-1 1zM6.18 41.24a1 1 0 0 1-1-1V1a1 1 0 0 1 1-1h58.58a1 1 0 0 1 1 1v10.52a1 1 0 1 1-2 0V2H7.18v38.24a1 1 0 0 1-1 1z" fill="#2088ff"></path><path d="M56.82 45.94H36.34a1 1 0 0 1-1-1v-11.1a1 1 0 0 1 1-1h20.48a1 1 0 0 1 1 1v11.1a1 1 0 0 1-1 1zm-19.48-2h18.48v-9.1H37.34z" fill="#79b8ff"></path><path d="M31.55 27.5a7.84 7.84 0 0 1-5.21-2.42c-1.06-1-11.87-9.74-12-9.83a1 1 0 0 1 .66-1.78h43.66a1 1 0 0 1 .64.24l11.26 9.48a1 1 0 0 1-1.29 1.53l-11-9.25H17.81c3.26 2.65 9.08 7.4 9.88 8.12a6 6 0 0 0 3.87 1.9 3.3 3.3 0 0 0 3-1.95 1 1 0 1 1 1.82.82 5.3 5.3 0 0 1-4.83 3.14z" fill="#2088ff"></path><path d="M40.24 27.5a5.26 5.26 0 0 1-1.86-.34 1 1 0 0 1 .71-1.87 3.26 3.26 0 0 0 1.16.21 3.3 3.3 0 0 0 3-1.95 1 1 0 1 1 1.82.82 5.3 5.3 0 0 1-4.83 3.13zM48.82 27.5a5.26 5.26 0 0 1-1.82-.34 1 1 0 0 1 .71-1.87 3.26 3.26 0 0 0 1.16.21 3.3 3.3 0 0 0 3-1.95 1 1 0 1 1 1.82.82 5.3 5.3 0 0 1-4.87 3.13zM57.41 27.5a5.26 5.26 0 0 1-1.86-.34 1 1 0 0 1 .71-1.87 3.26 3.26 0 0 0 1.16.21 3.3 3.3 0 0 0 3-1.95 1 1 0 1 1 1.82.82 5.3 5.3 0 0 1-4.83 3.13zM66 27.5a5.26 5.26 0 0 1-1.86-.34 1 1 0 0 1 .71-1.87 3.26 3.26 0 0 0 1.15.21 3.3 3.3 0 0 0 3-2 1 1 0 0 1 1.82.82A5.31 5.31 0 0 1 66 27.5zM15 27.22a1 1 0 0 1-1-1V14.71a1 1 0 0 1 2 0v11.51a1 1 0 0 1-1 1z" fill="#2088ff"></path><path d="M44.16 25a1 1 0 0 1-.65-.24L38.07 20a1 1 0 0 1 1.3-1.52l5.43 4.67a1 1 0 0 1-.64 1.85zM35.52 25a1 1 0 0 1-.65-.24L29.44 20a1 1 0 0 1 1.3-1.52l5.43 4.67a1 1 0 0 1-.65 1.85zM52.74 25a1 1 0 0 1-.65-.24L46.66 20A1 1 0 0 1 48 18.53l5.43 4.67a1 1 0 0 1-.69 1.8zM61.33 25a1 1 0 0 1-.65-.24L55.25 20a1 1 0 0 1 1.3-1.52L62 23.2a1 1 0 0 1-.67 1.8zM22.23 8.42H6.51a1 1 0 0 1 0-2h15.72a1 1 0 0 1 0 2zM64.76 8.42H58.4a1 1 0 0 1 0-2h6.36a1 1 0 0 1 0 2z" fill="#2088ff"></path><path d="M46.58 45.72a1 1 0 0 1-1-1V34.1a1 1 0 0 1 2 0v10.62a1 1 0 0 1-1 1z" fill="#79b8ff"></path></g></svg>
174 | 
175 |                               </div>
176 |                               <div>
177 |                                 <dt class="f4">Marketplace</dt>
178 |                                 <dd class="f6 text-gray">Find developer tools that work with GitHub</dd>
179 |                               </div>
180 |                             </a>
181 |                             <a href="https://developer.github.com" class="d-flex mb-3 link-gray-dark no-underline" data-ga-click="(Logged out) Header, go to Developers, experiment:site_header_dropdowns; group:dropdowns">
182 |                               <div class="mr-3">
183 |                                 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 61.23 60.12" class="d-block" width="34"><title>Asset 1</title><g data-name="Layer 2"><path fill="none" stroke="#79b8ff" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M14.75 13.22H3.52M18.02 6.33H6.54"></path><path fill="none" stroke="#2088ff" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M56.96 51.89H45.73M60.23 44.99H48.75"></path><circle cx="22.5" cy="37.62" r="7.48" transform="rotate(-45 22.502 37.62)" fill="none" stroke="#2088ff" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"></circle><path d="M36.72 29.79l1.35 3.4 5.93 2V40l-5.92 2.08-1.44 3.39 2.83 5.58L36 54.54l-5.66-2.7L27 53.22l-2 5.9h-4.89L18 53.18l-3.36-1.4L9 54.58 5.58 51.1l2.7-5.66-1.38-3.37-5.9-2v-4.89l5.92-2.08 1.44-3.39-2.82-5.57L9 20.69l5.66 2.7L18 22l2-5.9h4.89L27 22l3.36 1.4L36 20.66l3.45 3.48z" fill="none" stroke="#2088ff" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"></path><path d="M7.22 22.48L9 20.69l5.66 2.7L18 22l2-5.9h4.89L27 22l3.36 1.4L36 20.66l3.45 3.48-2.7 5.66 1.35 3.4 5.93 2L44 40l-5.92 2.08-1.44 3.39 2.83 5.58-1.83 1.82" fill="none" stroke="#2088ff" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" opacity=".1"></path><path d="M37.22 14.11a6.05 6.05 0 1 1 8.56 8.56M30.07 12l-2.29-4.51 2.81-2.79 4.58 2.19 2.72-1.12L39.5 1h4l1.64 4.8 2.71 1.14 4.54-2.26 2.79 2.81L53 12.06l1.09 2.75 4.8 1.58v4L54.09 22l-1.16 2.74 2.29 4.51-2.82 2.83-4.58-2.19" fill="none" stroke="#79b8ff" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"></path><path d="M29.14 6.14l1.45-1.44 4.58 2.19 2.72-1.12L39.5 1h4l1.64 4.8 2.71 1.14 4.54-2.26 2.79 2.81L53 12.06l1.09 2.75 4.8 1.58v4L54.09 22l-1.16 2.74 2.29 4.51-1.48 1.47" fill="none" stroke="#79b8ff" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" opacity=".1"></path></g></svg>
184 | 
185 |                               </div>
186 |                               <div>
187 |                                 <dt class="f4">GitHub API</dt>
188 |                                 <dd class="f6 text-gray">Start building on the GitHub platform</dd>
189 |                               </div>
190 |                             </a>
191 |                             <a href="https://partner.github.com/?source=github-header-loggedout&experiment=site_header_dropdowns&group=dropdowns" class="d-flex mb-3 link-gray-dark no-underline" data-ga-click="(Logged out) Header, go to Partner program, experiment:site_header_dropdowns; group:dropdowns">
192 |                               <div class="mr-3">
193 |                                 <!-- Generator: Adobe Illustrator 22.1.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  --><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" id="Layer_1" x="0px" y="0px" viewBox="0 0 37 23" style="enable-background:new 0 0 37 23;" xml:space="preserve" class="d-block" width="34">
194 | <style type="text/css">
195 | 	.st0{fill:none;stroke:#79B8FF;stroke-linecap:round;stroke-linejoin:round;}
196 | 	.st1{fill:none;stroke:#2088FF;stroke-linecap:round;stroke-linejoin:round;}
197 | </style>
198 | <path class="st0" d="M6,1l4.5,1.4L5.3,15.2L1,12.9L6,1z"></path>
199 | <path class="st1" d="M30.9,1.2L36,12.8l-3.9,2.6l-5.2-13L30.9,1.2z"></path>
200 | <path class="st1" d="M5.2,11.7c0-0.6-0.5-1-1-1c-0.6,0-1,0.5-1,1c0,0.6,0.5,1,1,1"></path>
201 | <path class="st0" d="M29.7,4.6c0.4,0,0.8-0.3,0.8-0.8v0c0-0.4-0.3-0.8-0.8-0.8h0c-0.4,0-0.8,0.3-0.8,0.8v0C29,4.3,29.3,4.6,29.7,4.6  L29.7,4.6z"></path>
202 | <path class="st0" d="M6.2,13.4c0,0,2.4,0.8,3.9,2.3L6.2,13.4z M9.6,4.8c0,0,4.3,2.3,6.9,2.8L9.6,4.8z"></path>
203 | <path class="st1" d="M30.4,11.5l-3.2,2.6"></path>
204 | <path class="st0" d="M19.7,9l8.7,6.2c0.4,0.3,1.1,1.2,0.4,2.2c-0.7,0.9-1.5,0.2-1.5,0.2l-6-4.2"></path>
205 | <path class="st1" d="M27.5,4.8c0,0-2,1.6-3.3,1.4c-1.3-0.2-4.7-0.9-4.7-0.9c-0.7,0.8-3.7,3.5-4.6,3.6c0,0-0.2,1.5,1.6,1.3  c1.7-0.2,2.7-0.9,3.7-1.7"></path>
206 | <path class="st0" d="M24.6,18.9c0,0,0.2,0.9-0.2,1.4c-0.2,0.3-0.7,0.6-1.3,0.3l-3.7-2.5L24.6,18.9z M27.1,17.6c0,0,0.1,0.9-0.5,1.4  c-0.3,0.3-0.8,0.6-1.5,0.3l-5.9-4.3L27.1,17.6z"></path>
207 | <path class="st0" d="M22.4,20.1c0,0,0,0.6-0.2,1.1c-0.2,0.3-0.4,0.6-1.1,0.4l-2.8-1.9"></path>
208 | <path class="st1" d="M17.8,17c0.3-0.3,1.6-0.2,1.4,1c-0.3,1.2-1.7,2.4-2.1,2.6c-0.4,0.2-1.4,0.2-1.4-0.7L17.8,17z M14.3,14.6  c0,0-0.7-1.5-1.5-1.2c-0.8,0.3-2.8,2-2.8,2.7c0,0.7,1,1.5,1.9,0.8L14.3,14.6z"></path>
209 | <path class="st1" d="M16.1,15.8c0.2-0.3,1.8-0.4,1.5,1.1c-0.3,1.4-2.1,2.7-2.4,2.9c-0.3,0.1-1.7,0.3-1.9-0.9"></path>
210 | <path class="st1" d="M15.9,16.1c0,0,0.5-0.9-0.1-1.5c-0.6-0.6-1.2-0.2-1.6,0.2c-0.3,0.3-2.9,2.8-2.9,2.8s0.5,1.8,1.7,1.3  C14.2,18.5,15.9,16.1,15.9,16.1z"></path>
211 | </svg>
212 | 
213 |                               </div>
214 |                               <div>
215 |                                 <dt class="f4">Partner program</dt>
216 |                                 <dd class="f6 text-gray">Help millions of developers do their best work</dd>
217 |                               </div>
218 |                             </a>
219 |                           </dl>
220 |                           <div class="d-none d-lg-block border-left position-absolute top-0 right-0 bottom-0"></div>
221 |                         </div>
222 |                         <div class="col-4">
223 |                           <strong class="d-block f5 text-bold mb-2 text-gray-dark">Apps by GitHub</strong>
224 |                           <ul class="list-style-none f5">
225 |                             <li class="mb-2"><a href="https://desktop.github.com/" class="link-gray" data-ga-click="(Logged out) Header, go to Desktop, experiment:site_header_dropdowns; group:dropdowns">Desktop <span style="color: #959da5;">&#8599;</span></a></li>
226 |                             <li class="mb-2"><a href="https://atom.io/" class="link-gray" data-ga-click="(Logged out) Header, go to Atom, experiment:site_header_dropdowns; group:dropdowns">Atom <span style="color: #959da5;">&#8599;</span></a></li>
227 |                             <li class="mb-2"><a href="https://visualstudio.github.com/" class="link-gray" data-ga-click="(Logged out) Header, go to Visual Studio, experiment:site_header_dropdowns; group:dropdowns">Visual Studio <span style="color: #959da5;">&#8599;</span></a></li>
228 |                             <li class="mb-2"><a href="https://unity.github.com/" class="link-gray" data-ga-click="(Logged out) Header, go to Unity Extension, experiment:site_header_dropdowns; group:dropdowns">Unity Extension <span style="color: #959da5;">&#8599;</span></a></li>
229 |                           </ul>
230 |                         </div>
231 |                       </div>
232 |                     </div>
233 |                   </details>
234 |                 </li>
235 |                 <li class="HeaderMenu-item dropdown mr-5">
236 |                   <details class="details-expanded details-reset js-dropdown-details ">
237 |                     <summary class="HeaderMenu-target text-white">
238 |                       <div class="d-flex flex-items-baseline flex-justify-between">
239 |                         <span class="d-inline-block mr-1">Business</span>
240 |                         <span class="dropdown-caret"></span>
241 |                       </div>
242 |                     </summary>
243 |                     <div class="dropdown-menu dropdown-menu-s p-4 ml-n4 mt-3 mt-lg-2">
244 |                       <a href="/business" class="d-block d-lg-flex flex-items-center flex-justify-between f5 link-gray-dark text-bold mb-2" data-ga-click="(Logged out) Header, go to Business, experiment:site_header_dropdowns; group:dropdowns"><span>Business overview</span> <svg height="16" class="octicon octicon-chevron-right text-gray-dark" viewBox="0 0 8 16" version="1.1" width="8" aria-hidden="true"><path fill-rule="evenodd" d="M7.5 8l-5 5L1 11.5 4.75 8 1 4.5 2.5 3l5 5z"/></svg></a>
245 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
246 |                       <ul class="list-style-none f5">
247 |                         <li class="mb-2"><a href="/business/customers" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Customers, experiment:site_header_dropdowns; group:dropdowns">Customers</a></li>
248 |                         <li class="mb-2"><a href="/business/security" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Security, experiment:site_header_dropdowns; group:dropdowns">Security</a></li>
249 |                         <li class="mb-2"><a href="https://enterprise.github.com/contact" class="d-block link-gray" data-ga-click="(Logged out) Header, go to Contact, experiment:site_header_dropdowns; group:dropdowns">Contact</a></li>
250 |                       </ul>
251 |                     </div>
252 |                   </details>
253 |                 </li>
254 | 
255 |                 <li class="HeaderMenu-item dropdown mr-5">
256 |                   <details class="details-expanded details-reset js-dropdown-details ">
257 |                     <summary class="HeaderMenu-target text-white">
258 |                       <div class="d-flex flex-items-baseline flex-justify-between">
259 |                         <span class="d-inline-block mr-1">Explore</span>
260 |                         <span class="dropdown-caret"></span>
261 |                       </div>
262 |                     </summary>
263 |                     <div class="dropdown-menu dropdown-menu-s p-4 ml-n4 mt-3 mt-lg-2">
264 |                       <ul class="list-style-none f5">
265 |                         <li class="mb-2"><a href="/explore" class="d-lg-flex flex-items-center flex-justify-between link-gray-dark text-bold" data-ga-click="(Logged out) Header, go to Explore GitHub, experiment:site_header_dropdowns; group:dropdowns"><span>Explore GitHub</span> <svg height="16" class="octicon octicon-chevron-right text-gray-dark" viewBox="0 0 8 16" version="1.1" width="8" aria-hidden="true"><path fill-rule="evenodd" d="M7.5 8l-5 5L1 11.5 4.75 8 1 4.5 2.5 3l5 5z"/></svg></a></li>
266 |                         <li class="mb-3"><a href="/trending" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Trending, experiment:site_header_dropdowns; group:dropdowns">Trending</a></li>
267 |                       </ul>
268 | 
269 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
270 | 
271 |                       <strong class="d-block f5 text-bold mb-2 text-gray-dark">Learn</strong>
272 |                       <ul class="list-style-none f5">
273 |                         <li class="mb-2"><a href="/topics" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Topics, experiment:site_header_dropdowns; group:dropdowns">Topics</a></li>
274 |                         <li class="mb-2"><a href="/collections" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Collections, experiment:site_header_dropdowns; group:dropdowns">Collections</a></li>
275 |                         <li class="mb-2"><a href="https://lab.github.com/" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Learning Lab, experiment:site_header_dropdowns; group:dropdowns">Learning Lab <span style="color: #959da5;">&#8599;</span></a></li>
276 |                         <li class="mb-3"><a href="https://opensource.guide/" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Open source guides, experiment:site_header_dropdowns; group:dropdowns">Open source guides <span style="color: #959da5;">&#8599;</span></a></li>
277 |                       </ul>
278 | 
279 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
280 | 
281 |                       <strong class="d-block f5 text-bold mb-2 text-gray-dark">Connect</strong>
282 |                       <ul class="list-style-none f5">
283 |                         <li class="mb-2"><a href="/events" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Events, experiment:site_header_dropdowns; group:dropdowns">Events</a></li>
284 |                         <li class="mb-2"><a href="https://github.community/" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Community forum, experiment:site_header_dropdowns; group:dropdowns">Community forum <span style="color: #959da5;">&#8599;</span></a></li>
285 |                         <li class="mb-3"><a href="https://education.github.community/" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Education community, experiment:site_header_dropdowns; group:dropdowns">Education community <span style="color: #959da5;">&#8599;</span></a></li>
286 |                       </ul>
287 |                     </div>
288 |                   </details>
289 |                 </li>
290 | 
291 |                 <li class="HeaderMenu-item dropdown mr-5">
292 |                   <details class="details-expanded details-reset js-dropdown-details ">
293 |                     <summary class="HeaderMenu-target text-white">
294 |                       <div class="d-flex flex-items-baseline flex-justify-between">
295 |                         <span class="d-inline-block mr-1">Pricing</span>
296 |                         <span class="dropdown-caret"></span>
297 |                       </div>
298 |                     </summary>
299 |                     <div class="dropdown-menu dropdown-menu-s p-4 ml-n4 mt-3 mt-lg-2">
300 |                       <a href="/pricing" class="d-block d-lg-flex flex-items-center flex-justify-between f5 link-gray-dark text-bold mb-3" data-ga-click="(Logged out) Header, go to Pricing, experiment:site_header_dropdowns; group:dropdowns"><span>Pricing overview</span> <svg height="16" class="octicon octicon-chevron-right text-gray-dark" viewBox="0 0 8 16" version="1.1" width="8" aria-hidden="true"><path fill-rule="evenodd" d="M7.5 8l-5 5L1 11.5 4.75 8 1 4.5 2.5 3l5 5z"/></svg></a>
301 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
302 |                       <strong class="d-block f5 text-bold text-gray-dark mb-2">Plans</strong>
303 |                       <ul class="list-style-none f5">
304 |                         <li class="mb-2"><a href="/pricing/developer" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Developer, experiment:site_header_dropdowns; group:dropdowns">Developer</a></li>
305 |                         <li class="mb-2"><a href="/pricing/team" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Team, experiment:site_header_dropdowns; group:dropdowns">Team</a></li>
306 |                         <li class="mb-2"><a href="/pricing/business-hosted" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Business, experiment:site_header_dropdowns; group:dropdowns">Business</a></li>
307 |                         <li class="mb-3"><a href="/pricing#feature-comparison" class="link-gray d-block" data-ga-click="(Logged out) Header, go to Compare plans, experiment:site_header_dropdowns; group:dropdowns">Compare plans</a></li>
308 |                       </ul>
309 |                       <hr class="border border-dashed my-3 d-none d-lg-block">
310 |                       <ul class="list-style-none f5">
311 |                         <li class="mb-2"><a href="https://github.com/nonprofit" class="link-gray-dark" data-ga-click="(Logged out) Header, go to Nonprofits, experiment:site_header_dropdowns; group:dropdowns">Nonprofits</a></li>
312 |                         <li class="mb-2"><a href="https://education.github.com/discount_requests/new" class="link-gray-dark" data-ga-click="(Logged out) Header, go to Education, experiment:site_header_dropdowns; group:dropdowns">Education <span style="color: #959da5;">&#8599;</span></a></li>
313 |                       </ul>
314 |                     </div>
315 |                   </details>
316 |                 </li>
317 |             </ul>
318 |           </nav>
319 | 
320 |         <div class="d-flex flex-items-center">
321 |             <div class="d-flex mr-3 flex-items-center">
322 |               <div class="header-search scoped-search site-scoped-search js-site-search position-relative js-jump-to"
323 |   role="search combobox"
324 |   aria-owns="jump-to-results"
325 |   aria-label="Search or jump to"
326 |   aria-haspopup="listbox"
327 |   aria-expanded="true"
328 | >
329 |   <div class="position-relative">
330 |     <!-- '"` --><!-- </textarea></xmp> --></option></form><form class="js-site-search-form" data-scope-type="Repository" data-scope-id="142387092" data-scoped-search-url="/G-Wang/some_files/search" data-unscoped-search-url="/search" action="/G-Wang/some_files/search" accept-charset="UTF-8" method="get"><input name="utf8" type="hidden" value="&#x2713;" />
331 |       <label class="form-control header-search-wrapper header-search-wrapper-jump-to position-relative d-flex flex-justify-between flex-items-center js-chromeless-input-container">
332 |         <input type="text"
333 |           class="form-control header-search-input jump-to-field js-jump-to-field js-site-search-focus js-site-search-field is-clearable"
334 |           data-hotkey="s,/"
335 |           name="q"
336 |           value=""
337 |           placeholder="Search"
338 |           data-unscoped-placeholder="Search GitHub"
339 |           data-scoped-placeholder="Search"
340 |           autocapitalize="off"
341 |           aria-autocomplete="list"
342 |           aria-controls="jump-to-results"
343 |           data-jump-to-suggestions-path="/_graphql/GetSuggestedNavigationDestinations#csrf-token=t9gyfpSWnoeNpTHC8seV7ddQOfednbPbqA+IR41gWosVW2HYdWrf9zrJwMC+mZodfLr9A6XWlziHt+w2q0lQ0w=="
344 |           spellcheck="false"
345 |           autocomplete="off"
346 |           >
347 |           <input type="hidden" class="js-site-search-type-field" name="type" >
348 |             <img src="https://assets-cdn.github.com/images/search-shortcut-hint.svg" alt="" class="mr-2 header-search-key-slash">
349 | 
350 |             <div class="Box position-absolute overflow-hidden d-none jump-to-suggestions js-jump-to-suggestions-container">
351 |               <ul class="d-none js-jump-to-suggestions-template-container">
352 |                 <li class="d-flex flex-justify-start flex-items-center p-0 f5 navigation-item js-navigation-item">
353 |                   <a tabindex="-1" class="no-underline d-flex flex-auto flex-items-center p-2 jump-to-suggestions-path js-jump-to-suggestion-path js-navigation-open" href="">
354 |                     <div class="jump-to-octicon js-jump-to-octicon mr-2 text-center d-none"></div>
355 |                     <img class="avatar mr-2 flex-shrink-0 js-jump-to-suggestion-avatar" alt="" aria-label="Team" src="" width="28" height="28">
356 | 
357 |                     <div class="jump-to-suggestion-name js-jump-to-suggestion-name flex-auto overflow-hidden text-left no-wrap css-truncate css-truncate-target">
358 |                     </div>
359 | 
360 |                     <div class="border rounded-1 flex-shrink-0 bg-gray px-1 text-gray-light ml-1 f6 d-none js-jump-to-badge-search">
361 |                       <span class="js-jump-to-badge-search-text-default d-none" aria-label="in this repository">
362 |                         In this repository
363 |                       </span>
364 |                       <span class="js-jump-to-badge-search-text-global d-none" aria-label="in all of GitHub">
365 |                         All GitHub
366 |                       </span>
367 |                       <span aria-hidden="true" class="d-inline-block ml-1 v-align-middle">↵</span>
368 |                     </div>
369 | 
370 |                     <div aria-hidden="true" class="border rounded-1 flex-shrink-0 bg-gray px-1 text-gray-light ml-1 f6 d-none d-on-nav-focus js-jump-to-badge-jump">
371 |                       Jump to
372 |                       <span class="d-inline-block ml-1 v-align-middle">↵</span>
373 |                     </div>
374 |                   </a>
375 |                 </li>
376 |                 <svg height="16" width="16" class="octicon octicon-repo flex-shrink-0 js-jump-to-repo-octicon-template" title="Repository" aria-label="Repository" viewBox="0 0 12 16" version="1.1" role="img"><path fill-rule="evenodd" d="M4 9H3V8h1v1zm0-3H3v1h1V6zm0-2H3v1h1V4zm0-2H3v1h1V2zm8-1v12c0 .55-.45 1-1 1H6v2l-1.5-1.5L3 16v-2H1c-.55 0-1-.45-1-1V1c0-.55.45-1 1-1h10c.55 0 1 .45 1 1zm-1 10H1v2h2v-1h3v1h5v-2zm0-10H2v9h9V1z"/></svg>
377 |                 <svg height="16" width="16" class="octicon octicon-project flex-shrink-0 js-jump-to-project-octicon-template" title="Project" aria-label="Project" viewBox="0 0 15 16" version="1.1" role="img"><path fill-rule="evenodd" d="M10 12h3V2h-3v10zm-4-2h3V2H6v8zm-4 4h3V2H2v12zm-1 1h13V1H1v14zM14 0H1a1 1 0 0 0-1 1v14a1 1 0 0 0 1 1h13a1 1 0 0 0 1-1V1a1 1 0 0 0-1-1z"/></svg>
378 |                 <svg height="16" width="16" class="octicon octicon-search flex-shrink-0 js-jump-to-search-octicon-template" title="Search" aria-label="Search" viewBox="0 0 16 16" version="1.1" role="img"><path fill-rule="evenodd" d="M15.7 13.3l-3.81-3.83A5.93 5.93 0 0 0 13 6c0-3.31-2.69-6-6-6S1 2.69 1 6s2.69 6 6 6c1.3 0 2.48-.41 3.47-1.11l3.83 3.81c.19.2.45.3.7.3.25 0 .52-.09.7-.3a.996.996 0 0 0 0-1.41v.01zM7 10.7c-2.59 0-4.7-2.11-4.7-4.7 0-2.59 2.11-4.7 4.7-4.7 2.59 0 4.7 2.11 4.7 4.7 0 2.59-2.11 4.7-4.7 4.7z"/></svg>
379 |               </ul>
380 |               <ul class="d-none js-jump-to-no-results-template-container">
381 |                 <li class="d-flex flex-justify-center flex-items-center p-3 f5 d-none">
382 |                   <span class="text-gray">No suggested jump to results</span>
383 |                 </li>
384 |               </ul>
385 | 
386 |               <ul id="jump-to-results" class="js-navigation-container jump-to-suggestions-results-container js-jump-to-suggestions-results-container" >
387 |                 <li class="d-flex flex-justify-center flex-items-center p-0 f5">
388 |                   <img src="https://assets-cdn.github.com/images/spinners/octocat-spinner-128.gif" alt="Octocat Spinner Icon" class="m-2" width="28">
389 |                 </li>
390 |               </ul>
391 |             </div>
392 |       </label>
393 | </form>  </div>
394 | </div>
395 | 
396 |             </div>
397 | 
398 |           <a class="HeaderMenu-target text-white no-underline mr-3" href="/login?return_to=%2FG-Wang%2Fsome_files%2Fblob%2Fmaster%2F20180510_mixture_lj_checkpoint_step000320000_ema.wav" data-ga-click="(Logged out) Header, clicked Sign in, text:sign-in; experiment:site_header_dropdowns; group:dropdowns">Sign&nbsp;in</a>
399 |             <a class="HeaderMenu-target d-inline-block text-white no-underline border border-gray-dark rounded-2 px-2 py-1" href="/join" data-ga-click="(Logged out) Header, clicked Sign up, text:sign-up; experiment:site_header_dropdowns; group:dropdowns">Sign&nbsp;up</a>
400 |         </div>
401 |       </div>
402 |     </div>
403 |   </header>
404 | 
405 | 
406 |   </div>
407 | 
408 |   <div id="start-of-content" class="show-on-focus"></div>
409 | 
410 |     <div id="js-flash-container">
411 | 
412 | 
413 | </div>
414 | 
415 | 
416 | 
417 |   <div role="main" class="application-main ">
418 |         <div itemscope itemtype="http://schema.org/SoftwareSourceCode" class="">
419 |     <div id="js-repo-pjax-container" data-pjax-container >
420 |       
421 | 
422 | 
423 | 
424 | 
425 | 
426 | 
427 | 
428 |   <div class="pagehead repohead instapaper_ignore readability-menu experiment-repo-nav  ">
429 |     <div class="repohead-details-container clearfix container">
430 | 
431 |       <ul class="pagehead-actions">
432 |   <li>
433 |       <a href="/login?return_to=%2FG-Wang%2Fsome_files"
434 |     class="btn btn-sm btn-with-count tooltipped tooltipped-s"
435 |     aria-label="You must be signed in to watch a repository" rel="nofollow">
436 |     <svg class="octicon octicon-eye v-align-text-bottom" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M8.06 2C3 2 0 8 0 8s3 6 8.06 6C13 14 16 8 16 8s-3-6-7.94-6zM8 12c-2.2 0-4-1.78-4-4 0-2.2 1.8-4 4-4 2.22 0 4 1.8 4 4 0 2.22-1.78 4-4 4zm2-4c0 1.11-.89 2-2 2-1.11 0-2-.89-2-2 0-1.11.89-2 2-2 1.11 0 2 .89 2 2z"/></svg>
437 |     Watch
438 |   </a>
439 |   <a class="social-count" href="/G-Wang/some_files/watchers"
440 |      aria-label="1 user is watching this repository">
441 |     1
442 |   </a>
443 | 
444 |   </li>
445 | 
446 |   <li>
447 |       <a href="/login?return_to=%2FG-Wang%2Fsome_files"
448 |     class="btn btn-sm btn-with-count tooltipped tooltipped-s"
449 |     aria-label="You must be signed in to star a repository" rel="nofollow">
450 |     <svg class="octicon octicon-star v-align-text-bottom" viewBox="0 0 14 16" version="1.1" width="14" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M14 6l-4.9-.64L7 1 4.9 5.36 0 6l3.6 3.26L2.67 14 7 11.67 11.33 14l-.93-4.74L14 6z"/></svg>
451 |     Star
452 |   </a>
453 | 
454 |     <a class="social-count js-social-count" href="/G-Wang/some_files/stargazers"
455 |       aria-label="0 users starred this repository">
456 |       0
457 |     </a>
458 | 
459 |   </li>
460 | 
461 |   <li>
462 |       <a href="/login?return_to=%2FG-Wang%2Fsome_files"
463 |         class="btn btn-sm btn-with-count tooltipped tooltipped-s"
464 |         aria-label="You must be signed in to fork a repository" rel="nofollow">
465 |         <svg class="octicon octicon-repo-forked v-align-text-bottom" viewBox="0 0 10 16" version="1.1" width="10" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1a1.993 1.993 0 0 0-1 3.72V6L5 8 3 6V4.72A1.993 1.993 0 0 0 2 1a1.993 1.993 0 0 0-1 3.72V6.5l3 3v1.78A1.993 1.993 0 0 0 5 15a1.993 1.993 0 0 0 1-3.72V9.5l3-3V4.72A1.993 1.993 0 0 0 8 1zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zm3 10c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zm3-10c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"/></svg>
466 |         Fork
467 |       </a>
468 | 
469 |     <a href="/G-Wang/some_files/network/members" class="social-count"
470 |        aria-label="0 users forked this repository">
471 |       0
472 |     </a>
473 |   </li>
474 | </ul>
475 | 
476 |       <h1 class="public ">
477 |   <svg class="octicon octicon-repo" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9H3V8h1v1zm0-3H3v1h1V6zm0-2H3v1h1V4zm0-2H3v1h1V2zm8-1v12c0 .55-.45 1-1 1H6v2l-1.5-1.5L3 16v-2H1c-.55 0-1-.45-1-1V1c0-.55.45-1 1-1h10c.55 0 1 .45 1 1zm-1 10H1v2h2v-1h3v1h5v-2zm0-10H2v9h9V1z"/></svg>
478 |   <span class="author" itemprop="author"><a class="url fn" rel="author" href="/G-Wang">G-Wang</a></span><!--
479 | --><span class="path-divider">/</span><!--
480 | --><strong itemprop="name"><a data-pjax="#js-repo-pjax-container" href="/G-Wang/some_files">some_files</a></strong>
481 | 
482 | </h1>
483 | 
484 |     </div>
485 |     
486 | <nav class="reponav js-repo-nav js-sidenav-container-pjax container"
487 |      itemscope
488 |      itemtype="http://schema.org/BreadcrumbList"
489 |      role="navigation"
490 |      data-pjax="#js-repo-pjax-container">
491 | 
492 |   <span itemscope itemtype="http://schema.org/ListItem" itemprop="itemListElement">
493 |     <a class="js-selected-navigation-item selected reponav-item" itemprop="url" data-hotkey="g c" data-selected-links="repo_source repo_downloads repo_commits repo_releases repo_tags repo_branches repo_packages /G-Wang/some_files" href="/G-Wang/some_files">
494 |       <svg class="octicon octicon-code" viewBox="0 0 14 16" version="1.1" width="14" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M9.5 3L8 4.5 11.5 8 8 11.5 9.5 13 14 8 9.5 3zm-5 0L0 8l4.5 5L6 11.5 2.5 8 6 4.5 4.5 3z"/></svg>
495 |       <span itemprop="name">Code</span>
496 |       <meta itemprop="position" content="1">
497 | </a>  </span>
498 | 
499 |     <span itemscope itemtype="http://schema.org/ListItem" itemprop="itemListElement">
500 |       <a itemprop="url" data-hotkey="g i" class="js-selected-navigation-item reponav-item" data-selected-links="repo_issues repo_labels repo_milestones /G-Wang/some_files/issues" href="/G-Wang/some_files/issues">
501 |         <svg class="octicon octicon-issue-opened" viewBox="0 0 14 16" version="1.1" width="14" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"/></svg>
502 |         <span itemprop="name">Issues</span>
503 |         <span class="Counter">0</span>
504 |         <meta itemprop="position" content="2">
505 | </a>    </span>
506 | 
507 |   <span itemscope itemtype="http://schema.org/ListItem" itemprop="itemListElement">
508 |     <a data-hotkey="g p" itemprop="url" class="js-selected-navigation-item reponav-item" data-selected-links="repo_pulls checks /G-Wang/some_files/pulls" href="/G-Wang/some_files/pulls">
509 |       <svg class="octicon octicon-git-pull-request" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"/></svg>
510 |       <span itemprop="name">Pull requests</span>
511 |       <span class="Counter">0</span>
512 |       <meta itemprop="position" content="3">
513 | </a>  </span>
514 | 
515 |     <a data-hotkey="g b" class="js-selected-navigation-item reponav-item" data-selected-links="repo_projects new_repo_project repo_project /G-Wang/some_files/projects" href="/G-Wang/some_files/projects">
516 |       <svg class="octicon octicon-project" viewBox="0 0 15 16" version="1.1" width="15" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M10 12h3V2h-3v10zm-4-2h3V2H6v8zm-4 4h3V2H2v12zm-1 1h13V1H1v14zM14 0H1a1 1 0 0 0-1 1v14a1 1 0 0 0 1 1h13a1 1 0 0 0 1-1V1a1 1 0 0 0-1-1z"/></svg>
517 |       Projects
518 |       <span class="Counter" >0</span>
519 | </a>
520 | 
521 | 
522 | 
523 |   <a class="js-selected-navigation-item reponav-item" data-selected-links="repo_graphs repo_contributors dependency_graph pulse /G-Wang/some_files/pulse" href="/G-Wang/some_files/pulse">
524 |     <svg class="octicon octicon-graph" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M16 14v1H0V0h1v14h15zM5 13H3V8h2v5zm4 0H7V3h2v10zm4 0h-2V6h2v7z"/></svg>
525 |     Insights
526 | </a>
527 | 
528 | </nav>
529 | 
530 | 
531 |   </div>
532 | 
533 | <div class="container new-discussion-timeline experiment-repo-nav  ">
534 |   <div class="repository-content ">
535 | 
536 |     
537 |   <a class="d-none js-permalink-shortcut" data-hotkey="y" href="/G-Wang/some_files/blob/13da4400c90ce495611f84526ff2f03e98f5ed5d/20180510_mixture_lj_checkpoint_step000320000_ema.wav">Permalink</a>
538 | 
539 |   <!-- blob contrib key: blob_contributors:v21:618cd214f0c27f43ca44186db612e534 -->
540 | 
541 |       <div class="signup-prompt-bg rounded-1">
542 |       <div class="signup-prompt p-4 text-center mb-4 rounded-1">
543 |         <div class="position-relative">
544 |           <!-- '"` --><!-- </textarea></xmp> --></option></form><form action="/site/dismiss_signup_prompt" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="QXwbDPpIh3TrobvgiC0rShR4B2KNs8wRLm2aKE+7Dmdb7jiflDogNYE0u67inI8DYXfmbjqCxNJUaOKVK263lQ==" />
545 |             <button type="submit" class="position-absolute top-0 right-0 btn-link link-gray" data-ga-click="(Logged out) Sign up prompt, clicked Dismiss, text:dismiss">
546 |               Dismiss
547 |             </button>
548 | </form>          <h3 class="pt-2">Join GitHub today</h3>
549 |           <p class="col-6 mx-auto">GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.</p>
550 |           <a class="btn btn-primary" href="/join?source=prompt-blob-show" data-ga-click="(Logged out) Sign up prompt, clicked Sign up, text:sign-up">Sign up</a>
551 |         </div>
552 |       </div>
553 |     </div>
554 | 
555 | 
556 |   <div class="file-navigation">
557 |     
558 | <div class="select-menu branch-select-menu js-menu-container js-select-menu float-left">
559 |   <button class=" btn btn-sm select-menu-button js-menu-target css-truncate" data-hotkey="w"
560 |     
561 |     type="button" aria-label="Switch branches or tags" aria-expanded="false" aria-haspopup="true">
562 |       <i>Branch:</i>
563 |       <span class="js-select-button css-truncate-target">master</span>
564 |   </button>
565 | 
566 |   <div class="select-menu-modal-holder js-menu-content js-navigation-container" data-pjax>
567 | 
568 |     <div class="select-menu-modal">
569 |       <div class="select-menu-header">
570 |         <svg class="octicon octicon-x js-menu-close" role="img" aria-label="Close" viewBox="0 0 12 16" version="1.1" width="12" height="16"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
571 |         <span class="select-menu-title">Switch branches/tags</span>
572 |       </div>
573 | 
574 |       <div class="select-menu-filters">
575 |         <div class="select-menu-text-filter">
576 |           <input type="text" aria-label="Filter branches/tags" id="context-commitish-filter-field" class="form-control js-filterable-field js-navigation-enable" placeholder="Filter branches/tags">
577 |         </div>
578 |         <div class="select-menu-tabs">
579 |           <ul>
580 |             <li class="select-menu-tab">
581 |               <a href="#" data-tab-filter="branches" data-filter-placeholder="Filter branches/tags" class="js-select-menu-tab" role="tab">Branches</a>
582 |             </li>
583 |             <li class="select-menu-tab">
584 |               <a href="#" data-tab-filter="tags" data-filter-placeholder="Find a tag…" class="js-select-menu-tab" role="tab">Tags</a>
585 |             </li>
586 |           </ul>
587 |         </div>
588 |       </div>
589 | 
590 |       <div class="select-menu-list select-menu-tab-bucket js-select-menu-tab-bucket" data-tab-filter="branches" role="menu">
591 | 
592 |         <div data-filterable-for="context-commitish-filter-field" data-filterable-type="substring">
593 | 
594 | 
595 |             <a class="select-menu-item js-navigation-item js-navigation-open selected"
596 |                href="/G-Wang/some_files/blob/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav"
597 |                data-name="master"
598 |                data-skip-pjax="true"
599 |                rel="nofollow">
600 |               <svg class="octicon octicon-check select-menu-item-icon" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M12 5l-8 8-4-4 1.5-1.5L4 10l6.5-6.5L12 5z"/></svg>
601 |               <span class="select-menu-item-text css-truncate-target js-select-menu-filter-text">
602 |                 master
603 |               </span>
604 |             </a>
605 |         </div>
606 | 
607 |           <div class="select-menu-no-results">Nothing to show</div>
608 |       </div>
609 | 
610 |       <div class="select-menu-list select-menu-tab-bucket js-select-menu-tab-bucket" data-tab-filter="tags">
611 |         <div data-filterable-for="context-commitish-filter-field" data-filterable-type="substring">
612 | 
613 | 
614 |         </div>
615 | 
616 |         <div class="select-menu-no-results">Nothing to show</div>
617 |       </div>
618 | 
619 |     </div>
620 |   </div>
621 | </div>
622 | 
623 |     <div class="BtnGroup float-right">
624 |       <a href="/G-Wang/some_files/find/master"
625 |             class="js-pjax-capture-input btn btn-sm BtnGroup-item"
626 |             data-pjax
627 |             data-hotkey="t">
628 |         Find file
629 |       </a>
630 |       <clipboard-copy for="blob-path" class="btn btn-sm BtnGroup-item">
631 |         Copy path
632 |       </clipboard-copy>
633 |     </div>
634 |     <div id="blob-path" class="breadcrumb">
635 |       <span class="repo-root js-repo-root"><span class="js-path-segment"><a data-pjax="true" href="/G-Wang/some_files"><span>some_files</span></a></span></span><span class="separator">/</span><strong class="final-path">20180510_mixture_lj_checkpoint_step000320000_ema.wav</strong>
636 |     </div>
637 |   </div>
638 | 
639 | 
640 |   
641 |   <div class="commit-tease">
642 |       <span class="float-right">
643 |         <a class="commit-tease-sha" href="/G-Wang/some_files/commit/13da4400c90ce495611f84526ff2f03e98f5ed5d" data-pjax>
644 |           13da440
645 |         </a>
646 |         <relative-time datetime="2018-07-26T04:08:04Z">Jul 25, 2018</relative-time>
647 |       </span>
648 |       <div>
649 |         <a rel="author" data-skip-pjax="true" data-hovercard-user-id="14854468" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="/G-Wang"><img class="avatar" src="https://avatars1.githubusercontent.com/u/14854468?s=40&amp;v=4" width="20" height="20" alt="@G-Wang" /></a>
650 |         <a class="user-mention" rel="author" data-hovercard-user-id="14854468" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="/G-Wang">G-Wang</a>
651 |           <a data-pjax="true" title="add" class="message" href="/G-Wang/some_files/commit/13da4400c90ce495611f84526ff2f03e98f5ed5d">add</a>
652 |       </div>
653 | 
654 |     <div class="commit-tease-contributors">
655 |       
656 | <details class="details-reset details-overlay details-overlay-dark lh-default text-gray-dark float-left mr-2" id="blob_contributors_box">
657 |   <summary class="btn-link" aria-haspopup="dialog" >
658 |     
659 |     <span><strong>1</strong> contributor</span>
660 |   </summary>
661 |   <details-dialog class="Box Box--overlay d-flex flex-column anim-fade-in fast " aria-label="Users who have contributed to this file">
662 |     <div class="Box-header">
663 |       <button class="Box-btn-octicon btn-octicon float-right" type="button" aria-label="Close dialog" data-close-dialog>
664 |         <svg class="octicon octicon-x" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
665 |       </button>
666 |       <h3 class="Box-title">Users who have contributed to this file</h3>
667 |     </div>
668 |     
669 |         <ul class="list-style-none overflow-auto">
670 |             <li class="Box-row">
671 |               <a class="link-gray-dark no-underline" href="/G-Wang">
672 |                 <img class="avatar mr-2" alt="" src="https://avatars1.githubusercontent.com/u/14854468?s=40&amp;v=4" width="20" height="20" />
673 |                 G-Wang
674 | </a>            </li>
675 |         </ul>
676 | 
677 |   </details-dialog>
678 | </details>
679 |       
680 |     </div>
681 |   </div>
682 | 
683 | 
684 | 
685 |   <div class="file">
686 |     <div class="file-header">
687 |   <div class="file-actions">
688 | 
689 |     <div class="BtnGroup">
690 |       <a id="raw-url" class="btn btn-sm BtnGroup-item" href="/G-Wang/some_files/raw/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav">Download</a>
691 |       <a rel="nofollow" class="btn btn-sm BtnGroup-item" href="/G-Wang/some_files/commits/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav">History</a>
692 |     </div>
693 | 
694 | 
695 |         <!-- '"` --><!-- </textarea></xmp> --></option></form><form class="inline-form" action="/G-Wang/some_files/delete/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="4+9KsRnIVeaNCkC0pt8VG74cTQ15hMLhj2oKuael475ktXNincA/7B0nx5YZZG56jz/j7sO49C7ljCVp2dtFBA==" />
696 |           <button class="btn-octicon btn-octicon-danger tooltipped tooltipped-nw" type="submit"
697 |             aria-label="You must be signed in to make or propose changes" data-disable-with>
698 |             <svg class="octicon octicon-trashcan" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M11 2H9c0-.55-.45-1-1-1H5c-.55 0-1 .45-1 1H2c-.55 0-1 .45-1 1v1c0 .55.45 1 1 1v9c0 .55.45 1 1 1h7c.55 0 1-.45 1-1V5c.55 0 1-.45 1-1V3c0-.55-.45-1-1-1zm-1 12H3V5h1v8h1V5h1v8h1V5h1v8h1V5h1v9zm1-10H2V3h9v1z"/></svg>
699 |           </button>
700 | </form>  </div>
701 | 
702 |   <div class="file-info">
703 |     440 KB
704 |   </div>
705 | </div>
706 | 
707 |     
708 | 
709 |   <div itemprop="text" class="blob-wrapper data type-text">
710 |       <div class="image">
711 |           <a href="/G-Wang/some_files/blob/master/20180510_mixture_lj_checkpoint_step000320000_ema.wav?raw=true">View Raw</a>
712 |       </div>
713 |   </div>
714 | 
715 |   </div>
716 | 
717 |   <details class="details-reset details-overlay details-overlay-dark">
718 |     <summary data-hotkey="l" aria-label="Jump to line"></summary>
719 |     <details-dialog class="Box Box--overlay d-flex flex-column anim-fade-in fast linejump" aria-label="Jump to line">
720 |       <!-- '"` --><!-- </textarea></xmp> --></option></form><form class="js-jump-to-line-form Box-body d-flex" action="" accept-charset="UTF-8" method="get"><input name="utf8" type="hidden" value="&#x2713;" />
721 |         <input class="form-control flex-auto mr-3 linejump-input js-jump-to-line-field" type="text" placeholder="Jump to line&hellip;" aria-label="Jump to line" autofocus>
722 |         <button type="submit" class="btn" data-close-dialog>Go</button>
723 | </form>    </details-dialog>
724 |   </details>
725 | 
726 | 
727 |   </div>
728 |   <div class="modal-backdrop js-touch-events"></div>
729 | </div>
730 | 
731 |     </div>
732 |   </div>
733 | 
734 |   </div>
735 | 
736 |         
737 | <div class="footer container-lg px-3" role="contentinfo">
738 |   <div class="position-relative d-flex flex-justify-between pt-6 pb-2 mt-6 f6 text-gray border-top border-gray-light ">
739 |     <ul class="list-style-none d-flex flex-wrap ">
740 |       <li class="mr-3">&copy; 2018 <span title="0.24817s from unicorn-9649c87bc-psnfb">GitHub</span>, Inc.</li>
741 |         <li class="mr-3"><a data-ga-click="Footer, go to terms, text:terms" href="https://github.com/site/terms">Terms</a></li>
742 |         <li class="mr-3"><a data-ga-click="Footer, go to privacy, text:privacy" href="https://github.com/site/privacy">Privacy</a></li>
743 |         <li class="mr-3"><a href="https://help.github.com/articles/github-security/" data-ga-click="Footer, go to security, text:security">Security</a></li>
744 |         <li class="mr-3"><a href="https://status.github.com/" data-ga-click="Footer, go to status, text:status">Status</a></li>
745 |         <li><a data-ga-click="Footer, go to help, text:help" href="https://help.github.com">Help</a></li>
746 |     </ul>
747 | 
748 |     <a aria-label="Homepage" title="GitHub" class="footer-octicon" href="https://github.com">
749 |       <svg height="24" class="octicon octicon-mark-github" viewBox="0 0 16 16" version="1.1" width="24" aria-hidden="true"><path fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0 0 16 8c0-4.42-3.58-8-8-8z"/></svg>
750 | </a>
751 |    <ul class="list-style-none d-flex flex-wrap ">
752 |         <li class="mr-3"><a data-ga-click="Footer, go to contact, text:contact" href="https://github.com/contact">Contact GitHub</a></li>
753 |       <li class="mr-3"><a href="https://developer.github.com" data-ga-click="Footer, go to api, text:api">API</a></li>
754 |       <li class="mr-3"><a href="https://training.github.com" data-ga-click="Footer, go to training, text:training">Training</a></li>
755 |       <li class="mr-3"><a href="https://shop.github.com" data-ga-click="Footer, go to shop, text:shop">Shop</a></li>
756 |         <li class="mr-3"><a href="https://blog.github.com" data-ga-click="Footer, go to blog, text:blog">Blog</a></li>
757 |         <li><a data-ga-click="Footer, go to about, text:about" href="https://github.com/about">About</a></li>
758 | 
759 |     </ul>
760 |   </div>
761 |   <div class="d-flex flex-justify-center pb-6">
762 |     <span class="f6 text-gray-light"></span>
763 |   </div>
764 | </div>
765 | 
766 | 
767 | 
768 |   <div id="ajax-error-message" class="ajax-error-message flash flash-error">
769 |     <svg class="octicon octicon-alert" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M8.893 1.5c-.183-.31-.52-.5-.887-.5s-.703.19-.886.5L.138 13.499a.98.98 0 0 0 0 1.001c.193.31.53.501.886.501h13.964c.367 0 .704-.19.877-.5a1.03 1.03 0 0 0 .01-1.002L8.893 1.5zm.133 11.497H6.987v-2.003h2.039v2.003zm0-3.004H6.987V5.987h2.039v4.006z"/></svg>
770 |     <button type="button" class="flash-close js-ajax-error-dismiss" aria-label="Dismiss error">
771 |       <svg class="octicon octicon-x" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
772 |     </button>
773 |     You can’t perform that action at this time.
774 |   </div>
775 | 
776 | 
777 |     <script crossorigin="anonymous" integrity="sha512-2VdGgXQE8W5ONZ4OsrbEo/noennUZaqXkevD9R8juTHiCsjT0HFTTF6MoBfySUc8G+eFqUcDgd7v+CAu8Gjxlg==" type="application/javascript" src="https://assets-cdn.github.com/assets/compat-f849c975b0ffaa01d6ca305e48417d08.js"></script>
778 |     <script crossorigin="anonymous" integrity="sha512-2EuAbCaxtrtpNeEq5yrLHWM1xGqaNKsuc6MuRtfuHgAR0yhsZH0fzhWAUtsIskUoiivK1d30h0gL57aR5VUDrw==" type="application/javascript" src="https://assets-cdn.github.com/assets/frameworks-dc359f6eb41e200e056d6763c2c2ad6a.js"></script>
779 |     
780 |     <script crossorigin="anonymous" async="async" integrity="sha512-RgTmJkjgBv6qrm5wFFCqxemcxPPL0dwRYZBx8wGHOUdoy5zN2iUIdug2V+82+lH+WpCBMdZ2ogD9pXSokyysOw==" type="application/javascript" src="https://assets-cdn.github.com/assets/github-c05962e21b15a4e47b820d4568e844f7.js"></script>
781 |     
782 |     
783 |     
784 |   <div class="js-stale-session-flash stale-session-flash flash flash-warn flash-banner d-none">
785 |     <svg class="octicon octicon-alert" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M8.893 1.5c-.183-.31-.52-.5-.887-.5s-.703.19-.886.5L.138 13.499a.98.98 0 0 0 0 1.001c.193.31.53.501.886.501h13.964c.367 0 .704-.19.877-.5a1.03 1.03 0 0 0 .01-1.002L8.893 1.5zm.133 11.497H6.987v-2.003h2.039v2.003zm0-3.004H6.987V5.987h2.039v4.006z"/></svg>
786 |     <span class="signed-in-tab-flash">You signed in with another tab or window. <a href="">Reload</a> to refresh your session.</span>
787 |     <span class="signed-out-tab-flash">You signed out in another tab or window. <a href="">Reload</a> to refresh your session.</span>
788 |   </div>
789 |   <div class="facebox" id="facebox" style="display:none;">
790 |   <div class="facebox-popup">
791 |     <div class="facebox-content" role="dialog" aria-labelledby="facebox-header" aria-describedby="facebox-description">
792 |     </div>
793 |     <button type="button" class="facebox-close js-facebox-close" aria-label="Close modal">
794 |       <svg class="octicon octicon-x" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
795 |     </button>
796 |   </div>
797 | </div>
798 | 
799 |   <template id="site-details-dialog">
800 |   <details class="details-reset details-overlay details-overlay-dark lh-default text-gray-dark" open>
801 |     <summary aria-haspopup="dialog" aria-label="Close dialog"></summary>
802 |     <details-dialog class="Box Box--overlay d-flex flex-column anim-fade-in fast">
803 |       <button class="m-3 btn-octicon position-absolute right-0 top-0" type="button" aria-label="Close dialog" data-close-dialog>
804 |         <svg class="octicon octicon-x" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.48 8l3.75 3.75-1.48 1.48L6 9.48l-3.75 3.75-1.48-1.48L4.52 8 .77 4.25l1.48-1.48L6 6.52l3.75-3.75 1.48 1.48L7.48 8z"/></svg>
805 |       </button>
806 |       <div class="octocat-spinner my-6 js-details-dialog-spinner"></div>
807 |     </details-dialog>
808 |   </details>
809 | </template>
810 | 
811 |   <div class="Popover js-hovercard-content position-absolute" style="display: none; outline: none;" tabindex="0">
812 |   <div class="Popover-message Popover-message--bottom-left Popover-message--large Box box-shadow-large" style="width:360px;">
813 |   </div>
814 | </div>
815 | 
816 | <div id="hovercard-aria-description" class="sr-only">
817 |   Press h to open a hovercard with more details.
818 | </div>
819 | 
820 | 
821 |   </body>
822 | </html>
823 | 
824 | 


--------------------------------------------------------------------------------
/loss_function.py:
--------------------------------------------------------------------------------
 1 | import torch
 2 | from torch.nn import functional as F
 3 | 
 4 | 
 5 | def nll_loss(y_hat, y, reduce=True):
 6 |     y_hat = y_hat.permute(0,2,1)
 7 |     y = y.squeeze(-1)
 8 |     loss = F.nll_loss(y_hat, y)
 9 |     return loss
10 | 
11 | def test_loss():
12 |     yhat = torch.rand(16, 100, 54)
13 |     y = torch.rand(16, 100, 1)
14 |     loss = nll_loss(yhat, y.squeeze(-1))


--------------------------------------------------------------------------------
/lrschedule.py:
--------------------------------------------------------------------------------
 1 | # reference: https://raw.githubusercontent.com/r9y9/wavenet_vocoder/master/lrschedule.py
 2 | 
 3 | import numpy as np
 4 | 
 5 | 
 6 | # https://github.com/tensorflow/tensor2tensor/issues/280#issuecomment-339110329
 7 | def noam_learning_rate_decay(init_lr, global_step, warmup_steps=4000):
 8 |     # Noam scheme from tensor2tensor:
 9 |     warmup_steps = float(warmup_steps)
10 |     step = global_step + 1.
11 |     lr = init_lr * warmup_steps**0.5 * np.minimum(
12 |         step * warmup_steps**-1.5, step**-0.5)
13 |     return lr
14 | 
15 | 
16 | def step_learning_rate_decay(init_lr, global_step,
17 |                              anneal_rate=0.98,
18 |                              anneal_interval=30000):
19 |     return init_lr * anneal_rate ** (global_step // anneal_interval)
20 | 
21 | 
22 | def cyclic_cosine_annealing(init_lr, global_step, T, M):
23 |     """Cyclic cosine annealing
24 | 
25 |     https://arxiv.org/pdf/1704.00109.pdf
26 | 
27 |     Args:
28 |         init_lr (float): Initial learning rate
29 |         global_step (int): Current iteration number
30 |         T (int): Total iteration number (i,e. nepoch)
31 |         M (int): Number of ensembles we want
32 | 
33 |     Returns:
34 |         float: Annealed learning rate
35 |     """
36 |     TdivM = T // M
37 |     return init_lr / 2.0 * (np.cos(np.pi * ((global_step - 1) % TdivM) / TdivM) + 1.0)
38 | 
39 | 
40 | def test_noam():
41 |     lr = 1e-3
42 |     init_lr = 1e-3
43 |     for i in range(50000):
44 |         print(i, lr)
45 |         lr = noam_learning_rate_decay(init_lr, i)
46 |         


--------------------------------------------------------------------------------
/model.py:
--------------------------------------------------------------------------------
  1 | import torch
  2 | from torch import nn
  3 | import torch.nn.functional as F
  4 | from hparams import hparams as hp
  5 | from torch.utils.data import DataLoader, Dataset
  6 | from distributions import *
  7 | from utils import num_params, mulaw_quantize, inv_mulaw_quantize
  8 | 
  9 | from tqdm import tqdm
 10 | import numpy as np
 11 | 
 12 | class ResBlock(nn.Module) :
 13 |     def __init__(self, dims) :
 14 |         super().__init__()
 15 |         self.conv1 = nn.Conv1d(dims, dims, kernel_size=1, bias=False)
 16 |         self.conv2 = nn.Conv1d(dims, dims, kernel_size=1, bias=False)
 17 |         self.batch_norm1 = nn.BatchNorm1d(dims)
 18 |         self.batch_norm2 = nn.BatchNorm1d(dims)
 19 |         
 20 |     def forward(self, x) :
 21 |         residual = x
 22 |         x = self.conv1(x)
 23 |         x = self.batch_norm1(x)
 24 |         x = F.relu(x)
 25 |         x = self.conv2(x)
 26 |         x = self.batch_norm2(x)
 27 |         return x + residual
 28 | 
 29 | class MelResNet(nn.Module) :
 30 |     def __init__(self, res_blocks, in_dims, compute_dims, res_out_dims) :
 31 |         super().__init__()
 32 |         self.conv_in = nn.Conv1d(in_dims, compute_dims, kernel_size=5, bias=False)
 33 |         self.batch_norm = nn.BatchNorm1d(compute_dims)
 34 |         self.layers = nn.ModuleList()
 35 |         for i in range(res_blocks) :
 36 |             self.layers.append(ResBlock(compute_dims))
 37 |         self.conv_out = nn.Conv1d(compute_dims, res_out_dims, kernel_size=1)
 38 |         
 39 |     def forward(self, x) :
 40 |         x = self.conv_in(x)
 41 |         x = self.batch_norm(x)
 42 |         x = F.relu(x)
 43 |         for f in self.layers : x = f(x)
 44 |         x = self.conv_out(x)
 45 |         return x
 46 | 
 47 | class Stretch2d(nn.Module) :
 48 |     def __init__(self, x_scale, y_scale) :
 49 |         super().__init__()
 50 |         self.x_scale = x_scale
 51 |         self.y_scale = y_scale
 52 |         
 53 |     def forward(self, x) :
 54 |         b, c, h, w = x.size()
 55 |         x = x.unsqueeze(-1).unsqueeze(3)
 56 |         x = x.repeat(1, 1, 1, self.y_scale, 1, self.x_scale)
 57 |         return x.view(b, c, h * self.y_scale, w * self.x_scale)
 58 | 
 59 | class UpsampleNetwork(nn.Module) :
 60 |     def __init__(self, feat_dims, upsample_scales, compute_dims, 
 61 |                  res_blocks, res_out_dims, pad) :
 62 |         super().__init__()
 63 |         total_scale = np.cumproduct(upsample_scales)[-1]
 64 |         self.indent = pad * total_scale
 65 |         self.resnet = MelResNet(res_blocks, feat_dims, compute_dims, res_out_dims)
 66 |         self.resnet_stretch = Stretch2d(total_scale, 1)
 67 |         self.up_layers = nn.ModuleList()
 68 |         for scale in upsample_scales :
 69 |             k_size = (1, scale * 2 + 1)
 70 |             padding = (0, scale)
 71 |             stretch = Stretch2d(scale, 1)
 72 |             conv = nn.Conv2d(1, 1, kernel_size=k_size, padding=padding, bias=False)
 73 |             conv.weight.data.fill_(1. / k_size[1])
 74 |             self.up_layers.append(stretch)
 75 |             self.up_layers.append(conv)
 76 |     
 77 |     def forward(self, m) :
 78 |         aux = self.resnet(m).unsqueeze(1)
 79 |         aux = self.resnet_stretch(aux)
 80 |         aux = aux.squeeze(1)
 81 |         m = m.unsqueeze(1)
 82 |         for f in self.up_layers : m = f(m)
 83 |         m = m.squeeze(1)[:, :, self.indent:-self.indent]
 84 |         return m.transpose(1, 2), aux.transpose(1, 2)
 85 | 
 86 | 
 87 | class Model(nn.Module) :
 88 |     def __init__(self, rnn_dims, fc_dims, bits, pad, upsample_factors,
 89 |                  feat_dims, compute_dims, res_out_dims, res_blocks):
 90 |         super().__init__()
 91 |         if hp.input_type == 'raw':
 92 |             self.n_classes = 2
 93 |         elif hp.input_type == 'mixture':
 94 |             # mixture requires multiple of 3, default at 10 component mixture, i.e 3 x 10 = 30
 95 |             self.n_classes = 30
 96 |         elif hp.input_type == 'mulaw':
 97 |             self.n_classes = hp.mulaw_quantize_channels
 98 |         elif hp.input_type == 'bits':
 99 |             self.n_classes = 2**bits
100 |         else:
101 |             raise ValueError("input_type: {hp.input_type} not supported")
102 |         self.rnn_dims = rnn_dims
103 |         self.aux_dims = res_out_dims // 4
104 |         self.upsample = UpsampleNetwork(feat_dims, upsample_factors, compute_dims, 
105 |                                         res_blocks, res_out_dims, pad)
106 |         self.I = nn.Linear(feat_dims + self.aux_dims + 1, rnn_dims)
107 |         self.rnn1 = nn.GRU(rnn_dims, rnn_dims, batch_first=True)
108 |         self.rnn2 = nn.GRU(rnn_dims + self.aux_dims, rnn_dims, batch_first=True)
109 |         self.fc1 = nn.Linear(rnn_dims + self.aux_dims, fc_dims)
110 |         self.fc2 = nn.Linear(fc_dims + self.aux_dims, fc_dims)
111 |         self.fc3 = nn.Linear(fc_dims, self.n_classes)
112 |         num_params(self)
113 |     
114 |     def forward(self, x, mels) :
115 |         bsize = x.size(0)
116 |         h1 = torch.zeros(1, bsize, self.rnn_dims).cuda()
117 |         h2 = torch.zeros(1, bsize, self.rnn_dims).cuda()
118 |         mels, aux = self.upsample(mels)
119 |         
120 |         aux_idx = [self.aux_dims * i for i in range(5)]
121 |         a1 = aux[:, :, aux_idx[0]:aux_idx[1]]
122 |         a2 = aux[:, :, aux_idx[1]:aux_idx[2]]
123 |         a3 = aux[:, :, aux_idx[2]:aux_idx[3]]
124 |         a4 = aux[:, :, aux_idx[3]:aux_idx[4]]
125 |         
126 |         x = torch.cat([x.unsqueeze(-1), mels, a1], dim=2)
127 |         x = self.I(x)
128 |         res = x
129 |         x, _ = self.rnn1(x, h1)
130 |         
131 |         x = x + res
132 |         res = x
133 |         x = torch.cat([x, a2], dim=2)
134 |         x, _ = self.rnn2(x, h2)
135 |         
136 |         x = x + res
137 |         x = torch.cat([x, a3], dim=2)
138 |         x = F.relu(self.fc1(x))
139 |         
140 |         x = torch.cat([x, a4], dim=2)
141 |         x = F.relu(self.fc2(x))
142 | 
143 |         x = self.fc3(x)
144 | 
145 |         if hp.input_type == 'raw':
146 |             return x
147 |         elif hp.input_type == 'mixture':
148 |             return x
149 |         elif hp.input_type == 'bits' or hp.input_type == 'mulaw':
150 |             return F.log_softmax(x, dim=-1)
151 |         else:
152 |             raise ValueError("input_type: {hp.input_type} not supported")
153 | 
154 | 
155 |     def preview_upsampling(self, mels) :
156 |         mels, aux = self.upsample(mels)
157 |         return mels, aux
158 |     
159 |     def generate(self, mels) :
160 |         self.eval()
161 |         output = []
162 |         rnn1 = self.get_gru_cell(self.rnn1)
163 |         rnn2 = self.get_gru_cell(self.rnn2)
164 |         
165 |         with torch.no_grad() :
166 |             x = torch.zeros(1, 1).cuda()
167 |             h1 = torch.zeros(1, self.rnn_dims).cuda()
168 |             h2 = torch.zeros(1, self.rnn_dims).cuda()
169 |             
170 |             mels = torch.FloatTensor(mels).cuda().unsqueeze(0)
171 |             mels, aux = self.upsample(mels)
172 |             
173 |             aux_idx = [self.aux_dims * i for i in range(5)]
174 |             a1 = aux[:, :, aux_idx[0]:aux_idx[1]]
175 |             a2 = aux[:, :, aux_idx[1]:aux_idx[2]]
176 |             a3 = aux[:, :, aux_idx[2]:aux_idx[3]]
177 |             a4 = aux[:, :, aux_idx[3]:aux_idx[4]]
178 |             
179 |             seq_len = mels.size(1)
180 |             
181 |             for i in tqdm(range(seq_len)) :
182 | 
183 |                 m_t = mels[:, i, :]
184 |                 a1_t = a1[:, i, :]
185 |                 a2_t = a2[:, i, :]
186 |                 a3_t = a3[:, i, :]
187 |                 a4_t = a4[:, i, :]
188 |                 
189 |                 x = torch.cat([x, m_t, a1_t], dim=1)
190 |                 x = self.I(x)
191 |                 h1 = rnn1(x, h1)
192 |                 
193 |                 x = x + h1
194 |                 inp = torch.cat([x, a2_t], dim=1)
195 |                 h2 = rnn2(inp, h2)
196 |                 
197 |                 x = x + h2
198 |                 x = torch.cat([x, a3_t], dim=1)
199 |                 x = F.relu(self.fc1(x))
200 |                 
201 |                 x = torch.cat([x, a4_t], dim=1)
202 |                 x = F.relu(self.fc2(x))
203 |                 x = self.fc3(x)
204 |                 if hp.input_type == 'raw':
205 |                     if hp.distribution == 'beta':
206 |                         sample = sample_from_beta_dist(x.unsqueeze(0))
207 |                     elif hp.distribution == 'gaussian':
208 |                         sample = sample_from_gaussian(x.unsqueeze(0))
209 |                 elif hp.input_type == 'mixture':
210 |                     sample = sample_from_discretized_mix_logistic(x.unsqueeze(-1),hp.log_scale_min)
211 |                 elif hp.input_type == 'bits':
212 |                     posterior = F.softmax(x, dim=1).view(-1)
213 |                     distrib = torch.distributions.Categorical(posterior)
214 |                     sample = 2 * distrib.sample().float() / (self.n_classes - 1.) - 1.
215 |                 elif hp.input_type == 'mulaw':
216 |                     posterior = F.softmax(x, dim=1).view(-1)
217 |                     distrib = torch.distributions.Categorical(posterior)
218 |                     sample = inv_mulaw_quantize(distrib.sample(), hp.mulaw_quantize_channels, True)
219 |                 output.append(sample.view(-1))
220 |                 x = torch.FloatTensor([[sample]]).cuda()
221 |         output = torch.stack(output).cpu().numpy()
222 |         self.train()
223 |         return output
224 | 
225 | 
226 |     def batch_generate(self, mels) :
227 |         """mel should be of shape [batch_size x 80 x mel_length]
228 |         """
229 |         self.eval()
230 |         output = []
231 |         rnn1 = self.get_gru_cell(self.rnn1)
232 |         rnn2 = self.get_gru_cell(self.rnn2)
233 |         b_size = mels.shape[0]
234 |         assert len(mels.shape) == 3, "mels should have shape [batch_size x 80 x mel_length]"
235 |         
236 |         with torch.no_grad() :
237 |             x = torch.zeros(b_size, 1).cuda()
238 |             h1 = torch.zeros(b_size, self.rnn_dims).cuda()
239 |             h2 = torch.zeros(b_size, self.rnn_dims).cuda()
240 |             
241 |             mels = torch.FloatTensor(mels).cuda()
242 |             mels, aux = self.upsample(mels)
243 |             
244 |             aux_idx = [self.aux_dims * i for i in range(5)]
245 |             a1 = aux[:, :, aux_idx[0]:aux_idx[1]]
246 |             a2 = aux[:, :, aux_idx[1]:aux_idx[2]]
247 |             a3 = aux[:, :, aux_idx[2]:aux_idx[3]]
248 |             a4 = aux[:, :, aux_idx[3]:aux_idx[4]]
249 |             
250 |             seq_len = mels.size(1)
251 |             
252 |             for i in tqdm(range(seq_len)) :
253 | 
254 |                 m_t = mels[:, i, :]
255 |                 a1_t = a1[:, i, :]
256 |                 a2_t = a2[:, i, :]
257 |                 a3_t = a3[:, i, :]
258 |                 a4_t = a4[:, i, :]
259 |                 
260 |                 x = torch.cat([x, m_t, a1_t], dim=1)
261 |                 x = self.I(x)
262 |                 h1 = rnn1(x, h1)
263 |                 
264 |                 x = x + h1
265 |                 inp = torch.cat([x, a2_t], dim=1)
266 |                 h2 = rnn2(inp, h2)
267 |                 
268 |                 x = x + h2
269 |                 x = torch.cat([x, a3_t], dim=1)
270 |                 x = F.relu(self.fc1(x))
271 |                 
272 |                 x = torch.cat([x, a4_t], dim=1)
273 |                 x = F.relu(self.fc2(x))
274 |                 x = self.fc3(x)
275 |                 if hp.input_type == 'raw':
276 |                     sample = sample_from_beta_dist(x.unsqueeze(0))
277 |                 elif hp.input_type == 'mixture':
278 |                     sample = sample_from_discretized_mix_logistic(x.unsqueeze(-1),hp.log_scale_min)
279 |                 elif hp.input_type == 'bits':
280 |                     posterior = F.softmax(x, dim=1).view(b_size, -1)
281 |                     distrib = torch.distributions.Categorical(posterior)
282 |                     sample = 2 * distrib.sample().float() / (self.n_classes - 1.) - 1.
283 |                 elif hp.input_type == 'mulaw':
284 |                     posterior = F.softmax(x, dim=1).view(b_size, -1)
285 |                     distrib = torch.distributions.Categorical(posterior)
286 |                     print(type(distrib.sample()))
287 |                     sample = inv_mulaw_quantize(distrib.sample(), hp.mulaw_quantize_channels, True)
288 |                 output.append(sample.view(-1))
289 |                 x = sample.view(b_size,1)
290 |         output = torch.stack(output).cpu().numpy()
291 |         self.train()
292 |         # output is a batch of wav segments of shape [batch_size x seq_len]
293 |         # will need to merge into one wav of size [batch_size * seq_len]
294 |         assert output.shape[1] == b_size
295 |         output = (output.swapaxes(1,0)).reshape(-1)
296 |         return output
297 |     
298 |     def get_gru_cell(self, gru) :
299 |         gru_cell = nn.GRUCell(gru.input_size, gru.hidden_size)
300 |         gru_cell.weight_hh.data = gru.weight_hh_l0.data
301 |         gru_cell.weight_ih.data = gru.weight_ih_l0.data
302 |         gru_cell.bias_hh.data = gru.bias_hh_l0.data
303 |         gru_cell.bias_ih.data = gru.bias_ih_l0.data
304 |         return gru_cell
305 | 
306 | 
307 | def build_model():
308 |     """build model with hparams settings
309 | 
310 |     """
311 |     if hp.input_type == 'raw':
312 |         print('building model with Beta distribution output')
313 |     elif hp.input_type == 'mixture':
314 |         print("building model with mixture of logistic output")
315 |     elif hp.input_type == 'bits':
316 |         print("building model with quantized bit audio")
317 |     elif hp.input_type == 'mulaw':
318 |         print("building model with quantized mulaw encoding")
319 |     else:
320 |         raise ValueError('input_type provided not supported')
321 |     model = Model(hp.rnn_dims, hp.fc_dims, hp.bits,
322 |         hp.pad, hp.upsample_factors, hp.num_mels,
323 |         hp.compute_dims, hp.res_out_dims, hp.res_blocks)
324 | 
325 |     return model 
326 | 
327 | def no_test_build_model():
328 |     model = Model(hp.rnn_dims, hp.fc_dims, hp.bits,
329 |         hp.pad, hp.upsample_factors, hp.num_mels,
330 |         hp.compute_dims, hp.res_out_dims, hp.res_blocks).cuda()
331 |     print(vars(model))
332 | 
333 | 
334 | def test_batch_generate():
335 |     model = Model(hp.rnn_dims, hp.fc_dims, hp.bits,
336 |         hp.pad, hp.upsample_factors, hp.num_mels,
337 |         hp.compute_dims, hp.res_out_dims, hp.res_blocks).cuda()
338 |     print(vars(model))
339 |     batch_mel = torch.rand(3, 80, 100)
340 |     output = model.batch_generate(batch_mel)
341 |     print(output.shape)


--------------------------------------------------------------------------------
/preprocess.py:
--------------------------------------------------------------------------------
  1 | """
  2 | Preprocess dataset
  3 | 
  4 | usage: preproess.py [options] <wav-dir>
  5 | 
  6 | options:
  7 |      --output-dir=<dir>      Directory where processed outputs are saved. [default: data_dir].
  8 |     -h, --help              Show help message.
  9 | """
 10 | import os
 11 | from docopt import docopt
 12 | import numpy as np
 13 | import math, pickle, os
 14 | from audio import *
 15 | from hparams import hparams as hp
 16 | from utils import *
 17 | from tqdm import tqdm
 18 | 
 19 | def get_wav_mel(path):
 20 |     """Given path to .wav file, get the quantized wav and mel spectrogram as numpy vectors
 21 | 
 22 |     """
 23 |     wav = load_wav(path)
 24 |     mel = melspectrogram(wav)
 25 |     if hp.input_type == 'raw':
 26 |         return wav.astype(np.float32), mel
 27 |     elif hp.input_type == 'mulaw':
 28 |         quant = mulaw_quantize(wav, hp.mulaw_quantize_channels)
 29 |         return quant.astype(np.int), mel
 30 |     elif hp.input_type == 'bits':
 31 |         quant = quantize(wav)
 32 |         return quant.astype(np.int), mel
 33 |     else:
 34 |         raise ValueError("hp.input_type {} not recognized".format(hp.input_type))
 35 | 
 36 | 
 37 | 
 38 | 
 39 | 
 40 | def process_data(wav_dir, output_path, mel_path, wav_path):
 41 |     """
 42 |     given wav directory and output directory, process wav files and save quantized wav and mel
 43 |     spectrogram to output directory
 44 |     """
 45 |     dataset_ids = []
 46 |     # get list of wav files
 47 |     wav_files = os.listdir(wav_dir)
 48 |     # check wav_file
 49 |     assert len(wav_files) != 0 or wav_files[0][-4:] == '.wav', "no wav files found!"
 50 |     # create training and testing splits
 51 |     test_wav_files = wav_files[:4]
 52 |     wav_files = wav_files[4:]
 53 |     for i, wav_file in enumerate(tqdm(wav_files)):
 54 |         # get the file id
 55 |         file_id = '{:d}'.format(i).zfill(5)
 56 |         wav, mel = get_wav_mel(os.path.join(wav_dir,wav_file))
 57 |         # save
 58 |         np.save(os.path.join(mel_path,file_id+".npy"), mel)
 59 |         np.save(os.path.join(wav_path,file_id+".npy"), wav)
 60 |         # add to dataset_ids
 61 |         dataset_ids.append(file_id)
 62 | 
 63 |     # save dataset_ids
 64 |     with open(os.path.join(output_path,'dataset_ids.pkl'), 'wb') as f:
 65 |         pickle.dump(dataset_ids, f)
 66 | 
 67 |     # process testing_wavs
 68 |     test_path = os.path.join(output_path,'test')
 69 |     os.makedirs(test_path, exist_ok=True)
 70 |     for i, wav_file in enumerate(test_wav_files):
 71 |         wav, mel = get_wav_mel(os.path.join(wav_dir,wav_file))
 72 |         # save test_wavs
 73 |         np.save(os.path.join(test_path,"test_{}_mel.npy".format(i)),mel)
 74 |         np.save(os.path.join(test_path,"test_{}_wav.npy".format(i)),wav)
 75 | 
 76 |     
 77 |     print("\npreprocessing done, total processed wav files:{}.\nProcessed files are located in:{}".format(len(wav_files), os.path.abspath(output_path)))
 78 | 
 79 | 
 80 | 
 81 | if __name__=="__main__":
 82 |     args = docopt(__doc__)
 83 |     wav_dir = args["<wav-dir>"]
 84 |     output_dir = args["--output-dir"]
 85 | 
 86 |     # create paths
 87 |     output_path = os.path.join(output_dir,"")
 88 |     mel_path = os.path.join(output_dir,"mel")
 89 |     wav_path = os.path.join(output_dir,"wav")
 90 | 
 91 |     # create dirs
 92 |     os.makedirs(output_path, exist_ok=True)
 93 |     os.makedirs(mel_path, exist_ok=True)
 94 |     os.makedirs(wav_path, exist_ok=True)
 95 | 
 96 |     # process data
 97 |     process_data(wav_dir, output_path, mel_path, wav_path)
 98 | 
 99 | 
100 | 
101 | def test_get_wav_mel():
102 |     wav, mel = get_wav_mel('sample.wav')
103 |     print(wav.shape, mel.shape)
104 |     print(wav)


--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | docopt
2 | librosa
3 | nnmnkwii
4 | tqdm
5 | lws


--------------------------------------------------------------------------------
/train.py:
--------------------------------------------------------------------------------
  1 | """Training WaveRNN Model.
  2 | 
  3 | usage: train.py [options] <data-root>
  4 | 
  5 | options:
  6 |     --checkpoint-dir=<dir>      Directory where to save model checkpoints [default: checkpoints].
  7 |     --checkpoint=<path>         Restore model from checkpoint path if given.
  8 |     -h, --help                  Show this help message and exit
  9 | """
 10 | from docopt import docopt
 11 | 
 12 | import os
 13 | from os.path import dirname, join, expanduser
 14 | from tqdm import tqdm
 15 | 
 16 | import numpy as np
 17 | import matplotlib.pyplot as plt
 18 | import librosa
 19 | 
 20 | from model import build_model
 21 | 
 22 | import torch
 23 | from torch import nn
 24 | import torch.nn.functional as F
 25 | from torch import optim
 26 | from torch.utils.data import DataLoader
 27 | 
 28 | from model import build_model
 29 | from distributions import *
 30 | from loss_function import nll_loss
 31 | from dataset import raw_collate, discrete_collate, AudiobookDataset
 32 | from hparams import hparams as hp
 33 | from lrschedule import noam_learning_rate_decay, step_learning_rate_decay
 34 | 
 35 | global_step = 0
 36 | global_epoch = 0
 37 | global_test_step = 0
 38 | use_cuda = torch.cuda.is_available()
 39 | 
 40 | def save_checkpoint(device, model, optimizer, step, checkpoint_dir, epoch):
 41 |     checkpoint_path = join(
 42 |         checkpoint_dir, "checkpoint_step{:09d}.pth".format(step))
 43 |     optimizer_state = optimizer.state_dict()
 44 |     global global_test_step
 45 |     torch.save({
 46 |         "state_dict": model.state_dict(),
 47 |         "optimizer": optimizer_state,
 48 |         "global_step": step,
 49 |         "global_epoch": epoch,
 50 |         "global_test_step": global_test_step,
 51 |     }, checkpoint_path)
 52 |     print("Saved checkpoint:", checkpoint_path)
 53 | 
 54 | 
 55 | def _load(checkpoint_path):
 56 |     if use_cuda:
 57 |         checkpoint = torch.load(checkpoint_path)
 58 |     else:
 59 |         checkpoint = torch.load(checkpoint_path,
 60 |                                 map_location=lambda storage, loc: storage)
 61 |     return checkpoint
 62 | 
 63 | 
 64 | def load_checkpoint(path, model, optimizer, reset_optimizer):
 65 |     global global_step
 66 |     global global_epoch
 67 |     global global_test_step
 68 | 
 69 |     print("Load checkpoint from: {}".format(path))
 70 |     checkpoint = _load(path)
 71 |     model.load_state_dict(checkpoint["state_dict"])
 72 |     if not reset_optimizer:
 73 |         optimizer_state = checkpoint["optimizer"]
 74 |         if optimizer_state is not None:
 75 |             print("Load optimizer state from {}".format(path))
 76 |             optimizer.load_state_dict(checkpoint["optimizer"])
 77 |     global_step = checkpoint["global_step"]
 78 |     global_epoch = checkpoint["global_epoch"]
 79 |     global_test_step = checkpoint.get("global_test_step", 0)
 80 | 
 81 |     return model
 82 | 
 83 | 
 84 | def test_save_checkpoint():
 85 |     checkpoint_path = "checkpoints/"
 86 |     device = torch.device("cuda" if use_cuda else "cpu")
 87 |     model = build_model()
 88 |     optimizer = optim.Adam(model.parameters(), lr=1e-4)
 89 |     global global_step, global_epoch, global_test_step
 90 |     save_checkpoint(device, model, optimizer, global_step, checkpoint_path, global_epoch)
 91 | 
 92 |     model = load_checkpoint(checkpoint_path+"checkpoint_step000000000.pth", model, optimizer, False)
 93 | 
 94 | 
 95 | def evaluate_model(model, data_loader, checkpoint_dir, limit_eval_to=5):
 96 |     """evaluate model and save generated wav and plot
 97 | 
 98 |     """
 99 |     test_path = data_loader.dataset.test_path
100 |     test_files = os.listdir(test_path)
101 |     counter = 0
102 |     output_dir = os.path.join(checkpoint_dir,'eval')
103 |     for f in test_files:
104 |         if f[-7:] == "mel.npy":
105 |             mel = np.load(os.path.join(test_path,f))
106 |             wav = model.generate(mel)
107 |             # save wav
108 |             wav_path = os.path.join(output_dir,"checkpoint_step{:09d}_wav_{}.wav".format(global_step,counter))
109 |             librosa.output.write_wav(wav_path, wav, sr=hp.sample_rate)
110 |             # save wav plot
111 |             fig_path = os.path.join(output_dir,"checkpoint_step{:09d}_wav_{}.png".format(global_step,counter))
112 |             fig = plt.plot(wav.reshape(-1))
113 |             plt.savefig(fig_path)
114 |             # clear fig to drawing to the same plot
115 |             plt.clf()
116 |             counter += 1
117 |         # stop evaluation early via limit_eval_to
118 |         if counter >= limit_eval_to:
119 |             break
120 | 
121 | 
122 | def train_loop(device, model, data_loader, optimizer, checkpoint_dir):
123 |     """Main training loop.
124 | 
125 |     """
126 |     # create loss and put on device
127 |     if hp.input_type == 'raw':
128 |         if hp.distribution == 'beta':
129 |             criterion = beta_mle_loss
130 |         elif hp.distribution == 'gaussian':
131 |             criterion = gaussian_loss
132 |     elif hp.input_type == 'mixture':
133 |         criterion = discretized_mix_logistic_loss
134 |     elif hp.input_type in ["bits", "mulaw"]:
135 |         criterion = nll_loss
136 |     else:
137 |         raise ValueError("input_type:{} not supported".format(hp.input_type))
138 | 
139 |     
140 | 
141 |     global global_step, global_epoch, global_test_step
142 |     while global_epoch < hp.nepochs:
143 |         running_loss = 0
144 |         for i, (x, m, y) in enumerate(tqdm(data_loader)):
145 |             x, m, y = x.to(device), m.to(device), y.to(device)
146 |             y_hat = model(x, m)
147 |             y = y.unsqueeze(-1)
148 |             loss = criterion(y_hat, y)
149 |             # calculate learning rate and update learning rate
150 |             if hp.fix_learning_rate:
151 |                 current_lr = hp.fix_learning_rate
152 |             elif hp.lr_schedule_type == 'step':
153 |                 current_lr = step_learning_rate_decay(hp.initial_learning_rate, global_step, hp.step_gamma, hp.lr_step_interval)
154 |             else:
155 |                 current_lr = noam_learning_rate_decay(hp.initial_learning_rate, global_step, hp.noam_warm_up_steps)
156 |             for param_group in optimizer.param_groups:
157 |                 param_group['lr'] = current_lr
158 |             optimizer.zero_grad()
159 |             loss.backward()
160 |             # clip gradient norm
161 |             nn.utils.clip_grad_norm_(model.parameters(), hp.grad_norm)
162 |             optimizer.step()
163 | 
164 |             running_loss += loss.item()
165 |             avg_loss = running_loss / (i+1)
166 |             # saving checkpoint if needed
167 |             if global_step != 0 and global_step % hp.save_every_step == 0:
168 |                 save_checkpoint(device, model, optimizer, global_step, checkpoint_dir, global_epoch)
169 |             # evaluate model if needed
170 |             if global_step != 0 and global_test_step !=True and global_step % hp.evaluate_every_step == 0:
171 |                 print("step {}, evaluating model: generating wav from mel...".format(global_step))
172 |                 evaluate_model(model, data_loader, checkpoint_dir)
173 |                 print("evaluation finished, resuming training...")
174 | 
175 |             # reset global_test_step status after evaluation
176 |             if global_test_step is True:
177 |                 global_test_step = False
178 |             global_step += 1
179 |         
180 |         print("epoch:{}, running loss:{}, average loss:{}, current lr:{}".format(global_epoch, running_loss, avg_loss, current_lr))
181 |         global_epoch += 1
182 | 
183 | 
184 | 
185 | if __name__=="__main__":
186 |     args = docopt(__doc__)
187 |     #print("Command line args:\n", args)
188 |     checkpoint_dir = args["--checkpoint-dir"]
189 |     checkpoint_path = args["--checkpoint"]
190 |     data_root = args["<data-root>"]
191 | 
192 |     # make dirs, load dataloader and set up device
193 |     os.makedirs(checkpoint_dir, exist_ok=True)
194 |     os.makedirs(os.path.join(checkpoint_dir,'eval'), exist_ok=True)
195 |     dataset = AudiobookDataset(data_root)
196 |     if hp.input_type == 'raw':
197 |         collate_fn = raw_collate
198 |     elif hp.input_type == 'mixture':
199 |         collate_fn = raw_collate
200 |     elif hp.input_type in ['bits', 'mulaw']:
201 |         collate_fn = discrete_collate
202 |     else:
203 |         raise ValueError("input_type:{} not supported".format(hp.input_type))
204 |     data_loader = DataLoader(dataset, collate_fn=collate_fn, shuffle=True, num_workers=0, batch_size=hp.batch_size)
205 |     device = torch.device("cuda" if use_cuda else "cpu")
206 |     print("using device:{}".format(device))
207 | 
208 |     # build model, create optimizer
209 |     model = build_model().to(device)
210 |     optimizer = optim.Adam(model.parameters(),
211 |                            lr=hp.initial_learning_rate, betas=(
212 |         hp.adam_beta1, hp.adam_beta2),
213 |         eps=hp.adam_eps, weight_decay=hp.weight_decay,
214 |         amsgrad=hp.amsgrad)
215 | 
216 |     if hp.fix_learning_rate:
217 |         print("using fixed learning rate of :{}".format(hp.fix_learning_rate))
218 |     elif hp.lr_schedule_type == 'step':
219 |         print("using exponential learning rate decay")
220 |     elif hp.lr_schedule_type == 'noam':
221 |         print("using noam learning rate decay")
222 | 
223 |     # load checkpoint
224 |     if checkpoint_path is None:
225 |         print("no checkpoint specified as --checkpoint argument, creating new model...")
226 |     else:
227 |         model = load_checkpoint(checkpoint_path, model, optimizer, False)
228 |         print("loading model from checkpoint:{}".format(checkpoint_path))
229 |         # set global_test_step to True so we don't evaluate right when we load in the model
230 |         global_test_step = True
231 | 
232 |     # main train loop
233 |     try:
234 |         train_loop(device, model, data_loader, optimizer, checkpoint_dir)
235 |     except KeyboardInterrupt:
236 |         print("Interrupted!")
237 |         pass
238 |     finally:
239 |         print("saving model....")
240 |         save_checkpoint(device, model, optimizer, global_step, checkpoint_dir, global_epoch)
241 |     
242 | 
243 | def test_eval():
244 |     data_root = "data_dir"
245 |     dataset = AudiobookDataset(data_root)
246 |     if hp.input_type == 'raw':
247 |         collate_fn = raw_collate
248 |     elif hp.input_type == 'bits':
249 |         collate_fn = discrete_collate
250 |     else:
251 |         raise ValueError("input_type:{} not supported".format(hp.input_type))
252 |     data_loader = DataLoader(dataset, collate_fn=collate_fn, shuffle=True, num_workers=0, batch_size=hp.batch_size)
253 |     device = torch.device("cuda" if use_cuda else "cpu")
254 |     print("using device:{}".format(device))
255 | 
256 |     # build model, create optimizer
257 |     model = build_model().to(device)
258 | 
259 |     evaluate_model(model, data_loader)
260 | 
261 |     
262 | 
263 |     
264 | 
265 | 


--------------------------------------------------------------------------------
/utils.py:
--------------------------------------------------------------------------------
 1 | import numpy as np
 2 | import torch
 3 | 
 4 | def num_params(model) :
 5 |     parameters = filter(lambda p: p.requires_grad, model.parameters())
 6 |     parameters = sum([np.prod(p.size()) for p in parameters]) / 1000000
 7 |     print('Trainable Parameters: %.3f million' % parameters)
 8 | 
 9 | 
10 | # for mulaw encoding and decoding in torch tensors, modified from: https://github.com/pytorch/audio/blob/master/torchaudio/transforms.py
11 | def mulaw_quantize(x, quantization_channels=256):
12 |     """Encode signal based on mu-law companding.  For more info see the
13 |     `Wikipedia Entry <https://en.wikipedia.org/wiki/%CE%9C-law_algorithm>`_
14 | 
15 |     This algorithm assumes the signal has been scaled to between -1 and 1 and
16 |     returns a signal encoded with values from 0 to quantization_channels - 1
17 | 
18 |     Args:
19 |         quantization_channels (int): Number of channels. default: 256
20 | 
21 |     """
22 |     mu = quantization_channels - 1
23 |     if isinstance(x, np.ndarray):
24 |         x_mu = np.sign(x) * np.log1p(mu * np.abs(x)) / np.log1p(mu)
25 |         x_mu = ((x_mu + 1) / 2 * mu + 0.5).astype(int)
26 |     elif isinstance(x, (torch.Tensor, torch.LongTensor)):
27 | 
28 |         if isinstance(x, torch.LongTensor):
29 |             x = x.float()
30 |         mu = torch.FloatTensor([mu])
31 |         x_mu = torch.sign(x) * torch.log1p(mu * torch.abs(x)) / torch.log1p(mu)
32 |         x_mu = ((x_mu + 1) / 2 * mu + 0.5).long()
33 |     return x_mu
34 | 
35 | 
36 | def inv_mulaw_quantize(x_mu, quantization_channels=256, cuda=False):
37 |     """Decode mu-law encoded signal.  For more info see the
38 |     `Wikipedia Entry <https://en.wikipedia.org/wiki/%CE%9C-law_algorithm>`_
39 | 
40 |     This expects an input with values between 0 and quantization_channels - 1
41 |     and returns a signal scaled between -1 and 1.
42 | 
43 |     Args:
44 |         quantization_channels (int): Number of channels. default: 256
45 | 
46 |     """
47 |     mu = quantization_channels - 1.
48 |     if isinstance(x_mu, np.ndarray):
49 |         x = ((x_mu) / mu) * 2 - 1.
50 |         x = np.sign(x) * (np.exp(np.abs(x) * np.log1p(mu)) - 1.) / mu
51 |     elif isinstance(x_mu, (torch.Tensor, torch.LongTensor)):
52 |         if isinstance(x_mu, (torch.LongTensor, torch.cuda.LongTensor)):
53 |             x_mu = x_mu.float()
54 |         if cuda:
55 |             mu = (torch.FloatTensor([mu])).cuda()
56 |         else:
57 |             mu = torch.FloatTensor([mu])
58 |         x = ((x_mu) / mu) * 2 - 1.
59 |         x = torch.sign(x) * (torch.exp(torch.abs(x) * torch.log1p(mu)) - 1.) / mu
60 |     return x
61 | 
62 | 
63 | def test_inv_mulaw():
64 |     wav = torch.rand(5, 5000)
65 |     wav = wav.cuda()
66 |     de_quant = inv_mulaw_quantize(wav, 512, True)


--------------------------------------------------------------------------------