Example pianoroll
26 | 27 | ## Multitrack Pianoroll 28 | 29 | We represent a multitrack music piece with a _multitrack pianoroll_, which is a 30 | set of pianorolls where each pianoroll represents one specific track in the 31 | original music piece. That is, a _M_-track music piece will be converted into a 32 | set of _M_ pianorolls. For instance, a bar in 4/4 time with _M_ tracks can be 33 | represented as a 96 x 128 x _M_ tensor. 34 | 35 |
36 | Example five-track pianorolls
37 | 38 | > The above pianoroll visualizations are produced using 39 | [Pypianoroll](https://salu133445.github.io/pypianoroll/). 40 | -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | The Lakh Pianoroll Dataset (LPD) is a collection of 174,154 2 | [multitrack pianorolls](representation) derived from the 3 | [Lakh MIDI Dataset](http://colinraffel.com/projects/lmd/) (LMD). 4 | 5 | ## Getting the dataset 6 | 7 | We provide multiple subsets and versions of the dataset (see 8 | [here](comparisons)). The dataset is available [here](dataset). 9 | 10 | ## Using LPD 11 | 12 | The multitrack pianorolls in LPD are stored in a special format for efficient 13 | I/O and to save space. We recommend to load the data with 14 | [Pypianoroll](https://salu133445.github.io/pypianoroll/) (The dataset is created 15 | using Pypianoroll v0.3.0.). See [here](https://salu133445.github.io/pypianoroll/save_load.html) 16 | to learn how the data is stored and how to load the data properly. 17 | 18 | ## License 19 | 20 | Lakh Pianoroll Dataset is a derivative of 21 | [Lakh MIDI Dataset](http://colinraffel.com/projects/lmd/) by 22 | [Colin Raffel](http://colinraffel.com), used under 23 | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 24 | Lakh Pianoroll Dataset is licensed under 25 | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by 26 | [Hao-Wen Dong](https://salu133445.github.io) and 27 | [Wen-Yi Hsiao](https://github.com/wayne391). 28 | 29 | Please cite the following papers if you use Lakh Pianoroll Dataset in a 30 | published work. 31 | 32 | - Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, and Yi-Hsuan Yang, 33 | "__MuseGAN: Multi-track Sequential Generative Adversarial Networks for 34 | Symbolic Music Generation and Accompaniment__," 35 | in _Proceedings of the 32nd AAAI Conference on Artificial Intelligence_ 36 | (AAAI), 2018. 37 | 38 | - Colin Raffel, 39 | "__Learning-Based Methods for Comparing Sequences, with Applications to 40 | Audio-to-MIDI Alignment and Matching__," 41 | _PhD Thesis_, 2016. 42 | 43 | ## Related projects 44 | 45 | - [MuseGAN](https://salu133445.github.io/musegan/) 46 | - [LeadSheetGAN](https://liuhaumin.github.io/LeadsheetArrangement/) 47 | -------------------------------------------------------------------------------- /scripts/synthesize.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script synthesizes a multitrack pianoroll (in .npz) to an mp3 file. 3 | # Usage: 4 | # synthesize.sh [-b bit_rate] [-f sampling_rate] [-k] [-t tempo] [-y] [dst] \ 5 | # [src] 6 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" > /dev/null && pwd )" 7 | SRC_DIR="$DIR/../src/" 8 | 9 | function usage { 10 | echo "Usage:" 11 | echo " synthesize.sh [-b bit_rate] [-f sampling_rate] [-k] [-t tempo] \\" 12 | echo " [dst] [src]" 13 | echo "Options:" 14 | echo " -b bit_rate bit rate in bit/sec (default to 192k)" 15 | echo " -f sampling_rate sampling rate in Hz (default to 44100)" 16 | echo " -h display help" 17 | echo " -k keep the intermediately-created .wav file" 18 | echo " -t tempo tempo in bpm (default to follow the MIDI file)" 19 | echo " -y overwrite output file if it exists" 20 | exit 1 21 | } 22 | 23 | if [[ $# -eq 0 ]] 24 | then 25 | usage 26 | fi 27 | 28 | SRC="${@: -1}" 29 | DST="${@:(-2):1}" 30 | BR="192k" 31 | FS=44100 32 | KEEP=false 33 | TEMPO=0 34 | YES=false 35 | while [[ $# -gt 0 ]] 36 | do 37 | key="$1" 38 | case $key in 39 | -b|--br|--bit_rate) 40 | BR="$2" 41 | shift 2 42 | ;; 43 | -f|--fs|--sampling_rate) 44 | FS="$2" 45 | shift 2 46 | ;; 47 | -k|--keep|--keep_wav) 48 | KEEP=true 49 | shift 50 | ;; 51 | -t|--tempo) 52 | TEMPO="$2" 53 | shift 2 54 | ;; 55 | -h|--help) 56 | usage 57 | ;; 58 | -y) 59 | YES=true 60 | shift 61 | ;; 62 | *) 63 | shift 64 | ;; 65 | esac 66 | done 67 | 68 | if [ "$TEMPO" -gt 0 ]; then 69 | python2 "$SRC_DIR/npz_to_audio.py" "${DST%.mp3}.wav" "$SRC" --fs "$FS" \ 70 | --tempo "$TEMPO" 71 | else 72 | python2 "$SRC_DIR/npz_to_audio.py" "${DST%.mp3}.wav" "$SRC" --fs "$FS" 73 | fi 74 | 75 | if $YES; then 76 | ffmpeg -y -i "${DST%.mp3}.wav" -vn -ar "$FS" -ab "$BR" -f mp3 "$DST" 77 | else 78 | ffmpeg -i "${DST%.mp3}.wav" -vn -ar "$FS" -ab "$BR" -f mp3 "$DST" 79 | fi 80 | 81 | if ! $KEEP; then 82 | rm -rf "${DST%.mp3}.wav" 83 | fi 84 | -------------------------------------------------------------------------------- /src/derive_id_lists_tagtraum.py: -------------------------------------------------------------------------------- 1 | """This file derives the ID lists for different labels in the Tagtraum genre 2 | annotations (see http://www.tagtraum.com/msd_genre_datasets.html).""" 3 | import os.path 4 | import argparse 5 | from utils import make_sure_path_exists 6 | 7 | def parse_args(): 8 | """Return the parsed command line arguments.""" 9 | parser = argparse.ArgumentParser() 10 | parser.add_argument('result_dir', help="path to save the ID lists") 11 | parser.add_argument('src', help="path to the labels") 12 | parser.add_argument('subset_ids_path', 13 | help="path to the ID list of the target subset") 14 | args = parser.parse_args() 15 | return args.result_dir, args.src, args.subset_ids_path 16 | 17 | def main(): 18 | """Main function.""" 19 | result_dir, src, subset_ids_path = parse_args() 20 | 21 | # Parse the label of each song 22 | tag_dict = {} 23 | with open(src) as f: 24 | for line in f: 25 | if line.startswith('#'): 26 | continue 27 | elif len(line.split()) == 2: 28 | tag_dict[line.split()[0]] = line.split()[1] 29 | elif len(line.split()) > 2: 30 | tag_dict[line.split()[0]] = '-'.join(line.split()[1:]) 31 | 32 | tags = set(tag_dict.values()) 33 | id_lists = {tag: [] for tag in tags} 34 | 35 | # Load the IDs of the songs in the subset 36 | with open(subset_ids_path) as f: 37 | subset_ids = [line.rstrip('\n').split()[1] for line in f] 38 | 39 | # Loop over all the songs in the subsets 40 | for msd_id in subset_ids: 41 | tag = tag_dict.get(msd_id) 42 | if tag is None: 43 | continue 44 | # Add the ID to the corresponding tag 45 | id_lists[tag].append(msd_id) 46 | 47 | # Save the ID lists to files 48 | make_sure_path_exists(result_dir) 49 | for tag in tags: 50 | filename = 'id_list_{}.txt'.format(tag) 51 | with open(os.path.join(result_dir, filename), 'w') as f: 52 | for msd_id in id_lists[tag]: 53 | f.write(msd_id + '\n') 54 | 55 | print("ID lists for Tagtraum genre annotations successfully saved.") 56 | 57 | if __name__ == "__main__": 58 | main() 59 | -------------------------------------------------------------------------------- /docs/labels.md: -------------------------------------------------------------------------------- 1 | # Labels 2 | 3 | We derived the labels for the Lakh Pianoroll Dataset (LPD) from three different 4 | sources: the Last.fm Dataset, the Million Song Dataset (MSD) Benchmarks and 5 | the Tagtraum genre annotations. Note that these labels are derived based on the 6 | mapping between the Lakh MIDI Dataset (LMD) and the MSD, which may contain 7 | incorrect pairs (see [here](https://colinraffel.com/projects/lmd/)). 8 | 9 | > All the labels below can be downloaded as a ZIP file [here](https://ucsdcloud-my.sharepoint.com/:u:/g/personal/h3dong_ucsd_edu/Ebyo4GUGkIhKop7TAIw98agB1qZ8NVerIvT5W9Nry9Cn5Q?e=xjgOiR). Please refer to the original sources for the license information. 10 | 11 | ## Last.fm Dataset 12 | 13 | - [id_lists_lastfm.tar.gz](https://ucsdcloud-my.sharepoint.com/:u:/g/personal/h3dong_ucsd_edu/EbVdGIKAuC9Hn5BTkCLhIzMBRaF-X91EvI77ZSTs9aO2lA?e=rgxLjm): 14 | we derived the ID lists for most common labels in the Last.fm Dataset (see 15 | [here](http://millionsongdataset.com/lastfm/)). 16 | 17 | ## Million Song Dataset Benchmarks 18 | 19 | - [id_lists_amg.tar.gz](https://ucsdcloud-my.sharepoint.com/:u:/g/personal/h3dong_ucsd_edu/EbqjoeUiIj9DhODGZOIVBIMBTK1PhDN86rRb29_iTmfhuQ): 20 | we derived the ID lists for all the labels in the MSD AllMusic Top Genre 21 | Dataset (TopMAGD) provided in the Million Song Dataset Benchmarks (see 22 | [here](http://www.ifs.tuwien.ac.at/mir/msd/)). 23 | - [masd_labels.txt](https://ucsdcloud-my.sharepoint.com/:t:/g/personal/h3dong_ucsd_edu/Ec1B0IGvabhJgzvhfskjKlQBcQQuePIGOpmM67O5twpbxg?e=yZpCdz): 24 | we also derived the labels from the MSD AllMusic Style Dataset (MASD) provided 25 | in the Million Song Dataset Benchmarks. Each entry has at most one label. 26 | - [masd_labels_cleansed.txt](https://ucsdcloud-my.sharepoint.com/:t:/g/personal/h3dong_ucsd_edu/EWait2PjXF9Pk7T_8D_zu0ABfDi-kWCBOI1M7PqLsMvLIA?e=Mcwz2I): 27 | this is a cleansed version of *masd_labels.txt*, where less common labels are 28 | discarded. 29 | 30 | ## Tagtraum genre annotations 31 | 32 | - [id_lists_tagtraum.tar.gz](https://ucsdcloud-my.sharepoint.com/:u:/g/personal/h3dong_ucsd_edu/EaPOfwt7maBHkxAAtKv5U58BdRSJp3c-_5TS18opcskkCA?e=D1Dhts): 33 | we derived the ID lists for all the labels in the Tagtraum genre annotations 34 | (see [here](http://www.tagtraum.com/msd_genre_datasets.html)). 35 | -------------------------------------------------------------------------------- /src/derive_id_lists_amg.py: -------------------------------------------------------------------------------- 1 | """This file derives the ID lists for different labels in the MSD AllMusic Top 2 | Genre Dataset (TopMAGD) provided in the Million Song Dataset Benchmarks ( 3 | see http://www.ifs.tuwien.ac.at/mir/msd/).""" 4 | import os.path 5 | import argparse 6 | from utils import make_sure_path_exists 7 | 8 | def parse_args(): 9 | """Return the parsed command line arguments.""" 10 | parser = argparse.ArgumentParser() 11 | parser.add_argument('result_dir', help="path to save the ID lists") 12 | parser.add_argument('src', help="path to the labels") 13 | parser.add_argument('subset_ids_path', 14 | help="path to the ID list of the target subset") 15 | args = parser.parse_args() 16 | return args.result_dir, args.src, args.subset_ids_path 17 | 18 | def main(): 19 | """Main function.""" 20 | result_dir, src, subset_ids_path = parse_args() 21 | 22 | # Parse the label of each song 23 | tag_dict = {} 24 | with open(src) as f: 25 | for line in f: 26 | if line.startswith('#'): 27 | continue 28 | elif len(line.split()) == 2: 29 | tag_dict[line.split()[0]] = line.split()[1] 30 | elif len(line.split()) > 2: 31 | tag_dict[line.split()[0]] = '-'.join(line.split()[1:]) 32 | 33 | tags = set(tag_dict.values()) 34 | id_lists = {tag: [] for tag in tags} 35 | 36 | # Load the IDs of the songs in the subset 37 | with open(subset_ids_path) as f: 38 | subset_ids = [line.rstrip('\n').split()[1] for line in f] 39 | 40 | # Loop over all the songs in the subset 41 | for msd_id in subset_ids: 42 | tag = tag_dict.get(msd_id) 43 | if tag is None: 44 | continue 45 | # Add the ID to the corresponding tag 46 | id_lists[tag].append(msd_id) 47 | 48 | # Save the ID lists to files 49 | make_sure_path_exists(result_dir) 50 | for tag in tags: 51 | filename = 'id_list_{}.txt'.format(tag) 52 | with open(os.path.join(result_dir, filename), 'w') as f: 53 | for msd_id in id_lists[tag]: 54 | f.write(msd_id + '\n') 55 | 56 | print("ID lists for Million Song Dataset Benchmarks successfully saved.") 57 | 58 | if __name__ == "__main__": 59 | main() 60 | -------------------------------------------------------------------------------- /src/cleanser.py: -------------------------------------------------------------------------------- 1 | """This script cleanses a dataset and write the qualified IDs to a file. 2 | 3 | Cleansing Rules 4 | --------------- 5 | 6 | - Remove those having more than one time signature change events 7 | - Remove those having a time signature other than 4/4 8 | - Remove those whose first beat not starting from time zero 9 | - Keep only one file that has the highest confidence score in matching for 10 | each song 11 | 12 | """ 13 | import argparse 14 | import os.path 15 | import json 16 | from config import CONFIG 17 | 18 | def parse_args(): 19 | """Return the parsed command line arguments.""" 20 | parser = argparse.ArgumentParser() 21 | parser.add_argument('filepath', help="path to save the resulting list") 22 | parser.add_argument('src', help="root path to the source dataset") 23 | parser.add_argument('match_scores_path', 24 | help="path to the match scores file") 25 | parser.add_argument('midi_info_path', help="path to the MIDI info file") 26 | 27 | args = parser.parse_args() 28 | return args.filepath, args.src, args.match_scores_path, args.midi_info_path 29 | 30 | def midi_filter(midi_info): 31 | """Return True for qualified MIDI files and False for unwanted ones.""" 32 | if midi_info['first_beat_time'] > 0.0: 33 | return False 34 | if midi_info['constant_time_signature'] not in CONFIG['time_signatures']: 35 | return False 36 | return True 37 | 38 | def main(): 39 | """Main function.""" 40 | filepath, src, match_scores_path, midi_info_path = parse_args() 41 | with open(match_scores_path) as f: 42 | match_score_dict = json.load(f) 43 | with open(midi_info_path) as f: 44 | midi_info = json.load(f) 45 | 46 | with open(filepath, 'w') as f: 47 | for msd_id in match_score_dict: 48 | midi_md5s = sorted(match_score_dict[msd_id], 49 | key=match_score_dict[msd_id].get, reverse=True) 50 | for midi_md5 in midi_md5s: 51 | npz_path = os.path.join(src, midi_md5[0], midi_md5 + '.npz') 52 | if os.path.isfile(npz_path): 53 | if midi_filter(midi_info[midi_md5]): 54 | f.write("{} {}\n".format(midi_md5, msd_id)) 55 | break 56 | 57 | print("Cleansed ID list successfully saved.") 58 | 59 | if __name__ == "__main__": 60 | main() 61 | -------------------------------------------------------------------------------- /src/constants.py: -------------------------------------------------------------------------------- 1 | """This file defines the mapping between the labels and the codes.""" 2 | LABEL_NUM_MAP = { 3 | 'Big_Band': 0, 4 | 'Blues_Contemporary': 1, 5 | 'Country_Traditional': 2, 6 | 'Dance': 3, 7 | 'Electronica': 4, 8 | 'Experimental': 5, 9 | 'Folk_International': 6, 10 | 'Gospel': 7, 11 | 'Grunge_Emo': 8, 12 | 'Hip_Hop_Rap': 9, 13 | 'Jazz_Classic': 10, 14 | 'Metal_Alternative': 11, 15 | 'Metal_Death': 12, 16 | 'Metal_Heavy': 13, 17 | 'Pop_Contemporary': 14, 18 | 'Pop_Indie': 15, 19 | 'Pop_Latin': 16, 20 | 'Punk': 17, 21 | 'Reggae': 18, 22 | 'RnB_Soul': 19, 23 | 'Rock_Alternative': 20, 24 | 'Rock_College': 21, 25 | 'Rock_Contemporary': 22, 26 | 'Rock_Hard': 23, 27 | 'Rock_Neo_Psychedelia': 24 28 | } 29 | 30 | NUM_LABEL_MAP = { 31 | 0: 'Big_Band', 32 | 1: 'Blues_Contemporary', 33 | 2: 'Country_Traditional', 34 | 3: 'Dance', 35 | 4: 'Electronica', 36 | 5: 'Experimental', 37 | 6: 'Folk_International', 38 | 7: 'Gospel', 39 | 8: 'Grunge_Emo', 40 | 9: 'Hip_Hop_Rap', 41 | 10: 'Jazz_Classic', 42 | 11: 'Metal_Alternative', 43 | 12: 'Metal_Death', 44 | 13: 'Metal_Heavy', 45 | 14: 'Pop_Contemporary', 46 | 15: 'Pop_Indie', 47 | 16: 'Pop_Latin', 48 | 17: 'Punk', 49 | 18: 'Reggae', 50 | 19: 'RnB_Soul', 51 | 20: 'Rock_Alternative', 52 | 21: 'Rock_College', 53 | 22: 'Rock_Contemporary', 54 | 23: 'Rock_Hard', 55 | 24: 'Rock_Neo Psychedelia' 56 | } 57 | 58 | CLEANSED_LABELS = [2, 3, 11, 14, 15, 16, 20, 21, 22, 23] 59 | 60 | CLEANSED_NUM_LABEL_MAP = { 61 | 0: 'Country_Traditional', # 2 62 | 1: 'Dance', # 3 63 | 2: 'Metal_Alternative', # 11 64 | 3: 'Pop_Contemporary', # 14 65 | 4: 'Pop_Indie', # 15 66 | 5: 'Pop_Latin', # 16 67 | 6: 'Rock_Alternative', # 20 68 | 7: 'Rock_College', # 21 69 | 8: 'Rock_Contemporary', # 22 70 | 9: 'Rock_Hard' # 23 71 | } 72 | 73 | CLEANSED_LABEL_NUM_MAP = { 74 | 'Country_Traditional': 0, # 2 75 | 'Dance': 1, # 3 76 | 'Metal_Alternative': 2, # 11 77 | 'Pop_Contemporary': 3, # 14 78 | 'Pop_Indie': 4, # 15 79 | 'Pop_Latin': 5, # 16 80 | 'Rock_Alternative': 6, # 20 81 | 'Rock_College': 7, # 21 82 | 'Rock_Contemporary': 8, # 22 83 | 'Rock_Hard': 9 # 23 84 | } 85 | -------------------------------------------------------------------------------- /src/derive_labels_amg.py: -------------------------------------------------------------------------------- 1 | """This file derives the labels in the MSD AllMusic Style Dataset (MASD) 2 | provided in the Million Song Dataset Benchmarks (see 3 | http://www.ifs.tuwien.ac.at/mir/msd/).""" 4 | import os.path 5 | import argparse 6 | from utils import make_sure_path_exists 7 | from constants import LABEL_NUM_MAP, CLEANSED_LABELS 8 | 9 | def parse_args(): 10 | """Return the parsed command line arguments.""" 11 | parser = argparse.ArgumentParser() 12 | parser.add_argument('result_dir', help="path to save the ID lists") 13 | parser.add_argument('src', help="path to the labels") 14 | parser.add_argument('subset_ids_path', 15 | help="path to the ID list of the target subset") 16 | args = parser.parse_args() 17 | return args.result_dir, args.src, args.subset_ids_path 18 | 19 | def main(): 20 | """Main function.""" 21 | result_dir, src, subset_ids_path = parse_args() 22 | 23 | # Parse the label of each song 24 | id_label_masd = {} 25 | with open(src) as f: 26 | for line in f: 27 | if line.startswith('#'): 28 | continue 29 | id_label_masd[line.split()[0]] = LABEL_NUM_MAP[line.split()[1]] 30 | 31 | # Load the IDs of the songs in the subset 32 | with open(subset_ids_path) as f: 33 | subset_ids = [line.rstrip('\n').split()[1] for line in f] 34 | 35 | # Loop over all the songs in the subset 36 | collected = {} 37 | for msd_id in subset_ids: 38 | label = id_label_masd.get(msd_id) 39 | if label is None: 40 | continue 41 | collected[msd_id] = label 42 | 43 | # Save the ID label pairs to a file 44 | make_sure_path_exists(result_dir) 45 | filepath = os.path.join(result_dir, 'masd_labels.txt') 46 | with open(filepath, 'w') as f: 47 | f.write("# msd_id, label_num\n") 48 | for msd_id in collected: 49 | f.write("{} {}\n".format(msd_id, collected[msd_id])) 50 | print("Labels successfully saved.") 51 | 52 | # Save the cleansed ID label pairs to a file 53 | cleansed = {} 54 | for msd_id in collected: 55 | if collected[msd_id] in CLEANSED_LABELS: 56 | cleansed[msd_id] = CLEANSED_LABELS.index(collected[msd_id]) 57 | filepath = os.path.join(result_dir, 'masd_labels_cleansed.txt') 58 | with open(filepath, 'w') as f: 59 | f.write("# msd_id, label_num\n") 60 | for msd_id in cleansed: 61 | f.write("{} {}\n".format(msd_id, cleansed[msd_id])) 62 | print("Cleansed labels successfully saved.") 63 | 64 | if __name__ == "__main__": 65 | main() 66 | -------------------------------------------------------------------------------- /src/derive_id_lists_lastfm.py: -------------------------------------------------------------------------------- 1 | """This file derives the ID lists for different labels in the Last.fm Dataset 2 | (see https://labrosa.ee.columbia.edu/millionsong/lastfm).""" 3 | import json 4 | import os.path 5 | import argparse 6 | from utils import make_sure_path_exists, msd_id_to_dirs 7 | 8 | # Define the tags to be considered 9 | TAGS = [ 10 | 'sad', 'Love', 'cool', 'electro', 'happy', 'indie', '70s', 'classic-rock', 11 | 'Progressive-rock', 'Mellow', 'oldies', 'british', 'piano', 'electronica', 12 | 'sexy', 'experimental', 'techno', 'funk', 'folk', 'psychedelic', 'hardcore', 13 | 'Favorite', 'pop', '60s', 'downtempo', 'relax', 'amazing', 'blues', 'House', 14 | 'catchy', 'Favourites', '80s', '00s', 'country', 'chillout', 'chill', 15 | 'electronic', 'punk', 'Soundtrack', 'fun', 'favourite', 'lounge', 'reggae', 16 | 'favorites', 'trance', 'american', 'cover', 'alternative', 'party', 17 | 'Awesome', 'melancholy', 'acoustic', 'dance', 'Hip-Hop', 'soul', 'classic', 18 | 'metal', 'guitar', 'rap', 'beautiful', 'loved', 'jazz', 'rock', 19 | 'heard-on-Pandora', '90s', 'female', 'ambient', 'instrumental', 'rnb' 20 | ] 21 | 22 | def parse_args(): 23 | """Return the parsed command line arguments.""" 24 | parser = argparse.ArgumentParser() 25 | parser.add_argument('result_dir', help="path to save the ID lists") 26 | parser.add_argument('src', help="path to the labels") 27 | parser.add_argument('subset_ids_path', 28 | help="path to the ID list of the target subset") 29 | args = parser.parse_args() 30 | return args.result_dir, args.src, args.subset_ids_path 31 | 32 | def main(): 33 | """Main function.""" 34 | result_dir, src, subset_ids_path = parse_args() 35 | 36 | id_lists = {tag: [] for tag in TAGS} 37 | 38 | # Load the IDs of the songs in the subset 39 | with open(subset_ids_path) as f: 40 | subset_ids = [line.rstrip('\n').split()[1] for line in f] 41 | 42 | # Loop over all the songs in the subsets 43 | for msd_id in subset_ids: 44 | for dataset in ('lastfm_train', 'lastfm_test'): 45 | filepath = os.path.join( 46 | src, dataset, msd_id_to_dirs(msd_id) + '.json') 47 | if os.path.exists(filepath): 48 | with open(filepath) as f: 49 | data = json.load(f) 50 | # Loop over all the tags annotated to the song 51 | for tag_freq_pair in data['tags']: 52 | if tag_freq_pair[0] in TAGS: 53 | # Add the ID to the corresponding tag 54 | id_lists[tag_freq_pair[0]].append(msd_id) 55 | 56 | # Save the ID lists to files 57 | make_sure_path_exists(result_dir) 58 | for tag in TAGS: 59 | filename = 'id_list_{}.txt'.format(tag.lower()) 60 | with open(os.path.join(result_dir, filename), 'w') as f: 61 | for msd_id in id_lists[tag]: 62 | f.write(msd_id + '\n') 63 | 64 | print("ID lists for Last.fm Dataset successfully saved.") 65 | 66 | if __name__ == "__main__": 67 | main() 68 | -------------------------------------------------------------------------------- /src/merger_5.py: -------------------------------------------------------------------------------- 1 | """This script merges multitrack pianorolls to five-track pianorolls. 2 | """ 3 | import os.path 4 | import argparse 5 | from pypianoroll import Multitrack, Track 6 | from utils import make_sure_path_exists, change_prefix, findall_endswith 7 | from config import CONFIG 8 | if CONFIG['multicore'] > 1: 9 | import joblib 10 | 11 | TRACK_INFO = ( 12 | ('Drums', 0), 13 | ('Piano', 0), 14 | ('Guitar', 24), 15 | ('Bass', 32), 16 | ('Strings', 48), 17 | ) 18 | 19 | def parse_args(): 20 | """Return the parsed command line arguments.""" 21 | parser = argparse.ArgumentParser() 22 | parser.add_argument('src', help="root path to the source dataset") 23 | parser.add_argument('dst', help="root path to the destination dataset") 24 | args = parser.parse_args() 25 | return args.src, args.dst 26 | 27 | def get_merged(multitrack): 28 | """Merge the multitrack pianorolls into five instrument families and 29 | return the resulting multitrack pianoroll object.""" 30 | track_lists_to_merge = [[] for _ in range(5)] 31 | for idx, track in enumerate(multitrack.tracks): 32 | if track.is_drum: 33 | track_lists_to_merge[0].append(idx) 34 | elif track.program//8 == 0: 35 | track_lists_to_merge[1].append(idx) 36 | elif track.program//8 == 3: 37 | track_lists_to_merge[2].append(idx) 38 | elif track.program//8 == 4: 39 | track_lists_to_merge[3].append(idx) 40 | elif track.program < 96 or 104 <= track.program < 112: 41 | track_lists_to_merge[4].append(idx) 42 | 43 | tracks = [] 44 | for idx, track_list_to_merge in enumerate(track_lists_to_merge): 45 | if track_list_to_merge: 46 | merged = multitrack[track_list_to_merge].get_merged_pianoroll('max') 47 | tracks.append(Track(merged, TRACK_INFO[idx][1], (idx == 0), 48 | TRACK_INFO[idx][0])) 49 | else: 50 | tracks.append(Track(None, TRACK_INFO[idx][1], (idx == 0), 51 | TRACK_INFO[idx][0])) 52 | return Multitrack(None, tracks, multitrack.tempo, multitrack.downbeat, 53 | multitrack.beat_resolution, multitrack.name) 54 | 55 | def merger(filepath, src, dst): 56 | """Load and merge a multitrack pianoroll and save to the given path.""" 57 | # Load and merge the multitrack pianoroll 58 | multitrack = Multitrack(filepath) 59 | merged = get_merged(multitrack) 60 | 61 | # Save the merged multitrack pianoroll 62 | result_path = change_prefix(filepath, src, dst) 63 | make_sure_path_exists(os.path.dirname(result_path)) 64 | merged.save(result_path) 65 | 66 | def main(): 67 | """Main function.""" 68 | src, dst = parse_args() 69 | make_sure_path_exists(dst) 70 | 71 | if CONFIG['multicore'] > 1: 72 | joblib.Parallel(n_jobs=CONFIG['multicore'], verbose=5)( 73 | joblib.delayed(merger)(npz_path, src, dst) 74 | for npz_path in findall_endswith('.npz', src)) 75 | else: 76 | for npz_path in findall_endswith('.npz', src): 77 | merger(npz_path, src, dst) 78 | 79 | if __name__ == "__main__": 80 | main() 81 | -------------------------------------------------------------------------------- /src/merger_17.py: -------------------------------------------------------------------------------- 1 | """This script merges multitrack pianorolls to seventeen-track pianorolls. 2 | """ 3 | import os.path 4 | import argparse 5 | from pypianoroll import Multitrack, Track 6 | from utils import make_sure_path_exists, change_prefix, findall_endswith 7 | from config import CONFIG 8 | if CONFIG['multicore'] > 1: 9 | import joblib 10 | 11 | TRACK_INFO = ( 12 | ('Drums', 0), 13 | ('Piano', 0), 14 | ('Chromatic Percussion', 8), 15 | ('Organ', 16), 16 | ('Guitar', 24), 17 | ('Bass', 32), 18 | ('Strings', 40), 19 | ('Ensemble', 48), 20 | ('Brass', 56), 21 | ('Reed', 64), 22 | ('Pipe', 72), 23 | ('Synth Lead', 80), 24 | ('Synth Pad', 88), 25 | ('Synth Effects', 96), 26 | ('Ethnic', 104), 27 | ('Percussive', 112), 28 | ('Sound Effects', 120), 29 | ) 30 | 31 | def parse_args(): 32 | """Return the parsed command line arguments.""" 33 | parser = argparse.ArgumentParser() 34 | parser.add_argument('src', help="root path to the source dataset") 35 | parser.add_argument('dst', help="root path to the destination dataset") 36 | args = parser.parse_args() 37 | return args.src, args.dst 38 | 39 | def get_merged(multitrack): 40 | """Merge the multitrack pianorolls into sixteen instrument families and 41 | return the resulting multitrack pianoroll object.""" 42 | track_lists_to_merge = [[] for _ in range(17)] 43 | for idx, track in enumerate(multitrack.tracks): 44 | if track.is_drum: 45 | track_lists_to_merge[0].append(idx) 46 | else: 47 | track_lists_to_merge[track.program//8 + 1].append(idx) 48 | 49 | tracks = [] 50 | for idx, track_list_to_merge in enumerate(track_lists_to_merge): 51 | if track_list_to_merge: 52 | merged = multitrack[track_list_to_merge].get_merged_pianoroll('max') 53 | tracks.append(Track(merged, TRACK_INFO[idx][1], (idx == 0), 54 | TRACK_INFO[idx][0])) 55 | else: 56 | tracks.append(Track(None, TRACK_INFO[idx][1], (idx == 0), 57 | TRACK_INFO[idx][0])) 58 | return Multitrack(None, tracks, multitrack.tempo, multitrack.downbeat, 59 | multitrack.beat_resolution, multitrack.name) 60 | 61 | def merger(filepath, src, dst): 62 | """Load and merge a multitrack pianoroll and save to the given path.""" 63 | # Load and merge the multitrack pianoroll 64 | multitrack = Multitrack(filepath) 65 | merged = get_merged(multitrack) 66 | 67 | # Save the merged multitrack pianoroll 68 | result_path = change_prefix(filepath, src, dst) 69 | make_sure_path_exists(os.path.dirname(result_path)) 70 | merged.save(result_path) 71 | 72 | def main(): 73 | """Main function.""" 74 | src, dst = parse_args() 75 | make_sure_path_exists(dst) 76 | 77 | if CONFIG['multicore'] > 1: 78 | joblib.Parallel(n_jobs=CONFIG['multicore'], verbose=5)( 79 | joblib.delayed(merger)(npz_path, src, dst) 80 | for npz_path in findall_endswith('.npz', src)) 81 | else: 82 | for npz_path in findall_endswith('.npz', src): 83 | merger(npz_path, src, dst) 84 | 85 | if __name__ == "__main__": 86 | main() 87 | -------------------------------------------------------------------------------- /docs/_layouts/default.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | {% seo %} 8 | 9 | 10 | 11 | 12 | 13 | 14 |