├── .gitignore ├── README.md ├── dockerfiles ├── musictransformer.Dockerfile └── onsets-and-frames.Dockerfile ├── generated-music └── my-sample-song-1.mid ├── images ├── logic-pro.png └── pachhub_screenshot.png ├── pipelines ├── midi.json ├── mp3-to-wav.json ├── music-transformer.json └── transformer-preprocess.json ├── samples └── midi │ ├── 1.mid │ ├── 2.mid │ ├── 3.mid │ ├── 4.mid │ ├── 5.mid │ ├── 6.mid │ └── 7.mid ├── scripts ├── name-standarizer.sh └── pachyderm_ingest.py └── trained-models ├── ambient-musictransformer-model-1-may-27-2020_500-epochs ├── checkpoint ├── ckpt.data-00000-of-00002 ├── ckpt.data-00001-of-00002 ├── ckpt.index └── config.json ├── ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data ├── checkpoint ├── ckpt.data-00000-of-00002 ├── ckpt.data-00001-of-00002 ├── ckpt.index └── config.json ├── techno-musictransformer-june-1-2020_1500-epochs ├── checkpoint ├── ckpt.data-00000-of-00002 ├── ckpt.data-00001-of-00002 ├── ckpt.index └── config.json └── techno-musictransformer-model-1-may-30-2020_500-epochs ├── checkpoint ├── ckpt.data-00000-of-00002 ├── ckpt.data-00001-of-00002 ├── ckpt.index └── config.json /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules 2 | .DS_Store 3 | tmp 4 | .ipynb_checkpoints 5 | audio-processed 6 | audio-unprocessed -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Ambient Music Generator 2 | 3 | An **ambient music generating** machine learning pipeline, using [Pachyderm](https://docs.pachyderm.com/latest/getting_started/beginner_tutorial/) and the [Magenta Project Music Transformer](https://magenta.tensorflow.org/music-transformer). 4 | 5 | Pachyderm makes it supremely simple to string together a bunch of loosely coupled frameworks into a smoothly scaling AI training platform. If you can package up your program in a Docker container you can easily run it in Pachyderm. 6 | 7 | - [Generating Songs](#generating-songs) 8 | - [Requirements](#requirements) 9 | - [Playing Your Song](#playing-your-song) 10 | - [Training Your Own Music Generator in Pachyderm](#training-your-own-music-generator-in-pachyderm) 11 | - [Training Requirements](#training-requirements) 12 | - [TLDR; Just give me the code!](#tldr-just-give-me-the-code) 13 | - [Step 1: Download Your Data](#step-1-download-your-data) 14 | - [Step 2: Get Pachyderm Running](#step-2-get-pachyderm-running) 15 | - [Step 3: Clean Your Data](#step-3-clean-your-data) 16 | - [Step 4: Convert MP3s to Waves](#step-4-convert-mp3s-to-waves) 17 | - [Step 5: Converting WAVs to MIDI](#step-5-converting-wavs-to-midi) 18 | - [Step 6: MIDI to TFRecord](#step-6-midi-to-tfrecord) 19 | - [Step 7: Train the Transformer](#step-7-train-the-transformer) 20 | - [Step 8: Download the transformer model](#step-8-download-the-transformer-model) 21 | 22 | 23 | ## Generating Songs 24 | In this repo, I've included a fully trained model you can download right here and try out yourself. Additionally, I included a Docker container with the fully trained model so you can start making music fast. 25 | 26 | ### Requirements 27 | Make sure you have [Docker Desktop](https://www.docker.com/get-started) installed and running. 28 | 29 | Pull down the Ambient Music Transformer container to get started. 30 | 31 | docker pull jimmywhitaker/music-transformer:0.1 32 | 33 | Clone this Github repo and then in your terminal be sure you change to the directory where the repo lives locally. 34 | 35 | Now launch the container and generate a song (with the defaults): 36 | 37 | docker run -it -v `pwd`:/data --entrypoint python jimmywhitaker/music-transformer:0.1 generate.py --load_path=/data/trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data --inputs /data/samples/midi/1.mid --length=2048 --save_path=/data/generated-music/my-sample-song-1.mid 38 | 39 | You should see something like this: 40 | ``` 41 | 2020-07-16 01:12:09.048700: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 42 | 2020-07-16 01:12:09.063616: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3504000000 Hz 43 | 2020-07-16 01:12:09.063934: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3160000b20 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 44 | 2020-07-16 01:12:09.064158: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 45 | >> generate with decoder wise... beam size is None 46 | generating |## | 182/2048 47 | ``` 48 | 49 | What just happened? We started a docker container with all the code and files to generate a song. When the container starts it maps the present working directory (``--v `pwd`:data``) to `/data` so that we have access to our local directory. The container then generates a song with our pre-trained ambient music generation model. 50 | 51 | A seed is passed to the generator to give the model a starting point for style. We've already included 7 midi seeds in [``samples/midi``](samples/midi). You can change them by changing the `--inputs` flag to point at a different file (e.g. `--inputs /data/samples/midi/2.mid`). 52 | 53 | Music Transformer will try to predict a good sequence in the style of that original song and deliver something brand new and never before heard by gods or mankind. 54 | 55 | The model will generate your song in the sub-directory ``generated-music`` with the file name ``my-sample-song-1.mid``. 56 | 57 | If you want to generate a second song, you can run the command again but be sure to change the `--save_path` file name ``my-sample-song-1.mid`` to a different name or you will overwrite the last file! 58 | 59 | If you want to add your own MIDI seed then you can do the following. Let's say you wanted to have the Music Transformer generate a model with Cuando el Sol Grita la Mañana, by Leandro Fresco as the seed, you’d put the file into the [``samples/midi``](samples/midi) and change the run command to: 60 | 61 | docker run -it -v `pwd`:/data --entrypoint /bin/python jimmywhitaker/music-transformer:0.1 generate.py --load_path=/src/music-transformer-model --inputs /data/samples/midi/leandrofresco-cuando-el-sol-grita-la-mananga.mid --length=2048 --save_path=/data/generated-music/my-sample-song-3.mid 62 | 63 | It will take a few minutes, depending on how long of a song you asked it to create. It can’t create sequences longer than 2048. It can create shorter tunes but I’ve found the longer ones more amazing because they have a consistent structure throughout the entire song, something a 10 second sample just can’t capture. 64 | 65 | ## Playing Your Song 66 | 67 | This step is where we really bring in our human-in-the-loop creativity. 68 | 69 | We play our MIDI through various software instruments to see how it sounds. The software instruments are what bring our song to life. 70 | 71 | Different instruments create very different songs. If you play your MIDI through a drum machine or a piano it will sound like random garbage because ambient music is more irregular than a drum or piano concert. 72 | 73 | But pick the right ambient instrument and you might just have musical magic at your fingertips. 74 | 75 | Of course, you could automate this step but you’ll need to find a rich collection of open source software instruments. They’re out there, but if you have a Mac you already have a rich collection of software instruments in [Apple’s Logic Pro](https://www.apple.com/logic-pro/). That felt like the best place to start so I could try lots of iterations fast. If you don’t own Logic Pro you can install a 60 day trial version from [the Mac Store](https://apps.apple.com/us/app/logic-pro-x/id634148309?mt=12) that is fully featured and not crippleware. 76 | 77 | If you don’t want to use Logic Pro, there’s lots of amazing music creation software to choose from, like Abelton Live and Cuebase. You can also use Apple's free [Garageband](https://www.apple.com/mac/garageband/) by installing it from the app store. If you’re a musical magician then go wild and unleash your favorite software collection on those AI generated songs. 78 | 79 | But if you’re using Logic Pro like me, then you just import the MIDI and change out the instrument in the software track. 80 | 81 | ![Logic Pro screenshot](images/logic-pro.png) 82 | 83 | After days of experimentation I found a few amazing software instruments that consistently delivered fantastic songs when the Music Transformer managed to spit out a great sample. 84 | 85 | - Stratosphere 86 | - Deterioration 87 | - Peaceful Meadow 88 | - Calm and Storm 89 | 90 | Here are some of my favorite samples: 91 | 92 | - https://soundcloud.com/laughing-monkey-962936176/ambient-ai-song-seed-aeoliah-angels-of-presence-instrument-peaceful-meadow/s-mrmfusrM76x 93 | 94 | - https://soundcloud.com/laughing-monkey-962936176/ambient-ai-song-seed-deuter-mystic-voyage-instrument-peaceful-meadow/s-RKK82x0t5Nm 95 | 96 | - https://soundcloud.com/laughing-monkey-962936176/ambient-ai-song-seed-biosphere-instrument-voyager/s-Ax1cDDlLQiW 97 | 98 | Several software instruments give the very same MIDI a strong, sci-fi vibe that feels otherworldly, as if I was flying through space or dropped down into a 1980s sci-fi [synthwave](https://open.spotify.com/playlist/6FgP4Vq2tctL3p1cNrgocs?si=uu71bBbSS3G3ozPeecpPzQ) blockbuster: 99 | 100 | - Underwater Temple 101 | - Parallel Universe 102 | 103 | Here are a few great sci-fi examples: 104 | 105 | - https://soundcloud.com/laughing-monkey-962936176/ambient-ai-song-seed-biosphere-instrument-parallel-universe/s-aWh5GkKQWWi 106 | 107 | - https://soundcloud.com/laughing-monkey-962936176/ambient-ai-song-seed-love-on-a-train-instrument-underwater-temple/s-NxCgMvHFsoV 108 | 109 | Music Transformer does a terrific job depending on what you feed it. Some seeds really make the Music Transformer sing. 110 | 111 | But not all seeds are created equal. It really struggles sometimes. 112 | 113 | No software instrument can save a terrible output no matter how hard you try. 114 | 115 | Choose your seed wisely! 116 | 117 | Happy music making! 118 | 119 | ## Training Your Own Music Generator in Pachyderm 120 | 121 | In this section, I walk through how to recreate the entire pipeline and train the model yourself. 122 | 123 | ![Pachyderm screenshot](images/pachhub_screenshot.png) 124 | 125 | I used Pachyderm to handle the full pipeline for scaling and management. Pachyderm is really unique because it will automatically run the rest of the pipeline and retrain your model when you add (or remove) data. This is super handy when you're still in the process of selecting your training data or if you want to try different styles of music without having to re-run scripts - just upload the data and everything is done. 126 | 127 | ### Training Requirements 128 | You can deploy a cluster on [PacHub](hub.pachyderm.com) which we did or deploy locally as described here: 129 | 130 | - [Pachyderm Getting Started](https://docs.pachyderm.com/latest/getting_started/) 131 | 132 | Once everything is up, we can check the setup by running: 133 | 1. `kubectl get all` to ensure all the pods are up. 134 | 2. `pachctl version` which will show both the `pachctl` and `pachd` versions. 135 | 136 | We’ve included all the containers and JSON you’ll need to build a complete end-to-end ML pipeline right here. There’s no need to spend time compiling your own containers. We did the hard work for you! 137 | 138 | Let's take a look at what it took to train this Transforming monster. 139 | 140 | Our ML pipeline had eight stages. 141 | 1. [Download Your Data](#step-1-download-your-data) 142 | 2. [Get Pachyderm Running](#step-2-get-pachyderm-running) 143 | 3. [Clean Your Data](#step-3-clean-your-data) 144 | 4. [Convert MP3s to Waves](#step-4-convert-mp3s-to-waves) 145 | 5. [Converting WAVs to MIDI](#step-5-converting-wavs-to-midi) 146 | 6. [MIDI to TFRecord](#step-6-midi-to-tfrecord) 147 | 7. [Train the Transformer](#step-7-train-the-transformer) 148 | 8. [Download the transformer model](#step-8-download-the-transformer-model) 149 | 150 | The first step was manual but you can automate it pretty simply. But since we you only have to download the dataset once there didn’t seem to be any reason to script that step. 151 | 152 | ### TLDR; Just give me the code! 153 | 154 | ```bash 155 | # Once Pachyderm is set up, run the following: 156 | pachctl create repo audio-unprocessed 157 | 158 | ### Clean Your Data 159 | mkdir audio-unprocessed # move downloaded mp3 files here 160 | mkdir audio-processed 161 | ./scripts/name-standarizer.sh # standardizes and uploads your data to pachyderm 162 | 163 | # Convert MP3s to Waves 164 | cd pipelines/ 165 | pachctl create pipeline -f ./mp3-to-wav.json 166 | 167 | # Converting WAVs to MIDI 168 | pachctl create pipeline -f ./midi.json 169 | 170 | # MIDI to TFRecord 171 | pachctl create pipeline -f ./transformer-preprocess.json 172 | 173 | # Train the Transformer 174 | pachctl create pipeline -f ./music-transformer.json 175 | ``` 176 | 177 | ### Step 1: Download Your Data 178 | 179 | I used my [Ambient Music for Creativity and Calm](https://open.spotify.com/playlist/6qaujvXpcysfuyFMtp7Ljn?si=p4Vme8YAS9mOlRNNM_xIsA) music collection on Spotify as my dataset. As I wrote earlier, I’ve carefully crafted and curated this list over many years so it has no false notes. By that I mean there are no surprise songs that sound radically different from the rest. You won’t suddenly find a dark techno song mixed in there or a dance beat. 180 | 181 | The big advantage of this playlist is that all the songs are very, very similar in terms of style, which means the patterns are very consistent throughout. Because of that I can use less data. A random mix of ambient music pulled from the net would take a lot more data and not have the same inner consistency. 182 | 183 | I also built a second playlist called [Ambient AI Training](https://open.spotify.com/playlist/16PvqqR02AL9zsNfPGeMLI?si=JWZ-q4qGTImpVu-eHRCnQg) that's less well curated but where I still took the time to listen to each song to make sure it matched the kinds of songs I love. 184 | 185 | You can find lots of free and paid programs out there to download your songs to MP3s. I used [AllToMP3](https://alltomp3.org/). 186 | 187 | ### Step 2: Get Pachyderm Running 188 | 189 | After I had my songs I pushed them up to Pachyderm. Pachyderm is simple and straightforward to set up. You can [install it locally](https://docs.pachyderm.com/latest/getting_started/local_installation/) or run it in the cloud. 190 | 191 | I ran Pachyderm on [PachHub](https://hub.pachyderm.com/orgs/109/workspace) which lives in the Google Cloud. It spins you up a cluster in seconds and gives you a one time login. You’ll eventually need a GPU node if you want to train the model in a few days versus a few months, but you don’t need one to start building your pipeline. When you’re ready you can launch a GPU node through the PachHub interface. It launches a graphics unit and installs the proper driver for you. Now you’re ready to schedule a job on a GPU node. 192 | 193 | If you run Pachyderm in the cloud you’ll still need a local instance of Pachyderm to talk with the cloud instance. That’s super simple. Just follow the [local installation instructions](https://docs.pachyderm.com/latest/getting_started/local_installation/) to get it running in Docker Desktop or Minikube in no time. 194 | 195 | Once you have Pachyderm up and running, you’ll want to create your ingest repo where all your data will live and where the automated pipeline can start doing its magic. You can do that with the following command: 196 | 197 | pachctl create repo audio-unprocessed 198 | 199 | That creates a starting repo called audio-unprocessed. 200 | 201 | ### Step 3: Clean Your Data 202 | 203 | I then used a little [Bash script](scripts/name-standarizer.sh) to rename all my MP3s, making them lowercase and removing all spaces to give me a standard naming convention. The script then pushes them up to the Pachyderm audio-unprocessed repo that gets it all started. 204 | 205 | ```bash 206 | mkdir audio-unprocessed # move mp3 files here 207 | mkdir audio-processed 208 | 209 | ./scripts/name-standarizer.sh 210 | ``` 211 | 212 | ### Step 4: Convert MP3s to Waves 213 | 214 | You create a pipeline with a JSON file. That file simply contains the following info for Pachy: 215 | 216 | - an input repo (where the system fetches data) 217 | - an output repo (where 218 | it puts transformed data) 219 | - a container with some code in it 220 | - the command and switches to run that code 221 | 222 | Here’s the JSON for stage three: 223 | 224 | # mp3-to-wav.json 225 | { 226 | "pipeline": { 227 | "name": "audio-processed-wav" 228 | }, 229 | "description": "A pipeline that converts MP3s to WAVs with ffmpeg.", 230 | "transform": { 231 | "cmd": [ "bash", "/ffmpeg-convert-pachy.sh" ], 232 | "image": "rabbit37/audio-preprocessor:v5" 233 | }, 234 | "input": { 235 | "pfs": { 236 | "repo": "audio-unprocessed", 237 | "glob": "/*" 238 | } 239 | } 240 | } 241 | 242 | My container runs Ubuntu and there’s a bash script in it that calls ffmpeg to convert my MP3s to WAVs. You can [grab that JSON file here](pipelines/mp3-to-wav.json) and then push it up to the cloud with the following command: 243 | 244 | ```bash 245 | cd pipelines 246 | pachctl create pipeline -f ./mp3-to-wav.json 247 | ``` 248 | 249 | That will fire up your new pipeline, chew through all your MP3s and turn them into WAVs before putting them in the audio-processed repo. 250 | 251 | ### Step 5: Converting WAVs to MIDI 252 | 253 | The third step uses the Magenta Project’s Onsets and Frames Python code to transform our WAVs into MIDI files. Onsets and Frames uses a fully trained ML model to transcribe the music. It’s not perfect but no MIDI transcription software is perfect. 254 | 255 | If you want to take this project farther then consider exploring different music transcription packages out there to get the best results. Better transcription will almost certainly create a better and more robots model. But for now this works great. 256 | 257 | [Grab the JSON file here](pipelines/midi.json) and then push it up to the Pachy cloud with the following command: 258 | 259 | ```bash 260 | pachctl create pipeline -f ./midi.json 261 | ``` 262 | 263 | The container installs and sets up Magenta's [Onsets & Frames](https://github.com/magenta/magenta/tree/master/magenta/models/onsets_frames_transcription). It relies on [this repo](https://github.com/the-laughing-monkey/onsets-and-frames-transcription) to orchestrate the reading of files and transcription. 264 | 265 | **Container:** [`hitheory/onsets-and-frames:v1.0`](https://hub.docker.com/repository/docker/hitheory/musictransformer) 266 | 267 | **Dockerfile:** [`dockerfiles/onsets-and-frames.Dockerfile`](dockerfiles/onsets-and-frames.Dockerfile) 268 | 269 | **Pipeline JSON:** [`pipelines/midi.json`](pipelines/midi.json) 270 | 271 | **Entry script:** [`/src/transcribe.py`](https://github.com/the-laughing-monkey/onsets-and-frames-transcription/blob/master/transcribe.py) 272 | 273 | **Arguments:** 274 | 275 | - `--input` - the directory containing the input files 276 | - `--output` - the directory in which to write the MIDI files 277 | 278 | Example: 279 | 280 | ```bash 281 | python3 /src/transcribe.py --input /pfs/audio-processed-wav --output /pfs/out 282 | 283 | ``` 284 | 285 | ### Step 6: MIDI to TFRecord 286 | 287 | Before training the model, we need to do one more step. We’ve got to convert those MIDI files into a format TensorFlow can understand, called the TFRecord format. The [JSON is here](pipelines/transformer-preprocess.json), and we push it with the following command: 288 | 289 | pachctl create pipeline -f ./transformer-preprocess.json 290 | 291 | We rely on [a fork of an implementation](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0) of MusicTransformer by [@jason9693](https://github.com/jason9693). 292 | 293 | **Container:** [`hitheory/musictranformer:v1.0.0`](https://hub.docker.com/repository/docker/hitheory/musictransformer) 294 | 295 | **Dockerfile:** [`dockerfiles/musictransformer.Dockerfile`](dockerfiles/musictransformer.Dockerfile) 296 | 297 | **Pipeline JSON:** [`pipelines/transformer-preprocess.json`](pipelines/transformer-preprocess.json) 298 | 299 | **Entry script:** [`/src/preprocess.py`](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0/blob/master/preprocess.py) 300 | 301 | **Arguments:** 302 | 303 | - `` - the directory containing the input files to pre-process 304 | - `` - the directory in which to write the pre-processed files 305 | 306 | Example: 307 | 308 | ``` 309 | python3 /src/preprocess.py /pfs/midi /pfs/out 310 | 311 | ``` 312 | 313 | ### Step 7: Train the Transformer 314 | 315 | Now we can finally unleash the power of GPUs to train our model. We used one monster Nvidia GPU, on a single node, and it took about three days running 24x7 to train, but you could use more GPUs and let [Pachyderm scale up the training for you](https://docs.pachyderm.com/latest/concepts/advanced-concepts/distributed_computing/) to train it a lot faster. 316 | 317 | The [training JSON file is right here](pipelines/music-transformer.json) and you push it like so: 318 | 319 | pachctl create pipeline -f ./music-transformer.json 320 | 321 | We rely on [the same fork of an implementation](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0) of MusicTransformer by [@jason9693](https://github.com/jason9693), and use the same container and Dockerfile. 322 | 323 | **Container:** [`hitheory/musictranformer:v1.0.0`](https://hub.docker.com/repository/docker/hitheory/musictransformer) 324 | 325 | **Dockerfile:** [`dockerfiles/musictransformer.Dockerfile`](dockerfiles/musictransformer.Dockerfile) 326 | 327 | **Pipeline JSON:** [`pipelines/musictransformer.json`](pipelines/musictransformer.json) 328 | 329 | **Entry script:** [`/src/preprocess.py`](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0/blob/master/train.py) 330 | 331 | **Arguments:** 332 | 333 | - `--l_r` - The learning rate to use. If `None`, [a custom learning rate is used, as defined in the original repo](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0#hyper-parameter). 334 | - `--batch_size` - The batch size to use 335 | - `--max_seq` - The sequence length to use ([more information in the paper](https://arxiv.org/pdf/1809.04281.pdf)). 336 | - `--epochs` - The number of epochs to use 337 | - `--input_path` - The directory containing the files to use for training 338 | - `--save_path` - The directory in which to write the trained model 339 | - `--num_layers` - [The number of layers to use](https://github.com/the-laughing-monkey/MusicTransformer-tensorflow2.0/blob/master/model.py#L15). 340 | 341 | Example: 342 | 343 | ``` 344 | python3 /src/train.py --epochs 500 --save_path /pfs/out --input_path /pfs/transformer-preprecess --batch_size 2 --max_seq 2048 345 | 346 | ``` 347 | 348 | ### Step 8: Download the transformer model 349 | Once the model is finished training, we can download it from pachyderm by running the following: 350 | ```bash 351 | pachctl list file music-transformer@master 352 | pachctl get file --recursive music-transformer@master 353 | ``` 354 | -------------------------------------------------------------------------------- /dockerfiles/musictransformer.Dockerfile: -------------------------------------------------------------------------------- 1 | FROM tensorflow/tensorflow:2.2.0-gpu 2 | ARG DEBIAN_FRONTEND=noninteractive 3 | ENV LANG C.UTF-8 4 | 5 | ######### Set up system with Python ######### 6 | RUN apt-get update && apt-get install -y \ 7 | libgirepository1.0-dev gcc libcairo2-dev \ 8 | libjpeg-dev libgif-dev libpango1.0-dev libssl-dev \ 9 | pkg-config gir1.2-gtk-3.0 software-properties-common \ 10 | git curl wget 11 | 12 | WORKDIR /src 13 | 14 | RUN git clone https://github.com/thekevinscott/MusicTransformer-tensorflow2.0.git /src 15 | RUN git clone https://github.com/jason9693/midi-neural-processor.git /src/midi_processor 16 | RUN pip3 install -r /src/requirements.txt --ignore-installed 17 | -------------------------------------------------------------------------------- /dockerfiles/onsets-and-frames.Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:18.04 2 | RUN apt-get clean \ 3 | && apt-get update \ 4 | && apt-get install -y \ 5 | python3-pip python3-dev wget zip git \ 6 | && cd /usr/local/bin \ 7 | && ln -s /usr/bin/python3 python \ 8 | && pip3 install --upgrade pip 9 | 10 | RUN pip3 install tensorflow==1.15.2 11 | RUN apt-get update -qq && apt-get install -qq libfluidsynth1 \ 12 | fluid-soundfont-gm build-essential libasound2-dev libjack-dev ffmpeg sox 13 | RUN pip3 install pyfluidsynth pretty_midi 14 | RUN pip3 install -qU magenta==1.3.1 15 | RUN apt-get autoremove -y && apt-get install --fix-broken 16 | RUN pip3 install librosa 17 | 18 | RUN mkdir -p /data 19 | RUN wget https://storage.googleapis.com/magentadata/models/onsets_frames_transcription/maestro_checkpoint.zip 20 | RUN unzip maestro_checkpoint.zip -d /data/maestro 21 | RUN rm maestro_checkpoint.zip 22 | 23 | ADD https://api.github.com/repos/thekevinscott/onsets-and-frames-transcription/git/refs/heads/master version.json 24 | RUN git clone https://github.com/thekevinscott/onsets-and-frames-transcription.git /src 25 | RUN rm version.json 26 | -------------------------------------------------------------------------------- /generated-music/my-sample-song-1.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/generated-music/my-sample-song-1.mid -------------------------------------------------------------------------------- /images/logic-pro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/images/logic-pro.png -------------------------------------------------------------------------------- /images/pachhub_screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/images/pachhub_screenshot.png -------------------------------------------------------------------------------- /pipelines/midi.json: -------------------------------------------------------------------------------- 1 | { 2 | "pipeline": { 3 | "name": "midi" 4 | }, 5 | "description": "A pipeline that performs midi transcription using Magenta's Onsets and Frames", 6 | "transform": { 7 | "cmd": [ 8 | "python3", 9 | "/src/transcribe.py", 10 | "--input", 11 | "/pfs/audio-processed-wav", 12 | "--output", 13 | "/pfs/out" 14 | ], 15 | "image": "hitheory/onsets-and-frames:v1.0" 16 | }, 17 | "input": { 18 | "pfs": { 19 | "repo": "audio-processed-wav", 20 | "glob": "/*" 21 | } 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /pipelines/mp3-to-wav.json: -------------------------------------------------------------------------------- 1 | { 2 | "pipeline": { 3 | "name": "audio-processed-wav" 4 | }, 5 | "description": "A pipeline that converts MP3s to WAVs with ffmpeg.", 6 | "transform": { 7 | "cmd": [ "bash", "/ffmpeg-convert-pachy.sh" ], 8 | "image": "rabbit37/audio-preprocessor:v5" 9 | }, 10 | "input": { 11 | "pfs": { 12 | "repo": "audio-unprocessed", 13 | "glob": "/*" 14 | } 15 | } 16 | } -------------------------------------------------------------------------------- /pipelines/music-transformer.json: -------------------------------------------------------------------------------- 1 | { 2 | "pipeline": { 3 | "name": "music-transformer" 4 | }, 5 | "transform": { 6 | "image": "hitheory/musictransformer:v1.0.0", 7 | "cmd": [ 8 | "python3", 9 | "/src/train.py", 10 | "--epochs", 11 | "500", 12 | "--save_path", 13 | "/pfs/out", 14 | "--input_path", 15 | "/pfs/transformer-preprocess", 16 | "--batch_size", 17 | "2", 18 | "--max_seq", 19 | "2048" 20 | ] 21 | }, 22 | "resource_limits": { 23 | "memory": "14336M", 24 | "gpu": { 25 | "type": "nvidia.com/gpu", 26 | "number": 1 27 | } 28 | }, 29 | "input": { 30 | "pfs": { 31 | "repo": "transformer-preprocess", 32 | "glob": "/" 33 | } 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /pipelines/transformer-preprocess.json: -------------------------------------------------------------------------------- 1 | { 2 | "pipeline": { 3 | "name": "transformer-preprocess" 4 | }, 5 | "transform": { 6 | "image": "hitheory/musictransformer:v1.0.0", 7 | "cmd": [ 8 | "python3", 9 | "/src/preprocess.py", 10 | "/pfs/midi", 11 | "/pfs/out" 12 | ] 13 | }, 14 | "input": { 15 | "pfs": { 16 | "repo": "midi", 17 | "glob": "/" 18 | } 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /samples/midi/1.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/1.mid -------------------------------------------------------------------------------- /samples/midi/2.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/2.mid -------------------------------------------------------------------------------- /samples/midi/3.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/3.mid -------------------------------------------------------------------------------- /samples/midi/4.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/4.mid -------------------------------------------------------------------------------- /samples/midi/5.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/5.mid -------------------------------------------------------------------------------- /samples/midi/6.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/6.mid -------------------------------------------------------------------------------- /samples/midi/7.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/samples/midi/7.mid -------------------------------------------------------------------------------- /scripts/name-standarizer.sh: -------------------------------------------------------------------------------- 1 | #/bin/bash 2 | 3 | # file name standarization 4 | # remove spaces and change upper case to lower case 5 | # move them to a new directory 6 | # push them to pachyderm 7 | 8 | INPUT_DIR="./audio-unprocessed" 9 | OUTPUT_DIR="./audio-processed" 10 | 11 | renamefiles () { 12 | for oldname in "$INPUT_DIR"/*.mp3; do 13 | newname=`echo $oldname | sed 's/ //g' | tr '[:upper:]' '[:lower:]'` 14 | mv "$oldname" "$newname" 15 | done 16 | } 17 | 18 | movefiles () { 19 | moveme=`mv $INPUT_DIR/*.mp3 $OUTPUT_DIR/` 20 | echo "$moveme" 21 | } 22 | 23 | pachyput () { 24 | cd $OUTPUT_DIR 25 | for file in ./*.mp3; do 26 | newfilename=`basename $file` 27 | pachypush=`pachctl put file audio-unprocessed@master:$newfilename -f $file` 28 | echo "$pachypush" 29 | done 30 | } 31 | 32 | renamefiles 33 | movefiles 34 | pachyput -------------------------------------------------------------------------------- /scripts/pachyderm_ingest.py: -------------------------------------------------------------------------------- 1 | #TODO create python-pachyderm client example for ingestion -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/checkpoint: -------------------------------------------------------------------------------- 1 | model_checkpoint_path: "ckpt" 2 | all_model_checkpoint_paths: "ckpt" 3 | -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.data-00000-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.data-00000-of-00002 -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.data-00001-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.data-00001-of-00002 -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/ckpt.index -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-1-may-27-2020_500-epochs/config.json: -------------------------------------------------------------------------------- 1 | {"debug": false, "max_seq": 2048, "num_layer": 6, "embedding_dim": 256, "vocab_size": 391, "dist": false} -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/checkpoint: -------------------------------------------------------------------------------- 1 | model_checkpoint_path: "ckpt" 2 | all_model_checkpoint_paths: "ckpt" 3 | -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.data-00000-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.data-00000-of-00002 -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.data-00001-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.data-00001-of-00002 -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/ckpt.index -------------------------------------------------------------------------------- /trained-models/ambient-musictransformer-model-2-june-3-2020-750-epochs-more-data/config.json: -------------------------------------------------------------------------------- 1 | {"debug": false, "max_seq": 2048, "num_layer": 6, "embedding_dim": 256, "vocab_size": 391, "dist": false} -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-june-1-2020_1500-epochs/checkpoint: -------------------------------------------------------------------------------- 1 | model_checkpoint_path: "ckpt" 2 | all_model_checkpoint_paths: "ckpt" 3 | -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.data-00000-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.data-00000-of-00002 -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.data-00001-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.data-00001-of-00002 -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-june-1-2020_1500-epochs/ckpt.index -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-june-1-2020_1500-epochs/config.json: -------------------------------------------------------------------------------- 1 | {"debug": false, "max_seq": 2048, "num_layer": 6, "embedding_dim": 256, "vocab_size": 391, "dist": false} -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/checkpoint: -------------------------------------------------------------------------------- 1 | model_checkpoint_path: "ckpt" 2 | all_model_checkpoint_paths: "ckpt" 3 | -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.data-00000-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.data-00000-of-00002 -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.data-00001-of-00002: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.data-00001-of-00002 -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pachyderm/ambient-music-generation/3487bcde6a6eb0543bd2ada419570d20678419b2/trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/ckpt.index -------------------------------------------------------------------------------- /trained-models/techno-musictransformer-model-1-may-30-2020_500-epochs/config.json: -------------------------------------------------------------------------------- 1 | {"debug": false, "max_seq": 2048, "num_layer": 6, "embedding_dim": 256, "vocab_size": 391, "dist": false} --------------------------------------------------------------------------------