├── LICENSE ├── README.md ├── config.py ├── inference.py ├── output_tunes └── Mon_01_Jul_2024_21_59_21.abc ├── prompt.txt ├── random_model.py ├── requirements.txt ├── train.py ├── utils.py └── xml2abc ├── batch_xml2abc.py └── xml2abc.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Sander Wood 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing 2 | 3 | This repository contains the code for the MelodyT5 model as described in the paper [MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing](https://arxiv.org/abs/2407.02277). 4 | 5 | ## Model Description 6 | 7 | MelodyT5 is an unified framework for symbolic music processing, using an encoder-decoder architecture to handle multiple melody-centric tasks, such as generation, harmonization, and segmentation, by treating them as score-to-score transformations. Pre-trained on [MelodyHub](https://huggingface.co/datasets/sander-wood/melodyhub), a large dataset of melodies in ABC notation, it demonstrates the effectiveness of multi-task transfer learning in symbolic music processing. 8 | 9 | We provide the weights of MelodyT5 on [Hugging Face](https://huggingface.co/sander-wood/melodyt5/blob/main/weights.pth), which are based on pre-training with over one million task instances encompassing seven melody-centric tasks. 10 | 11 | ## ABC Notation 12 | 13 | ABC notation is an ASCII-based plain text musical notation system that is commonly used for transcribing traditional music and sharing sheet music online. It provides a simple and concise way to represent musical elements such as notes, rhythms, chords, and more. 14 | 15 | For those looking to interact with ABC notation in various ways, there are several tools available: 16 | 17 | 1. **[Online ABC Player](https://abc.rectanglered.com/):** This web-based tool allows you to input ABC notation and hear the corresponding audio playback. By pasting your ABC code into the player, you can instantly listen to the tune as it would sound when played. 18 | 19 | 2. **[ABC Sheet Music Editor - EasyABC](https://easyabc.sourceforge.net/):** EasyABC is a user-friendly software application designed for creating, editing, and formatting ABC notation. Its graphical interface enables you to input your ABC code, preview the sheet music, and make adjustments as necessary. 20 | 21 | To learn more about ABC notaton, please see [ABC Examples](https://abcnotation.com/examples) and [ABC Strandard](https://abcnotation.com/wiki/abc:standard). 22 | 23 | ## Installation 24 | 25 | To set up the MelodyT5 environment and install the necessary dependencies, follow these steps: 26 | 27 | 1. **Create and Activate Conda Environment** 28 | 29 | ```bash 30 | conda create --name melodyt5 python=3.7.9 31 | conda activate melodyt5 32 | ``` 33 | 34 | 2. **Install Dependencies** 35 | 36 | ```bash 37 | pip install -r requirements.txt 38 | ``` 39 | 40 | 3. **Install Pytorch** 41 | 42 | ```bash 43 | pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116 44 | ``` 45 | 46 | 4. **Create Random Weights (Optional)** 47 | 48 | If `SHARE_WEIGHTS = True` in `config.py`, run `random_model.py` to generate random weights for initializing the patch-level encoder and decoder with shared weights. 49 | 50 | 5. **Download Pre-trained MelodyT5 Weights (Optional)** 51 | 52 | For those interested in starting with pre-trained models, MelodyT5 weights are available on [Hugging Face](https://huggingface.co/sander-wood/melodyt5/blob/main/weights.pth). 53 | 54 | ## Usage 55 | 56 | - `config.py`: Configuration settings for training and inference. 57 | - `inference.py`: Perform inference tasks (e.g., generation and harmonization) using trained models. 58 | - `prompt.txt`: Text file containing the prompt to specify a task and its input for the model. 59 | - `random_model.py`: Use this script to initialize the model with random weights when `SHARE_WEIGHTS = TRUE` in `config.py` (default setting). 60 | - `train.py`: Script for training the MelodyT5 model. 61 | - `utils.py`: Utility functions supporting model operations and data processing. 62 | 63 | ### Setting Up Inference Parameters 64 | 65 | Before running the inference script, you can configure the following parameters in `config.py` or directly via command-line arguments: 66 | 67 | - `-num_tunes`: Number of independently computed returned tunes (default: 3) 68 | - `-max_patch`: Maximum length in patches of each tune (default: 128) 69 | - `-top_p`: Cumulative probability threshold for nucleus sampling (default: 0.8) 70 | - `-top_k`: Number of highest probability tokens to keep for sampling (default: 8) 71 | - `-temperature`: Temperature of the sampling operation (default: 2.6) 72 | - `-seed`: Seed for random state (default: None) 73 | - `-show_control_code`: Whether to show control codes (default: True) 74 | 75 | These parameters control how the model generates melodies based on the input provided in `prompt.txt`. 76 | 77 | ### Running Inference 78 | 79 | To perform inference tasks using MelodyT5, follow these steps: 80 | 81 | 1. **Prepare Your Prompt** 82 | - Edit `prompt.txt` to specify the task and input for the model. 83 | 84 | 2. **Execute Inference** 85 | - Run the following command to execute the inference script: 86 | ```bash 87 | python inference.py -num_tunes 3 -max_patch 128 -top_p 0.8 -top_k 8 -temperature 2.6 -seed -show_control_code True 88 | ``` 89 | Replace `` with your chosen seed value or leave it as `None` for a random seed. 90 | - The script will generate melodies based on the prompt specified in `prompt.txt` using the configured parameters and save the results in the `output_tunes` folder. 91 | 92 | ## How to Use 93 | 94 | Follow these steps to utilize MelodyT5 for symbolic music processing: 95 | 96 | 1. **Prepare XML Data** 97 | 98 | Ensure your data is in single-track XML format, which can be `.xml`, `.musicxml`, or `.mxl` files. 99 | 100 | 2. **Convert to Standard Format** 101 | 102 | - Place the folder containing your XML data in the `xml2abc` directory. 103 | - Navigate to the `xml2abc` directory. 104 | - Run the following command to convert your XML files into standard ABC notation format: 105 | 106 | ```bash 107 | python batch_xml2abc.py path_to_xml_directory 108 | ``` 109 | 110 | - Replace `path_to_xml_directory` with the directory containing your XML files. 111 | 112 | 3. **Design Input-Output Pairs** 113 | 114 | After conversion, design input-output pairs based on your specific task requirements. You can refer to the data formats used in [MelodyHub](https://huggingface.co/datasets/sander-wood/melodyhub) for guidance. 115 | 116 | 4. **Configure Your Model** 117 | 118 | Adjust model hyperparameters, training parameters, and file paths in the `config.py` file as you see fit. 119 | 120 | 5. **Train the Model** 121 | 122 | Run the `train.py` script to train MelodyT5. Use the following command, adjusting for your specific setup: 123 | 124 | ``` 125 | python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py 126 | ``` 127 | This command utilizes distributed training across multiple GPUs (modify `--nproc_per_node` as needed). 128 | 129 | 6. **Run Inference** 130 | 131 | To perform inference tasks such as melody generation or harmonization, execute `inference.py`. The script reads prompts from `prompt.txt` to specify the task and input for the model. Customize prompts in `prompt.txt` to define different tasks and inputs for MelodyT5. Refer to the examples below for guidance on setting up prompts. 132 | 133 | ``` 134 | %%input 135 | %%variation 136 | L:1/8 137 | M:6/8 138 | K:D 139 | |: AFD DFA | Add B2 A | ABA F3 | GFG EFG | AFD DFA | Add B2 A | ABA F2 E | FDD D3 :: fdd ede | 140 | fdd d2 g | fdd def | gfg e2 g | fed B2 A | AdF A3 | ABA F2 E | FDD D3 :| 141 | %%output 142 | E:8 143 | L:1/8 144 | M:6/8 145 | K:D 146 | |: B | 147 | ``` 148 | 149 | Ensure the input (encoder input) is complete, while the output (decoder input) is optional. If you need the model to continue a given output, use `%%input` and `%%output` to mark the beginning of each section. Additionally, the output must not contain incomplete bars. 150 | 151 | ## Inference Examples 152 | Below are the MelodyT5 results on seven MelodyHub tasks, using random samples from the validation set. Three independent outputs were generated without cherry-picking. Each `X:0` output corresponds to the original input for that task and is not generated by the model, while `X:1`, `X:2`, and `X:3` are generated outputs. 153 | 154 | To view the musical scores and listen to the tunes, you can use online ABC notation platforms such as [Online ABC Player](https://abc.rectanglered.com/) or the [ABC Sheet Music Editor - EasyABC](https://easyabc.sourceforge.net/). Simply copy and paste the ABC notation into these tools to see the sheet music and hear the audio playback. 155 | 156 | 1. Cataloging 157 | ``` 158 | %%input 159 | %%cataloging 160 | T: 161 | O: 162 | Y: 163 | L:1/4 164 | Q:1/4=120 165 | M:3/4 166 | K:C 167 | A/0 A/0 B/0 (c/0 B/0) A/0 A/0 ^G/0 A/0 (e/0 f/0) (=g/0 f/0) e/0 | (f e) d | e2 e | (d c) B | 168 | (c B) A | ^G2 e | f2 e | d2 c | B2 A | ^G2 c | (d e) d | (e f) g | e2 e | (e c) c | (d B) B | 169 | (c A) A | ^G2 (A/B/) | (c d) c | c2 B | c2 (c/d/) | (e f) e | e2 d | e2 (e/f/) | g2 g | (g e) e | 170 | f2 e | d2 e | (e d) e | (c d) e | (d e) (c/d/) | B2 c | B2 A | ^G2 A | A2 c | B2 A | ^G2 A | 171 | A2 x |] 172 | ``` 173 | ``` 174 | %%output 175 | X:0 176 | T:HET GODEN BANKET. Ter Huwfeest van Muzika en Poezy 177 | O:Netherlands 178 | Y:vocal 179 | 180 | X:1 181 | T:WILHELLTOF GERSCHANDS 182 | O:Netherlands 183 | Y:vocal 184 | 185 | 186 | X:2 187 | T:MIEN IS LOVEN, VRIENDAAT OP EEN 188 | O:Netherlands 189 | Y:vocal 190 | 191 | 192 | X:3 193 | T:Het lied en tijd over 194 | O:Netherlands 195 | Y:vocal 196 | ``` 197 | 2. Generation 198 | ``` 199 | %%input 200 | %%generation 201 | ``` 202 | ``` 203 | %%output 204 | X:0 205 | S:2 206 | B:8 207 | E:5 208 | B:8 209 | L:1/8 210 | M:2/2 211 | K:G 212 | |: A3 c BG G2 | BGBd g2 fg | eA A2 BGBd | egdc BG G2 | A3 c BG G2 | BGBd g3 a | bgag egfa | 213 | gedc BG G2 :: bg g2 agef | g3 e dega | bg g2 aged | eaag a2 ga | bg g2 agef | g3 e dega | 214 | bgag egfa | gedc BG G2 :| 215 | 216 | X:1 217 | S:2 218 | B:5 219 | E:5 220 | B:9 221 | L:1/8 222 | M:2/2 223 | K:Amin 224 | cd | e3 c d2 cd | e3 d c2 Ac | B3 c A3 A | A4 :: G2 | c3 c c3 c | d3 d e2 fe | f2 f2 e2 d2 | 225 | g3 a g2 cd | e3 d c2 dc | e3 d c2 Ac | B3 c A3 A | A4 :| 226 | 227 | X:2 228 | S:3 229 | B:1 230 | E:0 231 | B:17 232 | E:0 233 | E:5 234 | B:17 235 | L:1/8 236 | M:3/4 237 | K:G 238 | D |:"G" G2 GABc |"G/B" d2 B3 G |"C" E3 G E2 |"G" D3 DEF |"G" G2 GABc |"G/B" d2 B3 G | 239 | "D" E2 G2 B2 |"D" A4 DD |"G" G2 GABc |"G/B" d2 B3 G |"C" E3 G E2 |"G" D3 DEF |"G" G2 GABc | 240 | "G/B" d2 B3 G |"D" E2 G2 F2 |1"G" G4 GD :|2"G" G4 DG |:"C" E G3 E2 |"G" D3 E D2 |"G" B3 A G2 | 241 | "G" B d3 B2 |"C" e c3 e2 |"G" d3 B G2 |"Am" E2 G2 B2 |"D" A4 DG |"C" E G3 E2 |"G" D3 E D2 | 242 | "G" B3 A G2 |"G" B d3 B2 |"C" e c3 e2 |"G" d3 B G2 |"D" E2 G2 F2 |1"G" G4 DG :|2"G" G4 z2 |] 243 | 244 | X:3 245 | S:3 246 | B:9 247 | E:5 248 | B:9 249 | E:5 250 | E:5 251 | B:9 252 | L:1/8 253 | M:4/4 254 | K:D 255 | |: A2 | d2 d2 c2 c2 | B2 A2 F2 A2 | B2 B2 c2 c2 | d2 d2 d2 c2 | d2 d2 c2 c2 | B2 A2 F2 A2 | 256 | B2 B2 c2 c2 | d6 ::[K:D] A2 | d2 d2 cB c2 | Bc BA G2 FG | A3 B AG FE | D2 DE FG AB | d2 d2 cB c2 | 257 | Bc BA G2 FG | A3 B AG FE | D6 :: A2 | F2 DF A2 FA | G2 EC E2 A2 | F2 DF A2 FA | G2 EC A,2 A2 | 258 | F2 DF A2 FA | G2 EC E2 A2 | F2 DF A2 AF | G2 EC D2 :| 259 | ``` 260 | 3. Harmonization 261 | ``` 262 | %%input 263 | %%harmonization 264 | L:1/4 265 | M:4/4 266 | K:B 267 | B, | F D/C/ B, F | G3/4A/8B/8 G !fermata!F F | 268 | G A B A | G/B/A/G/ !fermata!F D | 269 | G F E D | C2 !fermata!B, :| z | F2 !fermata!D2 | 270 | F2 !fermata!D2 | D D C C | D D C D | 271 | E D C2 | !fermata!B,2 B A | G F E D | 272 | C2 !fermata!B, |] 273 | ``` 274 | ``` 275 | %%output 276 | X:0 277 | E:5 278 | L:1/4 279 | M:4/4 280 | K:B 281 | "B" B, |"F#/A#" F"B" D/C/"G#m" B,"D#m" F |"G#m7/B" G3/4A/8B/8"C#" G"F#" !fermata!F"B" F | 282 | "E" G"A#dim/D#" A"B/D#" B"F#" A |"G#m7/B" G/B/"A#m/C#"A/G/"F#" !fermata!F"B" D | 283 | "E" G"B/D#" F"C#m7" E"B" D |"F#sus4" C2"B" !fermata!B, :| z |"F#/A#" F2"B" !fermata!D2 | 284 | "F#" F2"B" !fermata!D2 |"B" D"B/D#" D"F#" C"F#" C |"B/D#" D"B/D#" D"F#" C"B#dim/E" D | 285 | "C#m" E"G#m" D"F#7/E" C2 |"B" !fermata!B,2"G#m" B"D#" A |"E" G"D#m/F#" F"Emaj7/G#" E"B" D | 286 | "F#7" C2"B" !fermata!B, |] 287 | 288 | X:1 289 | E:5 290 | L:1/4 291 | M:4/4 292 | K:B 293 | "B" B, |"F#/A#" F"B" D/C/"G#m" B,"B/D#" F |"G#m7/B" G3/4A/8B/8"C#" G"F#" !fermata!F"B" F | 294 | G"F#/A#" A"B/D#" B"F#" A |"G#m7/B" G/B/"A#m/C#"A/G/"F#" !fermata!F"B" D | 295 | G"B/D#" F"C#m" E"D#m7b5/C#" D"A" |"F#7" C2"B" !fermata!B, :|"F#" z |"F#/A#" F2"B" !fermata!D2 | 296 | "B" F2"B" !fermata!D2 |"B" D"B/D#" D"F#" C"F#" C |"B/D#" D"B/D#" D"F#" C"B#dim/E" D | 297 | E"G#m" D"F#7/E" C"G#m"2 |"G#m" !fermata!B,2"G#m" B"A#" A |"G#m" G"D#m/F#" F"Emaj7/G#" E"B" D | 298 | "F#7" C2"B" !fermata!B, |] 299 | 300 | X:2 301 | E:5 302 | L:1/4 303 | M:4/4 304 | K:B 305 | "B" B, |"F#/A#" F"B" D/C/"G#m" B,"D#m" F |"G#m7/B" G3/4A/8B/8"C#" G"F#" !fermata!F"B" F | 306 | G"F#" A"B/D#" B"F#" A"B" |"G#m/B" G/B/"A#dim/C#"A/G/"D#" !fermata!F"G#m" D | 307 | "E" G"E/G#" F"A#m7b5/G#" E"B/F#" D |"F#7" C2"B" !fermata!B, :|"B" z |"D#m" F2"G#m" !fermata!D2 | 308 | "D#m" F2"G#m" !fermata!D2 | D"G#m" D"F#/A#" C"C#m" C | D"B/D#" D"B/D#" C"B" D"F#sus4" | 309 | "C#m" E"G#m" D"F#" C2 | !fermata!B,2"B" B"F#" A |"E" G"F#/A#" F"C#m" E"D#m" D | 310 | "F#7" C2"B" !fermata!B, |] 311 | 312 | X:3 313 | E:5 314 | L:1/4 315 | M:4/4 316 | K:B 317 | "B" B, |"F#/A#" F"B" D/C/"G#m" B,"B/D#" F |"G#m7/B" G3/4A/8B/8"C#" G"F#" !fermata!F"B" F | 318 | G"F#" A"B/D#" B"F#" A"B" |"G#m/B" G/B/"A#dim/C#"A/G/"D#" !fermata!F"B" D | 319 | G"E/G#" F"C#m7" E"B" D"F#" |"F#sus4" C2"B" !fermata!B, :| z |"F#/A#" F2"B" !fermata!D2 | 320 | "F#" F2"B" !fermata!D2 | D"B" D"B/D#" C"B/C#" C | D"B/D#" D"B/D#" C"B" D | 321 | E"C#m" D"F#sus4" C"D#7"2 |"G#m" !fermata!B,2"G#m" B"A#" A |"E" G"D#m/F#" F"E/G#" E"B" D | 322 | "F#7" C2"B" !fermata!B, |] 323 | ``` 324 | 4. Melodization 325 | ``` 326 | %%input 327 | %%melodization 328 | L:1/8 329 | M:6/8 330 | K:G 331 | |: z |"G" z6 | z6 |"Am" z6 |"C" z3"D7" z3 |"G" z6 | z6 |"Am" z3"D7" z3 |"G" z4 z :: z |"C" z6 | 332 | z6 |"Bm" z6 |"Em" z6 |"C" z3"D7" z3 |"Em" z3"Am" z3 |"D7" z6 |"G" z4 :| 333 | ``` 334 | ``` 335 | %%output 336 | X:0 337 | E:5 338 | L:1/8 339 | M:6/8 340 | K:G 341 | |: B/A/ |"G" GDE G2 A | Bgf gdB |"Am" ABc BGA |"C" BcA"D7" BGE |"G" GDE G2 A | Bgf gdB | 342 | "Am" ABc"D7" BcA |"G" BGG G2 :: B/d/ |"C" e2 e e2 e | egf edB |"Bm" d2 d d2 d |"Em" dge dBG | 343 | "C" c2 d"D7" e2 f |"Em" gdB"Am" A2 d |"D7" BGA BcA |"G" BGG G :| 344 | 345 | X:1 346 | E:5 347 | L:1/8 348 | M:6/8 349 | K:G 350 | |: d |"G" GBG GBG | BGG G2 B |"Am" cec ABc |"C" ecA"D7" A2 c |"G" BGG BGG | BGB Bcd |"Am" edc"D7" BcA | 351 | "G" BGG G2 :: d |"G" gfg GBd | gfg bag |"Bm" afd Adf |"Bm" afd def |"C" gfg"G" Bcd | 352 | "Em" gdB"Am" cde |"D7" dcB AGF |"G" BGG G2 :| 353 | 354 | X:2 355 | E:5 356 | L:1/8 357 | M:6/8 358 | K:G 359 | |: d/c/ |"G" BAB G2 E | D2 D DEG |"Am" ABA AGE |"C" cBc"D7" Adc |"G" BAB G2 E | D2 D DEG | 360 | "Am" ABA"D7" AGA |"G" BGG G2 :: B/c/ |"G" d2 d dBG | Bdd d2 B |"Am" c2 c cAA |"Em" B2 B B2 d | 361 | "C" e2 e"D7" dBA |"Em" B2 d"Am" dBA |"D7" GAB AGA |"G" BGG G2 :| 362 | 363 | X:3 364 | E:5 365 | L:1/8 366 | M:6/8 367 | K:G 368 | |: d/c/ |"G" BGG DGG | BGB dcB |"Am" cAA EAA |"C" cBc"D7" edc |"G" BGG DGG | BGB dcB | 369 | "Am" cBc"D7" Adc |"G" BGG G2 :: g/a/ |"G" bgg dgg | bgb bag |"Bm" aff dff |"Bm" afa agf | 370 | "C" egg"G" dgg |"Am" cgg"G" B2 B |"D7" cBc Adc |"G" BGG G2 :| 371 | ``` 372 | 5. Segmentation 373 | ``` 374 | %%input 375 | %%segmentation 376 | L:1/4 377 | M:4/4 378 | K:Eb 379 | "Cm" c"Cm" c"Cm/Eb" g"Cm" g/a/ |"Bb/D" b"Eb" g"Ab" e"Ddim/F" f/g/ |"Bb7/F" a2"Eb" g2 | 380 | "F7/A" f"Bbsus4" f"Cm" e"Bb/D" f |"Eb" g"Bbsus4" f"Cm" e"Bb/D" d |"Cm7/Eb" c2"Bb" B2 | 381 | "Cm" e"Csus2" d"Cm" e/f/"Cm/Eb" g |"Fm" f/e/"G" d"Ab" c2 | 382 | "G" d"Cm/Eb" c/d/"Cm" e"Gsus4" d |"C" c4 |] 383 | ``` 384 | ``` 385 | %%output 386 | X:0 387 | E:9 388 | L:1/4 389 | M:4/4 390 | K:Eb 391 | "Cm" c"Cm" c"Cm/Eb" g"Cm" g/a/ |"Bb/D" b"Eb" g"Ab" e"Ddim/F" f/g/ |"Bb7/F" a2"Eb" !breath!g2 | 392 | "F7/A" f"Bbsus4" f"Cm" e"Bb/D" f |"Eb" g"Bbsus4" f"Cm" e"Bb/D" d |"Cm7/Eb" c2"Bb" !breath!B2 | 393 | "Cm" e"Csus2" d"Cm" e/f/"Cm/Eb" g |"Fm" f/e/"G" d"Ab" !breath!c2 | 394 | "G" d"Cm/Eb" c/d/"Cm" e"Gsus4" d |"C" !breath!c4 |] 395 | 396 | X:1 397 | E:9 398 | L:1/4 399 | M:4/4 400 | K:Eb 401 | "Cm" c"Cm" c"Cm/Eb" g"Cm" g/a/ |"Bb/D" b"Eb" g"Ab" e"Ddim/F" f/g/ |"Bb7/F" a2"Eb" !breath!g2 | 402 | "F7/A" f"Bbsus4" f"Cm" e"Bb/D" f |"Eb" g"Bbsus4" f"Cm" e"Bb/D" d |"Cm7/Eb" c2"Bb" !breath!B2 | 403 | "Cm" e"Csus2" d"Cm" e/f/"Cm/Eb" g |"Fm" f/e/"G" d"Ab" !breath!c2 | 404 | "G" d"Cm/Eb" c/d/"Cm" e"Gsus4" d |"C" !breath!c4 |] 405 | 406 | X:2 407 | E:9 408 | L:1/4 409 | M:4/4 410 | K:Eb 411 | "Cm" c"Cm" c"Cm/Eb" g"Cm" g/a/ |"Bb/D" b"Eb" g"Ab" e"Ddim/F" f/g/ |"Bb7/F" a2"Eb" !breath!g2 | 412 | "F7/A" f"Bbsus4" f"Cm" e"Bb/D" f |"Eb" g"Bbsus4" f"Cm" e"Bb/D" d |"Cm7/Eb" c2"Bb" !breath!B2 | 413 | "Cm" e"Csus2" d"Cm" e/f/"Cm/Eb" g |"Fm" f/e/"G" d"Ab" !breath!c2 | 414 | "G" d"Cm/Eb" c/d/"Cm" e"Gsus4" d |"C" !breath!c4 |] 415 | 416 | X:3 417 | E:9 418 | L:1/4 419 | M:4/4 420 | K:Eb 421 | "Cm" c"Cm" c"Cm/Eb" g"Cm" g/a/ |"Bb/D" b"Eb" g"Ab" e"Ddim/F" f/g/ |"Bb7/F" a2"Eb" !breath!g2 | 422 | "F7/A" f"Bbsus4" f"Cm" e"Bb/D" f |"Eb" g"Bbsus4" f"Cm" e"Bb/D" d |"Cm7/Eb" c2"Bb" !breath!B2 | 423 | "Cm" e"Csus2" d"Cm" e/f/"Cm/Eb" g |"Fm" f/e/"G" d"Ab" !breath!c2 | 424 | "G" d"Cm/Eb" c/d/"Cm" e"Gsus4" d |"C" !breath!c4 |] 425 | ``` 426 | 6. Transcription 427 | ``` 428 | %%input 429 | %%transcription 430 | L:1/8 431 | M:3/4 432 | K:A 433 | EG A2 A2 | BA G2 A2 | Bc/d/ e2 A2 | BA GF E2 | F2 G2 A2 | Bc d2 e2 | 434 | fd c/4B/4c/4B/4c/4B/4c/4B/4 A2 | A2 A4 | EG A2 A2 | BA G2 A2 | Bc/d/ e2 A2 | BA GF E2 | 435 | F2 G2 A2 | Bc d2 e2 | fd c/4B/4c/4B/4c/4B/4c/4B/4 A2 | A2 A4 | cd e2 ef | =g2 f3 e | dc d2 dB | 436 | AB G>F E2 | F2 G2 A2 | Bc d2 e2 | fd c/4B/4c/4B/4c/4B/4c/4B/4 A2 | A2 A4- | A2 cd e2 | 437 | ef =g2 f2- | fe dc d2 | dB AB G>F | E2 F2 G2 | A2 Bc d2 | e2 fd c/4B/4c/4B/4c/4B/4c/4B/4 | 438 | A2 A2 A2- | A4 z2 |] 439 | ``` 440 | ``` 441 | %%output 442 | X:0 443 | E:3 444 | L:1/8 445 | M:3/4 446 | K:A 447 | EG | A2 A2 BA | G2 A2 Bc/d/ | e2 A2 BA | GF E2 F2 | G2 A2 Bc | d2 e2 fd | TB2 A2 A2 | A4 :: cd | 448 | e2 ef =g2 | f3 edc | d2 dB AB | G>F E2 F2 | G2 A2 Bc | d2 e2 fd | TB2 A2 A2 | A6 :| 449 | 450 | X:1 451 | E:3 452 | L:1/8 453 | M:3/4 454 | K:A 455 | EG |"A" A2 A2 BA |"E" G2 A2 Bc/d/ |"A" e2 A2 BA |"E" GF E2 F2 |"E" G2 A2 Bc |"D" d2 e2 fd | 456 | "E" TB2 A2 A2 |"A" A4 :| cd |"A" e2 ef =g2 |"D" f3 e dc |"D" d2 dB AB |"E" G>F E2 F2 | 457 | "E" G2 A2 Bc |"D" d2 e2 fd |"E" TB2 A2 A2 |"A" A6 cd |"A" e2 ef =g2 |"D" f3 e dc |"D" d2 dB AB | 458 | "E" G>F E2 F2 |"E" G2 A2 Bc |"D" d2 e2 fd |"E" TB2 A2 A2 |"A" A4 |] 459 | 460 | X:2 461 | E:3 462 | L:1/8 463 | M:3/4 464 | K:A 465 | |: EG |"A" A2 A2 BA |"E" G2 A2 Bc/d/ |"A" e2 A2 BA |"E" GF E2 F2 |"G" G2 A2 Bc |"D" d2 e2 fd | 466 | "E" TB2 A2 A2 |"A" A4 :| cd |"A" e2 ef =g2 |"D" f3 e dc |"D" d2 dB AB |"E" G>F E2 F2 | 467 | "G" G2 A2 Bc |"D" d2 e2 fd |"E" TB2 A2 A2 |"A" A6 cd |"A" e2 ef =g2 |"D" f3 e dc |"D" d2 dB AB | 468 | "E" G>F E2 F2 |"G" G2 A2 Bc |"D" d2 e2 fd |"E" TB2 A2 A2 |"A" A6 || 469 | 470 | X:3 471 | E:4 472 | L:1/8 473 | M:3/4 474 | K:A 475 | EG | A2 A2 BA | G2 A2 Bc/d/ | e2 A2 BA | GF E2 F2 | G2 A2 Bc | d2 e2 fd | TB2 A2 A2 | A4 :| cd | 476 | e2 ef =g2 | f3 e dc | d2 dB AB | G>F E2 F2 | G2 A2 Bc | d2 e2 fd | TB2 A2 A2 | A6 || cd | e2 ef =g2 | 477 | f3 e dc | d2 dB AB | G>F E2 F2 | G2 A2 Bc | d2 e2 fd | TB2 A2 A2 | A6 || 478 | ``` 479 | 7. Variation 480 | ``` 481 | %%input 482 | %%variation 483 | L:1/8 484 | M:6/8 485 | K:D 486 | |: AFD DFA | Add B2 A | ABA F3 | GFG EFG | AFD DFA | Add B2 A | ABA F2 E | FDD D3 :: fdd ede | 487 | fdd d2 g | fdd def | gfg e2 g | fed B2 A | AdF A3 | ABA F2 E | FDD D3 :| 488 | ``` 489 | ``` 490 | %%output 491 | X:0 492 | E:8 493 | L:1/8 494 | M:6/8 495 | K:D 496 | |: B | AFD DFA | Add B2 A | ABA F2 E | FEE E2 B | AFD DF/G/A | Add B2 A | ABA F2 E | FDD D2 :: e | 497 | fdd dcd | fdd d2 e | f^ef d=ef | g2 f efg | ff/e/d B2 d | Add F2 G | ABA F2 E | FDD D2 :| 498 | 499 | X:1 500 | E:8 501 | L:1/8 502 | M:6/8 503 | K:D 504 | |: B | AFD DFA | ded B2 A | ABA F2 D | GFG E2 B | AFD DF/G/A | df/e/d B2 A | ABA F2 E | EDD D2 :: 505 | e | fdd e^de | fdd d2 e | f2 f def | g2 f e2 g | fed B2 d | A2 d F3 | ABA F2 E | EDD D2 :| 506 | 507 | X:2 508 | E:8 509 | L:1/8 510 | M:6/8 511 | K:D 512 | |: B | AFD DFA | BdB BAF | ABA F2 D | FEE E2 B | AFD DFA | BdB BAF | ABA F2 E |1 FDD D2 :|2 513 | FDD D2 e |: fdd dcd | fdd d2 e | fef def | gfg eag | fed B2 d | A2 d F2 G | ABA F2 E |1 514 | FDD D2 e :|2 FDD D2 || 515 | 516 | X:3 517 | E:5 518 | L:1/8 519 | M:6/8 520 | K:D 521 | |: (d/B/) | AFD DFA | B2 d F2 A | AFD DEF | GFG EFG | AFD DFA | B2 d F2 A | Bdd F2 E | FDD D2 :: 522 | fed dB/c/d | efe efg | fed daa | agf eag | fed B2 d | A2 d F2 A | Bdd F2 E | FDD D2 :| 523 | ``` 524 | 525 | ## BibTeX Citation 526 | ``` 527 | @misc{wu2024melodyt5unifiedscoretoscoretransformer, 528 | title={MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing}, 529 | author={Shangda Wu and Yashan Wang and Xiaobing Li and Feng Yu and Maosong Sun}, 530 | year={2024}, 531 | eprint={2407.02277}, 532 | archivePrefix={arXiv}, 533 | primaryClass={cs.SD}, 534 | url={https://arxiv.org/abs/2407.02277}, 535 | } 536 | ``` 537 | -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- 1 | PATCH_LENGTH = 256 # Patch Length 2 | PATCH_SIZE = 64 # Patch Size 3 | 4 | PATCH_NUM_LAYERS = 9 # Number of layers in the encoder 5 | CHAR_NUM_LAYERS = 3 # Number of layers in the decoder 6 | 7 | NUM_EPOCHS = 32 # Number of epochs to train for (if early stopping doesn't intervene) 8 | LEARNING_RATE = 2e-4 # Learning rate for the optimizer 9 | ACCUMULATION_STEPS = 1 # Accumulation steps to simulate large batch size 10 | BATCH_SIZE = 10 # Batch size for training 11 | PATCH_SAMPLING_BATCH_SIZE = 0 # Batch size for patch during training, 0 for full context 12 | LOAD_FROM_CHECKPOINT = True # Whether to load weights from a checkpoint 13 | LOAD_FROM_PRETRAINED = True # Whether to load weights from a pretrained model 14 | SHARE_WEIGHTS = True # Whether to share weights between the encoder and decoder 15 | 16 | TRAIN_DATA_PATH = 'total_data_train.jsonl' 17 | VALIDATION_DATA_PATH = 'total_data_validation.jsonl' 18 | 19 | PRETRAINED_PATH = "weights.pth" # Path to the pretrained weights 20 | WEIGHTS_PATH = "weights.pth" # Path to save the weights 21 | LOGS_PATH = "logs.txt" # Path to save the logs -------------------------------------------------------------------------------- /inference.py: -------------------------------------------------------------------------------- 1 | import time 2 | import torch 3 | from utils import * 4 | import re 5 | from config import * 6 | from transformers import GPT2Config 7 | import argparse 8 | 9 | def get_args(parser): 10 | 11 | parser.add_argument('-num_tunes', type=int, default=3, help='the number of independently computed returned tunes') 12 | parser.add_argument('-max_patch', type=int, default=128, help='integer to define the maximum length in tokens of each tune') 13 | parser.add_argument('-top_p', type=float, default=0.8, help='float to define the tokens that are within the sample operation of text generation') 14 | parser.add_argument('-top_k', type=int, default=8, help='integer to define the tokens that are within the sample operation of text generation') 15 | parser.add_argument('-temperature', type=float, default=2.6, help='the temperature of the sampling operation') 16 | parser.add_argument('-seed', type=int, default=None, help='seed for randomstate') 17 | parser.add_argument('-show_control_code', type=bool, default=True, help='whether to show control code') 18 | args = parser.parse_args() 19 | 20 | return args 21 | 22 | def generate_abc(args): 23 | 24 | if torch.cuda.is_available(): 25 | device = torch.device("cuda") 26 | else: 27 | device = torch.device("cpu") 28 | patchilizer = Patchilizer() 29 | 30 | patch_config = GPT2Config(num_hidden_layers=PATCH_NUM_LAYERS, 31 | max_length=PATCH_LENGTH, 32 | max_position_embeddings=PATCH_LENGTH, 33 | vocab_size=1) 34 | char_config = GPT2Config(num_hidden_layers=CHAR_NUM_LAYERS, 35 | max_length=PATCH_SIZE, 36 | max_position_embeddings=PATCH_SIZE, 37 | vocab_size=128) 38 | model = MelodyT5(patch_config, char_config) 39 | 40 | checkpoint = torch.load(WEIGHTS_PATH, map_location=device) 41 | model.load_state_dict(checkpoint['model']) 42 | model = model.to(device) 43 | model.eval() 44 | # print parameter number 45 | print("Parameter Number: "+str(sum(p.numel() for p in model.parameters() if p.requires_grad))) 46 | 47 | with open('prompt.txt', 'r') as f: 48 | prompt = f.read() 49 | 50 | if "%%output\n" not in prompt: 51 | input_abc = prompt.replace("%%input\n", "") 52 | prompt = "" 53 | else: 54 | input_abc = prompt.replace("%%input\n", "").split("%%output\n")[0] 55 | prompt = prompt.split("%%output\n")[1] 56 | task = input_abc.split("\n")[0][2:] 57 | 58 | tunes = "" 59 | num_tunes = args.num_tunes 60 | max_patch = args.max_patch 61 | top_p = args.top_p 62 | top_k = args.top_k 63 | temperature = args.temperature 64 | seed = args.seed 65 | show_control_code = args.show_control_code 66 | 67 | print(" HYPERPARAMETERS ".center(60, "#"), '\n') 68 | args = vars(args) 69 | for key in args.keys(): 70 | print(key+': '+str(args[key])) 71 | print("\n"+" OUTPUT TUNES ".center(60, "#")) 72 | 73 | start_time = time.time() 74 | 75 | for i in range(num_tunes): 76 | if prompt=="": 77 | tune = "X:"+str(i+1) + "\n" 78 | else: 79 | tune = "X:"+str(i+1) + "\n" + prompt.strip() 80 | lines = re.split(r'(\n)', tune) 81 | tune = "" 82 | skip = False 83 | for line in lines: 84 | if show_control_code or line[:2] not in ["S:", "B:", "E:"]: 85 | if not skip: 86 | print(line, end="") 87 | tune += line 88 | skip = False 89 | else: 90 | skip = True 91 | 92 | patches = torch.tensor([patchilizer.encode(input_abc, add_special_patches=True)], device=device) 93 | decoder_patches = torch.tensor([patchilizer.encode(prompt, add_special_patches=True)[:-1]], device=device) 94 | tokens = None 95 | 96 | while decoder_patches.shape[1] 1: 24 | torch.cuda.set_device(local_rank) 25 | device = torch.device("cuda", local_rank) 26 | dist.init_process_group(backend='nccl') if world_size > 1 else None 27 | else: 28 | device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu") 29 | 30 | seed = 0 + global_rank 31 | random.seed(seed) 32 | np.random.seed(seed) 33 | torch.manual_seed(seed) 34 | torch.cuda.manual_seed_all(seed) 35 | torch.backends.cudnn.deterministic = True 36 | torch.backends.cudnn.benchmark = False 37 | 38 | batch_size = BATCH_SIZE 39 | patchilizer = Patchilizer() 40 | 41 | patch_config = GPT2Config(num_hidden_layers=PATCH_NUM_LAYERS, 42 | max_length=PATCH_LENGTH, 43 | max_position_embeddings=PATCH_LENGTH, 44 | vocab_size=1) 45 | char_config = GPT2Config(num_hidden_layers=CHAR_NUM_LAYERS, 46 | max_length=PATCH_SIZE, 47 | max_position_embeddings=PATCH_SIZE, 48 | vocab_size=128) 49 | 50 | model = MelodyT5(patch_config, char_config) 51 | model = model.to(device) 52 | # print parameter number 53 | print("Parameter Number: "+str(sum(p.numel() for p in model.parameters() if p.requires_grad))) 54 | 55 | if world_size > 1: 56 | model = DDP(model, device_ids=[local_rank], output_device=local_rank, find_unused_parameters=True) 57 | 58 | scaler = GradScaler() 59 | is_autocast = True 60 | optimizer = torch.optim.AdamW(model.parameters(), lr=LEARNING_RATE) 61 | 62 | def collate_batch(batch): 63 | input_patches, input_masks, output_patches, output_masks = [], [], [], [] 64 | 65 | for input_patch, output_patch in batch: 66 | input_patches.append(input_patch) 67 | input_masks.append(torch.tensor([1]*input_patch.shape[0])) 68 | output_patches.append(output_patch) 69 | output_masks.append(torch.tensor([1]*output_patch.shape[0])) 70 | 71 | input_patches = torch.nn.utils.rnn.pad_sequence(input_patches, batch_first=True, padding_value=0) 72 | input_masks = torch.nn.utils.rnn.pad_sequence(input_masks, batch_first=True, padding_value=0) 73 | output_patches = torch.nn.utils.rnn.pad_sequence(output_patches, batch_first=True, padding_value=0) 74 | output_masks = torch.nn.utils.rnn.pad_sequence(output_masks, batch_first=True, padding_value=0) 75 | 76 | return input_patches.to(device), input_masks.to(device), output_patches.to(device), output_masks.to(device) 77 | 78 | def split_data(data, eval_ratio=0.1): 79 | random.shuffle(data) 80 | split_idx = int(len(data)*eval_ratio) 81 | eval_set = data[:split_idx] 82 | train_set = data[split_idx:] 83 | return train_set, eval_set 84 | 85 | class MelodyHubDataset(Dataset): 86 | def __init__(self, items): 87 | self.inputs = [] 88 | self.outputs = [] 89 | 90 | for item in tqdm(items): 91 | input_patch = patchilizer.encode(item['input'], add_special_patches=True) 92 | input_patch = torch.tensor(input_patch) 93 | 94 | output_patch = patchilizer.encode(item["output"], add_special_patches=True) 95 | output_patch = torch.tensor(output_patch) 96 | if torch.sum(output_patch)!=0: 97 | self.inputs.append(input_patch) 98 | self.outputs.append(output_patch) 99 | 100 | def __len__(self): 101 | return len(self.inputs) 102 | 103 | def __getitem__(self, idx): 104 | return self.inputs[idx], self.outputs[idx] 105 | 106 | # call model with a batch of input 107 | def process_one_batch(batch): 108 | input_patches, input_masks, output_patches, output_masks = batch 109 | 110 | loss = model(input_patches, 111 | input_masks, 112 | output_patches, 113 | output_masks) 114 | 115 | # Reduce the loss on GPU 0 116 | if world_size > 1: 117 | loss = loss.unsqueeze(0) 118 | dist.reduce(loss, dst=0) 119 | loss = loss / world_size 120 | dist.broadcast(loss, src=0) 121 | 122 | return loss 123 | 124 | # do one epoch for training 125 | def train_epoch(): 126 | tqdm_train_set = tqdm(train_set) 127 | total_train_loss = 0 128 | iter_idx = 1 129 | model.train() 130 | 131 | for batch in tqdm_train_set: 132 | with autocast(): 133 | loss = process_one_batch(batch) 134 | if loss is None or torch.isnan(loss).item(): 135 | continue 136 | scaler.scale(loss).backward() 137 | scaler.step(optimizer) 138 | scaler.update() 139 | 140 | lr_scheduler.step() 141 | model.zero_grad(set_to_none=True) 142 | total_train_loss += loss.item() 143 | tqdm_train_set.set_postfix({str(global_rank)+'_train_loss': total_train_loss / iter_idx}) 144 | iter_idx += 1 145 | 146 | return total_train_loss / (iter_idx-1) 147 | 148 | # do one epoch for eval 149 | def eval_epoch(): 150 | tqdm_eval_set = tqdm(eval_set) 151 | total_eval_loss = 0 152 | iter_idx = 1 153 | model.eval() 154 | 155 | # Evaluate data for one epoch 156 | for batch in tqdm_eval_set: 157 | with torch.no_grad(): 158 | loss = process_one_batch(batch) 159 | if loss is None or torch.isnan(loss).item(): 160 | continue 161 | total_eval_loss += loss.item() 162 | tqdm_eval_set.set_postfix({str(global_rank)+'_eval_loss': total_eval_loss / iter_idx}) 163 | iter_idx += 1 164 | return total_eval_loss / (iter_idx-1) 165 | 166 | # train and eval 167 | if __name__ == "__main__": 168 | 169 | train_set = [] 170 | eval_set = [] 171 | with open(TRAIN_DATA_PATH, 'r', encoding='utf-8') as file: 172 | for line in file: 173 | data = json.loads(line.strip()) 174 | train_set.append(data) 175 | with open(VALIDATION_DATA_PATH, 'r', encoding='utf-8') as file: 176 | for line in file: 177 | data = json.loads(line.strip()) 178 | eval_set.append(data) 179 | 180 | train_batch_nums = int(len(train_set) / batch_size) 181 | eval_batch_nums = int(len(eval_set) / batch_size) 182 | 183 | random.shuffle(train_set) 184 | random.shuffle(eval_set) 185 | 186 | train_set = train_set[:train_batch_nums*batch_size] 187 | eval_set = eval_set[:eval_batch_nums*batch_size] 188 | 189 | train_set = MelodyHubDataset(train_set) 190 | eval_set = MelodyHubDataset(eval_set) 191 | 192 | train_sampler = DistributedSampler(train_set, num_replicas=world_size, rank=local_rank) 193 | eval_sampler = DistributedSampler(eval_set, num_replicas=world_size, rank=local_rank) 194 | 195 | train_set = DataLoader(train_set, batch_size=batch_size, collate_fn=collate_batch, sampler=train_sampler, shuffle = (train_sampler is None)) 196 | eval_set = DataLoader(eval_set, batch_size=batch_size, collate_fn=collate_batch, sampler=eval_sampler, shuffle = (train_sampler is None)) 197 | 198 | lr_scheduler = get_constant_schedule_with_warmup(optimizer = optimizer, num_warmup_steps = 1000) 199 | 200 | model = model.to(device) 201 | optimizer = torch.optim.AdamW(model.parameters(), lr=LEARNING_RATE) 202 | 203 | if LOAD_FROM_PRETRAINED and os.path.exists(PRETRAINED_PATH): 204 | # Load checkpoint to CPU 205 | checkpoint = torch.load(PRETRAINED_PATH, map_location='cpu') 206 | 207 | # Here, model is assumed to be on GPU 208 | # Load state dict to CPU model first, then move the model to GPU 209 | if torch.cuda.device_count() > 1: 210 | # If you have a DataParallel model, you need to load to model.module instead 211 | cpu_model = deepcopy(model.module) 212 | cpu_model.load_state_dict(checkpoint['model']) 213 | model.module.load_state_dict(cpu_model.state_dict()) 214 | else: 215 | # Load to a CPU clone of the model, then load back 216 | cpu_model = deepcopy(model) 217 | cpu_model.load_state_dict(checkpoint['model']) 218 | model.load_state_dict(cpu_model.state_dict()) 219 | 220 | print(f"Successfully Loaded Pretrained Checkpoint at Epoch {checkpoint['epoch']} with Loss {checkpoint['min_eval_loss']}") 221 | 222 | else: 223 | pre_epoch = 0 224 | best_epoch = 0 225 | min_eval_loss = float('inf') 226 | 227 | if LOAD_FROM_CHECKPOINT and os.path.exists(WEIGHTS_PATH): 228 | # Load checkpoint to CPU 229 | checkpoint = torch.load(WEIGHTS_PATH, map_location='cpu') 230 | 231 | # Here, model is assumed to be on GPU 232 | # Load state dict to CPU model first, then move the model to GPU 233 | if torch.cuda.device_count() > 1: 234 | # If you have a DataParallel model, you need to load to model.module instead 235 | cpu_model = deepcopy(model.module) 236 | cpu_model.load_state_dict(checkpoint['model']) 237 | model.module.load_state_dict(cpu_model.state_dict()) 238 | else: 239 | # Load to a CPU clone of the model, then load back 240 | cpu_model = deepcopy(model) 241 | cpu_model.load_state_dict(checkpoint['model']) 242 | model.load_state_dict(cpu_model.state_dict()) 243 | optimizer.load_state_dict(checkpoint['optimizer']) 244 | lr_scheduler.load_state_dict(checkpoint['lr_sched']) 245 | pre_epoch = checkpoint['epoch'] 246 | best_epoch = checkpoint['best_epoch'] 247 | min_eval_loss = checkpoint['min_eval_loss'] 248 | print("Successfully Loaded Checkpoint from Epoch %d" % pre_epoch) 249 | checkpoint = None 250 | 251 | else: 252 | pre_epoch = 0 253 | best_epoch = 0 254 | min_eval_loss = float('inf') 255 | 256 | for epoch in range(1+pre_epoch, NUM_EPOCHS+1): 257 | train_sampler.set_epoch(epoch) 258 | eval_sampler.set_epoch(epoch) 259 | print('-' * 21 + "Epoch " + str(epoch) + '-' * 21) 260 | train_loss = train_epoch() 261 | eval_loss = eval_epoch() 262 | if global_rank==0: 263 | with open(LOGS_PATH,'a') as f: 264 | f.write("Epoch " + str(epoch) + "\ntrain_loss: " + str(train_loss) + "\neval_loss: " +str(eval_loss) + "\ntime: " + time.asctime(time.localtime(time.time())) + "\n\n") 265 | if eval_loss < min_eval_loss: 266 | best_epoch = epoch 267 | min_eval_loss = eval_loss 268 | checkpoint = { 269 | 'model': model.module.state_dict() if hasattr(model, "module") else model.state_dict(), 270 | 'optimizer': optimizer.state_dict(), 271 | 'lr_sched': lr_scheduler.state_dict(), 272 | 'epoch': epoch, 273 | 'best_epoch': best_epoch, 274 | 'min_eval_loss': min_eval_loss 275 | } 276 | torch.save(checkpoint, WEIGHTS_PATH) 277 | 278 | if world_size > 1: 279 | dist.barrier() 280 | 281 | if global_rank==0: 282 | print("Best Eval Epoch : "+str(best_epoch)) 283 | print("Min Eval Loss : "+str(min_eval_loss)) 284 | 285 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | import re 2 | import torch 3 | import random 4 | from config import * 5 | from unidecode import unidecode 6 | from samplings import top_p_sampling, top_k_sampling, temperature_sampling 7 | from transformers import GPT2LMHeadModel, PreTrainedModel, EncoderDecoderConfig, EncoderDecoderModel 8 | 9 | class Patchilizer: 10 | """ 11 | A class for converting music bars to patches and vice versa. 12 | """ 13 | def __init__(self): 14 | self.delimiters = ["|:", "::", ":|", "[|", "||", "|]", "|"] 15 | self.regexPattern = '(' + '|'.join(map(re.escape, self.delimiters)) + ')' 16 | self.pad_token_id = 0 17 | self.bos_token_id = 1 18 | self.eos_token_id = 2 19 | 20 | def split_bars(self, body): 21 | """ 22 | Split a body of music into individual bars. 23 | """ 24 | bars = re.split(self.regexPattern, ''.join(body)) 25 | bars = list(filter(None, bars)) # remove empty strings 26 | if bars[0] in self.delimiters: 27 | bars[1] = bars[0] + bars[1] 28 | bars = bars[1:] 29 | bars = [bars[i * 2] + bars[i * 2 + 1] for i in range(len(bars) // 2)] 30 | return bars 31 | 32 | def bar2patch(self, bar, patch_size=PATCH_SIZE): 33 | """ 34 | Convert a bar into a patch of specified length. 35 | """ 36 | patch = [self.bos_token_id] + [ord(c) for c in bar] + [self.eos_token_id] 37 | patch = patch[:patch_size] 38 | patch += [self.pad_token_id] * (patch_size - len(patch)) 39 | return patch 40 | 41 | def patch2bar(self, patch): 42 | """ 43 | Convert a patch into a bar. 44 | """ 45 | return ''.join(chr(idx) if idx > self.eos_token_id else '' for idx in patch if idx != self.eos_token_id) 46 | 47 | def encode(self, abc_code, patch_length=PATCH_LENGTH, patch_size=PATCH_SIZE, add_special_patches=False): 48 | """ 49 | Encode music into patches of specified length. 50 | """ 51 | lines = unidecode(abc_code).split('\n') 52 | lines = list(filter(None, lines)) # remove empty lines 53 | 54 | body = "" 55 | patches = [] 56 | 57 | for line in lines: 58 | if len(line) > 1 and ((line[0].isalpha() and line[1] == ':') or line.startswith('%%')): 59 | if body: 60 | bars = self.split_bars(body) 61 | patches.extend(self.bar2patch(bar + '\n' if idx == len(bars) - 1 else bar, patch_size) 62 | for idx, bar in enumerate(bars)) 63 | body = "" 64 | patches.append(self.bar2patch(line + '\n', patch_size)) 65 | else: 66 | body += line + '\n' 67 | 68 | if body: 69 | patches.extend(self.bar2patch(bar, patch_size) for bar in self.split_bars(body)) 70 | 71 | if add_special_patches: 72 | bos_patch = [self.bos_token_id] * (patch_size-1) + [self.eos_token_id] 73 | eos_patch = [self.bos_token_id] + [self.eos_token_id] * (patch_size-1) 74 | patches = [bos_patch] + patches + [eos_patch] 75 | 76 | return patches[:patch_length] 77 | 78 | def decode(self, patches): 79 | """ 80 | Decode patches into music. 81 | """ 82 | return ''.join(self.patch2bar(patch) for patch in patches) 83 | 84 | class PatchLevelEnDecoder(PreTrainedModel): 85 | """ 86 | An Patch-level Decoder model for generating patch features in an auto-regressive manner. 87 | It inherits PreTrainedModel from transformers. 88 | """ 89 | 90 | def __init__(self, config): 91 | super().__init__(config) 92 | self.patch_embedding = torch.nn.Linear(PATCH_SIZE * 128, config.n_embd) 93 | torch.nn.init.normal_(self.patch_embedding.weight, std=0.02) 94 | if SHARE_WEIGHTS: 95 | try: 96 | self.base = EncoderDecoderModel.from_encoder_decoder_pretrained("random_model", "random_model", tie_encoder_decoder=True) 97 | except Exception as e: 98 | print("Error loading 'random_model':", e) 99 | print("Please run 'random_model.py' to create randomly initialized weights.") 100 | raise 101 | else: 102 | config = EncoderDecoderConfig.from_encoder_decoder_configs(config, config) 103 | self.config = config 104 | self.base = EncoderDecoderModel(config=self.config) 105 | 106 | self.base.config.pad_token_id = 0 107 | self.base.config.decoder_start_token_id = 1 108 | 109 | def forward(self, 110 | patches: torch.Tensor, 111 | masks: torch.Tensor, 112 | decoder_patches: torch.Tensor, 113 | decoder_masks: torch.Tensor): 114 | """ 115 | The forward pass of the patch-level decoder model. 116 | :param patches: the patches to be encoded 117 | :return: the encoded patches 118 | """ 119 | patches = torch.nn.functional.one_hot(patches, num_classes=128).float() 120 | patches = patches.reshape(len(patches), -1, PATCH_SIZE * 128) 121 | patches = self.patch_embedding(patches.to(self.device)) 122 | 123 | decoder_patches = torch.nn.functional.one_hot(decoder_patches, num_classes=128).float() 124 | decoder_patches = decoder_patches.reshape(len(decoder_patches), -1, PATCH_SIZE * 128) 125 | decoder_patches = self.patch_embedding(decoder_patches.to(self.device)) 126 | 127 | if masks==None or decoder_masks==None: 128 | return self.base(inputs_embeds=patches, 129 | decoder_inputs_embeds=decoder_patches, 130 | output_hidden_states = True)["decoder_hidden_states"][-1] 131 | else: 132 | return self.base(inputs_embeds=patches, 133 | attention_mask=masks, 134 | decoder_inputs_embeds=decoder_patches, 135 | decoder_attention_mask=decoder_masks, 136 | output_hidden_states = True)["decoder_hidden_states"][-1] 137 | 138 | class CharLevelDecoder(PreTrainedModel): 139 | """ 140 | A Char-level Decoder model for generating the characters within each bar patch sequentially. 141 | It inherits PreTrainedModel from transformers. 142 | """ 143 | def __init__(self, config): 144 | super().__init__(config) 145 | self.pad_token_id = 0 146 | self.bos_token_id = 1 147 | self.eos_token_id = 2 148 | 149 | self.base = GPT2LMHeadModel(config) 150 | 151 | def forward(self, encoded_patches: torch.Tensor, target_patches: torch.Tensor): 152 | """ 153 | The forward pass of the char-level decoder model. 154 | :param encoded_patches: the encoded patches 155 | :param target_patches: the target patches 156 | :return: the decoded patches 157 | """ 158 | # preparing the labels for model training 159 | target_masks = target_patches == self.pad_token_id 160 | labels = target_patches.clone().masked_fill_(target_masks, -100) 161 | 162 | # masking the labels for model training 163 | target_masks = torch.ones_like(labels) 164 | target_masks = target_masks.masked_fill_(labels == -100, 0) 165 | 166 | # select patches 167 | if PATCH_SAMPLING_BATCH_SIZE!=0 and PATCH_SAMPLING_BATCH_SIZE 1) 246 | decoder_masks[:, 0] = 0 247 | 248 | encoded_patches = encoded_patches[left_shift_masks == 1] 249 | decoder_patches = decoder_patches[decoder_masks == 1] 250 | 251 | return self.char_level_decoder(encoded_patches, 252 | decoder_patches)["loss"] 253 | 254 | def generate(self, 255 | patches: torch.Tensor, 256 | decoder_patches: torch.Tensor, 257 | tokens: torch.Tensor, 258 | task: str, 259 | top_p: float=1, 260 | top_k: int=0, 261 | temperature: float=1, 262 | seed: int=None): 263 | """ 264 | The generate function for generating patches based on patches. 265 | :param patches: the patches to be encoded 266 | :return: the generated patches 267 | """ 268 | patches = patches.reshape(len(patches), -1, PATCH_SIZE) 269 | decoder_patches = decoder_patches.reshape(len(decoder_patches), -1, PATCH_SIZE) 270 | encoded_patches = self.patch_level_decoder(patches=patches, 271 | masks=None, 272 | decoder_patches=decoder_patches, 273 | decoder_masks=None) 274 | if tokens==None: 275 | tokens = torch.tensor([self.bos_token_id], device=self.device) 276 | generated_patch = [] 277 | random.seed(seed) 278 | 279 | if task in ["harmonization", "segmentation"] and decoder_patches.shape[1] > 1: 280 | if task == "harmonization": 281 | special_token = ord('"') 282 | else: 283 | special_token = ord('!') 284 | copy_flag = True 285 | reference_patch = patches[0][decoder_patches.shape[1]] 286 | reference_idx = tokens.shape[0] 287 | 288 | while True: 289 | if seed!=None: 290 | n_seed = random.randint(0, 1000000) 291 | random.seed(n_seed) 292 | else: 293 | n_seed = None 294 | prob = self.char_level_decoder.generate(encoded_patches[0][-1], tokens).cpu().detach().numpy() 295 | prob = top_p_sampling(prob, top_p=top_p, return_probs=True) 296 | prob = top_k_sampling(prob, top_k=top_k, return_probs=True) 297 | token = temperature_sampling(prob, temperature=temperature, seed=n_seed) 298 | 299 | if token == self.eos_token_id or len(tokens) >= PATCH_SIZE - 1: 300 | generated_patch.append(token) 301 | break 302 | else: 303 | if task in ["harmonization", "segmentation"] and decoder_patches.shape[1] > 1: 304 | reference_token = reference_patch[reference_idx].item() 305 | reference_idx += 1 306 | n_special_tokens = sum([1 for t in tokens if t == special_token]) 307 | 308 | if token == special_token and token != reference_token: 309 | reference_idx -= 1 310 | if n_special_tokens % 2 == 0: 311 | copy_flag = False 312 | else: 313 | copy_flag = True 314 | 315 | if token != special_token: 316 | if copy_flag: 317 | token = reference_token 318 | else: 319 | reference_idx -= 1 320 | 321 | generated_patch.append(token) 322 | tokens = torch.cat((tokens, torch.tensor([token], device=self.device)), dim=0) 323 | 324 | return generated_patch, n_seed 325 | -------------------------------------------------------------------------------- /xml2abc/batch_xml2abc.py: -------------------------------------------------------------------------------- 1 | import os 2 | import math 3 | import argparse 4 | import subprocess 5 | from tqdm import trange 6 | from unidecode import unidecode 7 | from multiprocessing import Pool 8 | 9 | cmd = 'cd '+os.getcwd() 10 | output = os.popen(cmd).read() 11 | cmd = 'cmd /u /c python xml2abc.py -m 2 -c 6 -x ' 12 | 13 | def parse_args(): 14 | parser = argparse.ArgumentParser(description='Process some files.') 15 | parser.add_argument('path', type=str, help='The path to the directory containing the files') 16 | return parser.parse_args() 17 | 18 | def run_filter(lines): 19 | score = "" 20 | for line in lines.replace("\r", "").split("\n"): 21 | if line[:2] in ['A:', 'B:', 'C:', 'D:', 'F:', 'G', 'H:', 'N:', 'O:', 'R:', 'r:', 'S:', 'T:', 'V:', 'W:', 'X:', 'Z:'] \ 22 | or line=='\n' \ 23 | or line.startswith('%'): 24 | continue 25 | else: 26 | if '%' in line: 27 | line = line.split('%') 28 | line = ''.join(line[:-1]) 29 | score += line + '\n' 30 | score = score.strip() 31 | if score.endswith(" |"): 32 | score += "]" 33 | return score 34 | 35 | def convert_abc(file_list): 36 | for file_idx in trange(len(file_list)): 37 | file = file_list[file_idx] 38 | filename = os.path.basename(file) 39 | directory = os.path.dirname(file) 40 | path_parts = directory.split(os.sep) 41 | if path_parts: 42 | path_parts[0] = path_parts[0] + "_abc" 43 | new_directory = os.path.join(*path_parts) 44 | os.makedirs(new_directory, exist_ok=True) 45 | try: 46 | p = subprocess.Popen(cmd+'"'+file+'"', stdout=subprocess.PIPE, stderr=subprocess.DEVNULL) 47 | result = p.communicate() 48 | output = result[0].decode('utf-8') 49 | output = run_filter(output) 50 | output = unidecode(output) 51 | 52 | if output=='': 53 | continue 54 | else: 55 | with open(new_directory+'/'+filename[:-4]+'.abc', 'w', encoding='utf-8') as f: 56 | f.write(output) 57 | except Exception as e: 58 | print(e) 59 | pass 60 | 61 | if __name__ == '__main__': 62 | args = parse_args() 63 | file_list = [] 64 | abc_list = [] 65 | 66 | for root, dirs, files in os.walk(args.path): 67 | for file in files: 68 | filename = os.path.join(root, file) 69 | file_list.append(filename) 70 | 71 | file_lists = [] 72 | for i in range(os.cpu_count()): 73 | start_idx = int(math.floor(i*len(file_list)/os.cpu_count())) 74 | end_idx = int(math.floor((i+1)*len(file_list)/os.cpu_count())) 75 | file_lists.append(file_list[start_idx:end_idx]) 76 | 77 | pool = Pool(processes=os.cpu_count()) 78 | pool.map(convert_abc, file_lists) -------------------------------------------------------------------------------- /xml2abc/xml2abc.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding=latin-1 3 | ''' 4 | Copyright (C) 2012-2018: W.G. Vree 5 | Contributions: M. Tarenskeen, N. Liberg, Paul Villiger, Janus Meuris, Larry Myerscough, 6 | Dick Jackson, Jan Wybren de Jong, Mark Zealey. 7 | 8 | This program is free software; you can redistribute it and/or modify it under the terms of the 9 | Lesser GNU General Public License as published by the Free Software Foundation; 10 | 11 | This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; 12 | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 13 | See the Lesser GNU General Public License for more details. . 14 | ''' 15 | 16 | try: import xml.etree.cElementTree as E 17 | except: import xml.etree.ElementTree as E 18 | import os, sys, types, re, math 19 | 20 | VERSION = 143 21 | 22 | python3 = sys.version_info.major > 2 23 | if python3: 24 | tupletype = tuple 25 | listtype = list 26 | max_int = sys.maxsize 27 | else: 28 | tupletype = types.TupleType 29 | listtype = types.ListType 30 | max_int = sys.maxint 31 | 32 | note_ornamentation_map = { # for notations/, modified from EasyABC 33 | 'ornaments/trill-mark': 'T', 34 | 'ornaments/mordent': 'M', 35 | 'ornaments/inverted-mordent': 'P', 36 | 'ornaments/turn': '!turn!', 37 | 'ornaments/inverted-turn': '!invertedturn!', 38 | 'technical/up-bow': 'u', 39 | 'technical/down-bow': 'v', 40 | 'technical/harmonic': '!open!', 41 | 'technical/open-string': '!open!', 42 | 'technical/stopped': '!plus!', 43 | 'technical/snap-pizzicato': '!snap!', 44 | 'technical/thumb-position': '!thumb!', 45 | 'articulations/accent': '!>!', 46 | 'articulations/strong-accent':'!^!', 47 | 'articulations/staccato': '.', 48 | 'articulations/staccatissimo':'!wedge!', 49 | 'articulations/scoop': '!slide!', 50 | 'fermata': '!fermata!', 51 | 'arpeggiate': '!arpeggio!', 52 | 'articulations/tenuto': '!tenuto!', 53 | 'articulations/staccatissimo':'!wedge!', # not sure whether this is the right translation 54 | 'articulations/spiccato': '!wedge!', # not sure whether this is the right translation 55 | 'articulations/breath-mark': '!breath!', # this may need to be tested to make sure it appears on the right side of the note 56 | 'articulations/detached-legato': '!tenuto!.', 57 | } 58 | 59 | dynamics_map = { # for direction/direction-type/dynamics/ 60 | 'p': '!p!', 61 | 'pp': '!pp!', 62 | 'ppp': '!ppp!', 63 | 'pppp': '!pppp!', 64 | 'f': '!f!', 65 | 'ff': '!ff!', 66 | 'fff': '!fff!', 67 | 'ffff': '!ffff!', 68 | 'mp': '!mp!', 69 | 'mf': '!mf!', 70 | 'sfz': '!sfz!', 71 | } 72 | 73 | percSvg = '''%%beginsvg 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | %%endsvg''' 94 | 95 | tabSvg = '''%%beginsvg 96 | 99 | 100 | 101 | ''' 102 | 103 | kopSvg = '%s\n' 104 | kopSvg2 = '%s\n' 105 | 106 | def info (s, warn=1): sys.stderr.write ((warn and '-- ' or '') + s + '\n') 107 | 108 | #------------------- 109 | # data abstractions 110 | #------------------- 111 | class Measure: 112 | def __init__ (s, p): 113 | s.reset () 114 | s.ixp = p # part number 115 | s.ixm = 0 # measure number 116 | s.mdur = 0 # measure duration (nominal metre value in divisions) 117 | s.divs = 0 # number of divisions per 1/4 118 | s.mtr = 4,4 # meter 119 | 120 | def reset (s): # reset each measure 121 | s.attr = '' # measure signatures, tempo 122 | s.lline = '' # left barline, but only holds ':' at start of repeat, otherwise empty 123 | s.rline = '|' # right barline 124 | s.lnum = '' # (left) volta number 125 | 126 | class Note: 127 | def __init__ (s, dur=0, n=None): 128 | s.tijd = 0 # the time in XML division units 129 | s.dur = dur # duration of a note in XML divisions 130 | s.fact = None # time modification for tuplet notes (num, div) 131 | s.tup = [''] # start(s) and/or stop(s) of tuplet 132 | s.tupabc = '' # abc tuplet string to issue before note 133 | s.beam = 0 # 1 = beamed 134 | s.grace = 0 # 1 = grace note 135 | s.before = [] # abc string that goes before the note/chord 136 | s.after = '' # the same after the note/chord 137 | s.ns = n and [n] or [] # notes in the chord 138 | s.lyrs = {} # {number -> syllabe} 139 | s.tab = None # (string number, fret number) 140 | s.ntdec = '' # !string!, !courtesy! 141 | 142 | class Elem: 143 | def __init__ (s, string): 144 | s.tijd = 0 # the time in XML division units 145 | s.str = string # any abc string that is not a note 146 | 147 | class Counter: 148 | def inc (s, key, voice): s.counters [key][voice] = s.counters [key].get (voice, 0) + 1 149 | def clear (s, vnums): # reset all counters 150 | tups = list( zip (vnums.keys (), len (vnums) * [0])) 151 | s.counters = {'note': dict (tups), 'nopr': dict (tups), 'nopt': dict (tups)} 152 | def getv (s, key, voice): return s.counters[key][voice] 153 | def prcnt (s, ip): # print summary of all non zero counters 154 | for iv in s.counters ['note']: 155 | if s.getv ('nopr', iv) != 0: 156 | info ( 'part %d, voice %d has %d skipped non printable notes' % (ip, iv, s.getv ('nopr', iv))) 157 | if s.getv ('nopt', iv) != 0: 158 | info ( 'part %d, voice %d has %d notes without pitch' % (ip, iv, s.getv ('nopt', iv))) 159 | if s.getv ('note', iv) == 0: # no real notes counted in this voice 160 | info ( 'part %d, skipped empty voice %d' % (ip, iv)) 161 | 162 | class Music: 163 | def __init__(s, options): 164 | s.tijd = 0 # the current time 165 | s.maxtime = 0 # maximum time in a measure 166 | s.gMaten = [] # [voices,.. for all measures in a part] 167 | s.gLyrics = [] # [{num: (abc_lyric_string, melis)},.. for all measures in a part] 168 | s.vnums = {} # all used voice id's in a part (xml voice id's == numbers) 169 | s.cnt = Counter () # global counter object 170 | s.vceCnt = 1 # the global voice count over all parts 171 | s.lastnote = None # the last real note record inserted in s.voices 172 | s.bpl = options.b # the max number of bars per line when writing abc 173 | s.cpl = options.n # the number of chars per line when writing abc 174 | s.repbra = 0 # true if volta is used somewhere 175 | s.nvlt = options.v # no volta on higher voice numbers 176 | s.jscript = options.j # compatibility with javascript version 177 | 178 | def initVoices (s, newPart=0): 179 | s.vtimes, s.voices, s.lyrics = {}, {}, {} 180 | for v in s.vnums: 181 | s.vtimes [v] = 0 # {voice: the end time of the last item in each voice} 182 | s.voices [v] = [] # {voice: [Note|Elem, ..]} 183 | s.lyrics [v] = [] # {voice: [{num: syl}, ..]} 184 | if newPart: s.cnt.clear (s.vnums) # clear counters once per part 185 | 186 | def incTime (s, dt): 187 | s.tijd += dt 188 | if s.tijd < 0: s.tijd = 0 # erroneous element 189 | if s.tijd > s.maxtime: s.maxtime = s.tijd 190 | 191 | def appendElemCv (s, voices, elem): 192 | for v in voices: 193 | s.appendElem (v, elem) # insert element in all voices 194 | 195 | def insertElem (s, v, elem): # insert at the start of voice v in the current measure 196 | obj = Elem (elem) 197 | obj.tijd = 0 # because voice is sorted later 198 | s.voices [v].insert (0, obj) 199 | 200 | def appendObj (s, v, obj, dur): 201 | obj.tijd = s.tijd 202 | s.voices [v].append (obj) 203 | s.incTime (dur) 204 | if s.tijd > s.vtimes[v]: s.vtimes[v] = s.tijd # don't update for inserted earlier items 205 | 206 | def appendElem (s, v, elem, tel=0): 207 | s.appendObj (v, Elem (elem), 0) 208 | if tel: s.cnt.inc ('note', v) # count number of certain elements in each voice (in addition to notes) 209 | 210 | def appendElemT (s, v, elem, tijd): # insert element at specified time 211 | obj = Elem (elem) 212 | obj.tijd = tijd 213 | s.voices [v].append (obj) 214 | 215 | def appendNote (s, v, note, noot): 216 | note.ns.append (note.ntdec + noot) 217 | s.appendObj (v, note, int (note.dur)) 218 | s.lastnote = note # remember last note/rest for later modifications (chord, grace) 219 | if noot != 'z' and noot != 'x': # real notes and grace notes 220 | s.cnt.inc ('note', v) # count number of real notes in each voice 221 | if not note.grace: # for every real note 222 | s.lyrics[v].append (note.lyrs) # even when it has no lyrics 223 | 224 | def getLastRec (s, voice): 225 | if s.gMaten: return s.gMaten[-1][voice][-1] # the last record in the last measure 226 | return None # no previous records in the first measure 227 | 228 | def getLastMelis (s, voice, num): # get melisma of last measure 229 | if s.gLyrics: 230 | lyrdict = s.gLyrics[-1][voice] # the previous lyrics dict in this voice 231 | if num in lyrdict: return lyrdict[num][1] # lyrdict = num -> (lyric string, melisma) 232 | return 0 # no previous lyrics in voice or line number 233 | 234 | def addChord (s, note, noot): # careful: we assume that chord notes follow immediately 235 | for d in note.before: # put all decorations before chord 236 | if d not in s.lastnote.before: 237 | s.lastnote.before += [d] 238 | s.lastnote.ns.append (note.ntdec + noot) 239 | 240 | def addBar (s, lbrk, m): # linebreak, measure data 241 | if m.mdur and s.maxtime > m.mdur: info ('measure %d in part %d longer than metre' % (m.ixm+1, m.ixp+1)) 242 | s.tijd = s.maxtime # the time of the bar lines inserted here 243 | for v in s.vnums: 244 | if m.lline or m.lnum: # if left barline or left volta number 245 | p = s.getLastRec (v) # get the previous barline record 246 | if p: # in measure 1 no previous measure is available 247 | x = p.str # p.str is the ABC barline string 248 | if m.lline: # append begin of repeat, m.lline == ':' 249 | x = (x + m.lline).replace (':|:','::').replace ('||','|') 250 | if s.nvlt == 3: # add volta number only to lowest voice in part 0 251 | if m.ixp + v == min (s.vnums): x += m.lnum 252 | elif m.lnum: # new behaviour with I:repbra 0 253 | x += m.lnum # add volta number(s) or text to all voices 254 | s.repbra = 1 # signal occurrence of a volta 255 | p.str = x # modify previous right barline 256 | elif m.lline: # begin of new part and left repeat bar is required 257 | s.insertElem (v, '|:') 258 | if lbrk: 259 | p = s.getLastRec (v) # get the previous barline record 260 | if p: p.str += lbrk # insert linebreak char after the barlines+volta 261 | if m.attr: # insert signatures at front of buffer 262 | s.insertElem (v, '%s' % m.attr) 263 | s.appendElem (v, ' %s' % m.rline) # insert current barline record at time maxtime 264 | s.voices[v] = sortMeasure (s.voices[v], m) # make all times consistent 265 | lyrs = s.lyrics[v] # [{number: sylabe}, .. for all notes] 266 | lyrdict = {} # {number: (abc_lyric_string, melis)} for this voice 267 | nums = [num for d in lyrs for num in d.keys ()] # the lyrics numbers in this measure 268 | maxNums = max (nums + [0]) # the highest lyrics number in this measure 269 | for i in range (maxNums, 0, -1): 270 | xs = [syldict.get (i, '') for syldict in lyrs] # collect the syllabi with number i 271 | melis = s.getLastMelis (v, i) # get melisma from last measure 272 | lyrdict [i] = abcLyr (xs, melis) 273 | s.lyrics[v] = lyrdict # {number: (abc_lyric_string, melis)} for this measure 274 | mkBroken (s.voices[v]) 275 | s.gMaten.append (s.voices) 276 | s.gLyrics.append (s.lyrics) 277 | s.tijd = s.maxtime = 0 278 | s.initVoices () 279 | 280 | def outVoices (s, divs, ip, isSib): # output all voices of part ip 281 | vvmap = {} # xml voice number -> abc voice number (one part) 282 | vnum_keys = list (s.vnums.keys ()) 283 | if s.jscript or isSib: vnum_keys.sort () 284 | lvc = min (vnum_keys or [1]) # lowest xml voice number of this part 285 | for iv in vnum_keys: 286 | if s.cnt.getv ('note', iv) == 0: # no real notes counted in this voice 287 | continue # skip empty voices 288 | if abcOut.denL: unitL = abcOut.denL # take the unit length from the -d option 289 | else: unitL = compUnitLength (iv, s.gMaten, divs) # compute the best unit length for this voice 290 | abcOut.cmpL.append (unitL) # remember for header output 291 | vn, vl = [], {} # for voice iv: collect all notes to vn and all lyric lines to vl 292 | for im in range (len (s.gMaten)): 293 | measure = s.gMaten [im][iv] 294 | vn.append (outVoice (measure, divs [im], im, ip, unitL)) 295 | checkMelismas (s.gLyrics, s.gMaten, im, iv) 296 | for n, (lyrstr, melis) in s.gLyrics [im][iv].items (): 297 | if n in vl: 298 | while len (vl[n]) < im: vl[n].append ('') # fill in skipped measures 299 | vl[n].append (lyrstr) 300 | else: 301 | vl[n] = im * [''] + [lyrstr] # must skip im measures 302 | for n, lyrs in vl.items (): # fill up possibly empty lyric measures at the end 303 | mis = len (vn) - len (lyrs) 304 | lyrs += mis * [''] 305 | abcOut.add ('V:%d' % s.vceCnt) 306 | if s.repbra: 307 | if s.nvlt == 1 and s.vceCnt > 1: abcOut.add ('I:repbra 0') # only volta on first voice 308 | if s.nvlt == 2 and iv > lvc: abcOut.add ('I:repbra 0') # only volta on first voice of each part 309 | if s.cpl > 0: s.bpl = 0 # option -n (max chars per line) overrules -b (max bars per line) 310 | elif s.bpl == 0: s.cpl = 100 # the default: 100 chars per line 311 | bn = 0 # count bars 312 | while vn: # while still measures available 313 | ib = 1 314 | chunk = vn [0] 315 | while ib < len (vn): 316 | if s.cpl > 0 and len (chunk) + len (vn [ib]) >= s.cpl: break # line full (number of chars) 317 | if s.bpl > 0 and ib >= s.bpl: break # line full (number of bars) 318 | chunk += vn [ib] 319 | ib += 1 320 | bn += ib 321 | abcOut.add (chunk + ' %%%d' % bn) # line with barnumer 322 | del vn[:ib] # chop ib bars 323 | lyrlines = sorted (vl.items ()) # order the numbered lyric lines for output 324 | for n, lyrs in lyrlines: 325 | abcOut.add ('w: ' + '|'.join (lyrs[:ib]) + '|') 326 | del lyrs[:ib] 327 | vvmap [iv] = s.vceCnt # xml voice number -> abc voice number 328 | s.vceCnt += 1 # count voices over all parts 329 | s.gMaten = [] # reset the follwing instance vars for each part 330 | s.gLyrics = [] 331 | s.cnt.prcnt (ip+1) # print summary of skipped items in this part 332 | return vvmap 333 | 334 | class ABCoutput: 335 | pagekeys = 'scale,pageheight,pagewidth,leftmargin,rightmargin,topmargin,botmargin'.split (',') 336 | def __init__ (s, fnmext, pad, X, options): 337 | s.fnmext = fnmext 338 | s.outlist = [] # list of ABC strings 339 | s.title = 'T:Title' 340 | s.key = 'none' 341 | s.clefs = {} # clefs for all abc-voices 342 | s.mtr = 'none' 343 | s.tempo = 0 # 0 -> no tempo field 344 | s.tempo_units = (1,4) # note type of tempo direction 345 | s.pad = pad # the output path or none 346 | s.X = X + 1 # the abc tune number 347 | s.denL = options.d # denominator of the unit length (L:) from -d option 348 | s.volpan = int (options.m) # 0 -> no %%MIDI, 1 -> only program, 2 -> all %%MIDI 349 | s.cmpL = [] # computed optimal unit length for all voices 350 | s.jscript = options.j # compatibility with javascript version 351 | s.tstep = options.t # translate percmap to voicemap 352 | s.stemless = 0 # use U:s=!stemless! 353 | s.shiftStem = options.s # shift note heads 3 units left 354 | if pad: 355 | _, base_name = os.path.split (fnmext) 356 | s.outfile = open (os.path.join (pad, base_name), 'w') 357 | else: s.outfile = sys.stdout 358 | if s.jscript: s.X = 1 # always X:1 in javascript version 359 | s.pageFmt = {} 360 | for k in s.pagekeys: s.pageFmt [k] = None 361 | if len (options.p) == 7: 362 | for k, v in zip (s.pagekeys, options.p): 363 | try: s.pageFmt [k] = float (v) 364 | except: info ('illegal float %s for %s', (k, v)); continue 365 | 366 | def add (s, str): 367 | s.outlist.append (str + '\n') # collect all ABC output 368 | 369 | def mkHeader (s, stfmap, partlist, midimap, vmpdct, koppen): # stfmap = [parts], part = [staves], stave = [voices] 370 | accVce, accStf, staffs = [], [], stfmap[:] # staffs is consumed 371 | for x in partlist: # collect partnames into accVce and staff groups into accStf 372 | try: prgroupelem (x, ('', ''), '', stfmap, accVce, accStf) 373 | except: info ('lousy musicxml: error in part-list') 374 | staves = ' '.join (accStf) 375 | clfnms = {} 376 | for part, (partname, partabbrv) in zip (staffs, accVce): 377 | if not part: continue # skip empty part 378 | firstVoice = part[0][0] # the first voice number in this part 379 | nm = partname.replace ('\n','\\n').replace ('.:','.').strip (':') 380 | snm = partabbrv.replace ('\n','\\n').replace ('.:','.').strip (':') 381 | clfnms [firstVoice] = (nm and 'nm="%s"' % nm or '') + (snm and ' snm="%s"' % snm or '') 382 | hd = ['X:%d\n%s\n' % (s.X, s.title)] 383 | for i, k in enumerate (s.pagekeys): 384 | if s.jscript and k in ['pageheight','topmargin', 'botmargin']: continue 385 | if s.pageFmt [k] != None: hd.append ('%%%%%s %.2f%s\n' % (k, s.pageFmt [k], i > 0 and 'cm' or '')) 386 | if staves and len (accStf) > 1: hd.append ('%%score ' + staves + '\n') 387 | tempo = s.tempo and 'Q:%d/%d=%s\n' % (s.tempo_units [0], s.tempo_units [1], s.tempo) or '' # default no tempo field 388 | d = {} # determine the most frequently occurring unit length over all voices 389 | for x in s.cmpL: d[x] = d.get (x, 0) + 1 390 | if s.jscript: defLs = sorted (d.items (), key=lambda x: (-x[1], x[0])) # when tie (1) sort on key (0) 391 | else: defLs = sorted (d.items (), key=lambda x: -x[1]) 392 | defL = s.denL and s.denL or defLs [0][0] # override default unit length with -d option 393 | hd.append ('L:1/%d\n%sM:%s\n' % (defL, tempo, s.mtr)) 394 | hd.append ('K:%s\n' % s.key) 395 | if s.stemless: hd.append ('U:s=!stemless!\n') 396 | vxs = sorted (vmpdct.keys ()) 397 | for vx in vxs: hd.extend (vmpdct [vx]) 398 | s.dojef = 0 # translate percmap to voicemap 399 | for vnum, clef in s.clefs.items (): 400 | ch, prg, vol, pan = midimap [vnum-1][:4] 401 | dmap = midimap [vnum - 1][4:] # map of abc percussion notes to midi notes 402 | if dmap and 'perc' not in clef: clef = (clef + ' map=perc').strip (); 403 | hd.append ('V:%d %s %s\n' % (vnum, clef, clfnms.get (vnum, ''))) 404 | if vnum in vmpdct: 405 | hd.append ('%%%%voicemap tab%d\n' % vnum) 406 | hd.append ('K:none\nM:none\n%%clef none\n%%staffscale 1.6\n%%flatbeams true\n%%stemdir down\n') 407 | if 'perc' in clef: hd.append ('K:none\n'); # no key for a perc voice 408 | if s.volpan > 1: # option -m 2 -> output all recognized midi commands when needed and present in xml 409 | if ch > 0 and ch != vnum: hd.append ('%%%%MIDI channel %d\n' % ch) 410 | if prg > 0: hd.append ('%%%%MIDI program %d\n' % (prg - 1)) 411 | if vol >= 0: hd.append ('%%%%MIDI control 7 %.0f\n' % vol) # volume == 0 is possible ... 412 | if pan >= 0: hd.append ('%%%%MIDI control 10 %.0f\n' % pan) 413 | elif s.volpan > 0: # default -> only output midi program command when present in xml 414 | if dmap and ch > 0: hd.append ('%%%%MIDI channel %d\n' % ch) # also channel if percussion part 415 | if prg > 0: hd.append ('%%%%MIDI program %d\n' % (prg - 1)) 416 | for abcNote, step, midiNote, notehead in dmap: 417 | if not notehead: notehead = 'normal' 418 | if abcMid (abcNote) != midiNote or abcNote != step: 419 | if s.volpan > 0: hd.append ('%%%%MIDI drummap %s %s\n' % (abcNote, midiNote)) 420 | hd.append ('I:percmap %s %s %s %s\n' % (abcNote, step, midiNote, notehead)) 421 | s.dojef = s.tstep 422 | if defL != s.cmpL [vnum-1]: # only if computed unit length different from header 423 | hd.append ('L:1/%d\n' % s.cmpL [vnum-1]) 424 | s.outlist = hd + s.outlist 425 | if koppen: # output SVG stuff needed for tablature 426 | k1 = kopSvg.replace ('-2','-5') if s.shiftStem else kopSvg # shift note heads 3 units left 427 | k2 = kopSvg2.replace ('-2','-5') if s.shiftStem else kopSvg2 428 | tb = tabSvg.replace ('-3','-6') if s.shiftStem else tabSvg 429 | ks = sorted (koppen.keys ()) # javascript compatibility 430 | ks = [k2 % (k, k) if len (k) == 2 else k1 % (k, k) for k in ks] 431 | tbs = map (lambda x: x.strip () + '\n', tb.splitlines ()) # javascript compatibility 432 | s.outlist = tbs + ks + ['\n%%endsvg\n'] + s.outlist 433 | 434 | def writeall (s): # determine the required encoding of the entire ABC output 435 | str = ''.join (s.outlist) 436 | if s.dojef: str = perc2map (str) 437 | if python3: s.outfile.write (str) 438 | else: s.outfile.write (str.encode ('utf-8')) 439 | if s.pad: s.outfile.close () # close each file with -o option 440 | else: s.outfile.write ('\n') # add empty line between tunes on stdout 441 | info ('%s written with %d voices' % (s.fnmext, len (s.clefs)), warn=0) 442 | 443 | #---------------- 444 | # functions 445 | #---------------- 446 | def abcLyr (xs, melis): # Convert list xs to abc lyrics. 447 | if not ''.join (xs): return '', 0 # there is no lyrics in this measure 448 | res = [] 449 | for x in xs: # xs has for every note a lyrics syllabe or an empty string 450 | if x == '': # note without lyrics 451 | if melis: x = '_' # set melisma 452 | else: x = '*' # skip note 453 | elif x.endswith ('_') and not x.endswith ('\_'): # start of new melisma 454 | x = x.replace ('_', '') # remove and set melis boolean 455 | melis = 1 # so next skips will become melisma 456 | else: melis = 0 # melisma stops on first syllable 457 | res.append (x) 458 | return (' '.join (res), melis) 459 | 460 | def simplify (a, b): # divide a and b by their greatest common divisor 461 | x, y = a, b 462 | while b: a, b = b, a % b 463 | return x // a, y // a 464 | 465 | def abcdur (nx, divs, uL): # convert an musicXML duration d to abc units with L:1/uL 466 | if nx.dur == 0: return '' # when called for elements without duration 467 | num, den = simplify (uL * nx.dur, divs * 4) # L=1/8 -> uL = 8 units 468 | if nx.fact: # apply tuplet time modification 469 | numfac, denfac = nx.fact 470 | num, den = simplify (num * numfac, den * denfac) 471 | if den > 64: # limit the denominator to a maximum of 64 472 | x = float (num) / den; n = math.floor (x); # when just above an integer n 473 | if x - n < 0.1 * x: num, den = n, 1; # round to n 474 | num64 = 64. * num / den + 1.0e-15 # to get Python2 behaviour of round 475 | num, den = simplify (int (round (num64)), 64) 476 | if num == 1: 477 | if den == 1: dabc = '' 478 | elif den == 2: dabc = '/' 479 | else: dabc = '/%d' % den 480 | elif den == 1: dabc = '%d' % num 481 | else: dabc = '%d/%d' % (num, den) 482 | return dabc 483 | 484 | def abcMid (note): # abc note -> midi pitch 485 | r = re.search (r"([_^]*)([A-Ga-g])([',]*)", note) 486 | if not r: return -1 487 | acc, n, oct = r.groups () 488 | nUp = n.upper () 489 | p = 60 + [0,2,4,5,7,9,11]['CDEFGAB'.index (nUp)] + (12 if nUp != n else 0); 490 | if acc: p += (1 if acc[0] == '^' else -1) * len (acc) 491 | if oct: p += (12 if oct[0] == "'" else -12) * len (oct) 492 | return p 493 | 494 | def staffStep (ptc, o, clef, tstep): 495 | ndif = 0 496 | if 'stafflines=1' in clef: ndif += 4 # meaning of one line: E (xml) -> B (abc) 497 | if not tstep and clef.startswith ('bass'): ndif += 12 # transpose bass -> treble (C3 -> A4) 498 | if ndif: # diatonic transposition == addition modulo 7 499 | nm7 = 'C,D,E,F,G,A,B'.split (',') 500 | n = nm7.index (ptc) + ndif 501 | ptc, o = nm7 [n % 7], o + n // 7 502 | if o > 4: ptc = ptc.lower () 503 | if o > 5: ptc = ptc + (o-5) * "'" 504 | if o < 4: ptc = ptc + (4-o) * "," 505 | return ptc 506 | 507 | def setKey (fifths, mode): 508 | sharpness = ['Fb', 'Cb','Gb','Db','Ab','Eb','Bb','F','C','G','D','A', 'E', 'B', 'F#','C#','G#','D#','A#','E#','B#'] 509 | offTab = {'maj':8, 'ion':8, 'm':11, 'min':11, 'aeo':11, 'mix':9, 'dor':10, 'phr':12, 'lyd':7, 'loc':13, 'non':8} 510 | mode = mode.lower ()[:3] # only first three chars, no case 511 | key = sharpness [offTab [mode] + fifths] + (mode if offTab [mode] != 8 else '') 512 | accs = ['F','C','G','D','A','E','B'] 513 | if fifths >= 0: msralts = dict (zip (accs[:fifths], fifths * [1])) 514 | else: msralts = dict (zip (accs[fifths:], -fifths * [-1])) 515 | return key, msralts 516 | 517 | def insTup (ix, notes, fact): # read one nested tuplet 518 | tupcnt = 0 519 | nx = notes [ix] 520 | if 'start' in nx.tup: 521 | nx.tup.remove ('start') # do recursive calls when starts remain 522 | tix = ix # index of first tuplet note 523 | fn, fd = fact # xml time-mod of the higher level 524 | fnum, fden = nx.fact # xml time-mod of the current level 525 | tupfact = fnum//fn, fden//fd # abc time mod of this level 526 | while ix < len (notes): 527 | nx = notes [ix] 528 | if isinstance (nx, Elem) or nx.grace: 529 | ix += 1 # skip all non tuplet elements 530 | continue 531 | if 'start' in nx.tup: # more nested tuplets to start 532 | ix, tupcntR = insTup (ix, notes, tupfact) # ix is on the stop note! 533 | tupcnt += tupcntR 534 | elif nx.fact: 535 | tupcnt += 1 # count tuplet elements 536 | if 'stop' in nx.tup: 537 | nx.tup.remove ('stop') 538 | break 539 | if not nx.fact: # stop on first non tuplet note 540 | ix = lastix # back to last tuplet note 541 | break 542 | lastix = ix 543 | ix += 1 544 | # put abc tuplet notation before the recursive ones 545 | tup = (tupfact[0], tupfact[1], tupcnt) 546 | if tup == (3, 2, 3): tupPrefix = '(3' 547 | else: tupPrefix = '(%d:%d:%d' % tup 548 | notes [tix].tupabc = tupPrefix + notes [tix].tupabc 549 | return ix, tupcnt # ix is on the last tuplet note 550 | 551 | def mkBroken (vs): # introduce broken rhythms (vs: one voice, one measure) 552 | vs = [n for n in vs if isinstance (n, Note)] 553 | i = 0 554 | while i < len (vs) - 1: 555 | n1, n2 = vs[i], vs[i+1] # scan all adjacent pairs 556 | # skip if note in tuplet or has no duration or outside beam 557 | if not n1.fact and not n2.fact and n1.dur > 0 and n2.beam: 558 | if n1.dur * 3 == n2.dur: 559 | n2.dur = (2 * n2.dur) // 3 560 | n1.dur = n1.dur * 2 561 | n1.after = '<' + n1.after 562 | i += 1 # do not chain broken rhythms 563 | elif n2.dur * 3 == n1.dur: 564 | n1.dur = (2 * n1.dur) // 3 565 | n2.dur = n2.dur * 2 566 | n1.after = '>' + n1.after 567 | i += 1 # do not chain broken rhythms 568 | i += 1 569 | 570 | def outVoice (measure, divs, im, ip, unitL): # note/elem objects of one measure in one voice 571 | ix = 0 572 | while ix < len (measure): # set all (nested) tuplet annotations 573 | nx = measure [ix] 574 | if isinstance (nx, Note) and nx.fact and not nx.grace: 575 | ix, tupcnt = insTup (ix, measure, (1, 1)) # read one tuplet, insert annotation(s) 576 | ix += 1 577 | vs = [] 578 | for nx in measure: 579 | if isinstance (nx, Note): 580 | durstr = abcdur (nx, divs, unitL) # xml -> abc duration string 581 | chord = len (nx.ns) > 1 582 | cns = [nt[:-1] for nt in nx.ns if nt.endswith ('-')] 583 | tie = '' 584 | if chord and len (cns) == len (nx.ns): # all chord notes tied 585 | nx.ns = cns # chord notes without tie 586 | tie = '-' # one tie for whole chord 587 | s = nx.tupabc + ''.join (nx.before) 588 | if chord: s += '[' 589 | for nt in nx.ns: s += nt 590 | if chord: s += ']' + tie 591 | if s.endswith ('-'): s, tie = s[:-1], '-' # split off tie 592 | s += durstr + tie # and put it back again 593 | s += nx.after 594 | nospace = nx.beam 595 | else: 596 | if isinstance (nx.str, listtype): nx.str = nx.str [0] 597 | s = nx.str 598 | nospace = 1 599 | if nospace: vs.append (s) 600 | else: vs.append (' ' + s) 601 | vs = ''.join (vs) # ad hoc: remove multiple pedal directions 602 | while vs.find ('!ped!!ped!') >= 0: vs = vs.replace ('!ped!!ped!','!ped!') 603 | while vs.find ('!ped-up!!ped-up!') >= 0: vs = vs.replace ('!ped-up!!ped-up!','!ped-up!') 604 | while vs.find ('!8va(!!8va)!') >= 0: vs = vs.replace ('!8va(!!8va)!','') # remove empty ottava's 605 | return vs 606 | 607 | def sortMeasure (voice, m): 608 | voice.sort (key=lambda o: o.tijd) # sort on time 609 | time = 0 610 | v = [] 611 | rs = [] # holds rests in between notes 612 | for i, nx in enumerate (voice): # establish sequentiality 613 | if nx.tijd > time and chkbug (nx.tijd - time, m): 614 | v.append (Note (nx.tijd - time, 'x')) # fill hole with invisble rest 615 | rs.append (len (v) - 1) 616 | if isinstance (nx, Elem): 617 | if nx.tijd < time: nx.tijd = time # shift elems without duration to where they fit 618 | v.append (nx) 619 | time = nx.tijd 620 | continue 621 | if nx.tijd < time: # overlapping element 622 | if nx.ns[0] == 'z': continue # discard overlapping rest 623 | if v[-1].tijd <= nx.tijd: # we can do something 624 | if v[-1].ns[0] == 'z': # shorten rest 625 | v[-1].dur = nx.tijd - v[-1].tijd 626 | if v[-1].dur == 0: del v[-1] # nothing left 627 | info ('overlap in part %d, measure %d: rest shortened' % (m.ixp+1, m.ixm+1)) 628 | else: # make a chord of overlap 629 | v[-1].ns += nx.ns 630 | info ('overlap in part %d, measure %d: added chord' % (m.ixp+1, m.ixm+1)) 631 | nx.dur = (nx.tijd + nx.dur) - time # the remains 632 | if nx.dur <= 0: continue # nothing left 633 | nx.tijd = time # append remains 634 | else: # give up 635 | info ('overlapping notes in one voice! part %d, measure %d, note %s discarded' % (m.ixp+1, m.ixm+1, isinstance (nx, Note) and nx.ns or nx.str)) 636 | continue 637 | v.append (nx) 638 | if isinstance (nx, Note): 639 | if nx.ns [0] in 'zx': 640 | rs.append (len (v) - 1) # remember rests between notes 641 | elif len (rs): 642 | if nx.beam and not nx.grace: # copy beam into rests 643 | for j in rs: v[j].beam = nx.beam 644 | rs = [] # clear rests on each note 645 | time = nx.tijd + nx.dur 646 | # when a measure contains no elements and no forwards -> no incTime -> s.maxtime = 0 -> right barline 647 | # is inserted at time == 0 (in addbar) and is only element in the voice when sortMeasure is called 648 | if time == 0: info ('empty measure in part %d, measure %d, it should contain at least a rest to advance the time!' % (m.ixp+1, m.ixm+1)) 649 | return v 650 | 651 | def getPartlist (ps): # correct part-list (from buggy xml-software) 652 | xs = [] # the corrected part-list 653 | e = [] # stack of opened part-groups 654 | for x in list (ps): # insert missing stops, delete double starts 655 | if x.tag == 'part-group': 656 | num, type = x.get ('number'), x.get ('type') 657 | if type == 'start': 658 | if num in e: # missing stop: insert one 659 | xs.append (E.Element ('part-group', number = num, type = 'stop')) 660 | xs.append (x) 661 | else: # normal start 662 | xs.append (x) 663 | e.append (num) 664 | else: 665 | if num in e: # normal stop 666 | e.remove (num) 667 | xs.append (x) 668 | else: pass # double stop: skip it 669 | else: xs.append (x) 670 | for num in reversed (e): # fill missing stops at the end 671 | xs.append (E.Element ('part-group', number = num, type = 'stop')) 672 | return xs 673 | 674 | def parseParts (xs, d, e): # -> [elems on current level], rest of xs 675 | if not xs: return [],[] 676 | x = xs.pop (0) 677 | if x.tag == 'part-group': 678 | num, type = x.get ('number'), x.get ('type') 679 | if type == 'start': # go one level deeper 680 | s = [x.findtext (n, '') for n in ['group-symbol','group-barline','group-name','group-abbreviation']] 681 | d [num] = s # remember groupdata by group number 682 | e.append (num) # make stack of open group numbers 683 | elemsnext, rest1 = parseParts (xs, d, e) # parse one level deeper to next stop 684 | elems, rest2 = parseParts (rest1, d, e) # parse the rest on this level 685 | return [elemsnext] + elems, rest2 686 | else: # stop: close level and return group-data 687 | nums = e.pop () # last open group number in stack order 688 | if xs and xs[0].get ('type') == 'stop': # two consequetive stops 689 | if num != nums: # in the wrong order (tempory solution) 690 | d[nums], d[num] = d[num], d[nums] # exchange values (only works for two stops!!!) 691 | sym = d[num] # retrieve an return groupdata as last element of the group 692 | return [sym], xs 693 | else: 694 | elems, rest = parseParts (xs, d, e) # parse remaining elements on current level 695 | name = x.findtext ('part-name',''), x.findtext ('part-abbreviation','') 696 | return [name] + elems, rest 697 | 698 | def bracePart (part): # put a brace on multistaff part and group voices 699 | if not part: return [] # empty part in the score 700 | brace = [] 701 | for ivs in part: 702 | if len (ivs) == 1: # stave with one voice 703 | brace.append ('%s' % ivs[0]) 704 | else: # stave with multiple voices 705 | brace += ['('] + ['%s' % iv for iv in ivs] + [')'] 706 | brace.append ('|') 707 | del brace[-1] # no barline at the end 708 | if len (part) > 1: 709 | brace = ['{'] + brace + ['}'] 710 | return brace 711 | 712 | def prgroupelem (x, gnm, bar, pmap, accVce, accStf): # collect partnames (accVce) and %%score map (accStf) 713 | if type (x) == tupletype: # partname-tuple = (part-name, part-abbrev) 714 | y = pmap.pop (0) 715 | if gnm[0]: x = [n1 + ':' + n2 for n1, n2 in zip (gnm, x)] # put group-name before part-name 716 | accVce.append (x) 717 | accStf.extend (bracePart (y)) 718 | elif len (x) == 2 and type (x[0]) == tupletype: # misuse of group just to add extra name to stave 719 | y = pmap.pop (0) 720 | nms = [n1 + ':' + n2 for n1, n2 in zip (x[0], x[1][2:])] # x[0] = partname-tuple, x[1][2:] = groupname-tuple 721 | accVce.append (nms) 722 | accStf.extend (bracePart (y)) 723 | else: 724 | prgrouplist (x, bar, pmap, accVce, accStf) 725 | 726 | def prgrouplist (x, pbar, pmap, accVce, accStf): # collect partnames, scoremap for a part-group 727 | sym, bar, gnm, gabbr = x[-1] # bracket symbol, continue barline, group-name-tuple 728 | bar = bar == 'yes' or pbar # pbar -> the parent has bar 729 | accStf.append (sym == 'brace' and '{' or '[') 730 | for z in x[:-1]: 731 | prgroupelem (z, (gnm, gabbr), bar, pmap, accVce, accStf) 732 | if bar: accStf.append ('|') 733 | if bar: del accStf [-1] # remove last one before close 734 | accStf.append (sym == 'brace' and '}' or ']') 735 | 736 | def compUnitLength (iv, maten, divs): # compute optimal unit length 737 | uLmin, minLen = 0, max_int 738 | for uL in [4,8,16]: # try 1/4, 1/8 and 1/16 739 | vLen = 0 # total length of abc duration strings in this voice 740 | for im, m in enumerate (maten): # all measures 741 | for e in m[iv]: # all notes in voice iv 742 | if isinstance (e, Elem) or e.dur == 0: continue # no real durations 743 | vLen += len (abcdur (e, divs [im], uL)) # add len of duration string 744 | if vLen < minLen: uLmin, minLen = uL, vLen # remember the smallest 745 | return uLmin 746 | 747 | def doSyllable (syl): 748 | txt = '' 749 | for e in syl: 750 | if e.tag == 'elision': txt += '~' 751 | elif e.tag == 'text': # escape - and space characters 752 | txt += (e.text or '').replace ('_','\_').replace('-', r'\-').replace(' ', '~') 753 | if not txt: return txt 754 | if syl.findtext('syllabic') in ['begin', 'middle']: txt += '-' 755 | if syl.find('extend') is not None: txt += '_' 756 | return txt 757 | 758 | def checkMelismas (lyrics, maten, im, iv): 759 | if im == 0: return 760 | maat = maten [im][iv] # notes of the current measure 761 | curlyr = lyrics [im][iv] # lyrics dict of current measure 762 | prvlyr = lyrics [im-1][iv] # lyrics dict of previous measure 763 | for n, (lyrstr, melis) in prvlyr.items (): # all lyric numbers in the previous measure 764 | if n not in curlyr and melis: # melisma required, but no lyrics present -> make one! 765 | ms = getMelisma (maat) # get a melisma for the current measure 766 | if ms: curlyr [n] = (ms, 0) # set melisma as the n-th lyrics of the current measure 767 | 768 | def getMelisma (maat): # get melisma from notes in maat 769 | ms = [] 770 | for note in maat: # every note should get an underscore 771 | if not isinstance (note, Note): continue # skip Elem's 772 | if note.grace: continue # skip grace notes 773 | if note.ns [0] in 'zx': break # stop on first rest 774 | ms.append ('_') 775 | return ' '.join (ms) 776 | 777 | def perc2map (abcIn): 778 | fillmap = {'diamond':1, 'triangle':1, 'square':1, 'normal':1}; 779 | abc = map (lambda x: x.strip (), percSvg.splitlines ()) 780 | id='default' 781 | maps = {'default': []}; 782 | dmaps = {'default': []} 783 | r1 = re.compile (r'V:\s*(\S+)') 784 | ls = abcIn.splitlines () 785 | for x in ls: 786 | if 'I:percmap' in x: 787 | noot, step, midi, kop = map (lambda x: x.strip (), x.split ()[1:]) 788 | if kop in fillmap: kop = kop + '+' + ',' + kop 789 | x = '%%%%map perc%s %s print=%s midi=%s heads=%s' % (id, noot, step, midi, kop) 790 | maps [id].append (x) 791 | if '%%MIDI' in x: dmaps [id].append (x) 792 | if 'V:' in x: 793 | r = r1.match (x) 794 | if r: 795 | id = r.group (1); 796 | if id not in maps: maps [id] = []; dmaps [id] = [] 797 | ids = sorted (maps.keys ()) 798 | for id in ids: abc += maps [id] 799 | id='default' 800 | for x in ls: 801 | if 'I:percmap' in x: continue 802 | if '%%MIDI' in x: continue 803 | if 'V:' in x or 'K:' in x: 804 | r = r1.match (x) 805 | if r: id = r.group (1) 806 | abc.append (x) 807 | if id in dmaps and len (dmaps [id]) > 0: abc.extend (dmaps [id]); del dmaps [id] 808 | if 'perc' in x and 'map=' not in x: x += ' map=perc'; 809 | if 'map=perc' in x and len (maps [id]) > 0: abc.append ('%%voicemap perc' + id); 810 | if 'map=off' in x: abc.append ('%%voicemap'); 811 | else: 812 | abc.append (x) 813 | return '\n'.join (abc) + '\n' 814 | 815 | def addoct (ptc, o): # xml staff step, xml octave number 816 | p = ptc 817 | if o > 4: p = ptc.lower () 818 | if o > 5: p = p + (o-5) * "'" 819 | if o < 4: p = p + (4-o) * "," 820 | return p # abc pitch == abc note without accidental 821 | 822 | def chkbug (dt, m): 823 | if dt > m.divs / 16: return 1 # duration should be > 1/64 note 824 | info ('MuseScore bug: incorrect duration, smaller then 1/64! in measure %d, part %d' % (m.ixm, m.ixp)) 825 | return 0 826 | 827 | #---------------- 828 | # parser 829 | #---------------- 830 | class Parser: 831 | note_alts = [ # 3 alternative notations of the same note for tablature mapping 832 | [x.strip () for x in '=C, ^C, =D, ^D, =E, =F, ^F, =G, ^G, =A, ^A, =B'.split (',')], 833 | [x.strip () for x in '^B, _D,^^C, _E, _F, ^E, _G,^^F, _A,^^G, _B, _C'.split (',')], 834 | [x.strip () for x in '__D,^^B,__E,__F,^^D,__G,^^E,__A,_/A,__B,__C,^^A'.split (',')] ] 835 | step_map = {'C':0,'D':2,'E':4,'F':5,'G':7,'A':9,'B':11} 836 | def __init__ (s, options): 837 | # unfold repeats, number of chars per line, credit filter level, volta option 838 | s.slurBuf = {} # dict of open slurs keyed by slur number 839 | s.dirStk = {} # {direction-type + number -> (type, voice | time)} dict for proper closing 840 | s.ingrace = 0 # marks a sequence of grace notes 841 | s.msc = Music (options) # global music data abstraction 842 | s.unfold = options.u # turn unfolding repeats on 843 | s.ctf = options.c # credit text filter level 844 | s.gStfMap = [] # [[abc voice numbers] for all parts] 845 | s.midiMap = [] # midi-settings for each abc voice, in order 846 | s.drumInst = {} # inst_id -> midi pitch for channel 10 notes 847 | s.drumNotes = {} # (xml voice, abc note) -> (midi note, note head) 848 | s.instMid = [] # [{inst id -> midi-settings} for all parts] 849 | s.midDflt = [-1,-1,-1,-91] # default midi settings for channel, program, volume, panning 850 | s.msralts = {} # xml-notenames (without octave) with accidentals from the key 851 | s.curalts = {} # abc-notenames (with voice number) with passing accidentals 852 | s.stfMap = {} # xml staff number -> [xml voice number] 853 | s.vce2stf = {} # xml voice number -> allocated staff number 854 | s.clefMap = {} # xml staff number -> abc clef (for header only) 855 | s.curClef = {} # xml staff number -> current abc clef 856 | s.stemDir = {} # xml voice number -> current stem direction 857 | s.clefOct = {} # xml staff number -> current clef-octave-change 858 | s.curStf = {} # xml voice number -> current xml staff number 859 | s.nolbrk = options.x; # generate no linebreaks ($) 860 | s.jscript = options.j # compatibility with javascript version 861 | s.ornaments = sorted (note_ornamentation_map.items ()) 862 | s.doPageFmt = len (options.p) == 1 # translate xml page format 863 | s.tstep = options.t # clef determines step on staff (percussion) 864 | s.dirtov1 = options.v1 # all directions to first voice of staff 865 | s.ped = options.ped # render pedal directions 866 | s.wstems = options.stm # translate stem elements 867 | s.pedVce = None # voice for pedal directions 868 | s.repeat_str = {} # staff number -> [measure number, repeat-text] 869 | s.tabVceMap = {} # abc voice num -> [%%map ...] for tab voices 870 | s.koppen = {} # noteheads needed for %%map 871 | 872 | def matchSlur (s, type2, n, v2, note2, grace, stopgrace): # match slur number n in voice v2, add abc code to before/after 873 | if type2 not in ['start', 'stop']: return # slur type continue has no abc equivalent 874 | if n == None: n = '1' 875 | if n in s.slurBuf: 876 | type1, v1, note1, grace1 = s.slurBuf [n] 877 | if type2 != type1: # slur complete, now check the voice 878 | if v2 == v1: # begins and ends in the same voice: keep it 879 | if type1 == 'start' and (not grace1 or not stopgrace): # normal slur: start before stop and no grace slur 880 | note1.before = ['('] + note1.before # keep left-right order! 881 | note2.after += ')' 882 | # no else: don't bother with reversed stave spanning slurs 883 | del s.slurBuf [n] # slur finished, remove from stack 884 | else: # double definition, keep the last 885 | info ('double slur numbers %s-%s in part %d, measure %d, voice %d note %s, first discarded' % (type2, n, s.msr.ixp+1, s.msr.ixm+1, v2, note2.ns)) 886 | s.slurBuf [n] = (type2, v2, note2, grace) 887 | else: # unmatched slur, put in dict 888 | s.slurBuf [n] = (type2, v2, note2, grace) 889 | 890 | def doNotations (s, note, nttn, isTab): 891 | for key, val in s.ornaments: 892 | if nttn.find (key) != None: note.before += [val] # just concat all ornaments 893 | trem = nttn.find ('ornaments/tremolo') 894 | if trem != None: 895 | type = trem.get ('type') 896 | if type == 'single': 897 | note.before.insert (0, '!%s!' % (int (trem.text) * '/')) 898 | else: 899 | note.fact = None # no time modification in ABC 900 | if s.tstep: # abc2svg version 901 | if type == 'stop': note.before.insert (0, '!trem%s!' % trem.text); 902 | else: # abc2xml version 903 | if type == 'start': note.before.insert (0, '!%s-!' % (int (trem.text) * '/')); 904 | fingering = nttn.findall ('technical/fingering') 905 | for finger in fingering: # handle multiple finger annotations 906 | if not isTab: note.before += ['!%s!' % finger.text] # fingering goes before chord (addChord) 907 | snaar = nttn.find ('technical/string') 908 | if snaar != None and isTab: 909 | if s.tstep: 910 | fret = nttn.find ('technical/fret') 911 | if fret != None: note.tab = (snaar.text, fret.text) 912 | else: 913 | deco = '!%s!' % snaar.text # no double string decos (bug in musescore) 914 | if deco not in note.ntdec: note.ntdec += deco 915 | wvln = nttn.find ('ornaments/wavy-line') 916 | if wvln != None: 917 | if wvln.get ('type') == 'start': note.before = ['!trill(!'] + note.before # keep left-right order! 918 | elif wvln.get ('type') == 'stop': note.before = ['!trill)!'] + note.before 919 | glis = nttn.find ('glissando') 920 | if glis == None: glis = nttn.find ('slide') # treat slide as glissando 921 | if glis != None: 922 | lt = '~' if glis.get ('line-type') =='wavy' else '-' 923 | if glis.get ('type') == 'start': note.before = ['!%s(!' % lt] + note.before # keep left-right order! 924 | elif glis.get ('type') == 'stop': note.before = ['!%s)!' % lt] + note.before 925 | 926 | def tabnote (s, alt, ptc, oct, v, ntrec): 927 | p = s.step_map [ptc] + int (alt or '0') # p in -2 .. 13 928 | if p > 11: oct += 1 # octave correction 929 | if p < 0: oct -= 1 930 | p = p % 12 # remap p into 0..11 931 | snaar_nw, fret_nw = ntrec.tab # the computed/annotated allocation of nt 932 | for i in range (4): # support same note on 4 strings 933 | na = s.note_alts [i % 3] [p] # get alternative representation of same note 934 | o = oct 935 | if na in ['^B', '^^B']: o -= 1 # because in adjacent octave 936 | if na in ['_C', '__C']: o += 1 937 | if '/' in na or i == 3: o = 9 # emergency notation for 4th string case 938 | nt = addoct (na, o) 939 | snaar, fret = s.tabmap.get ((v, nt), ('', '')) # the current allocation of nt 940 | if not snaar: break # note not yet allocated 941 | if snaar_nw == snaar: return nt # use present allocation 942 | if i == 3: # new allocaion needed but none is free 943 | fmt = 'rejected: voice %d note %3s string %s fret %2s remains: string %s fret %s' 944 | info (fmt % (v, nt, snaar_nw, fret_nw, snaar, fret), 1) 945 | ntrec.tab = (snaar, fret) 946 | s.tabmap [v, nt] = ntrec.tab # for tablature map (voice, note) -> (string, fret) 947 | return nt # ABC code always in key C (with midi pitch alterations) 948 | 949 | def ntAbc (s, ptc, oct, note, v, ntrec, isTab): # pitch, octave -> abc notation 950 | acc2alt = {'double-flat':-2,'flat-flat':-2,'flat':-1,'natural':0,'sharp':1,'sharp-sharp':2,'double-sharp':2} 951 | oct += s.clefOct.get (s.curStf [v], 0) # minus clef-octave-change value 952 | acc = note.findtext ('accidental') # should be the notated accidental 953 | alt = note.findtext ('pitch/alter') # pitch alteration (midi) 954 | if ntrec.tab: return s.tabnote (alt, ptc, oct, v, ntrec) # implies s.tstep is true (options.t was given) 955 | elif isTab and s.tstep: 956 | nt = ['__','_','','^','^^'][int (alt or '0') + 2] + addoct (ptc, oct) 957 | info ('no string notation found for note %s in voice %d' % (nt, v), 1) 958 | p = addoct (ptc, oct) 959 | if alt == None and s.msralts.get (ptc, 0): alt = 0 # no alt but key implies alt -> natural!! 960 | if alt == None and (p, v) in s.curalts: alt = 0 # no alt but previous note had one -> natural!! 961 | if acc == None and alt == None: return p # no acc, no alt 962 | elif acc != None: 963 | alt = acc2alt [acc] # acc takes precedence over the pitch here! 964 | else: # now see if we really must add an accidental 965 | alt = int (float (alt)) 966 | if (p, v) in s.curalts: # the note in this voice has been altered before 967 | if alt == s.curalts [(p, v)]: return p # alteration still the same 968 | elif alt == s.msralts.get (ptc, 0): return p # alteration implied by the key 969 | tieElms = note.findall ('tie') + note.findall ('notations/tied') # in xml we have separate notated ties and playback ties 970 | if 'stop' in [e.get ('type') for e in tieElms]: return p # don't alter tied notes 971 | info ('accidental %d added in part %d, measure %d, voice %d note %s' % (alt, s.msr.ixp+1, s.msr.ixm+1, v+1, p)) 972 | s.curalts [(p, v)] = alt 973 | p = ['__','_','=','^','^^'][alt+2] + p # and finally ... prepend the accidental 974 | return p 975 | 976 | def doNote (s, n): # parse a musicXML note tag 977 | note = Note () 978 | v = int (n.findtext ('voice', '1')) 979 | if s.isSib: v += 100 * int (n.findtext ('staff', '1')) # repair bug in Sibelius 980 | chord = n.find ('chord') != None 981 | p = n.findtext ('pitch/step') or n.findtext ('unpitched/display-step') 982 | o = n.findtext ('pitch/octave') or n.findtext ('unpitched/display-octave') 983 | r = n.find ('rest') 984 | numer = n.findtext ('time-modification/actual-notes') 985 | if numer: 986 | denom = n.findtext ('time-modification/normal-notes') 987 | note.fact = (int (numer), int (denom)) 988 | note.tup = [x.get ('type') for x in n.findall ('notations/tuplet')] 989 | dur = n.findtext ('duration') 990 | grc = n.find ('grace') 991 | note.grace = grc != None 992 | note.before, note.after = [], '' # strings with ABC stuff that goes before or after a note/chord 993 | if note.grace and not s.ingrace: # open a grace sequence 994 | s.ingrace = 1 995 | note.before = ['{'] 996 | if grc.get ('slash') == 'yes': note.before += ['/'] # acciaccatura 997 | stopgrace = not note.grace and s.ingrace 998 | if stopgrace: # close the grace sequence 999 | s.ingrace = 0 1000 | s.msc.lastnote.after += '}' # close grace on lastenote.after 1001 | if dur == None or note.grace: dur = 0 1002 | if r == None and n.get ('print-object') == 'no': 1003 | if chord: return 1004 | r = 1 # turn invisible notes (that advance the time) into invisible rests 1005 | note.dur = int (dur) 1006 | if r == None and (not p or not o): # not a rest and no pitch 1007 | s.msc.cnt.inc ('nopt', v) # count unpitched notes 1008 | o, p = 5,'E' # make it an E5 ?? 1009 | isTab = s.curClef and s.curClef.get (s.curStf [v], '').startswith ('tab') 1010 | nttn = n.find ('notations') # add ornaments 1011 | if nttn != None: s.doNotations (note, nttn, isTab) 1012 | e = n.find ('stem') if r == None else None # no !stemless! before rest 1013 | if e != None and e.text == 'none' and (not isTab or v in s.hasStems or s.tstep): 1014 | note.before += ['s']; abcOut.stemless = 1; 1015 | e = n.find ('accidental') 1016 | if e != None and e.get ('parentheses') == 'yes': note.ntdec += '!courtesy!' 1017 | if r != None: noot = 'x' if n.get ('print-object') == 'no' or isTab else 'z' 1018 | else: noot = s.ntAbc (p, int (o), n, v, note, isTab) 1019 | if n.find ('unpitched') != None: 1020 | clef = s.curClef [s.curStf [v]] # the current clef for this voice 1021 | step = staffStep (p, int (o), clef, s.tstep) # (clef independent) step value of note on the staff 1022 | instr = n.find ('instrument') 1023 | instId = instr.get ('id') if instr != None else 'dummyId' 1024 | midi = s.drumInst.get (instId, abcMid (noot)) 1025 | nh = n.findtext ('notehead', '').replace (' ','-') # replace spaces in xml notehead names for percmap 1026 | if nh == 'x': noot = '^' + noot.replace ('^','').replace ('_','') 1027 | if nh in ['circle-x','diamond','triangle']: noot = '_' + noot.replace ('^','').replace ('_','') 1028 | if nh and n.find ('notehead').get ('filled','') == 'yes': nh += '+' 1029 | if nh and n.find ('notehead').get ('filled','') == 'no': nh += '-' 1030 | s.drumNotes [(v, noot)] = (step, midi, nh) # keep data for percussion map 1031 | tieElms = n.findall ('tie') + n.findall ('notations/tied') # in xml we have separate notated ties and playback ties 1032 | if 'start' in [e.get ('type') for e in tieElms]: # n can have stop and start tie 1033 | noot = noot + '-' 1034 | note.beam = sum ([1 for b in n.findall('beam') if b.text in ['continue', 'end']]) + int (note.grace) 1035 | lyrlast = 0; rsib = re.compile (r'^.*verse') 1036 | for e in n.findall ('lyric'): 1037 | lyrnum = int (rsib.sub ('', e.get ('number', '1'))) # also do Sibelius numbers 1038 | if lyrnum == 0: lyrnum = lyrlast + 1 # and correct Sibelius bugs 1039 | else: lyrlast = lyrnum 1040 | note.lyrs [lyrnum] = doSyllable (e) 1041 | stemdir = n.findtext ('stem') 1042 | if s.wstems and (stemdir == 'up' or stemdir == 'down'): 1043 | if stemdir != s.stemDir.get (v, ''): 1044 | s.stemDir [v] = stemdir 1045 | s.msc.appendElem (v, '[I:stemdir %s]' % stemdir) 1046 | if chord: s.msc.addChord (note, noot) 1047 | else: 1048 | xmlstaff = int (n.findtext ('staff', '1')) 1049 | if s.curStf [v] != xmlstaff: # the note should go to another staff 1050 | dstaff = xmlstaff - s.curStf [v] # relative new staff number 1051 | s.curStf [v] = xmlstaff # remember the new staff for this voice 1052 | s.msc.appendElem (v, '[I:staff %+d]' % dstaff) # insert a move before the note 1053 | s.msc.appendNote (v, note, noot) 1054 | for slur in n.findall ('notations/slur'): # s.msc.lastnote points to the last real note/chord inserted above 1055 | s.matchSlur (slur.get ('type'), slur.get ('number'), v, s.msc.lastnote, note.grace, stopgrace) # match slur definitions 1056 | 1057 | def doAttr (s, e): # parse a musicXML attribute tag 1058 | teken = {'C1':'alto1','C2':'alto2','C3':'alto','C4':'tenor','F4':'bass','F3':'bass3','G2':'treble','TAB':'tab','percussion':'perc'} 1059 | dvstxt = e.findtext ('divisions') 1060 | if dvstxt: s.msr.divs = int (dvstxt) 1061 | steps = int (e.findtext ('transpose/chromatic', '0')) # for transposing instrument 1062 | fifths = e.findtext ('key/fifths') 1063 | first = s.msc.tijd == 0 and s.msr.ixm == 0 # first attributes in first measure 1064 | if fifths: 1065 | key, s.msralts = setKey (int (fifths), e.findtext ('key/mode','major')) 1066 | if first and not steps and abcOut.key == 'none': 1067 | abcOut.key = key # first measure -> header, if not transposing instrument or percussion part! 1068 | elif key != abcOut.key or not first: 1069 | s.msr.attr += '[K:%s]' % key # otherwise -> voice 1070 | beats = e.findtext ('time/beats') 1071 | if beats: 1072 | unit = e.findtext ('time/beat-type') 1073 | mtr = beats + '/' + unit 1074 | if first: abcOut.mtr = mtr # first measure -> header 1075 | else: s.msr.attr += '[M:%s]' % mtr # otherwise -> voice 1076 | s.msr.mtr = int (beats), int (unit) 1077 | s.msr.mdur = (s.msr.divs * s.msr.mtr[0] * 4) // s.msr.mtr[1] # duration of measure in xml-divisions 1078 | for ms in e.findall('measure-style'): 1079 | n = int (ms.get ('number', '1')) # staff number 1080 | voices = s.stfMap [n] # all voices of staff n 1081 | for mr in ms.findall('measure-repeat'): 1082 | ty = mr.get('type') 1083 | if ty == 'start': # remember start measure number and text voor each staff 1084 | s.repeat_str [n] = [s.msr.ixm, mr.text] 1085 | for v in voices: # insert repeat into all voices, value will be overwritten at stop 1086 | s.msc.insertElem (v, s.repeat_str [n]) 1087 | elif ty == 'stop': # calculate repeat measure count for this staff n 1088 | start_ix, text_ = s.repeat_str [n] 1089 | repeat_count = s.msr.ixm - start_ix 1090 | if text_: 1091 | mid_str = "%s " % text_ 1092 | repeat_count /= int (text_) 1093 | else: 1094 | mid_str = "" # overwrite repeat with final string 1095 | s.repeat_str [n][0] = '[I:repeat %s%d]' % (mid_str, repeat_count) 1096 | del s.repeat_str [n] # remove closed repeats 1097 | toct = e.findtext ('transpose/octave-change', '') 1098 | if toct: steps += 12 * int (toct) # extra transposition of toct octaves 1099 | for clef in e.findall ('clef'): # a part can have multiple staves 1100 | n = int (clef.get ('number', '1')) # local staff number for this clef 1101 | sgn = clef.findtext ('sign') 1102 | line = clef.findtext ('line', '') if sgn not in ['percussion','TAB'] else '' 1103 | cs = teken.get (sgn + line, '') 1104 | oct = clef.findtext ('clef-octave-change', '') or '0' 1105 | if oct: cs += {-2:'-15', -1:'-8', 1:'+8', 2:'+15'}.get (int (oct), '') 1106 | s.clefOct [n] = -int (oct); # xml playback pitch -> abc notation pitch 1107 | if steps: cs += ' transpose=' + str (steps) 1108 | stfdtl = e.find ('staff-details') 1109 | if stfdtl and int (stfdtl.get ('number', '1')) == n: 1110 | lines = stfdtl.findtext ('staff-lines') 1111 | if lines: 1112 | lns= '|||' if lines == '3' and sgn == 'TAB' else lines 1113 | cs += ' stafflines=%s' % lns 1114 | s.stafflines = int (lines) # remember for tab staves 1115 | strings = stfdtl.findall ('staff-tuning') 1116 | if strings: 1117 | tuning = [st.findtext ('tuning-step') + st.findtext ('tuning-octave') for st in strings] 1118 | cs += ' strings=%s' % ','.join (tuning) 1119 | capo = stfdtl.findtext ('capo') 1120 | if capo: cs += ' capo=%s' % capo 1121 | s.curClef [n] = cs # keep track of current clef (for percmap) 1122 | if first: s.clefMap [n] = cs # clef goes to header (where it is mapped to voices) 1123 | else: 1124 | voices = s.stfMap[n] # clef change to all voices of staff n 1125 | for v in voices: 1126 | if n != s.curStf [v]: # voice is not at its home staff n 1127 | dstaff = n - s.curStf [v] 1128 | s.curStf [v] = n # reset current staff at start of measure to home position 1129 | s.msc.appendElem (v, '[I:staff %+d]' % dstaff) 1130 | s.msc.appendElem (v, '[K:%s]' % cs) 1131 | 1132 | def findVoice (s, i, es): 1133 | stfnum = int (es[i].findtext ('staff',1)) # directions belong to a staff 1134 | vs = s.stfMap [stfnum] # voices in this staff 1135 | v1 = vs [0] if vs else 1 # directions to first voice of staff 1136 | if s.dirtov1: return stfnum, v1, v1 # option --v1 1137 | for e in es [i+1:]: # or to the voice of the next note 1138 | if e.tag == 'note': 1139 | v = int (e.findtext ('voice', '1')) 1140 | if s.isSib: v += 100 * int (e.findtext ('staff', '1')) # repair bug in Sibelius 1141 | stf = s.vce2stf [v] # use our own staff allocation 1142 | return stf, v, v1 # voice of next note, first voice of staff 1143 | if e.tag == 'backup': break 1144 | return stfnum, v1, v1 # no note found, fall back to v1 1145 | 1146 | def doDirection (s, e, i, es): # parse a musicXML direction tag 1147 | def addDirection (x, vs, tijd, stfnum): 1148 | if not x: return 1149 | vs = s.stfMap [stfnum] if '!8v' in x else [vs] # ottava's go to all voices of staff 1150 | for v in vs: 1151 | if tijd != None: # insert at time of encounter 1152 | s.msc.appendElemT (v, x.replace ('(',')').replace ('ped','ped-up'), tijd) 1153 | else: 1154 | s.msc.appendElem (v, x) 1155 | def startStop (dtype, vs, stfnum=1): 1156 | typmap = {'down':'!8va(!', 'up':'!8vb(!', 'crescendo':'!<(!', 'diminuendo':'!>(!', 'start':'!ped!'} 1157 | type = t.get ('type', '') 1158 | k = dtype + t.get ('number', '1') # key to match the closing direction 1159 | if type in typmap: # opening the direction 1160 | x = typmap [type] 1161 | if k in s.dirStk: # closing direction already encountered 1162 | stype, tijd = s.dirStk [k]; del s.dirStk [k] 1163 | if stype == 'stop': 1164 | addDirection (x, vs, tijd, stfnum) 1165 | else: 1166 | info ('%s direction %s has no stop in part %d, measure %d, voice %d' % (dtype, stype, s.msr.ixp+1, s.msr.ixm+1, vs+1)) 1167 | s.dirStk [k] = ((type , vs)) # remember voice and type for closing 1168 | else: 1169 | s.dirStk [k] = ((type , vs)) # remember voice and type for closing 1170 | elif type == 'stop': 1171 | if k in s.dirStk: # matching open direction found 1172 | type, vs = s.dirStk [k]; del s.dirStk [k] # into the same voice 1173 | if type == 'stop': 1174 | info ('%s direction %s has double stop in part %d, measure %d, voice %d' % (dtype, type, s.msr.ixp+1, s.msr.ixm+1, vs+1)) 1175 | x = '' 1176 | else: 1177 | x = typmap [type].replace ('(',')').replace ('ped','ped-up') 1178 | else: # closing direction found before opening 1179 | s.dirStk [k] = ('stop', s.msc.tijd) 1180 | x = '' # delay code generation until opening found 1181 | else: raise ValueError ('wrong direction type') 1182 | addDirection (x, vs, None, stfnum) 1183 | tempo, wrdstxt = None, '' 1184 | plcmnt = e.get ('placement') 1185 | stf, vs, v1 = s.findVoice (i, es) 1186 | jmp = '' # for jump sound elements: dacapo, dalsegno and family 1187 | jmps = [('dacapo','D.C.'),('dalsegno','D.S.'),('tocoda','dacoda'),('fine','fine'),('coda','O'),('segno','S')] 1188 | t = e.find ('sound') # there are many possible attributes for sound 1189 | if t != None: 1190 | minst = t.find ('midi-instrument') 1191 | if minst: 1192 | prg = t.findtext ('midi-instrument/midi-program') 1193 | chn = t.findtext ('midi-instrument/midi-channel') 1194 | vids = [v for v, id in s.vceInst.items () if id == minst.get ('id')] 1195 | if vids: vs = vids [0] # direction for the indentified voice, not the staff 1196 | parm, inst = ('program', str (int (prg) - 1)) if prg else ('channel', chn) 1197 | if inst and abcOut.volpan > 0: s.msc.appendElem (vs, '[I:MIDI= %s %s]' % (parm, inst)) 1198 | tempo = t.get ('tempo') # look for tempo attribute 1199 | if tempo: 1200 | tempo = '%.0f' % float (tempo) # hope it is a number and insert in voice 1 1201 | tempo_units = (1,4) # always 1/4 for sound elements! 1202 | for r, v in jmps: 1203 | if t.get (r, ''): jmp = v; break 1204 | dirtypes = e.findall ('direction-type') 1205 | for dirtyp in dirtypes: 1206 | units = { 'whole': (1,1), 'half': (1,2), 'quarter': (1,4), 'eighth': (1,8) } 1207 | metr = dirtyp.find ('metronome') 1208 | if metr != None: 1209 | t = metr.findtext ('beat-unit', '') 1210 | if t in units: tempo_units = units [t] 1211 | else: tempo_units = units ['quarter'] 1212 | if metr.find ('beat-unit-dot') != None: 1213 | tempo_units = simplify (tempo_units [0] * 3, tempo_units [1] * 2) 1214 | tmpro = re.search ('[.\d]+', metr.findtext ('per-minute')) # look for a number 1215 | if tmpro: tempo = tmpro.group () # overwrites the value set by the sound element of this direction 1216 | t = dirtyp.find ('wedge') 1217 | if t != None: startStop ('wedge', vs) 1218 | allwrds = dirtyp.findall ('words') # insert text annotations 1219 | if not allwrds: allwrds = dirtyp.findall ('rehearsal') # treat rehearsal mark as text annotation 1220 | for wrds in allwrds: 1221 | if jmp: # ignore the words when a jump sound element is present in this direction 1222 | s.msc.appendElem (vs, '!%s!' % jmp , 1) # to voice 1223 | break 1224 | plc = plcmnt == 'below' and '_' or '^' 1225 | if float (wrds.get ('default-y', '0')) < 0: plc = '_' 1226 | wrdstxt += (wrds.text or '').replace ('"','\\"').replace ('\n', '\\n') 1227 | wrdstxt = wrdstxt.strip () 1228 | for key, val in dynamics_map.items (): 1229 | if dirtyp.find ('dynamics/' + key) != None: 1230 | s.msc.appendElem (vs, val, 1) # to voice 1231 | if dirtyp.find ('coda') != None: s.msc.appendElem (vs, 'O', 1) 1232 | if dirtyp.find ('segno') != None: s.msc.appendElem (vs, 'S', 1) 1233 | t = dirtyp.find ('octave-shift') 1234 | if t != None: startStop ('octave-shift', vs, stf) # assume size == 8 for the time being 1235 | t = dirtyp.find ('pedal') 1236 | if t != None and s.ped: 1237 | if not s.pedVce: s.pedVce = vs 1238 | startStop ('pedal', s.pedVce) 1239 | if dirtyp.findtext ('other-direction') == 'diatonic fretting': s.diafret = 1; 1240 | if tempo: 1241 | tempo = '%.0f' % float (tempo) # hope it is a number and insert in voice 1 1242 | if s.msc.tijd == 0 and s.msr.ixm == 0: # first measure -> header 1243 | abcOut.tempo = tempo 1244 | abcOut.tempo_units = tempo_units 1245 | else: 1246 | s.msc.appendElem (v1, '[Q:%d/%d=%s]' % (tempo_units [0], tempo_units [1], tempo)) # otherwise -> 1st voice 1247 | if wrdstxt: s.msc.appendElem (vs, '"%s%s"' % (plc, wrdstxt), 1) # to voice, but after tempo 1248 | 1249 | def doHarmony (s, e, i, es): # parse a musicXMl harmony tag 1250 | _, vt, _ = s.findVoice (i, es) 1251 | short = {'major':'', 'minor':'m', 'augmented':'+', 'diminished':'dim', 'dominant':'7', 'half-diminished':'m7b5'} 1252 | accmap = {'major':'maj', 'dominant':'', 'minor':'m', 'diminished':'dim', 'augmented':'+', 'suspended':'sus'} 1253 | modmap = {'second':'2', 'fourth':'4', 'seventh':'7', 'sixth':'6', 'ninth':'9', '11th':'11', '13th':'13'} 1254 | altmap = {'1':'#', '0':'', '-1':'b'} 1255 | root = e.findtext ('root/root-step','') 1256 | alt = altmap.get (e.findtext ('root/root-alter'), '') 1257 | sus = '' 1258 | kind = e.findtext ('kind', '') 1259 | if kind in short: kind = short [kind] 1260 | elif '-' in kind: # xml chord names: - 1261 | triad, mod = kind.split ('-') 1262 | kind = accmap.get (triad, '') + modmap.get (mod, '') 1263 | if kind.startswith ('sus'): kind, sus = '', kind # sus-suffix goes to the end 1264 | elif kind == 'none': kind = e.find ('kind').get ('text','') 1265 | degrees = e.findall ('degree') 1266 | for d in degrees: # chord alterations 1267 | kind += altmap.get (d.findtext ('degree-alter'),'') + d.findtext ('degree-value','') 1268 | kind = kind.replace ('79','9').replace ('713','13').replace ('maj6','6') 1269 | bass = e.findtext ('bass/bass-step','') + altmap.get (e.findtext ('bass/bass-alter'),'') 1270 | s.msc.appendElem (vt, '"%s%s%s%s%s"' % (root, alt, kind, sus, bass and '/' + bass), 1) 1271 | 1272 | def doBarline (s, e): # 0 = no repeat, 1 = begin repeat, 2 = end repeat 1273 | rep = e.find ('repeat') 1274 | if rep != None: rep = rep.get ('direction') 1275 | if s.unfold: # unfold repeat, don't translate barlines 1276 | return rep and (rep == 'forward' and 1 or 2) or 0 1277 | loc = e.get ('location', 'right') # right is the default 1278 | if loc == 'right': # only change style for the right side 1279 | style = e.findtext ('bar-style') 1280 | if style == 'light-light': s.msr.rline = '||' 1281 | elif style == 'light-heavy': s.msr.rline = '|]' 1282 | if rep != None: # repeat found 1283 | if rep == 'forward': s.msr.lline = ':' 1284 | else: s.msr.rline = ':|' # override barline style 1285 | end = e.find ('ending') 1286 | if end != None: 1287 | if end.get ('type') == 'start': 1288 | n = end.get ('number', '1').replace ('.','').replace (' ','') 1289 | try: list (map (int, n.split (','))) # should be a list of integers 1290 | except: n = '"%s"' % n.strip () # illegal musicXML 1291 | s.msr.lnum = n # assume a start is always at the beginning of a measure 1292 | elif s.msr.rline == '|': # stop and discontinue the same in ABC ? 1293 | s.msr.rline = '||' # to stop on a normal barline use || in ABC ? 1294 | return 0 1295 | 1296 | def doPrint (s, e): # print element, measure number -> insert a line break 1297 | if e.get ('new-system') == 'yes' or e.get ('new-page') == 'yes': 1298 | if not s.nolbrk: return '$' # a line break 1299 | 1300 | def doPartList (s, e): # translate the start/stop-event-based xml-partlist into proper tree 1301 | for sp in e.findall ('part-list/score-part'): 1302 | midi = {} 1303 | for m in sp.findall ('midi-instrument'): 1304 | x = [m.findtext (p, s.midDflt [i]) for i,p in enumerate (['midi-channel','midi-program','volume','pan'])] 1305 | pan = float (x[3]) 1306 | if pan >= -90 and pan <= 90: # would be better to map behind-pannings 1307 | pan = (float (x[3]) + 90) / 180 * 127 # xml between -90 and +90 1308 | midi [m.get ('id')] = [int (x[0]), int (x[1]), float (x[2]) * 1.27, pan] # volume 100 -> midi 127 1309 | up = m.findtext ('midi-unpitched') 1310 | if up: s.drumInst [m.get ('id')] = int (up) - 1 # store midi-pitch for channel 10 notes 1311 | s.instMid.append (midi) 1312 | ps = e.find ('part-list') # partlist = [groupelem] 1313 | xs = getPartlist (ps) # groupelem = partname | grouplist 1314 | partlist, _ = parseParts (xs, {}, []) # grouplist = [groupelem, ..., groupdata] 1315 | return partlist # groupdata = [group-symbol, group-barline, group-name, group-abbrev] 1316 | 1317 | def mkTitle (s, e): 1318 | def filterCredits (y): # y == filter level, higher filters less 1319 | cs = [] 1320 | for x in credits: # skip redundant credit lines 1321 | if y < 6 and (x in title or x in mvttl): continue # sure skip 1322 | if y < 5 and (x in composer or x in lyricist): continue # almost sure skip 1323 | if y < 4 and ((title and title in x) or (mvttl and mvttl in x)): continue # may skip too much 1324 | if y < 3 and ([1 for c in composer if c in x] or [1 for c in lyricist if c in x]): continue # skips too much 1325 | if y < 2 and re.match (r'^[\d\W]*$', x): continue # line only contains numbers and punctuation 1326 | cs.append (x) 1327 | if y == 0 and (title + mvttl): cs = '' # default: only credit when no title set 1328 | return cs 1329 | title = e.findtext ('work/work-title', '').strip () 1330 | mvttl = e.findtext ('movement-title', '').strip () 1331 | composer, lyricist, credits = [], [], [] 1332 | for creator in e.findall ('identification/creator'): 1333 | if creator.text: 1334 | if creator.get ('type') == 'composer': 1335 | composer += [line.strip () for line in creator.text.split ('\n')] 1336 | elif creator.get ('type') in ('lyricist', 'transcriber'): 1337 | lyricist += [line.strip () for line in creator.text.split ('\n')] 1338 | for rights in e.findall ('identification/rights'): 1339 | if rights.text: 1340 | lyricist += [line.strip () for line in rights.text.split ('\n')] 1341 | for credit in e.findall('credit'): 1342 | cs = ''.join (e.text or '' for e in credit.findall('credit-words')) 1343 | credits += [re.sub (r'\s*[\r\n]\s*', ' ', cs)] 1344 | credits = filterCredits (s.ctf) 1345 | if title: title = 'T:%s\n' % title.replace ('\n', '\nT:') 1346 | if mvttl: title += 'T:%s\n' % mvttl.replace ('\n', '\nT:') 1347 | if credits: title += '\n'.join (['T:%s' % c for c in credits]) + '\n' 1348 | if composer: title += '\n'.join (['C:%s' % c for c in composer]) + '\n' 1349 | if lyricist: title += '\n'.join (['Z:%s' % c for c in lyricist]) + '\n' 1350 | if title: abcOut.title = title[:-1] 1351 | s.isSib = 'Sibelius' in (e.findtext ('identification/encoding/software') or '') 1352 | if s.isSib: info ('Sibelius MusicXMl is unreliable') 1353 | 1354 | def doDefaults (s, e): 1355 | if not s.doPageFmt: return # return if -pf option absent 1356 | d = e.find ('defaults'); 1357 | if d == None: return; 1358 | mils = d.findtext ('scaling/millimeters') # mills == staff height (mm) 1359 | tenths = d.findtext ('scaling/tenths') # staff height in tenths 1360 | if not mils or not tenths: return 1361 | xmlScale = float (mils) / float (tenths) / 10 # tenths -> mm 1362 | space = 10 * xmlScale # space between staff lines == 10 tenths 1363 | abcScale = space / 0.2117 # 0.2117 cm = 6pt = space between staff lines for scale = 1.0 in abcm2ps 1364 | abcOut.pageFmt ['scale'] = abcScale 1365 | eks = 2 * ['page-layout/'] + 4 * ['page-layout/page-margins/'] 1366 | eks = [a+b for a,b in zip (eks, 'page-height,page-width,left-margin,right-margin,top-margin,bottom-margin'.split (','))] 1367 | for i in range (6): 1368 | v = d.findtext (eks [i]) 1369 | k = abcOut.pagekeys [i+1] # pagekeys [0] == scale already done, skip it 1370 | if not abcOut.pageFmt [k] and v: 1371 | try: abcOut.pageFmt [k] = float (v) * xmlScale # -> cm 1372 | except: info ('illegal value %s for xml element %s', (v, eks [i])); continue # just skip illegal values 1373 | 1374 | def locStaffMap (s, part, maten): # map voice to staff with majority voting 1375 | vmap = {} # {voice -> {staff -> n}} count occurrences of voice in staff 1376 | s.vceInst = {} # {voice -> instrument id} for this part 1377 | s.msc.vnums = {} # voice id's 1378 | s.hasStems = {} # XML voice nums with at least one note with a stem (for tab key) 1379 | s.stfMap, s.clefMap = {}, {} # staff -> [voices], staff -> clef 1380 | ns = part.findall ('measure/note') 1381 | for n in ns: # count staff allocations for all notes 1382 | v = int (n.findtext ('voice', '1')) 1383 | if s.isSib: v += 100 * int (n.findtext ('staff', '1')) # repair bug in Sibelius 1384 | s.msc.vnums [v] = 1 # collect all used voice id's in this part 1385 | sn = int (n.findtext ('staff', '1')) 1386 | s.stfMap [sn] = [] 1387 | if v not in vmap: 1388 | vmap [v] = {sn:1} 1389 | else: 1390 | d = vmap[v] # counter for voice v 1391 | d[sn] = d.get (sn, 0) + 1 # ++ number of allocations for staff sn 1392 | x = n.find ('instrument') 1393 | if x != None: s.vceInst [v] = x.get ('id') 1394 | x, noRest = n.findtext ('stem'), n.find ('rest') == None 1395 | if noRest and (not x or x != 'none'): s.hasStems [v] = 1 # XML voice v has at least one stem 1396 | vks = list (vmap.keys ()) 1397 | if s.jscript or s.isSib: vks.sort () 1398 | for v in vks: # choose staff with most allocations for each voice 1399 | xs = [(n, sn) for sn, n in vmap[v].items ()] 1400 | xs.sort () 1401 | stf = xs[-1][1] # the winner: staff with most notes of voice v 1402 | s.stfMap [stf].append (v) 1403 | s.vce2stf [v] = stf # reverse map 1404 | s.curStf [v] = stf # current staff of XML voice v 1405 | 1406 | def addStaffMap (s, vvmap): # vvmap: xml voice number -> global abc voice number 1407 | part = [] # default: brace on staffs of one part 1408 | for stf, voices in sorted (s.stfMap.items ()): # s.stfMap has xml staff and voice numbers 1409 | locmap = [vvmap [iv] for iv in voices if iv in vvmap] 1410 | nostem = [(iv not in s.hasStems) for iv in voices if iv in vvmap] # same order as locmap 1411 | if locmap: # abc voice number of staff stf 1412 | part.append (locmap) 1413 | clef = s.clefMap.get (stf, 'treble') # {xml staff number -> clef} 1414 | for i, iv in enumerate (locmap): 1415 | clef_attr = '' 1416 | if clef.startswith ('tab'): 1417 | if nostem [i] and 'nostems' not in clef: clef_attr = ' nostems' 1418 | if s.diafret and 'diafret' not in clef: clef_attr += ' diafret' # for all voices in the part 1419 | abcOut.clefs [iv] = clef + clef_attr # add nostems when all notes of voice had no stem 1420 | s.gStfMap.append (part) 1421 | 1422 | def addMidiMap (s, ip, vvmap): # map abc voices to midi settings 1423 | instr = s.instMid [ip] # get the midi settings for this part 1424 | if instr.values (): defInstr = list(instr.values ())[0] # default settings = first instrument 1425 | else: defInstr = s.midDflt # no instruments defined 1426 | xs = [] 1427 | for v, vabc in vvmap.items (): # xml voice num, abc voice num 1428 | ks = sorted (s.drumNotes.items ()) 1429 | ds = [(nt, step, midi, head) for (vd, nt), (step, midi, head) in ks if v == vd] # map perc notes 1430 | id = s.vceInst.get (v, '') # get the instrument-id for part with multiple instruments 1431 | if id in instr: # id is defined as midi-instrument in part-list 1432 | xs.append ((vabc, instr [id] + ds)) # get midi settings for id 1433 | else: xs.append ((vabc, defInstr + ds)) # only one instrument for this part 1434 | xs.sort () # put abc voices in order 1435 | s.midiMap.extend ([midi for v, midi in xs]) 1436 | snaarmap = ['E','G','B','d', 'f', 'a', "c'", "e'"] 1437 | diamap = '0,1-,1,1+,2,3,3,4,4,5,6,6+,7,8-,8,8+,9,10,10,11,11,12,13,13+,14'.split (',') 1438 | for k in sorted (s.tabmap.keys ()): # add %%map's for all tab voices 1439 | v, noot = k; 1440 | snaar, fret = s.tabmap [k]; 1441 | if s.diafret: fret = diamap [int (fret)] 1442 | vabc = vvmap [v] 1443 | snaar = s.stafflines - int (snaar) 1444 | xs = s.tabVceMap.get (vabc, []) 1445 | xs.append ('%%%%map tab%d %s print=%s heads=kop%s\n' % (vabc, noot, snaarmap [snaar], fret)) 1446 | s.tabVceMap [vabc] = xs 1447 | s.koppen [fret] = 1 # collect noteheads for SVG defs 1448 | 1449 | def parse (s, fobj): 1450 | vvmapAll = {} # collect xml->abc voice maps (vvmap) of all parts 1451 | e = E.parse (fobj) 1452 | s.mkTitle (e) 1453 | s.doDefaults (e) 1454 | partlist = s.doPartList (e) 1455 | parts = e.findall ('part') 1456 | for ip, p in enumerate (parts): 1457 | maten = p.findall ('measure') 1458 | s.locStaffMap (p, maten) # {voice -> staff} for this part 1459 | s.drumNotes = {} # (xml voice, abc note) -> (midi note, note head) 1460 | s.clefOct = {} # xml staff number -> current clef-octave-change 1461 | s.curClef = {} # xml staff number -> current abc clef 1462 | s.stemDir = {} # xml voice number -> current stem direction 1463 | s.tabmap = {} # (xml voice, abc note) -> (string, fret) 1464 | s.diafret = 0 # use diatonic fretting 1465 | s.stafflines = 5 1466 | s.msc.initVoices (newPart = 1) # create all voices 1467 | aantalHerhaald = 0 # keep track of number of repititions 1468 | herhaalMaat = 0 # target measure of the repitition 1469 | divisions = [] # current value of for each measure 1470 | s.msr = Measure (ip) # various measure data 1471 | while s.msr.ixm < len (maten): 1472 | maat = maten [s.msr.ixm] 1473 | herhaal, lbrk = 0, '' 1474 | s.msr.reset () 1475 | s.curalts = {} # passing accidentals are reset each measure 1476 | es = list (maat) 1477 | for i, e in enumerate (es): 1478 | if e.tag == 'note': s.doNote (e) 1479 | elif e.tag == 'attributes': s.doAttr (e) 1480 | elif e.tag == 'direction': s.doDirection (e, i, es) 1481 | elif e.tag == 'sound': s.doDirection (maat, i, es) # sound element directly in measure! 1482 | elif e.tag == 'harmony': s.doHarmony (e, i, es) 1483 | elif e.tag == 'barline': herhaal = s.doBarline (e) 1484 | elif e.tag == 'backup': 1485 | dt = int (e.findtext ('duration')) 1486 | if chkbug (dt, s.msr): s.msc.incTime (-dt) 1487 | elif e.tag == 'forward': 1488 | dt = int (e.findtext ('duration')) 1489 | if chkbug (dt, s.msr): s.msc.incTime (dt) 1490 | elif e.tag == 'print': lbrk = s.doPrint (e) 1491 | s.msc.addBar (lbrk, s.msr) 1492 | divisions.append (s.msr.divs) 1493 | if herhaal == 1: 1494 | herhaalMaat = s.msr.ixm 1495 | s.msr.ixm += 1 1496 | elif herhaal == 2: 1497 | if aantalHerhaald < 1: # jump 1498 | s.msr.ixm = herhaalMaat 1499 | aantalHerhaald += 1 1500 | else: 1501 | aantalHerhaald = 0 # reset 1502 | s.msr.ixm += 1 # just continue 1503 | else: s.msr.ixm += 1 # on to the next measure 1504 | for rv in s.repeat_str.values (): # close hanging measure-repeats without stop 1505 | rv [0] = '[I:repeat %s %d]' % (rv [1], 1) 1506 | vvmap = s.msc.outVoices (divisions, ip, s.isSib) 1507 | s.addStaffMap (vvmap) # update global staff map 1508 | s.addMidiMap (ip, vvmap) 1509 | vvmapAll.update (vvmap) 1510 | if vvmapAll: # skip output if no part has any notes 1511 | abcOut.mkHeader (s.gStfMap, partlist, s.midiMap, s.tabVceMap, s.koppen) 1512 | abcOut.writeall () 1513 | else: info ('nothing written, %s has no notes ...' % abcOut.fnmext) 1514 | 1515 | #---------------- 1516 | # Main Program 1517 | #---------------- 1518 | if __name__ == '__main__': 1519 | from optparse import OptionParser 1520 | from glob import glob 1521 | from zipfile import ZipFile 1522 | ustr = '%prog [-h] [-u] [-m] [-c C] [-d D] [-n CPL] [-b BPL] [-o DIR] [-v V]\n' 1523 | ustr += '[-x] [-p PFMT] [-t] [-s] [-i] [--v1] [--noped] [--stems] [ ...]' 1524 | parser = OptionParser (usage=ustr, version=str(VERSION)) 1525 | parser.add_option ("-u", action="store_true", help="unfold simple repeats") 1526 | parser.add_option ("-m", action="store", help="0 -> no %%MIDI, 1 -> minimal %%MIDI, 2-> all %%MIDI", default=0) 1527 | parser.add_option ("-c", action="store", type="int", help="set credit text filter to C", default=0, metavar='C') 1528 | parser.add_option ("-d", action="store", type="int", help="set L:1/D", default=0, metavar='D') 1529 | parser.add_option ("-n", action="store", type="int", help="CPL: max number of characters per line (default 100)", default=0, metavar='CPL') 1530 | parser.add_option ("-b", action="store", type="int", help="BPL: max number of bars per line", default=0, metavar='BPL') 1531 | parser.add_option ("-o", action="store", help="store abc files in DIR", default='', metavar='DIR') 1532 | parser.add_option ("-v", action="store", type="int", help="set volta typesetting behaviour to V", default=0, metavar='V') 1533 | parser.add_option ("-x", action="store_true", help="output no line breaks") 1534 | parser.add_option ("-p", action="store", help="pageformat PFMT (cm) = scale, pageheight, pagewidth, leftmargin, rightmargin, topmargin, botmargin", default='', metavar='PFMT') 1535 | parser.add_option ("-j", action="store_true", help="switch for compatibility with javascript version") 1536 | parser.add_option ("-t", action="store_true", help="translate perc- and tab-staff to ABC code with %%map, %%voicemap") 1537 | parser.add_option ("-s", action="store_true", help="shift node heads 3 units left in a tab staff") 1538 | parser.add_option ("--v1", action="store_true", help="start-stop directions allways to first voice of staff") 1539 | parser.add_option ("--noped", action="store_false", help="skip all pedal directions", dest='ped', default=True) 1540 | parser.add_option ("--stems", action="store_true", help="translate stem directions", dest='stm', default=False) 1541 | parser.add_option ("-i", action="store_true", help="read xml file from standard input") 1542 | options, args = parser.parse_args () 1543 | if options.n < 0: parser.error ('only values >= 0') 1544 | if options.b < 0: parser.error ('only values >= 0') 1545 | if options.d and options.d not in [2**n for n in range (10)]: 1546 | parser.error ('D should be on of %s' % ','.join ([str(2**n) for n in range (10)])) 1547 | options.p = options.p and options.p.split (',') or [] # ==> [] | [string] 1548 | if len (args) == 0 and not options.i: parser.error ('no input file given') 1549 | pad = options.o 1550 | if pad: 1551 | if not os.path.exists (pad): os.mkdir (pad) 1552 | if not os.path.isdir (pad): parser.error ('%s is not a directory' % pad) 1553 | fnmext_list = [] 1554 | for i in args: fnmext_list += glob (i) 1555 | if options.i: fnmext_list = ['stdin.xml'] 1556 | if not fnmext_list: parser.error ('none of the input files exist') 1557 | for X, fnmext in enumerate (fnmext_list): 1558 | fnm, ext = os.path.splitext (fnmext) 1559 | if ext.lower () not in ('.xml','.mxl','.musicxml'): 1560 | info ('skipped input file %s, it should have extension .xml or .mxl' % fnmext) 1561 | continue 1562 | if os.path.isdir (fnmext): 1563 | info ('skipped directory %s. Only files are accepted' % fnmext) 1564 | continue 1565 | if fnmext == 'stdin.xml': 1566 | fobj = sys.stdin 1567 | elif ext.lower () == '.mxl': # extract .xml file from .mxl file 1568 | z = ZipFile(fnmext) 1569 | for n in z.namelist(): # assume there is always an xml file in a mxl archive !! 1570 | if (n[:4] != 'META') and (n[-4:].lower() == '.xml'): 1571 | fobj = z.open (n) 1572 | break # assume only one MusicXML file per archive 1573 | else: 1574 | fobj = open (fnmext, 'rb') # open regular xml file 1575 | 1576 | abcOut = ABCoutput (fnm + '.abc', pad, X, options) # create global ABC output object 1577 | psr = Parser (options) # xml parser 1578 | try: 1579 | psr.parse (fobj) # parse file fobj and write abc to .abc 1580 | except: 1581 | etype, value, traceback = sys.exc_info () # works in python 2 & 3 1582 | info ('** %s occurred: %s in %s' % (etype, value, fnmext), 0) 1583 | --------------------------------------------------------------------------------