├── LICENSE ├── README.md ├── harmony_visualizer.py ├── icon.ico ├── icon.png ├── icon_128x128.png ├── images └── harmony_visualizer.png └── requirements.txt /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2022, Frieve-A 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | 3. Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Harmony Visualizer 2 | A tool for visualizing the harmony of MIDI input. 3 | 4 | (英語に続き日本語の解説があります) 5 | 6 | ![Harmony Visualizer](./images/harmony_visualizer.png) 7 | 8 | A tool for visualizing the harmony of the sound using virtual overtones based on MIDI input. 9 | 10 | 11 |
12 | 13 | ## How to download and install on Windows 14 | 15 | Latest version for 64bit Windows can be downloaded from the following page. 16 | 17 | https://github.com/Frieve-A/harmony_visualizer/releases 18 | 19 | Unzip the downloaded zip file and run the harmony_visualizer.exe to launch the app. No installation is required. 20 | 21 |
22 | 23 | ## How to execute on other platforms 24 | 25 | This application is written in Python. 26 | Follow the steps below to execute the Python code of the Harmony Visualizer. 27 | 28 | 1. git clone https://github.com/Frieve-A/harmony_visualizer.git 29 | 2. pip install -r requirements.txt 30 | 3. python harmony_visualizer.py 31 | 32 |
33 | 34 | ## How to use 35 | 36 | If a MIDI keyboard is available, connect the MIDI keyboard to your PC and launch the app. 37 | 38 | When you play a note with the MIDI keyboard or click on the keyboard, the notes you play are displayed in different colors on the keyboard for each note name. For example, Do is displayed in red, and So is displayed in blue which is a little closer to green. 39 | 40 | Above the keyboard display, a vertical line is displayed to indicate the volumme of the virtual overtones. For example, if you play Do, a vertical line indicating the second overtone is displayed at the Do position one octave higher, and the third overtone is displayed at the So position. 41 | 42 | If you input two or more sounds at the same time and the overtones are in harmony with each other, the overtones that are in harmony with the input sound are connected by a curve. The numbers on the curve indicate which overtones are in harmony. 43 | 44 | The graph on the upper left shows the volume of each overtone as a polar coordinate graph with a radius as a pitch and an angle that makes one round in one octave. 45 | 46 | To hold the display, press the CTRL key on your computer keyboard or press the damper pedal on your MIDI keyboard. 47 | 48 | Play your favorite songs and visually enjoy the harmony of the overtones! 49 | 50 |
51 | 52 | ## Keyboard shortcuts 53 | 54 | ESC : Exit the app 55 | 56 | F11 : Switch to full screen 57 | 58 |
59 | 60 | --- 61 | 62 |
63 | 64 | MIDI入力を元にした仮想的な倍音を用いながら、音のハーモニーを可視化するツールです。 65 | 66 | 67 |
68 | 69 | ## ダウンロードとインストール（Windows） 70 | 71 | 以下のページより64bit Windows用の最新バージョンをダウンロードします。 72 | 73 | https://github.com/Frieve-A/harmony_visualizer/releases 74 | 75 | ダウンロードしたzipファイルを解凍し、harmony_visualizer.exeファイル起動します。インストールは不要です。 76 | 77 |
78 | 79 | ## その他のプラットフォームで実行するには 80 | 81 | 本アプリケーションはPythonで作成されています。 82 | 以下の手順でHarmony VisualizerのPythonコードを実行します。 83 | 84 | 1. git clone https://github.com/Frieve-A/harmony_visualizer.git 85 | 2. pip install -r requirements.txt 86 | 3. python harmony_visualizer.py 87 | 88 |
89 | 90 | ## 使い方 91 | 92 | MIDIキーボードが利用できる場合、PCにMIDIキーボードを接続してアプリを起動します。 93 | 94 | MIDIキーボードで音を弾くか、鍵盤上をクリックすると、弾いた音は音名毎にそれぞれ異なる色で鍵盤上に色分け表示されます。例えばドは赤色で、ソは少しグリーン寄りの青で表示されます。 95 | 96 | 鍵盤表示の上には、仮想的な倍音の大きさを示す縦線が表示されます。例えばドを弾いた場合、1オクターブ上のドの位置に2倍音、ソの位置に3倍音を示す縦線が表示されます。 97 | 98 | 2つ以上のを同時に入力しその倍音同士が調和している場合、入力した音と調和している倍音は曲線で結ばれます。曲線の上の数字は調和しているのが何番目の倍音であるかを示しています。 99 | 100 | 左上のグラフはそれぞれの倍音の強度を、半径を音程とし、角度を1オクターブで1周するように設定した極座標グラフで表示したものです。 101 | 102 | 表示をホールドするには、パソコンのキーボードのCTRLキーを押すか、MIDIキーボードのダンパーペダルを踏みます。 103 | 104 | 好きな楽曲を演奏し、倍音同士が調和する様子を視覚的に楽しみましょう！ 105 | 106 | ## キーボードショートカット 107 | 108 | ESC : アプリを終了 109 | 110 | F11 : フルスクリーンへの切り替え 111 | 112 | 113 | 114 | 115 | -------------------------------------------------------------------------------- /harmony_visualizer.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import math 4 | import time 5 | import pygame 6 | import pygame.midi 7 | from pygame.locals import * 8 | import numpy as np 9 | 10 | app_title = 'Harmony visualizer' 11 | screen_size = (1920, 1080) 12 | #screen_size = (2560, 1440) 13 | #screen_size = (3840, 2160) 14 | line_width = screen_size[1] // 1081 + 1 15 | bold_line_width = line_width * 3 16 | base_size = screen_size[0] // 240 # 8 pixel in 1920 x 1080 17 | keyboard_margin_x = base_size * 3 18 | key_width = (screen_size[0] - keyboard_margin_x * 2) / 52 19 | keyboard_top = base_size * 100 20 | energy_bottom = keyboard_top - keyboard_margin_x 21 | white_key_width = round(key_width * 33 / 36) #22.5 / 23.5 22 | white_key_height = round(key_width * 150 / 23.5) 23 | black_key_width = round(key_width * 23 / 36) #15 / 23.5 24 | black_key_height = round(key_width * 100 / 23.5) 25 | min_note_no = 21 26 | max_note_no = 109.2 # max spectrum frequency in note no 27 | energy_width = (key_width * 7) / 12 - 2 28 | polar_overtone_radius = screen_size[0] // 10 29 | polar_overtone_center = (polar_overtone_radius + base_size * 7, polar_overtone_radius + base_size * 19) 30 | tone_color = np.array([ 31 | (255, 128, 128), 32 | (224, 160, 128), 33 | (192, 192, 128), 34 | (160, 224, 128), 35 | (128, 255, 128), 36 | (128, 224, 160), 37 | (128, 192, 192), 38 | (128, 160, 224), 39 | (128, 128, 255), 40 | (160, 128, 224), 41 | (192, 128, 192), 42 | (224, 128, 160) 43 | ], dtype=np.int32) 44 | 45 | def prepare_midi_ins(): 46 | pygame.midi.init() 47 | midi_ins = [] 48 | midi_in_info = '' 49 | for i in range(pygame.midi.get_count()): 50 | midi_device_info = pygame.midi.get_device_info(i) 51 | if midi_device_info[2] and not midi_device_info[4]: # is input device and not opened. 52 | try: 53 | midi_ins.append(pygame.midi.Input(i)) 54 | except: 55 | # Failed to open MIDI in 56 | continue 57 | if midi_in_info: 58 | midi_in_info += ' and ' 59 | midi_in_info += '"' + midi_device_info[1].decode('utf-8') + '"' 60 | if midi_in_info: 61 | midi_in_info = 'Listening ' + midi_in_info + ' MIDI in(s).' 62 | else: 63 | midi_in_info = 'No available MIDI In found.' 64 | return midi_ins, midi_in_info 65 | 66 | def prepare_keyboard(): 67 | keys = [] 68 | 69 | class Key: 70 | pass 71 | for i in range(128): 72 | oct = i // 12 - 1 73 | note = i % 12 74 | black_key = note in [1, 3, 6, 8, 10] 75 | x = round(keyboard_margin_x + key_width * (oct * 7 + [0.5, 0.925, 1.5, 2.075, 2.5, 3.5, 3.85, 4.5, 5, 5.5, 6.15, 6.5][note] - 5)) 76 | key= Key() 77 | key.note_no = i 78 | key.black_key = black_key 79 | key.x = x 80 | key.normalized_x = round(keyboard_margin_x + 0.5 * key_width + (key_width * 7) / 12 * (i - 21)) 81 | key.note_on = False # Whether there is a sound including the damper pedal 82 | key.key_on = False # Whether the keyboard is pressed 83 | key.db = 0.0 # Virtual volume in DBs 84 | key.attack = 0.0 # Whether immediately after the attack. 1.0 indicates that it is just after the attack 85 | key.harmony = 0.0 # The volume of that harmony in DBs if the pitch is in harmony 86 | key.display_harmony = 0.0 # Display value with fluctuations added to the volume of the harmony 87 | keys.append(key) 88 | return keys 89 | 90 | def note_no_to_x(note_no): 91 | return keyboard_margin_x + 0.5 * key_width + (key_width * 7) / 12 * (note_no - 21) # 1 oct. lower 92 | 93 | def resource_path(relative_path): 94 | if hasattr(sys, '_MEIPASS'): 95 | return os.path.join(sys._MEIPASS, relative_path) 96 | return os.path.join(os.path.abspath("."), relative_path) 97 | 98 | def main(): 99 | # initialization 100 | pygame.init() 101 | pygame.display.set_icon(pygame.image.load(resource_path('icon_128x128.png'))) 102 | screen = pygame.display.set_mode(screen_size) 103 | full_screen = False 104 | 105 | pygame.display.set_caption(app_title) 106 | 107 | keyboard = prepare_keyboard() 108 | damper = False 109 | midi_ins, midi_in_info = prepare_midi_ins() 110 | 111 | tone_names_font = pygame.font.Font(None, base_size * 5) 112 | tone_names = ['C', '', 'D', '', 'E', 'F', '', 'G', '', 'A', '', 'B'] 113 | tone_name_sizes = [tone_names_font.size(tone_name) for tone_name in tone_names] 114 | tone_names = [tone_names_font.render(tone_name, True, tone_color[i]) for i, tone_name in enumerate(tone_names)] 115 | 116 | # main_loop 117 | font = pygame.font.Font(None, base_size * 4) 118 | app_title_text = font.render(app_title + ' / Frieve 2022', True, (255,255,255)) 119 | midi_info_text = font.render(midi_in_info, True, (255,255,255)) 120 | terminated = False 121 | last_time = time.perf_counter() - 10 122 | harmony_radius = np.zeros(128) 123 | mouse_click_note = -1 124 | while (not terminated): 125 | current_time = time.perf_counter() 126 | last_wait_ms = (current_time - last_time) * 1000 127 | last_time = current_time 128 | 129 | # handle events 130 | for event in pygame.event.get(): 131 | if event.type == QUIT: 132 | terminated = True 133 | elif event.type == KEYDOWN: 134 | if event.key == K_ESCAPE: 135 | terminated = True 136 | elif event.key == K_F11: 137 | full_screen = not full_screen 138 | if full_screen: 139 | screen = pygame.display.set_mode(screen_size, FULLSCREEN) 140 | else: 141 | screen = pygame.display.set_mode(screen_size) 142 | elif event.key == K_LCTRL or event.key == K_RCTRL: 143 | damper = True 144 | elif event.type == KEYUP: 145 | if event.key == K_LCTRL or event.key == K_RCTRL: 146 | damper = False 147 | elif event.type == MOUSEBUTTONDOWN and event.button == 1: 148 | for key in [key for key in keyboard[21:109] if key.black_key]: 149 | if event.pos[0] >= key.x - black_key_width / 2 and event.pos[0] < key.x + black_key_width / 2 and event.pos[1] >= keyboard_top and event.pos[1] < keyboard_top + black_key_height: 150 | mouse_click_note = key.note_no 151 | if mouse_click_note < 0: 152 | for key in [key for key in keyboard[21:109] if not key.black_key]: 153 | if event.pos[0] >= key.x - white_key_width / 2 and event.pos[0] < key.x + white_key_width / 2 and event.pos[1] >= keyboard_top and event.pos[1] < keyboard_top + white_key_height: 154 | mouse_click_note = key.note_no 155 | if mouse_click_note >= 0: 156 | keyboard[mouse_click_note].key_on = True 157 | keyboard[mouse_click_note].attack = 1.0 158 | keyboard[mouse_click_note].db = 96.0 159 | 160 | elif event.type == MOUSEBUTTONUP and event.button == 1: 161 | keyboard[mouse_click_note].key_on = False 162 | mouse_click_note = -1 163 | 164 | # midi in 165 | for midi_in in midi_ins: 166 | if midi_in.poll(): 167 | midi_events = midi_in.read(88) 168 | for midi_event in midi_events: 169 | if midi_event[0][0] % 16 != 9: # ignore ch 10 170 | midi_event_type = midi_event[0][0] // 16 171 | 172 | if midi_event_type == 9 and midi_event[0][2] > 0: # note on 173 | keyboard[midi_event[0][1]].key_on = True 174 | keyboard[midi_event[0][1]].attack = 1.0 175 | keyboard[midi_event[0][1]].db = max(math.log(midi_event[0][2] / 127) / math.log(2.0) * 6.0 + 96.0, keyboard[midi_event[0][1]].db) 176 | elif midi_event_type == 8 or (midi_event_type == 9 and midi_event[0][2] == 0): # note off 177 | keyboard[midi_event[0][1]].key_on = False 178 | elif midi_event_type == 11: 179 | cc_no = midi_event[0][1] 180 | if cc_no == 64: # damper 181 | damper = midi_event[0][2] > 0 182 | # print(midi_event) 183 | 184 | # update keyboard status 185 | overtone_list = [] 186 | harmony_buf = [[] for i in range(128)] 187 | harmony_list = [] 188 | for key in keyboard: 189 | key.harmony = 0.0 # reset harmony volume 190 | for key in keyboard: 191 | key.note_on = key.key_on or (key.note_on and damper) 192 | key.db -= last_wait_ms * (0.003 if key.note_on else 0.024) 193 | if key.db < 0.0: 194 | key.db = 0 195 | else: 196 | processed_note_no = [] 197 | for overtone_index in range(1, 384): 198 | energy = key.db - math.log(overtone_index) / math.log(2.0) * 3.0 199 | freq = key.note_no + math.log(overtone_index) / math.log(2.0) * 12 200 | if freq < min_note_no or energy < 60.0 or freq >= max_note_no: 201 | break 202 | overtone_list.append([key.note_no, freq, energy, key.attack]) 203 | 204 | if overtone_index <= 10: 205 | overtone_note_no = int(round(freq)) 206 | octave = overtone_index & (overtone_index - 1) == 0 207 | for harmony_note_no, harmony_overtone_index in harmony_buf[overtone_note_no]: 208 | harmony_octave = harmony_overtone_index & (harmony_overtone_index - 1) == 0 209 | if key.note_no % 12 != harmony_note_no % 12 and harmony_note_no not in processed_note_no and (not octave or not harmony_octave):# ignore octave 210 | brightness = min(key.db, keyboard[harmony_note_no].db) - 60 211 | keyboard[overtone_note_no].harmony = max(brightness, keyboard[overtone_note_no].harmony) 212 | if overtone_index > 1: 213 | harmony_list.append([key.note_no, overtone_index, brightness]) 214 | if harmony_overtone_index > 1: 215 | harmony_list.append([harmony_note_no, harmony_overtone_index, brightness]) 216 | processed_note_no.append(harmony_note_no) 217 | harmony_buf[overtone_note_no].append([key.note_no, overtone_index]) 218 | 219 | key.attack = max(key.attack - last_wait_ms * 0.02, 0.0) 220 | overtone_list.sort(key = lambda x: x[2]) # sort with energy 221 | harmony_list.sort(key = lambda x: x[2]) # sort with brightness 222 | 223 | # fill screen 224 | screen.fill((0,0,0)) 225 | 226 | # draw white keybed 227 | for key in [key for key in keyboard[21:109] if not key.black_key]: 228 | screen.fill((255, 255, 255) if not key.key_on else tone_color[key.note_no % 12], Rect(key.x - white_key_width // 2, keyboard_top, white_key_width, white_key_height - (base_size // 3 if key.attack > 0.0 else 0))) 229 | # draw black keybed 230 | for key in [key for key in keyboard[21:109] if key.black_key]: 231 | screen.fill((0, 0, 0), Rect(key.x - black_key_width // 2, keyboard_top, black_key_width, black_key_height)) 232 | if key.key_on: 233 | screen.fill(tone_color[key.note_no % 12], Rect(key.x - black_key_width / 2 + line_width, keyboard_top, black_key_width - line_width * 2, black_key_height - line_width - (base_size // 3 if key.attack > 0.0 else 0))) 234 | 235 | # draw harmony volume 236 | for key in keyboard[21:109]: 237 | key.display_harmony = key.display_harmony * 0.05 + ((key.harmony + np.random.randn() * 6)* 0.95 if key.harmony > 0.0 else 0.0) 238 | if key.display_harmony > base_size: 239 | pygame.draw.circle(screen, (160, 160, 160), (key.normalized_x + 1, energy_bottom + 1), key.display_harmony * base_size / 24, line_width * 2) 240 | 241 | # draw harmony 242 | for i, (note_no, overtone_index, brightness) in enumerate(harmony_list): 243 | x1 = note_no_to_x(note_no) 244 | overtone_note_no = note_no + math.log(overtone_index) / math.log(2.0) * 12 245 | x2 = note_no_to_x(overtone_note_no) 246 | height = base_size * 4 + math.log(overtone_index) / math.log(2.0) * base_size * 20 247 | color = (tone_color[note_no % 12] * brightness / 36).astype(np.int32) 248 | 249 | point = [] 250 | resolution = int(x2 - x1) // 6 251 | for i in range(resolution + 1): 252 | point.append((x1 + (x2 - x1) * i / resolution, energy_bottom - math.sqrt(math.sin(i / resolution * 3.1415926)) * height)) 253 | pygame.draw.lines(screen, color, False, point, line_width) 254 | 255 | # draw overtone_index 256 | if i >= len(harmony_list) - 16: 257 | overtone_text = font.render(str(overtone_index), True, color) 258 | overtone_text_size = font.size(str(overtone_index)) 259 | screen.blit(overtone_text, [(x1 + x2 - overtone_text_size[0]) // 2, energy_bottom - height - overtone_text_size[0] - base_size * 2]) 260 | 261 | # draw overtone (linear) 262 | for note_no, freq, energy, attack in overtone_list: 263 | height = (energy - 60.0) * base_size 264 | x = note_no_to_x(freq) 265 | pygame.draw.line(screen, tone_color[note_no % 12], (x, energy_bottom - height), (x, energy_bottom), bold_line_width if attack > 0 else line_width) 266 | 267 | # draw key center dot 268 | for key in keyboard[21:109]: 269 | pygame.draw.circle(screen, (255, 255, 255), (key.normalized_x + 1, energy_bottom + 1), (3 * base_size) // 8, 0) 270 | 271 | # draw overtone (polar) 272 | for i in range(6): 273 | x = math.sin(i / 12 * math.pi * 2.0) * polar_overtone_radius 274 | y = math.cos(i / 12 * math.pi * 2.0) * polar_overtone_radius 275 | pygame.draw.line(screen, (64, 64, 64), (polar_overtone_center[0] + x, polar_overtone_center[1] - y), (polar_overtone_center[0] - x, polar_overtone_center[1] + y), line_width) 276 | pygame.draw.circle(screen, (96, 96, 96), polar_overtone_center, polar_overtone_radius, line_width) 277 | for i in range(12): 278 | x = math.sin(i / 12 * math.pi * 2.0) * (polar_overtone_radius + base_size * 3) 279 | y = math.cos(i / 12 * math.pi * 2.0) * (polar_overtone_radius + base_size * 3) 280 | screen.blit(tone_names[i], [polar_overtone_center[0] + x - tone_name_sizes[i][0] / 2, polar_overtone_center[1] - y - tone_name_sizes[i][0] / 2]) 281 | for note_no, freq, energy, attack in overtone_list: 282 | r = (freq - 24) / (max_note_no - 24) * polar_overtone_radius 283 | angle = freq / 12 284 | pos = (polar_overtone_center[0] + math.sin(angle * math.pi * 2.0) * r + 1, polar_overtone_center[1] - math.cos(angle * math.pi * 2.0) * r + 1) 285 | if r > 0.0: 286 | radius = ((energy / 8 - 7 + attack * 2) * base_size) // 8 287 | if radius > 1: 288 | pygame.draw.circle(screen, tone_color[note_no % 12], pos, radius, 0) 289 | else: 290 | screen.set_at((int(pos[0]), int(pos[1])), tone_color[note_no % 12]) 291 | 292 | # display info 293 | screen.blit(app_title_text, [keyboard_margin_x, keyboard_margin_x]) 294 | screen.blit(midi_info_text, [keyboard_margin_x, keyboard_margin_x // 2 * 5]) 295 | 296 | # draw 297 | pygame.display.update() 298 | 299 | # wait 300 | pygame.time.wait(10) 301 | 302 | for midi_in in midi_ins: 303 | midi_in.close() 304 | pygame.midi.quit() 305 | pygame.quit() 306 | 307 | 308 | if __name__ == "__main__": 309 | main() 310 | -------------------------------------------------------------------------------- /icon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Frieve-A/harmony_visualizer/b492eb4bd25eb02adb3b94a020f7de371653cb5e/icon.ico -------------------------------------------------------------------------------- /icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Frieve-A/harmony_visualizer/b492eb4bd25eb02adb3b94a020f7de371653cb5e/icon.png -------------------------------------------------------------------------------- /icon_128x128.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Frieve-A/harmony_visualizer/b492eb4bd25eb02adb3b94a020f7de371653cb5e/icon_128x128.png -------------------------------------------------------------------------------- /images/harmony_visualizer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Frieve-A/harmony_visualizer/b492eb4bd25eb02adb3b94a020f7de371653cb5e/images/harmony_visualizer.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pygame 2 | numpy 3 | --------------------------------------------------------------------------------