├── .Rhistory ├── .github └── workflows │ └── release.yml ├── .gitignore ├── .python-version ├── README.md ├── bootstrap.sh ├── images ├── MIDISetup1.png └── MIDISetup2.png ├── mini_dj.spec ├── requirements.txt ├── src └── main.py └── tests └── test_main.py /.Rhistory: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JustinTjitra/trackstar/b90239257aa2d1b65b5d621b20e2e5d2477e96d0/.Rhistory -------------------------------------------------------------------------------- /.github/workflows/release.yml: -------------------------------------------------------------------------------- 1 | name: build-and-release 2 | 3 | on: 4 | push: 5 | tags: 6 | - "v*" 7 | 8 | jobs: 9 | build: 10 | runs-on: ${{ matrix.os }} 11 | strategy: 12 | matrix: 13 | os: [macos-13] 14 | steps: 15 | - uses: actions/checkout@v4 16 | - uses: actions/setup-python@v5 17 | with: 18 | python-version: "3.11" 19 | - name: Install dependencies 20 | run: | 21 | python -m pip install --upgrade pip pyinstaller 22 | if [ -f dj-hand-controller/requirements.txt ]; then pip install -r dj-hand-controller/requirements.txt; fi 23 | - name: Build binary (macOS) 24 | run: | 25 | cd dj-hand-controller 26 | pyinstaller -F -n mini-dj src/main.py 27 | - name: Upload artifact 28 | uses: actions/upload-artifact@v4 29 | with: 30 | name: mini-dj-${{ runner.os }} 31 | path: dj-hand-controller/dist/* 32 | 33 | release: 34 | needs: build 35 | runs-on: ubuntu-22.04 36 | steps: 37 | - uses: actions/download-artifact@v4 38 | with: 39 | path: dist 40 | - name: Create GitHub Release 41 | uses: softprops/action-gh-release@v2 42 | with: 43 | files: dist/**/* 44 | 45 | 46 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | venv/ 2 | __pycache__/ 3 | *.py[cod] 4 | -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | 3.8.16 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Watch the demo](https://img.youtube.com/vi/HVagoIxUzM0/maxresdefault.jpg)](https://youtu.be/HVagoIxUzM0) 2 | 3 | # DJ Hand Controller 4 | 5 | A gesture-to-MIDI/OSC DJ controller that uses a webcam to interpret hand movements and control DJ software. This project leverages Python with OpenCV for video processing and MediaPipe for hand tracking. 6 | 7 | ## Features (for now) 8 | 9 | - **Dual Deck Control:** Control two DJ decks independently using your left and right hands. 10 | - **Play/Pause Toggle:** Start and stop tracks with a simple gesture. 11 | - **Volume & Tempo Control:** Adjust gain and tempo with vertical hand movements. 12 | - **Multi-Band EQ Control:** Dynamically control Low, Mid, and High EQs. 13 | - **Stem Toggles:** Mute and unmute Drums, Vocals, and Instrumentals using gestures with the back of your hand. 14 | - **Looping:** Activate 1-beat, 2-beat and 4-beat loops. 15 | - **Beat Sync:** Synchronize the tempo of both decks. 16 | 17 | ## Prerequisites 18 | 19 | 1. **Python:** This project requires **Python 3.8, 3.9, or 3.10**. **Mediapipe does NOT support Python 3.11 or newer.** You can download compatible versions from [python.org](https://www.python.org/downloads/). 20 | 2. **Webcam:** A standard webcam is required for hand tracking. 21 | 3. **Virtual MIDI Driver:** You must have a virtual MIDI driver installed and running to route MIDI signals from this script to your DJ software. 22 | - **macOS:** The built-in **IAC Driver** is perfect. To enable it: 23 | 1. Open the "Audio MIDI Setup" application. 24 | 2. Go to `Window > Show MIDI Studio`. 25 | 3. Double-click on "IAC Driver". 26 | 4. Make sure the "Device is online" checkbox is checked. 27 | - **Windows:** A great free option is [**loopMIDI**](https://www.tobias-erichsen.de/software/loopmidi.html). Download, install, and create at least one MIDI port. 28 | 4. **DJ Software:** Your DJ software (e.g., Traktor, Serato, Rekordbox, VirtualDJ) must be configured to accept MIDI input from the virtual driver you set up. 29 | 30 | ## Setup 31 | 32 | 1. **Clone the repository:** 33 | 34 | ```bash 35 | git clone https://github.com/JustinTjitra/mini-dj.git 36 | cd mini-dj 37 | ``` 38 | 39 | 2. **Set up a Python 3.10/3.9/3.8 environment** 40 | 41 | > **Note:** Mediapipe only supports Python 3.8, 3.9, or 3.10. If your system default is Python 3.11 or newer, you must install an older version. 42 | 43 | - **macOS / Linux:** 44 | ```bash 45 | python3 --version 46 | python3.8 --version # or python3.10, python3.9 47 | python3.8 -m venv venv # or python3.10, python3.9 48 | source venv/bin/activate 49 | ``` 50 | - **Windows:** 51 | ```powershell 52 | py -0 53 | # or 54 | python --version 55 | py -3.8 --version # or -3.9, -3.10 56 | py -3.8 -m venv venv # or -3.9, -3.10 57 | .\venv\Scripts\activate 58 | ``` 59 | 60 | 3. **Install the required libraries:** 61 | ```bash 62 | pip install mediapipe opencv-python 63 | pip install -r requirements.txt 64 | ``` 65 | 66 | ## Example MIDI Setup 67 | 68 | ![The image here shows the functions under the DECK section of MIDI settings in rekordbox. To edit the MIDI IN and MIDI OUT, double click on the box, type in your corresponding value, and press ENTER.](images/MIDISetup1.png) 69 | 70 | The image here shows the functions under the DECK section of MIDI settings in rekordbox. To edit the MIDI IN and MIDI OUT, double click on the box, type in your corresponding value, and press ENTER. 71 | 72 | ![The image here shows the functions under the MIXER section of MIDI settings in rekordbox. To edit the MIDI IN and MIDI OUT, double click on the box, type in your corresponding value, and press ENTER.](images/MIDISetup2.png) 73 | 74 | The image here shows the functions under the MIXER section of MIDI settings in rekordbox. To edit the MIDI IN and MIDI OUT, double click on the box, type in your corresponding value, and press ENTER. 75 | 76 | ## Usage 77 | 78 | 1. Ensure your virtual MIDI driver is running and your DJ software is open and listening for MIDI input. 79 | 2. Run the script from your terminal: 80 | ```bash 81 | python src/main.py 82 | ``` 83 | 3. The script will list the available MIDI ports. Enter the number corresponding to your virtual MIDI driver and press Enter. 84 | 4. The webcam window will open, and you can begin controlling the software with your hands. Press `ESC` to quit. 85 | 86 | ## Gesture Guide 87 | 88 | IMPORTANT: Gestures are unfinished and are set to change according to their corresponding function. 89 | Ensure that your hand is not too far from the camera in order for it to read your gestures. 90 | More features to come... (longer loops, filters, cues, pads, etc.) 91 | 92 | ### Palm Facing Camera 93 | 94 | | Gesture | Action | Control Mechanism | 95 | | ------------------------------- | --------------------- | ---------------------------------------------------- | 96 | | **Open Hand** (Wrist in Zone) | Adjust Volume / EQ | Wrist in GAIN bar / Enters EQ Increase/Decrease zone | 97 | | **Closed Fist** (Wrist in Zone) | Adjust Tempo | Wrist in TEMPO bar | 98 | | **Index Finger Up** | Toggle Play/Pause | Rising edge of gesture | 99 | | **Index & Middle Fingers Up** | Toggle 1-Beat Loop | Rising edge of gesture | 100 | | **Middle, Ring, Pinky Up** | Toggle 2-Beat Loop | Rising edge of gesture | 101 | | **Ring & Pinky Up** | Toggle 4-Beat Loop | Rising edge of festure | 102 | | **Wrist in EQ Mode Button** | **Select EQ Mode:** | | 103 | |   ↳ Index Finger Up |   → Low EQ | Sets active EQ to Low | 104 | |   ↳ Index/Middle Up |   → Mid EQ | Sets active EQ to Mid | 105 | |   ↳ Mid/Ring/Pinky Up |   → High EQ | Sets active EQ to High | 106 | 107 | _Note: When an EQ mode (Low, Mid, or High) is selected, place your wrist in the "INC. EQ" or "DEQ. EQ" zones and adjust the value by changing the distance between your index and middle fingertips._ 108 | 109 | ### Back of Hand Facing Camera 110 | 111 | | Gesture | Action | Control Mechanism | 112 | | ----------------------------- | ------------------------- | ---------------------- | 113 | | **Index Finger Up** | Toggle Drums Mute | Rising edge of gesture | 114 | | **Index & Middle Fingers Up** | Toggle Vocals Mute | Rising edge of gesture | 115 | | **Middle, Ring, Pinky Up** | Toggle Instrumentals Mute | Rising edge of gesture | 116 | 117 | --- 118 | -------------------------------------------------------------------------------- /bootstrap.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -e 3 | 4 | PROJECT_NAME="dj-hand-controller" 5 | 6 | # 1. Make folder structure 7 | mkdir -p "$PROJECT_NAME"/{src,tests} 8 | cd "$PROJECT_NAME" 9 | 10 | # 2. Git & venv 11 | git init 12 | python -m venv venv 13 | 14 | # 3. .gitignore 15 | cat > .gitignore < README.md < src/main.py < tests/test_main.py < requirements.txt 66 | 67 | echo "✔️ Bootstrapped $PROJECT_NAME" 68 | -------------------------------------------------------------------------------- /images/MIDISetup1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JustinTjitra/trackstar/b90239257aa2d1b65b5d621b20e2e5d2477e96d0/images/MIDISetup1.png -------------------------------------------------------------------------------- /images/MIDISetup2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JustinTjitra/trackstar/b90239257aa2d1b65b5d621b20e2e5d2477e96d0/images/MIDISetup2.png -------------------------------------------------------------------------------- /mini_dj.spec: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | exceptiongroup==1.3.0 2 | iniconfig==2.1.0 3 | packaging==23.1 4 | pluggy==1.5.0 5 | pytest==8.3.5 6 | tomli==2.2.1 7 | typing_extensions==4.13.2 8 | opencv-python==4.11.0.86 9 | mediapipe==0.10.8 10 | mido==1.3.2 11 | numpy==1.26.4 12 | python-rtmidi==1.5.8 13 | -------------------------------------------------------------------------------- /src/main.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import mediapipe as mp 3 | import mido 4 | import numpy as np 5 | from collections import defaultdict 6 | import time 7 | 8 | # At the top of the file, after imports: 9 | FONT_FACE = cv2.FONT_HERSHEY_COMPLEX 10 | FONT_COLOR = (255,255,255) 11 | CURRENT_GESTURE_MSG = {'msg': ''} 12 | 13 | # ——— MIDI assignments ——— 14 | TOGGLE_NOTE = 60 # Play/Pause toggle (single button) 15 | VOLUME_CC = 7 # Mixer volume fader 16 | TEMPO_CC = 22 # Deck tempo slider 17 | LOOP_NOTE = 62 # Loop 1 beat toggle 18 | LOOP2_NOTE = 63 # Loop 2 beat toggle 19 | LOOP4_NOTE = 65 # Loop 4 beat toggle 20 | BEAT_SYNC_NOTE = 64 # Beat sync toggle 21 | LOW_EQ_CC = 14 # CC number for low EQ, adjust as needed 22 | MID_EQ_CC = 15 # CC number for mid EQ 23 | HIGH_EQ_CC = 16 # CC number for high EQ 24 | STEM_DRUMS_NOTE = 70 25 | STEM_VOCALS_NOTE = 71 26 | STEM_INST_NOTE = 72 27 | 28 | # ——— Gesture thresholds ——— 29 | OPEN_THRESH = 0.5 # threshold for open palm 30 | CLOSE_THRESH = 0.5 # threshold for closed fist 31 | DEBOUNCE_FRAMES = 2 32 | DEBOUNCE_TIME = 0 33 | VOL_THRESHOLD = 3 34 | TEMPO_THRESHOLD = 3 35 | LOOP_DEBOUNCE_FRAMES = 5 36 | LOOP_DEBOUNCE_TIME = 0.2 37 | BEAT_SYNC_DEBOUNCE_FRAMES = 1 38 | BEAT_SYNC_DEBOUNCE_TIME = 0 39 | 40 | # ——— Default-zone settings ——— 41 | # Default zones are small squares of size 25% of the smaller screen dimension 42 | ZONE_SIZE_FACTOR = 0.10 # fraction of min(width,height) 43 | DEFAULT_CC = 64 # middle CC value 44 | DEFAULT_VOLUME_WIDTH = 0.1 45 | 46 | # ——— Helper functions ——— 47 | def palm_distance(lm): 48 | wrist = np.array([lm[0].x, lm[0].y]) 49 | tips = np.array([[lm[i].x, lm[i].y] for i in (4,8,12,16,20)]) 50 | return np.mean(np.linalg.norm(tips - wrist, axis=1)) 51 | 52 | # hand-state checks 53 | def is_hand_open(lm): 54 | # Open if palm spread exceeds threshold 55 | return palm_distance(lm) > OPEN_THRESH 56 | 57 | def is_hand_closed(lm, h): 58 | # Closed if palm spread below threshold and all fingers folded 59 | if palm_distance(lm) > CLOSE_THRESH: 60 | return False 61 | for tip,pip in ((8,6),(12,10),(16,14),(20,18)): 62 | if lm[tip].y * h < lm[pip].y * h: 63 | return False 64 | return True 65 | 66 | # thumb up/down checks 67 | def is_thumbs_up(lm, h): 68 | if lm[4].y * h >= lm[2].y * h: 69 | return False 70 | for tip,pip in ((8,6),(12,10),(16,14),(20,18)): 71 | if lm[tip].y * h < lm[pip].y * h: 72 | return False 73 | return True 74 | 75 | def is_thumbs_down(lm, h): 76 | if lm[4].y * h <= lm[2].y * h: 77 | return False 78 | for tip,pip in ((8,6),(12,10),(16,14),(20,18)): 79 | if lm[tip].y * h < lm[pip].y * h: 80 | return False 81 | return True 82 | 83 | def is_index_up(lm, h): 84 | # index tip above PIP and other fingers folded 85 | if lm[8].y * h >= lm[6].y * h: 86 | return False 87 | for tip,pip in ((12,10),(16,14),(20,18)): 88 | if lm[tip].y * h < lm[pip].y * h: 89 | return False 90 | return True 91 | 92 | def is_two_fingers_up(lm, h): 93 | # index and middle tips above their PIPs 94 | if lm[8].y * h >= lm[6].y * h or lm[12].y * h >= lm[10].y * h: 95 | return False 96 | # other fingers folded 97 | for tip,pip in ((16,14),(20,18)): 98 | if lm[tip].y * h < lm[pip].y * h: 99 | return False 100 | return True 101 | 102 | def palm_normal(lm): 103 | # Use wrist (0), index_mcp (5), pinky_mcp (17) 104 | p0 = np.array([lm[0].x, lm[0].y, lm[0].z]) 105 | p1 = np.array([lm[5].x, lm[5].y, lm[5].z]) 106 | p2 = np.array([lm[17].x, lm[17].y, lm[17].z]) 107 | v1 = p1 - p0 108 | v2 = p2 - p0 109 | normal = np.cross(v1, v2) 110 | return normal 111 | 112 | def is_palm_facing_camera(lm, label=None): 113 | normal = palm_normal(lm) 114 | # For right hand, the normal direction is inverted 115 | if label == 'Right': 116 | return normal[2] > 0 117 | else: 118 | return normal[2] < 0 119 | 120 | def count_fingers_up_back(lm, h): 121 | # For back of hand, finger up means tip is above PIP in y (screen coordinates) 122 | up = [] 123 | for tip, pip in [(8,6), (12,10), (16,14), (20,18)]: 124 | up.append(lm[tip].y * h < lm[pip].y * h) 125 | return up # [index, middle, ring, pinky] 126 | 127 | def is_finger_up(lm, tip, pip, h, min_dist=0.04): 128 | return (lm[tip].y * h < lm[pip].y * h) and (abs(lm[tip].y - lm[pip].y) > min_dist) 129 | 130 | def is_index_and_middle_up(lm, h): 131 | # index and middle tips above their PIPs, others folded 132 | if lm[8].y * h >= lm[6].y * h or lm[12].y * h >= lm[10].y * h: 133 | return False 134 | for tip,pip in ((16,14),(20,18)): 135 | if lm[tip].y * h < lm[pip].y * h: 136 | return False 137 | return True 138 | 139 | def is_middle_ring_pinky_up(lm, h, min_dist=0.04): 140 | # middle, ring, pinky tips above their PIPs, index folded 141 | # Check that index is clearly folded 142 | if lm[8].y * h < lm[6].y * h + min_dist * h: 143 | return False 144 | # Check that middle, ring, pinky are clearly up 145 | for tip,pip in ((12,10),(16,14),(20,18)): 146 | if lm[tip].y * h >= lm[pip].y * h - min_dist * h: 147 | return False 148 | return True 149 | 150 | def is_ring_pinky_up(lm, h, min_dist=0.04): 151 | # ring and pinky tips above their PIPs, index and middle folded 152 | # Check that index and middle are clearly folded 153 | if lm[8].y * h < lm[6].y * h + min_dist * h or lm[12].y * h < lm[10].y * h + min_dist * h: 154 | return False 155 | # Check that ring and pinky are clearly up 156 | for tip,pip in ((16,14),(20,18)): 157 | if lm[tip].y * h >= lm[pip].y * h - min_dist * h: 158 | return False 159 | return True 160 | 161 | 162 | # Add a helper to update the gesture message 163 | def set_gesture_msg(msg): 164 | CURRENT_GESTURE_MSG['msg'] = msg 165 | 166 | 167 | def get_eq_zones(label, w, h): 168 | zone_size = int(min(w, h) * ZONE_SIZE_FACTOR) 169 | offset = 120 170 | if label == 'Left': 171 | tempo_bar_x = 250 172 | eq_x1 = tempo_bar_x + int(DEFAULT_VOLUME_WIDTH * w * 0.75) + 20 173 | eq_x2 = eq_x1 + zone_size 174 | else: 175 | tempo_bar_x = w - 250 - int(DEFAULT_VOLUME_WIDTH * w * 0.75) 176 | eq_x2 = tempo_bar_x - 20 177 | eq_x1 = eq_x2 - zone_size 178 | inc_y1 = int((h - 3 * zone_size - 40) / 2) + offset 179 | inc_y2 = inc_y1 + zone_size 180 | dec_y1 = inc_y2 + 20 181 | dec_y2 = dec_y1 + zone_size 182 | mode_y1 = dec_y2 + 20 183 | mode_y2 = mode_y1 + zone_size 184 | inc_zone = (eq_x1, inc_y1, eq_x2, inc_y2) 185 | dec_zone = (eq_x1, dec_y1, eq_x2, dec_y2) 186 | mode_zone = (eq_x1, mode_y1, eq_x2, mode_y2) 187 | return {'inc': inc_zone, 'dec': dec_zone, 'mode': mode_zone} 188 | 189 | # ——— Handlers ——— 190 | def handle_playpause(label, gesture, trackers, outport, palm_forward, lm, frame): 191 | if lm and frame is not None: 192 | h, w, _ = frame.shape 193 | zones = get_eq_zones(label, w, h) 194 | mode_zone = zones['mode'] 195 | x = lm[0].x * w 196 | y = lm[0].y * h 197 | in_mode = (mode_zone[0] <= x <= mode_zone[2] and mode_zone[1] <= y <= mode_zone[3]) 198 | if in_mode: 199 | return 200 | 201 | # gesture is 'IndexUp' for index finger up only 202 | playing = trackers['is_playing'] 203 | if gesture != 'IndexUp' or not palm_forward: 204 | trackers['last_gesture'][label] = None 205 | return 206 | # Toggle logic: only trigger on rising edge 207 | if trackers['last_gesture'][label] != gesture: 208 | trackers['last_gesture'][label] = gesture 209 | # toggle play/pause 210 | playing[label] = not playing[label] 211 | ch = 0 if label == 'Left' else 1 212 | outport.send(mido.Message('note_on', channel=ch, 213 | note=TOGGLE_NOTE, velocity=127)) 214 | state = 'Play' if playing[label] else 'Pause' 215 | print(f"{label} hand IndexUp → {state} (ch{ch})") 216 | set_gesture_msg(f"{label} hand IndexUp → {state} (ch{ch})") 217 | 218 | 219 | def handle_volume(label, open_flag, lm, vs, outport, frame): 220 | h, w, _ = frame.shape 221 | # Thinner bar 222 | bar_w = int(DEFAULT_VOLUME_WIDTH * w * 0.75) 223 | if label == 'Left': 224 | bar_x = 50 225 | else: 226 | bar_x = w - 50 - bar_w 227 | x = lm[0].x * w if open_flag else None 228 | # Transparent volume bar with colored border 229 | cv2.rectangle(frame, (bar_x, 500), (bar_x+bar_w, h), (0, 200, 255), 3) 230 | # Draw current indicator for both bars 231 | for side, color in zip(['Left', 'Right'], [(0,255,0), (255,0,0)]): 232 | last = vs['last'][side] 233 | if last >= 0: 234 | ch = 0 if side == 'Left' else 1 235 | top = 500 236 | bottom = frame.shape[0] 237 | if side == 'Left': 238 | x1, x2 = 50, 50+bar_w 239 | else: 240 | x1, x2 = w-50-bar_w, w-50 241 | y_pos = int(bottom - (last / 127) * (bottom - top)) 242 | cv2.rectangle(frame, (x1, y_pos-2), (x2, y_pos+2), (0,0,0), -1) 243 | # Draw modern GAIN text with background 244 | font = FONT_FACE 245 | font_scale = 1.2 246 | thickness = 1 247 | text = 'GAIN' 248 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 249 | while text_size[0] > bar_w - 20 and font_scale > 0.3: 250 | font_scale -= 0.05 251 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 252 | text_x = bar_x + (bar_w - text_size[0]) // 2 253 | text_y = 490 254 | 255 | # Simple text with shadow 256 | cv2.putText(frame, text, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 257 | cv2.putText(frame, text, (text_x, text_y), font, font_scale, (0, 200, 255), thickness, cv2.LINE_AA) 258 | if not open_flag: 259 | return 260 | if label == 'Left' and (x<50 or x > bar_w+50): 261 | return 262 | if label == 'Right' and (x < w - bar_w-50 or x > w-50): 263 | return 264 | y = lm[0].y * h 265 | top=500 266 | bottom = frame.shape[0] 267 | last = vs['last'][label] 268 | indicator_y = int(bottom - (last / 127) * (bottom - top)) 269 | touch_threshold = 50 # pixels 270 | if abs(y - indicator_y) > touch_threshold: 271 | return 272 | cc = int(np.clip((bottom - y) / (bottom - top) * 127, 0, 127)) 273 | if abs(cc - last) >= vs['threshold']: 274 | vs['last'][label] = cc 275 | ch = 0 if label == 'Left' else 1 276 | outport.send(mido.Message('control_change', channel=ch, 277 | control=VOLUME_CC, value=cc)) 278 | print(f"{label} Volume → {cc} (ch{ch})") 279 | set_gesture_msg(f"{label} Volume → {cc} (ch{ch})") 280 | 281 | def handle_tempo(label, closed_flag, lm, ts, outport, frame): 282 | h, w, _ = frame.shape 283 | # Thinner bar 284 | bar_w = int(DEFAULT_VOLUME_WIDTH * w * 0.75) 285 | if label == 'Left': 286 | bar_x = 250 287 | else: 288 | bar_x = w - 250 - bar_w 289 | x = lm[0].x * w if closed_flag else None 290 | # Transparent tempo bar with colored border 291 | cv2.rectangle(frame, (bar_x, 500), (bar_x+bar_w, h), (255, 100, 0), 3) 292 | # Draw current indicator for both bars 293 | for side, color in zip(['Left', 'Right'], [(0,255,0), (255,0,0)]): 294 | last = ts['last'][side] 295 | if last >= 0: 296 | ch = 0 if side == 'Left' else 1 297 | top = 500 298 | bottom = frame.shape[0] 299 | if side == 'Left': 300 | x1, x2 = 250, 250+bar_w 301 | else: 302 | x1, x2 = w-250-bar_w, w-250 303 | y_pos = int(bottom - (last / 127) * (bottom - top)) 304 | cv2.rectangle(frame, (x1, y_pos-2), (x2, y_pos+2), (0,0,0), -1) 305 | # Draw modern TEMPO text with background 306 | font = FONT_FACE 307 | font_scale = 1.2 308 | thickness = 1 309 | text = 'TEMPO' 310 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 311 | while text_size[0] > bar_w - 20 and font_scale > 0.3: 312 | font_scale -= 0.05 313 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 314 | text_x = bar_x + (bar_w - text_size[0]) // 2 315 | text_y = 490 316 | 317 | # Simple text with shadow 318 | cv2.putText(frame, text, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 319 | cv2.putText(frame, text, (text_x, text_y), font, font_scale, (255, 100, 0), thickness, cv2.LINE_AA) 320 | if not closed_flag: 321 | return 322 | if label == 'Left' and (x<250 or x > bar_w+250): 323 | return 324 | if label == 'Right' and (x < w - bar_w-250 or x > w-250): 325 | return 326 | y = lm[0].y * h 327 | top=500 328 | bottom = frame.shape[0] 329 | last = ts['last'][label] 330 | indicator_y = int(bottom - (last / 127) * (bottom - top)) 331 | touch_threshold = 50 # pixels 332 | if abs(y - indicator_y) > touch_threshold: 333 | return 334 | cc = int(np.clip((bottom - y) / (bottom - top) * 127, 0, 127)) 335 | if abs(cc - last) >= ts['threshold']: 336 | ts['last'][label] = cc 337 | ch = 0 if label == 'Left' else 1 338 | outport.send(mido.Message('control_change', channel=ch, 339 | control=TEMPO_CC, value=cc)) 340 | print(f"{label} Tempo → {cc} (ch{ch})") 341 | set_gesture_msg(f"{label} Tempo → {cc} (ch{ch})") 342 | 343 | def handle_eq(label, open_flag, lm, outport, frame, eq_state, gestures, palm_forward): 344 | h, w, _ = frame.shape 345 | zones = get_eq_zones(label, w, h) 346 | inc_zone = zones['inc'] 347 | dec_zone = zones['dec'] 348 | mode_zone = zones['mode'] 349 | zone_size = int(min(w, h) * ZONE_SIZE_FACTOR) 350 | 351 | # Draw transparent EQ zones with colored borders 352 | # INC zone 353 | cv2.rectangle(frame, (inc_zone[0], inc_zone[1]), (inc_zone[2], inc_zone[3]), (0, 200, 255), 2) 354 | 355 | # DEC zone 356 | cv2.rectangle(frame, (dec_zone[0], dec_zone[1]), (dec_zone[2], dec_zone[3]), (255, 100, 0), 2) 357 | 358 | # MODE zone 359 | cv2.rectangle(frame, (mode_zone[0], mode_zone[1]), (mode_zone[2], mode_zone[3]), (255, 0, 255), 2) 360 | 361 | # Draw text 362 | font = FONT_FACE 363 | thickness = 1 364 | for zone, text in [(inc_zone, 'INC. EQ'), (dec_zone, 'DEQ. EQ')]: 365 | font_scale = 0.9 366 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 367 | while text_size[0] > zone_size - 10 and font_scale > 0.3: 368 | font_scale -= 0.05 369 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 370 | text_x = zone[0] + (zone_size - text_size[0]) // 2 371 | text_y = zone[1] + (zone_size + text_size[1]) // 2 372 | cv2.putText(frame, text, (text_x, text_y), font, font_scale, FONT_COLOR, thickness, cv2.LINE_AA) 373 | mode_text = f"MODE: {eq_state['mode'][label].upper()}" 374 | font_scale = 0.7 375 | text_size = cv2.getTextSize(mode_text, font, font_scale, thickness)[0] 376 | while text_size[0] > zone_size - 10 and font_scale > 0.3: 377 | font_scale -= 0.05 378 | text_size = cv2.getTextSize(mode_text, font, font_scale, thickness)[0] 379 | text_x = mode_zone[0] + (zone_size - text_size[0]) // 2 380 | text_y = mode_zone[1] + (zone_size + text_size[1]) // 2 381 | cv2.putText(frame, mode_text, (text_x, text_y), font, font_scale, FONT_COLOR, thickness, cv2.LINE_AA) 382 | 383 | if not lm: 384 | return 385 | 386 | x = lm[0].x * w 387 | y = lm[0].y * h 388 | 389 | in_inc = (inc_zone[0] <= x <= inc_zone[2] and inc_zone[1] <= y <= inc_zone[3]) 390 | in_dec = (dec_zone[0] <= x <= dec_zone[2] and dec_zone[1] <= y <= dec_zone[3]) 391 | in_mode = (mode_zone[0] <= x <= mode_zone[2] and mode_zone[1] <= y <= mode_zone[3]) 392 | 393 | if in_mode and palm_forward: 394 | active_gesture = None 395 | if gestures['four_beat_loop']: 396 | active_gesture = 'high' 397 | elif gestures['two_beat_loop']: 398 | active_gesture = 'mid' 399 | elif gestures['one_beat_loop']: 400 | active_gesture = 'low' 401 | elif gestures['index_up']: 402 | active_gesture = 'low' 403 | 404 | if active_gesture and active_gesture != eq_state['last_gesture'][label]: 405 | eq_state['mode'][label] = active_gesture 406 | set_gesture_msg(f"{label} EQ Mode: {active_gesture.upper()}") 407 | print(f"{label} EQ Mode set to {active_gesture.upper()}") 408 | 409 | eq_state['last_gesture'][label] = active_gesture 410 | else: 411 | eq_state['last_gesture'][label] = None 412 | 413 | current_mode = eq_state['mode'][label] 414 | if current_mode != 'none' and (in_inc or in_dec): 415 | idx_tip = np.array([lm[8].x * w, lm[8].y * h]) 416 | mid_tip = np.array([lm[12].x * w, lm[12].y * h]) 417 | dist = np.linalg.norm(idx_tip - mid_tip) 418 | min_dist, max_dist = 0, 200 # Generic range 419 | cc_val = int(np.clip((dist - min_dist) / (max_dist - min_dist) * 127, 0, 127)) 420 | last = eq_state['last'][label] 421 | 422 | if abs(cc_val - last) > 2 or not eq_state['active'][label]: 423 | eq_state['last'][label] = cc_val 424 | eq_state['active'][label] = True 425 | ch = 0 if label == 'Left' else 1 426 | cc_map = {'low': LOW_EQ_CC, 'mid': MID_EQ_CC, 'high': HIGH_EQ_CC} 427 | control_cc = cc_map[current_mode] 428 | value_to_send = cc_val if in_inc else (127 - cc_val) 429 | outport.send(mido.Message('control_change', channel=ch, control=control_cc, value=value_to_send)) 430 | direction = '↑' if in_inc else '↓' 431 | print(f"{label} {current_mode.capitalize()} EQ {direction} {value_to_send} (ch{ch})") 432 | set_gesture_msg(f"{label} {current_mode.capitalize()} EQ {direction} {value_to_send} (ch{ch})") 433 | elif not (in_inc or in_dec): 434 | eq_state['active'][label] = False 435 | 436 | def handle_loop(label, gesture, trackers, outport, lm, frame): 437 | if lm and frame is not None: 438 | h, w, _ = frame.shape 439 | zones = get_eq_zones(label, w, h) 440 | mode_zone = zones['mode'] 441 | x = lm[0].x * w 442 | y = lm[0].y * h 443 | in_mode = (mode_zone[0] <= x <= mode_zone[2] and mode_zone[1] <= y <= mode_zone[3]) 444 | if in_mode: 445 | return 446 | 447 | # Only trigger for the one_beat_loop gesture (True/False) 448 | if not gesture: 449 | trackers['last_gesture'][label] = None 450 | return 451 | # Toggle logic: only trigger on rising edge 452 | if trackers['last_gesture'][label] != gesture: 453 | trackers['last_gesture'][label] = gesture 454 | # Toggle state 455 | state = trackers['state'][label] 456 | trackers['state'][label] = not state 457 | ch = 0 if label == 'Left' else 1 458 | outport.send(mido.Message('note_on', channel=ch, 459 | note=LOOP_NOTE, velocity=127)) 460 | print(f"{label} 1-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 461 | set_gesture_msg(f"{label} 1-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 462 | 463 | def handle_loop2(label, gesture, trackers, outport, lm, frame): 464 | if lm and frame is not None: 465 | h, w, _ = frame.shape 466 | zones = get_eq_zones(label, w, h) 467 | mode_zone = zones['mode'] 468 | x = lm[0].x * w 469 | y = lm[0].y * h 470 | in_mode = (mode_zone[0] <= x <= mode_zone[2] and mode_zone[1] <= y <= mode_zone[3]) 471 | if in_mode: 472 | return 473 | 474 | # Only trigger for the two_beat_loop gesture (True/False) 475 | if not gesture: 476 | trackers['last_gesture'][label] = None 477 | trackers['last_time'][label] = 0 478 | return 479 | 480 | # Add debouncing with time delay 481 | current_time = time.time() 482 | if current_time - trackers.get('last_time', {}).get(label, 0) < LOOP_DEBOUNCE_TIME: 483 | return 484 | 485 | # Toggle logic: only trigger on rising edge 486 | if trackers['last_gesture'][label] != gesture: 487 | trackers['last_gesture'][label] = gesture 488 | trackers['last_time'][label] = current_time 489 | # Toggle state 490 | state = trackers['state'][label] 491 | trackers['state'][label] = not state 492 | ch = 0 if label == 'Left' else 1 493 | outport.send(mido.Message('note_on', channel=ch, 494 | note=LOOP2_NOTE, velocity=127)) 495 | print(f"{label} 2-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 496 | set_gesture_msg(f"{label} 2-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 497 | 498 | def handle_loop4(label, gesture, trackers, outport, lm, frame): 499 | if lm and frame is not None: 500 | h, w, _ = frame.shape 501 | zones = get_eq_zones(label, w, h) 502 | mode_zone = zones['mode'] 503 | x = lm[0].x * w 504 | y = lm[0].y * h 505 | in_mode = (mode_zone[0] <= x <= mode_zone[2] and mode_zone[1] <= y <= mode_zone[3]) 506 | if in_mode: 507 | return 508 | 509 | # Only trigger for the four_beat_loop gesture (True/False) 510 | if not gesture: 511 | trackers['last_gesture'][label] = None 512 | trackers['last_time'][label] = 0 513 | return 514 | 515 | # Add debouncing with time delay 516 | current_time = time.time() 517 | if current_time - trackers.get('last_time', {}).get(label, 0) < LOOP_DEBOUNCE_TIME: 518 | return 519 | 520 | # Toggle logic: only trigger on rising edge 521 | if trackers['last_gesture'][label] != gesture: 522 | trackers['last_gesture'][label] = gesture 523 | trackers['last_time'][label] = current_time 524 | # Toggle state 525 | state = trackers['state'][label] 526 | trackers['state'][label] = not state 527 | ch = 0 if label == 'Left' else 1 528 | outport.send(mido.Message('note_on', channel=ch, 529 | note=LOOP4_NOTE, velocity=127)) 530 | print(f"{label} 4-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 531 | set_gesture_msg(f"{label} 4-beat loop {'ON' if not state else 'OFF'} (ch{ch})") 532 | 533 | 534 | def handle_beat_sync(label, _, trackers, outport, frame): 535 | last_time, sync_order, active, prev_in_button = ( 536 | trackers['last_time'], trackers['sync_order'], trackers['active'], trackers['prev_in_button'] 537 | ) 538 | now = time.time() 539 | 540 | # Draw sync toggle buttons as small circles 541 | h, w, _ = frame.shape 542 | circle_radius = 40 543 | y_center = 1000 544 | left_center = (450 + circle_radius, y_center) 545 | right_center = (w - 450 - circle_radius, y_center) 546 | 547 | # Draw modern circular buttons 548 | overlay = frame.copy() 549 | # Colors 550 | edge_color = (0, 200, 255) # Cyan 551 | fill_color = (20, 20, 20) # Dark background 552 | alpha_on = 0.9 553 | alpha_off = 0.3 554 | font = FONT_FACE 555 | font_scale = 0.5 556 | thickness = 2 557 | text_color = FONT_COLOR 558 | 559 | # For left button 560 | if not active['Left']: 561 | # Active state - bright 562 | cv2.circle(overlay, left_center, circle_radius, fill_color, -1) 563 | cv2.circle(overlay, left_center, circle_radius, edge_color, 4) 564 | # Glow effect 565 | cv2.circle(overlay, left_center, circle_radius-2, edge_color, 2) 566 | cv2.addWeighted(overlay, alpha_on, frame, 1-alpha_on, 0, frame) 567 | else: 568 | # Inactive state - dim 569 | cv2.circle(overlay, left_center, circle_radius, fill_color, -1) 570 | cv2.circle(overlay, left_center, circle_radius, (100, 100, 100), 2) 571 | cv2.addWeighted(overlay, alpha_off, frame, 1-alpha_off, 0, frame) 572 | # Draw 'BEAT' and 'SYNC' on separate lines, centered 573 | beat_text = 'BEAT' 574 | sync_text = 'SYNC' 575 | beat_size = cv2.getTextSize(beat_text, font, font_scale, thickness)[0] 576 | sync_size = cv2.getTextSize(sync_text, font, font_scale, thickness)[0] 577 | center_x = left_center[0] 578 | beat_y = left_center[1] - 4 579 | sync_y = left_center[1] + sync_size[1] + 4 580 | cv2.putText(frame, beat_text, (center_x - beat_size[0] // 2, beat_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 581 | cv2.putText(frame, sync_text, (center_x - sync_size[0] // 2, sync_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 582 | # For right button 583 | overlay = frame.copy() 584 | if not active['Right']: 585 | # Active state - bright 586 | cv2.circle(overlay, right_center, circle_radius, fill_color, -1) 587 | cv2.circle(overlay, right_center, circle_radius, edge_color, 4) 588 | # Glow effect 589 | cv2.circle(overlay, right_center, circle_radius-2, edge_color, 2) 590 | cv2.addWeighted(overlay, alpha_on, frame, 1-alpha_on, 0, frame) 591 | else: 592 | # Inactive state - dim 593 | cv2.circle(overlay, right_center, circle_radius, fill_color, -1) 594 | cv2.circle(overlay, right_center, circle_radius, (100, 100, 100), 2) 595 | cv2.addWeighted(overlay, alpha_off, frame, 1-alpha_off, 0, frame) 596 | beat_size = cv2.getTextSize(beat_text, font, font_scale, thickness)[0] 597 | sync_size = cv2.getTextSize(sync_text, font, font_scale, thickness)[0] 598 | center_x = right_center[0] 599 | beat_y = right_center[1] - 4 600 | sync_y = right_center[1] + sync_size[1] + 4 601 | cv2.putText(frame, beat_text, (center_x - beat_size[0] // 2, beat_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 602 | cv2.putText(frame, sync_text, (center_x - sync_size[0] // 2, sync_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 603 | 604 | # Check if wrist is in button area 605 | x = trackers['current_pos'][label][0] * w 606 | y = trackers['current_pos'][label][1] * h 607 | in_left = (np.linalg.norm(np.array([x, y]) - np.array(left_center)) <= circle_radius) 608 | in_right = (np.linalg.norm(np.array([x, y]) - np.array(right_center)) <= circle_radius) 609 | # Edge trigger logic 610 | if label == 'Left': 611 | if in_left and not prev_in_button['Left'] and now - last_time[label] > BEAT_SYNC_DEBOUNCE_TIME: 612 | if not active['Left']: 613 | active['Left'] = True 614 | if 'Left' not in sync_order: 615 | sync_order.append('Left') 616 | print('Left beat sync activated') 617 | outport.send(mido.Message('note_on', channel=0, note=BEAT_SYNC_NOTE, velocity=127)) 618 | else: 619 | active['Left'] = False 620 | if 'Left' in sync_order: 621 | sync_order.remove('Left') 622 | print('Left beat sync deactivated') 623 | outport.send(mido.Message('note_on', channel=0, note=BEAT_SYNC_NOTE, velocity=0)) 624 | last_time[label] = now 625 | prev_in_button['Left'] = in_left 626 | elif label == 'Right': 627 | if in_right and not prev_in_button['Right'] and now - last_time[label] > BEAT_SYNC_DEBOUNCE_TIME: 628 | if not active['Right']: 629 | active['Right'] = True 630 | if 'Right' not in sync_order: 631 | sync_order.append('Right') 632 | print('Right beat sync activated') 633 | outport.send(mido.Message('note_on', channel=1, note=BEAT_SYNC_NOTE, velocity=127)) 634 | else: 635 | active['Right'] = False 636 | if 'Right' in sync_order: 637 | sync_order.remove('Right') 638 | print('Right beat sync deactivated') 639 | outport.send(mido.Message('note_on', channel=1, note=BEAT_SYNC_NOTE, velocity=0)) 640 | last_time[label] = now 641 | prev_in_button['Right'] = in_right 642 | # Optionally, print which track is leading if both are active 643 | if active['Left'] and active['Right'] and len(sync_order) == 2: 644 | ch = 0 if sync_order[0] == 'Left' else 1 645 | print(f"Beat sync: {sync_order[0]} track leading (ch{ch})") 646 | set_gesture_msg(f"Beat sync: {sync_order[0]} track leading (ch{ch})") 647 | 648 | def handle_stem_toggles(label, lm, h, trackers, outport): 649 | # Only trigger if back of hand is facing camera 650 | if is_palm_facing_camera(lm, label): 651 | trackers['last_gesture'][label] = None 652 | return 653 | fingers = count_fingers_up_back(lm, h) 654 | # Only index up 655 | if fingers == [True, False, False, False]: 656 | gesture = 'drums' 657 | note = STEM_DRUMS_NOTE 658 | # Index + middle up 659 | elif fingers == [True, True, False, False]: 660 | gesture = 'vocals' 661 | note = STEM_VOCALS_NOTE 662 | # Robust instrumentals: index down, middle/ring/pinky up (with threshold) 663 | elif ( 664 | not is_finger_up(lm, 8, 6, h) and 665 | is_finger_up(lm, 12, 10, h) and 666 | is_finger_up(lm, 16, 14, h) and 667 | is_finger_up(lm, 20, 18, h) 668 | ): 669 | gesture = 'inst' 670 | note = STEM_INST_NOTE 671 | else: 672 | trackers['last_gesture'][label] = None 673 | return 674 | # Toggle logic: only trigger on rising edge 675 | if trackers['last_gesture'][label] != gesture: 676 | trackers['last_gesture'][label] = gesture 677 | # Toggle state 678 | state = trackers['state'][label][gesture] 679 | trackers['state'][label][gesture] = not state 680 | ch = 0 if label == 'Left' else 1 681 | velocity = 127 if not state else 0 # 127 = mute, 0 = unmute 682 | outport.send(mido.Message('note_on', channel=ch, note=note, velocity=velocity)) 683 | print(f"{label} hand: {'MUTE' if velocity==127 else 'UNMUTE'} {gesture.upper()} (ch{ch})") 684 | set_gesture_msg(f"{label} hand: {'MUTE' if velocity==127 else 'UNMUTE'} {gesture.upper()} (ch{ch})") 685 | 686 | def draw_stem_bars(frame, stem_state): 687 | h, w, _ = frame.shape 688 | bar_height = 25 689 | bar_width = 90 690 | gap = 12 691 | top_margin = 20 692 | left_margin = 50 693 | right_margin = 50 694 | colors = [(0, 100, 255), (0, 255, 100), (255, 100, 0)] # Blue, Green, Orange 695 | labels = ['DRUMS', 'VOCALS', 'INST'] 696 | 697 | # Left deck 698 | for i, (stem, color, label) in enumerate(zip(['drums','vocals','inst'], colors, labels)): 699 | x1 = left_margin + i*(bar_width+gap) 700 | y1 = top_margin 701 | x2 = x1 + bar_width 702 | y2 = y1 + bar_height 703 | 704 | # Create transparent rectangle with colored fill when active 705 | is_active = not stem_state['Left'][stem] 706 | 707 | if is_active: 708 | # Fill with color when active 709 | overlay = frame.copy() 710 | cv2.rectangle(overlay, (x1, y1), (x2, y2), color, -1) 711 | cv2.addWeighted(overlay, 0.7, frame, 0.3, 0, frame) 712 | else: 713 | # Transparent when inactive - just draw border 714 | cv2.rectangle(frame, (x1, y1), (x2, y2), (100, 100, 100), 2) 715 | 716 | # Text with shadow effect 717 | font = FONT_FACE 718 | font_scale = 0.6 719 | thickness = 1 720 | text_color = (255, 255, 255) if is_active else (150, 150, 150) 721 | 722 | # Shadow 723 | text_size = cv2.getTextSize(label, font, font_scale, thickness)[0] 724 | text_x = x1 + (bar_width - text_size[0]) // 2 725 | text_y = y1 + (bar_height + text_size[1]) // 2 726 | cv2.putText(frame, label, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 727 | cv2.putText(frame, label, (text_x, text_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 728 | 729 | # Right deck 730 | for i, (stem, color, label) in enumerate(zip(['drums','vocals','inst'], colors, labels)): 731 | x1 = w - right_margin - (3-i)*(bar_width+gap) 732 | y1 = top_margin 733 | x2 = x1 + bar_width 734 | y2 = y1 + bar_height 735 | 736 | # Create transparent rectangle with colored fill when active 737 | is_active = not stem_state['Right'][stem] 738 | 739 | if is_active: 740 | # Fill with color when active 741 | overlay = frame.copy() 742 | cv2.rectangle(overlay, (x1, y1), (x2, y2), color, -1) 743 | cv2.addWeighted(overlay, 0.7, frame, 0.3, 0, frame) 744 | else: 745 | # Transparent when inactive - just draw border 746 | cv2.rectangle(frame, (x1, y1), (x2, y2), (100, 100, 100), 2) 747 | 748 | # Text with shadow effect 749 | font = FONT_FACE 750 | font_scale = 0.6 751 | thickness = 1 752 | text_color = (255, 255, 255) if is_active else (150, 150, 150) 753 | 754 | # Shadow 755 | text_size = cv2.getTextSize(label, font, font_scale, thickness)[0] 756 | text_x = x1 + (bar_width - text_size[0]) // 2 757 | text_y = y1 + (bar_height + text_size[1]) // 2 758 | cv2.putText(frame, label, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 759 | cv2.putText(frame, label, (text_x, text_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 760 | 761 | def draw_loop_indicators(frame, loop_states): 762 | h, w, _ = frame.shape 763 | bar_height = 30 764 | bar_width = 70 765 | gap = 15 766 | top_margin = 60 767 | left_margin = 50 768 | right_margin = 50 769 | colors = [(0, 200, 255), (255, 200, 0), (0, 255, 100)] # Cyan, Yellow, Green 770 | labels = ['1B', '2B', '4B'] 771 | loop_keys = ['loop1', 'loop2', 'loop4'] 772 | 773 | # Left deck 774 | for i, (loop_key, color, label) in enumerate(zip(loop_keys, colors, labels)): 775 | x1 = left_margin + i*(bar_width+gap) 776 | y1 = top_margin 777 | x2 = x1 + bar_width 778 | y2 = y1 + bar_height 779 | 780 | is_active = loop_states['Left'][loop_key] 781 | 782 | if is_active: 783 | # Fill with color when active 784 | overlay = frame.copy() 785 | cv2.rectangle(overlay, (x1, y1), (x2, y2), color, -1) 786 | cv2.addWeighted(overlay, 0.8, frame, 0.2, 0, frame) 787 | else: 788 | # Transparent when inactive - just draw border 789 | cv2.rectangle(frame, (x1, y1), (x2, y2), (80, 80, 80), 2) 790 | 791 | # Text with shadow 792 | font = FONT_FACE 793 | font_scale = 0.7 794 | thickness = 1 795 | text_color = (255, 255, 255) if is_active else (120, 120, 120) 796 | 797 | text_size = cv2.getTextSize(label, font, font_scale, thickness)[0] 798 | text_x = x1 + (bar_width - text_size[0]) // 2 799 | text_y = y1 + (bar_height + text_size[1]) // 2 800 | 801 | # Shadow 802 | cv2.putText(frame, label, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 803 | cv2.putText(frame, label, (text_x, text_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 804 | 805 | # Right deck 806 | for i, (loop_key, color, label) in enumerate(zip(loop_keys, colors, labels)): 807 | x1 = w - right_margin - (3-i)*(bar_width+gap) 808 | y1 = top_margin 809 | x2 = x1 + bar_width 810 | y2 = y1 + bar_height 811 | 812 | is_active = loop_states['Right'][loop_key] 813 | 814 | if is_active: 815 | # Fill with color when active 816 | overlay = frame.copy() 817 | cv2.rectangle(overlay, (x1, y1), (x2, y2), color, -1) 818 | cv2.addWeighted(overlay, 0.8, frame, 0.2, 0, frame) 819 | else: 820 | # Transparent when inactive - just draw border 821 | cv2.rectangle(frame, (x1, y1), (x2, y2), (80, 80, 80), 2) 822 | 823 | # Text with shadow 824 | font = FONT_FACE 825 | font_scale = 0.7 826 | thickness = 1 827 | text_color = (255, 255, 255) if is_active else (120, 120, 120) 828 | 829 | text_size = cv2.getTextSize(label, font, font_scale, thickness)[0] 830 | text_x = x1 + (bar_width - text_size[0]) // 2 831 | text_y = y1 + (bar_height + text_size[1]) // 2 832 | 833 | # Shadow 834 | cv2.putText(frame, label, (text_x+1, text_y+1), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 835 | cv2.putText(frame, label, (text_x, text_y), font, font_scale, text_color, thickness, cv2.LINE_AA) 836 | 837 | # ——— Main ——— 838 | def main(): 839 | # Let user select MIDI port 840 | available_ports = mido.get_output_names() 841 | if not available_ports: 842 | print("Error: No MIDI output ports found.") 843 | print("Please make sure a virtual MIDI driver (like 'IAC Driver' on Mac or 'loopMIDI' on Windows) is installed and running.") 844 | return 845 | 846 | print("Please select a MIDI output port:") 847 | for i, port_name in enumerate(available_ports): 848 | print(f" {i}: {port_name}") 849 | 850 | selected_port_index = -1 851 | while True: 852 | try: 853 | raw_input = input(f"Enter the number of the port you want to use (0-{len(available_ports)-1}): ") 854 | selected_port_index = int(raw_input) 855 | if 0 <= selected_port_index < len(available_ports): 856 | break 857 | else: 858 | print(f"Invalid number. Please enter a number between 0 and {len(available_ports)-1}.") 859 | except ValueError: 860 | print("Invalid input. Please enter a number.") 861 | except (KeyboardInterrupt, EOFError): 862 | print("\nSelection cancelled. Exiting.") 863 | return 864 | 865 | selected_port_name = available_ports[selected_port_index] 866 | print(f"Opening port: {selected_port_name}") 867 | 868 | try: 869 | out = mido.open_output(selected_port_name) 870 | except Exception as e: 871 | print(f"Error opening MIDI port: {e}") 872 | return 873 | 874 | hands = mp.solutions.hands.Hands( 875 | static_image_mode=False, 876 | max_num_hands=2, 877 | min_detection_confidence=0.7, 878 | model_complexity=0 879 | ) 880 | cap = cv2.VideoCapture(0) 881 | if not cap.isOpened(): 882 | print("Cannot open camera") 883 | return 884 | 885 | pp = { 886 | 'prev_thumb': {'Left':None,'Right':None}, 887 | 'last_raw': {'Left':None,'Right':None}, 888 | 'counts': defaultdict(int), 889 | 'last_time': defaultdict(lambda:0.0), 890 | 'is_playing': {'Left':False,'Right':False}, 891 | 'last_gesture': {'Left': None, 'Right': None} 892 | } 893 | loop_st = { 894 | 'last_gesture': {'Left': None, 'Right': None}, 895 | 'state': {'Left': False, 'Right': False} 896 | } 897 | loop2_st = { 898 | 'last_gesture': {'Left': None, 'Right': None}, 899 | 'last_time': {'Left': 0, 'Right': 0}, 900 | 'state': {'Left': False, 'Right': False} 901 | } 902 | loop4_st = { 903 | 'last_gesture': {'Left': None, 'Right': None}, 904 | 'last_time': {'Left': 0, 'Right': 0}, 905 | 'state': {'Left': False, 'Right': False} 906 | } 907 | beat_sync_st = { 908 | 'last_raw': {'Left':False,'Right':False}, 909 | 'counts': defaultdict(int), 910 | 'last_time': defaultdict(lambda:0.0), 911 | 'sync_order': [], 912 | 'current_pos': {'Left': (0,0), 'Right': (0,0)}, 913 | 'active': {'Left': False, 'Right': False}, 914 | 'prev_in_button': {'Left': False, 'Right': False} 915 | } 916 | stem_trackers = { 917 | 'last_gesture': {'Left': None, 'Right': None}, 918 | 'state': { 919 | 'Left': {'drums': False, 'vocals': False, 'inst': False}, 920 | 'Right': {'drums': False, 'vocals': False, 'inst': False} 921 | } 922 | } 923 | 924 | # Combined loop states for visual feedback 925 | loop_states = { 926 | 'Left': { 927 | 'loop1': False, 'loop2': False, 'loop4': False 928 | }, 929 | 'Right': { 930 | 'loop1': False, 'loop2': False, 'loop4': False 931 | } 932 | } 933 | 934 | vol = {'last':{'Left':64,'Right':64}, 'threshold':VOL_THRESHOLD} 935 | tmp = {'last':{'Left':64,'Right':64}, 'threshold':TEMPO_THRESHOLD} 936 | df = {'last':{'Left':False,'Right':False}} 937 | eq_st = { 938 | 'mode': {'Left': 'none', 'Right': 'none'}, 939 | 'last': {'Left': -1, 'Right': -1}, 940 | 'active': {'Left': False, 'Right': False}, 941 | 'last_gesture': {'Left': None, 'Right': None} 942 | } 943 | 944 | while True: 945 | ret, frame = cap.read() 946 | if not ret: 947 | break 948 | # Mirror the camera feed 949 | frame = cv2.flip(frame, 1) 950 | h, w, _ = frame.shape 951 | rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) 952 | res = hands.process(rgb) 953 | 954 | draw_stem_bars(frame, stem_trackers['state']) 955 | draw_loop_indicators(frame, loop_states) 956 | 957 | if res.multi_hand_landmarks: 958 | for lm, hd in zip(res.multi_hand_landmarks, res.multi_handedness): 959 | label = hd.classification[0].label 960 | palm_forward = is_palm_facing_camera(lm.landmark, label) 961 | open_flag = is_hand_open(lm.landmark) 962 | closed_flag = is_hand_closed(lm.landmark, h) 963 | 964 | # detect gestures 965 | idx_up = is_index_up(lm.landmark,h) 966 | one_beat_loop = is_index_and_middle_up(lm.landmark, h) 967 | two_beat_loop = is_middle_ring_pinky_up(lm.landmark, h) 968 | four_beat_loop = is_ring_pinky_up(lm.landmark, h) 969 | 970 | eq_gestures = { 971 | 'index_up': idx_up, 972 | 'one_beat_loop': one_beat_loop, 973 | 'two_beat_loop': two_beat_loop, 974 | 'four_beat_loop': four_beat_loop 975 | } 976 | 977 | # Store current hand position for beat sync 978 | beat_sync_st['current_pos'][label] = (lm.landmark[0].x, lm.landmark[0].y) 979 | 980 | # Consolidated EQ handler 981 | handle_eq(label, open_flag, lm.landmark, out, frame, eq_st, eq_gestures, palm_forward) 982 | 983 | # Other handlers are conditional on the gesture not being consumed by EQ 984 | play_pause_gesture = idx_up and not one_beat_loop 985 | handle_playpause(label, 'IndexUp' if play_pause_gesture and palm_forward else None, pp, out, palm_forward, lm.landmark, frame) 986 | 987 | if palm_forward: 988 | handle_loop(label, one_beat_loop, loop_st, out, lm.landmark, frame) 989 | handle_loop2(label, two_beat_loop, loop2_st, out, lm.landmark, frame) 990 | handle_loop4(label, four_beat_loop, loop4_st, out, lm.landmark, frame) 991 | 992 | # Update visual feedback states 993 | loop_states[label]['loop1'] = loop_st['state'][label] 994 | loop_states[label]['loop2'] = loop2_st['state'][label] 995 | loop_states[label]['loop4'] = loop4_st['state'][label] 996 | 997 | handle_beat_sync(label, None, beat_sync_st, out, frame) 998 | handle_stem_toggles(label, lm.landmark, h, stem_trackers, out) 999 | if not df['last'][label]: 1000 | handle_volume(label,open_flag,lm.landmark,vol,out,frame) 1001 | handle_tempo(label,closed_flag,lm.landmark,tmp,out,frame) 1002 | 1003 | mp.solutions.drawing_utils.draw_landmarks(frame, lm, mp.solutions.hands.HAND_CONNECTIONS) 1004 | 1005 | # Draw the current gesture message with modern styling 1006 | if CURRENT_GESTURE_MSG['msg']: 1007 | font = FONT_FACE 1008 | font_scale = 1.2 1009 | thickness = 1 1010 | text = CURRENT_GESTURE_MSG['msg'] 1011 | h, w, _ = frame.shape 1012 | text_size = cv2.getTextSize(text, font, font_scale, thickness)[0] 1013 | text_x = (w - text_size[0]) // 2 1014 | text_y = 150 1015 | 1016 | # Background rectangle with rounded corners effect 1017 | padding = 15 1018 | bg_x1 = text_x - padding 1019 | bg_y1 = text_y - text_size[1] - padding 1020 | bg_x2 = text_x + text_size[0] + padding 1021 | bg_y2 = text_y + padding 1022 | 1023 | # Draw simple background 1024 | cv2.rectangle(frame, (bg_x1, bg_y1), (bg_x2, bg_y2), (20, 20, 20), -1) 1025 | cv2.rectangle(frame, (bg_x1, bg_y1), (bg_x2, bg_y2), (0, 200, 255), 3) 1026 | 1027 | # Draw text with shadow 1028 | cv2.putText(frame, text, (text_x+2, text_y+2), font, font_scale, (0, 0, 0), thickness, cv2.LINE_AA) 1029 | cv2.putText(frame, text, (text_x, text_y), font, font_scale, (255, 255, 255), thickness, cv2.LINE_AA) 1030 | 1031 | cv2.imshow("DJ Controller", frame) 1032 | if cv2.waitKey(1) & 0xFF == 27: 1033 | break 1034 | 1035 | cap.release() 1036 | cv2.destroyAllWindows() 1037 | hands.close() 1038 | out.close() 1039 | 1040 | if __name__ == "__main__": 1041 | main() 1042 | -------------------------------------------------------------------------------- /tests/test_main.py: -------------------------------------------------------------------------------- 1 | def test_dummy(): 2 | assert True 3 | --------------------------------------------------------------------------------