├── .gitignore ├── README.md ├── communicate.py ├── mcwebcam.py ├── mcwebcamWin10.py └── pilibs └── adafruit_hid ├── __init__.py ├── consumer_control.py ├── consumer_control_code.py ├── gamepad.py ├── keyboard.py ├── keyboard_layout_us.py ├── keycode.py └── mouse.py /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | .DS_Store 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | **WARNING:** If you come here from 2022 posts (Reddit posts, PhoenixSC), this is NOT what you're looking for. This is an old version which does not have the same functionality as what you've seen. It's pretty buggy and is a nightmare to set up. The new version will be released in the second half of summer 2022. To stay tuned for updates, subscribe to me on YouTube. Thank you! 2 | 3 | 4 | This is the official repo for my video "Playing MINECRAFT with a WEBCAM" on YouTube 5 | Original video can be found here: https://youtu.be/701TPxL0Skg 6 | Reddit Post: https://www.reddit.com/r/Minecraft/comments/o7vjh5/playing_with_a_webcam_is_really_fun/ 7 | 8 | A few things before attempting to use this program: 9 | 10 | 1. You must have a Python 3 interpreter installed. Found here https://www.python.org/downloads/ 11 | 2. A few packages must be installed prior to launching the program 12 | `pip install opencv-python` 13 | `pip install numpy` 14 | `pip install pyautogui` 15 | `pip install mediapipe` 16 | `pip install serial` 17 | 3. If you are on a (non Windows) Unix based system like MacOS or Linux, you will need a Raspberry Pi Pico to control the mouse input. To set up the Pico: 18 | Install CircuitPython by pressing and holding the button on the pi, while plugging it into the computer. A disk will be shown connected to your computer. Drag the UF2 to the disk named something close to "RPI". The CircuitPython UF2 can be found here https://circuitpython.org/board/raspberry_pi_pico/ 19 | 20 | Install the necessary libraries on the Raspberry Pi Pico. They can be found in the "pilibs" folder in the repo. Just drag them to the Pi's "lib" folder. 21 | 22 | Every time you plug your pico into your computer, run these 3 lines of code after connecting to the Pi's python REPL (`screen /dev/tty.usbmodem*` on MacOS) 23 | `import usb_hid` 24 | `from adafruit_hid.mouse import Mouse` 25 | `mouse = Mouse(usb_hid.devices)` 26 | 27 | 4. If you are using a Windows computer, install the following package 28 | `pip install pydirectinput` 29 | Also turn on Raw Input in Minecraft controls menu 30 | 5. To run the program, run `python mcwebcam.py` from the program folder if you are on a Unix based computer, or `python mcwebcamWin10.py` if you are on a Windows computer. Then quickly tab to Minecraft. TO STOP THE PROGRAM, ALT TAB/COMMAND TAB INTO CMD OR TERMINAL AND PRESS CTRL+C AT THE SAME TIME. YOU MIGHT NEED TO SPAM IT. 31 | ALTERNATIVELY, MOVE THE MOUSE TO A CORNER OF THE SCREEN (0,0). I AM NOT RESPONSIBLE FOR INABILITY TO STOP THE PROGRAM. 32 | 33 | The control might either be too sensitive or not sensitive at all, you can change this by editing the respective python file in a text editor or IDE. The scale factors can be found on line 196 for mcwebcam.py and line 201 for mcwebcamWin10.py. There will be two lines under the format `x = x * 18`, one for x and one for y. You can change the number that the coordinates are multiplied by. 34 | 35 | This code is licensed under Apache 2.0 36 | 37 | The setup for the python version is overly long and requires lots of technical knowledge to get set up, so I plan on making a more polished version in C++ in the near future. 38 | 39 | I AM NOT RESPONSIBLE FOR ANY HARM DONE TO PERSONS OR PROPERTY AS A RESULT OF THIS CODE. 40 | -------------------------------------------------------------------------------- /communicate.py: -------------------------------------------------------------------------------- 1 | # Code written by Rishabh Roy 2 | 3 | # Copyright 2021 Rishabh Roy 4 | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | import os 18 | import time 19 | import serial 20 | 21 | def sendCommand(command): 22 | os.system('screen -S pico -X stuff \'' + command + '\\r\'') 23 | 24 | def openConnection(): 25 | os.system('screen -d -m -S pico /dev/tty.usbmodem* 9600') 26 | sendCommand('import usb_hid\\r') 27 | sendCommand('from adafruit_hid.mouse import Mouse\\r') 28 | sendCommand('mouse = Mouse(usb_hid.devices)\\r') 29 | print('initialized') 30 | 31 | def closeConnection(): 32 | os.system('screen -XS pico quit') 33 | print('uninitialized') 34 | 35 | def moveMouse(x, y): 36 | print(str(x) + ' ' + str(y)) 37 | sendCommand('mouse.move(x=' + str(x) + ',y=' + str(y) + ')') 38 | 39 | def getRelativeCoords(curx, cury, x, y): 40 | relx = x - curx 41 | rely = y - cury 42 | 43 | return relx, rely 44 | 45 | def sendSerialCommand(command, ser): 46 | command = command + '\r' 47 | command = str.encode(command) 48 | ser.write(command) # write a string 49 | 50 | def openSerialConnection(): 51 | ser = serial.Serial('/dev/tty.usbmodem1112301', 115200) # open serial port 52 | return ser 53 | 54 | def closeSerialConnection(ser): 55 | ser.close() 56 | 57 | def serialMoveMouse(ser, x, y): 58 | if x == 0: 59 | command = "mouse.move(y=" + str(y) + ")\r" 60 | else: 61 | if y == 0: 62 | command = "mouse.move(x=" + str(x) + ")\r" 63 | else: 64 | command = "mouse.move(x=" + str(x) + ", y=" + str(y) + ")\r" 65 | command = str.encode(command) 66 | ser.write(command) 67 | 68 | def getCoordinatePlane(x, y, dimx=960, dimy=540): 69 | x = dimx -x 70 | y = dimy -y 71 | 72 | return x, y 73 | 74 | -------------------------------------------------------------------------------- /mcwebcam.py: -------------------------------------------------------------------------------- 1 | # Code written by Rishabh Roy 2 | 3 | # Copyright 2021 Rishabh Roy 4 | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | # Import Statements 18 | import cv2 as cv 19 | import numpy as np 20 | import mediapipe as mp 21 | import pyautogui as gui 22 | import time 23 | import communicate 24 | import serial 25 | import itertools 26 | import signal 27 | import sys 28 | 29 | # Cursor smoothening value 30 | smoothening = 15 31 | 32 | # Handle CTRL-C to exit gracefully 33 | def signal_handler(signal, frame): 34 | print("Exiting program") 35 | cap.release() 36 | cv.destroyAllWindows() 37 | communicate.closeSerialConnection(ser) 38 | sys.exit() 39 | 40 | signal.signal(signal.SIGINT, signal_handler) 41 | 42 | # Set mouse movement pause to 0 for smooth movement 43 | gui.PAUSE = 0 44 | 45 | # Get solution objects from mediapipe 46 | mp_drawing = mp.solutions.drawing_utils 47 | mp_pose = mp.solutions.pose 48 | 49 | # Get video capture 50 | cap = cv.VideoCapture(0) 51 | ser = communicate.openSerialConnection() 52 | 53 | # Variables to keep track of where the crosshair is 54 | findx = 0 55 | findy = 0 56 | 57 | # Old values to average 58 | oldx = 0 59 | oldy = 0 60 | 61 | # Shoulder movement list 62 | walkingdiff = [] 63 | jumpingdiff = [] 64 | miningdiff = [] 65 | placingdiff = [] 66 | 67 | walk = False 68 | jump = False 69 | mining = False 70 | placing = False 71 | 72 | # Main while loop 73 | with mp_pose.Pose(min_detection_confidence=0.5, min_tracking_confidence=0.5, static_image_mode=False, smooth_landmarks=True) as pose: 74 | for i in itertools.count(): 75 | 76 | # Get frame 77 | ret, frame = cap.read() 78 | 79 | # Flip frame 80 | frame = cv.flip(frame, 1) 81 | 82 | # Get pose data 83 | results = pose.process(cv.cvtColor(frame, cv.COLOR_BGR2RGB)) 84 | 85 | # Draw landmarks 86 | mp_drawing.draw_landmarks(frame, results.pose_landmarks, mp_pose.POSE_CONNECTIONS) 87 | 88 | # pose_landmarks variable 89 | pose_landmarks = results.pose_landmarks 90 | 91 | # print pose landmarks 92 | if not results.pose_landmarks: 93 | continue 94 | 95 | # Get image dimensions 96 | image_height, image_width, _ = frame.shape 97 | 98 | # region Mining and using 99 | righthandy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_INDEX].y * image_height 100 | lefthandy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_INDEX].y * image_height 101 | 102 | miningdiff.append(righthandy) 103 | 104 | if len(miningdiff) < 5: 105 | continue 106 | 107 | miningworkingdata = np.diff(miningdiff) 108 | miningworkingdata = abs(miningworkingdata) > 100 109 | 110 | if True in miningworkingdata: 111 | #gui.click(button='left') 112 | gui.mouseDown(button='left') 113 | mining = True 114 | if not True in miningworkingdata and mining: 115 | gui.mouseUp(button='left') 116 | mining = False 117 | 118 | if len(miningdiff) == 20: 119 | miningdiff = [] 120 | 121 | placingdiff.append(righthandy) 122 | 123 | if len(placingdiff) < 5: 124 | continue 125 | 126 | if lefthandy < 400 and not placing: 127 | gui.click(button='right') 128 | placing = True 129 | elif placing: 130 | placing = False 131 | 132 | 133 | # endregion 134 | 135 | # region Walking + Jumping 136 | walkingx = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * image_width 137 | walkingdiff.append(walkingx) 138 | 139 | if len(walkingdiff) < 5: 140 | continue 141 | 142 | walkingworkingdata = np.diff(walkingdiff) 143 | walkingworkingdata = abs(walkingworkingdata) > 15 144 | if True in walkingworkingdata and not mining: 145 | #print('w!') 146 | gui.keyDown('alt') 147 | gui.keyDown('w') 148 | walk = True 149 | 150 | if not True in walkingworkingdata: 151 | #print('not w') 152 | gui.keyUp('w') 153 | gui.keyUp('alt') 154 | walk = False 155 | 156 | # Reset every 60 samples 157 | if len(walkingdiff) == 15: 158 | walkingdiff = [] 159 | 160 | 161 | # Jumping 162 | jumpingy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * image_height 163 | jumpingdiff.append(jumpingy) 164 | 165 | jumpingworkingdata = np.diff(jumpingdiff) 166 | jumpingworkingdata = abs(jumpingworkingdata) > 40 167 | 168 | if True in jumpingworkingdata: 169 | gui.keyDown('space') 170 | jump = True 171 | else: 172 | gui.keyUp('space') 173 | jump = False 174 | 175 | if len(jumpingdiff) == 20: 176 | jumpingdiff = [] 177 | 178 | # endregion 179 | 180 | # region Move Mouse 181 | x = results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].x * image_width 182 | y = results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].y * image_height 183 | x = int(x) 184 | y = int(y) 185 | 186 | # Scale to display size 187 | x = x * 1.5 188 | y = y * 1.5 189 | 190 | x = oldx + (x - oldx) / smoothening 191 | y = oldy + (y - oldy) / smoothening 192 | 193 | oldx = x 194 | oldy = y 195 | 196 | # Increase movement (trial and error) 197 | x = x * 18 198 | y = y * 15 199 | 200 | # Get the relative coordinates to move to 201 | x, y = communicate.getRelativeCoords(findx, findy, x, y) 202 | 203 | # Change to integer 204 | x = int(x) 205 | y = int(y) 206 | 207 | # Move the mouse 208 | if not walk and not jump and not mining: 209 | communicate.serialMoveMouse(ser, x, y) 210 | 211 | # Update the find coordinates to keep track of location 212 | findx = findx + x 213 | findy = findy + y 214 | 215 | # endregion 216 | 217 | #frame = cv.resize(frame, (640, 360)) 218 | 219 | # Show frame 220 | #cv.imshow("OpenCV", frame) 221 | 222 | # Exit key 223 | if cv.waitKey(20) & 0xFF==ord('d'): 224 | break 225 | 226 | # Exit program 227 | cap.release() 228 | cv.destroyAllWindows() 229 | communicate.closeSerialConnection(ser) -------------------------------------------------------------------------------- /mcwebcamWin10.py: -------------------------------------------------------------------------------- 1 | # Code written by Rishabh Roy 2 | 3 | # Copyright 2021 Rishabh Roy 4 | 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | # Import Statements 18 | import cv2 as cv 19 | import numpy as np 20 | import mediapipe as mp 21 | import pyautogui as gui 22 | import time 23 | import serial 24 | import itertools 25 | import signal 26 | import sys 27 | import pydirectinput 28 | 29 | # Get relative coords 30 | def getRelativeCoords(curx, cury, x, y): 31 | relx = x - curx 32 | rely = y - cury 33 | 34 | return relx, rely 35 | 36 | # Cursor smoothening value 37 | smoothening = 15 38 | 39 | # Handle CTRL-C to exit gracefully 40 | def signal_handler(signal, frame): 41 | print("Exiting program") 42 | cap.release() 43 | cv.destroyAllWindows() 44 | sys.exit() 45 | 46 | signal.signal(signal.SIGINT, signal_handler) 47 | 48 | # Set mouse movement pause to 0 for smooth movement 49 | gui.PAUSE = 0 50 | 51 | # Get solution objects from mediapipe 52 | mp_drawing = mp.solutions.drawing_utils 53 | mp_pose = mp.solutions.pose 54 | 55 | # Get video capture 56 | cap = cv.VideoCapture(0) 57 | 58 | # Variables to keep track of where the crosshair is 59 | findx = 0 60 | findy = 0 61 | 62 | # Old values to average 63 | oldx = 0 64 | oldy = 0 65 | 66 | # Shoulder movement list 67 | walkingdiff = [] 68 | jumpingdiff = [] 69 | miningdiff = [] 70 | placingdiff = [] 71 | 72 | walk = False 73 | jump = False 74 | mining = False 75 | placing = False 76 | 77 | # Main while loop 78 | with mp_pose.Pose(min_detection_confidence=0.5, min_tracking_confidence=0.5, static_image_mode=False, smooth_landmarks=True) as pose: 79 | for i in itertools.count(): 80 | 81 | # Get frame 82 | ret, frame = cap.read() 83 | 84 | # Flip frame 85 | frame = cv.flip(frame, 1) 86 | 87 | # Get pose data 88 | results = pose.process(cv.cvtColor(frame, cv.COLOR_BGR2RGB)) 89 | 90 | # Draw landmarks 91 | mp_drawing.draw_landmarks(frame, results.pose_landmarks, mp_pose.POSE_CONNECTIONS) 92 | 93 | # pose_landmarks variable 94 | pose_landmarks = results.pose_landmarks 95 | 96 | # print pose landmarks 97 | if not results.pose_landmarks: 98 | continue 99 | 100 | # Get image dimensions 101 | image_height, image_width, _ = frame.shape 102 | 103 | # region Mining and using 104 | righthandy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_INDEX].y * image_height 105 | lefthandy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_INDEX].y * image_height 106 | 107 | miningdiff.append(righthandy) 108 | 109 | if len(miningdiff) < 5: 110 | continue 111 | 112 | miningworkingdata = np.diff(miningdiff) 113 | miningworkingdata = abs(miningworkingdata) > 100 114 | 115 | if True in miningworkingdata: 116 | #gui.click(button='left') 117 | gui.mouseDown(button='left') 118 | mining = True 119 | if not True in miningworkingdata and mining: 120 | gui.mouseUp(button='left') 121 | mining = False 122 | 123 | if len(miningdiff) == 20: 124 | miningdiff = [] 125 | 126 | placingdiff.append(righthandy) 127 | 128 | if len(placingdiff) < 5: 129 | continue 130 | 131 | if lefthandy < 400 and not placing: 132 | gui.click(button='right') 133 | placing = True 134 | elif placing: 135 | placing = False 136 | 137 | 138 | # endregion 139 | 140 | # region Walking + Jumping 141 | walkingx = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * image_width 142 | walkingdiff.append(walkingx) 143 | 144 | if len(walkingdiff) < 5: 145 | continue 146 | 147 | walkingworkingdata = np.diff(walkingdiff) 148 | walkingworkingdata = abs(walkingworkingdata) > 15 149 | if True in walkingworkingdata and not mining: 150 | #print('w!') 151 | gui.keyDown('ctrl') 152 | gui.keyDown('w') 153 | walk = True 154 | 155 | if not True in walkingworkingdata: 156 | #print('not w') 157 | gui.keyUp('w') 158 | gui.keyUp('ctrl') 159 | walk = False 160 | 161 | # Reset every 60 samples 162 | if len(walkingdiff) == 15: 163 | walkingdiff = [] 164 | 165 | 166 | # Jumping 167 | jumpingy = results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * image_height 168 | jumpingdiff.append(jumpingy) 169 | 170 | jumpingworkingdata = np.diff(jumpingdiff) 171 | jumpingworkingdata = abs(jumpingworkingdata) > 40 172 | 173 | if True in jumpingworkingdata: 174 | gui.keyDown('space') 175 | jump = True 176 | else: 177 | gui.keyUp('space') 178 | jump = False 179 | 180 | if len(jumpingdiff) == 20: 181 | jumpingdiff = [] 182 | 183 | # endregion 184 | 185 | # region Move Mouse 186 | x = results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].x * image_width 187 | y = results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].y * image_height 188 | x = int(x) 189 | y = int(y) 190 | 191 | # Scale to display size 192 | #x = x * 1 193 | #y = y * 1 194 | 195 | x = oldx + (x - oldx) / smoothening 196 | y = oldy + (y - oldy) / smoothening 197 | 198 | oldx = x 199 | oldy = y 200 | 201 | # Increase movement (trial and error) 202 | x = x * 18 203 | y = y * 15 204 | 205 | # Get the relative coordinates to move to 206 | x, y = getRelativeCoords(findx, findy, x, y) 207 | 208 | # Change to integer 209 | x = int(x) 210 | y = int(y) 211 | 212 | # Move the mouse 213 | if not walk and not jump and not mining: 214 | pydirectinput.move(x, y) 215 | 216 | # Update the find coordinates to keep track of location 217 | findx = findx + x 218 | findy = findy + y 219 | 220 | # endregion 221 | 222 | # Show frame 223 | #cv.imshow("OpenCV", frame) 224 | 225 | # Exit key 226 | if cv.waitKey(20) & 0xFF==ord('d'): 227 | break 228 | 229 | # Exit program 230 | cap.release() 231 | cv.destroyAllWindows() -------------------------------------------------------------------------------- /pilibs/adafruit_hid/__init__.py: -------------------------------------------------------------------------------- 1 | # SPDX-FileCopyrightText: 2017 Scott Shawcroft for Adafruit Industries 2 | # 3 | # SPDX-License-Identifier: MIT 4 | 5 | """ 6 | `adafruit_hid` 7 | ==================================================== 8 | 9 | This driver simulates USB HID devices. 10 | 11 | * Author(s): Scott Shawcroft, Dan Halbert 12 | 13 | Implementation Notes 14 | -------------------- 15 | **Software and Dependencies:** 16 | * Adafruit CircuitPython firmware for the supported boards: 17 | https://github.com/adafruit/circuitpython/releases 18 | """ 19 | 20 | # imports 21 | 22 | __version__ = "0.0.0-auto.0" 23 | __repo__ = "https://github.com/adafruit/Adafruit_CircuitPython_HID.git" 24 | 25 | 26 | def find_device(devices, *, usage_page, usage): 27 | """Search through the provided list of devices to find the one with the matching usage_page and 28 | usage.""" 29 | if hasattr(devices, "send_report"): 30 | devices = [devices] 31 | for device in devices: 32 | if ( 33 | device.usage_page == usage_page 34 | and device.usage == usage 35 | and hasattr(device, "send_report") 36 | ): 37 | return device 38 | raise ValueError("Could not find matching HID device.") 39 | -------------------------------------------------------------------------------- /pilibs/adafruit_hid/consumer_control.py: -------------------------------------------------------------------------------- 1 | # SPDX-FileCopyrightText: 2018 Dan Halbert for Adafruit Industries 2 | # 3 | # SPDX-License-Identifier: MIT 4 | 5 | """ 6 | `adafruit_hid.consumer_control.ConsumerControl` 7 | ==================================================== 8 | 9 | * Author(s): Dan Halbert 10 | """ 11 | 12 | import sys 13 | 14 | if sys.implementation.version[0] < 3: 15 | raise ImportError( 16 | "{0} is not supported in CircuitPython 2.x or lower".format(__name__) 17 | ) 18 | 19 | # pylint: disable=wrong-import-position 20 | import struct 21 | import time 22 | from . import find_device 23 | 24 | 25 | class ConsumerControl: 26 | """Send ConsumerControl code reports, used by multimedia keyboards, remote controls, etc.""" 27 | 28 | def __init__(self, devices): 29 | """Create a ConsumerControl object that will send Consumer Control Device HID reports. 30 | 31 | Devices can be a list of devices that includes a Consumer Control device or a CC device 32 | itself. A device is any object that implements ``send_report()``, ``usage_page`` and 33 | ``usage``. 34 | """ 35 | self._consumer_device = find_device(devices, usage_page=0x0C, usage=0x01) 36 | 37 | # Reuse this bytearray to send consumer reports. 38 | self._report = bytearray(2) 39 | 40 | # Do a no-op to test if HID device is ready. 41 | # If not, wait a bit and try once more. 42 | try: 43 | self.send(0x0) 44 | except OSError: 45 | time.sleep(1) 46 | self.send(0x0) 47 | 48 | def send(self, consumer_code): 49 | """Send a report to do the specified consumer control action, 50 | and then stop the action (so it will not repeat). 51 | 52 | :param consumer_code: a 16-bit consumer control code. 53 | 54 | Examples:: 55 | 56 | from adafruit_hid.consumer_control_code import ConsumerControlCode 57 | 58 | # Raise volume. 59 | consumer_control.send(ConsumerControlCode.VOLUME_INCREMENT) 60 | 61 | # Advance to next track (song). 62 | consumer_control.send(ConsumerControlCode.SCAN_NEXT_TRACK) 63 | """ 64 | self.press(consumer_code) 65 | self.release() 66 | 67 | def press(self, consumer_code): 68 | """Send a report to indicate that the given key has been pressed. 69 | Only one consumer control action can be pressed at a time, so any one 70 | that was previously pressed will be released. 71 | 72 | :param consumer_code: a 16-bit consumer control code. 73 | 74 | Examples:: 75 | 76 | from adafruit_hid.consumer_control_code import ConsumerControlCode 77 | 78 | # Raise volume for 0.5 seconds 79 | consumer_control.press(ConsumerControlCode.VOLUME_INCREMENT) 80 | time.sleep(0.5) 81 | consumer_control.release() 82 | """ 83 | struct.pack_into(" x37|SHIFT_FLAG (shift .) 99 | b"\xb8" # ? x38|SHIFT_FLAG (shift /) 100 | b"\x9f" # @ x1f|SHIFT_FLAG (shift 2) 101 | b"\x84" # A x04|SHIFT_FLAG (shift a) 102 | b"\x85" # B x05|SHIFT_FLAG (etc.) 103 | b"\x86" # C x06|SHIFT_FLAG 104 | b"\x87" # D x07|SHIFT_FLAG 105 | b"\x88" # E x08|SHIFT_FLAG 106 | b"\x89" # F x09|SHIFT_FLAG 107 | b"\x8a" # G x0a|SHIFT_FLAG 108 | b"\x8b" # H x0b|SHIFT_FLAG 109 | b"\x8c" # I x0c|SHIFT_FLAG 110 | b"\x8d" # J x0d|SHIFT_FLAG 111 | b"\x8e" # K x0e|SHIFT_FLAG 112 | b"\x8f" # L x0f|SHIFT_FLAG 113 | b"\x90" # M x10|SHIFT_FLAG 114 | b"\x91" # N x11|SHIFT_FLAG 115 | b"\x92" # O x12|SHIFT_FLAG 116 | b"\x93" # P x13|SHIFT_FLAG 117 | b"\x94" # Q x14|SHIFT_FLAG 118 | b"\x95" # R x15|SHIFT_FLAG 119 | b"\x96" # S x16|SHIFT_FLAG 120 | b"\x97" # T x17|SHIFT_FLAG 121 | b"\x98" # U x18|SHIFT_FLAG 122 | b"\x99" # V x19|SHIFT_FLAG 123 | b"\x9a" # W x1a|SHIFT_FLAG 124 | b"\x9b" # X x1b|SHIFT_FLAG 125 | b"\x9c" # Y x1c|SHIFT_FLAG 126 | b"\x9d" # Z x1d|SHIFT_FLAG 127 | b"\x2f" # [ 128 | b"\x31" # \ backslash 129 | b"\x30" # ] 130 | b"\xa3" # ^ x23|SHIFT_FLAG (shift 6) 131 | b"\xad" # _ x2d|SHIFT_FLAG (shift -) 132 | b"\x35" # ` 133 | b"\x04" # a 134 | b"\x05" # b 135 | b"\x06" # c 136 | b"\x07" # d 137 | b"\x08" # e 138 | b"\x09" # f 139 | b"\x0a" # g 140 | b"\x0b" # h 141 | b"\x0c" # i 142 | b"\x0d" # j 143 | b"\x0e" # k 144 | b"\x0f" # l 145 | b"\x10" # m 146 | b"\x11" # n 147 | b"\x12" # o 148 | b"\x13" # p 149 | b"\x14" # q 150 | b"\x15" # r 151 | b"\x16" # s 152 | b"\x17" # t 153 | b"\x18" # u 154 | b"\x19" # v 155 | b"\x1a" # w 156 | b"\x1b" # x 157 | b"\x1c" # y 158 | b"\x1d" # z 159 | b"\xaf" # { x2f|SHIFT_FLAG (shift [) 160 | b"\xb1" # | x31|SHIFT_FLAG (shift \) 161 | b"\xb0" # } x30|SHIFT_FLAG (shift ]) 162 | b"\xb5" # ~ x35|SHIFT_FLAG (shift `) 163 | b"\x4c" # DEL DELETE (called Forward Delete in usb.org document) 164 | ) 165 | 166 | def __init__(self, keyboard): 167 | """Specify the layout for the given keyboard. 168 | 169 | :param keyboard: a Keyboard object. Write characters to this keyboard when requested. 170 | 171 | Example:: 172 | 173 | kbd = Keyboard(usb_hid.devices) 174 | layout = KeyboardLayoutUS(kbd) 175 | """ 176 | 177 | self.keyboard = keyboard 178 | 179 | def write(self, string): 180 | """Type the string by pressing and releasing keys on my keyboard. 181 | 182 | :param string: A string of ASCII characters. 183 | :raises ValueError: if any of the characters are not ASCII or have no keycode 184 | (such as some control characters). 185 | 186 | Example:: 187 | 188 | # Write abc followed by Enter to the keyboard 189 | layout.write('abc\\n') 190 | """ 191 | for char in string: 192 | keycode = self._char_to_keycode(char) 193 | # If this is a shifted char, clear the SHIFT flag and press the SHIFT key. 194 | if keycode & self.SHIFT_FLAG: 195 | keycode &= ~self.SHIFT_FLAG 196 | self.keyboard.press(Keycode.SHIFT) 197 | self.keyboard.press(keycode) 198 | self.keyboard.release_all() 199 | 200 | def keycodes(self, char): 201 | """Return a tuple of keycodes needed to type the given character. 202 | 203 | :param char: A single ASCII character in a string. 204 | :type char: str of length one. 205 | :returns: tuple of Keycode keycodes. 206 | :raises ValueError: if ``char`` is not ASCII or there is no keycode for it. 207 | 208 | Examples:: 209 | 210 | # Returns (Keycode.TAB,) 211 | keycodes('\t') 212 | # Returns (Keycode.A,) 213 | keycode('a') 214 | # Returns (Keycode.SHIFT, Keycode.A) 215 | keycode('A') 216 | # Raises ValueError because it's a accented e and is not ASCII 217 | keycode('é') 218 | """ 219 | keycode = self._char_to_keycode(char) 220 | if keycode & self.SHIFT_FLAG: 221 | return (Keycode.SHIFT, keycode & ~self.SHIFT_FLAG) 222 | 223 | return (keycode,) 224 | 225 | def _char_to_keycode(self, char): 226 | """Return the HID keycode for the given ASCII character, with the SHIFT_FLAG possibly set. 227 | 228 | If the character requires pressing the Shift key, the SHIFT_FLAG bit is set. 229 | You must clear this bit before passing the keycode in a USB report. 230 | """ 231 | char_val = ord(char) 232 | if char_val > 128: 233 | raise ValueError("Not an ASCII character.") 234 | keycode = self.ASCII_TO_KEYCODE[char_val] 235 | if keycode == 0: 236 | raise ValueError("No keycode available for character.") 237 | return keycode 238 | -------------------------------------------------------------------------------- /pilibs/adafruit_hid/keycode.py: -------------------------------------------------------------------------------- 1 | # SPDX-FileCopyrightText: 2017 Scott Shawcroft for Adafruit Industries 2 | # 3 | # SPDX-License-Identifier: MIT 4 | 5 | """ 6 | `adafruit_hid.keycode.Keycode` 7 | ==================================================== 8 | 9 | * Author(s): Scott Shawcroft, Dan Halbert 10 | """ 11 | 12 | 13 | class Keycode: 14 | """USB HID Keycode constants. 15 | 16 | This list is modeled after the names for USB keycodes defined in 17 | https://usb.org/sites/default/files/hut1_21_0.pdf#page=83. 18 | This list does not include every single code, but does include all the keys on 19 | a regular PC or Mac keyboard. 20 | 21 | Remember that keycodes are the names for key *positions* on a US keyboard, and may 22 | not correspond to the character that you mean to send if you want to emulate non-US keyboard. 23 | For instance, on a French keyboard (AZERTY instead of QWERTY), 24 | the keycode for 'q' is used to indicate an 'a'. Likewise, 'y' represents 'z' on 25 | a German keyboard. This is historical: the idea was that the keycaps could be changed 26 | without changing the keycodes sent, so that different firmware was not needed for 27 | different variations of a keyboard. 28 | """ 29 | 30 | # pylint: disable-msg=invalid-name 31 | A = 0x04 32 | """``a`` and ``A``""" 33 | B = 0x05 34 | """``b`` and ``B``""" 35 | C = 0x06 36 | """``c`` and ``C``""" 37 | D = 0x07 38 | """``d`` and ``D``""" 39 | E = 0x08 40 | """``e`` and ``E``""" 41 | F = 0x09 42 | """``f`` and ``F``""" 43 | G = 0x0A 44 | """``g`` and ``G``""" 45 | H = 0x0B 46 | """``h`` and ``H``""" 47 | I = 0x0C 48 | """``i`` and ``I``""" 49 | J = 0x0D 50 | """``j`` and ``J``""" 51 | K = 0x0E 52 | """``k`` and ``K``""" 53 | L = 0x0F 54 | """``l`` and ``L``""" 55 | M = 0x10 56 | """``m`` and ``M``""" 57 | N = 0x11 58 | """``n`` and ``N``""" 59 | O = 0x12 60 | """``o`` and ``O``""" 61 | P = 0x13 62 | """``p`` and ``P``""" 63 | Q = 0x14 64 | """``q`` and ``Q``""" 65 | R = 0x15 66 | """``r`` and ``R``""" 67 | S = 0x16 68 | """``s`` and ``S``""" 69 | T = 0x17 70 | """``t`` and ``T``""" 71 | U = 0x18 72 | """``u`` and ``U``""" 73 | V = 0x19 74 | """``v`` and ``V``""" 75 | W = 0x1A 76 | """``w`` and ``W``""" 77 | X = 0x1B 78 | """``x`` and ``X``""" 79 | Y = 0x1C 80 | """``y`` and ``Y``""" 81 | Z = 0x1D 82 | """``z`` and ``Z``""" 83 | 84 | ONE = 0x1E 85 | """``1`` and ``!``""" 86 | TWO = 0x1F 87 | """``2`` and ``@``""" 88 | THREE = 0x20 89 | """``3`` and ``#``""" 90 | FOUR = 0x21 91 | """``4`` and ``$``""" 92 | FIVE = 0x22 93 | """``5`` and ``%``""" 94 | SIX = 0x23 95 | """``6`` and ``^``""" 96 | SEVEN = 0x24 97 | """``7`` and ``&``""" 98 | EIGHT = 0x25 99 | """``8`` and ``*``""" 100 | NINE = 0x26 101 | """``9`` and ``(``""" 102 | ZERO = 0x27 103 | """``0`` and ``)``""" 104 | ENTER = 0x28 105 | """Enter (Return)""" 106 | RETURN = ENTER 107 | """Alias for ``ENTER``""" 108 | ESCAPE = 0x29 109 | """Escape""" 110 | BACKSPACE = 0x2A 111 | """Delete backward (Backspace)""" 112 | TAB = 0x2B 113 | """Tab and Backtab""" 114 | SPACEBAR = 0x2C 115 | """Spacebar""" 116 | SPACE = SPACEBAR 117 | """Alias for SPACEBAR""" 118 | MINUS = 0x2D 119 | """``-` and ``_``""" 120 | EQUALS = 0x2E 121 | """``=` and ``+``""" 122 | LEFT_BRACKET = 0x2F 123 | """``[`` and ``{``""" 124 | RIGHT_BRACKET = 0x30 125 | """``]`` and ``}``""" 126 | BACKSLASH = 0x31 127 | r"""``\`` and ``|``""" 128 | POUND = 0x32 129 | """``#`` and ``~`` (Non-US keyboard)""" 130 | SEMICOLON = 0x33 131 | """``;`` and ``:``""" 132 | QUOTE = 0x34 133 | """``'`` and ``"``""" 134 | GRAVE_ACCENT = 0x35 135 | r""":literal:`\`` and ``~``""" 136 | COMMA = 0x36 137 | """``,`` and ``<``""" 138 | PERIOD = 0x37 139 | """``.`` and ``>``""" 140 | FORWARD_SLASH = 0x38 141 | """``/`` and ``?``""" 142 | 143 | CAPS_LOCK = 0x39 144 | """Caps Lock""" 145 | 146 | F1 = 0x3A 147 | """Function key F1""" 148 | F2 = 0x3B 149 | """Function key F2""" 150 | F3 = 0x3C 151 | """Function key F3""" 152 | F4 = 0x3D 153 | """Function key F4""" 154 | F5 = 0x3E 155 | """Function key F5""" 156 | F6 = 0x3F 157 | """Function key F6""" 158 | F7 = 0x40 159 | """Function key F7""" 160 | F8 = 0x41 161 | """Function key F8""" 162 | F9 = 0x42 163 | """Function key F9""" 164 | F10 = 0x43 165 | """Function key F10""" 166 | F11 = 0x44 167 | """Function key F11""" 168 | F12 = 0x45 169 | """Function key F12""" 170 | 171 | PRINT_SCREEN = 0x46 172 | """Print Screen (SysRq)""" 173 | SCROLL_LOCK = 0x47 174 | """Scroll Lock""" 175 | PAUSE = 0x48 176 | """Pause (Break)""" 177 | 178 | INSERT = 0x49 179 | """Insert""" 180 | HOME = 0x4A 181 | """Home (often moves to beginning of line)""" 182 | PAGE_UP = 0x4B 183 | """Go back one page""" 184 | DELETE = 0x4C 185 | """Delete forward""" 186 | END = 0x4D 187 | """End (often moves to end of line)""" 188 | PAGE_DOWN = 0x4E 189 | """Go forward one page""" 190 | 191 | RIGHT_ARROW = 0x4F 192 | """Move the cursor right""" 193 | LEFT_ARROW = 0x50 194 | """Move the cursor left""" 195 | DOWN_ARROW = 0x51 196 | """Move the cursor down""" 197 | UP_ARROW = 0x52 198 | """Move the cursor up""" 199 | 200 | KEYPAD_NUMLOCK = 0x53 201 | """Num Lock (Clear on Mac)""" 202 | KEYPAD_FORWARD_SLASH = 0x54 203 | """Keypad ``/``""" 204 | KEYPAD_ASTERISK = 0x55 205 | """Keypad ``*``""" 206 | KEYPAD_MINUS = 0x56 207 | """Keyapd ``-``""" 208 | KEYPAD_PLUS = 0x57 209 | """Keypad ``+``""" 210 | KEYPAD_ENTER = 0x58 211 | """Keypad Enter""" 212 | KEYPAD_ONE = 0x59 213 | """Keypad ``1`` and End""" 214 | KEYPAD_TWO = 0x5A 215 | """Keypad ``2`` and Down Arrow""" 216 | KEYPAD_THREE = 0x5B 217 | """Keypad ``3`` and PgDn""" 218 | KEYPAD_FOUR = 0x5C 219 | """Keypad ``4`` and Left Arrow""" 220 | KEYPAD_FIVE = 0x5D 221 | """Keypad ``5``""" 222 | KEYPAD_SIX = 0x5E 223 | """Keypad ``6`` and Right Arrow""" 224 | KEYPAD_SEVEN = 0x5F 225 | """Keypad ``7`` and Home""" 226 | KEYPAD_EIGHT = 0x60 227 | """Keypad ``8`` and Up Arrow""" 228 | KEYPAD_NINE = 0x61 229 | """Keypad ``9`` and PgUp""" 230 | KEYPAD_ZERO = 0x62 231 | """Keypad ``0`` and Ins""" 232 | KEYPAD_PERIOD = 0x63 233 | """Keypad ``.`` and Del""" 234 | KEYPAD_BACKSLASH = 0x64 235 | """Keypad ``\\`` and ``|`` (Non-US)""" 236 | 237 | APPLICATION = 0x65 238 | """Application: also known as the Menu key (Windows)""" 239 | POWER = 0x66 240 | """Power (Mac)""" 241 | KEYPAD_EQUALS = 0x67 242 | """Keypad ``=`` (Mac)""" 243 | F13 = 0x68 244 | """Function key F13 (Mac)""" 245 | F14 = 0x69 246 | """Function key F14 (Mac)""" 247 | F15 = 0x6A 248 | """Function key F15 (Mac)""" 249 | F16 = 0x6B 250 | """Function key F16 (Mac)""" 251 | F17 = 0x6C 252 | """Function key F17 (Mac)""" 253 | F18 = 0x6D 254 | """Function key F18 (Mac)""" 255 | F19 = 0x6E 256 | """Function key F19 (Mac)""" 257 | 258 | LEFT_CONTROL = 0xE0 259 | """Control modifier left of the spacebar""" 260 | CONTROL = LEFT_CONTROL 261 | """Alias for LEFT_CONTROL""" 262 | LEFT_SHIFT = 0xE1 263 | """Shift modifier left of the spacebar""" 264 | SHIFT = LEFT_SHIFT 265 | """Alias for LEFT_SHIFT""" 266 | LEFT_ALT = 0xE2 267 | """Alt modifier left of the spacebar""" 268 | ALT = LEFT_ALT 269 | """Alias for LEFT_ALT; Alt is also known as Option (Mac)""" 270 | OPTION = ALT 271 | """Labeled as Option on some Mac keyboards""" 272 | LEFT_GUI = 0xE3 273 | """GUI modifier left of the spacebar""" 274 | GUI = LEFT_GUI 275 | """Alias for LEFT_GUI; GUI is also known as the Windows key, Command (Mac), or Meta""" 276 | WINDOWS = GUI 277 | """Labeled with a Windows logo on Windows keyboards""" 278 | COMMAND = GUI 279 | """Labeled as Command on Mac keyboards, with a clover glyph""" 280 | RIGHT_CONTROL = 0xE4 281 | """Control modifier right of the spacebar""" 282 | RIGHT_SHIFT = 0xE5 283 | """Shift modifier right of the spacebar""" 284 | RIGHT_ALT = 0xE6 285 | """Alt modifier right of the spacebar""" 286 | RIGHT_GUI = 0xE7 287 | """GUI modifier right of the spacebar""" 288 | 289 | # pylint: enable-msg=invalid-name 290 | @classmethod 291 | def modifier_bit(cls, keycode): 292 | """Return the modifer bit to be set in an HID keycode report if this is a 293 | modifier key; otherwise return 0.""" 294 | return ( 295 | 1 << (keycode - 0xE0) if cls.LEFT_CONTROL <= keycode <= cls.RIGHT_GUI else 0 296 | ) 297 | -------------------------------------------------------------------------------- /pilibs/adafruit_hid/mouse.py: -------------------------------------------------------------------------------- 1 | # SPDX-FileCopyrightText: 2017 Dan Halbert for Adafruit Industries 2 | # 3 | # SPDX-License-Identifier: MIT 4 | 5 | """ 6 | `adafruit_hid.mouse.Mouse` 7 | ==================================================== 8 | 9 | * Author(s): Dan Halbert 10 | """ 11 | import time 12 | 13 | from . import find_device 14 | 15 | 16 | class Mouse: 17 | """Send USB HID mouse reports.""" 18 | 19 | LEFT_BUTTON = 1 20 | """Left mouse button.""" 21 | RIGHT_BUTTON = 2 22 | """Right mouse button.""" 23 | MIDDLE_BUTTON = 4 24 | """Middle mouse button.""" 25 | 26 | def __init__(self, devices): 27 | """Create a Mouse object that will send USB mouse HID reports. 28 | 29 | Devices can be a list of devices that includes a keyboard device or a keyboard device 30 | itself. A device is any object that implements ``send_report()``, ``usage_page`` and 31 | ``usage``. 32 | """ 33 | self._mouse_device = find_device(devices, usage_page=0x1, usage=0x02) 34 | 35 | # Reuse this bytearray to send mouse reports. 36 | # report[0] buttons pressed (LEFT, MIDDLE, RIGHT) 37 | # report[1] x movement 38 | # report[2] y movement 39 | # report[3] wheel movement 40 | self.report = bytearray(4) 41 | 42 | # Do a no-op to test if HID device is ready. 43 | # If not, wait a bit and try once more. 44 | try: 45 | self._send_no_move() 46 | except OSError: 47 | time.sleep(1) 48 | self._send_no_move() 49 | 50 | def press(self, buttons): 51 | """Press the given mouse buttons. 52 | 53 | :param buttons: a bitwise-or'd combination of ``LEFT_BUTTON``, 54 | ``MIDDLE_BUTTON``, and ``RIGHT_BUTTON``. 55 | 56 | Examples:: 57 | 58 | # Press the left button. 59 | m.press(Mouse.LEFT_BUTTON) 60 | 61 | # Press the left and right buttons simultaneously. 62 | m.press(Mouse.LEFT_BUTTON | Mouse.RIGHT_BUTTON) 63 | """ 64 | self.report[0] |= buttons 65 | self._send_no_move() 66 | 67 | def release(self, buttons): 68 | """Release the given mouse buttons. 69 | 70 | :param buttons: a bitwise-or'd combination of ``LEFT_BUTTON``, 71 | ``MIDDLE_BUTTON``, and ``RIGHT_BUTTON``. 72 | """ 73 | self.report[0] &= ~buttons 74 | self._send_no_move() 75 | 76 | def release_all(self): 77 | """Release all the mouse buttons.""" 78 | self.report[0] = 0 79 | self._send_no_move() 80 | 81 | def click(self, buttons): 82 | """Press and release the given mouse buttons. 83 | 84 | :param buttons: a bitwise-or'd combination of ``LEFT_BUTTON``, 85 | ``MIDDLE_BUTTON``, and ``RIGHT_BUTTON``. 86 | 87 | Examples:: 88 | 89 | # Click the left button. 90 | m.click(Mouse.LEFT_BUTTON) 91 | 92 | # Double-click the left button. 93 | m.click(Mouse.LEFT_BUTTON) 94 | m.click(Mouse.LEFT_BUTTON) 95 | """ 96 | self.press(buttons) 97 | self.release(buttons) 98 | 99 | def move(self, x=0, y=0, wheel=0): 100 | """Move the mouse and turn the wheel as directed. 101 | 102 | :param x: Move the mouse along the x axis. Negative is to the left, positive 103 | is to the right. 104 | :param y: Move the mouse along the y axis. Negative is upwards on the display, 105 | positive is downwards. 106 | :param wheel: Rotate the wheel this amount. Negative is toward the user, positive 107 | is away from the user. The scrolling effect depends on the host. 108 | 109 | Examples:: 110 | 111 | # Move 100 to the left. Do not move up and down. Do not roll the scroll wheel. 112 | m.move(-100, 0, 0) 113 | # Same, with keyword arguments. 114 | m.move(x=-100) 115 | 116 | # Move diagonally to the upper right. 117 | m.move(50, 20) 118 | # Same. 119 | m.move(x=50, y=-20) 120 | 121 | # Roll the mouse wheel away from the user. 122 | m.move(wheel=1) 123 | """ 124 | # Send multiple reports if necessary to move or scroll requested amounts. 125 | while x != 0 or y != 0 or wheel != 0: 126 | partial_x = self._limit(x) 127 | partial_y = self._limit(y) 128 | partial_wheel = self._limit(wheel) 129 | self.report[1] = partial_x & 0xFF 130 | self.report[2] = partial_y & 0xFF 131 | self.report[3] = partial_wheel & 0xFF 132 | self._mouse_device.send_report(self.report) 133 | x -= partial_x 134 | y -= partial_y 135 | wheel -= partial_wheel 136 | 137 | def _send_no_move(self): 138 | """Send a button-only report.""" 139 | self.report[1] = 0 140 | self.report[2] = 0 141 | self.report[3] = 0 142 | self._mouse_device.send_report(self.report) 143 | 144 | @staticmethod 145 | def _limit(dist): 146 | return min(127, max(-127, dist)) 147 | --------------------------------------------------------------------------------