├── src ├── utils │ ├── __init__.py │ └── picamera_utils.py ├── object-tracking-feature │ ├── simple.png │ └── orb.py ├── object-detection-yolo │ └── object_detection_yolo.py ├── camera-test │ └── cv_camera_test.py ├── face-detection │ └── face-detection.py ├── motion-detection │ └── cv_motion_detection.py ├── object-tracking-shape │ └── cv_object_tracking_shape.py └── object-tracking-color │ └── cv_object_tracking_color.py ├── images ├── cover.png ├── cv_orb.png ├── rpi-logo.png ├── cv_camera_test.png ├── cv_face-detection.png ├── cv_motion_detection.png ├── cv_object_tracking_color.png └── cv_object_tracking_shape.png ├── requirements.txt ├── .gitignore ├── LICENSE ├── install.sh └── README.md /src/utils/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /images/cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cover.png -------------------------------------------------------------------------------- /images/cv_orb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_orb.png -------------------------------------------------------------------------------- /images/rpi-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/rpi-logo.png -------------------------------------------------------------------------------- /images/cv_camera_test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_camera_test.png -------------------------------------------------------------------------------- /images/cv_face-detection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_face-detection.png -------------------------------------------------------------------------------- /images/cv_motion_detection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_motion_detection.png -------------------------------------------------------------------------------- /images/cv_object_tracking_color.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_object_tracking_color.png -------------------------------------------------------------------------------- /images/cv_object_tracking_shape.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/images/cv_object_tracking_shape.png -------------------------------------------------------------------------------- /src/object-tracking-feature/simple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/automaticdai/rpi-object-detection/HEAD/src/object-tracking-feature/simple.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | matplotlib 2 | numpy 3 | opencv-contrib-python 4 | opencv-python 5 | packaging 6 | pillow 7 | pyparsing 8 | python-dateutil 9 | scipy 10 | six 11 | ultralytics 12 | -------------------------------------------------------------------------------- /src/utils/picamera_utils.py: -------------------------------------------------------------------------------- 1 | try: 2 | from picamera2 import Picamera2 3 | picam2_available = True 4 | except ImportError: 5 | picam2_available = False 6 | 7 | 8 | def is_raspberry_camera(): 9 | """Check if the Raspberry Pi Camera is being used.""" 10 | return picam2_available 11 | 12 | def get_picamera(width, height): 13 | """Get the camera object.""" 14 | if picam2_available: 15 | picam2 = Picamera2() 16 | picam2.configure(picam2.create_preview_configuration(main={"format": 'XRGB8888', "size": (width, height)})) 17 | return picam2 18 | return None 19 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | *.egg-info/ 23 | .installed.cfg 24 | *.egg 25 | 26 | # PyInstaller 27 | # Usually these files are written by a python script from a template 28 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 29 | *.manifest 30 | *.spec 31 | 32 | # Installer logs 33 | pip-log.txt 34 | pip-delete-this-directory.txt 35 | 36 | # Unit test / coverage reports 37 | htmlcov/ 38 | .tox/ 39 | .coverage 40 | .coverage.* 41 | .cache 42 | nosetests.xml 43 | coverage.xml 44 | *,cover 45 | 46 | # Translations 47 | *.mo 48 | *.pot 49 | 50 | # Django stuff: 51 | *.log 52 | 53 | # Sphinx documentation 54 | docs/_build/ 55 | 56 | # PyBuilder 57 | target/ 58 | .idea/ 59 | 60 | # venv 61 | venv/ 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License 2 | 3 | Copyright (c) 2010-2020 YunFei Robotics Lab. https://www.yfrl.org 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/object-detection-yolo/object_detection_yolo.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # Use Ultralytics YOLO to detect objects in a video stream 5 | # ------------------------------------------------------------------------------ 6 | # automaticdai 7 | # YF Robotics Labrotary 8 | # Instagram: yfrobotics 9 | # Twitter: @yfrobotics 10 | # Website: https://yfrobotics.github.io/ 11 | # ------------------------------------------------------------------------------ 12 | # Referecens: 13 | # - https://docs.ultralytics.com/guides/raspberry-pi/ 14 | # ------------------------------------------------------------------------------ 15 | 16 | import cv2 17 | from ultralytics import YOLO 18 | 19 | def main(): 20 | # Load a YOLO11n PyTorch model 21 | model = YOLO("yolo11n.pt") 22 | 23 | # Open default camera 24 | cap = cv2.VideoCapture(0) 25 | if not cap.isOpened(): 26 | print("Error: Could not open video stream.") 27 | return 28 | 29 | while True: 30 | ret, frame = cap.read() 31 | if not ret: 32 | print("Error: Failed to grab frame.") 33 | break 34 | 35 | # Run YOLO inference on the frame 36 | results = model(frame) 37 | 38 | # Draw results on the frame 39 | annotated_frame = results[0].plot() 40 | 41 | # Show the frame 42 | cv2.imshow("YOLOv11 Object Detection", annotated_frame) 43 | 44 | # Exit on 'q' or ESC key 45 | key = cv2.waitKey(1) 46 | if key == 27 or key == ord('q'): 47 | break 48 | 49 | cap.release() 50 | cv2.destroyAllWindows() 51 | 52 | if __name__ == "__main__": 53 | main() 54 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "Welcome to use Raspberry Pi Real-Time Obeject Detection and Tracking" 4 | echo "For more information: https://github.com/automaticdai/rpi-object-detection" 5 | 6 | is_raspberry_pi() { 7 | if grep -q "Raspberry" /proc/cpuinfo || grep -q "BCM" /proc/cpuinfo; then 8 | return 0 # Is raspberry Pi 9 | else 10 | return 1 11 | fi 12 | } 13 | 14 | if is_raspberry_pi; then 15 | echo "Raspberry Pi detected. Running Raspberry-specific script..." 16 | 17 | echo "Update packages" 18 | sudo apt-get update 19 | 20 | echo "Installing libopencv-dev and libatlas-base-dev..." 21 | sudo apt-get install -y libopencv-dev libatlas-base-dev 22 | 23 | if ! command -v pip3 &> /dev/null; then 24 | echo "pip3 not found. Installing pip3..." 25 | sudo apt-get install -y python3-pip 26 | fi 27 | 28 | echo "Installing Pillow, numpy, scipy, matplotlib..." 29 | pip3 install Pillow numpy scipy matplotlib 30 | 31 | echo "Installing opencv-python and opencv-contrib-python..." 32 | pip3 install opencv-python opencv-contrib-python ultralytics 33 | 34 | echo "Installation of dependencies completed for Raspberry Pi!" 35 | else 36 | echo "Non-Raspberry Pi system detected. Running standard script..." 37 | 38 | echo "Updating package list..." 39 | sudo apt-get update 40 | 41 | echo "Installing libopencv-dev and libatlas-base-dev..." 42 | sudo apt-get install -y libopencv-dev libatlas-base-dev 43 | 44 | if ! command -v pip3 &> /dev/null; then 45 | echo "pip3 not found. Installing pip3..." 46 | sudo apt-get install -y python3-pip 47 | fi 48 | 49 | echo "Installing python3-venv" 50 | sudo apt install -y python3-venv 51 | 52 | echo "Creating a virtual environment..." 53 | python3 -m venv venv 54 | 55 | echo "Activating virtual environment..." 56 | source venv/bin/activate 57 | 58 | echo "Installing dependencies: Pillow, numpy, scipy, matplotlib, opencv-python, and opencv-contrib-python..." 59 | pip install --upgrade pip setuptools 60 | pip install Pillow numpy scipy matplotlib opencv-python opencv-contrib-python ultralytics 61 | 62 | if [ -f requirements.txt ]; then 63 | echo "Installing dependencies from requirements.txt..." 64 | pip install -r requirements.txt 65 | else 66 | echo "requirements.txt not found. Skipping installation from requirements file." 67 | fi 68 | 69 | echo "Installation of dependencies completed for non-Raspberry Pi system!" 70 | fi 71 | -------------------------------------------------------------------------------- /src/camera-test/cv_camera_test.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # Display the image captured from the camera. Used as a test program to verify 5 | # if OpenCV has been properly installed. 6 | # ------------------------------------------------------------------------------ 7 | # automaticdai 8 | # YF Robotics Labrotary 9 | # Instagram: yfrobotics 10 | # Twitter: @yfrobotics 11 | # Website: https://yfrobotics.github.io/ 12 | # ------------------------------------------------------------------------------ 13 | 14 | import os 15 | import sys 16 | import cv2 17 | import time 18 | import numpy as np 19 | import time 20 | 21 | # Add src directory to the path 22 | sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) 23 | 24 | from utils.picamera_utils import is_raspberry_camera, get_picamera 25 | 26 | CAMERA_DEVICE_ID = 0 27 | IMAGE_WIDTH = 320 28 | IMAGE_HEIGHT = 240 29 | IS_RASPI_CAMERA = is_raspberry_camera() 30 | fps = 0 31 | 32 | print("Using raspi camera: ", IS_RASPI_CAMERA) 33 | 34 | def visualize_fps(image, fps: int): 35 | if len(np.shape(image)) < 3: 36 | text_color = (255, 255, 255) # white 37 | else: 38 | text_color = (0, 255, 0) # green 39 | row_size = 20 # pixels 40 | left_margin = 24 # pixels 41 | 42 | font_size = 1 43 | font_thickness = 1 44 | 45 | # Draw the FPS counter 46 | fps_text = 'FPS = {:.1f}'.format(fps) 47 | text_location = (left_margin, row_size) 48 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 49 | font_size, text_color, font_thickness) 50 | 51 | return image 52 | 53 | if __name__ == "__main__": 54 | try: 55 | if IS_RASPI_CAMERA: 56 | cap = get_picamera(IMAGE_WIDTH, IMAGE_HEIGHT) 57 | cap.start() 58 | else: 59 | # create video capture 60 | cap = cv2.VideoCapture(CAMERA_DEVICE_ID) 61 | # set resolution to 320x240 to reduce latency 62 | cap.set(3, IMAGE_WIDTH) 63 | cap.set(4, IMAGE_HEIGHT) 64 | 65 | # Loop to continuously get images 66 | while True: 67 | # ---------------------------------------------------------------------- 68 | # record start time 69 | start_time = time.time() 70 | 71 | # Read the frames from a camera 72 | if IS_RASPI_CAMERA: 73 | frame = cap.capture_array() 74 | else: 75 | _, frame = cap.read() 76 | 77 | # show image 78 | cv2.imshow('frame', visualize_fps(frame, fps)) 79 | 80 | # ---------------------------------------------------------------------- 81 | # record end time 82 | end_time = time.time() 83 | 84 | # calculate FPS 85 | seconds = end_time - start_time 86 | fps = 1.0 / seconds 87 | print("Estimated fps:{0:0.1f}".format(fps)) 88 | 89 | # if key pressed is 'Esc' then exit the loop 90 | if cv2.waitKey(33) == 27: 91 | break 92 | except Exception as e: 93 | print(e) 94 | finally: 95 | # Clean up and exit the program 96 | cv2.destroyAllWindows() 97 | cap.close() if IS_RASPI_CAMERA else cap.release() 98 | -------------------------------------------------------------------------------- /src/face-detection/face-detection.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # automaticdai 5 | # YF Robotics Labrotary 6 | # Instagram: yfrobotics 7 | # Twitter: @yfrobotics 8 | # Website: https://yfrobotics.github.io/ 9 | # ------------------------------------------------------------------------------ 10 | # Reference: 11 | # - https://towardsdatascience.com/face-detection-in-2-minutes-using-opencv-python-90f89d7c0f81 12 | # ------------------------------------------------------------------------------ 13 | import os 14 | import sys 15 | import cv2 16 | import time 17 | import numpy as np 18 | import time 19 | 20 | # Add src directory to the path 21 | sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) 22 | 23 | from utils.picamera_utils import is_raspberry_camera, get_picamera 24 | 25 | CAMERA_DEVICE_ID = 0 26 | IMAGE_WIDTH = 320 27 | IMAGE_HEIGHT = 240 28 | IS_RASPI_CAMERA = is_raspberry_camera() 29 | fps = 0 30 | base_dir = os.path.dirname(os.path.abspath(__file__)) 31 | 32 | print("Using raspi camera: ", IS_RASPI_CAMERA) 33 | 34 | def visualize_fps(image, fps: int): 35 | if len(np.shape(image)) < 3: 36 | text_color = (255, 255, 255) # white 37 | else: 38 | text_color = (0, 255, 0) # green 39 | row_size = 20 # pixels 40 | left_margin = 24 # pixels 41 | 42 | font_size = 1 43 | font_thickness = 1 44 | 45 | # Draw the FPS counter 46 | fps_text = 'FPS = {:.1f}'.format(fps) 47 | text_location = (left_margin, row_size) 48 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 49 | font_size, text_color, font_thickness) 50 | 51 | return image 52 | 53 | 54 | # Load the cascade 55 | face_cascade = cv2.CascadeClassifier(os.path.join(base_dir, 'haarcascade_frontalface_default.xml')) 56 | 57 | # To capture video from webcam. 58 | if IS_RASPI_CAMERA: 59 | cap = get_picamera(IMAGE_WIDTH, IMAGE_HEIGHT) 60 | cap.start() 61 | else: 62 | # create video capture 63 | cap = cv2.VideoCapture(CAMERA_DEVICE_ID) 64 | # set resolution to 320x240 to reduce latency 65 | cap.set(3, IMAGE_WIDTH) 66 | cap.set(4, IMAGE_HEIGHT) 67 | # To use a video file as input 68 | # cap = cv2.VideoCapture('filename.mp4') 69 | 70 | while True: 71 | # ---------------------------------------------------------------------- 72 | # record start time 73 | start_time = time.time() 74 | # Read the frames from a camera 75 | if IS_RASPI_CAMERA: 76 | frame = cap.capture_array() 77 | else: 78 | _, frame = cap.read() 79 | # Convert to grayscale 80 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 81 | # Detect the faces 82 | faces = face_cascade.detectMultiScale(gray, 1.1, 4) 83 | # Draw the rectangle around each face 84 | for (x, y, w, h) in faces: 85 | cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2) 86 | # Display 87 | cv2.imshow('img', visualize_fps(frame, fps)) 88 | # ---------------------------------------------------------------------- 89 | # record end time 90 | end_time = time.time() 91 | # calculate FPS 92 | seconds = end_time - start_time 93 | fps = 1.0 / seconds 94 | print("Estimated fps:{0:0.1f}".format(fps)) 95 | # Stop if escape key is pressed 96 | k = cv2.waitKey(30) & 0xff 97 | if k==27: 98 | break 99 | 100 | # Release the VideoCapture object 101 | cap.close() if IS_RASPI_CAMERA else cap.release() 102 | -------------------------------------------------------------------------------- /src/motion-detection/cv_motion_detection.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # Detect any motion in the frame. 5 | # ------------------------------------------------------------------------------ 6 | # automaticdai 7 | # YF Robotics Labrotary 8 | # Instagram: yfrobotics 9 | # Twitter: @yfrobotics 10 | # Website: https://yfrobotics.github.io/ 11 | # ------------------------------------------------------------------------------ 12 | 13 | import os 14 | import sys 15 | import cv2 16 | import time 17 | import numpy as np 18 | import time 19 | 20 | # Add src directory to the path 21 | sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) 22 | 23 | from utils.picamera_utils import is_raspberry_camera, get_picamera 24 | 25 | CAMERA_DEVICE_ID = 0 26 | IMAGE_WIDTH = 320 27 | IMAGE_HEIGHT = 240 28 | IS_RASPI_CAMERA = is_raspberry_camera() 29 | MOTION_BLUR = True 30 | 31 | cnt_frame = 0 32 | fps = 0 33 | 34 | print("Using raspi camera: ", IS_RASPI_CAMERA) 35 | 36 | def mse(image_a, image_b): 37 | # the 'Mean Squared Error' between the two images is the 38 | # sum of the squared difference between the two images; 39 | # NOTE: the two images must have the same dimension 40 | err = np.sum((image_a.astype("float") - image_b.astype("float")) ** 2) 41 | err /= float(image_a.shape[0] * image_a.shape[1]) 42 | 43 | # return the MSE, the lower the error, the more "similar" 44 | # the two images are 45 | return err 46 | 47 | 48 | def visualize_fps(image, fps: int): 49 | if len(np.shape(image)) < 3: 50 | text_color = (255, 255, 255) # white 51 | else: 52 | text_color = (0, 255, 0) # green 53 | row_size = 20 # pixels 54 | left_margin = 24 # pixels 55 | 56 | font_size = 1 57 | font_thickness = 1 58 | 59 | # Draw the FPS counter 60 | fps_text = 'FPS = {:.1f}'.format(fps) 61 | text_location = (left_margin, row_size) 62 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 63 | font_size, text_color, font_thickness) 64 | 65 | return image 66 | 67 | 68 | if __name__ == "__main__": 69 | try: 70 | # create video capture 71 | if IS_RASPI_CAMERA: 72 | cap = get_picamera(IMAGE_WIDTH, IMAGE_HEIGHT) 73 | cap.start() 74 | else: 75 | # create video capture 76 | cap = cv2.VideoCapture(CAMERA_DEVICE_ID) 77 | # set resolution to 320x240 to reduce latency 78 | cap.set(3, IMAGE_WIDTH) 79 | cap.set(4, IMAGE_HEIGHT) 80 | 81 | while True: 82 | # ---------------------------------------------------------------------- 83 | # record start time 84 | start_time = time.time() 85 | # ---------------------------------------------------------------------- 86 | # Read the frames from a camera 87 | if IS_RASPI_CAMERA: 88 | frame_raw = cap.capture_array() 89 | else: 90 | _, frame_raw = cap.read() 91 | 92 | if MOTION_BLUR: 93 | # Denoise the frame 94 | frame = cv2.GaussianBlur(frame_raw, (3,3),0) 95 | else: 96 | frame = frame_raw 97 | 98 | # Convert to gray image 99 | frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 100 | 101 | # Find edges 102 | edges = cv2.Canny(frame_gray,100,200) 103 | 104 | # Show the original and processed image 105 | cv2.imshow('gray', visualize_fps(frame_gray, fps)) 106 | cv2.imshow('edge', visualize_fps(edges, fps)) 107 | 108 | # Calculate MSE 109 | if cnt_frame > 0: 110 | if mse(frame_gray, frame_gray_p) > 100: 111 | print('Frame{0}: Motion Detected!'.format(cnt_frame)) 112 | 113 | # ---------------------------------------------------------------------- 114 | # record end time 115 | end_time = time.time() 116 | 117 | # calculate FPS 118 | seconds = end_time - start_time 119 | fps = 1.0 / seconds 120 | print("Estimated fps:{0:0.1f}".format(fps)); 121 | 122 | cnt_frame = cnt_frame + 1 123 | edges_p = edges 124 | frame_gray_p = frame_gray 125 | # ---------------------------------------------------------------------- 126 | 127 | # if key pressed is 'Esc' then exit the loop 128 | if cv2.waitKey(1)== 27: 129 | break 130 | except Exception as e: 131 | print(e) 132 | finally: 133 | # Clean up and exit the program 134 | cv2.destroyAllWindows() 135 | cap.close() if IS_RASPI_CAMERA else cap.release() 136 | -------------------------------------------------------------------------------- /src/object-tracking-feature/orb.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # automaticdai 5 | # YF Robotics Labrotary 6 | # Instagram: yfrobotics 7 | # Twitter: @yfrobotics 8 | # Website: https://yfrobotics.github.io/ 9 | # ------------------------------------------------------------------------------ 10 | # References: 11 | # - https://circuitdigest.com/tutorial/real-life-object-detection-using-opencv-python-detecting-objects-in-live-video 12 | # - https://docs.opencv.org/4.x/d1/d89/tutorial_py_orb.html 13 | # ------------------------------------------------------------------------------ 14 | 15 | import cv2 16 | import numpy as np 17 | import time 18 | 19 | def ORB_detector(new_image, image_template): 20 | # Function that compares input image to template 21 | # It then returns the number of ORB matches between them 22 | image1 = cv2.cvtColor(new_image, cv2.COLOR_BGR2GRAY) 23 | 24 | # Create ORB detector with 1000 keypoints with a scaling pyramid factor of 1.2 25 | orb = cv2.ORB_create(1000, 1.2) 26 | 27 | # Detect keypoints of original image 28 | (kp1, des1) = orb.detectAndCompute(image1, None) 29 | 30 | # Detect keypoints of rotated image 31 | (kp2, des2) = orb.detectAndCompute(image_template, None) 32 | 33 | # Create matcher 34 | # Note we're no longer using Flannbased matching 35 | bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) 36 | # Do matching 37 | matches = bf.match(des1,des2) 38 | 39 | # Sort the matches based on distance. Least distance 40 | # is better 41 | matches = sorted(matches, key=lambda val: val.distance) 42 | return len(matches) 43 | 44 | 45 | def visualize_fps(image, fps: int): 46 | if len(np.shape(image)) < 3: 47 | text_color = (255, 255, 255) # white 48 | else: 49 | text_color = (0, 255, 0) # green 50 | row_size = 20 # pixels 51 | left_margin = 24 # pixels 52 | 53 | font_size = 1 54 | font_thickness = 1 55 | 56 | # Draw the FPS counter 57 | fps_text = 'FPS = {:.1f}'.format(fps) 58 | text_location = (left_margin, row_size) 59 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 60 | font_size, text_color, font_thickness) 61 | 62 | return image 63 | 64 | 65 | fps = 0 66 | 67 | cap = cv2.VideoCapture(0) 68 | 69 | # Load our image template, this is our reference image 70 | image_template = cv2.imread('simple.png', 0) 71 | 72 | while True: 73 | # ---------------------------------------------------------------------- 74 | # record start time 75 | start_time = time.time() 76 | # Get webcam images 77 | ret, frame = cap.read() 78 | 79 | # Get height and width of webcam frame 80 | height, width = frame.shape[:2] 81 | 82 | # Define ROI Box Dimensions (Note some of these things should be outside the loop) 83 | top_left_x = int(width / 3) 84 | top_left_y = int((height / 2) + (height / 4)) 85 | bottom_right_x = int((width / 3) * 2) 86 | bottom_right_y = int((height / 2) - (height / 4)) 87 | 88 | # Draw rectangular window for our region of interest 89 | cv2.rectangle(frame, (top_left_x,top_left_y), (bottom_right_x,bottom_right_y), 255, 3) 90 | 91 | # Crop window of observation we defined above 92 | cropped = frame[bottom_right_y:top_left_y , top_left_x:bottom_right_x] 93 | 94 | # Flip frame orientation horizontally 95 | frame = cv2.flip(frame,1) 96 | 97 | # Get number of ORB matches 98 | matches = ORB_detector(cropped, image_template) 99 | 100 | # Display status string showing the current no. of matches 101 | output_string = "# of Matches = " + str(matches) 102 | cv2.putText(frame, output_string, (50,450), cv2.FONT_HERSHEY_COMPLEX, 1, (250,0,0), 2) 103 | 104 | # Our threshold to indicate object deteciton 105 | # For new images or lightening conditions you may need to experiment a bit 106 | # Note: The ORB detector to get the top 1000 matches, 350 is essentially a min 35% match 107 | threshold = 200 108 | 109 | # If matches exceed our threshold then object has been detected 110 | if matches > threshold: 111 | cv2.rectangle(frame, (top_left_x,top_left_y), (bottom_right_x,bottom_right_y), (0,255,0), 3) 112 | cv2.putText(frame,'Object Found',(50,50), cv2.FONT_HERSHEY_COMPLEX, 2 ,(0,255,0), 2) 113 | 114 | cv2.imshow('Object Detector using ORB', visualize_fps(frame, fps)) 115 | # ---------------------------------------------------------------------- 116 | # record end time 117 | end_time = time.time() 118 | # calculate FPS 119 | seconds = end_time - start_time 120 | fps = 1.0 / seconds 121 | print("Estimated fps:{0:0.1f}".format(fps)) 122 | if cv2.waitKey(1) == 13: #13 is the Enter Key 123 | break 124 | 125 | cap.release() 126 | cv2.destroyAllWindows() 127 | -------------------------------------------------------------------------------- /src/object-tracking-shape/cv_object_tracking_shape.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # Find geometric shaped objects in the image using houghCircles(). 5 | # ------------------------------------------------------------------------------ 6 | # automaticdai 7 | # YF Robotics Labrotary 8 | # Instagram: yfrobotics 9 | # Twitter: @yfrobotics 10 | # Website: https://yfrobotics.github.io/ 11 | # ------------------------------------------------------------------------------ 12 | # Reference: 13 | # - https://www.pyimagesearch.com/2014/07/21/detecting-circles-images-using-opencv-hough-circles/ 14 | # ------------------------------------------------------------------------------ 15 | 16 | import os 17 | import sys 18 | import cv2 19 | import time 20 | import numpy as np 21 | import time 22 | 23 | # Add src directory to the path 24 | sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) 25 | 26 | from utils.picamera_utils import is_raspberry_camera, get_picamera 27 | 28 | CAMERA_DEVICE_ID = 0 29 | IMAGE_WIDTH = 320 30 | IMAGE_HEIGHT = 240 31 | IS_RASPI_CAMERA = is_raspberry_camera() 32 | fps = 0 33 | print("Using raspi camera: ", IS_RASPI_CAMERA) 34 | 35 | def isset(v): 36 | try: 37 | type (eval(v)) 38 | except: 39 | return 0 40 | else: 41 | return 1 42 | 43 | 44 | def visualize_fps(image, fps: int): 45 | if len(np.shape(image)) < 3: 46 | text_color = (255, 255, 255) # white 47 | else: 48 | text_color = (0, 255, 0) # green 49 | row_size = 20 # pixels 50 | left_margin = 24 # pixels 51 | 52 | font_size = 1 53 | font_thickness = 1 54 | 55 | # Draw the FPS counter 56 | fps_text = 'FPS = {:.1f}'.format(fps) 57 | text_location = (left_margin, row_size) 58 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 59 | font_size, text_color, font_thickness) 60 | 61 | return image 62 | 63 | 64 | if __name__ == "__main__": 65 | try: 66 | # create video capture 67 | if IS_RASPI_CAMERA: 68 | cap = get_picamera(IMAGE_WIDTH, IMAGE_HEIGHT) 69 | cap.start() 70 | else: 71 | # create video capture 72 | cap = cv2.VideoCapture(CAMERA_DEVICE_ID) 73 | # set resolution to 320x240 to reduce latency 74 | cap.set(3, IMAGE_WIDTH) 75 | cap.set(4, IMAGE_HEIGHT) 76 | 77 | while True: 78 | # ---------------------------------------------------------------------- 79 | # record start time 80 | start_time = time.time() 81 | # Read the frames frome a camera 82 | if IS_RASPI_CAMERA: 83 | frame = cap.capture_array() 84 | else: 85 | _, frame = cap.read() 86 | 87 | frame = cv2.blur(frame,(3,3)) 88 | 89 | # Or get it from a JPEG 90 | # frame = cv2.imread('frame0010.jpg', 1) 91 | 92 | # convert the image into gray color 93 | output = frame.copy() 94 | gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 95 | 96 | # detect circles in the image 97 | circles = cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1.2, 100) 98 | 99 | # ensure at least some circles were found 100 | if circles is not None: 101 | # convert the (x, y) coordinates and radius of the circles to integers 102 | circles = np.round(circles[0, :]).astype("int") 103 | # loop over the (x, y) coordinates and radius of the circles 104 | for (x, y, r) in circles: 105 | # draw the circle in the output image, then draw a rectangle 106 | # corresponding to the center of the circle 107 | cv2.circle(output, (x, y), r, (0, 255, 0), 4) 108 | cv2.rectangle(output, (x - 5, y - 5), (x + 5, y + 5), (0, 128, 255), -1) 109 | 110 | # show the output image 111 | cv2.imshow('img', np.hstack([frame, output])) 112 | # cv2.imshow("frame", np.hstack([visualize_fps(frame, fps), visualize_fps(output, fps)])) 113 | # ---------------------------------------------------------------------- 114 | # record end time 115 | end_time = time.time() 116 | # calculate FPS 117 | seconds = end_time - start_time 118 | fps = 1.0 / seconds 119 | print("Estimated fps:{0:0.1f}".format(fps)) 120 | # if key pressed is 'Esc' then exit the loop 121 | if cv2.waitKey(33) == 27: 122 | break 123 | except Exception as e: 124 | print(e) 125 | finally: 126 | # Clean up and exit the program 127 | cv2.destroyAllWindows() 128 | cap.close() if IS_RASPI_CAMERA else cap.release() 129 | -------------------------------------------------------------------------------- /src/object-tracking-color/cv_object_tracking_color.py: -------------------------------------------------------------------------------- 1 | # ------------------------------------------------------------------------------ 2 | # rpi-object-detection 3 | # ------------------------------------------------------------------------------ 4 | # This is a blob detection program which intend to find the biggest blob 5 | # in a given picture taken by a camera and return its central position. 6 | # 7 | # Key Steps: 8 | # 1. Image Filtering 9 | # 2. Image Segmentation 10 | # 3. Detect Blobs 11 | # 4. Filter Blobs using a criteria 12 | # 5. Track Blobs 13 | # ------------------------------------------------------------------------------ 14 | # automaticdai 15 | # YF Robotics Labrotary 16 | # Instagram: yfrobotics 17 | # Twitter: @yfrobotics 18 | # Website: https://yfrobotics.github.io/ 19 | # ------------------------------------------------------------------------------ 20 | 21 | import os 22 | import sys 23 | import cv2 24 | import time 25 | import numpy as np 26 | import time 27 | import math 28 | 29 | # Add src directory to the path 30 | sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) 31 | 32 | from utils.picamera_utils import is_raspberry_camera, get_picamera 33 | 34 | CAMERA_DEVICE_ID = 0 35 | IMAGE_WIDTH = 320 36 | IMAGE_HEIGHT = 240 37 | IS_RASPI_CAMERA = is_raspberry_camera() 38 | 39 | fps = 0 40 | hsv_min = np.array((50, 80, 80)) 41 | hsv_max = np.array((120, 255, 255)) 42 | 43 | colors = [] 44 | 45 | print("Using raspi camera: ", IS_RASPI_CAMERA) 46 | 47 | def isset(v): 48 | try: 49 | type (eval(v)) 50 | except: 51 | return 0 52 | else: 53 | return 1 54 | 55 | 56 | def on_mouse_click(event, x, y, flags, frame): 57 | global colors 58 | 59 | if event == cv2.EVENT_LBUTTONUP: 60 | color_bgr = frame[y, x] 61 | color_rgb = tuple(reversed(color_bgr)) 62 | #frame[y,x].tolist() 63 | 64 | print(color_rgb) 65 | 66 | color_hsv = rgb2hsv(color_rgb[0], color_rgb[1], color_rgb[2]) 67 | print(color_hsv) 68 | 69 | colors.append(color_hsv) 70 | 71 | print(colors) 72 | 73 | 74 | # R, G, B values are [0, 255]. 75 | # Normally H value is [0, 359]. S, V values are [0, 1]. 76 | # However in opencv, H is [0,179], S, V values are [0, 255]. 77 | # Reference: https://docs.opencv.org/3.4/de/d25/imgproc_color_conversions.html 78 | def hsv2rgb(h, s, v): 79 | h = float(h) * 2 80 | s = float(s) / 255 81 | v = float(v) / 255 82 | h60 = h / 60.0 83 | h60f = math.floor(h60) 84 | hi = int(h60f) % 6 85 | f = h60 - h60f 86 | p = v * (1 - s) 87 | q = v * (1 - f * s) 88 | t = v * (1 - (1 - f) * s) 89 | r, g, b = 0, 0, 0 90 | if hi == 0: r, g, b = v, t, p 91 | elif hi == 1: r, g, b = q, v, p 92 | elif hi == 2: r, g, b = p, v, t 93 | elif hi == 3: r, g, b = p, q, v 94 | elif hi == 4: r, g, b = t, p, v 95 | elif hi == 5: r, g, b = v, p, q 96 | r, g, b = int(r * 255), int(g * 255), int(b * 255) 97 | return (r, g, b) 98 | 99 | 100 | def rgb2hsv(r, g, b): 101 | r, g, b = r/255.0, g/255.0, b/255.0 102 | mx = max(r, g, b) 103 | mn = min(r, g, b) 104 | df = mx-mn 105 | if mx == mn: 106 | h = 0 107 | elif mx == r: 108 | h = (60 * ((g-b)/df) + 360) % 360 109 | elif mx == g: 110 | h = (60 * ((b-r)/df) + 120) % 360 111 | elif mx == b: 112 | h = (60 * ((r-g)/df) + 240) % 360 113 | if mx == 0: 114 | s = 0 115 | else: 116 | s = df/mx 117 | v = mx 118 | 119 | h = int(h / 2) 120 | s = int(s * 255) 121 | v = int(v * 255) 122 | 123 | return (h, s, v) 124 | 125 | 126 | def visualize_fps(image, fps: int): 127 | if len(np.shape(image)) < 3: 128 | text_color = (255, 255, 255) # white 129 | else: 130 | text_color = (0, 255, 0) # green 131 | row_size = 20 # pixels 132 | left_margin = 24 # pixels 133 | 134 | font_size = 1 135 | font_thickness = 1 136 | 137 | # Draw the FPS counter 138 | fps_text = 'FPS = {:.1f}'.format(fps) 139 | text_location = (left_margin, row_size) 140 | cv2.putText(image, fps_text, text_location, cv2.FONT_HERSHEY_PLAIN, 141 | font_size, text_color, font_thickness) 142 | 143 | return image 144 | 145 | 146 | if __name__ == "__main__": 147 | try: 148 | # To capture video from webcam. 149 | if IS_RASPI_CAMERA: 150 | cap = get_picamera(IMAGE_WIDTH, IMAGE_HEIGHT) 151 | cap.start() 152 | else: 153 | # create video capture 154 | cap = cv2.VideoCapture(CAMERA_DEVICE_ID) 155 | # set resolution to 320x240 to reduce latency 156 | cap.set(3, IMAGE_WIDTH) 157 | cap.set(4, IMAGE_HEIGHT) 158 | 159 | while True: 160 | # ---------------------------------------------------------------------- 161 | # record start time 162 | start_time = time.time() 163 | # Read the frames frome a camera 164 | if IS_RASPI_CAMERA: 165 | frame = cap.capture_array() 166 | else: 167 | _, frame = cap.read() 168 | 169 | frame = cv2.blur(frame,(3,3)) 170 | 171 | # Or get it from a JPEG 172 | # frame = cv2.imread('frame0010.jpg', 1) 173 | 174 | # Convert the image to hsv space and find range of colors 175 | hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) 176 | cv2.namedWindow('frame') 177 | cv2.setMouseCallback('frame', on_mouse_click, frame) 178 | 179 | # Uncomment this for RED tag 180 | # thresh = cv2.inRange(hsv,np.array((120, 80, 80)), np.array((180, 255, 255))) 181 | 182 | # find the color using a color threshhold 183 | if colors: 184 | # find max & min h, s, v 185 | minh = min(c[0] for c in colors) 186 | mins = min(c[1] for c in colors) 187 | minv = min(c[2] for c in colors) 188 | maxh = max(c[0] for c in colors) 189 | maxs = max(c[1] for c in colors) 190 | maxv = max(c[2] for c in colors) 191 | 192 | print("New HSV threshold: ", (minh, mins, minv), (maxh, maxs, maxv)) 193 | hsv_min = np.array((minh, mins, minv)) 194 | hsv_max = np.array((maxh, maxs, maxv)) 195 | 196 | thresh = cv2.inRange(hsv, hsv_min, hsv_max) 197 | thresh2 = thresh.copy() 198 | 199 | # find contours in the threshold image 200 | (major_ver, minor_ver, subminor_ver) = (cv2.__version__).split('.') 201 | #print(major_ver, minor_ver, subminor_ver) 202 | 203 | # findContours() has different form for opencv2 and opencv3 204 | if major_ver == "2" or major_ver == "3": 205 | _, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) 206 | else: 207 | contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) 208 | 209 | # finding contour with maximum area and store it as best_cnt 210 | max_area = 0 211 | for cnt in contours: 212 | area = cv2.contourArea(cnt) 213 | if area > max_area: 214 | max_area = area 215 | best_cnt = cnt 216 | 217 | # finding centroids of best_cnt and draw a circle there 218 | if isset('best_cnt'): 219 | M = cv2.moments(best_cnt) 220 | cx,cy = int(M['m10']/M['m00']), int(M['m01']/M['m00']) 221 | cv2.circle(frame,(cx,cy),5,255,-1) 222 | print("Central pos: (%d, %d)" % (cx,cy)) 223 | else: 224 | print("[Warning]Tag lost...") 225 | 226 | # Show the original and processed image 227 | #res = cv2.bitwise_and(frame, frame, mask=thresh2) 228 | cv2.imshow('frame', visualize_fps(frame, fps)) 229 | cv2.imshow('thresh', visualize_fps(thresh2, fps)) 230 | # ---------------------------------------------------------------------- 231 | # record end time 232 | end_time = time.time() 233 | # calculate FPS 234 | seconds = end_time - start_time 235 | fps = 1.0 / seconds 236 | print("Estimated fps:{0:0.1f}".format(fps)); 237 | # if key pressed is 'Esc' then exit the loop 238 | if cv2.waitKey(33) == 27: 239 | break 240 | except Exception as e: 241 | print(e) 242 | finally: 243 | # Clean up and exit the program 244 | cv2.destroyAllWindows() 245 | cap.close() if IS_RASPI_CAMERA else cap.release() 246 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Raspberry Pi Real-Time Object Detection and Tracking 2 | 3 | ![](https://img.shields.io/github/stars/automaticdai/rpi-object-detection) ![](https://img.shields.io/github/forks/automaticdai/rpi-object-detection) ![](https://img.shields.io/github/license/automaticdai/rpi-object-detection) 4 | 5 | ![rpi-logo](./images/cover.png) 6 | 7 | - [Raspberry Pi Real-Time Object Detection and Tracking](#raspberry-pi-real-time-object-detection-and-tracking) 8 | * [1. Introduction](#1-introduction) 9 | * [2. Dependency](#2-dependency) 10 | + [2.1. Packages requirement](#21-packages-requirement) 11 | + [2.2. Hardware support](#22-hardware-support) 12 | * [3. What's in this repository](#3-what-s-in-this-repository) 13 | + [3.1. Camera Test](#31-camera-test) 14 | + [3.2. Motion Detection](#32-motion-detection) 15 | + [3.3. Color-based Object Detection and Tracking](#33-color-based-object-detection-and-tracking) 16 | + [3.4. Shape-based Object Detection and Tracking](#34-shape-based-object-detection-and-tracking) 17 | + [3.5. Feature-based Object Detection and Tracking (with ORB)](#35-feature-based-object-detection-and-tracking--with-orb-) 18 | + [3.6. Face Detection and Tracking](#36-face-detection-and-tracking) 19 | + [3.7. Object Detection using Neural Network (TensorFlow Lite)](#37-object-detection-using-neural-network--tensorflow-lite-) 20 | * [4. How to Run](#4-how-to-run) 21 | + [4.1. Install the environment on Raspberry Pi](#41-install-the-environment-on-raspberry-pi) 22 | + [4.2. Install TensorFlow Lite (optional; only if you want to use the neural network example)](#42-install-tensorflow-lite--optional--only-if-you-want-to-use-the-neural-network-example-) 23 | + [4.3. Run the scripts](#43-run-the-scripts) 24 | + [4.4. Change camera resolution](#44-change-camera-resolution) 25 | * [5. Q&A](#5-q-a) 26 | * [License](#license) 27 | 28 | Table of contents generated with markdown-toc 29 | 30 | 31 | 32 | ![rpi-logo](./images/rpi-logo.png) 33 | 34 | ## 1. Introduction 35 | Using a Raspberry Pi and a camera module for computer vision with OpenCV, YOLO, and TensorFlow Lite. The aim of this project is to provide a starting point for using RPi & CV in your own DIY / maker projects. Computer vision based on cameras is very powerful and will bring your project to the next level. This allows you to track complicated objects that would otherwise not be possible with other types of sensors (infrared, ultrasonic, LiDAR, etc). 36 | 37 | Note the code is based on Python and OpenCV meaning it is cross-platform. You can run this on other Linux-based platforms as well, e.g. x86/x64 PC, IPC, Jetson, Banana Pi, LattaPanda, BeagleBoard, etc. 38 | 39 | 40 | ## 2. Dependency 41 | ### 2.1. Packages requirement 42 | This project is dependent on the following packages: 43 | - Python >= 3.6.9 44 | - OpenCV-Python 45 | - OpenCV-Contrib-Python 46 | - NumPy 47 | - SciPy 48 | - Matplotlib 49 | - Ultralytics 50 | 51 | ### 2.2. Hardware support 52 | - Support Raspberry Pi 1 Model B, Raspberry Pi 2, Raspberry Pi Zero, and Raspberry Pi 3/4/5 (preferable) 53 | - Different boards will have very varied performances. 54 | - RPi 3/4/5 are preferable as they have more powerful CPUs; 55 | - RPi 1/2 may be struggling and produce very low FPS, in which case you can further reduce the camera resolution (160 x 120). 56 | - Nvidia Jetson 57 | - Jetson Nano (A01) also passed the test. 58 | - Jetson Orin Nano should work as well. 59 | - Jetson has its own repo now: [jetson-object-detection](https://github.com/automaticdai/jetson-object-detection) 60 | - Any USB camera supported by Raspberry Pi 61 | - To see a list of all supportive cameras, visit http://elinux.org/RPi_USB_Webcams 62 | - The official RPi camera module is supported through `Picamera2`. 63 | 64 | 65 | ## 3. What's in this repository 66 | Currently, the following applications are implemented: 67 | 68 | - `src/camera-test`: Test if the camera is working 69 | - `src/motion-detection`: Detect any motion in the frame 70 | - `src/object-tracking-color`: Object detection & tracking based on color 71 | - `src/object-tracking-shape`: Object detection & tracking based on shape 72 | - `src/object-tracking-feature`: Object detection & tracking based on features using ORB 73 | - `src/face-detection`: Face detection & tracking 74 | - `src/object-detection-yolo`: Object detection using YOLO (RPi 3/4/5 only) 75 | - (*Todo*) Object detection using Neural Network (TensorFlow Lite) 76 | 77 | 78 | ### 3.1. Camera Test 79 | Test the RPi and OpenCV environment. You are expected to see a pop-up window that has video streams from your USB camera if everything is set up correctly. If the window does not appear, you need to check both of (1) your environment; (2) camera connection. 80 | 81 | ![alt text](./images/cv_camera_test.png) 82 | 83 | ### 3.2. Motion Detection 84 | Detect object movements in the image and print a warning message if any movement is detected. This detection is based on the mean squared error (MSE) of the difference between two images. 85 | 86 | ![alt text](./images/cv_motion_detection.png) 87 | 88 | ### 3.3. Color-based Object Detection and Tracking 89 | Track an object based on its color in HSV and print its center position. You can choose your own color by clicking on the object of interest. Click multiple times on different points so a full color space is coveraged. You can hard code the parameters so you don't need to pick them again for the next run. The following demo shows how I track a Nintendo game controller in real-time: 90 | 91 | ![alt text](./images/cv_object_tracking_color.png) 92 | 93 | ### 3.4. Shape-based Object Detection and Tracking 94 | Detect and track round objects using HoughCircles(). 95 | Support of squares is coming soon. 96 | 97 | ![alt text](./images/cv_object_tracking_shape.png) 98 | 99 | ### 3.5. Feature-based Object Detection and Tracking (with ORB) 100 | Detect and track an object using its feature. The algorithm I selected here is ORB (Oriented FAST and Rotated BRIEF) for its fast calculation speed to enable real-time detection. To use the example, please prepare an Arduino UNO board in hand (or replace the `simple.png`). 101 | 102 | ![alt text](./images/cv_orb.png) 103 | 104 | ### 3.6. Face Detection and Tracking 105 | Detecting face using Harr Cascade detector. 106 | 107 | ![cv_face-detection](images/cv_face-detection.png) 108 | 109 | ### 3.7. Object Detection using YOLO 110 | Use YOLO (You Only Look Once) for object detection. 111 | Note this code is based on Ultralytics YOLO. The instruction can be found at their website: [Quick Start Guide: Raspberry Pi with Ultralytics YOLO11](https://docs.ultralytics.com/guides/raspberry-pi/). Double check if you need to use it in a commercialised project! 112 | 113 | ### 3.8. Object Detection using Neural Network (TensorFlow Lite) 114 | (ongoing) Use TensorFlow Lite to recognise objects. 115 | 116 | 117 | ## 4. How to Run 118 | ### 4.1. Install the environment on Raspberry Pi 119 | ``` 120 | sudo apt-get install -y libopencv-dev libatlas-base-dev 121 | pip3 install virtualenv Pillow numpy scipy matplotlib opencv-python opencv-contrib-python 122 | ``` 123 | 124 | or use the installation script: 125 | 126 | ``` 127 | chmod +x install.sh 128 | ./install.sh 129 | ``` 130 | 131 | ### 4.2. Install TensorFlow Lite (optional; only if you want to use the neural network example) 132 | ``` 133 | wget https://github.com/PINTO0309/Tensorflow-bin/raw/master/tensorflow-2.1.0-cp37-cp37m-linux_armv7l.whl 134 | pip3 install --upgrade setuptools 135 | pip3 install tensorflow-2.1.0-cp37-cp37m-linux_armv7l.whl 136 | pip3 install -e . 137 | ``` 138 | 139 | ### 4.3. Run the scripts 140 | Run scripts in the `/src` folder by: 141 | 142 | ``` 143 | python3 src/$FOLDER_NAME$/$SCRIPT_NAME$.py 144 | ``` 145 | 146 | To stop the code, press the `ESC` key on your keyboard. 147 | 148 | ### 4.4. Change camera resolution 149 | Changing the resolution will significantly impact the FPS. By default it is set to be `320 x 240`, but you can change it to any value that your camera supports at the beginning of each source code (defined by `IMAGE_WIDTH` and `IMAGE_HEIGHT`). Typical resolutions are: 150 | 151 | - 160 x 120 152 | - 320 x 240 153 | - 640 x 480 (480p) 154 | - 1280 x 720 (720p) 155 | - 1920 x 1080 (1080p: make sure your camera supports this high resolution.) 156 | 157 | 158 | ## 5. Q&A 159 | **Q1: Does this support Nvidia Jetson?** 160 | A1: Yes. I have tested with my Jetson Nano 4GB. Note that Jetson has its own repo now: [jetson-object-detection](https://github.com/automaticdai/jetson-object-detection). 161 | 162 | **Q2: Does this support the Raspberry Pi camera?** 163 | A2: This is implemented in [issue [#16]](https://github.com/automaticdai/rpi-object-detection/pull/16). 164 | 165 | **Q3: Does this support Raspberry Pi 5?** 166 | A3: This is not officially tested (as I haven't received my Pi 5 yet) but it should work out of the box. 167 | 168 | **Q4: Can we run this project on Ubuntu server 22.04/24.04?** 169 | A4: It is not officially tested but you should be able to run 99% of the things here. 170 | 171 | **Q5: I am using virtual environment and get a message "no module called libcamera" issue** 172 | A5: A simple solution would be [(Reference)](https://forums.raspberrypi.com/viewtopic.php?t=361758): 173 | `python3 -m venv --system-site-packages env` 174 | 175 | Thanks [VgerTest](https://github.com/VgerTest) for [issue [#20]](https://github.com/automaticdai/rpi-object-detection/issues/20). 176 | 177 | ## License 178 | © This source code is licensed under the [MIT License](LICENSE). 179 | --------------------------------------------------------------------------------