├── .gitignore ├── .vscode └── settings.json ├── README.md ├── __pycache__ └── utils.cpython-36.pyc ├── demos ├── __pycache__ │ └── utils.cpython-36.pyc ├── background_subtraction.py ├── gesture_recognizer.py ├── images │ ├── moving_subject.mp4 │ ├── moving_subject_scale.mp4 │ └── running.mp4 ├── model │ ├── adder_model.hdf5 │ ├── hand_model_gray.hdf5 │ ├── model_history.txt │ └── model_json.json ├── tracker.py └── utils.py ├── images ├── bg.jpg ├── bg_subtract_movement.mp4 ├── cnn1.png ├── cnn2.png ├── cnn3.png ├── cnn4.png ├── cnn5.png ├── cnn6.png ├── cnn7.png ├── current_frame.jpg ├── dnn.jpg ├── earth.jpg ├── neuralnetwork.png ├── nnslides │ ├── Slide10.PNG │ ├── Slide11.PNG │ └── Slide8.PNG └── preview │ ├── fist_0_1039.jpeg │ ├── fist_0_1228.jpeg │ ├── fist_0_1285.jpeg │ ├── fist_0_1311.jpeg │ ├── fist_0_1438.jpeg │ ├── fist_0_1523.jpeg │ ├── fist_0_1773.jpeg │ ├── fist_0_282.jpeg │ ├── fist_0_351.jpeg │ ├── fist_0_423.jpeg │ ├── fist_0_481.jpeg │ ├── fist_0_510.jpeg │ ├── fist_0_635.jpeg │ └── fist_0_975.jpeg ├── model ├── adder_model.hdf5 ├── hand_model_gray.hdf5 ├── model_history.txt └── model_json.json ├── notebook.ipynb └── pres-notebook.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints/ 2 | 3 | training_data/* 4 | validation_data/* 5 | 6 | *.whl 7 | pyenv/ 8 | -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "python.pythonPath": "pyenv/bin/python3" 3 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tutorial: Computer Vision and Machine Learning with Python, Keras and OpenCV 2 | 3 | ### Includes a demonstration of concepts with Gesture Recognition. 4 | This was created as part of an educational for the [Western Founders Network](https://foundersnetwork.ca/) computer vision and machine learning educational session. 5 | 6 | ## Demo 7 | 8 | The final demo can be seen [here](https://www.youtube.com/watch?v=IJV11OGTNT8) and below: 9 | 10 | 11 | 12 | ## Contents 13 | [notebook.ipynb](https://github.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/blob/master/notebook.ipynb) contains a full tutorial of basic computer vision and machine learning concepts, including: 14 | 15 | * *What computers see* 16 | * Image Filters and Functions 17 | - Blurring 18 | - Dilating 19 | - Erosion 20 | - Canny Edge Detectors 21 | - Thresholding 22 | * Background Subtraction Techniques 23 | - Using a background image to find differences 24 | - Using motion based background subtraction algorithms 25 | * Contours 26 | - Finding and sorting contours 27 | * Tracking 28 | * (Deep) Neural Networks 29 | * (Deep) Convolutional Neural Networks 30 | * Demo Project: Gesture Recognition 31 | - Extracting the subject 32 | - Tracking the hand 33 | - Collecting data 34 | - Building the Neural Network 35 | - Preparing Data for Training 36 | - Training the Network 37 | - Plotting Model History 38 | 39 | *Note: Please check the [issues](https://github.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/issues) on this repo if you're having problems with the notebook.* 40 | 41 | ## Installation Instructions ('$' means run this in terminal/command prompt, do not type '$') 42 | ### Windows: 43 | * Install Anaconda (https://www.anaconda.com/download/) or Miniconda (https://conda.io/miniconda.html) to save hard drive space 44 | * Start an Anaconda Prompt. (Search Anaconda in the start menu.) 45 | ### Mac/Linux: Manually installing packages 46 | * Install Anaconda (https://www.anaconda.com/download/) or Miniconda (https://conda.io/miniconda.html) to save hard drive space 47 | #### Mac: 48 | * For Miniconda, open terminal and navigate to the directory you downloaded Miniconda3-latest-MacOSX-x86_64.sh to and run: 49 | 50 | $ bash Miniconda3-latest-MacOSX-x86_64.sh 51 | 52 | * For Anaconda, double click the Anaconda3-5.0.1-MacOSX-x86_64.pkg file you downloaded 53 | 54 | #### Linux: 55 | * For Miniconda, open a terminal and navigate to the directory you downloaded Miniconda3-latest-Linux-x86_64.sh to and run: 56 | 57 | $ bash Miniconda3-latest-Linux-x86_64.sh 58 | 59 | * For Anaconda, open a terminal and navigate to the directory you downloaded Anaconda3-5.0.1-Linux-x86_64.sh to and run: 60 | 61 | $ bash Anaconda3-5.0.1-Linux-x86_64.sh 62 | 63 | ### All Platforms: 64 | * Create and activate a Python 3.5 conda environment called cv. 65 | 66 | $ conda create -n cv python=3.5 67 | 68 | $ source activate cv 69 | * Install Numpy (http://www.numpy.org/) 70 | 71 | $ conda install numpy 72 | * Install Matplotlib (https://matplotlib.org/) 73 | 74 | $ conda install matplotlib 75 | * Install Keras (https://keras.io/) 76 | 77 | $ conda install keras 78 | - This should also install tensorflow 79 | * Install h5py (http://www.h5py.org/) 80 | 81 | $ conda install h5py 82 | * Install Jupyter Notebook (http://jupyter.org/) 83 | 84 | $ conda install jupyter notebook 85 | * Install IPython (https://ipython.org/) 86 | 87 | $ conda install ipython 88 | * Install OpenCV3 (https://opencv.org/) 89 | 90 | $ conda install -c conda-forge opencv 91 | 92 | if the `import cv2` does not work with this install, try instead: 93 | 94 | $ conda install -c https://conda.anaconda.org/menpo opencv3 95 | -------------------------------------------------------------------------------- /__pycache__/utils.cpython-36.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/__pycache__/utils.cpython-36.pyc -------------------------------------------------------------------------------- /demos/__pycache__/utils.cpython-36.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/__pycache__/utils.cpython-36.pyc -------------------------------------------------------------------------------- /demos/background_subtraction.py: -------------------------------------------------------------------------------- 1 | import sys # system functions (ie. exiting the program) 2 | import os # operating system functions (ie. path building on Windows vs. MacOs) 3 | import time # for time operations 4 | import uuid # for generating unique file names 5 | import math # math functions 6 | import argparse 7 | 8 | from IPython.display import display as ipydisplay, Image, clear_output, HTML # for interacting with the notebook better 9 | 10 | import numpy as np # matrix operations (ie. difference between two matricies) 11 | import cv2 # (OpenCV) computer vision functions (ie. tracking) 12 | (major_ver, minor_ver, subminor_ver) = (cv2.__version__).split('.') 13 | print('OpenCV Version: {}.{}.{}'.format(major_ver, minor_ver, subminor_ver)) 14 | 15 | from utils import setup_tracker 16 | 17 | # Helper function for applying a mask to an array 18 | def mask_array(array, imask): 19 | if array.shape[:2] != imask.shape: 20 | raise Exception("Shapes of input and imask are incompatible") 21 | output = np.zeros_like(array, dtype=np.uint8) 22 | for i, row in enumerate(imask): 23 | output[i, row] = array[i, row] 24 | return output 25 | 26 | 27 | # Begin capturing video 28 | video = cv2.VideoCapture(0) 29 | if not video.isOpened(): 30 | print("Could not open video") 31 | sys.exit() 32 | 33 | 34 | # Read first frame 35 | ok, frame = video.read() 36 | if not ok: 37 | print("Cannot read video") 38 | sys.exit() 39 | # Use the first frame as an initial background frame 40 | bg = frame.copy() 41 | 42 | 43 | # Kernel for erosion and dilation of masks 44 | kernel = np.ones((3,3),np.uint8) 45 | 46 | 47 | # Capture, process, display loop 48 | while True: 49 | # Read a new frame 50 | ok, frame = video.read() 51 | display = frame.copy() 52 | if not ok: 53 | break 54 | 55 | 56 | # Start timer 57 | timer = cv2.getTickCount() 58 | 59 | # Processing 60 | # First find the absolute difference between the two images 61 | diff = cv2.absdiff(bg, frame) 62 | mask = cv2.cvtColor(diff, cv2.COLOR_BGR2GRAY) 63 | # Threshold the mask 64 | th, thresh = cv2.threshold(mask, 10, 255, cv2.THRESH_BINARY) 65 | # Opening, closing and dilation 66 | opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel) 67 | closing = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel) 68 | img_dilation = cv2.dilate(closing, kernel, iterations=2) 69 | # Get mask indexes 70 | imask = img_dilation > 0 71 | # Get foreground from mask 72 | foreground = mask_array(frame, imask) 73 | 74 | 75 | # Calculate Frames per second (FPS) 76 | fps = cv2.getTickFrequency() / (cv2.getTickCount() - timer) 77 | # Display FPS on frame 78 | 79 | 80 | # Display diff 81 | cv2.imshow("diff", diff) 82 | # Display thresh 83 | cv2.imshow("thresh", thresh) 84 | # Display mask 85 | cv2.imshow("img_dilation", img_dilation) 86 | # Display foreground 87 | cv2.imshow("foreground", foreground) 88 | 89 | cv2.putText(display, "FPS : " + str(int(fps)), (100, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.65, (50, 170, 50), 2) 90 | # Display result 91 | cv2.imshow("original", display) 92 | 93 | k = cv2.waitKey(1) & 0xff 94 | if k == 27: break # ESC pressed 95 | elif k == 114 or k == 112: 96 | # r pressed 97 | bg = frame.copy() 98 | elif k != 255: print(k) 99 | 100 | cv2.destroyAllWindows() 101 | video.release() -------------------------------------------------------------------------------- /demos/gesture_recognizer.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | import cv2 3 | import numpy as np 4 | from keras.models import load_model 5 | 6 | MODEL_PATH = "model/hand_model_gray.hdf5" 7 | 8 | class GestureRecognizer: 9 | """ 10 | Class for tracking the hand and labeling the gesture. 11 | """ 12 | 13 | # Mapping between the output of our model and a textual representation 14 | CLASSES = { 15 | 0: 'fist', 16 | 1: 'five', 17 | 2: 'point', 18 | 3: 'swing' 19 | } 20 | 21 | positions = { 22 | 'hand_pose': (15, 40), # hand pose text 23 | 'fps': (15, 20), # fps counter 24 | 'null_pos': (200, 200) # used as null point for mouse control 25 | } 26 | 27 | def __init__(self, capture=0, model_path=MODEL_PATH): 28 | self.bg = None 29 | self.frame = None # current frame 30 | self.tracker = cv2.TrackerKCF_create() 31 | self.kernel = np.ones((3, 3), np.uint8) 32 | self.recognizer = load_model(model_path) 33 | self.is_tracking = False 34 | self.hand_bbox = (116, 116, 170, 170) 35 | 36 | # Begin capturing video 37 | self.video = cv2.VideoCapture(capture) 38 | if not self.video.isOpened(): 39 | print("Could not open video") 40 | 41 | # def __del__(self): 42 | # cv2.destroyAllWindows() 43 | # self.video.release() 44 | 45 | # Helper function for applying a mask to an array 46 | def mask_array(self, array, imask): 47 | if array.shape[:2] != imask.shape: 48 | raise Exception("Shapes of input and imask are incompatible") 49 | output = np.zeros_like(array, dtype=np.uint8) 50 | for i, row in enumerate(imask): 51 | output[i, row] = array[i, row] 52 | return output 53 | 54 | def extract_foreground(self): 55 | # Find the absolute difference between the background frame and current frame 56 | diff = cv2.absdiff(self.bg, self.frame) 57 | mask = cv2.cvtColor(diff, cv2.COLOR_BGR2GRAY) 58 | 59 | # Threshold the mask 60 | th, thresh = cv2.threshold(mask, 10, 255, cv2.THRESH_BINARY) 61 | 62 | # Opening, closing and dilation 63 | opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, self.kernel) 64 | closing = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, self.kernel) 65 | img_dilation = cv2.dilate(closing, self.kernel, iterations=2) 66 | 67 | # Get mask indexes 68 | imask = img_dilation > 0 69 | 70 | # Get foreground from mask 71 | foreground = self.mask_array(self.frame, imask) 72 | 73 | return foreground, img_dilation 74 | 75 | def run(self): 76 | # Capture, process, display loop 77 | while True: 78 | # Read a new frame 79 | ok, self.frame = self.video.read() 80 | display = self.frame.copy() # Frame we'll do all the graphical drawing to 81 | data_display = np.zeros_like(display, dtype=np.uint8) # Black screen to display data 82 | if not ok: 83 | break 84 | 85 | # Start FPS timer 86 | timer = cv2.getTickCount() 87 | 88 | if self.bg is None: 89 | cv2.putText(display, 90 | "Press 'r' to reset the foreground extraction.", 91 | self.positions['hand_pose'], 92 | cv2.FONT_HERSHEY_SIMPLEX, 93 | 0.75, (0, 127, 64), 2) 94 | cv2.imshow("display", display) 95 | 96 | k = cv2.waitKey(1) & 0xff 97 | if k == 27: break # ESC pressed 98 | elif k == 114 or k == 108: 99 | # r pressed 100 | self.bg = self.frame.copy() 101 | self.hand_bbox = (116, 116, 170, 170) 102 | self.is_tracking = False 103 | else: 104 | # Extract the foreground 105 | foreground, mask = self.extract_foreground() 106 | foreground_display = foreground.copy() 107 | 108 | # Get hand from mask using the bounding box 109 | hand_crop = mask[int(self.hand_bbox[1]):int(self.hand_bbox[1]+self.hand_bbox[3]), 110 | int(self.hand_bbox[0]):int(self.hand_bbox[0]+self.hand_bbox[2])] 111 | 112 | # Update tracker 113 | if self.is_tracking: 114 | tracking, self.hand_bbox = self.tracker.update(foreground) 115 | 116 | try: 117 | # Resize cropped hand and make prediction on gesture 118 | hand_crop_resized = np.expand_dims(cv2.resize(hand_crop, (54, 54)), axis=0).reshape((1, 54, 54, 1)) 119 | prediction = self.recognizer.predict(hand_crop_resized) 120 | predi = prediction[0].argmax() # Get the index of the greatest confidence 121 | gesture = self.CLASSES[predi] 122 | 123 | for i, pred in enumerate(prediction[0]): 124 | # Draw confidence bar for each gesture 125 | barx = self.positions['hand_pose'][0] 126 | bary = 60 + i*60 127 | bar_height = 20 128 | bar_length = int(400 * pred) + barx # calculate length of confidence bar 129 | 130 | # Make the most confidence prediction green 131 | if i == predi: 132 | colour = (0, 255, 0) 133 | else: 134 | colour = (0, 0, 255) 135 | 136 | cv2.putText(data_display, 137 | "{}: {}".format(self.CLASSES[i], pred), 138 | (self.positions['hand_pose'][0], 30 + i*60), 139 | cv2.FONT_HERSHEY_SIMPLEX, 140 | 0.75, (255, 255, 255), 2) 141 | cv2.rectangle(data_display, 142 | (barx, bary), 143 | (bar_length, 144 | bary - bar_height), 145 | colour, 146 | -1, 1) 147 | 148 | cv2.putText(display, 149 | "hand pose: {}".format(gesture), 150 | self.positions['hand_pose'], 151 | cv2.FONT_HERSHEY_SIMPLEX, 152 | 0.75, (0, 0, 255), 2) 153 | cv2.putText(foreground_display, 154 | "hand pose: {}".format(gesture), 155 | self.positions['hand_pose'], 156 | cv2.FONT_HERSHEY_SIMPLEX, 157 | 0.75, (0, 0, 255), 2) 158 | except Exception as ex: 159 | cv2.putText(display, 160 | "hand pose: error", 161 | self.positions['hand_pose'], 162 | cv2.FONT_HERSHEY_SIMPLEX, 163 | 0.75, (0, 0, 255), 2) 164 | cv2.putText(foreground_display, 165 | "hand pose: error", 166 | self.positions['hand_pose'], 167 | cv2.FONT_HERSHEY_SIMPLEX, 168 | 0.75, (0, 0, 255), 2) 169 | 170 | # Draw bounding box 171 | p1 = (int(self.hand_bbox[0]), int(self.hand_bbox[1])) 172 | p2 = (int(self.hand_bbox[0] + self.hand_bbox[2]), int(self.hand_bbox[1] + self.hand_bbox[3])) 173 | cv2.rectangle(foreground_display, p1, p2, (255, 0, 0), 2, 1) 174 | cv2.rectangle(display, p1, p2, (255, 0, 0), 2, 1) 175 | 176 | # Calculate difference in hand position 177 | hand_pos = ((p1[0] + p2[0])//2, (p1[1] + p2[1])//2) 178 | mouse_change = ((p1[0] + p2[0])//2 - self.positions['null_pos'][0], self.positions['null_pos'][0] - (p1[1] + p2[1])//2) 179 | 180 | # Draw hand moved difference 181 | cv2.circle(display, self.positions['null_pos'], 5, (0,0,255), -1) 182 | cv2.circle(display, hand_pos, 5, (0,255,0), -1) 183 | cv2.line(display, self.positions['null_pos'], hand_pos, (255,0,0),5) 184 | 185 | # Calculate Frames per second (FPS) 186 | fps = cv2.getTickFrequency() / (cv2.getTickCount() - timer) 187 | 188 | # Display FPS on frame 189 | cv2.putText(foreground_display, "FPS : " + str(int(fps)), self.positions['fps'], cv2.FONT_HERSHEY_SIMPLEX, 0.65, (50, 170, 50), 2) 190 | cv2.putText(display, "FPS : " + str(int(fps)), self.positions['fps'], cv2.FONT_HERSHEY_SIMPLEX, 0.65, (50, 170, 50), 2) 191 | 192 | # Display pause command text 193 | cv2.putText(foreground_display, 194 | "hold 'r' to recalibrate until the screen is black", 195 | (15, 400), 196 | cv2.FONT_HERSHEY_SIMPLEX, 197 | 0.65, (0, 0, 255), 2) 198 | cv2.putText(foreground_display, 199 | "to recalibrate", 200 | (15, 420), 201 | cv2.FONT_HERSHEY_SIMPLEX, 202 | 0.65, (0, 0, 255), 2) 203 | cv2.putText(foreground_display, 204 | "press 'p' to return to paused state", 205 | (15, 450), 206 | cv2.FONT_HERSHEY_SIMPLEX, 207 | 0.65, (0, 0, 255), 2) 208 | 209 | # Display foreground_display 210 | cv2.imshow("foreground_display", foreground_display) 211 | cv2.imshow("display", display) 212 | cv2.imshow("hand_crop", hand_crop) 213 | # Display result 214 | cv2.imshow("data", data_display) 215 | 216 | k = cv2.waitKey(1) & 0xff 217 | if k == 27: break # ESC pressed 218 | elif k == 114 or k == 108: # r pressed 219 | self.bg = self.frame.copy() 220 | self.hand_bbox = (116, 116, 170, 170) 221 | self.is_tracking = False 222 | elif k == 116: # t pressed 223 | # Initialize tracker with first frame and bounding box 224 | self.is_tracking = True 225 | self.tracker = cv2.TrackerKCF_create() 226 | self.tracker.init(self.frame, self.hand_bbox) 227 | elif k == 112: # p pressed 228 | # Reset to paused state 229 | self.is_tracking = False 230 | self.bg = None 231 | cv2.destroyAllWindows() 232 | elif k != 255: print(k) 233 | 234 | if __name__ == "__main__": 235 | recognizer = GestureRecognizer(0) 236 | recognizer.run() -------------------------------------------------------------------------------- /demos/images/moving_subject.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/images/moving_subject.mp4 -------------------------------------------------------------------------------- /demos/images/moving_subject_scale.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/images/moving_subject_scale.mp4 -------------------------------------------------------------------------------- /demos/images/running.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/images/running.mp4 -------------------------------------------------------------------------------- /demos/model/adder_model.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/model/adder_model.hdf5 -------------------------------------------------------------------------------- /demos/model/hand_model_gray.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/demos/model/hand_model_gray.hdf5 -------------------------------------------------------------------------------- /demos/model/model_history.txt: -------------------------------------------------------------------------------- 1 | Epoch 1/50 2 | 125/125 [==============================] - 6s 47ms/step - loss: 11.9161 - acc: 0.2601 - val_loss: 11.5849 - val_acc: 0.2812 3 | Epoch 2/50 4 | 125/125 [==============================] - 6s 47ms/step - loss: 12.0070 - acc: 0.2551 - val_loss: 11.5849 - val_acc: 0.2812 5 | Epoch 3/50 6 | 125/125 [==============================] - 6s 46ms/step - loss: 11.9408 - acc: 0.2581 - val_loss: 11.5849 - val_acc: 0.2812 7 | Epoch 4/50 8 | 125/125 [==============================] - 6s 48ms/step - loss: 11.8833 - acc: 0.2617 - val_loss: 11.5849 - val_acc: 0.2812 9 | Epoch 5/50 10 | 125/125 [==============================] - 6s 47ms/step - loss: 11.9187 - acc: 0.2581 - val_loss: 11.5849 - val_acc: 0.2812 11 | Epoch 6/50 12 | 125/125 [==============================] - 6s 51ms/step - loss: 12.0361 - acc: 0.2505 - val_loss: 11.5849 - val_acc: 0.2812 13 | Epoch 7/50 14 | 125/125 [==============================] - 6s 48ms/step - loss: 11.9017 - acc: 0.2604 - val_loss: 11.5849 - val_acc: 0.2812 15 | Epoch 8/50 16 | 125/125 [==============================] - 6s 48ms/step - loss: 11.9578 - acc: 0.2572 - val_loss: 11.5849 - val_acc: 0.2812 17 | Epoch 9/50 18 | 125/125 [==============================] - 6s 47ms/step - loss: 9.9074 - acc: 0.2650 - val_loss: 1.3807 - val_acc: 0.2240 19 | Epoch 10/50 20 | 125/125 [==============================] - 6s 46ms/step - loss: 1.3254 - acc: 0.3456 - val_loss: 1.1493 - val_acc: 0.5260 21 | Epoch 11/50 22 | 125/125 [==============================] - 6s 48ms/step - loss: 1.2466 - acc: 0.4234 - val_loss: 0.9881 - val_acc: 0.6823 23 | Epoch 12/50 24 | 125/125 [==============================] - 6s 49ms/step - loss: 1.1583 - acc: 0.5020 - val_loss: 0.7385 - val_acc: 0.7083 25 | Epoch 13/50 26 | 125/125 [==============================] - 6s 46ms/step - loss: 1.0516 - acc: 0.5400 - val_loss: 0.5796 - val_acc: 0.8594 27 | Epoch 14/50 28 | 125/125 [==============================] - 6s 46ms/step - loss: 0.8610 - acc: 0.6300 - val_loss: 0.3554 - val_acc: 0.8542 29 | Epoch 15/50 30 | 125/125 [==============================] - 6s 47ms/step - loss: 0.7976 - acc: 0.6725 - val_loss: 0.2068 - val_acc: 0.9531 31 | Epoch 16/50 32 | 125/125 [==============================] - 6s 46ms/step - loss: 0.7047 - acc: 0.7019 - val_loss: 0.2234 - val_acc: 0.9635 33 | Epoch 17/50 34 | 125/125 [==============================] - 6s 46ms/step - loss: 0.6517 - acc: 0.7343 - val_loss: 0.1734 - val_acc: 0.9688 35 | Epoch 18/50 36 | 125/125 [==============================] - 6s 46ms/step - loss: 0.5707 - acc: 0.7683 - val_loss: 0.1242 - val_acc: 0.9792 37 | Epoch 19/50 38 | 125/125 [==============================] - 6s 46ms/step - loss: 0.5629 - acc: 0.7815 - val_loss: 0.1700 - val_acc: 0.9479 39 | Epoch 20/50 40 | 125/125 [==============================] - 6s 47ms/step - loss: 0.5375 - acc: 0.7920 - val_loss: 0.1190 - val_acc: 0.9896 41 | Epoch 21/50 42 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4960 - acc: 0.8100 - val_loss: 0.0803 - val_acc: 0.9844 43 | Epoch 22/50 44 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4834 - acc: 0.8275 - val_loss: 0.1308 - val_acc: 0.9688 45 | Epoch 23/50 46 | 125/125 [==============================] - 6s 47ms/step - loss: 0.4031 - acc: 0.8570 - val_loss: 0.1189 - val_acc: 0.9688 47 | Epoch 24/50 48 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4140 - acc: 0.8509 - val_loss: 0.0801 - val_acc: 0.9844 49 | Epoch 25/50 50 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4094 - acc: 0.8488 - val_loss: 0.0960 - val_acc: 0.9740 51 | Epoch 26/50 52 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3757 - acc: 0.8645 - val_loss: 0.0712 - val_acc: 0.9740 53 | Epoch 27/50 54 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3576 - acc: 0.8603 - val_loss: 0.1682 - val_acc: 0.9167 55 | Epoch 28/50 56 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3522 - acc: 0.8710 - val_loss: 0.0303 - val_acc: 0.9948 57 | Epoch 29/50 58 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3299 - acc: 0.8860 - val_loss: 0.0979 - val_acc: 0.9688 59 | Epoch 30/50 60 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3207 - acc: 0.8885 - val_loss: 0.0451 - val_acc: 0.9844 61 | Epoch 31/50 62 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3240 - acc: 0.8854 - val_loss: 0.0609 - val_acc: 0.9896 63 | Epoch 32/50 64 | 125/125 [==============================] - 6s 49ms/step - loss: 0.3174 - acc: 0.8864 - val_loss: 0.0661 - val_acc: 0.9844 65 | Epoch 33/50 66 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3291 - acc: 0.8905 - val_loss: 0.0304 - val_acc: 0.9948 67 | Epoch 34/50 68 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2965 - acc: 0.8994 - val_loss: 0.0424 - val_acc: 0.9948 69 | Epoch 35/50 70 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3055 - acc: 0.8969 - val_loss: 0.0611 - val_acc: 0.9844 71 | Epoch 36/50 72 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3180 - acc: 0.8840 - val_loss: 0.0538 - val_acc: 0.9896 73 | Epoch 37/50 74 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2719 - acc: 0.9079 - val_loss: 0.0549 - val_acc: 0.9792 75 | Epoch 38/50 76 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2625 - acc: 0.9065 - val_loss: 0.0257 - val_acc: 0.9948 77 | Epoch 39/50 78 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2612 - acc: 0.9105 - val_loss: 0.0555 - val_acc: 0.9896 79 | Epoch 40/50 80 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2641 - acc: 0.9075 - val_loss: 0.0722 - val_acc: 0.9896 81 | Epoch 41/50 82 | 125/125 [==============================] - 6s 45ms/step - loss: 0.2635 - acc: 0.9110 - val_loss: 0.0436 - val_acc: 0.9896 83 | Epoch 42/50 84 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2446 - acc: 0.9174 - val_loss: 0.0343 - val_acc: 1.0000 85 | Epoch 43/50 86 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2459 - acc: 0.9150 - val_loss: 0.0529 - val_acc: 0.9844 87 | Epoch 44/50 88 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2770 - acc: 0.9035 - val_loss: 0.1011 - val_acc: 0.9792 89 | Epoch 45/50 90 | 125/125 [==============================] - 6s 48ms/step - loss: 0.2289 - acc: 0.9205 - val_loss: 0.0273 - val_acc: 0.9896 91 | Epoch 46/50 92 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2426 - acc: 0.9175 - val_loss: 0.0295 - val_acc: 0.9948 93 | Epoch 47/50 94 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2487 - acc: 0.9190 - val_loss: 0.0369 - val_acc: 0.9844 95 | Epoch 48/50 96 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2229 - acc: 0.9229 - val_loss: 0.0469 - val_acc: 0.9740 97 | Epoch 49/50 98 | 125/125 [==============================] - 6s 48ms/step - loss: 0.2392 - acc: 0.9185 - val_loss: 0.0485 - val_acc: 0.9896 99 | Epoch 50/50 100 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2300 - acc: 0.9210 - val_loss: 0.0205 - val_acc: 1.0000 -------------------------------------------------------------------------------- /demos/model/model_json.json: -------------------------------------------------------------------------------- 1 | "{\"backend\": \"tensorflow\", \"config\": [{\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"filters\": 32, \"name\": \"conv2d_1\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"same\", \"bias_regularizer\": null, \"dtype\": \"float32\", \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"batch_input_shape\": [null, 28, 28, 1], \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"trainable\": true, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_1\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_2\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"valid\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 32, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_2\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [2, 2], \"padding\": \"valid\", \"pool_size\": [2, 2], \"name\": \"max_pooling2d_1\", \"data_format\": \"channels_last\", \"trainable\": true}, \"class_name\": \"MaxPooling2D\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.25, \"seed\": null, \"trainable\": true, \"name\": \"dropout_1\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_3\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"same\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 64, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_3\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_4\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"valid\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 64, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_4\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [2, 2], \"padding\": \"valid\", \"pool_size\": [2, 2], \"name\": \"max_pooling2d_2\", \"data_format\": \"channels_last\", \"trainable\": true}, \"class_name\": \"MaxPooling2D\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.25, \"seed\": null, \"trainable\": true, \"name\": \"dropout_2\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"trainable\": true, \"name\": \"flatten_1\"}, \"class_name\": \"Flatten\"}, {\"config\": {\"activation\": \"linear\", \"name\": \"dense_1\", \"units\": 512, \"kernel_constraint\": null, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"bias_constraint\": null, \"bias_regularizer\": null, \"kernel_regularizer\": null, \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"activity_regularizer\": null, \"trainable\": true, \"use_bias\": true}, \"class_name\": \"Dense\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_5\"}, \"class_name\": \"Activation\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.5, \"seed\": null, \"trainable\": true, \"name\": \"dropout_3\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"activation\": \"linear\", \"name\": \"dense_2\", \"units\": 24, \"kernel_constraint\": null, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"bias_constraint\": null, \"bias_regularizer\": null, \"kernel_regularizer\": null, \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"activity_regularizer\": null, \"trainable\": true, \"use_bias\": true}, \"class_name\": \"Dense\"}, {\"config\": {\"activation\": \"softmax\", \"trainable\": true, \"name\": \"activation_6\"}, \"class_name\": \"Activation\"}], \"keras_version\": \"2.1.2\", \"class_name\": \"Sequential\"}" -------------------------------------------------------------------------------- /demos/tracker.py: -------------------------------------------------------------------------------- 1 | import sys # system functions (ie. exiting the program) 2 | import os # operating system functions (ie. path building on Windows vs. MacOs) 3 | import time # for time operations 4 | import uuid # for generating unique file names 5 | import math # math functions 6 | import argparse 7 | 8 | from IPython.display import display as ipydisplay, Image, clear_output, HTML # for interacting with the notebook better 9 | 10 | import numpy as np # matrix operations (ie. difference between two matricies) 11 | import cv2 # (OpenCV) computer vision functions (ie. tracking) 12 | (major_ver, minor_ver, subminor_ver) = (cv2.__version__).split('.') 13 | print('OpenCV Version: {}.{}.{}'.format(major_ver, minor_ver, subminor_ver)) 14 | 15 | from utils import setup_tracker 16 | 17 | parser = argparse.ArgumentParser(description='Process some integers.') 18 | parser.add_argument('src', default='video') 19 | args = parser.parse_args() 20 | 21 | try: 22 | src = int(args.src) 23 | video = cv2.VideoCapture(src) 24 | except ValueError: 25 | if args.src == 'running': 26 | video = cv2.VideoCapture(os.path.join('images', 'running.mp4')) 27 | elif args.src == 'bottle': 28 | video = cv2.VideoCapture(os.path.join('images', 'moving_subject.mp4')) 29 | else: 30 | raise Exception('video not found') 31 | 32 | 33 | cv2.namedWindow('frame', cv2.WINDOW_KEEPRATIO) 34 | 35 | # Read first frame 36 | success, frame = video.read() 37 | if not success: 38 | print("first frame not read") 39 | sys.exit() 40 | 41 | tracker = setup_tracker(4) 42 | 43 | # Select roi for bbox 44 | bbox = cv2.selectROI('frame', frame, False) 45 | cv2.destroyAllWindows() 46 | 47 | # Initialize tracker with first frame and bounding box 48 | tracking_success = tracker.init(frame, bbox) 49 | 50 | while True: 51 | time.sleep(0.02) 52 | 53 | timer = cv2.getTickCount() 54 | 55 | # Read a new frame 56 | success, frame = video.read() 57 | if not success: 58 | # Frame not successfully read from video capture 59 | break 60 | 61 | # Update tracker 62 | tracking_success, bbox = tracker.update(frame) 63 | 64 | # Draw bounding box 65 | if tracking_success: 66 | # Tracking success 67 | p1 = (int(bbox[0]), int(bbox[1])) 68 | p2 = (int(bbox[0] + bbox[2]), int(bbox[1] + bbox[3])) 69 | cv2.rectangle(frame, p1, p2, (255, 0, 0), 2, 1) 70 | else: 71 | # Tracking failure 72 | cv2.putText(frame, "Tracking failure detected", (100, 80), cv2.FONT_HERSHEY_SIMPLEX, 0.75, (0, 0, 255), 2) 73 | 74 | # Calculate Frames per second (FPS) 75 | fps = cv2.getTickFrequency() / (cv2.getTickCount() - timer) 76 | # Display FPS on frame 77 | cv2.putText(frame, "FPS : " + str(int(fps)), (100, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.65, (255, 0, 0), 2) 78 | 79 | # Display result 80 | cv2.namedWindow('frame', cv2.WINDOW_KEEPRATIO) 81 | cv2.imshow("frame", frame) 82 | 83 | k = cv2.waitKey(1) & 0xff 84 | if k == 27: break # ESC pressed 85 | 86 | cv2.destroyAllWindows() 87 | video.release() -------------------------------------------------------------------------------- /demos/utils.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | 3 | # Set up tracker. 4 | def setup_tracker(ttype): 5 | tracker_types = ['BOOSTING', 'MIL', 'KCF', 'TLD', 'MEDIANFLOW', 'GOTURN'] 6 | tracker_type = tracker_types[ttype] 7 | 8 | if tracker_type == 'BOOSTING': 9 | tracker = cv2.TrackerBoosting_create() 10 | if tracker_type == 'MIL': 11 | tracker = cv2.TrackerMIL_create() 12 | if tracker_type == 'KCF': 13 | tracker = cv2.TrackerKCF_create() 14 | if tracker_type == 'TLD': 15 | tracker = cv2.TrackerTLD_create() 16 | if tracker_type == 'MEDIANFLOW': 17 | tracker = cv2.TrackerMedianFlow_create() 18 | if tracker_type == 'GOTURN': 19 | tracker = cv2.TrackerGOTURN_create() 20 | 21 | return tracker -------------------------------------------------------------------------------- /images/bg.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/bg.jpg -------------------------------------------------------------------------------- /images/bg_subtract_movement.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/bg_subtract_movement.mp4 -------------------------------------------------------------------------------- /images/cnn1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn1.png -------------------------------------------------------------------------------- /images/cnn2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn2.png -------------------------------------------------------------------------------- /images/cnn3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn3.png -------------------------------------------------------------------------------- /images/cnn4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn4.png -------------------------------------------------------------------------------- /images/cnn5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn5.png -------------------------------------------------------------------------------- /images/cnn6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn6.png -------------------------------------------------------------------------------- /images/cnn7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/cnn7.png -------------------------------------------------------------------------------- /images/current_frame.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/current_frame.jpg -------------------------------------------------------------------------------- /images/dnn.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/dnn.jpg -------------------------------------------------------------------------------- /images/earth.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/earth.jpg -------------------------------------------------------------------------------- /images/neuralnetwork.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/neuralnetwork.png -------------------------------------------------------------------------------- /images/nnslides/Slide10.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/nnslides/Slide10.PNG -------------------------------------------------------------------------------- /images/nnslides/Slide11.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/nnslides/Slide11.PNG -------------------------------------------------------------------------------- /images/nnslides/Slide8.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/nnslides/Slide8.PNG -------------------------------------------------------------------------------- /images/preview/fist_0_1039.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1039.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1228.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1228.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1285.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1285.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1311.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1311.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1438.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1438.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1523.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1523.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_1773.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_1773.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_282.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_282.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_351.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_351.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_423.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_423.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_481.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_481.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_510.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_510.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_635.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_635.jpeg -------------------------------------------------------------------------------- /images/preview/fist_0_975.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/images/preview/fist_0_975.jpeg -------------------------------------------------------------------------------- /model/adder_model.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/model/adder_model.hdf5 -------------------------------------------------------------------------------- /model/hand_model_gray.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV/dfd76833fee1000c1d891386c14e589402294544/model/hand_model_gray.hdf5 -------------------------------------------------------------------------------- /model/model_history.txt: -------------------------------------------------------------------------------- 1 | Epoch 1/50 2 | 125/125 [==============================] - 6s 47ms/step - loss: 11.9161 - acc: 0.2601 - val_loss: 11.5849 - val_acc: 0.2812 3 | Epoch 2/50 4 | 125/125 [==============================] - 6s 47ms/step - loss: 12.0070 - acc: 0.2551 - val_loss: 11.5849 - val_acc: 0.2812 5 | Epoch 3/50 6 | 125/125 [==============================] - 6s 46ms/step - loss: 11.9408 - acc: 0.2581 - val_loss: 11.5849 - val_acc: 0.2812 7 | Epoch 4/50 8 | 125/125 [==============================] - 6s 48ms/step - loss: 11.8833 - acc: 0.2617 - val_loss: 11.5849 - val_acc: 0.2812 9 | Epoch 5/50 10 | 125/125 [==============================] - 6s 47ms/step - loss: 11.9187 - acc: 0.2581 - val_loss: 11.5849 - val_acc: 0.2812 11 | Epoch 6/50 12 | 125/125 [==============================] - 6s 51ms/step - loss: 12.0361 - acc: 0.2505 - val_loss: 11.5849 - val_acc: 0.2812 13 | Epoch 7/50 14 | 125/125 [==============================] - 6s 48ms/step - loss: 11.9017 - acc: 0.2604 - val_loss: 11.5849 - val_acc: 0.2812 15 | Epoch 8/50 16 | 125/125 [==============================] - 6s 48ms/step - loss: 11.9578 - acc: 0.2572 - val_loss: 11.5849 - val_acc: 0.2812 17 | Epoch 9/50 18 | 125/125 [==============================] - 6s 47ms/step - loss: 9.9074 - acc: 0.2650 - val_loss: 1.3807 - val_acc: 0.2240 19 | Epoch 10/50 20 | 125/125 [==============================] - 6s 46ms/step - loss: 1.3254 - acc: 0.3456 - val_loss: 1.1493 - val_acc: 0.5260 21 | Epoch 11/50 22 | 125/125 [==============================] - 6s 48ms/step - loss: 1.2466 - acc: 0.4234 - val_loss: 0.9881 - val_acc: 0.6823 23 | Epoch 12/50 24 | 125/125 [==============================] - 6s 49ms/step - loss: 1.1583 - acc: 0.5020 - val_loss: 0.7385 - val_acc: 0.7083 25 | Epoch 13/50 26 | 125/125 [==============================] - 6s 46ms/step - loss: 1.0516 - acc: 0.5400 - val_loss: 0.5796 - val_acc: 0.8594 27 | Epoch 14/50 28 | 125/125 [==============================] - 6s 46ms/step - loss: 0.8610 - acc: 0.6300 - val_loss: 0.3554 - val_acc: 0.8542 29 | Epoch 15/50 30 | 125/125 [==============================] - 6s 47ms/step - loss: 0.7976 - acc: 0.6725 - val_loss: 0.2068 - val_acc: 0.9531 31 | Epoch 16/50 32 | 125/125 [==============================] - 6s 46ms/step - loss: 0.7047 - acc: 0.7019 - val_loss: 0.2234 - val_acc: 0.9635 33 | Epoch 17/50 34 | 125/125 [==============================] - 6s 46ms/step - loss: 0.6517 - acc: 0.7343 - val_loss: 0.1734 - val_acc: 0.9688 35 | Epoch 18/50 36 | 125/125 [==============================] - 6s 46ms/step - loss: 0.5707 - acc: 0.7683 - val_loss: 0.1242 - val_acc: 0.9792 37 | Epoch 19/50 38 | 125/125 [==============================] - 6s 46ms/step - loss: 0.5629 - acc: 0.7815 - val_loss: 0.1700 - val_acc: 0.9479 39 | Epoch 20/50 40 | 125/125 [==============================] - 6s 47ms/step - loss: 0.5375 - acc: 0.7920 - val_loss: 0.1190 - val_acc: 0.9896 41 | Epoch 21/50 42 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4960 - acc: 0.8100 - val_loss: 0.0803 - val_acc: 0.9844 43 | Epoch 22/50 44 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4834 - acc: 0.8275 - val_loss: 0.1308 - val_acc: 0.9688 45 | Epoch 23/50 46 | 125/125 [==============================] - 6s 47ms/step - loss: 0.4031 - acc: 0.8570 - val_loss: 0.1189 - val_acc: 0.9688 47 | Epoch 24/50 48 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4140 - acc: 0.8509 - val_loss: 0.0801 - val_acc: 0.9844 49 | Epoch 25/50 50 | 125/125 [==============================] - 6s 46ms/step - loss: 0.4094 - acc: 0.8488 - val_loss: 0.0960 - val_acc: 0.9740 51 | Epoch 26/50 52 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3757 - acc: 0.8645 - val_loss: 0.0712 - val_acc: 0.9740 53 | Epoch 27/50 54 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3576 - acc: 0.8603 - val_loss: 0.1682 - val_acc: 0.9167 55 | Epoch 28/50 56 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3522 - acc: 0.8710 - val_loss: 0.0303 - val_acc: 0.9948 57 | Epoch 29/50 58 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3299 - acc: 0.8860 - val_loss: 0.0979 - val_acc: 0.9688 59 | Epoch 30/50 60 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3207 - acc: 0.8885 - val_loss: 0.0451 - val_acc: 0.9844 61 | Epoch 31/50 62 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3240 - acc: 0.8854 - val_loss: 0.0609 - val_acc: 0.9896 63 | Epoch 32/50 64 | 125/125 [==============================] - 6s 49ms/step - loss: 0.3174 - acc: 0.8864 - val_loss: 0.0661 - val_acc: 0.9844 65 | Epoch 33/50 66 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3291 - acc: 0.8905 - val_loss: 0.0304 - val_acc: 0.9948 67 | Epoch 34/50 68 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2965 - acc: 0.8994 - val_loss: 0.0424 - val_acc: 0.9948 69 | Epoch 35/50 70 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3055 - acc: 0.8969 - val_loss: 0.0611 - val_acc: 0.9844 71 | Epoch 36/50 72 | 125/125 [==============================] - 6s 46ms/step - loss: 0.3180 - acc: 0.8840 - val_loss: 0.0538 - val_acc: 0.9896 73 | Epoch 37/50 74 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2719 - acc: 0.9079 - val_loss: 0.0549 - val_acc: 0.9792 75 | Epoch 38/50 76 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2625 - acc: 0.9065 - val_loss: 0.0257 - val_acc: 0.9948 77 | Epoch 39/50 78 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2612 - acc: 0.9105 - val_loss: 0.0555 - val_acc: 0.9896 79 | Epoch 40/50 80 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2641 - acc: 0.9075 - val_loss: 0.0722 - val_acc: 0.9896 81 | Epoch 41/50 82 | 125/125 [==============================] - 6s 45ms/step - loss: 0.2635 - acc: 0.9110 - val_loss: 0.0436 - val_acc: 0.9896 83 | Epoch 42/50 84 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2446 - acc: 0.9174 - val_loss: 0.0343 - val_acc: 1.0000 85 | Epoch 43/50 86 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2459 - acc: 0.9150 - val_loss: 0.0529 - val_acc: 0.9844 87 | Epoch 44/50 88 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2770 - acc: 0.9035 - val_loss: 0.1011 - val_acc: 0.9792 89 | Epoch 45/50 90 | 125/125 [==============================] - 6s 48ms/step - loss: 0.2289 - acc: 0.9205 - val_loss: 0.0273 - val_acc: 0.9896 91 | Epoch 46/50 92 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2426 - acc: 0.9175 - val_loss: 0.0295 - val_acc: 0.9948 93 | Epoch 47/50 94 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2487 - acc: 0.9190 - val_loss: 0.0369 - val_acc: 0.9844 95 | Epoch 48/50 96 | 125/125 [==============================] - 6s 47ms/step - loss: 0.2229 - acc: 0.9229 - val_loss: 0.0469 - val_acc: 0.9740 97 | Epoch 49/50 98 | 125/125 [==============================] - 6s 48ms/step - loss: 0.2392 - acc: 0.9185 - val_loss: 0.0485 - val_acc: 0.9896 99 | Epoch 50/50 100 | 125/125 [==============================] - 6s 46ms/step - loss: 0.2300 - acc: 0.9210 - val_loss: 0.0205 - val_acc: 1.0000 -------------------------------------------------------------------------------- /model/model_json.json: -------------------------------------------------------------------------------- 1 | "{\"backend\": \"tensorflow\", \"config\": [{\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"filters\": 32, \"name\": \"conv2d_1\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"same\", \"bias_regularizer\": null, \"dtype\": \"float32\", \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"batch_input_shape\": [null, 28, 28, 1], \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"trainable\": true, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_1\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_2\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"valid\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 32, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_2\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [2, 2], \"padding\": \"valid\", \"pool_size\": [2, 2], \"name\": \"max_pooling2d_1\", \"data_format\": \"channels_last\", \"trainable\": true}, \"class_name\": \"MaxPooling2D\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.25, \"seed\": null, \"trainable\": true, \"name\": \"dropout_1\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_3\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"same\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 64, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_3\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [1, 1], \"activation\": \"linear\", \"name\": \"conv2d_4\", \"data_format\": \"channels_last\", \"kernel_constraint\": null, \"trainable\": true, \"dilation_rate\": [1, 1], \"bias_constraint\": null, \"padding\": \"valid\", \"bias_regularizer\": null, \"kernel_size\": [3, 3], \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"activity_regularizer\": null, \"filters\": 64, \"kernel_regularizer\": null, \"use_bias\": true}, \"class_name\": \"Conv2D\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_4\"}, \"class_name\": \"Activation\"}, {\"config\": {\"strides\": [2, 2], \"padding\": \"valid\", \"pool_size\": [2, 2], \"name\": \"max_pooling2d_2\", \"data_format\": \"channels_last\", \"trainable\": true}, \"class_name\": \"MaxPooling2D\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.25, \"seed\": null, \"trainable\": true, \"name\": \"dropout_2\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"trainable\": true, \"name\": \"flatten_1\"}, \"class_name\": \"Flatten\"}, {\"config\": {\"activation\": \"linear\", \"name\": \"dense_1\", \"units\": 512, \"kernel_constraint\": null, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"bias_constraint\": null, \"bias_regularizer\": null, \"kernel_regularizer\": null, \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"activity_regularizer\": null, \"trainable\": true, \"use_bias\": true}, \"class_name\": \"Dense\"}, {\"config\": {\"activation\": \"relu\", \"trainable\": true, \"name\": \"activation_5\"}, \"class_name\": \"Activation\"}, {\"config\": {\"noise_shape\": null, \"rate\": 0.5, \"seed\": null, \"trainable\": true, \"name\": \"dropout_3\"}, \"class_name\": \"Dropout\"}, {\"config\": {\"activation\": \"linear\", \"name\": \"dense_2\", \"units\": 24, \"kernel_constraint\": null, \"kernel_initializer\": {\"config\": {\"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\", \"distribution\": \"uniform\"}, \"class_name\": \"VarianceScaling\"}, \"bias_constraint\": null, \"bias_regularizer\": null, \"kernel_regularizer\": null, \"bias_initializer\": {\"config\": {}, \"class_name\": \"Zeros\"}, \"activity_regularizer\": null, \"trainable\": true, \"use_bias\": true}, \"class_name\": \"Dense\"}, {\"config\": {\"activation\": \"softmax\", \"trainable\": true, \"name\": \"activation_6\"}, \"class_name\": \"Activation\"}], \"keras_version\": \"2.1.2\", \"class_name\": \"Sequential\"}" --------------------------------------------------------------------------------