├── .gitignore ├── LICENSE ├── README.md ├── cli_commands.txt ├── object_detection_multi_framework.py ├── openvino_basic_object_detection.py ├── openvino_face_detection.py └── openvino_object_detection_configs.py /.gitignore: -------------------------------------------------------------------------------- 1 | venv/ 2 | .idea/ 3 | models/ 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Onur Dundar onur.dundar1@gmail.com 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Introduction 2 | 3 | This repo contains couple python sample applications to teach about Intel(R) Distribution of OpenVINO(TM). 4 | 5 | ## Object Detection Application 6 | 7 | `openvino_basic_object_detection.py` 8 | 9 | This applications intends to showcase how a model is being used with OpenVINO(TM) Toolkit. 10 | 11 | Aim is to show initial use case of Inference Engine API and Async Mode. 12 | 13 | ```bash 14 | usage: openvino_basic_object_detection.py [-h] [--model-xml MODEL_XML] 15 | [--model-bin MODEL_BIN] 16 | [--target-device TARGET_DEVICE] 17 | [--input-type INPUT_TYPE] 18 | [--input INPUT] 19 | [--detection-threshold DETECTION_THRESHOLD] 20 | [--async] 21 | [--request-number REQUEST_NUMBER] 22 | 23 | Basic OpenVINO Example with MobileNet-SSD 24 | 25 | optional arguments: 26 | -h, --help show this help message and exit 27 | --model-xml MODEL_XML 28 | XML File 29 | --model-bin MODEL_BIN 30 | BIN File 31 | --target-device TARGET_DEVICE 32 | Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, 33 | HETERO:FPGA,CPU 34 | --input-type INPUT_TYPE 35 | Type of Input: image, video, cam 36 | --input INPUT Path to Input: WebCam: 0, Video File or Image file 37 | --detection-threshold DETECTION_THRESHOLD 38 | Object Detection Accuracy Threshold 39 | --async Run Async Mode 40 | --request-number REQUEST_NUMBER 41 | Number of Requests 42 | ``` 43 | 44 | ### mobilenet-ssd Model Download & Conversion 45 | 46 | Downloaded model with OpenVINO(TM) Toolkit Model Downloader 47 | 48 | ```bash 49 | python3 /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py \ 50 | --name mobilenet-ssd \ 51 | --output_dir ~/openvino_models/ 52 | ``` 53 | 54 | Converted Model using OpenVINO(TM) Toolkit Model Optimizer 55 | 56 | - FP32 Conversion 57 | 58 | ```bash 59 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo.py \ 60 | --input_model ~/openvino_models/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel \ 61 | --output_dir ~/openvino_models/object_detection/common/mobilenet-ssd/FP32 \ 62 | --data_type FP32 \ 63 | --scale 127.50223128904757 \ 64 | --input_shape [1,3,300,300] \ 65 | --mean_values [127.5,127.5,127.5] \ 66 | --framework caffe 67 | ``` 68 | 69 | - FP16 Conversion 70 | 71 | ```bash 72 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo.py \ 73 | --input_model ~/openvino_models/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel \ 74 | --output_dir ~/openvino_models/object_detection/common/mobilenet-ssd/FP16 \ 75 | --data_type FP16 \ 76 | --scale 127.50223128904757 \ 77 | --input_shape [1,3,300,300] \ 78 | --mean_values [127.5,127.5,127.5] \ 79 | --framework caffe 80 | ``` 81 | 82 | ## Face Detection Application 83 | 84 | `openvino_basic_object_detection.py` 85 | 86 | This application intends to showcase how two model from Open Model Zoo (https://github.com/opencv/open_model_zoo) is being used. 87 | 88 | Aim is to show initial use case of Inference Engine API Dynamic Batching with using Face Detection and Gender-Age Detection. 89 | 90 | ```bash 91 | usage: openvino_face_detection.py [-h] [--face-model-xml FACE_MODEL_XML] 92 | [--face-model-bin FACE_MODEL_BIN] 93 | [--face-detection-threshold FACE_DETECTION_THRESHOLD] 94 | [--face-target-device FACE_TARGET_DEVICE] 95 | [--ag-model-xml AG_MODEL_XML] 96 | [--ag-model-bin AG_MODEL_BIN] 97 | [--ag-max-batch-size AG_MAX_BATCH_SIZE] 98 | [--ag-dynamic-batch] 99 | [--ag-target-device AG_TARGET_DEVICE] 100 | [--input-type INPUT_TYPE] [--input INPUT] 101 | 102 | Basic OpenVINO Example to Face, Age and Gender Detection 103 | 104 | optional arguments: 105 | -h, --help show this help message and exit 106 | --face-model-xml FACE_MODEL_XML 107 | Face Detection Model XML File 108 | --face-model-bin FACE_MODEL_BIN 109 | Face Detection Model BIN File 110 | --face-detection-threshold FACE_DETECTION_THRESHOLD 111 | Face Detection Accuracy Threshold 112 | --face-target-device FACE_TARGET_DEVICE 113 | Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, 114 | HETERO:FPGA,CPU 115 | --ag-model-xml AG_MODEL_XML 116 | Age-Gender Detection XML File 117 | --ag-model-bin AG_MODEL_BIN 118 | Age-Gender Detection BIN File 119 | --ag-max-batch-size AG_MAX_BATCH_SIZE 120 | Age Gender Detection Max Batch Size 121 | --ag-dynamic-batch Age Gender Detection Enable Dynamic Batching 122 | --ag-target-device AG_TARGET_DEVICE 123 | Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, 124 | HETERO:FPGA,CPU 125 | --input-type INPUT_TYPE 126 | Type of Input: image, video, cam 127 | --input INPUT Path to Input: WebCam: 0, Video File or Image file 128 | ``` 129 | 130 | ### Face Detection Model 131 | 132 | Downloaded Face Detection model as below: 133 | 134 | ```bash 135 | python3 /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py \ 136 | --name face-detection-retail* \ 137 | --output_dir ~/openvino_models/ 138 | ``` 139 | 140 | Detailed information about the model: 141 | 142 | - https://github.com/opencv/open_model_zoo/blob/master/intel_models/face-detection-retail-0004/description/face-detection-retail-0004.md 143 | 144 | ### Age Gender Detection Model 145 | 146 | Downloaded Age Gender Detection Model as below: 147 | 148 | ```bash 149 | python3 /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py \ 150 | --name age-gender-recognition-retail* \ 151 | --output_dir ~/openvino_models/ 152 | ``` 153 | 154 | - https://github.com/opencv/open_model_zoo/blob/master/intel_models/age-gender-recognition-retail-0013/description/age-gender-recognition-retail-0013.md 155 | 156 | ## Object Detection Application with Configurations 157 | 158 | `openvino_object_detection_configs.py` 159 | 160 | This application is similar to Object Detection Application above, however some additional configurations options added to see the performance and understand their effect to overall performance. 161 | 162 | ```bash 163 | usage: openvino_object_detection_configs.py [-h] [--model-xml MODEL_XML] 164 | [--model-bin MODEL_BIN] 165 | [--target-device TARGET_DEVICE] 166 | [--input-type INPUT_TYPE] 167 | [--input INPUT] 168 | [--detection-threshold DETECTION_THRESHOLD] 169 | [--async] 170 | [--request-number REQUEST_NUMBER] 171 | [--pc] 172 | [--cpu-num-threads CPU_NUM_THREADS] 173 | [--cpu-bind-thread] 174 | [--cpu-throughput-streams CPU_THROUGHPUT_STREAMS] 175 | [--gpu-throughput-streams GPU_THROUGHPUT_STREAMS] 176 | [--gpu-throttle] 177 | 178 | Basic OpenVINO Example to Test Object Detection Model 179 | 180 | optional arguments: 181 | -h, --help show this help message and exit 182 | --model-xml MODEL_XML 183 | XML File 184 | --model-bin MODEL_BIN 185 | BIN File 186 | --target-device TARGET_DEVICE 187 | Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, 188 | HETERO:FPGA,CPU 189 | --input-type INPUT_TYPE 190 | Type of Input: image, video, cam 191 | --input INPUT Path to Input: WebCam: 0, Video File or Image file 192 | --detection-threshold DETECTION_THRESHOLD 193 | Object Detection Accuracy Threshold 194 | --async Run Async Mode 195 | --request-number REQUEST_NUMBER 196 | Number of Requests 197 | --pc Enable Performance Counters 198 | --cpu-num-threads CPU_NUM_THREADS 199 | Limit CPU Threads 200 | --cpu-bind-thread Bind Threads to CPU 201 | --cpu-throughput-streams CPU_THROUGHPUT_STREAMS 202 | Int Values or CPU_THROUGHPUT_NUMA if not set 203 | CPU_THROUGHPUT_AUTO 204 | --gpu-throughput-streams GPU_THROUGHPUT_STREAMS 205 | Int Values if not set GPU_THROUGHPUT_AUTO 206 | --gpu-throttle multi-device execution with the CPU+GPU performs best 207 | with GPU trottling hint 208 | ``` 209 | 210 | ## Object Detection Multi Framework Sample 211 | 212 | This sample application will be used to compare inference between multiple frameworks; opencv, openvino 213 | -------------------------------------------------------------------------------- /cli_commands.txt: -------------------------------------------------------------------------------- 1 | CPU: 2 | 3 | python3 object_detection_multi_platform.py --framework openvino --device CPU --mconfig /home/intel/openvino_models/ir/mobilenet-ssd/FP32/mobilenet-ssd.xml --mweight /home/intel/openvino_models/ir/mobilenet-ssd/FP32/mobilenet-ssd.bin --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc True 4 | 5 | GPU: 6 | 7 | python3 object_detection_multi_platform.py --framework openvino --device GPU --mconfig /home/intel/openvino_models/ir/mobilenet-ssd/FP32/mobilenet-ssd.xml --mweight /home/intel/openvino_models/ir/mobilenet-ssd/FP32/mobilenet-ssd.bin --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc True 8 | 9 | python3 object_detection_multi_platform.py --framework openvino --device GPU --mconfig /home/intel/openvino_models/ir/mobilenet-ssd/FP16/mobilenet-ssd.xml --mweight /home/intel/openvino_models/ir/mobilenet-ssd/FP16/mobilenet-ssd.bin --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc True 10 | 11 | MYRIAD: 12 | 13 | python3 object_detection_multi_platform.py --framework openvino --device MYRIAD --mconfig /home/intel/openvino_models/ir/mobilenet-ssd/FP16/mobilenet-ssd.xml --mweight /home/intel/openvino_models/ir/mobilenet-ssd/FP16/mobilenet-ssd.bin --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc True 14 | 15 | OpenCV: 16 | 17 | python3 object_detection_multi_platform.py --framework caffe --device CPU --backend OPENCV_DNN --mconfig /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.prototxt --mweight /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 18 | 19 | python3 object_detection_multi_platform.py --framework caffe --device CPU --backend OPENCV_DNN --mconfig /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.prototxt --mweight /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 20 | 21 | python3 object_detection_multi_platform.py --framework caffe --device OPENCL --backend OPENCV_DNN --mconfig /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.prototxt --mweight /home/intel/openvino_models/FP32/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --mlabels models/caffe_ssd_labels.txt -i offline --source /home/intel/Videos/traffic__winter_pk__pk_ave.MTS --model_image_height 224 --model_image_width 224 --confidence 0.60 22 | 23 | 24 | Extra Demos 25 | 26 | Interactive Face Demo 27 | 28 | ./interactive_face_detection_demo -i "cam" -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP16/face-detection-adas-0001.xml -m_ag /opt/intel/computer_vision_sdk/deployment_tools/intel_models/age-gender-recognition-retail-0013/FP32/age-gender-recognition-retail-0013.xml -m_em /opt/intel/computer_vision_sdk/deployment_tools/intel_models/emotions-recognition-retail-0003/FP16/emotions-recognition-retail-0003.xml -m_lm /opt/intel/computer_vision_sdk/deployment_tools/intel_models/facial-landmarks-35-adas-0001/FP16/facial-landmarks-35-adas-0001.xml -d GPU -d_ag CPU -d_em GPU -d_lm GPU -l lib/libcpu_extension.so 29 | 30 | WINDOWS: 31 | 32 | python.exe object_detection_multi_platform.py --framework openvino --device MYRIAD --mconfig C:\Users\odundar\Documents\Intel\OpenVINO\openvino_models/ir/mobilenet-ssd/FP16\mobilenet-ssd.xml --mweight C:\Users\odundar\Documents\Intel\OpenVINO\openvino_models/ir/mobilenet-ssd/FP16\mobilenet-ssd.bin --mlabels labels -i live --source 0 --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc False --openvino_cpulib_path C:\Users\odundar\Documents\Intel\OpenVINO\inference_engine_samples_2017\intel64\Release\cpu_extension.dll --openvino_libpath C:\Intel\computer_vision_sdk\deployment_tools\inference_engine\bin\intel64\Release 33 | 34 | python.exe object_detection_multi_platform.py --framework openvino --device MYRIAD --mconfig C:\Users\odundar\Documents\Intel\OpenVINO\openvino_models/ir/mobilenet-ssd/FP16\mobilenet-ssd.xml --mweight C:\Users\odundar\Documents\Intel\OpenVINO\openvino_models/ir/mobilenet-ssd/FP16\mobilenet-ssd.bin --mlabels labels -i live --source 0 --model_image_height 224 --model_image_width 224 --confidence 0.60 --async True --number_of_async_requests 10 --pc False --openvino_cpulib_path C:\Users\odundar\Documents\Intel\OpenVINO\inference_engine_samples_2017\intel64\Release\cpu_extension.dll --openvino_libpath C:\Intel\computer_vision_sdk\deployment_tools\inference_engine\bin\intel64\Release 35 | 36 | 37 | YOLOv3: 38 | 39 | python3 object_detection_multi_platform.py --framework openvino --device CPU --mconfig ~/smart-video-workshop/object-detection/tensorflow-yolo-v3/FP32/frozen_darknet_yolov3_model.xml --mweight ~/smart-video-workshop/object-detection/tensorflow-yolo-v3/FP32/frozen_darknet_yolov3_model.bin --mlabels ~/smart-video-workshop/object-detection/tensorflow-yolo-v3/coco.names -i offline --source /home/intel/Workshop2/videos/cars_1900.mp4 --model_image_height 224 --model_image_width 224 --confidence 0.60 --iou_threshold 0.4 --yolomodel True -------------------------------------------------------------------------------- /object_detection_multi_framework.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | # Required Libraries Imported 24 | import sys 25 | import os 26 | import time 27 | import cv2 as cv 28 | import numpy as np 29 | import psutil 30 | from math import exp as exp 31 | 32 | # Import OpenVINO 33 | from openvino.inference_engine import IENetwork, IEPlugin 34 | 35 | # OpenCV Backends and Targets (Inference Engine is opensource OpenVINO Distribution 36 | # opencv_backends = (cv.dnn.DNN_BACKEND_DEFAULT, cv.dnn.DNN_BACKEND_HALIDE, cv.dnn.DNN_BACKEND_INFERENCE_ENGINE, cv.dnn.DNN_BACKEND_OPENCV) 37 | # opencv_targets = (cv.dnn.DNN_TARGET_CPU, cv.dnn.DNN_TARGET_OPENCL, cv.dnn.DNN_TARGET_OPENCL_FP16, cv.dnn.DNN_TARGET_MYRIAD) 38 | 39 | # Performance Counters 40 | global inference_time_duration 41 | global resize_time_durations 42 | global resize_time_duration 43 | global post_process_durations 44 | post_process_durations = 0. 45 | global inferred_frame_count 46 | global frame_read_times 47 | 48 | 49 | class Config: 50 | """ 51 | Config Model to Store Arguments 52 | """ 53 | # Detection Threshold 54 | CONFIDENCE_THRESHOLD = 0.6 55 | 56 | # Source Type of Video 57 | VIDEOSOURCE = str() 58 | 59 | # Inference Framework if caffe or tensorflow opencv will be used 60 | INFERENCE_FRAMEWORK = str() 61 | 62 | # Source of Video, Full path of video file or index of Camera 63 | VIDEOPATH = str() # full path or 0 for live 64 | 65 | # Platform to 66 | INFERENCE_PLATFORM = "CPU" # cpu, gpu, movidius 67 | OPENCV_INFERENCE_BACKEND = "OPENCV_DNN" 68 | 69 | # Labels List which is created by reading labels file 70 | MODEL_LABELS = list() 71 | 72 | # Random Colors When Rectangle to be drawn for detected objects 73 | LABEL_COLORS = list() # colors to draw at post process 74 | 75 | # Label Names File for the detection 76 | MODEL_LABELS_FILE = str() 77 | 78 | # Path to protobuf file or .xml file for openvino 79 | MODEL_FILE = str() 80 | # Path to weights/biases file or .bin file for openvino 81 | MODEL_WEIGHT_FILE = str() 82 | 83 | # Manually Set Number of Frames to Infer, if not set app tries to infer all the frames 84 | INFERENCE_FRAMERATE = -1 85 | 86 | # Model Input Expected Image Size 87 | MODEL_IMAGE_HEIGHT = 300 88 | MODEL_IMAGE_WIDTH = 300 89 | 90 | # Actual Frame Size from Video Source 91 | IMAGE_HEIGHT = 300 92 | IMAGE_WIDTH = 300 93 | 94 | # Used to Determine Output Window Size 95 | OUTPUT_IMAGE_HEIGHT = 600 96 | OUTPUT_IMAGE_WIDTH = 800 97 | 98 | # Model Mean/Scale 99 | MODEL_MEANS = [127.5, 127.5, 127.5] 100 | MODEL_SCALE = (1. / 127.5) 101 | 102 | # used when reading from a video file, according to system speed it can be fast 103 | FPS_DELAY = 100 104 | 105 | # OpenVINO Has Feature to Run Async Inferrence 106 | ASYNC = False 107 | 108 | # OpenVINO Has Feature to Run Inference on Multiple Batches 109 | BATCH_SIZE = 1 110 | 111 | # Number of simultaneous Requests to Handle 112 | OPENVINO_NUM_REQUESTS = 1 113 | 114 | # Enable Performance Counter Report 115 | OPENVINO_PERFORMANCE_COUNTER = False 116 | 117 | # If Given Object Detection Model is YOLOv3 118 | YOLO_MODEL_DEFINED = False 119 | IOU_THRESHOLD = 0.4 120 | 121 | # Libray Paths 122 | OPENVINO_CPU_LIBPATH = '/home/intel/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so' 123 | OPENVINO_LIBPATH = '/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64' 124 | 125 | @staticmethod 126 | def print_summary(): 127 | """Prints values of Config Fields 128 | 129 | Args: 130 | 131 | Returns: 132 | None 133 | """ 134 | print("Inference Framework :{}".format(Config.INFERENCE_FRAMEWORK)) 135 | print("Inference Device :{}".format(Config.INFERENCE_PLATFORM)) 136 | print("OpenCV Inference Backend :{}".format(Config.OPENCV_INFERENCE_BACKEND)) 137 | print("Video Source :{}".format(Config.VIDEOSOURCE)) 138 | print("Video Path :{}".format(Config.VIDEOPATH)) 139 | print("Model Network Path :{}".format(Config.MODEL_FILE)) 140 | print("Model Weights Path :{}".format(Config.MODEL_WEIGHT_FILE)) 141 | print("Model Labels Path :{}".format(Config.MODEL_LABELS_FILE)) 142 | print("Detection Confidence Threshold :{}".format(Config.CONFIDENCE_THRESHOLD)) 143 | print("Inference Frame Rate :{}".format(Config.INFERENCE_FRAMERATE)) 144 | print("Inference Async :{}".format(Config.ASYNC)) 145 | print("FPS Delay :{}".format(Config.FPS_DELAY)) 146 | print("Performance Counter Report :{}".format(Config.OPENVINO_PERFORMANCE_COUNTER)) 147 | print("Is It YOLOv3 Model :{}".format(Config.YOLO_MODEL_DEFINED)) 148 | print("Intersection Over Union Thres :{}".format(Config.IOU_THRESHOLD)) 149 | print("Batch Size :{}".format(Config.BATCH_SIZE)) 150 | print("Number of Async Requests :{}".format(Config.OPENVINO_NUM_REQUESTS)) 151 | print("Model Image Width :{}".format(Config.MODEL_IMAGE_WIDTH)) 152 | print("Model Image Height :{}".format(Config.MODEL_IMAGE_HEIGHT)) 153 | print("Model Mean Substract :{}".format(Config.MODEL_MEANS)) 154 | print("Model Scale :{}".format(Config.MODEL_SCALE)) 155 | print("Image Width :{}".format(Config.IMAGE_WIDTH)) 156 | print("Image Height :{}".format(Config.IMAGE_HEIGHT)) 157 | print("Image Output Width :{}".format(Config.OUTPUT_IMAGE_WIDTH)) 158 | print("Image Output Height :{}".format(Config.OUTPUT_IMAGE_HEIGHT)) 159 | print("OpenVINO CPU Lib Path :{}".format(Config.OPENVINO_CPU_LIBPATH)) 160 | print("OpenVINO Lib Path :{}".format(Config.OPENVINO_LIBPATH)) 161 | 162 | return None 163 | 164 | 165 | class YoloV3Params: 166 | # ------------------------------------------- Extracting layer parameters ------------------------------------------ 167 | # Magic numbers are copied from yolo samples 168 | # Code from: https://docs.openvinotoolkit.org/latest/_inference_engine_ie_bridges_python_sample_object_detection_demo_yolov3_async_README.html 169 | def __init__(self, param, side): 170 | self.num = 3 if 'num' not in param else len(param['mask'].split(',')) if 'mask' in param else int(param['num']) 171 | self.coords = 4 if 'coords' not in param else int(param['coords']) 172 | self.classes = 80 if 'classes' not in param else int(param['classes']) 173 | self.anchors = [10.0, 13.0, 16.0, 30.0, 33.0, 23.0, 30.0, 61.0, 62.0, 45.0, 59.0, 119.0, 116.0, 90.0, 156.0, 174 | 198.0, 175 | 373.0, 326.0] if 'anchors' not in param else [float(a) for a in param['anchors'].split(',')] 176 | self.side = side 177 | if self.side == 13: 178 | self.anchor_offset = 2 * 6 179 | elif self.side == 26: 180 | self.anchor_offset = 2 * 3 181 | elif self.side == 52: 182 | self.anchor_offset = 2 * 0 183 | else: 184 | assert False, "Invalid output size. Only 13, 26 and 52 sizes are supported for output spatial dimensions" 185 | 186 | def log_params(self): 187 | params_to_print = {'classes': self.classes, 'num': self.num, 'coords': self.coords, 'anchors': self.anchors} 188 | [print(" {:8}: {}".format(param_name, param)) for param_name, param in params_to_print.items()] 189 | 190 | 191 | def entry_index(side, coord, classes, location, entry): 192 | side_power_2 = side ** 2 193 | n = location // side_power_2 194 | loc = location % side_power_2 195 | return int(side_power_2 * (n * (coord + classes + 1) + entry) + loc) 196 | 197 | 198 | def scale_bbox(x, y, h, w, class_id, confidence, h_scale, w_scale): 199 | xmin = int((x - w / 2) * w_scale) 200 | ymin = int((y - h / 2) * h_scale) 201 | xmax = int(xmin + w * w_scale) 202 | ymax = int(ymin + h * h_scale) 203 | return dict(xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, class_id=class_id, confidence=confidence) 204 | 205 | 206 | def intersection_over_union(box_1, box_2): 207 | width_of_overlap_area = min(box_1['xmax'], box_2['xmax']) - max(box_1['xmin'], box_2['xmin']) 208 | height_of_overlap_area = min(box_1['ymax'], box_2['ymax']) - max(box_1['ymin'], box_2['ymin']) 209 | if width_of_overlap_area < 0 or height_of_overlap_area < 0: 210 | area_of_overlap = 0 211 | else: 212 | area_of_overlap = width_of_overlap_area * height_of_overlap_area 213 | box_1_area = (box_1['ymax'] - box_1['ymin']) * (box_1['xmax'] - box_1['xmin']) 214 | box_2_area = (box_2['ymax'] - box_2['ymin']) * (box_2['xmax'] - box_2['xmin']) 215 | area_of_union = box_1_area + box_2_area - area_of_overlap 216 | if area_of_union == 0: 217 | return 0 218 | return area_of_overlap / area_of_union 219 | 220 | 221 | def parse_yolo_region(blob, resized_image_shape, originial_height, original_width, params, threshold): 222 | # ------------------------------------------ Validating output parameters ------------------------------------------ 223 | # _, _, out_blob_h, out_blob_w = blob.shape 224 | # assert out_blob_w == out_blob_h, "Invalid size of output blob. It sould be in NCHW layout and height should " \ 225 | # "be equal to width. Current height = {}, current width = {}" \ 226 | # "".format(out_blob_h, out_blob_w) 227 | 228 | # ------------------------------------------ Extracting layer parameters ------------------------------------------- 229 | orig_im_h = originial_height 230 | orig_im_w = original_width 231 | resized_image_h, resized_image_w = resized_image_shape 232 | objects = list() 233 | predictions = blob.flatten() 234 | side_square = params.side * params.side 235 | 236 | # ------------------------------------------- Parsing YOLO Region output ------------------------------------------- 237 | for i in range(side_square): 238 | row = i // params.side 239 | col = i % params.side 240 | for n in range(params.num): 241 | obj_index = entry_index(params.side, params.coords, params.classes, n * side_square + i, params.coords) 242 | scale = predictions[obj_index] 243 | if scale < threshold: 244 | continue 245 | box_index = entry_index(params.side, params.coords, params.classes, n * side_square + i, 0) 246 | x = (col + predictions[box_index + 0 * side_square]) / params.side * resized_image_w 247 | y = (row + predictions[box_index + 1 * side_square]) / params.side * resized_image_h 248 | # Value for exp is very big number in some cases so following construction is using here 249 | try: 250 | w_exp = exp(predictions[box_index + 2 * side_square]) 251 | h_exp = exp(predictions[box_index + 3 * side_square]) 252 | except OverflowError: 253 | continue 254 | w = w_exp * params.anchors[params.anchor_offset + 2 * n] 255 | h = h_exp * params.anchors[params.anchor_offset + 2 * n + 1] 256 | for j in range(params.classes): 257 | class_index = entry_index(params.side, params.coords, params.classes, n * side_square + i, 258 | params.coords + 1 + j) 259 | confidence = scale * predictions[class_index] 260 | if confidence < threshold: 261 | continue 262 | objects.append(scale_bbox(x=x, y=y, h=h, w=w, class_id=j, confidence=confidence, 263 | h_scale=orig_im_h / resized_image_h, w_scale=orig_im_w / resized_image_w)) 264 | 265 | return objects 266 | 267 | 268 | def help_menu(): 269 | """Help Menu for the application 270 | 271 | Args: 272 | 273 | Returns: 274 | None 275 | """ 276 | 277 | print('This script helps you to run real time object detection sample using OpenCV ' 278 | 'and NCSDK APIs using Tensorflow or Caffe DNN Models') 279 | 280 | print('./RealTimeObjectDetection [option] [value]') 281 | 282 | print('options:') 283 | print('--help print this menu') 284 | print('-d, --device CPU|GPU|MYRIAD|HETERO:CPU,GPU|HETERO:GPU,CPU|HETERO:MYRIAD,GPU : hw platform for openvino') 285 | print('-d, --device CPU|OPENCL|OPENCL_FP16|MYRIAD : hw platform for opencv, CPU default') 286 | print('-b, --backend OPENCV_DNN|OPENVINO : hw platform for opencv, OPENCV_DNN default') 287 | print('-i, --input live|offline : source of video, either webcam or video on disk') 288 | print('-s, --source : video file full e.g. /home/intel/videos/test.mp4') 289 | print('-f, --framework openvino|caffe|tensorflow : framework of models being used') 290 | print('--yolomodel True|False if object detection model is YOLOv3') 291 | print('--iou_threshold True|False if object detection model is YOLOv3') 292 | print('--mconfig file') 293 | print('--mweight file') 294 | print('--mlabels ') 295 | print('--model_image_height DNN model image height to be used for inferrence') 296 | print('--model_image_width DNN model image width to be used for inferrence') 297 | print('--model_image_scale DNN model image input scale for inference') 298 | print('--model_image_mean DNN model image mean substract eg. 127.5,127.5,127.5 ') 299 | print('-c, --confidence confidence threshold value, default 0.6') 300 | print('--pc True|False, report performance counters for Deep Learning Layers') 301 | print('--infer_fc <1, 2 ..> number of frames to infer, by default program tries to infer as much as it can') 302 | print('--async True|False determine if request is async or not') 303 | print('--number_of_async_requests 1 to N, enabled if async is being used') 304 | print('--batch_size 1,2 .. number of frames to infer at once by OpenVINO, default 1') 305 | print('--openvino_cpulib_path path to libcpu_extension.so') 306 | print('--openvino_libpath path to openvino libraries') 307 | 308 | return None 309 | 310 | 311 | def parse_model_labels_from_file(label_file_name=str()): 312 | """Method used to read each line as a label for a given model 313 | 314 | Args: 315 | label_file_name: File name of labels 316 | 317 | Returns: 318 | Returns list of string 319 | """ 320 | 321 | if not os.path.isfile(label_file_name): 322 | print('Label File Not Found ...') 323 | help_menu() 324 | sys.exit(2) 325 | 326 | label_list = list() 327 | print('Labels: ') 328 | with open(label_file_name) as f: 329 | lines = f.readlines() 330 | for line in lines: 331 | label = line.replace('\n', '') 332 | label_list.append(label) 333 | 334 | print(label_list) 335 | 336 | return label_list 337 | 338 | 339 | def get_label_colors(labels_len=0): 340 | """Method used to generate random RGB values for drawing on the frames for detection layers 341 | 342 | Args: 343 | labels_len: Length of labels list 344 | 345 | Returns: 346 | Return list RGB Values [[R,G,B] ...] 347 | """ 348 | return np.random.uniform(0, 255, (labels_len, 3)) 349 | 350 | 351 | def parse_cli_arguments(argv): 352 | """Method used to parse command line arguments and set Config fields 353 | 354 | Args: 355 | argv: Arguments and Options list 356 | 357 | Returns: 358 | None 359 | """ 360 | 361 | opts = list() 362 | 363 | for i in range(0, len(argv), 2): 364 | if argv[0] in ('-h', '--help'): 365 | help_menu() 366 | sys.exit(0) 367 | else: 368 | opts.append((argv[i], argv[i + 1])) 369 | 370 | # Iterate over arguments 371 | for opt, arg in opts: 372 | if opt in ('-i', '--input'): 373 | Config.VIDEOSOURCE = arg 374 | elif opt in ('-s', '--source'): 375 | Config.VIDEOPATH = arg 376 | elif opt in ('-f', '--framework'): 377 | Config.INFERENCE_FRAMEWORK = arg 378 | elif opt in ('-d', '--device'): 379 | Config.INFERENCE_PLATFORM = arg 380 | elif opt in ('-b', '--backend'): 381 | Config.OPENCV_INFERENCE_BACKEND = arg 382 | elif opt in ('-c', '--confidence'): 383 | Config.CONFIDENCE_THRESHOLD = float(arg) 384 | elif opt == '--mconfig': 385 | Config.MODEL_FILE = arg 386 | elif opt == '--mweight': 387 | Config.MODEL_WEIGHT_FILE = arg 388 | elif opt == '--mlabels': 389 | Config.MODEL_LABELS_FILE = arg 390 | elif opt == '--infer_fc': 391 | Config.INFERENCE_FRAMERATE = int(arg) 392 | elif opt == '--model_image_height': 393 | Config.MODEL_IMAGE_HEIGHT = int(arg) 394 | elif opt == '--model_image_width': 395 | Config.MODEL_IMAGE_WIDTH = int(arg) 396 | elif opt == '--model_image_mean': 397 | vals = arg.split(',') 398 | Config.MODEL_MEANS = [float(vals[0]), float(vals[1]), float(vals[2])] 399 | elif opt == '--model_image_scale': 400 | Config.MODEL_IMAGE_SCALE = float(arg) 401 | elif opt == '--batch_size': 402 | Config.BATCH_SIZE = int(arg) 403 | elif opt == '--number_of_async_requests': 404 | Config.OPENVINO_NUM_REQUESTS = int(arg) 405 | elif opt == '--async': 406 | Config.ASYNC = (arg == 'True') 407 | elif opt == '--openvino_cpulib_path': 408 | Config.OPENVINO_CPU_LIBPATH = arg 409 | elif opt == '--openvino_libpath': 410 | Config.OPENVINO_LIBPATH = arg 411 | elif opt == '--pc': 412 | Config.OPENVINO_PERFORMANCE_COUNTER = (arg == 'True') 413 | elif opt == '--yolomodel': 414 | Config.YOLO_MODEL_DEFINED = (arg == 'True') 415 | elif opt == '--iou_threshold': 416 | Config.YOLO_MODEL_DEFINED = float(arg) 417 | else: 418 | print('Unknown argument {} exiting ...'.format(arg)) 419 | sys.exit(2) 420 | 421 | return None 422 | 423 | 424 | def load_video(source, path): 425 | """Method used to load given video source with OpenCV VideoCapture 426 | 427 | Args: 428 | source: Type of video source live (webcam) or file 429 | path: Path to video source 430 | 431 | Returns: 432 | cv.VideoCapture object 433 | """ 434 | 435 | if source == 'live': 436 | print('Loading {} Video '.format(source)) 437 | cap = cv.VideoCapture(0) 438 | print('Video FPS :{}'.format(cap.get(cv.CAP_PROP_FPS))) 439 | if cap.get(cv.CAP_PROP_FPS) > 0.0: 440 | Config.FPS_DELAY = int(1000 / cap.get(cv.CAP_PROP_FPS)) 441 | else: 442 | Config.FPS_DELAY = int(1) 443 | return cap 444 | elif source == 'offline': 445 | print('Loading {} Video '.format(source)) 446 | if not os.path.isfile(path): 447 | print('Video File Not Found, Exiting ...') 448 | sys.exit(2) 449 | 450 | cap = cv.VideoCapture(path) 451 | print('Video FPS :{}'.format(cap.get(cv.CAP_PROP_FPS))) 452 | Config.FPS_DELAY = int(1000 / cap.get(cv.CAP_PROP_FPS)) 453 | return cap 454 | else: 455 | print("Unidentified Source Input :{}".format(source)) 456 | print('-i, --input live|offline : source of video, either webcam or video on disk, Exiting ...') 457 | sys.exit(2) 458 | 459 | 460 | def opencv_inference(blob, network): 461 | """Method used to run inference on given frame with given network 462 | 463 | Args: 464 | blob: Resized and objectified frame to run forward propagation on the given DNN object 465 | network: DNN Network loaded 466 | 467 | Returns: 468 | cv.VideoCapture object 469 | """ 470 | 471 | # Send blob data to Network 472 | network.setInput(blob) 473 | 474 | # Make network do a forward propagation to get recognition matrix 475 | out = network.forward() 476 | 477 | return out[0, 0, :, :] 478 | 479 | 480 | def post_process(frame, detections): 481 | """Method used to draw rectangels on the detected objects according to defined confidence 482 | 483 | Args: 484 | frame: Original frame to be shown 485 | detections: detection list [N, 7] shape 486 | 487 | Returns: 488 | Processed frame 489 | """ 490 | 491 | for detection in detections: 492 | # confidence score 493 | score = float(detection[2]) 494 | 495 | # draw rectangle and write the name of the object if above given confidence 496 | if score > Config.CONFIDENCE_THRESHOLD: 497 | # label index 498 | # print(score) 499 | 500 | # print(detection) 501 | 502 | label_index = int(detection[1]) 503 | 504 | if label_index >= len(Config.MODEL_LABELS): 505 | label_text = str(label_index) 506 | else: 507 | label_text = Config.MODEL_LABELS[label_index] + " " + str(round(score, 4)) 508 | 509 | # print('Possibly Detected: ', label_text, score) 510 | 511 | xmin = detection[3] 512 | ymin = detection[4] 513 | xmax = detection[5] 514 | ymax = detection[6] 515 | 516 | col_factor = Config.IMAGE_WIDTH 517 | row_factor = Config.IMAGE_HEIGHT 518 | 519 | xmin = (xmin * col_factor) 520 | xmax = (xmax * col_factor) 521 | ymin = (ymin * row_factor) 522 | ymax = (ymax * row_factor) 523 | 524 | cv.putText(frame, 525 | label_text, 526 | (int(xmin), int(ymin)), 527 | cv.FONT_HERSHEY_SIMPLEX, 528 | 0.5, 529 | Config.LABEL_COLORS[label_index], 530 | 2) 531 | 532 | cv.rectangle(frame, 533 | (int(xmin), int(ymin)), 534 | (int(xmax), int(ymax)), 535 | Config.LABEL_COLORS[label_index], 536 | thickness=3) 537 | 538 | return frame 539 | 540 | 541 | def post_process_yolo(frame, objects): 542 | """Method used to draw rectangels on the detected objects according to defined confidence when Yolo Model Used 543 | 544 | Args: 545 | frame: Original frame to be shown 546 | objects: 547 | 548 | Returns: 549 | Processed frame 550 | """ 551 | # Filtering overlapping boxes with respect to the --iou_threshold CLI parameter 552 | 553 | print(objects) 554 | 555 | for i in range(len(objects)): 556 | if objects[i]['confidence'] == 0: 557 | continue 558 | for j in range(i + 1, len(objects)): 559 | if intersection_over_union(objects[i], objects[j]) > Config.IOU_THRESHOLD: 560 | objects[j]['confidence'] = 0 561 | 562 | # Drawing objects with respect to the --prob_threshold CLI parameter 563 | objects = [obj for obj in objects if obj['confidence'] >= Config.CONFIDENCE_THRESHOLD] 564 | 565 | for obj in objects: 566 | # Validation bbox of detected object 567 | if obj['xmax'] > frame.shape[1] or obj['ymax'] > frame.shape[0] or obj['xmin'] < 0 or obj['ymin'] < 0: 568 | continue 569 | 570 | label_index = obj['class_id'] 571 | 572 | label_text = Config.MODEL_LABELS[label_index] if Config.MODEL_LABELS and len( 573 | Config.MODEL_LABELS) >= label_index else \ 574 | str(obj['class_id']) 575 | 576 | label_text += ' ' + str(round(obj['confidence'], 4)) 577 | 578 | print(label_text) 579 | 580 | cv.rectangle(frame, (obj['xmin'], obj['ymin']), (obj['xmax'], obj['ymax']), Config.LABEL_COLORS[label_index], 3) 581 | cv.putText(frame, label_text, (obj['xmin'], obj['ymin'] - 7), cv.FONT_HERSHEY_COMPLEX, 0.5, 582 | Config.LABEL_COLORS[label_index], 2) 583 | 584 | return frame 585 | 586 | 587 | def get_openvino_plugin(openvino_network, inference_platform, library_path, cpu_libpath): 588 | """ 589 | Method used to load IEPlugin according to given target platform 590 | :param openvino_network: IENetwork object 591 | :param inference_platform: Target Device Plugin name (CPU, GPU, HETERO:MYRIAD,GPU,CPU etc. 592 | :param library_path: Lib path to Shared Libraries /opt/intel/openvino/deployment_tools/inference_engine/lib/ubuntu.. 593 | :return: IEPlugin object 594 | """ 595 | openvino_plugin = None 596 | 597 | # If OpenVINO Selected, Check for Hardware (GPU, MYRIAD or CPU) is supported for this example 598 | # Load corresponding device library from the indicated paths, this application requires the environment 599 | # variables are already set correctly 600 | # source /opt/intel/openvino/bin/setupvars.sh 601 | if inference_platform == 'GPU': 602 | print('Trying to Load OpenVINO GPU Plugin') 603 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 604 | elif inference_platform == 'MYRIAD': 605 | print('Trying to Load OpenVINO Myriad Plugin') 606 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 607 | openvino_plugin.set_config({"VPU_FORCE_RESET": "NO"}) 608 | elif inference_platform == 'HETERO:CPU,GPU' or inference_platform == 'HETERO:GPU,CPU': 609 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 610 | openvino_plugin.add_cpu_extension(cpu_libpath) 611 | openvino_plugin.set_config({"TARGET_FALLBACK": inference_platform.split(':')[1]}) 612 | # Enable graph visualization 613 | # openvino_plugin.set_config({"HETERO_DUMP_GRAPH_DOT": "YES"}) 614 | openvino_plugin.set_initial_affinity(openvino_network) 615 | supported_layers = openvino_plugin.get_supported_layers(openvino_network) 616 | print('Supported Layers') 617 | # [print(layer) for layer in supported_layers] 618 | not_supported_layers = [l for l in openvino_network.layers.keys() if l not in supported_layers] 619 | print('UnSupported Layers') 620 | # [print(layer) for layer in not_supported_layers] 621 | elif inference_platform == 'HETERO:MYRIAD,GPU' or inference_platform == 'HETERO:GPU,MYRIAD': 622 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 623 | openvino_plugin.set_config({"TARGET_FALLBACK": inference_platform.split(':')[1]}) 624 | # Enable graph visualization 625 | # openvino_plugin.set_config({"HETERO_DUMP_GRAPH_DOT": "YES"}) 626 | openvino_plugin.set_initial_affinity(openvino_network) 627 | supported_layers = openvino_plugin.get_supported_layers(openvino_network) 628 | print('Supported Layers') 629 | # [print(layer) for layer in supported_layers] 630 | not_supported_layers = [l for l in openvino_network.layers.keys() if l not in supported_layers] 631 | print('UnSupported Layers') 632 | # [print(layer) for layer in not_supported_layers] 633 | elif inference_platform == 'HETERO:MYRIAD,CPU' or inference_platform == 'HETERO:CPU,MYRIAD': 634 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 635 | openvino_plugin.add_cpu_extension(cpu_libpath) 636 | openvino_plugin.set_config({"TARGET_FALLBACK": inference_platform.split(':')[1]}) 637 | # Enable graph visualization 638 | # openvino_plugin.set_config({"HETERO_DUMP_GRAPH_DOT": "YES"}) 639 | openvino_plugin.set_initial_affinity(openvino_network) 640 | supported_layers = openvino_plugin.get_supported_layers(openvino_network) 641 | print('Supported Layers') 642 | # [print(layer) for layer in supported_layers] 643 | not_supported_layers = [l for l in openvino_network.layers.keys() if l not in supported_layers] 644 | print('UnSupported Layers') 645 | # [print(layer) for layer in not_supported_layers] 646 | elif inference_platform == "CPU": 647 | # By default try to load CPU library 648 | print('Trying to Load OpenVINO CPU Plugin') 649 | openvino_plugin = IEPlugin(device=inference_platform, plugin_dirs=library_path) 650 | openvino_plugin.add_cpu_extension(cpu_libpath) 651 | else: 652 | print('Undefined Target Platform for OpenVINO: {}'.format(inference_platform)) 653 | help_menu() 654 | exit(-2) 655 | 656 | return openvino_plugin 657 | 658 | 659 | def main(argv): 660 | """Main method runs the application logic 661 | 662 | Args: 663 | argv: Command line arguments 664 | 665 | Returns: 666 | None 667 | """ 668 | # Get process id , required to show CPU load 669 | process = psutil.Process(os.getpid()) 670 | 671 | global inference_time_duration 672 | inference_time_duration = 0. 673 | global resize_time_durations 674 | resize_time_durations = dict() 675 | global resize_time_duration 676 | resize_time_duration = 0. 677 | global inferred_frame_count 678 | inferred_frame_count = 0 679 | global frame_read_times 680 | frame_read_times = 0. 681 | global frame_display_times 682 | frame_display_times = 0. 683 | global post_process_durations 684 | 685 | # Parse cli arguments 686 | parse_cli_arguments(argv) 687 | 688 | # Read Labels From Given Text File 689 | Config.MODEL_LABELS = parse_model_labels_from_file(Config.MODEL_LABELS_FILE) 690 | # Generate Random Colors for each Label 691 | Config.LABEL_COLORS = get_label_colors(len(Config.MODEL_LABELS)) 692 | 693 | # Print Config Summary 694 | Config.print_summary() 695 | 696 | # Open Video with OpenCV 697 | cap = load_video(Config.VIDEOSOURCE, Config.VIDEOPATH) 698 | 699 | print("Loaded Video :{}".format(Config.VIDEOSOURCE)) 700 | print("Video Path :{}".format(Config.VIDEOPATH)) 701 | 702 | # Actual Frame Width/Height 703 | Config.IMAGE_WIDTH = cap.get(cv.CAP_PROP_FRAME_WIDTH) 704 | Config.IMAGE_HEIGHT = cap.get(cv.CAP_PROP_FRAME_HEIGHT) 705 | 706 | print("Video Resolution :{} x {}".format(Config.IMAGE_WIDTH, Config.IMAGE_HEIGHT)) 707 | 708 | # Deep Learning Network Object 709 | openvino_net = None 710 | openvino_plugin = None 711 | 712 | # OpenVINO Input/Output Definitions 713 | input_blob = None 714 | out_blob = None 715 | 716 | # OpenVINO Input Properties (Number of Inputs, Channels, Height, Width) 717 | n = 0 718 | c = 0 719 | h = 0 720 | w = 0 721 | 722 | # Request ID Queue for Async Inference 723 | request_ids = list() 724 | 725 | network_load_time_start = time.time() 726 | # Select Framework according to Options 727 | if Config.INFERENCE_FRAMEWORK == 'openvino': 728 | print('OpenVINO Framework Selected ...') 729 | 730 | # Read Inference Engine Network with given .bin/.xml files 731 | print('Loading DL Model Files : {} - {}'.format(Config.MODEL_FILE, Config.MODEL_WEIGHT_FILE)) 732 | network = IENetwork(model=Config.MODEL_FILE, weights=Config.MODEL_WEIGHT_FILE) 733 | 734 | openvino_plugin = get_openvino_plugin(network, 735 | Config.INFERENCE_PLATFORM, 736 | Config.OPENVINO_LIBPATH, 737 | Config.OPENVINO_CPU_LIBPATH) 738 | 739 | input_blob = next(iter(network.inputs)) 740 | print('OpenVINO Model Input Blob :', type(input_blob)) 741 | 742 | n, c, h, w = network.inputs[input_blob].shape 743 | Config.MODEL_IMAGE_HEIGHT = h 744 | Config.MODEL_IMAGE_WIDTH = w 745 | print('Input Properties') 746 | print('Batch :{}'.format(n)) 747 | print('Channels :{}'.format(c)) 748 | print('Height :{}'.format(h)) 749 | print('Width :{}'.format(w)) 750 | 751 | out_blob = next(iter(network.outputs)) 752 | print('OpenVINO Model Output Blob :', type(out_blob)) 753 | 754 | network.batch_size = Config.BATCH_SIZE 755 | print('Batch Size :', network.batch_size) 756 | 757 | print("Loading Given Model with IEPlugin ...") 758 | openvino_net = openvino_plugin.load(network=network, num_requests=Config.OPENVINO_NUM_REQUESTS) 759 | 760 | if Config.ASYNC: 761 | request_ids = list(np.arange(0, Config.OPENVINO_NUM_REQUESTS)) 762 | print("Number of Requests to Handle :", Config.OPENVINO_NUM_REQUESTS) 763 | else: 764 | request_ids.append(0) 765 | 766 | if openvino_net is None: 767 | print('Can not Load Given Network, Exiting ....') 768 | exit(-2) 769 | 770 | elif Config.INFERENCE_FRAMEWORK == 'tensorflow': 771 | print('OpenCV DNN will use Tensoflow Models for Inference') 772 | network = cv.dnn.readNetFromTensorflow(Config.MODEL_WEIGHT_FILE, Config.MODEL_FILE) 773 | 774 | elif Config.INFERENCE_FRAMEWORK == 'caffe': 775 | print('OpenCV DNN will use Caffe Models for Inference') 776 | network = cv.dnn.readNetFromCaffe(Config.MODEL_FILE, Config.MODEL_WEIGHT_FILE) 777 | 778 | else: 779 | print("{} Framework Not Supported, Exiting ...".format(Config.INFERENCE_FRAMEWORK)) 780 | help_menu() 781 | sys.exit(2) 782 | 783 | if Config.INFERENCE_FRAMEWORK == 'tensorflow' or Config.INFERENCE_FRAMEWORK == 'caffe': 784 | print('Setting OpenCV Backend and Target Device ...') 785 | if Config.OPENCV_INFERENCE_BACKEND == 'OPENVINO': 786 | network.setPreferableBackend(cv.dnn.DNN_BACKEND_INFERENCE_ENGINE) 787 | elif Config.OPENCV_INFERENCE_BACKEND == 'OPENCV_DNN': 788 | network.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV) 789 | else: 790 | print('Undefined OpenCV Backend: {}'.format(Config.OPENCV_INFERENCE_BACKEND)) 791 | help_menu() 792 | sys.exit(2) 793 | 794 | if Config.INFERENCE_PLATFORM == 'OPENCL': 795 | network.setPreferableTarget(cv.dnn.DNN_TARGET_OPENCL) 796 | elif Config.INFERENCE_PLATFORM == 'OPENCL_FP16': 797 | network.setPreferableTarget(cv.dnn.DNN_TARGET_OPENCL_FP16) 798 | elif Config.INFERENCE_PLATFORM == 'MYRIAD': 799 | network.setPreferableTarget(cv.dnn.DNN_TARGET_MYRIAD) 800 | elif Config.INFERENCE_PLATFORM == 'CPU': 801 | network.setPreferableTarget(cv.dnn.DNN_TARGET_CPU) 802 | else: 803 | print('Undefined OpenCV Target Device: {}'.format(Config.INFERENCE_PLATFORM)) 804 | help_menu() 805 | sys.exit(2) 806 | 807 | network_load_time_end = time.time() 808 | # Start Counting frames to Calculate FPS 809 | 810 | detections = None 811 | 812 | cur_request_id = 0 813 | next_request_id = 1 814 | # Queue to be used for request ids 815 | if Config.INFERENCE_FRAMEWORK == 'openvino' and Config.ASYNC: 816 | cur_request_id = request_ids.pop(0) 817 | next_request_id = request_ids.pop(0) 818 | 819 | next_frame = None 820 | # Continuous loop to read frames 821 | has_frame, frame = cap.read() 822 | 823 | openvino_detection_starts = dict() 824 | frame_count = 0 825 | start_time = time.time() 826 | 827 | cpu_count = psutil.cpu_count() 828 | 829 | # Start Reading Frames 830 | while True: 831 | # read frame from capture 832 | frame_read_start = time.time() 833 | if Config.ASYNC: 834 | has_frame, next_frame = cap.read() 835 | else: 836 | has_frame, frame = cap.read() 837 | frame_read_end = time.time() 838 | frame_read_times += (frame_read_end - frame_read_start) 839 | 840 | if not has_frame: 841 | break 842 | 843 | yolo_objects = list() 844 | 845 | if Config.INFERENCE_FRAMEWORK == 'openvino': 846 | if Config.ASYNC: 847 | # Read and pre-process input images 848 | resize_start = time.time() 849 | resized_frame = cv.resize(next_frame, (Config.MODEL_IMAGE_HEIGHT, Config.MODEL_IMAGE_WIDTH)) 850 | resized_frame = resized_frame.transpose((2, 0, 1)) # Change layout to HWC 851 | resized_frame = resized_frame.reshape((n, c, h, w)) 852 | resize_end = time.time() 853 | resize_time_duration += (resize_end - resize_start) 854 | 855 | infer_start = time.time() 856 | openvino_net.start_async(request_id=next_request_id, inputs={input_blob: resized_frame}) 857 | openvino_detection_starts[next_request_id] = infer_start 858 | 859 | else: 860 | resize_start = time.time() 861 | resized_frame = cv.resize(frame, (Config.MODEL_IMAGE_HEIGHT, Config.MODEL_IMAGE_WIDTH)) 862 | resized_frame = resized_frame.transpose((2, 0, 1)) # Change layout to HWC 863 | resized_frame = resized_frame.reshape((n, c, h, w)) 864 | resize_end = time.time() 865 | resize_time_duration += (resize_end - resize_start) 866 | 867 | infer_start = time.time() 868 | openvino_net.infer(inputs={input_blob: resized_frame}) 869 | openvino_detection_starts[cur_request_id] = infer_start 870 | 871 | if openvino_net.requests[cur_request_id].wait(-1) == 0: 872 | if not Config.YOLO_MODEL_DEFINED: 873 | openvino_detections = openvino_net.requests[cur_request_id].outputs[out_blob] 874 | detections = openvino_detections[0][0] 875 | 876 | else: 877 | output = openvino_net.requests[cur_request_id].outputs 878 | 879 | for layer_name, out_blob in output.items(): 880 | layer_params = YoloV3Params(network.layers[layer_name].params, out_blob.shape[2]) 881 | # print("Layer {} parameters: ".format(layer_name)) 882 | layer_params.log_params() 883 | yolo_objects += parse_yolo_region(out_blob, 884 | resized_frame.shape[2:], 885 | Config.IMAGE_HEIGHT, 886 | Config.IMAGE_WIDTH, 887 | layer_params, 888 | Config.CONFIDENCE_THRESHOLD) 889 | detection_ends = time.time() 890 | inference_time_duration += (detection_ends - openvino_detection_starts[cur_request_id]) 891 | inferred_frame_count += 1 892 | 893 | elif Config.INFERENCE_FRAMEWORK == 'tensorflow' or Config.INFERENCE_FRAMEWORK == 'caffe': 894 | resize_start = time.time() 895 | resized_frame = cv.resize(frame, (Config.MODEL_IMAGE_HEIGHT, Config.MODEL_IMAGE_WIDTH)) 896 | 897 | # MobileNetSSD Expects 224x224 resized frames 898 | blob = cv.dnn.blobFromImage(resized_frame, 899 | Config.MODEL_SCALE, 900 | (Config.MODEL_IMAGE_HEIGHT, Config.MODEL_IMAGE_WIDTH), 901 | (Config.MODEL_MEANS[0], Config.MODEL_MEANS[1], Config.MODEL_MEANS[2])) 902 | 903 | resize_end = time.time() 904 | resize_time_duration += (resize_end - resize_start) 905 | 906 | infer_start = time.time() 907 | detections = opencv_inference(blob, network) 908 | infer_end = time.time() 909 | inference_time_duration += (infer_end - infer_start) 910 | inferred_frame_count += 1 911 | else: 912 | print('Framework Not Found, Exiting ...') 913 | sys.exit(2) 914 | 915 | # Post Process over Detections 916 | post_process_start = time.time() 917 | if detections is not None and not Config.YOLO_MODEL_DEFINED: 918 | post_process(frame, detections) 919 | 920 | if yolo_objects is not None and Config.YOLO_MODEL_DEFINED: 921 | post_process_yolo(frame, yolo_objects) 922 | 923 | # display text to let user know how to quit 924 | cv.rectangle(frame, (0, 0), (220, 60), (50, 50, 50, 100), -1) 925 | cv.putText(frame, 926 | "Q to Quit", 927 | (10, 12), 928 | cv.FONT_HERSHEY_SIMPLEX, 929 | 0.4, 930 | (255, 255, 255), 931 | 1) 932 | # CPU Load 933 | current_cpu_load = process.cpu_percent() 934 | cpu_load = current_cpu_load / cpu_count 935 | cv.putText(frame, 936 | 'CPU Load %: {} '.format(cpu_load), 937 | (10, 25), 938 | cv.FONT_HERSHEY_SIMPLEX, 939 | 0.4, 940 | (255, 255, 255), 941 | 1) 942 | current_end = time.time() 943 | current_fps = frame_count / (current_end - start_time) 944 | cv.putText(frame, 945 | 'FPS : {} '.format((round(current_fps, 3))), 946 | (10, 38), 947 | cv.FONT_HERSHEY_SIMPLEX, 948 | 0.4, 949 | (255, 255, 255), 950 | 1) 951 | 952 | cv.imshow('Real Time Object Detection', frame) 953 | 954 | if Config.ASYNC: 955 | request_ids.append(cur_request_id) 956 | cur_request_id = next_request_id 957 | next_request_id = request_ids.pop(0) 958 | frame = next_frame 959 | 960 | if cv.waitKey(Config.FPS_DELAY) & 0xFF == ord('q'): 961 | break 962 | 963 | post_process_end = time.time() 964 | global post_process_durations 965 | post_process_durations += post_process_end - post_process_start 966 | frame_count += 1 967 | 968 | # Summarize Performance Metrics 969 | end_time = time.time() 970 | elapsed_time = end_time - start_time 971 | network_load_time = network_load_time_end - network_load_time_start 972 | 973 | print('Total Execution Time :', 974 | elapsed_time, ' Seconds') 975 | print('Processed Frame Count :', 976 | inferred_frame_count, ' Frames') 977 | print('Network Load Time: ' + 978 | str(network_load_time) + ' Seconds') 979 | print('Reading 1 Frame in :' + 980 | str(round((frame_read_times / frame_count) * 1000, 3)) + ' Milliseconds') 981 | print('Frames Per Second :' + 982 | str(round(frame_count / elapsed_time, 3))) 983 | print('Pre-process for 1 Frame :' + 984 | str(round((resize_time_duration / inferred_frame_count) * 1000, 3)), 985 | ' milliseconds') 986 | 987 | global post_process_durations 988 | if not Config.ASYNC: 989 | print('Inference for 1 Frame :' + 990 | str(round((inference_time_duration / inferred_frame_count) * 1000, 3)), 991 | ' milliseconds') 992 | else: 993 | print('Inference for 1 Frame :', 994 | str(round(((elapsed_time - frame_read_times - 995 | resize_time_duration - post_process_durations) 996 | / frame_count) * 1000, 3)), 997 | ' milliseconds') 998 | 999 | print('Post-process for 1 Frame :' + 1000 | str(round((post_process_durations / inferred_frame_count) * 1000, 3)), 1001 | ' milliseconds (including display, key wait time ...)') 1002 | 1003 | print('Final Time Table in Milliseconds') 1004 | print('Elapsed Time - ' 1005 | 'Frame Read Time - Pre Process Time - ' 1006 | 'Infer Time - Post Process Time') 1007 | 1008 | print('{} - {} - {} - {} - {} \n'.format(elapsed_time * 1000., 1009 | frame_read_times * 1000, 1010 | resize_time_duration * 1000, 1011 | inference_time_duration * 1000, 1012 | post_process_durations * 1000)) 1013 | 1014 | # print('Total Elapsed Time: {} Milliseconds'.format(elapsed_time * 1000)) 1015 | 1016 | # time_sums = frame_display_times + resize_time_duration + \ 1017 | # inference_time_duration + post_process_durations 1018 | 1019 | # print('Sum of Measured Time: {} Milliseconds'.format(time_sums * 1000)) 1020 | 1021 | # When frames finished 1022 | if Config.INFERENCE_FRAMEWORK == 'openvino' and Config.OPENVINO_PERFORMANCE_COUNTER: 1023 | print("No more frame from from video source, exiting ....") 1024 | 1025 | perf_counts = openvino_net.requests[0].get_perf_counts() 1026 | print("Performance counters:") 1027 | print("{:<70} {:<15} {:<15} {:<15} {:<10}".format('name', 1028 | 'layer_type', 1029 | 'exet_type', 1030 | 'status', 1031 | 'real_time, us')) 1032 | for layer, stats in perf_counts.items(): 1033 | print("{:<70} {:<15} {:<15} {:<15} {:<10}".format(layer, stats['layer_type'], stats['exec_type'], 1034 | stats['status'], stats['real_time'])) 1035 | 1036 | # Release Resources 1037 | cv.destroyAllWindows() 1038 | cap.release() 1039 | 1040 | del openvino_net 1041 | del network 1042 | del openvino_plugin 1043 | 1044 | 1045 | # Application Entry Point 1046 | if __name__ == "__main__": 1047 | main(sys.argv[1:]) 1048 | -------------------------------------------------------------------------------- /openvino_basic_object_detection.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import time 24 | import cv2 as cv 25 | import argparse 26 | import psutil 27 | import os 28 | 29 | # Import OpenVINO Inference Engine 30 | from openvino.inference_engine import IECore, IENetwork 31 | 32 | 33 | mobilenet_ssd_labels = {0:'background', 34 | 1: 'aeroplane', 35 | 2: 'bicycle', 36 | 3: 'bird', 37 | 4: 'boat', 38 | 5: 'bottle', 39 | 6: 'bus', 40 | 7: 'car', 41 | 8: 'cat', 42 | 9: 'chair', 43 | 10: 'cow', 44 | 11: 'diningtable', 45 | 12: 'dog', 46 | 13: 'horse', 47 | 14: 'motorbike', 48 | 15: 'person', 49 | 16: 'pottedplant', 50 | 17: 'sheep', 51 | 18: 'sofa', 52 | 19: 'train', 53 | 20: 'tvmonitor' } 54 | 55 | 56 | def run_app(): 57 | """ 58 | Run Object Detection Application 59 | :return: 60 | """ 61 | frame_count = 0 62 | 63 | # Load Network 64 | OpenVinoNetwork = IENetwork(model=arguments.model_xml, weights=arguments.model_bin) 65 | 66 | # Get Input Layer Information 67 | InputLayer = next(iter(OpenVinoNetwork.inputs)) 68 | print("Input Layer: ", InputLayer) 69 | 70 | # Get Output Layer Information 71 | OutputLayer = next(iter(OpenVinoNetwork.outputs)) 72 | print("Output Layer: ", OutputLayer) 73 | 74 | # Get Input Shape of Model 75 | InputShape = OpenVinoNetwork.inputs[InputLayer].shape 76 | print("Input Shape: ", InputShape) 77 | 78 | # Get Output Shape of Model 79 | OutputShape = OpenVinoNetwork.outputs[OutputLayer].shape 80 | print("Output Shape: ", OutputShape) 81 | 82 | # Load IECore Object 83 | OpenVinoIE = IECore() 84 | print("Available Devices: ", OpenVinoIE.available_devices) 85 | 86 | # Load CPU Extensions if Necessary 87 | if 'CPU' in arguments.target_device: 88 | OpenVinoIE.add_extension('/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension.so', "CPU") 89 | 90 | # Create Executable Network 91 | if arguments.async: 92 | print("Async Mode Enabled") 93 | OpenVinoExecutable = OpenVinoIE.load_network(network=OpenVinoNetwork, device_name=arguments.target_device, num_requests=number_of_async_req) 94 | else: 95 | OpenVinoExecutable = OpenVinoIE.load_network(network=OpenVinoNetwork, device_name=arguments.target_device) 96 | 97 | # Generate a Named Window to Show Output 98 | cv.namedWindow('Window', cv.WINDOW_NORMAL) 99 | cv.resizeWindow('Window', 800, 600) 100 | 101 | start_time = time.time() 102 | 103 | if arguments.input_type == 'image': 104 | frame_count += 1 105 | # Read Image 106 | image = cv.imread(arguments.input) 107 | 108 | # Get Shape Values 109 | N, C, H, W = OpenVinoNetwork.inputs[InputLayer].shape 110 | 111 | # Pre-process Image 112 | resized = cv.resize(image, (W, H)) 113 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 114 | input_image = resized.reshape((N, C, H, W)) 115 | 116 | # Start Inference 117 | start = time.time() 118 | results = OpenVinoExecutable.infer(inputs={InputLayer: input_image}) 119 | end = time.time() 120 | inf_time = end - start 121 | print('Inference Time: {} Seconds Single Image'.format(inf_time)) 122 | 123 | fps = 1./(end-start) 124 | print('Estimated FPS: {} FPS Single Image'.format(fps)) 125 | 126 | fh = image.shape[0] 127 | fw = image.shape[1] 128 | 129 | # Write Information on Image 130 | text = 'FPS: {}, INF: {}'.format(round(fps, 2), round(inf_time, 2)) 131 | cv.putText(image, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 132 | 133 | # Print Bounding Boxes on Image 134 | detections = results[OutputLayer][0][0] 135 | for detection in detections: 136 | if detection[2] > arguments.detection_threshold: 137 | print('Original Frame Shape: ', fw, fh) 138 | xmin = int(detection[3] * fw) 139 | ymin = int(detection[4] * fh) 140 | xmax = int(detection[5] * fw) 141 | ymax = int(detection[6] * fh) 142 | cv.rectangle(image, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 143 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], round(detection[2], 2)) 144 | cv.putText(image, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 145 | 146 | cv.imshow('Window', image) 147 | cv.waitKey(0) 148 | else: 149 | print("Running Inference for {} - {}".format(arguments.input_type, arguments.input)) 150 | 151 | process_id = os.getpid() 152 | process = psutil.Process(process_id) 153 | 154 | total_inference_time = 0.0 155 | # Implementation for CAM or Video File 156 | # Read Image 157 | capture = cv.VideoCapture(arguments.input) 158 | has_frame, frame = capture.read() 159 | frame_count += 1 160 | 161 | if not has_frame: 162 | print("Can't Open Input Video Source {}".format(arguments.input)) 163 | exit(-1) 164 | 165 | # Get Shape Values 166 | N, C, H, W = OpenVinoNetwork.inputs[InputLayer].shape 167 | fh = frame.shape[0] 168 | fw = frame.shape[1] 169 | print('Original Frame Shape: ', fw, fh) 170 | 171 | request_order = list() 172 | process_order = list() 173 | frame_order = list() 174 | if arguments.async: 175 | print("Async Mode Set") 176 | for i in range(number_of_async_req): 177 | request_order.append(i) 178 | print('Request Id {} Created'.format(i)) 179 | 180 | print('Request Ids {}'.format(request_order)) 181 | 182 | while has_frame: 183 | if arguments.async: 184 | if len(request_order) > 0: 185 | resized = cv.resize(frame, (W, H)) 186 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 187 | input_data = resized.reshape((N, C, H, W)) 188 | req_id = request_order[0] 189 | request_order.pop(0) 190 | OpenVinoExecutable.start_async(req_id, inputs={InputLayer: input_data}) 191 | process_order.append(req_id) 192 | frame_order.append(frame) 193 | 194 | if len(process_order) > 0: 195 | first = process_order[0] 196 | if OpenVinoExecutable.requests[first].wait(0) == 0: 197 | results = OpenVinoExecutable.requests[first].outputs[OutputLayer] 198 | process_order.pop(0) 199 | request_order.append(first) 200 | show_frame = frame_order[0] 201 | frame_order.pop(0) 202 | 203 | detections = results[0][0] 204 | for detection in detections: 205 | if detection[2] > arguments.detection_threshold: 206 | xmin = int(detection[3] * fw) 207 | ymin = int(detection[4] * fh) 208 | xmax = int(detection[5] * fw) 209 | ymax = int(detection[6] * fh) 210 | cv.rectangle(show_frame, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 211 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], 212 | round(detection[2], 3)) 213 | cv.putText(show_frame, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 214 | 215 | fps = frame_count / (time.time() - start_time) 216 | # Write Information on Image 217 | text = 'FPS: {}, INF: {} ms'.format(round(fps, 3), "-") 218 | cv.putText(show_frame, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.8, (0, 125, 255), 1) 219 | 220 | text = "SYS CPU% {} SYS MEM% {} \n " \ 221 | "PROC CPU Affinity {} \n " \ 222 | "NUM Threads {} \n " \ 223 | "PROC CPU% {} \n " \ 224 | "PROC MEM% {}".format(psutil.cpu_percent(), 225 | psutil.virtual_memory()[2], 226 | process.cpu_affinity(), 227 | process.num_threads(), 228 | process.cpu_percent(), 229 | round(process.memory_percent(), 4)) 230 | 231 | cv.putText(show_frame, text, (0, 50), cv.FONT_HERSHEY_COMPLEX, 1, (250, 0, 255), 1) 232 | cv.imshow('Window', show_frame) 233 | if cv.waitKey(1) & 0xFF == ord('q'): 234 | break 235 | 236 | if len(process_order) > 0: 237 | has_frame, frame = capture.read() 238 | frame_count += 1 239 | else: 240 | frame_count += 1 241 | resized = cv.resize(frame, (W, H)) 242 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 243 | input_data = resized.reshape((N, C, H, W)) 244 | # Start Inference 245 | results = OpenVinoExecutable.infer(inputs={InputLayer: input_data}) 246 | 247 | fps = frame_count / (time.time() - start_time) 248 | inf_time = (time.time() - start_time) / frame_count 249 | # Write Information on Image 250 | text = 'FPS: {}, INF: {} ms'.format(round(fps, 3), round(inf_time, 3)) 251 | cv.putText(frame, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.8, (0, 125, 255), 1) 252 | 253 | # Print Bounding Boxes on Image 254 | detections = results[OutputLayer][0][0] 255 | for detection in detections: 256 | if detection[2] > arguments.detection_threshold: 257 | xmin = int(detection[3] * fw) 258 | ymin = int(detection[4] * fh) 259 | xmax = int(detection[5] * fw) 260 | ymax = int(detection[6] * fh) 261 | cv.rectangle(frame, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 262 | detection_percentage = round(detection[2], 4) 263 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], detection_percentage) 264 | cv.putText(frame, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 265 | 266 | text = "SYS CPU% {} SYS MEM% {} \n " \ 267 | "PROC CPU Affinity {} \n " \ 268 | "NUM Threads {} \n " \ 269 | "PROC CPU% {} \n " \ 270 | "PROC MEM% {}".format(psutil.cpu_percent(), 271 | psutil.virtual_memory()[2], 272 | process.cpu_affinity(), 273 | process.num_threads(), 274 | process.cpu_percent(), 275 | round(process.memory_percent(), 4)) 276 | 277 | cv.putText(frame, text, (0, 50), cv.FONT_HERSHEY_COMPLEX, 0.8, (250, 0, 250), 1) 278 | cv.imshow('Window', frame) 279 | if cv.waitKey(1) & 0xFF == ord('q'): 280 | break 281 | has_frame, frame = capture.read() 282 | 283 | end_time = time.time() 284 | print('Elapsed Time: {} Seconds'.format(end_time - start_time)) 285 | print('Number of Frames: {} '.format(frame_count)) 286 | print('Estimated FPS: {}'.format(frame_count / (end_time - start_time))) 287 | 288 | 289 | global arguments 290 | global number_of_async_req 291 | 292 | """ 293 | Entry Point of Application 294 | """ 295 | if __name__ == '__main__': 296 | # Parse Arguments 297 | parser = argparse.ArgumentParser(description='Basic OpenVINO Example with MobileNet-SSD') 298 | parser.add_argument('--model-xml', 299 | default='/home/intel/openvino_models/object_detection/common/mobilenet-ssd/FP32/mobilenet-ssd.xml', 300 | help='XML File') 301 | parser.add_argument('--model-bin', 302 | default='/home/intel/openvino_models/object_detection/common/mobilenet-ssd/FP32/mobilenet-ssd.bin', 303 | help='BIN File') 304 | parser.add_argument('--target-device', default='CPU', 305 | help='Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, HETERO:FPGA,CPU') 306 | parser.add_argument('--input-type', default='image', help='Type of Input: image, video, cam') 307 | parser.add_argument('--input', default='/home/intel/Pictures/cars.png', 308 | help='Path to Input: WebCam: 0, Video File or Image file') 309 | 310 | parser.add_argument('--detection-threshold', default=0.6, help='Object Detection Accuracy Threshold') 311 | 312 | parser.add_argument('--async',action="store_true", default=False, help='Run Async Mode') 313 | parser.add_argument('--request-number', default=1, help='Number of Requests') 314 | 315 | global arguments 316 | arguments = parser.parse_args() 317 | 318 | global number_of_async_req 319 | number_of_async_req = int(arguments.request_number) 320 | 321 | print('WARNING: No Argument Control Done, You Can GET Runtime Errors') 322 | run_app() 323 | -------------------------------------------------------------------------------- /openvino_face_detection.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import time 24 | import cv2 as cv 25 | import argparse 26 | import numpy as np 27 | 28 | # Import OpenVINO Inference Engine 29 | from openvino.inference_engine import IECore, IENetwork 30 | 31 | global arguments 32 | 33 | 34 | def crop_frame(frame, coordinate, normalized=True): 35 | """ 36 | Crop Frame as Given Coordinates 37 | :param frame: 38 | :param coordinate: 39 | :param normalized: 40 | :return: 41 | """ 42 | 43 | x1 = coordinate[0] 44 | y1 = coordinate[1] 45 | x2 = coordinate[2] 46 | y2 = coordinate[3] 47 | 48 | if normalized: 49 | h = frame.shape[0] 50 | w = frame.shape[1] 51 | 52 | x1 = int(x1 * w) 53 | x2 = int(x2 * w) 54 | 55 | y1 = int(y1 * h) 56 | y2 = int(y2 * h) 57 | 58 | return frame[y1:y2, x1:x2] 59 | 60 | 61 | def run_app(): 62 | # Load IECore Object 63 | OpenVinoIE = IECore() 64 | print("Available Devices: ", OpenVinoIE.available_devices) 65 | 66 | # Load CPU Extensions if Necessary 67 | if "CPU" in arguments.face_target_device or "CPU" in arguments.ag_target_device: 68 | OpenVinoIE.add_extension('/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension.so', "CPU") 69 | 70 | print('Loading Face Detection Model ....') 71 | # Load Network 72 | FaceDetectionNetwork = IENetwork(model=arguments.face_model_xml, weights=arguments.face_model_bin) 73 | # Get Input Layer Information 74 | FaceDetectionInputLayer = next(iter(FaceDetectionNetwork.inputs)) 75 | print("Face Detection Input Layer: ", FaceDetectionInputLayer) 76 | # Get Output Layer Information 77 | FaceDetectionOutputLayer = next(iter(FaceDetectionNetwork.outputs)) 78 | print("Face Detection Output Layer: ", FaceDetectionOutputLayer) 79 | # Get Input Shape of Model 80 | FaceDetectionInputShape = FaceDetectionNetwork.inputs[FaceDetectionInputLayer].shape 81 | print("Face Detection Input Shape: ", FaceDetectionInputShape) 82 | # Get Output Shape of Model 83 | FaceDetectionOutputShape = FaceDetectionNetwork.outputs[FaceDetectionOutputLayer].shape 84 | print("Face Detection Output Shape: ", FaceDetectionOutputShape) 85 | 86 | # Load Executable Network 87 | FaceDetectionExecutable = OpenVinoIE.load_network(network=FaceDetectionNetwork, device_name=arguments.face_target_device) 88 | 89 | print('Loading Age - Gender Detection Model .....') 90 | # Load Network 91 | AgeGenderDetectionNetwork = IENetwork(model=arguments.ag_model_xml, weights=arguments.ag_model_bin) 92 | # Get Input Layer Information 93 | AgeGenderDetectionInputLayer = next(iter(AgeGenderDetectionNetwork.inputs)) 94 | print("Age Gender Detection Input Layer: ", AgeGenderDetectionInputLayer) 95 | # Get Output Layer Information 96 | AgeGenderDetectionOutputLayers = list(AgeGenderDetectionNetwork.outputs.keys()) 97 | print("Age Gender Detection Output Layer: ", AgeGenderDetectionOutputLayers) 98 | # Get Input Shape of Model 99 | AgeGenderDetectionInputShape = AgeGenderDetectionNetwork.inputs[AgeGenderDetectionInputLayer].shape 100 | print("Age Gender Detection Input Shape: ", AgeGenderDetectionInputShape) 101 | # Get Output Shape of Model 102 | AgeOutputShape = AgeGenderDetectionNetwork.outputs[AgeGenderDetectionOutputLayers[0]].shape 103 | print("Age Gender Detection Output Layer {} Shape: ".format(AgeGenderDetectionOutputLayers[0]), AgeOutputShape) 104 | GenderOutputShape = AgeGenderDetectionNetwork.outputs[AgeGenderDetectionOutputLayers[0]].shape 105 | print("Age Gender Detection Output Layer {} Shape: ".format(AgeGenderDetectionOutputLayers[1]), GenderOutputShape) 106 | 107 | # Set Maximum Batch Size for Age-Gender Detection 108 | AgeGenderDetectionNetwork.batch_size = int(arguments.ag_max_batch_size) 109 | 110 | # Check if Dynamic Batching Enabled for Age Gender Detection 111 | config = {} 112 | 113 | # Get the Batch Size and Allocate Input for Dynamic Batch Process 114 | NAG, CAG, HAG, WAG = AgeGenderDetectionNetwork.inputs[AgeGenderDetectionInputLayer].shape 115 | 116 | if arguments.ag_dynamic_batch: 117 | config = {"DYN_BATCH_ENABLED": "YES"} 118 | print("Dynamic Batch Enabled") 119 | 120 | if NAG > 1: 121 | age_detection_input = np.zeros(shape=(NAG, CAG, HAG, WAG), dtype=float) 122 | 123 | # Load Executable Network 124 | AgeGenderDetectionExecutable = OpenVinoIE.load_network(network=AgeGenderDetectionNetwork, config=config, device_name=arguments.ag_target_device) 125 | 126 | # Get Shape Values for Face Detection Network 127 | N, C, H, W = FaceDetectionNetwork.inputs[FaceDetectionInputLayer].shape 128 | 129 | # Generate a Named Window 130 | cv.namedWindow('Window', cv.WINDOW_NORMAL) 131 | cv.resizeWindow('Window', 800, 600) 132 | 133 | # Enables Single Image Inference ... 134 | if arguments.input_type == 'image': 135 | # Read Image 136 | image = cv.imread(arguments.input) 137 | fh = image.shape[0] 138 | fw = image.shape[1] 139 | 140 | # Pre-process Image 141 | resized = cv.resize(image, (W, H)) 142 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 143 | input_image = resized.reshape((N, C, H, W)) 144 | 145 | # Start Inference 146 | fdetect_start = time.time() 147 | results = FaceDetectionExecutable.infer(inputs={FaceDetectionInputLayer: input_image}) 148 | fdetect_end = time.time() 149 | 150 | inf_time = fdetect_end - fdetect_start 151 | fps = 1./inf_time 152 | # Write Information on Image 153 | text = 'Face Detection - FPS: {}, INF: {} Sec'.format(round(fps, 2), round(inf_time, 4)) 154 | print(text) 155 | cv.putText(image, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 156 | 157 | # Get Results 158 | detections = results[FaceDetectionOutputLayer][0][0] 159 | 160 | # Get Faces 161 | i = 0 162 | face_coordinates = list() 163 | face_frames = list() 164 | for detection in detections: 165 | if detection[2] > arguments.face_detection_threshold: 166 | # Get Coordinates 167 | xmin = int(detection[3] * fw) 168 | ymin = int(detection[4] * fh) 169 | xmax = int(detection[5] * fw) 170 | ymax = int(detection[6] * fh) 171 | coordinates = [xmin, ymin, xmax, ymax] 172 | face_coordinates.append(coordinates) 173 | 174 | # Crop Face 175 | face_image = crop_frame(frame=image, coordinate=coordinates, normalized=False) 176 | r_frame = cv.resize(face_image, (WAG, HAG)) 177 | r_frame = cv.cvtColor(r_frame, cv.COLOR_BGR2RGB) 178 | r_frame = np.transpose(r_frame, (2, 0, 1)) 179 | r_frame = np.expand_dims(r_frame, axis=0) 180 | 181 | if NAG > 1: 182 | age_detection_input[i-1:i, ] = r_frame 183 | else: 184 | face_frames.append(r_frame) 185 | 186 | i += 1 187 | cv.rectangle(image, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 188 | print('Detected {} Faces'.format(i)) 189 | ag_total_infer_time = 0.0 190 | if NAG > 1 and arguments.ag_dynamic_batch: 191 | AgeGenderDetectionExecutable.requests[0].set_batch(i) 192 | ag_infer_start = time.time() 193 | AgeGenderDetectionExecutable.infer({AgeGenderDetectionInputLayer: age_detection_input}) 194 | ag_infer_end = time.time() 195 | ag_total_infer_time = (ag_infer_end - ag_infer_start) 196 | for f in range(age_detection_input.shape[0]): 197 | age = int( 198 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[0]][f][0][0][ 199 | 0] * 100) 200 | gender = 'male' 201 | if AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][f][0][0][0] > \ 202 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][f][1][0][0]: 203 | gender = 'female' 204 | 205 | text = "Face {} - A: {} - G: {}".format(f, age, gender) 206 | print(text) 207 | cv.putText(image, text, (face_coordinates[f][0], face_coordinates[f][1] - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 208 | 209 | fps = i / ag_total_infer_time 210 | inf_time = ag_total_infer_time / i 211 | text = 'AG Detection FPS: {}, INF: {} Sec Per Face'.format(round(fps, 2), round(inf_time, 4)) 212 | print(text) 213 | cv.putText(image, text, (0, 40), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 214 | else: 215 | f = 0 216 | for face in face_frames: 217 | ag_infer_start = time.time() 218 | AgeGenderDetectionExecutable.infer({AgeGenderDetectionInputLayer: face}) 219 | ag_infer_end = time.time() 220 | ag_total_infer_time += (ag_infer_end - ag_infer_start) 221 | age = int( 222 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[0]][0][0][0][ 223 | 0] * 100) 224 | 225 | gender = 'male' 226 | if AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][0][0][0] > \ 227 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][1][ 228 | 0][0]: 229 | gender = 'female' 230 | 231 | text = "Face {} - A: {} - G: {}".format(f, age, gender) 232 | print(text) 233 | cv.putText(image, text, (face_coordinates[f][0], face_coordinates[f][1] - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 234 | f += 1 235 | 236 | fps = 1.0 / ag_total_infer_time 237 | inf_time = ag_total_infer_time 238 | text = 'Age Detection - FPS: {}, INF: {} Sec'.format(round(fps, 2), round(inf_time, 4)) 239 | print(text) 240 | cv.putText(image, text, (0, 40), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 241 | 242 | cv.imshow('Window', image) 243 | cv.waitKey(0) 244 | 245 | # Enables Video File or Web Cam 246 | else: 247 | # Implementation for CAM or Video File 248 | # Read Image 249 | capture = cv.VideoCapture(arguments.input) 250 | has_frame, frame = capture.read() 251 | 252 | fh = frame.shape[0] 253 | fw = frame.shape[1] 254 | print('Original Frame Shape: ', fw, fh) 255 | 256 | # Variables to Hold Inference Time Information 257 | total_ag_inference_time = 0 258 | inferred_face_count = 0 259 | 260 | while has_frame: 261 | # Pre-process Image 262 | resized = cv.resize(frame, (W, H)) 263 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 264 | input_image = resized.reshape((N, C, H, W)) 265 | 266 | # Start Inference 267 | fdetect_start = time.time() 268 | results = FaceDetectionExecutable.infer(inputs={FaceDetectionInputLayer: input_image}) 269 | fdetect_end = time.time() 270 | inf_time = fdetect_end - fdetect_start 271 | fps = 1. / inf_time 272 | # Write Information on Image 273 | text = 'Face Detection - FPS: {}, INF: {}'.format(round(fps, 2), round(inf_time, 4)) 274 | cv.putText(frame, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 275 | 276 | # Print Bounding Boxes on Image 277 | detections = results[FaceDetectionOutputLayer][0][0] 278 | 279 | face_count = 0 280 | face_coordinates = list() 281 | face_frames = list() 282 | # Check All Detections 283 | for detection in detections: 284 | if detection[2] > arguments.face_detection_threshold: 285 | # Crop Frame 286 | xmin = int(detection[3] * fw) 287 | ymin = int(detection[4] * fh) 288 | xmax = int(detection[5] * fw) 289 | ymax = int(detection[6] * fh) 290 | 291 | coordinates = [xmin, ymin, xmax, ymax] 292 | face_coordinates.append(coordinates) 293 | face = crop_frame(frame=frame, coordinate=coordinates, normalized=False) 294 | r_frame = cv.resize(face, (WAG, HAG)) 295 | r_frame = cv.cvtColor(r_frame, cv.COLOR_BGR2RGB) 296 | r_frame = np.transpose(r_frame, (2, 0, 1)) 297 | r_frame = np.expand_dims(r_frame, axis=0) 298 | 299 | if NAG > 1: 300 | age_detection_input[face_count - 1:face_count, ] = r_frame 301 | else: 302 | face_frames.append(r_frame) 303 | 304 | face_count += 1 305 | cv.rectangle(frame, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 306 | 307 | agdetect_start = time.time() 308 | inferred_face_count += face_count 309 | 310 | if face_count > 0 and NAG > 1: 311 | if arguments.ag_dynamic_batch: 312 | AgeGenderDetectionExecutable.requests[0].set_batch(face_count) 313 | 314 | AgeGenderDetectionExecutable.infer({AgeGenderDetectionInputLayer: age_detection_input}) 315 | for f in range(face_count): 316 | age = int( 317 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[0]][0][0][0][ 318 | 0] * 100) 319 | 320 | gender = 'male' 321 | if AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][0][0][0] > \ 322 | AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][1][ 323 | 0][0]: 324 | gender = 'female' 325 | 326 | text = "A: {} - G: {}".format(age, gender) 327 | cv.putText(frame, text, (face_coordinates[f][0], face_coordinates[f][1] - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 328 | 329 | agdetect_end = time.time() 330 | # Write Information on Image 331 | inf_time = (agdetect_end - agdetect_start) / face_count 332 | fps = face_count / inf_time 333 | elif face_count > 0: 334 | f = 0 335 | for face in face_frames: 336 | AgeGenderDetectionExecutable.infer({AgeGenderDetectionInputLayer: face}) 337 | age = int(AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[0]][0][0][0][0] * 100) 338 | 339 | gender = 'male' 340 | if AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][0][0][0] > AgeGenderDetectionExecutable.requests[0].outputs[AgeGenderDetectionOutputLayers[1]][0][1][0][0]: 341 | gender = 'female' 342 | 343 | text = "A: {} - G: {}".format(age, gender) 344 | cv.putText(frame, text, (face_coordinates[f][0], face_coordinates[f][1] - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 345 | 346 | f += 1 347 | 348 | agdetect_end = time.time() 349 | # Write Information on Image 350 | inf_time = (agdetect_end - agdetect_start) / f 351 | fps = f / inf_time 352 | 353 | if face_count > 0: 354 | text = 'AG Detection - FPS: {}, INF Per Face: {}'.format(round(fps, 2), round(inf_time, 4)) 355 | cv.putText(frame, text, (0, 40), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 356 | total_ag_inference_time += inf_time 357 | 358 | cv.imshow('Window', frame) 359 | 360 | if cv.waitKey(1) & 0xFF == ord('q'): 361 | break 362 | 363 | has_frame, frame = capture.read() 364 | 365 | print('Total AG Inference Time : {}'.format(total_ag_inference_time)) 366 | print('Number of Face Inferred : {}'.format(inferred_face_count)) 367 | print('Average AG Inference Time : {}'.format(total_ag_inference_time/inferred_face_count)) 368 | 369 | 370 | """ 371 | Entry Point of Application 372 | """ 373 | if __name__ == '__main__': 374 | # Parse Arguments 375 | parser = argparse.ArgumentParser(description='Basic OpenVINO Example to Face, Age and Gender Detection') 376 | parser.add_argument('--face-model-xml', 377 | default='/home/intel/openvino_models/Transportation/object_detection/face/pruned_mobilenet_reduced_ssd_shared_weights/dldt/face-detection-adas-0001.xml', 378 | help='Face Detection Model XML File') 379 | parser.add_argument('--face-model-bin', 380 | default='/home/intel/openvino_models/Transportation/object_detection/face/pruned_mobilenet_reduced_ssd_shared_weights/dldt/face-detection-adas-0001.bin', 381 | help='Face Detection Model BIN File') 382 | parser.add_argument('--face-detection-threshold', default=0.5, help='Face Detection Accuracy Threshold') 383 | 384 | parser.add_argument('--face-target-device', default='CPU', 385 | help='Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, HETERO:FPGA,CPU') 386 | 387 | parser.add_argument('--ag-model-xml', 388 | default='/home/intel/openvino/Retail/object_attributes/age_gender/dldt/FP32/age-gender-recognition-retail-0013.xml', 389 | help='Age-Gender Detection XML File') 390 | parser.add_argument('--ag-model-bin', 391 | default='/home/intel/openvino/Retail/object_attributes/age_gender/dldt/FP32/age-gender-recognition-retail-0013.bin', 392 | help='Age-Gender Detection BIN File') 393 | 394 | parser.add_argument('--ag-max-batch-size', default=1, help='Age Gender Detection Max Batch Size') 395 | parser.add_argument('--ag-dynamic-batch', action="store_true", default=False, help='Age Gender Detection Enable Dynamic Batching') 396 | parser.add_argument('--ag-target-device', default='CPU', 397 | help='Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, HETERO:FPGA,CPU') 398 | 399 | parser.add_argument('--input-type', default='image', help='Type of Input: image, video, cam') 400 | parser.add_argument('--input', default='/home/intel/Pictures/faces.jpg', 401 | help='Path to Input: WebCam: 0, Video File or Image file') 402 | 403 | arguments = parser.parse_args() 404 | print('WARNING: No Argument Control Done, You Can GET Runtime Errors') 405 | run_app() 406 | -------------------------------------------------------------------------------- /openvino_object_detection_configs.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import time 24 | import cv2 as cv 25 | import argparse 26 | import psutil 27 | import os 28 | 29 | # Import OpenVINO Inference Engine 30 | from openvino.inference_engine import IECore, IENetwork 31 | 32 | 33 | mobilenet_ssd_labels = {0:'background', 34 | 1: 'aeroplane', 35 | 2: 'bicycle', 36 | 3: 'bird', 37 | 4: 'boat', 38 | 5: 'bottle', 39 | 6: 'bus', 40 | 7: 'car', 41 | 8: 'cat', 42 | 9: 'chair', 43 | 10: 'cow', 44 | 11: 'diningtable', 45 | 12: 'dog', 46 | 13: 'horse', 47 | 14: 'motorbike', 48 | 15: 'person', 49 | 16: 'pottedplant', 50 | 17: 'sheep', 51 | 18: 'sofa', 52 | 19: 'train', 53 | 20: 'tvmonitor' } 54 | 55 | 56 | def run_app(): 57 | """ 58 | Run Object Detection Application 59 | :return: 60 | """ 61 | 62 | frame_count = 0 63 | 64 | # Load Network 65 | OpenVinoNetwork = IENetwork(model=arguments.model_xml, weights=arguments.model_bin) 66 | 67 | # Get Input Layer Information 68 | InputLayer = next(iter(OpenVinoNetwork.inputs)) 69 | print("Input Layer: ", InputLayer) 70 | 71 | # Get Output Layer Information 72 | OutputLayer = next(iter(OpenVinoNetwork.outputs)) 73 | print("Output Layer: ", OutputLayer) 74 | 75 | # Get Input Shape of Model 76 | InputShape = OpenVinoNetwork.inputs[InputLayer].shape 77 | print("Input Shape: ", InputShape) 78 | 79 | # Get Output Shape of Model 80 | OutputShape = OpenVinoNetwork.outputs[OutputLayer].shape 81 | print("Output Shape: ", OutputShape) 82 | 83 | # Load IECore Object 84 | OpenVinoIE = IECore() 85 | print("Available Devices: ", OpenVinoIE.available_devices) 86 | 87 | # Load CPU Extensions if Necessary 88 | if 'CPU' in arguments.target_device: 89 | OpenVinoIE.add_extension('/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension.so', "CPU") 90 | 91 | # Configs 92 | if "CPU" in arguments.target_device: 93 | if int(arguments.cpu_num_threads) > 0: 94 | print("Setting CPU Threads to {}".format(arguments.cpu_num_threads)) 95 | OpenVinoIE.set_config({"CPU_THREADS_NUM" : arguments.cpu_num_threads}, "CPU") 96 | 97 | if "GPU" not in arguments.target_device: 98 | if arguments.cpu_bind_thread: 99 | print("Setting CPU Threads Binding") 100 | OpenVinoIE.set_config({"CPU_BIND_THREAD": "YES"}, "CPU") 101 | else: 102 | OpenVinoIE.set_config({"CPU_BIND_THREAD": "NO"}, "CPU") 103 | 104 | if arguments.async: 105 | print("Setting CPU Stream {}".format(arguments.cpu_throughput_streams)) 106 | try: 107 | stream = int(arguments.cpu_throughput_streams) 108 | if stream > 0: 109 | OpenVinoIE.set_config({"CPU_THROUGHPUT_STREAMS": arguments.cpu_throughput_streams}, "CPU") 110 | else: 111 | OpenVinoIE.set_config({"CPU_THROUGHPUT_STREAMS": "CPU_THROUGHPUT_AUTO"}, "CPU") 112 | except ValueError: 113 | if arguments.cpu_throughput_streams == "CPU_THROUGHPUT_NUMA": 114 | OpenVinoIE.set_config({"CPU_THROUGHPUT_STREAMS": "CPU_THROUGHPUT_NUMA"}, "CPU") 115 | else: 116 | OpenVinoIE.set_config({"CPU_THROUGHPUT_STREAMS": "CPU_THROUGHPUT_AUTO"}, "CPU") 117 | 118 | if "GPU" in arguments.target_device: 119 | if arguments.async: 120 | print("Setting GPU Stream {}".format(arguments.gpu_throughput_streams)) 121 | try: 122 | stream = int(arguments.cpu_throughput_streams) 123 | if stream > 0: 124 | OpenVinoIE.set_config({"GPU_THROUGHPUT_STREAMS": arguments.gpu_throughput_streams}, "GPU") 125 | else: 126 | OpenVinoIE.set_config({"GPU_THROUGHPUT_STREAMS": "GPU_THROUGHPUT_AUTO"}, "GPU") 127 | except ValueError: 128 | OpenVinoIE.set_config({"GPU_THROUGHPUT_STREAMS": "GPU_THROUGHPUT_AUTO"}, "GPU") 129 | 130 | if "MULTI" in arguments.target_device and arguments.gpu_throttle: 131 | print("Enabling GPU Throttle") 132 | OpenVinoIE.set_config({"CLDNN_PLUGIN_THROTTLE": "1"}, "GPU") 133 | 134 | config = {} 135 | 136 | if arguments.pc: 137 | print("Setting Performance Counters") 138 | config["PERF_COUNT"] = "YES" 139 | 140 | # Create Executable Network 141 | if arguments.async: 142 | print("Async Mode Enabled") 143 | OpenVinoExecutable = OpenVinoIE.load_network(network=OpenVinoNetwork, config=config, device_name=arguments.target_device, num_requests=number_of_async_req) 144 | else: 145 | OpenVinoExecutable = OpenVinoIE.load_network(network=OpenVinoNetwork, config=config, device_name=arguments.target_device) 146 | 147 | # Generate a Named Window 148 | cv.namedWindow('Window', cv.WINDOW_NORMAL) 149 | cv.resizeWindow('Window', 800, 600) 150 | 151 | start_time = time.time() 152 | 153 | if arguments.input_type == 'image': 154 | frame_count += 1 155 | # Read Image 156 | image = cv.imread(arguments.input) 157 | 158 | # Get Shape Values 159 | N, C, H, W = OpenVinoNetwork.inputs[InputLayer].shape 160 | 161 | # Pre-process Image 162 | resized = cv.resize(image, (W, H)) 163 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 164 | input_image = resized.reshape((N, C, H, W)) 165 | 166 | # Start Inference 167 | start = time.time() 168 | results = OpenVinoExecutable.infer(inputs={InputLayer: input_image}) 169 | end = time.time() 170 | inf_time = end - start 171 | print('Inference Time: {} Seconds'.format(inf_time)) 172 | 173 | fps = 1./(end-start) 174 | print('Estimated FPS: {} FPS'.format(fps)) 175 | 176 | fh = image.shape[0] 177 | fw = image.shape[1] 178 | 179 | # Write Information on Image 180 | text = 'FPS: {}, INF: {}'.format(round(fps, 2), round(inf_time, 2)) 181 | cv.putText(image, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.6, (0, 125, 255), 1) 182 | 183 | # Print Bounding Boxes on Image 184 | detections = results[OutputLayer][0][0] 185 | for detection in detections: 186 | if detection[2] > arguments.detection_threshold: 187 | print('Original Frame Shape: ', fw, fh) 188 | xmin = int(detection[3] * fw) 189 | ymin = int(detection[4] * fh) 190 | xmax = int(detection[5] * fw) 191 | ymax = int(detection[6] * fh) 192 | cv.rectangle(image, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 193 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], round(detection[2], 2)) 194 | cv.putText(image, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 195 | 196 | cv.imshow('Window', image) 197 | cv.waitKey(0) 198 | 199 | else: 200 | print("Running Inference for {} - {}".format(arguments.input_type, arguments.input)) 201 | 202 | process_id = os.getpid() 203 | process = psutil.Process(process_id) 204 | 205 | total_inference_time = 0.0 206 | # Implementation for CAM or Video File 207 | # Read Image 208 | capture = cv.VideoCapture(arguments.input) 209 | has_frame, frame = capture.read() 210 | frame_count += 1 211 | 212 | if not has_frame: 213 | print("Can't Open Input Video Source {}".format(arguments.input)) 214 | exit(-1) 215 | 216 | # Get Shape Values 217 | N, C, H, W = OpenVinoNetwork.inputs[InputLayer].shape 218 | fh = frame.shape[0] 219 | fw = frame.shape[1] 220 | print('Original Frame Shape: ', fw, fh) 221 | 222 | request_order = list() 223 | process_order = list() 224 | frame_order = list() 225 | if arguments.async: 226 | print("Async Mode Set") 227 | for i in range(number_of_async_req): 228 | request_order.append(i) 229 | print('Request Id {} Created'.format(i)) 230 | 231 | print('Request Ids {}'.format(request_order)) 232 | 233 | while has_frame: 234 | if arguments.async: 235 | if len(request_order) > 0: 236 | resized = cv.resize(frame, (W, H)) 237 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 238 | input_data = resized.reshape((N, C, H, W)) 239 | req_id = request_order[0] 240 | request_order.pop(0) 241 | OpenVinoExecutable.start_async(req_id, inputs={InputLayer: input_data}) 242 | process_order.append(req_id) 243 | frame_order.append(frame) 244 | 245 | if len(process_order) > 0: 246 | first = process_order[0] 247 | if OpenVinoExecutable.requests[first].wait(0) == 0: 248 | results = OpenVinoExecutable.requests[first].outputs[OutputLayer] 249 | process_order.pop(0) 250 | request_order.append(first) 251 | show_frame = frame_order[0] 252 | frame_order.pop(0) 253 | 254 | detections = results[0][0] 255 | for detection in detections: 256 | if detection[2] > arguments.detection_threshold: 257 | xmin = int(detection[3] * fw) 258 | ymin = int(detection[4] * fh) 259 | xmax = int(detection[5] * fw) 260 | ymax = int(detection[6] * fh) 261 | cv.rectangle(show_frame, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 262 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], 263 | round(detection[2], 3)) 264 | cv.putText(show_frame, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 265 | 266 | fps = frame_count / (time.time() - start_time) 267 | # Write Information on Image 268 | text = 'FPS: {}, INF: {} ms'.format(round(fps, 3), "-") 269 | cv.putText(show_frame, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.8, (0, 125, 255), 1) 270 | 271 | text = "SYS CPU% {} SYS MEM% {} \n " \ 272 | "PROC CPU Affinity {} \n " \ 273 | "NUM Threads {} \n " \ 274 | "PROC CPU% {} \n " \ 275 | "PROC MEM% {}".format(psutil.cpu_percent(), 276 | psutil.virtual_memory()[2], 277 | process.cpu_affinity(), 278 | process.num_threads(), 279 | process.cpu_percent(), 280 | round(process.memory_percent(), 4)) 281 | 282 | cv.putText(show_frame, text, (0, 50), cv.FONT_HERSHEY_COMPLEX, 0.8, (250, 0, 255), 1) 283 | 284 | if arguments.pc: 285 | perf_counts = OpenVinoExecutable.requests[0].get_perf_counts() 286 | print("Pefrormance counts for infer request") 287 | for layer, stats in perf_counts.items(): 288 | max_layer_name = 30 289 | print("{:<30}{:<15}{:<30}{:<20}{:<20}{:<20}".format( 290 | layer[:max_layer_name - 4] + '...' if (len(layer) >= max_layer_name) else layer, 291 | stats['status'], 292 | 'layerType: ' + str(stats['layer_type']), 293 | 'realTime: ' + str(stats['real_time']), 294 | 'cpu: ' + str(stats['cpu_time']), 295 | 'execType: ' + str(stats['exec_type']))) 296 | 297 | 298 | cv.imshow('Window', show_frame) 299 | if cv.waitKey(1) & 0xFF == ord('q'): 300 | break 301 | 302 | if len(process_order) > 0: 303 | has_frame, frame = capture.read() 304 | frame_count += 1 305 | else: 306 | frame_count += 1 307 | resized = cv.resize(frame, (W, H)) 308 | resized = resized.transpose((2, 0, 1)) # Change data layout from HWC to CHW 309 | input_data = resized.reshape((N, C, H, W)) 310 | # Start Inference 311 | results = OpenVinoExecutable.infer(inputs={InputLayer: input_data}) 312 | 313 | fps = frame_count / (time.time() - start_time) 314 | inf_time = (time.time() - start_time) / frame_count 315 | # Write Information on Image 316 | text = 'FPS: {}, INF: {} ms'.format(round(fps, 3), round(inf_time, 3)) 317 | cv.putText(frame, text, (0, 20), cv.FONT_HERSHEY_COMPLEX, 0.8, (0, 125, 255), 1) 318 | 319 | # Print Bounding Boxes on Image 320 | detections = results[OutputLayer][0][0] 321 | for detection in detections: 322 | if detection[2] > arguments.detection_threshold: 323 | xmin = int(detection[3] * fw) 324 | ymin = int(detection[4] * fh) 325 | xmax = int(detection[5] * fw) 326 | ymax = int(detection[6] * fh) 327 | cv.rectangle(frame, (xmin, ymin), (xmax, ymax), (0, 125, 255), 3) 328 | detection_percentage = round(detection[2], 4) 329 | text = '{}, %: {}'.format(mobilenet_ssd_labels[int(detection[1])], detection_percentage) 330 | cv.putText(frame, text, (xmin, ymin - 7), cv.FONT_HERSHEY_PLAIN, 0.8, (0, 125, 255), 1) 331 | 332 | text = "SYS CPU% {} SYS MEM% {} \n " \ 333 | "PROC CPU Affinity {} \n " \ 334 | "NUM Threads {} \n " \ 335 | "PROC CPU% {} \n " \ 336 | "PROC MEM% {}".format(psutil.cpu_percent(), 337 | psutil.virtual_memory()[2], 338 | process.cpu_affinity(), 339 | process.num_threads(), 340 | process.cpu_percent(), 341 | round(process.memory_percent(), 4)) 342 | 343 | cv.putText(frame, text, (0, 50), cv.FONT_HERSHEY_COMPLEX, 0.8, (250, 0, 250), 1) 344 | cv.imshow('Window', frame) 345 | if cv.waitKey(1) & 0xFF == ord('q'): 346 | break 347 | has_frame, frame = capture.read() 348 | 349 | if arguments.pc: 350 | perf_counts = OpenVinoExecutable.requests[0].get_perf_counts() 351 | print("Pefrormance counts for infer request") 352 | for layer, stats in perf_counts.items(): 353 | max_layer_name = 30 354 | print("{:<30}{:<15}{:<30}{:<20}{:<20}{:<20}".format( 355 | layer[:max_layer_name - 4] + '...' if (len(layer) >= max_layer_name) else layer, 356 | stats['status'], 357 | 'layerType: ' + str(stats['layer_type']), 358 | 'realTime: ' + str(stats['real_time']), 359 | 'cpu: ' + str(stats['cpu_time']), 360 | 'execType: ' + str(stats['exec_type']))) 361 | 362 | end_time = time.time() 363 | print('Elapsed Time: {} Seconds'.format(end_time - start_time)) 364 | print('Number of Frames: {} '.format(frame_count)) 365 | print('Estimated FPS: {}'.format(frame_count / (end_time - start_time))) 366 | 367 | 368 | global arguments 369 | global number_of_async_req 370 | 371 | """ 372 | Entry Point of Application 373 | """ 374 | if __name__ == '__main__': 375 | # Parse Arguments 376 | parser = argparse.ArgumentParser(description='Basic OpenVINO Example to Test Object Detection Model') 377 | parser.add_argument('--model-xml', 378 | default='/home/intel/openvino_models/object_detection/common/mobilenet-ssd/FP32/mobilenet-ssd.xml', 379 | help='XML File') 380 | parser.add_argument('--model-bin', 381 | default='/home/intel/openvino_models/object_detection/common/mobilenet-ssd/FP32/mobilenet-ssd.bin', 382 | help='BIN File') 383 | parser.add_argument('--target-device', default='CPU', 384 | help='Target Plugin: CPU, GPU, FPGA, MYRIAD, MULTI:CPU,GPU, HETERO:FPGA,CPU') 385 | parser.add_argument('--input-type', default='image', help='Type of Input: image, video, cam') 386 | parser.add_argument('--input', default='/home/intel/Pictures/faces.jpg', 387 | help='Path to Input: WebCam: 0, Video File or Image file') 388 | parser.add_argument('--detection-threshold', default=0.6, help='Object Detection Accuracy Threshold') 389 | 390 | parser.add_argument('--async', action="store_true", default=False, help='Run Async Mode') 391 | parser.add_argument('--request-number', default=1, help='Number of Requests') 392 | 393 | parser.add_argument('--pc', action="store_true", default=False, help='Enable Performance Counters') 394 | 395 | parser.add_argument('--cpu-num-threads', default=0, help='Limit CPU Threads') 396 | parser.add_argument('--cpu-bind-thread', action="store_true", default=False, help='Bind Threads to CPU') 397 | parser.add_argument('--cpu-throughput-streams', default="CPU_THROUGHPUT_AUTO", 398 | help="Int Values or CPU_THROUGHPUT_NUMA if not set CPU_THROUGHPUT_AUTO") 399 | parser.add_argument('--gpu-throughput-streams', default="GPU_THROUGHPUT_AUTO", 400 | help="Int Values if not set GPU_THROUGHPUT_AUTO") 401 | parser.add_argument('--gpu-throttle', action="store_true", default=False, 402 | help="multi-device execution with the CPU+GPU performs best with GPU trottling hint") 403 | 404 | global arguments 405 | arguments = parser.parse_args() 406 | 407 | global number_of_async_req 408 | number_of_async_req = int(arguments.request_number) 409 | 410 | print('WARNING: No Argument Control Done, You Can GET Runtime Errors') 411 | run_app() 412 | --------------------------------------------------------------------------------