├── .gitignore ├── LICENSE ├── README.md ├── app ├── README.md └── face_detection_openvino.py ├── config └── config.json ├── detection ├── __init__.py ├── age_gender_detection_ov.py ├── detection_base_ov.py └── face_detection_ov.py ├── files ├── README.md ├── intel-openvino.conf └── intel-openvino.sh ├── inference_services ├── README.md ├── facedetection │ ├── Dockerfile │ ├── README.md │ ├── detection │ │ ├── __init__.py │ │ ├── age_gender_detection_ov.py │ │ ├── detection_base_ov.py │ │ └── face_detection_ov.py │ ├── face_detection_service.py │ ├── inference_config.json │ ├── models │ │ ├── age-gender-recognition-retail-0013.bin │ │ ├── age-gender-recognition-retail-0013.xml │ │ ├── det1-0001.bin │ │ ├── det1-0001.mapping │ │ ├── det1-0001.xml │ │ ├── det2-0001.bin │ │ ├── det2-0001.mapping │ │ ├── det2-0001.xml │ │ ├── det3-0001.bin │ │ ├── det3-0001.mapping │ │ └── det3-0001.xml │ ├── requirements.txt │ └── utils │ │ ├── __init__.py │ │ └── image_utils.py └── flask_hello_world │ ├── Dockerfile │ ├── README.md │ ├── flask_app.py │ └── requirements.txt ├── models └── README.md └── utils ├── __init__.py └── image_utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | venv/ 2 | .idea/ 3 | __pycache__/ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python AI Project with Intel(R) OpenVINO(TM) Inference Engine Python API 2 | 3 | Here is a template project to reuse for production ready applications to use Deep Learning models focusing on Face, Age, Gender detection models. 4 | 5 | At this stage, only OpenVINO has been integrated. 6 | 7 | # OpenVINO(TM) Toolkit Installation and Configuration for Ubuntu 18.04 8 | 9 | ``OpenVINO(TM) Version: 2019.2.242`` 10 | 11 | You can install OpenVINO(TM) by following the instructions published online documentation. 12 | 13 | - https://docs.openvinotoolkit.org/ 14 | 15 | - https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_linux.html 16 | 17 | Before running this Python application: 18 | 19 | 1. Set Environment Variables on the current workspace: 20 | 21 | ```bash 22 | source /opt/intel/openvino/bin/setupvars.sh 23 | ``` 24 | 25 | **OR** 26 | 27 | 2. Set Environment Variables System Wide 28 | 29 | Copy `files/intel-openvino.sh` & `files/intel-openvino.conf` file as shown below: 30 | 31 | ```bash 32 | sudo cp files/intel-openvino.sh /etc/profile.d/ 33 | sudo cp files/intel-openvino.conf /etc/ld.so.conf.d/ 34 | sudo reboot 35 | ``` 36 | 37 | ## Clone This Repository 38 | 39 | ```bash 40 | git clone https://github.com/odundar/face_detection.git 41 | ``` 42 | 43 | ## Quick Run for Face Detection Application 44 | If all setup completed successfully, you can use the default configurations to give a start for face detection application. 45 | 46 | ```bash 47 | python3 face_detection_openvino.py config/config.json 48 | ``` 49 | 50 | ## app/ 51 | 52 | `app` folder includes apps ready to run for face, age, gender detection applications. 53 | 54 | ## services/ 55 | 56 | service modules stored here which are to be deployed in docker microservices 57 | 58 | ## config/ 59 | 60 | `config` folder includes app and service default configurations to be used as template. 61 | 62 | ## detection/ 63 | 64 | `detection` folder contains the modules and classes to reuse for inference application development. 65 | 66 | ``` 67 | detection\ 68 | detection_base_ov.py\ 69 | InferenceConfig 70 | InferenceBase 71 | 72 | age_gender_detection_ov.py\ 73 | AgeGenderDetectionTypes 74 | AgeGenderDetection 75 | AgeGenderConfig 76 | MTCNNAgeGenderDetection 77 | MTCNNAgeGenderConfig 78 | 79 | face_detection_ov.py\ 80 | FaceDetectionModelTypes 81 | FaceDetectionConfig 82 | OpenMZooFaceDetection 83 | MTCNNFaceDetectionConfig 84 | MtCNNFaceDetection 85 | ``` 86 | 87 | ## docker/ 88 | 89 | Includes instructions to deploy face, age-gender detection services as docker services. 90 | 91 | ## files/ 92 | 93 | This folder includes system configuration files for OpenVINO(TM) Toolkit 94 | 95 | ## models/ 96 | 97 | Folder includes instructions how to fetch models, convert and use them. 98 | 99 | -------------------------------------------------------------------------------- /app/README.md: -------------------------------------------------------------------------------- 1 | # Run Application 2 | 3 | This is a test application to run face-age-gender detections configured with the json file stored in config folder. 4 | 5 | Before running this application make sure OpenVINO environment has been setup correctly and `detection`, `utils` modules are in same directory as `app` folder. 6 | 7 | ```bash 8 | python3 app/face_detection_openvino.py config/config.json 9 | ``` 10 | 11 | If you run inside the `app` folder make sure `detection` and `utils` modules are included in `PYTHONPATH` 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /app/face_detection_openvino.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import sys 24 | import cv2 as cv 25 | import json 26 | import logging 27 | 28 | from detection.face_detection_ov import FaceDetectionConfig, OpenMZooFaceDetection, FaceDetectionModelTypes, MtCNNFaceDetection, MTCNNFaceDetectionConfig 29 | from detection.age_gender_detection_ov import AgeGenderConfig, MTCNNAgeGenderDetection, AgeGenderDetectionTypes, MTCNNAgeGenderConfig, AgeGenderDetection 30 | from utils.image_utils import ImageUtil 31 | 32 | 33 | def prepare_configs(): 34 | """ 35 | Set Configurations for Face, Age Gender Models 36 | :return: face config, age_gender config 37 | """ 38 | logging.log(logging.INFO, "Setting Configurations") 39 | if face_detection_model == FaceDetectionModelTypes.MTCNN: 40 | face_infer_cfg = MTCNNFaceDetectionConfig() 41 | else: 42 | face_infer_cfg = FaceDetectionConfig() 43 | 44 | face_infer_cfg.parse_json(config_file) 45 | 46 | age_gender_cfg = None 47 | 48 | if run_age_gender: 49 | if age_gender_model == AgeGenderDetectionTypes.MTCNN: 50 | age_gender_cfg = MTCNNAgeGenderConfig() 51 | else: 52 | age_gender_cfg = AgeGenderConfig() 53 | age_gender_cfg.parse_json(config_file) 54 | 55 | return face_infer_cfg, age_gender_cfg 56 | 57 | 58 | def run_app(): 59 | """ 60 | Runs Face Detection Application 61 | """ 62 | face_cfg, age_cfg = prepare_configs() 63 | 64 | '''Open Web Cam (change 0 to any video file if required)''' 65 | 66 | if input_type == "video" or input_type == "webcam": 67 | if input_type == "video": 68 | capture = cv.VideoCapture(input_path) 69 | has_frame, frame = capture.read() 70 | elif input_type == "webcam": 71 | capture = cv.VideoCapture(web_cam_index) 72 | has_frame, frame = capture.read() 73 | 74 | face_cfg.InputHeight = frame.shape[0] 75 | face_cfg.InputWidth = frame.shape[1] 76 | 77 | if face_detection_model == FaceDetectionModelTypes.MTCNN: 78 | face_infer = MtCNNFaceDetection(face_cfg) 79 | else: 80 | face_infer = OpenMZooFaceDetection(face_cfg) 81 | 82 | if run_age_gender: 83 | if age_gender_model == AgeGenderDetectionTypes.MTCNN: 84 | age_gender_infer = MTCNNAgeGenderDetection(age_cfg) 85 | else: 86 | age_gender_infer = AgeGenderDetection(age_cfg) 87 | 88 | video_inference(face_inference=face_infer, age_inference=age_gender_infer, source=capture) 89 | 90 | elif input_type == "image": 91 | frame = cv.imread(input_path) 92 | 93 | face_cfg.InputHeight = frame.shape[0] 94 | face_cfg.InputWidth = frame.shape[1] 95 | 96 | if face_detection_model == FaceDetectionModelTypes.MTCNN: 97 | face_infer = MtCNNFaceDetection(face_cfg) 98 | else: 99 | face_infer = OpenMZooFaceDetection(face_cfg) 100 | 101 | if run_age_gender: 102 | if age_gender_model == AgeGenderDetectionTypes.MTCNN: 103 | age_gender_infer = MTCNNAgeGenderDetection(age_cfg) 104 | else: 105 | age_gender_infer = AgeGenderDetection(age_cfg) 106 | 107 | image_inference(face_inference=face_infer, age_inference=age_gender_infer, source=frame) 108 | else: 109 | logging.log(logging.ERROR, "Invalid Input Type: {}".format(input_type)) 110 | exit(-1) 111 | 112 | return None 113 | 114 | 115 | def image_inference(face_inference=None, age_inference=None, source=None): 116 | """ 117 | 118 | :param face_inference: 119 | :param age_inference: 120 | :param source: 121 | :return: 122 | """ 123 | cv.namedWindow(cv_window_name, cv.WINDOW_NORMAL) 124 | cv.resizeWindow(cv_window_name, 800, 600) 125 | 126 | frame_order = [] 127 | frame_id = 1 128 | 129 | face_inference.infer(source) 130 | faces = face_inference.get_face_detection_data() 131 | if face_inference.Config.ModelType == FaceDetectionModelTypes.MTCNN: 132 | landmarks = face_inference.get_face_landmarks_data() 133 | 134 | if len(faces) > 0: 135 | print("Detected {} Faces with {} Threshold".format(len(faces), face_inference.Config.FaceDetectionThreshold)) 136 | for idx, face in enumerate(faces): 137 | ImageUtil.draw_rectangle(source, (face[0], face[1], face[2], face[3])) 138 | if face_inference.Config.ModelType == FaceDetectionModelTypes.MTCNN: 139 | for coordinate in range(0, len(landmarks[idx]), 2): 140 | ImageUtil.draw_ellipse(source, [landmarks[idx][coordinate], landmarks[idx][coordinate + 1]]) 141 | 142 | if run_age_gender: 143 | cropped_image = ImageUtil.crop_frame(source, (face[0], face[1], face[2], face[3])) 144 | if cropped_image.size > 0: 145 | age_inference.infer(cropped_image) 146 | age, gender = age_inference.get_age_gender_data() 147 | age_gender_text = '{} - {}' 148 | age_gender_text = age_gender_text.format(age, gender) 149 | ImageUtil.draw_text(source, age_gender_text, 150 | (face[0], face[1], face[2], face[3])) 151 | 152 | cv.imshow(cv_window_name, source) 153 | cv.waitKey(0) 154 | 155 | return None 156 | 157 | 158 | def video_inference(face_inference=None, age_inference=None, source=None): 159 | """ 160 | 161 | :param face_inference: 162 | :param age_inference: 163 | :param source: 164 | :return: 165 | """ 166 | face_request_order = list() 167 | face_process_order = list() 168 | 169 | for i in range(face_inference.Config.RequestCount): 170 | face_request_order.append(i) 171 | 172 | cv.namedWindow(cv_window_name, cv.WINDOW_NORMAL) 173 | cv.resizeWindow(cv_window_name, 800, 600) 174 | 175 | frame_order = [] 176 | frame_id = 1 177 | 178 | has_frame, frame = source.read() 179 | 180 | while has_frame: 181 | logging.log(logging.DEBUG, "Processing Frame {}".format(frame_id)) 182 | if len(face_request_order) > 0: 183 | req_id = face_request_order[0] 184 | face_request_order.pop(0) 185 | face_inference.infer(frame, req_id) 186 | face_process_order.append(req_id) 187 | frame_order.append(frame) 188 | 189 | if len(face_process_order) > 0: 190 | first = face_process_order[0] 191 | if face_inference.request_ready(request_id=first): 192 | detected_faces = face_inference.get_face_detection_data(first) 193 | if face_inference.Config.ModelType == FaceDetectionModelTypes.MTCNN: 194 | face_landmarks = face_inference.get_face_landmarks_data(first) 195 | face_process_order.pop(0) 196 | face_request_order.append(first) 197 | show_frame = frame_order[0] 198 | frame_order.pop(0) 199 | if len(detected_faces) > 0: 200 | for idx, face in enumerate(detected_faces): 201 | ImageUtil.draw_rectangle(show_frame, (face[0], face[1], face[2], face[3])) 202 | 203 | if face_inference.Config.ModelType == FaceDetectionModelTypes.MTCNN: 204 | for coordinate in range(0, len(face_landmarks[idx]), 2): 205 | ImageUtil.draw_ellipse(show_frame, [face_landmarks[idx][coordinate], 206 | face_landmarks[idx][coordinate + 1]]) 207 | 208 | if run_age_gender: 209 | cropped_image = ImageUtil.crop_frame(show_frame, (face[0], face[1], face[2], face[3])) 210 | if cropped_image.size > 0: 211 | age_inference.infer(cropped_image) 212 | age, gender = age_inference.get_age_gender_data() 213 | age_gender_text = '{} - {}' 214 | age_gender_text = age_gender_text.format(age, gender) 215 | ImageUtil.draw_text(show_frame, age_gender_text, (face[0], face[1], face[2], face[3])) 216 | 217 | cv.imshow(cv_window_name, show_frame) 218 | if cv.waitKey(1) & 0xFF == ord('q'): 219 | break 220 | 221 | if len(face_request_order) > 0: 222 | has_frame, frame = source.read() 223 | frame_id += 1 224 | 225 | face_inference.print_inference_performance_metrics() 226 | if run_age_gender: 227 | age_inference.print_inference_performance_metrics() 228 | 229 | return None 230 | 231 | 232 | """ 233 | Global Parameters Used for Application Configuration 234 | """ 235 | 236 | cv_window_name = 'Face-Detection' 237 | run_age_gender = False 238 | input_type = "image" 239 | input_path = '' 240 | web_cam_index = 0 241 | face_detection_model = FaceDetectionModelTypes.OPENMODELZOO 242 | age_gender_model = AgeGenderDetectionTypes.OPENMODELZOO 243 | config_file = "~/Projects/face_detection/config/config.json" 244 | 245 | 246 | def parse_config_file(config_json='config.json'): 247 | """ 248 | Parse Config File 249 | :param config_json: 250 | :return: 251 | """ 252 | global config_file 253 | config_file = config_json 254 | 255 | try: 256 | with open(config_json) as json_file: 257 | data = json.load(json_file) 258 | 259 | global cv_window_name 260 | cv_window_name = data['output_window_name'] 261 | 262 | global input_path 263 | input_path = data["input_path"] 264 | 265 | global input_type 266 | input_type = data["input_type"] 267 | 268 | global web_cam_index 269 | web_cam_index = int(data["web_cam_index"]) 270 | 271 | global run_age_gender 272 | if data['run_age_gender'] == "True": 273 | run_age_gender = True 274 | 275 | global face_detection_model 276 | if data['face_detection_model'] == FaceDetectionModelTypes.MTCNN: 277 | face_detection_model = FaceDetectionModelTypes.MTCNN 278 | 279 | global age_gender_model 280 | if data['age_gender_detection_model'] == AgeGenderDetectionTypes.MTCNN: 281 | age_gender_model = AgeGenderDetectionTypes.MTCNN 282 | 283 | if data["log_level"] == "DEBUG": 284 | logging.basicConfig(level=logging.DEBUG) 285 | elif data["log_level"] == "INFO": 286 | logging.basicConfig(level=logging.INFO) 287 | elif data["log_level"] == "WARN": 288 | logging.basicConfig(level=logging.WARN) 289 | else: 290 | logging.basicConfig(level=logging.ERROR) 291 | 292 | logging.log(logging.WARN, "Log Level Set to: {}".format(data["log_level"])) 293 | 294 | except FileNotFoundError: 295 | print('{} FileNotFound'.format(config_json)) 296 | exit(-1) 297 | 298 | 299 | def print_help(): 300 | print('Usage: python3 face_detection_openvino.py ') 301 | 302 | 303 | # Application Entry Point 304 | if __name__ == "__main__": 305 | 306 | if len(sys.argv) is not 2: 307 | print_help() 308 | print('Using default config file: {}'.format(config_file)) 309 | parse_config_file(config_file) 310 | else: 311 | parse_config_file(sys.argv[1]) 312 | 313 | # Run FD App 314 | run_app() 315 | -------------------------------------------------------------------------------- /config/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "omz_facedetection" : { 3 | "model_path" : "~/openvino_models/Retail/object_detection/face/sqnet1.0modif-ssd/0004/dldt/", 4 | "model_name" : "face-detection-retail-0004", 5 | "target_device" : "CPU", 6 | "cpu_extension" : "True", 7 | "cpu_extension_path" : "~/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so", 8 | "face_detection_threshold" : 0.6, 9 | "async" : "False", 10 | "request_count" : 1, 11 | "dynamic_batch" : "False", 12 | "batch_size" : 1, 13 | "limit_cpu_threads" : "False", 14 | "number_of_cpu_threads" : 4, 15 | "bind_cpu_threads" : "True", 16 | "cpu_stream" : "AUTO", 17 | "gpu_stream" : "AUTO" 18 | }, 19 | 20 | "omz_age_gender" : { 21 | "model_path" : "~/openvino_models/Retail/object_attributes/age_gender/dldt/FP32/", 22 | "model_name" : "age-gender-recognition-retail-0013", 23 | "target_device" : "CPU", 24 | "cpu_extension" : "True", 25 | "cpu_extension_path" : "~/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so", 26 | "async" : "False", 27 | "request_count" : 1, 28 | "dynamic_batch" : "False", 29 | "batch_size" : 1, 30 | "limit_cpu_threads" : "False", 31 | "number_of_cpu_threads" : 4, 32 | "bind_cpu_threads" : "True", 33 | "cpu_stream" : "AUTO", 34 | "gpu_stream" : "AUTO" 35 | }, 36 | 37 | "mtcnn_facedetection" : { 38 | "model_path" : "~/Projects/customer_tests/face_detection_mtcnn_opr/FP32/", 39 | "p_model_file_name" : "det1-0001", 40 | "r_model_file_name" : "det2-0001", 41 | "o_model_file_name" : "det3-0001", 42 | "target_device" : "CPU", 43 | "cpu_extension" : "True", 44 | "cpu_extension_path" : "~/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so", 45 | 46 | "p_network_threshold" : 0.6, 47 | "r_network_threshold" : 0.7, 48 | "o_network_threshold" : 0.8, 49 | 50 | "minimum_face_size" : 15.0, 51 | "minimum_length" : 720, 52 | "factor_count" : 0, 53 | "factor" : 0.707, 54 | "min_detection_size" : 12, 55 | 56 | "nms_thresholds" : [0.6, 0.6, 0.6], 57 | "r_input_batch_size" : 256, 58 | "o_input_batch_size" : 256, 59 | 60 | "limit_cpu_threads" : "False", 61 | "number_of_cpu_threads" : 4, 62 | "bind_cpu_threads" : "True", 63 | "cpu_stream" : "AUTO", 64 | "gpu_stream" : "AUTO" 65 | }, 66 | 67 | "mtcnn_age_gender" : { 68 | "model_path" : "~/DeepLearning/Models/gender-age/", 69 | "model_name" : "model-0000", 70 | "target_device" : "CPU", 71 | "cpu_extension" : "True", 72 | "cpu_extension_path" : "~/inference_engine_samples_build/intel64/Release/lib/libcpu_extension.so", 73 | "async" : "False", 74 | "request_count" : 1, 75 | "dynamic_batch" : "False", 76 | "batch_size" : 1, 77 | "limit_cpu_threads" : "False", 78 | "number_of_cpu_threads" : 4, 79 | "bind_cpu_threads" : "True", 80 | "cpu_stream" : "AUTO", 81 | "gpu_stream" : "AUTO" 82 | }, 83 | 84 | "face_detection_model" : "mtcnn_facedetection", 85 | "age_gender_detection_model" : "mtcnn_age_gender", 86 | "run_age_gender" : "True", 87 | "show_output" : "True", 88 | "output_window_name" : "Face-Detection", 89 | 90 | "input_type" : "webcam", 91 | "input_path" : "~/Videos/facedetection.mp4", 92 | "web_cam_index" : 0, 93 | 94 | "log_level" : "DEBUG" 95 | } -------------------------------------------------------------------------------- /detection/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/detection/__init__.py -------------------------------------------------------------------------------- /detection/age_gender_detection_ov.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import json 24 | import numpy as np 25 | import logging 26 | 27 | from .detection_base_ov import InferenceBase, InferenceConfig 28 | 29 | 30 | class AgeGenderDetectionTypes: 31 | MTCNN = "mtcnn_age_gender" 32 | OPENMODELZOO = "omz_age_gender" 33 | 34 | 35 | class AgeGenderConfig(InferenceConfig): 36 | ModelType = AgeGenderDetectionTypes.OPENMODELZOO 37 | 38 | def parse_json(self, json_file): 39 | try: 40 | logging.log(logging.INFO, "Loading JSON File {}".format(json_file)) 41 | logging.log(logging.INFO, "Model Type {}".format(self.ModelType)) 42 | 43 | with open(json_file) as json_file: 44 | data = json.load(json_file) 45 | 46 | self.ModelPath = data[self.ModelType]["model_path"] 47 | self.ModelName = data[self.ModelType]["model_name"] 48 | 49 | self.TargetDevice = data[self.ModelType]["target_device"] 50 | 51 | if data[self.ModelType]["async"] == "True": 52 | self.Async = True 53 | 54 | self.RequestCount = int(data[self.ModelType]["request_count"]) 55 | 56 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 57 | 58 | if data[self.ModelType]["cpu_extension"] == "True": 59 | self.CpuExtension = True 60 | 61 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 62 | 63 | if data[self.ModelType]["dynamic_batch"] == "True": 64 | self.DynamicBatch = True 65 | 66 | if data[self.ModelType]["limit_cpu_threads"] == "True": 67 | self.LimitCPUThreads = True 68 | 69 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 70 | 71 | if data[self.ModelType]["bind_cpu_threads"] == "True": 72 | self.LimitCPUThreads = True 73 | 74 | self.CPUStream = data[self.ModelType]["cpu_stream"] 75 | 76 | except FileNotFoundError: 77 | logging.log(logging.ERROR, '{} FileNotFound'.format(json_file)) 78 | exit(-1) 79 | 80 | def read_dict(self, data=None): 81 | """ 82 | Used When JSON Already Parsed as Dict 83 | :return: 84 | """ 85 | if data is None: 86 | data = dict() 87 | self.ModelPath = data[self.ModelType]["model_path"] 88 | self.ModelName = data[self.ModelType]["model_name"] 89 | 90 | self.TargetDevice = data[self.ModelType]["target_device"] 91 | 92 | if data[self.ModelType]["async"] == "True": 93 | self.Async = True 94 | 95 | self.RequestCount = int(data[self.ModelType]["request_count"]) 96 | 97 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 98 | 99 | if data[self.ModelType]["cpu_extension"] == "True": 100 | self.CpuExtension = True 101 | 102 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 103 | 104 | if data[self.ModelType]["dynamic_batch"] == "True": 105 | self.DynamicBatch = True 106 | 107 | if data[self.ModelType]["limit_cpu_threads"] == "True": 108 | self.LimitCPUThreads = True 109 | 110 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 111 | 112 | if data[self.ModelType]["bind_cpu_threads"] == "True": 113 | self.LimitCPUThreads = True 114 | 115 | self.CPUStream = data[self.ModelType]["cpu_stream"] 116 | 117 | 118 | class MTCNNAgeGenderConfig(AgeGenderConfig): 119 | ModelType = AgeGenderDetectionTypes.MTCNN 120 | 121 | 122 | class MTCNNAgeGenderDetection(InferenceBase): 123 | 124 | Config = MTCNNAgeGenderConfig() 125 | 126 | def get_age_gender_data(self, request_id=0): 127 | """ 128 | Parse Output Data for Age-Gender Detection Model 129 | :param request_id: 130 | :return: 131 | """ 132 | detection = self.OpenVinoExecutable.requests[request_id].outputs[self.OutputLayer] 133 | # Parse detection vector to get age and gender 134 | gender_vector = detection[:, 0:2].flatten() 135 | gender = int(np.argmax(gender_vector)) 136 | 137 | gender_text = 'female' 138 | if gender == 1: 139 | gender_text = 'male' 140 | 141 | age_matrix = detection[:, 2:202].reshape((100, 2)) 142 | ages = np.argmax(age_matrix, axis=1) 143 | age = int(sum(ages)) 144 | 145 | return age, gender_text 146 | 147 | 148 | class AgeGenderDetection(InferenceBase): 149 | 150 | Config = AgeGenderConfig() 151 | 152 | def get_age_gender_data(self, request_id=0): 153 | """ 154 | Parse Output Data for Age-Gender Detection Model 155 | :param request_id: 156 | :return: 157 | """ 158 | age = int(self.OpenVinoExecutable.requests[request_id].outputs["age_conv3"][0][0][0][0] * 100) 159 | genders = self.OpenVinoExecutable.requests[request_id].outputs["prob"] 160 | # Parse detection vector to get age and gender 161 | 162 | gender_text = 'female' 163 | if genders[0][0][0][0] < genders[0][1][0][0]: 164 | gender_text = 'male' 165 | 166 | return age, gender_text -------------------------------------------------------------------------------- /detection/detection_base_ov.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import time, logging, json 24 | import cv2 as cv 25 | import numpy as np 26 | 27 | # Import OpenVINO 28 | # Make sure environment variables set correctly for this to work 29 | # Check on README.md file 30 | from openvino.inference_engine import IENetwork, IECore, ExecutableNetwork 31 | 32 | 33 | class InferenceConfig(object): 34 | """ 35 | Inference Configuration Model 36 | """ 37 | ModelType = "" 38 | ModelPath = str() 39 | ModelName = str() 40 | TargetDevice = str() 41 | Async = False 42 | RequestCount = 1 43 | DynamicBatch = False 44 | BatchSize = 1 45 | CpuExtension = False 46 | CpuExtensionPath = "/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension.so" 47 | LimitCPUThreads = False 48 | CPUThreadNum = 1 49 | BindCPUThreads = True 50 | CPUStream = "AUTO" 51 | 52 | def parse_json(self, json_file): 53 | """ 54 | Parse JSON Parameters 55 | :param json_file: 56 | :return: 57 | """ 58 | try: 59 | logging.log(logging.INFO, "Loading JSON File".format(json_file)) 60 | with open(json_file) as json_file: 61 | data = json.load(json_file) 62 | 63 | self.ModelPath = data[self.ModelType]["model_path"] 64 | self.ModelName = data[self.ModelType]["model_name"] 65 | self.TargetDevice = data[self.ModelType]["target_device"] 66 | 67 | if data[self.ModelType]["async"] == "True": 68 | self.Async = True 69 | 70 | self.RequestCount = int(data[self.ModelType]["request_count"]) 71 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 72 | 73 | if data[self.ModelType]["cpu_extension"] == "True": 74 | self.CpuExtension = True 75 | 76 | self.CpuExtensionPath = data["cpu_extension_path"] 77 | 78 | if data[self.ModelType]["dynamic_batch"] == "True": 79 | self.DynamicBatch = True 80 | 81 | if data[self.ModelType]["limit_cpu_threads"] == "True": 82 | self.LimitCPUThreads = True 83 | 84 | self.CPUThreadNum = int(data[self.ModelType]["cpu_thread_num"]) 85 | 86 | if data[self.ModelType]["bind_cpu_threads"] == "True": 87 | self.LimitCPUThreads = True 88 | 89 | self.CPUStream = data[self.ModelType]["cpu_stream"] 90 | 91 | except FileNotFoundError: 92 | logging.log(logging.ERROR,'{} FileNotFound'.format(json_file)) 93 | exit(-1) 94 | 95 | def read_dict(self, data=None): 96 | """ 97 | Used When JSON Already Parsed as Dict 98 | :return: 99 | """ 100 | if data is None: 101 | data = dict() 102 | self.ModelPath = data[self.ModelType]["model_path"] 103 | self.ModelName = data[self.ModelType]["model_name"] 104 | self.TargetDevice = data[self.ModelType]["target_device"] 105 | 106 | if data[self.ModelType]["async"] == "True": 107 | self.Async = True 108 | 109 | self.RequestCount = int(data[self.ModelType]["request_count"]) 110 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 111 | 112 | if data[self.ModelType]["cpu_extension"] == "True": 113 | self.CpuExtension = True 114 | 115 | self.CpuExtensionPath = data["cpu_extension_path"] 116 | 117 | if data[self.ModelType]["dynamic_batch"] == "True": 118 | self.DynamicBatch = True 119 | 120 | if data[self.ModelType]["limit_cpu_threads"] == "True": 121 | self.LimitCPUThreads = True 122 | 123 | self.CPUThreadNum = int(data[self.ModelType]["cpu_thread_num"]) 124 | 125 | if data[self.ModelType]["bind_cpu_threads"] == "True": 126 | self.LimitCPUThreads = True 127 | 128 | self.CPUStream = data[self.ModelType]["cpu_stream"] 129 | 130 | 131 | class InferenceBase(object): 132 | """ 133 | Base Class to Load a Model with Inference Engine 134 | """ 135 | 136 | Config = InferenceConfig() 137 | 138 | '''Inference Engine Components''' 139 | OpenVinoIE = IECore() 140 | OpenVinoNetwork = IENetwork() 141 | OpenVinoExecutable = ExecutableNetwork() 142 | 143 | '''Model Components''' 144 | InputLayer = str() 145 | InputLayers = list() 146 | OutputLayer = str() 147 | OutputLayers = list() 148 | InputShape = None 149 | OutputShape = None 150 | 151 | '''Performance Metrics Storage''' 152 | ElapsedInferenceTime = 0.0 153 | InferenceCount = 0.0 154 | 155 | def __init__(self, infer_config): 156 | self.Config = infer_config 157 | self.prepare_detector() 158 | 159 | def prepare_detector(self): 160 | """ 161 | Load Model, Libraries According to Given Configuration. 162 | :return: 163 | """ 164 | if self.Config.ModelPath is None or self.Config.ModelName is None: 165 | return None 166 | 167 | ''' Model File Paths ''' 168 | model_file = self.Config.ModelPath + self.Config.ModelName + '.xml' 169 | model_weights = self.Config.ModelPath + self.Config.ModelName + '.bin' 170 | 171 | logging.log(logging.INFO, "Model File {}".format(model_file)) 172 | logging.log(logging.INFO, "Model Weights {}".format(model_weights)) 173 | 174 | ''' Create IECore Object ''' 175 | self.OpenVinoIE = IECore() 176 | 177 | ''' If target device is CPU add extensions ''' 178 | if self.Config.CpuExtension and 'CPU' in self.Config.TargetDevice: 179 | logging.log(logging.INFO, "Adding CPU Extensions, Path {}".format(self.Config.CpuExtensionPath)) 180 | self.OpenVinoIE.add_extension(self.Config.CpuExtensionPath, "CPU") 181 | 182 | ''' Try loading network ''' 183 | try: 184 | self.OpenVinoNetwork = IENetwork(model=model_file, weights=model_weights) 185 | logging.log(logging.INFO, "Loaded IENetwork") 186 | except FileNotFoundError: 187 | logging.log(logging.ERROR, FileNotFoundError.strerror + " " + FileNotFoundError.filename) 188 | logging.log(logging.ERROR, "Exiting ....") 189 | exit(-1) 190 | 191 | ''' Print supported/not-supported layers ''' 192 | if "CPU" in self.Config.TargetDevice: 193 | supported_layers = self.OpenVinoIE.query_network(self.OpenVinoNetwork, "CPU") 194 | not_supported_layers = [l for l in self.OpenVinoNetwork.layers.keys() if l not in supported_layers] 195 | if len(not_supported_layers) != 0: 196 | logging.log(logging.WARN, "Following layers are not supported by the plugin for specified device {}:\n {}".format(self.Config.TargetDevice, ', '.join(not_supported_layers))) 197 | logging.log(logging.WARN, "Please try to specify cpu extensions library path in config.json file ") 198 | 199 | '''Input / Output Memory Allocations to feed input or get output values''' 200 | self.InputLayer = next(iter(self.OpenVinoNetwork.inputs)) 201 | logging.log(logging.INFO, "Input Layer ".format(self.InputLayer)) 202 | 203 | N, C, H, W = self.OpenVinoNetwork.inputs[self.InputLayer].shape 204 | 205 | if self.Config.BatchSize > N: 206 | self.OpenVinoNetwork.batch_size = self.Config.BatchSize 207 | else: 208 | self.Config.BatchSize = self.OpenVinoNetwork.batch_size 209 | 210 | self.OutputLayer = next(iter(self.OpenVinoNetwork.outputs)) 211 | logging.log(logging.INFO, "Output Layer ".format(self.OutputLayer)) 212 | 213 | self.InputLayers = list(self.OpenVinoNetwork.inputs) 214 | logging.log(logging.INFO, "Input Layers ".format(self.InputLayers)) 215 | 216 | self.OutputLayers = list(self.OpenVinoNetwork.outputs) 217 | logging.log(logging.INFO, "Output Layers ".format(self.OutputLayers)) 218 | 219 | self.InputShape = self.OpenVinoNetwork.inputs[self.InputLayer].shape 220 | logging.log(logging.INFO, "Input Shape: {}".format(self.InputShape)) 221 | 222 | self.OutputShape = self.OpenVinoNetwork.outputs[self.OutputLayer].shape 223 | logging.log(logging.INFO, "Output Shape: {}".format(self.OutputShape)) 224 | 225 | '''Set Configurations''' 226 | 227 | config = {} 228 | 229 | if self.Config.DynamicBatch: 230 | config["DYN_BATCH_ENABLE"] = "YES" 231 | logging.log(logging.INFO, "Enabling Dynamic Batch Mode") 232 | 233 | if self.Config.Async: 234 | logging.log(logging.INFO, "Async Mode Enabled") 235 | 236 | self.OpenVinoExecutable = self.OpenVinoIE.load_network(network=self.OpenVinoNetwork, 237 | device_name=self.Config.TargetDevice, 238 | config=config, 239 | num_requests=self.Config.RequestCount) 240 | 241 | logging.log(logging.INFO, "Completed Loading Neural Network") 242 | 243 | return None 244 | 245 | def preprocess_input(self, input_data): 246 | """ 247 | Pre-process Input According to Loaded Network 248 | :param input_data: 249 | :return: 250 | """ 251 | 252 | n, c, h, w = self.OpenVinoNetwork.inputs[self.InputLayer].shape 253 | logging.log(logging.DEBUG, "Pre-processing Input to Shape {}".format(self.OpenVinoNetwork.inputs[self.InputLayer].shape)) 254 | 255 | resized = cv.resize(input_data, (w, h)) 256 | color_converted = cv.cvtColor(resized, cv.COLOR_BGR2RGB) 257 | transposed = np.transpose(color_converted, (2, 0, 1)) 258 | reshaped = np.expand_dims(transposed, axis=0) 259 | 260 | return reshaped 261 | 262 | def infer(self, input_data, request_id=0): 263 | """ 264 | Used to send data to network and start forward propagation. 265 | :param input_data: 266 | :param request_id: 267 | :return: 268 | """ 269 | if self.Config.Async: 270 | logging.log(logging.DEBUG, "Async Infer Request Id {}".format(request_id)) 271 | self.infer_async(input_data, request_id) 272 | else: 273 | logging.log(logging.DEBUG, "Infer Request Id {}".format(request_id)) 274 | self.infer_sync(input_data, request_id) 275 | 276 | def infer_async(self, input_data, request_id=0): 277 | """ 278 | Start Async Infer for Given Request Id 279 | :param input_data: 280 | :param request_id: 281 | :return: 282 | """ 283 | self.InferenceCount += 1 284 | processed_input = self.preprocess_input(input_data) 285 | self.OpenVinoExecutable.requests[request_id].async_infer(inputs={self.InputLayer: processed_input}) 286 | 287 | def infer_sync(self, input_data, request_id=0): 288 | """ 289 | Start Sync Infer 290 | :param input_data: 291 | :param request_id: 292 | :return: 293 | """ 294 | self.InferenceCount += 1 295 | processed_input = self.preprocess_input(input_data) 296 | start = time.time() 297 | self.OpenVinoExecutable.requests[request_id].infer(inputs={self.InputLayer: processed_input}) 298 | end = time.time() 299 | self.ElapsedInferenceTime += (end - start) 300 | 301 | def request_ready(self, request_id): 302 | """ 303 | Check if request is ready 304 | :param request_id: id to check request 305 | :return: bool 306 | """ 307 | if self.Config.Async: 308 | if self.OpenVinoExecutable.requests[request_id].wait(0) == 0: 309 | return True 310 | else: 311 | return True 312 | 313 | return False 314 | 315 | def get_results(self, output_layer, request_id=0): 316 | """ 317 | Get results from the network. 318 | :param output_layer: output layer 319 | :param request_id: request id 320 | :return: 321 | """ 322 | logging.log(logging.DEBUG, "Getting Results Request Id {}".format(request_id)) 323 | return self.OpenVinoExecutable.requests[request_id].outputs[output_layer] 324 | 325 | def print_inference_performance_metrics(self): 326 | """ 327 | Print Performance Data Collection 328 | :return: 329 | """ 330 | if self.Config.Async: 331 | logging.log(logging.WARN, 'Async Mode Inferred Frame Count {}'.format(self.InferenceCount)) 332 | else: 333 | logging.log(logging.WARN, "Sync Mode Inferred Frame Count {}".format(self.InferenceCount)) 334 | logging.log(logging.WARN, "Inference Per Input: {} MilliSeconds".format((self.ElapsedInferenceTime / self.InferenceCount) * 1000)) 335 | -------------------------------------------------------------------------------- /files/README.md: -------------------------------------------------------------------------------- 1 | Copy `files/intel-openvino.sh` & `files/intel-openvino.conf` file as shown below, they set the environment variables for OpenVINO(TM) system wide. 2 | 3 | ```bash 4 | sudo cp files/intel-openvino.sh /etc/profile.d/ 5 | sudo cp files/intel-openvino.conf /etc/ld.so.conf.d/ 6 | sudo reboot 7 | ``` -------------------------------------------------------------------------------- /files/intel-openvino.conf: -------------------------------------------------------------------------------- 1 | /opt/intel/openvino/opencv/lib 2 | /opt/intel/opencl 3 | /opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib 4 | /opt/intel/openvino/deployment_tools/inference_engine/external/gna/lib 5 | /opt/intel/openvino/deployment_tools/inference_engine/external/mkltiny_lnx/lib 6 | /opt/intel/openvino/deployment_tools/inference_engine/external/tbb/lib 7 | /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64 8 | /opt/intel/openvino/openvx/lib 9 | -------------------------------------------------------------------------------- /files/intel-openvino.sh: -------------------------------------------------------------------------------- 1 | export PATH="$PATH:/opt/intel/openvino/deployment_tools/model_optimizer" 2 | export PYTHONPATH="/opt/intel/openvino/python/python3.5:/opt/intel/openvino/deployment_tools/model_optimizer:/opt/intel/openvino/python/python3:/opt/intel/openvino/deployment_tools/model_optimizer:" 3 | export OpenCV_DIR="/opt/intel/openvino/opencv/cmake" 4 | export InferenceEngine_DIR="/opt/intel/openvino/deployment_tools/inference_engine/share" 5 | export IE_PLUGINS_PATH="/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64" 6 | export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/opencv/lib:/opt/intel/opencl:/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib" 7 | export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/external/gna/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/mkltiny_lnx/lib" 8 | export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/external/tbb/lib" 9 | export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64" 10 | export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/openvx/lib" 11 | export INTEL_OPENVINO_DIR="/opt/intel/openvino" 12 | export INTEL_CVSDK_DIR="/opt/intel/openvino" 13 | export HDDL_INSTALL_DIR="/opt/intel/openvino/deployment_tools/inference_engine/external/hddl" 14 | -------------------------------------------------------------------------------- /inference_services/README.md: -------------------------------------------------------------------------------- 1 | # Dockerize AI Applications 2 | 3 | AWS, Google Cloud & Azure already provides their services for AI. 4 | 5 | They mainly aims to enable you to use services to send data and get the predictions. 6 | 7 | Let's try to utilize this face detection modules to be used as a docker service. 8 | 9 | ## Services 10 | 11 | 1. Face Detection Service 12 | 13 | I want to deploy a face detection service which I can connect a network stream to get the detected faces from that video stream. 14 | 15 | 2. Age Gender Detection Service 16 | 17 | I want to deploy a age-gender detection service which I can send face frames to get the age and gender data. 18 | 19 | # System Design 20 | 21 | ## How to send data? 22 | 23 | JSON is used to send data, which can also include the remote video stream or remote image url to send to docker application to read from. 24 | 25 | ## How to receive data? 26 | 27 | JSON files can be send to user to get the information; 28 | 29 | - For face detection, number of faces detected each second etc. 30 | - For age-gender detection, age and gender information. 31 | 32 | -------------------------------------------------------------------------------- /inference_services/facedetection/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:18.04 2 | 3 | MAINTAINER ONUR DUNDAR "onur.dundar1@gmail.com" 4 | 5 | # ARG DOWNLOAD_LINK=~/Projects/workshop_installation/openvino_installer/l_openvino_toolkit_p_2019.2.242.tgz 6 | ARG INSTALL_DIR=/opt/intel/openvino 7 | ARG TEMP_DIR=/tmp/openvino_installer 8 | 9 | RUN apt-get update && apt-get install -y --no-install-recommends \ 10 | wget \ 11 | cpio \ 12 | sudo \ 13 | lsb-release && \ 14 | rm -rf /var/lib/apt/lists/* 15 | 16 | RUN mkdir -p $TEMP_DIR && cd $TEMP_DIR 17 | 18 | COPY l_openvino_toolkit_p_2019.2.242.tgz $TEMP_DIR 19 | 20 | WORKDIR $TEMP_DIR 21 | 22 | RUN tar xf l_openvino_toolkit_p_2019.2.242.tgz && \ 23 | cd l_openvino_toolkit_p_2019.2.242 && \ 24 | sed -i 's/decline/accept/g' silent.cfg && \ 25 | ./install.sh -s silent.cfg && \ 26 | rm -rf $TEMP_DIR 27 | 28 | RUN $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh 29 | 30 | RUN apt-get update -y && \ 31 | apt-get install -y python3-pip python3-dev libgtk-3-0 32 | 33 | WORKDIR /app 34 | COPY requirements.txt /app 35 | 36 | RUN pip3 install -r requirements.txt 37 | 38 | ENV PATH="$PATH:/opt/intel/openvino/deployment_tools/model_optimizer" 39 | ENV PYTHONPATH="/opt/intel/openvino/python/python3.6:/opt/intel/openvino/deployment_tools/model_optimizer:/opt/intel/openvino/python/python3:" 40 | ENV OpenCV_DIR="/opt/intel/openvino/opencv/cmake" 41 | ENV InferenceEngine_DIR="/opt/intel/openvino/deployment_tools/inference_engine/share" 42 | ENV IE_PLUGINS_PATH="/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64" 43 | ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/opencv/lib:/opt/intel/opencl:/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib" 44 | ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/external/gna/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/mkltiny_lnx/lib" 45 | ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/external/tbb/lib" 46 | ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64" 47 | ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/openvino/openvx/lib" 48 | ENV INTEL_OPENVINO_DIR="/opt/intel/openvino" 49 | ENV INTEL_CVSDK_DIR="/opt/intel/openvino" 50 | ENV HDDL_INSTALL_DIR="/opt/intel/openvino/deployment_tools/inference_engine/external/hddl" 51 | 52 | COPY detection /app/detection 53 | COPY utils /app/utils 54 | COPY models /app/models 55 | COPY videos /app/videos 56 | 57 | COPY face_detection_service.py /app 58 | 59 | EXPOSE 8000 60 | 61 | CMD ["python3", "/app/face_detection_service.py", "-p 8000"] -------------------------------------------------------------------------------- /inference_services/facedetection/README.md: -------------------------------------------------------------------------------- 1 | # Face Detection Service with Docker 2 | 3 | Here I have a very basic application developed with OpenVINO(TM) & Flask to start inference on given source. 4 | 5 | This app can handle single request at one time, multiple requests are not possible. 6 | 7 | # Build & Start Docker with WebCam 8 | 9 | Note: I had problems with webcam loading on docker so feel free to test as below. 10 | 11 | ```bash 12 | docker build -t facedetection:latest . 13 | 14 | docker run -d -p 8000:8000 facedetection --device=/dev/video0:/dev/video0 15 | ``` 16 | 17 | # JSON Config 18 | 19 | JSON Configuration have to made beforeh and in order to correctly start the inference 20 | 21 | # Curl Request to Start Face Detection Application 22 | 23 | ```bash 24 | curl --header "Content-Type: application/json" --request POST --data '@inference_config.json' http://127.0.0.1:8000/ 25 | 26 | curl --header "Content-Type: application/json" --request POST --data '@~/Projects/face_detection/inference_services/facedetection/inference_config.json' http://127.0.0.1:8000 27 | ``` 28 | 29 | # Retrieve Status 30 | 31 | You can retrieve the frame-id and face coordinates by a get request from: 32 | 33 | ```bash 34 | curl --request GET http://127.0.0.1:8000/status 35 | ``` 36 | 37 | # Retrieve Results 38 | 39 | You can retrieve the frame-id and face coordinates by a get request from: 40 | 41 | ```bash 42 | curl --request GET http://127.0.0.1:8000/results 43 | ``` 44 | 45 | # Check Logs 46 | 47 | ```bash 48 | curl --request GET http://127.0.0.1:8000/logs 49 | ``` 50 | 51 | # Play Video 52 | 53 | NOT IMPLEMENTED -------------------------------------------------------------------------------- /inference_services/facedetection/detection/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/detection/__init__.py -------------------------------------------------------------------------------- /inference_services/facedetection/detection/age_gender_detection_ov.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import json 24 | import numpy as np 25 | import logging 26 | 27 | from .detection_base_ov import InferenceBase, InferenceConfig 28 | 29 | 30 | class AgeGenderDetectionTypes: 31 | MTCNN = "mtcnn_age_gender" 32 | OPENMODELZOO = "omz_age_gender" 33 | 34 | 35 | class AgeGenderConfig(InferenceConfig): 36 | ModelType = AgeGenderDetectionTypes.OPENMODELZOO 37 | 38 | def parse_json(self, json_file): 39 | try: 40 | logging.log(logging.INFO, "Loading JSON File {}".format(json_file)) 41 | logging.log(logging.INFO, "Model Type {}".format(self.ModelType)) 42 | 43 | with open(json_file) as json_file: 44 | data = json.load(json_file) 45 | 46 | self.ModelPath = data[self.ModelType]["model_path"] 47 | self.ModelName = data[self.ModelType]["model_name"] 48 | 49 | self.TargetDevice = data[self.ModelType]["target_device"] 50 | 51 | if data[self.ModelType]["async"] == "True": 52 | self.Async = True 53 | 54 | self.RequestCount = int(data[self.ModelType]["request_count"]) 55 | 56 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 57 | 58 | if data[self.ModelType]["cpu_extension"] == "True": 59 | self.CpuExtension = True 60 | 61 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 62 | 63 | if data[self.ModelType]["dynamic_batch"] == "True": 64 | self.DynamicBatch = True 65 | 66 | if data[self.ModelType]["limit_cpu_threads"] == "True": 67 | self.LimitCPUThreads = True 68 | 69 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 70 | 71 | if data[self.ModelType]["bind_cpu_threads"] == "True": 72 | self.LimitCPUThreads = True 73 | 74 | self.CPUStream = data[self.ModelType]["cpu_stream"] 75 | 76 | except FileNotFoundError: 77 | logging.log(logging.ERROR, '{} FileNotFound'.format(json_file)) 78 | exit(-1) 79 | 80 | def read_dict(self, data=None): 81 | """ 82 | Used When JSON Already Parsed as Dict 83 | :return: 84 | """ 85 | if data is None: 86 | data = dict() 87 | self.ModelPath = data[self.ModelType]["model_path"] 88 | self.ModelName = data[self.ModelType]["model_name"] 89 | 90 | self.TargetDevice = data[self.ModelType]["target_device"] 91 | 92 | if data[self.ModelType]["async"] == "True": 93 | self.Async = True 94 | 95 | self.RequestCount = int(data[self.ModelType]["request_count"]) 96 | 97 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 98 | 99 | if data[self.ModelType]["cpu_extension"] == "True": 100 | self.CpuExtension = True 101 | 102 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 103 | 104 | if data[self.ModelType]["dynamic_batch"] == "True": 105 | self.DynamicBatch = True 106 | 107 | if data[self.ModelType]["limit_cpu_threads"] == "True": 108 | self.LimitCPUThreads = True 109 | 110 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 111 | 112 | if data[self.ModelType]["bind_cpu_threads"] == "True": 113 | self.LimitCPUThreads = True 114 | 115 | self.CPUStream = data[self.ModelType]["cpu_stream"] 116 | 117 | 118 | class MTCNNAgeGenderConfig(AgeGenderConfig): 119 | ModelType = AgeGenderDetectionTypes.MTCNN 120 | 121 | 122 | class MTCNNAgeGenderDetection(InferenceBase): 123 | 124 | Config = MTCNNAgeGenderConfig() 125 | 126 | def get_age_gender_data(self, request_id=0): 127 | """ 128 | Parse Output Data for Age-Gender Detection Model 129 | :param request_id: 130 | :return: 131 | """ 132 | detection = self.OpenVinoExecutable.requests[request_id].outputs[self.OutputLayer] 133 | # Parse detection vector to get age and gender 134 | gender_vector = detection[:, 0:2].flatten() 135 | gender = int(np.argmax(gender_vector)) 136 | 137 | gender_text = 'female' 138 | if gender == 1: 139 | gender_text = 'male' 140 | 141 | age_matrix = detection[:, 2:202].reshape((100, 2)) 142 | ages = np.argmax(age_matrix, axis=1) 143 | age = int(sum(ages)) 144 | 145 | return age, gender_text 146 | 147 | 148 | class AgeGenderDetection(InferenceBase): 149 | 150 | Config = AgeGenderConfig() 151 | 152 | def get_age_gender_data(self, request_id=0): 153 | """ 154 | Parse Output Data for Age-Gender Detection Model 155 | :param request_id: 156 | :return: 157 | """ 158 | age = int(self.OpenVinoExecutable.requests[request_id].outputs["age_conv3"][0][0][0][0] * 100) 159 | genders = self.OpenVinoExecutable.requests[request_id].outputs["prob"] 160 | # Parse detection vector to get age and gender 161 | 162 | gender_text = 'female' 163 | if genders[0][0][0][0] < genders[0][1][0][0]: 164 | gender_text = 'male' 165 | 166 | return age, gender_text -------------------------------------------------------------------------------- /inference_services/facedetection/detection/detection_base_ov.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import time, logging, json 24 | import cv2 as cv 25 | import numpy as np 26 | 27 | # Import OpenVINO 28 | # Make sure environment variables set correctly for this to work 29 | # Check on README.md file 30 | from openvino.inference_engine import IENetwork, IECore, ExecutableNetwork 31 | 32 | 33 | class InferenceConfig(object): 34 | """ 35 | Inference Configuration Model 36 | """ 37 | ModelType = "" 38 | ModelPath = str() 39 | ModelName = str() 40 | TargetDevice = str() 41 | Async = False 42 | RequestCount = 1 43 | DynamicBatch = False 44 | BatchSize = 1 45 | CpuExtension = False 46 | CpuExtensionPath = "/opt/intel/inference_engine/lib/intel64/libcpu_extension.so" 47 | LimitCPUThreads = False 48 | CPUThreadNum = 1 49 | BindCPUThreads = True 50 | CPUStream = "AUTO" 51 | 52 | def parse_json(self, json_file): 53 | """ 54 | Parse JSON Parameters 55 | :param json_file: 56 | :return: 57 | """ 58 | try: 59 | logging.log(logging.INFO, "Loading JSON File".format(json_file)) 60 | with open(json_file) as json_file: 61 | data = json.load(json_file) 62 | 63 | self.ModelPath = data[self.ModelType]["model_path"] 64 | self.ModelName = data[self.ModelType]["model_name"] 65 | self.TargetDevice = data[self.ModelType]["target_device"] 66 | 67 | if data[self.ModelType]["async"] == "True": 68 | self.Async = True 69 | 70 | self.RequestCount = int(data[self.ModelType]["request_count"]) 71 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 72 | 73 | if data[self.ModelType]["cpu_extension"] == "True": 74 | self.CpuExtension = True 75 | 76 | self.CpuExtensionPath = data["cpu_extension_path"] 77 | 78 | if data[self.ModelType]["dynamic_batch"] == "True": 79 | self.DynamicBatch = True 80 | 81 | if data[self.ModelType]["limit_cpu_threads"] == "True": 82 | self.LimitCPUThreads = True 83 | 84 | self.CPUThreadNum = int(data[self.ModelType]["cpu_thread_num"]) 85 | 86 | if data[self.ModelType]["bind_cpu_threads"] == "True": 87 | self.LimitCPUThreads = True 88 | 89 | self.CPUStream = data[self.ModelType]["cpu_stream"] 90 | 91 | except FileNotFoundError: 92 | logging.log(logging.ERROR,'{} FileNotFound'.format(json_file)) 93 | exit(-1) 94 | 95 | def read_dict(self, data=None): 96 | """ 97 | Used When JSON Already Parsed as Dict 98 | :return: 99 | """ 100 | if data is None: 101 | data = dict() 102 | self.ModelPath = data[self.ModelType]["model_path"] 103 | self.ModelName = data[self.ModelType]["model_name"] 104 | self.TargetDevice = data[self.ModelType]["target_device"] 105 | 106 | if data[self.ModelType]["async"] == "True": 107 | self.Async = True 108 | 109 | self.RequestCount = int(data[self.ModelType]["request_count"]) 110 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 111 | 112 | if data[self.ModelType]["cpu_extension"] == "True": 113 | self.CpuExtension = True 114 | 115 | self.CpuExtensionPath = data["cpu_extension_path"] 116 | 117 | if data[self.ModelType]["dynamic_batch"] == "True": 118 | self.DynamicBatch = True 119 | 120 | if data[self.ModelType]["limit_cpu_threads"] == "True": 121 | self.LimitCPUThreads = True 122 | 123 | self.CPUThreadNum = int(data[self.ModelType]["cpu_thread_num"]) 124 | 125 | if data[self.ModelType]["bind_cpu_threads"] == "True": 126 | self.LimitCPUThreads = True 127 | 128 | self.CPUStream = data[self.ModelType]["cpu_stream"] 129 | 130 | 131 | class InferenceBase(object): 132 | """ 133 | Base Class to Load a Model with Inference Engine 134 | """ 135 | 136 | Config = InferenceConfig() 137 | 138 | '''Inference Engine Components''' 139 | OpenVinoIE = IECore() 140 | OpenVinoNetwork = IENetwork() 141 | OpenVinoExecutable = ExecutableNetwork() 142 | 143 | '''Model Components''' 144 | InputLayer = str() 145 | InputLayers = list() 146 | OutputLayer = str() 147 | OutputLayers = list() 148 | InputShape = None 149 | OutputShape = None 150 | 151 | '''Performance Metrics Storage''' 152 | ElapsedInferenceTime = 0.0 153 | InferenceCount = 0.0 154 | 155 | def __init__(self, infer_config): 156 | self.Config = infer_config 157 | self.prepare_detector() 158 | 159 | def prepare_detector(self): 160 | """ 161 | Load Model, Libraries According to Given Configuration. 162 | :return: 163 | """ 164 | if self.Config.ModelPath is None or self.Config.ModelName is None: 165 | return None 166 | 167 | ''' Model File Paths ''' 168 | model_file = self.Config.ModelPath + self.Config.ModelName + '.xml' 169 | model_weights = self.Config.ModelPath + self.Config.ModelName + '.bin' 170 | 171 | logging.log(logging.INFO, "Model File {}".format(model_file)) 172 | logging.log(logging.INFO, "Model Weights {}".format(model_weights)) 173 | 174 | ''' Create IECore Object ''' 175 | self.OpenVinoIE = IECore() 176 | 177 | ''' If target device is CPU add extensions ''' 178 | if self.Config.CpuExtension and 'CPU' in self.Config.TargetDevice: 179 | logging.log(logging.INFO, "Adding CPU Extensions, Path {}".format(self.Config.CpuExtensionPath)) 180 | self.OpenVinoIE.add_extension(self.Config.CpuExtensionPath, "CPU") 181 | 182 | ''' Try loading network ''' 183 | try: 184 | self.OpenVinoNetwork = IENetwork(model=model_file, weights=model_weights) 185 | logging.log(logging.INFO, "Loaded IENetwork") 186 | except FileNotFoundError: 187 | logging.log(logging.ERROR, FileNotFoundError.strerror + " " + FileNotFoundError.filename) 188 | logging.log(logging.ERROR, "Exiting ....") 189 | exit(-1) 190 | 191 | ''' Print supported/not-supported layers ''' 192 | if "CPU" in self.Config.TargetDevice: 193 | supported_layers = self.OpenVinoIE.query_network(self.OpenVinoNetwork, "CPU") 194 | not_supported_layers = [l for l in self.OpenVinoNetwork.layers.keys() if l not in supported_layers] 195 | if len(not_supported_layers) != 0: 196 | logging.log(logging.WARN, "Following layers are not supported by the plugin for specified device {}:\n {}".format(self.Config.TargetDevice, ', '.join(not_supported_layers))) 197 | logging.log(logging.WARN, "Please try to specify cpu extensions library path in config.json file ") 198 | 199 | '''Input / Output Memory Allocations to feed input or get output values''' 200 | self.InputLayer = next(iter(self.OpenVinoNetwork.inputs)) 201 | logging.log(logging.INFO, "Input Layer ".format(self.InputLayer)) 202 | 203 | N, C, H, W = self.OpenVinoNetwork.inputs[self.InputLayer].shape 204 | 205 | if self.Config.BatchSize > N: 206 | self.OpenVinoNetwork.batch_size = self.Config.BatchSize 207 | else: 208 | self.Config.BatchSize = self.OpenVinoNetwork.batch_size 209 | 210 | self.OutputLayer = next(iter(self.OpenVinoNetwork.outputs)) 211 | logging.log(logging.INFO, "Output Layer ".format(self.OutputLayer)) 212 | 213 | self.InputLayers = list(self.OpenVinoNetwork.inputs) 214 | logging.log(logging.INFO, "Input Layers ".format(self.InputLayers)) 215 | 216 | self.OutputLayers = list(self.OpenVinoNetwork.outputs) 217 | logging.log(logging.INFO, "Output Layers ".format(self.OutputLayers)) 218 | 219 | self.InputShape = self.OpenVinoNetwork.inputs[self.InputLayer].shape 220 | logging.log(logging.INFO, "Input Shape: {}".format(self.InputShape)) 221 | 222 | self.OutputShape = self.OpenVinoNetwork.outputs[self.OutputLayer].shape 223 | logging.log(logging.INFO, "Output Shape: {}".format(self.OutputShape)) 224 | 225 | '''Set Configurations''' 226 | 227 | config = {} 228 | 229 | if self.Config.DynamicBatch: 230 | config["DYN_BATCH_ENABLE"] = "YES" 231 | logging.log(logging.INFO, "Enabling Dynamic Batch Mode") 232 | 233 | if self.Config.Async: 234 | logging.log(logging.INFO, "Async Mode Enabled") 235 | 236 | self.OpenVinoExecutable = self.OpenVinoIE.load_network(network=self.OpenVinoNetwork, 237 | device_name=self.Config.TargetDevice, 238 | config=config, 239 | num_requests=self.Config.RequestCount) 240 | 241 | logging.log(logging.INFO, "Completed Loading Neural Network") 242 | 243 | return None 244 | 245 | def preprocess_input(self, input_data): 246 | """ 247 | Pre-process Input According to Loaded Network 248 | :param input_data: 249 | :return: 250 | """ 251 | 252 | n, c, h, w = self.OpenVinoNetwork.inputs[self.InputLayer].shape 253 | logging.log(logging.DEBUG, "Pre-processing Input to Shape {}".format(self.OpenVinoNetwork.inputs[self.InputLayer].shape)) 254 | 255 | resized = cv.resize(input_data, (w, h)) 256 | color_converted = cv.cvtColor(resized, cv.COLOR_BGR2RGB) 257 | transposed = np.transpose(color_converted, (2, 0, 1)) 258 | reshaped = np.expand_dims(transposed, axis=0) 259 | 260 | return reshaped 261 | 262 | def infer(self, input_data, request_id=0): 263 | """ 264 | Used to send data to network and start forward propagation. 265 | :param input_data: 266 | :param request_id: 267 | :return: 268 | """ 269 | if self.Config.Async: 270 | logging.log(logging.DEBUG, "Async Infer Request Id {}".format(request_id)) 271 | self.infer_async(input_data, request_id) 272 | else: 273 | logging.log(logging.DEBUG, "Infer Request Id {}".format(request_id)) 274 | self.infer_sync(input_data, request_id) 275 | 276 | def infer_async(self, input_data, request_id=0): 277 | """ 278 | Start Async Infer for Given Request Id 279 | :param input_data: 280 | :param request_id: 281 | :return: 282 | """ 283 | self.InferenceCount += 1 284 | processed_input = self.preprocess_input(input_data) 285 | self.OpenVinoExecutable.requests[request_id].async_infer(inputs={self.InputLayer: processed_input}) 286 | 287 | def infer_sync(self, input_data, request_id=0): 288 | """ 289 | Start Sync Infer 290 | :param input_data: 291 | :param request_id: 292 | :return: 293 | """ 294 | self.InferenceCount += 1 295 | processed_input = self.preprocess_input(input_data) 296 | start = time.time() 297 | self.OpenVinoExecutable.requests[request_id].infer(inputs={self.InputLayer: processed_input}) 298 | end = time.time() 299 | self.ElapsedInferenceTime += (end - start) 300 | 301 | def request_ready(self, request_id): 302 | """ 303 | Check if request is ready 304 | :param request_id: id to check request 305 | :return: bool 306 | """ 307 | if self.Config.Async: 308 | if self.OpenVinoExecutable.requests[request_id].wait(0) == 0: 309 | return True 310 | else: 311 | return True 312 | 313 | return False 314 | 315 | def get_results(self, output_layer, request_id=0): 316 | """ 317 | Get results from the network. 318 | :param output_layer: output layer 319 | :param request_id: request id 320 | :return: 321 | """ 322 | logging.log(logging.DEBUG, "Getting Results Request Id {}".format(request_id)) 323 | return self.OpenVinoExecutable.requests[request_id].outputs[output_layer] 324 | 325 | def print_inference_performance_metrics(self): 326 | """ 327 | Print Performance Data Collection 328 | :return: 329 | """ 330 | if self.Config.Async: 331 | logging.log(logging.WARN, 'Async Mode Inferred Frame Count {}'.format(self.InferenceCount)) 332 | else: 333 | logging.log(logging.WARN, "Sync Mode Inferred Frame Count {}".format(self.InferenceCount)) 334 | logging.log(logging.WARN, "Inference Per Input: {} MilliSeconds".format((self.ElapsedInferenceTime / self.InferenceCount) * 1000)) 335 | -------------------------------------------------------------------------------- /inference_services/facedetection/detection/face_detection_ov.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import logging 24 | import math 25 | import json 26 | import time 27 | import cv2 as cv 28 | import numpy as np 29 | from PIL import Image 30 | 31 | from openvino.inference_engine import IENetwork, IECore, ExecutableNetwork 32 | from .detection_base_ov import InferenceConfig, InferenceBase 33 | 34 | 35 | class FaceDetectionModelTypes: 36 | """ 37 | Face Detection Model Type to Be Used 38 | """ 39 | # MTCNN Face Detection 40 | MTCNN = 'mtcnn_facedetection' 41 | # Open Model Zoo Face Detection 42 | OPENMODELZOO = 'omz_facedetection' 43 | 44 | 45 | class FaceDetectionConfig(InferenceConfig): 46 | """ 47 | Face Detection Module Configurations based on Open Model Zoo Face Detection Model 48 | """ 49 | ModelType = FaceDetectionModelTypes.OPENMODELZOO 50 | FaceDetectionThreshold = 1.0 51 | InputHeight = 720 52 | InputWidth = 1080 53 | 54 | def parse_json(self, json_file): 55 | try: 56 | logging.log(logging.INFO, "Loading JSON File {}".format(json_file)) 57 | logging.log(logging.INFO, "Model Type {}".format(self.ModelType)) 58 | 59 | with open(json_file) as json_file: 60 | data = json.load(json_file) 61 | 62 | self.ModelPath = data[self.ModelType]["model_path"] 63 | self.ModelName = data[self.ModelType]["model_name"] 64 | 65 | self.TargetDevice = data[self.ModelType]["target_device"] 66 | 67 | self.FaceDetectionThreshold = data[self.ModelType]["face_detection_threshold"] 68 | 69 | if data[self.ModelType]["async"] == "True": 70 | self.Async = True 71 | 72 | self.RequestCount = int(data[self.ModelType]["request_count"]) 73 | 74 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 75 | 76 | if data[self.ModelType]["cpu_extension"] == "True": 77 | self.CpuExtension = True 78 | 79 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 80 | 81 | if data[self.ModelType]["dynamic_batch"] == "True": 82 | self.DynamicBatch = True 83 | 84 | if data[self.ModelType]["limit_cpu_threads"] == "True": 85 | self.LimitCPUThreads = True 86 | 87 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 88 | 89 | if data[self.ModelType]["bind_cpu_threads"] == "True": 90 | self.LimitCPUThreads = True 91 | 92 | self.CPUStream = data[self.ModelType]["cpu_stream"] 93 | 94 | except FileNotFoundError: 95 | logging.log(logging.ERROR, '{} FileNotFound'.format(json_file)) 96 | exit(-1) 97 | 98 | def read_dict(self, data=None): 99 | """ 100 | Used When JSON Already Parsed as Dict 101 | :return: 102 | """ 103 | if data is None: 104 | logging.log(logging.ERROR, "No Parameters Passed") 105 | exit(-1) 106 | 107 | self.ModelPath = data[self.ModelType]["model_path"] 108 | self.ModelName = data[self.ModelType]["model_name"] 109 | 110 | self.TargetDevice = data[self.ModelType]["target_device"] 111 | 112 | self.FaceDetectionThreshold = data[self.ModelType]["face_detection_threshold"] 113 | 114 | if data[self.ModelType]["async"] == "True": 115 | self.Async = True 116 | 117 | self.RequestCount = int(data[self.ModelType]["request_count"]) 118 | 119 | self.BatchSize = int(data[self.ModelType]["batch_size"]) 120 | 121 | if data[self.ModelType]["cpu_extension"] == "True": 122 | self.CpuExtension = True 123 | 124 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 125 | 126 | if data[self.ModelType]["dynamic_batch"] == "True": 127 | self.DynamicBatch = True 128 | 129 | if data[self.ModelType]["limit_cpu_threads"] == "True": 130 | self.LimitCPUThreads = True 131 | 132 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 133 | 134 | if data[self.ModelType]["bind_cpu_threads"] == "True": 135 | self.LimitCPUThreads = True 136 | 137 | self.CPUStream = data[self.ModelType]["cpu_stream"] 138 | 139 | 140 | class OpenMZooFaceDetection(InferenceBase): 141 | """ 142 | Face Detection Module Configured to Get Results using Open Model Zoo Face Detection Model 143 | """ 144 | Config = FaceDetectionConfig() 145 | 146 | def __init__(self, config=FaceDetectionConfig()): 147 | super(OpenMZooFaceDetection, self).__init__(config) 148 | self.Config = config 149 | 150 | def get_face_detection_data(self, request_id=0): 151 | """ 152 | Parse Face Detection Output 153 | :param output_layer: 154 | :param request_id: 155 | :return: face coordinates 156 | """ 157 | face_coordinates = [] 158 | 159 | detections = self.get_results(self.OutputLayer, request_id)[0][0] 160 | 161 | logging.log(logging.INFO, "Fetched Face Detection Results") 162 | 163 | for detection in detections: 164 | if detection[2] > self.Config.FaceDetectionThreshold: 165 | face_coordinates.append([detection[3], detection[4], detection[5], detection[6]]) 166 | 167 | logging.log(logging.INFO, "Number of Detected Faces: {}".format(len(face_coordinates))) 168 | return face_coordinates 169 | 170 | 171 | class MTCNNFaceDetectionConfig(InferenceConfig): 172 | """ 173 | Face Detection Module Configurations based on MTCNN Face Detection Model 174 | """ 175 | ModelType = FaceDetectionModelTypes.MTCNN 176 | 177 | InputHeight = 720 178 | InputWidth = 1080 179 | 180 | PNetworkThreshold = 0.6 181 | RNetworkThreshold = 0.7 182 | ONetworkThreshold = 0.8 183 | 184 | NMSThresholds = [0.7, 0.7, 0.7] 185 | MinDetectionSize = 12 186 | Factor = 0.707 187 | 188 | MinimumFaceSize = 15.0 189 | MinLength = 720 190 | FactorCount = 0 191 | 192 | RInputBatchSize = 128 193 | OInputBatchSize = 128 194 | 195 | PModelFileName = "det1-0001" 196 | RModelFileName = "det2-0001" 197 | OModelFileName = "det3-0001" 198 | 199 | def parse_json(self, json_file): 200 | try: 201 | logging.log(logging.INFO, "Loading JSON File {}".format(json_file)) 202 | logging.log(logging.INFO, "Model Type {}".format(self.ModelType)) 203 | with open(json_file) as json_file: 204 | data = json.load(json_file) 205 | 206 | self.ModelPath = data[self.ModelType]["model_path"] 207 | self.PModelFileName = data[self.ModelType]["p_model_file_name"] 208 | self.RModelFileName = data[self.ModelType]["r_model_file_name"] 209 | self.OModelFileName = data[self.ModelType]["o_model_file_name"] 210 | 211 | self.TargetDevice = data[self.ModelType]["target_device"] 212 | 213 | if data[self.ModelType]["cpu_extension"] == "True": 214 | self.CpuExtension = True 215 | 216 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 217 | 218 | self.PNetworkThreshold = float(data[self.ModelType]["p_network_threshold"]) 219 | self.RNetworkThreshold = float(data[self.ModelType]["r_network_threshold"]) 220 | self.ONetworkThreshold = float(data[self.ModelType]["o_network_threshold"]) 221 | 222 | self.MinimumFaceSize = float(data[self.ModelType]["minimum_face_size"]) 223 | self.MinLength = float(data[self.ModelType]["minimum_length"]) 224 | self.FactorCount = float(data[self.ModelType]["factor_count"]) 225 | self.Factor = float(data[self.ModelType]["factor"]) 226 | self.MinDetectionSize = int(data[self.ModelType]["min_detection_size"]) 227 | 228 | self.NMSThresholds = list(data[self.ModelType]["nms_thresholds"]) 229 | 230 | self.RInputBatchSize = int(data[self.ModelType]["r_input_batch_size"]) 231 | self.OInputBatchSize = int(data[self.ModelType]["o_input_batch_size"]) 232 | 233 | if data[self.ModelType]["limit_cpu_threads"] == "True": 234 | self.LimitCPUThreads = True 235 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 236 | if data[self.ModelType]["bind_cpu_threads"] == "True": 237 | self.LimitCPUThreads = True 238 | self.CPUStream = data[self.ModelType]["cpu_stream"] 239 | 240 | except FileNotFoundError: 241 | logging.log(logging.ERROR, '{} FileNotFound'.format(json_file)) 242 | exit(-1) 243 | 244 | def read_dict(self, data=None): 245 | """ 246 | Used When JSON Already Parsed as Dict 247 | :return: 248 | """ 249 | if data is None: 250 | logging.log(logging.ERROR, "No Parameters Passed") 251 | exit(-1) 252 | 253 | self.ModelPath = data[self.ModelType]["model_path"] 254 | self.PModelFileName = data[self.ModelType]["p_model_file_name"] 255 | self.RModelFileName = data[self.ModelType]["r_model_file_name"] 256 | self.OModelFileName = data[self.ModelType]["o_model_file_name"] 257 | 258 | self.TargetDevice = data[self.ModelType]["target_device"] 259 | 260 | if data[self.ModelType]["cpu_extension"] == "True": 261 | self.CpuExtension = True 262 | 263 | self.CpuExtensionPath = data[self.ModelType]["cpu_extension_path"] 264 | 265 | self.PNetworkThreshold = float(data[self.ModelType]["p_network_threshold"]) 266 | self.RNetworkThreshold = float(data[self.ModelType]["r_network_threshold"]) 267 | self.ONetworkThreshold = float(data[self.ModelType]["o_network_threshold"]) 268 | 269 | self.MinimumFaceSize = float(data[self.ModelType]["minimum_face_size"]) 270 | self.MinLength = float(data[self.ModelType]["minimum_length"]) 271 | self.FactorCount = float(data[self.ModelType]["factor_count"]) 272 | self.Factor = float(data[self.ModelType]["factor"]) 273 | self.MinDetectionSize = int(data[self.ModelType]["min_detection_size"]) 274 | 275 | self.NMSThresholds = list(data[self.ModelType]["nms_thresholds"]) 276 | 277 | self.RInputBatchSize = int(data[self.ModelType]["r_input_batch_size"]) 278 | self.OInputBatchSize = int(data[self.ModelType]["o_input_batch_size"]) 279 | 280 | if data[self.ModelType]["limit_cpu_threads"] == "True": 281 | self.LimitCPUThreads = True 282 | self.CPUThreadNum = int(data[self.ModelType]["number_of_cpu_threads"]) 283 | if data[self.ModelType]["bind_cpu_threads"] == "True": 284 | self.LimitCPUThreads = True 285 | self.CPUStream = data[self.ModelType]["cpu_stream"] 286 | 287 | 288 | class MtCNNFaceDetection(InferenceBase): 289 | 290 | Config = MTCNNFaceDetectionConfig() 291 | 292 | OpenVinoExecutablesP = list() 293 | OpenVinoExecutableR = ExecutableNetwork() 294 | OpenVinoExecutableO = ExecutableNetwork() 295 | 296 | OpenVinoNetworkP = IENetwork() 297 | OpenVinoNetworkR = IENetwork() 298 | OpenVinoNetworkO = IENetwork() 299 | 300 | Scales = [] 301 | 302 | RINPUT = [] 303 | OINPUT = [] 304 | 305 | LastFaceDetections = [] 306 | LastLandmarkDetections = [] 307 | 308 | InputLayerP = str() 309 | InputLayerR = str() 310 | InputLayerO = str() 311 | 312 | OutputLayersP = list() 313 | OutputLayersR = list() 314 | OutputLayersO = list() 315 | 316 | InputShapeP = [] 317 | InputShapeR = [] 318 | InputShapeO = [] 319 | 320 | def __init__(self, config=MTCNNFaceDetectionConfig()): 321 | super(MtCNNFaceDetection, self).__init__(config) 322 | self.Config = config 323 | 324 | def prepare_detector(self): 325 | """ 326 | Override Base Class Since MTCNN works with three different model 327 | :return: None 328 | """ 329 | 330 | if self.Config.ModelPath is None or self.Config.ModelName is None: 331 | return None 332 | 333 | logging.log(logging.INFO, "Setting Up R - O Network Input Storage") 334 | self.RINPUT = np.zeros(dtype=float, shape=(self.Config.RInputBatchSize, 3, 24, 24)) 335 | self.OINPUT = np.zeros(dtype=float, shape=(self.Config.OInputBatchSize, 3, 48, 48)) 336 | 337 | self.OpenVinoIE = IECore() 338 | 339 | if self.Config.CpuExtension and 'CPU' in self.Config.TargetDevice: 340 | logging.log(logging.INFO, "CPU Extensions Added") 341 | self.OpenVinoIE.add_extension(self.Config.CpuExtensionPath, "CPU") 342 | 343 | try: 344 | # Model File Paths 345 | model_file = self.Config.ModelPath + self.Config.PModelFileName + ".xml" 346 | model_weights = self.Config.ModelPath + self.Config.PModelFileName + ".bin" 347 | logging.log(logging.INFO, "Loading Models File {}".format(model_file)) 348 | logging.log(logging.INFO, "Loading Weights File {}".format(model_weights)) 349 | 350 | self.OpenVinoNetworkP = IENetwork(model=model_file, weights=model_weights) 351 | logging.log(logging.INFO, "Loading P Network") 352 | 353 | model_file = self.Config.ModelPath + self.Config.RModelFileName + ".xml" 354 | model_weights = self.Config.ModelPath + self.Config.RModelFileName + ".bin" 355 | logging.log(logging.INFO, "Loading Models File {}".format(model_file)) 356 | logging.log(logging.INFO, "Loading Weights File {}".format(model_weights)) 357 | 358 | self.OpenVinoNetworkR = IENetwork(model=model_file, weights=model_weights) 359 | self.OpenVinoNetworkR.batch_size = self.Config.RInputBatchSize 360 | logging.log(logging.INFO, "Loading R Network") 361 | 362 | model_file = self.Config.ModelPath + self.Config.OModelFileName + ".xml" 363 | model_weights = self.Config.ModelPath + self.Config.OModelFileName + ".bin" 364 | logging.log(logging.INFO, "Loading Models File {}".format(model_file)) 365 | logging.log(logging.INFO, "Loading Weights File {}".format(model_weights)) 366 | 367 | self.OpenVinoNetworkO = IENetwork(model=model_file, weights=model_weights) 368 | self.OpenVinoNetworkO.batch_size = self.Config.OInputBatchSize 369 | logging.log(logging.INFO, "Loading O Network") 370 | 371 | except FileNotFoundError: 372 | logging.log(logging.ERROR, FileNotFoundError.strerror, " ", FileNotFoundError.filename) 373 | exit(-1) 374 | 375 | if "CPU" in self.Config.TargetDevice: 376 | supported_layers = self.OpenVinoIE.query_network(self.OpenVinoNetworkP, "CPU") 377 | not_supported_layers = [l for l in self.OpenVinoNetworkP.layers.keys() if l not in supported_layers] 378 | if len(not_supported_layers) != 0: 379 | logging.log(logging.INFO, "Following layers are not supported by the plugin for specified device {}:\n {}". 380 | format(self.Config.TargetDevice, ', '.join(not_supported_layers))) 381 | logging.log(logging.INFO, "Please try to specify cpu extensions library path in config.json file ") 382 | 383 | # Input / Output Memory Allocations to feed input or get output values 384 | self.InputLayerP = next(iter(self.OpenVinoNetworkP.inputs)) 385 | self.InputLayerR = next(iter(self.OpenVinoNetworkP.inputs)) 386 | self.InputLayerO = next(iter(self.OpenVinoNetworkP.inputs)) 387 | 388 | self.OutputLayersP = list(self.OpenVinoNetworkP.outputs) 389 | self.OutputLayersR = list(self.OpenVinoNetworkR.outputs) 390 | self.OutputLayersO = list(self.OpenVinoNetworkO.outputs) 391 | 392 | self.InputShapeP = self.OpenVinoNetworkP.inputs[self.InputLayerP].shape 393 | self.InputShapeR = self.OpenVinoNetworkR.inputs[self.InputLayerR].shape 394 | self.InputShapeO = self.OpenVinoNetworkO.inputs[self.InputLayerO].shape 395 | 396 | # Enable Dynamic Batch By Default 397 | config = {"DYN_BATCH_ENABLED": "YES"} 398 | 399 | self.OpenVinoExecutableR = self.OpenVinoIE.load_network(network=self.OpenVinoNetworkR, 400 | device_name=self.Config.TargetDevice, 401 | config=config, 402 | num_requests=self.Config.RequestCount) 403 | logging.log(logging.INFO, "Created R Network Executable") 404 | 405 | self.OpenVinoExecutableO = self.OpenVinoIE.load_network(network=self.OpenVinoNetworkO, 406 | device_name=self.Config.TargetDevice, 407 | config=config, 408 | num_requests=self.Config.RequestCount) 409 | logging.log(logging.INFO, "Created O Network Executable") 410 | 411 | self.Config.MinLength = min(self.Config.InputHeight, self.Config.InputWidth) 412 | M = self.Config.MinDetectionSize / self.Config.MinimumFaceSize 413 | self.Config.MinLength *= M 414 | 415 | while self.Config.MinLength > self.Config.MinDetectionSize: 416 | scale = (M*self.Config.Factor**self.Config.FactorCount) 417 | self.Scales.append(scale) 418 | self.Config.MinLength *= self.Config.Factor 419 | self.Config.FactorCount += 1 420 | 421 | sw, sh = math.ceil(self.Config.InputWidth * scale), math.ceil(self.Config.InputHeight * scale) 422 | 423 | self.OpenVinoNetworkP.reshape({self.InputLayerP: (1, 3, sh, sw)}) 424 | 425 | self.OpenVinoExecutablesP.append(self.OpenVinoIE.load_network(network=self.OpenVinoNetworkP, 426 | device_name=self.Config.TargetDevice, 427 | num_requests=self.Config.RequestCount)) 428 | 429 | logging.log(logging.INFO, "Created Scaled P Networks {}".format(len(self.OpenVinoExecutablesP))) 430 | 431 | def run_mtcnn_face_detection(self, images, request_id=0): 432 | """ 433 | Get Detected Face Coordinates 434 | :param images: 435 | :param request_id: 436 | :return: 437 | """ 438 | self.InferenceCount += 1 439 | start_time = time.time() 440 | bounding_boxes = [] 441 | landmarks = [] 442 | 443 | cv_img = cv.cvtColor(images, cv.COLOR_BGR2RGB) 444 | image = Image.fromarray(cv_img) 445 | 446 | none_count = 0 447 | 448 | for i, scale in enumerate(self.Scales): 449 | width, height = image.size 450 | sw, sh = math.ceil(width * scale), math.ceil(height * scale) 451 | img = image.resize((sw, sh), Image.BILINEAR) 452 | img = np.asarray(img, 'float32') 453 | img = self.preprocess(img) 454 | 455 | output = self.OpenVinoExecutablesP[i].infer({self.InputLayerP: img}) 456 | 457 | probs = output["prob1"][0, 1, :, :] 458 | offsets = output["conv4_2"] 459 | 460 | boxes = self.generate_bboxes(probs, offsets, scale, self.Config.PNetworkThreshold) 461 | 462 | if len(boxes) == 0: 463 | bounding_boxes.append(None) 464 | none_count += 1 465 | else: 466 | keep = self.nms(boxes[:, 0:5], overlap_threshold=0.5) 467 | bounding_boxes.append(boxes[keep]) 468 | 469 | if len(bounding_boxes) > none_count: 470 | bounding_boxes = [i for i in bounding_boxes if i is not None] 471 | bounding_boxes = np.vstack(bounding_boxes) 472 | keep = self.nms(bounding_boxes[:, 0:5], self.Config.NMSThresholds[0]) 473 | bounding_boxes = bounding_boxes[keep] 474 | bounding_boxes = self.calibrate_box(bounding_boxes[:, 0:5], bounding_boxes[:, 5:]) 475 | bounding_boxes = self.convert_to_square(bounding_boxes) 476 | bounding_boxes[:, 0:4] = np.round(bounding_boxes[:, 0:4]) 477 | 478 | img_boxes = self.get_image_boxes(bounding_boxes, image, size=24) 479 | 480 | if img_boxes.shape[0] > 0: 481 | shp = img_boxes.shape 482 | self.RINPUT[0:shp[0], ] = img_boxes 483 | self.OpenVinoExecutableR.requests[request_id].set_batch(shp[0]) 484 | self.OpenVinoExecutableR.requests[request_id].infer({self.InputLayerR: self.RINPUT}) 485 | 486 | offsets = self.OpenVinoExecutableR.requests[0].outputs['conv5_2'][:shp[0], ] 487 | probs = self.OpenVinoExecutableR.requests[0].outputs['prob1'][:shp[0]] 488 | 489 | keep = np.where(probs[:, 1] > self.Config.RNetworkThreshold)[0] 490 | bounding_boxes = bounding_boxes[keep] 491 | bounding_boxes[:, 4] = probs[keep, 1].reshape((-1,)) 492 | offsets = offsets[keep] 493 | keep = self.nms(bounding_boxes, self.Config.NMSThresholds[1]) 494 | bounding_boxes = bounding_boxes[keep] 495 | bounding_boxes = self.calibrate_box(bounding_boxes, offsets[keep]) 496 | bounding_boxes = self.convert_to_square(bounding_boxes) 497 | bounding_boxes[:, 0:4] = np.round(bounding_boxes[:, 0:4]) 498 | img_boxes = self.get_image_boxes(bounding_boxes, image, size=48) 499 | 500 | if img_boxes.shape[0] > 0: 501 | shp = img_boxes.shape 502 | self.OINPUT[0:shp[0], ] = img_boxes 503 | 504 | self.OpenVinoExecutableO.requests[0].set_batch(shp[0]) 505 | self.OpenVinoExecutableO.requests[0].infer({self.InputLayerO: self.OINPUT}) 506 | 507 | landmarks = self.OpenVinoExecutableO.requests[0].outputs['conv6_3'][:shp[0]] 508 | offsets = self.OpenVinoExecutableO.requests[0].outputs['conv6_2'][:shp[0]] 509 | probs = self.OpenVinoExecutableO.requests[0].outputs['prob1'][:shp[0]] 510 | 511 | keep = np.where(probs[:, 1] > self.Config.ONetworkThreshold)[0] 512 | bounding_boxes = bounding_boxes[keep] 513 | bounding_boxes[:, 4] = probs[keep, 1].reshape((-1,)) 514 | offsets = offsets[keep] 515 | landmarks = landmarks[keep] 516 | # compute landmark points 517 | width = bounding_boxes[:, 2] - bounding_boxes[:, 0] + 1.0 518 | height = bounding_boxes[:, 3] - bounding_boxes[:, 1] + 1.0 519 | xmin, ymin = bounding_boxes[:, 0], bounding_boxes[:, 1] 520 | landmarks[:, 0:5] = np.expand_dims(xmin, 1) + np.expand_dims(width, 1) * landmarks[:, 0:5] 521 | landmarks[:, 5:10] = np.expand_dims(ymin, 1) + np.expand_dims(height, 1) * landmarks[:, 5:10] 522 | bounding_boxes = self.calibrate_box(bounding_boxes, offsets) 523 | keep = self.nms(bounding_boxes, self.Config.NMSThresholds[2], mode='min') 524 | bounding_boxes = bounding_boxes[keep] 525 | landmarks = landmarks[keep] 526 | 527 | none_count = 0 528 | 529 | face_detections = [] 530 | landmark_detections = [] 531 | i = 0 532 | for box in bounding_boxes: 533 | if type(box) is type(None): 534 | none_count += 1 535 | else: 536 | scale = box[4] 537 | xmin = float((box[0] / scale) / self.Config.InputWidth) 538 | ymin = float((box[1] / scale) / self.Config.InputHeight) 539 | xmax = float((box[2] / scale) / self.Config.InputWidth) 540 | ymax = float((box[3] / scale) / self.Config.InputHeight) 541 | face_detections.append([xmin, ymin, xmax, ymax]) 542 | lands = [] 543 | for l in range(5): 544 | lands.append(float((landmarks[i][l] / scale) / self.Config.InputWidth)) 545 | lands.append(float((landmarks[i][l + 5] / scale) / self.Config.InputHeight)) 546 | 547 | landmark_detections.append(lands) 548 | i += 1 549 | 550 | if none_count == len(bounding_boxes): 551 | return [], [] 552 | 553 | self.LastFaceDetections = face_detections 554 | self.LastLandmarkDetections = landmark_detections 555 | 556 | self.ElapsedInferenceTime += (time.time() - start_time) 557 | 558 | def infer(self, images, request_id=0): 559 | """ 560 | Run inference 561 | :param images: image to get faces 562 | :param request_id: request id 563 | :return: 564 | """ 565 | self.run_mtcnn_face_detection(images, request_id=0) 566 | 567 | def request_ready(self, request_id): 568 | """ 569 | This is true by default since there is no ASYNC mode for MTCNN 570 | :param request_id: 571 | :return: 572 | """ 573 | return True 574 | 575 | def get_face_detection_data(self, request_id=0): 576 | """ 577 | Get Latest Results for Face Coordinates 578 | :param request_id: 579 | :return: 580 | """ 581 | return self.LastFaceDetections 582 | 583 | def get_face_landmarks_data(self, request_id=0): 584 | """ 585 | Get Latest Results for Landmark Coordinates 586 | :param request_id: 587 | :return: 588 | """ 589 | return self.LastLandmarkDetections 590 | 591 | @staticmethod 592 | def preprocess(img): 593 | """Preprocessing step before feeding the network. 594 | 595 | Arguments: 596 | img: a float numpy array of shape [h, w, c]. 597 | 598 | Returns: 599 | a float numpy array of shape [1, c, h, w]. 600 | """ 601 | img = img.transpose((2, 0, 1)) 602 | img = np.expand_dims(img, 0) 603 | img = (img - 127.5) * 0.0078125 604 | return img 605 | 606 | @staticmethod 607 | def generate_bboxes(probs, offsets, scale, threshold): 608 | """Generate bounding boxes at places 609 | where there is probably a face. 610 | 611 | Arguments: 612 | probs: a float numpy array of shape [n, m]. 613 | offsets: a float numpy array of shape [1, 4, n, m]. 614 | scale: a float number, 615 | width and height of the image were scaled by this number. 616 | threshold: a float number. 617 | 618 | Returns: 619 | a float numpy array of shape [n_boxes, 9] 620 | """ 621 | 622 | # applying P-Net is equivalent, in some sense, to 623 | # moving 12x12 window with stride 2 624 | stride = 2 625 | cell_size = 12 626 | 627 | # indices of boxes where there is probably a face 628 | inds = np.where(probs > threshold) 629 | 630 | if inds[0].size == 0: 631 | return np.array([]) 632 | 633 | # transformations of bounding boxes 634 | tx1, ty1, tx2, ty2 = [offsets[0, i, inds[0], inds[1]] for i in range(4)] 635 | # they are defined as: 636 | # w = x2 - x1 + 1 637 | # h = y2 - y1 + 1 638 | # x1_true = x1 + tx1*w 639 | # x2_true = x2 + tx2*w 640 | # y1_true = y1 + ty1*h 641 | # y2_true = y2 + ty2*h 642 | 643 | offsets = np.array([tx1, ty1, tx2, ty2]) 644 | score = probs[inds[0], inds[1]] 645 | 646 | # P-Net is applied to scaled images 647 | # so we need to rescale bounding boxes back 648 | bounding_boxes = np.vstack([ 649 | np.round((stride * inds[1] + 1.0) / scale), 650 | np.round((stride * inds[0] + 1.0) / scale), 651 | np.round((stride * inds[1] + 1.0 + cell_size) / scale), 652 | np.round((stride * inds[0] + 1.0 + cell_size) / scale), 653 | score, offsets 654 | ]) 655 | # why one is added? 656 | 657 | return bounding_boxes.T 658 | 659 | @staticmethod 660 | def nms(boxes, overlap_threshold=0.5, mode='union'): 661 | """Non-maximum suppression. 662 | 663 | Arguments: 664 | boxes: a float numpy array of shape [n, 5], 665 | where each row is (xmin, ymin, xmax, ymax, score). 666 | overlap_threshold: a float number. 667 | mode: 'union' or 'min'. 668 | 669 | Returns: 670 | list with indices of the selected boxes 671 | """ 672 | 673 | # if there are no boxes, return the empty list 674 | if len(boxes) == 0: 675 | return [] 676 | 677 | # list of picked indices 678 | pick = [] 679 | 680 | # grab the coordinates of the bounding boxes 681 | x1, y1, x2, y2, score = [boxes[:, i] for i in range(5)] 682 | 683 | area = (x2 - x1 + 1.0) * (y2 - y1 + 1.0) 684 | ids = np.argsort(score) # in increasing order 685 | 686 | while len(ids) > 0: 687 | 688 | # grab index of the largest value 689 | last = len(ids) - 1 690 | i = ids[last] 691 | pick.append(i) 692 | 693 | # compute intersections 694 | # of the box with the largest score 695 | # with the rest of boxes 696 | 697 | # left top corner of intersection boxes 698 | ix1 = np.maximum(x1[i], x1[ids[:last]]) 699 | iy1 = np.maximum(y1[i], y1[ids[:last]]) 700 | 701 | # right bottom corner of intersection boxes 702 | ix2 = np.minimum(x2[i], x2[ids[:last]]) 703 | iy2 = np.minimum(y2[i], y2[ids[:last]]) 704 | 705 | # width and height of intersection boxes 706 | w = np.maximum(0.0, ix2 - ix1 + 1.0) 707 | h = np.maximum(0.0, iy2 - iy1 + 1.0) 708 | 709 | # intersections' areas 710 | inter = w * h 711 | if mode == 'min': 712 | overlap = inter / np.minimum(area[i], area[ids[:last]]) 713 | elif mode == 'union': 714 | # intersection over union (IoU) 715 | overlap = inter / (area[i] + area[ids[:last]] - inter) 716 | 717 | # delete all boxes where overlap is too big 718 | ids = np.delete( 719 | ids, 720 | np.concatenate([[last], np.where(overlap > overlap_threshold)[0]]) 721 | ) 722 | 723 | return pick 724 | 725 | @staticmethod 726 | def calibrate_box(bboxes, offsets): 727 | """Transform bounding boxes to be more like true bounding boxes. 728 | 'offsets' is one of the outputs of the nets. 729 | 730 | Arguments: 731 | bboxes: a float numpy array of shape [n, 5]. 732 | offsets: a float numpy array of shape [n, 4]. 733 | 734 | Returns: 735 | a float numpy array of shape [n, 5]. 736 | """ 737 | x1, y1, x2, y2 = [bboxes[:, i] for i in range(4)] 738 | w = x2 - x1 + 1.0 739 | h = y2 - y1 + 1.0 740 | w = np.expand_dims(w, 1) 741 | h = np.expand_dims(h, 1) 742 | 743 | # this is what happening here: 744 | # tx1, ty1, tx2, ty2 = [offsets[:, i] for i in range(4)] 745 | # x1_true = x1 + tx1*w 746 | # y1_true = y1 + ty1*h 747 | # x2_true = x2 + tx2*w 748 | # y2_true = y2 + ty2*h 749 | # below is just more compact form of this 750 | 751 | # are offsets always such that 752 | # x1 < x2 and y1 < y2 ? 753 | 754 | translation = np.hstack([w, h, w, h]) * offsets 755 | bboxes[:, 0:4] = bboxes[:, 0:4] + translation 756 | return bboxes 757 | 758 | @staticmethod 759 | def convert_to_square(bboxes): 760 | """Convert bounding boxes to a square form. 761 | 762 | Arguments: 763 | bboxes: a float numpy array of shape [n, 5]. 764 | 765 | Returns: 766 | a float numpy array of shape [n, 5], 767 | squared bounding boxes. 768 | """ 769 | 770 | square_bboxes = np.zeros_like(bboxes) 771 | x1, y1, x2, y2 = [bboxes[:, i] for i in range(4)] 772 | h = y2 - y1 + 1.0 773 | w = x2 - x1 + 1.0 774 | max_side = np.maximum(h, w) 775 | square_bboxes[:, 0] = x1 + w * 0.5 - max_side * 0.5 776 | square_bboxes[:, 1] = y1 + h * 0.5 - max_side * 0.5 777 | square_bboxes[:, 2] = square_bboxes[:, 0] + max_side - 1.0 778 | square_bboxes[:, 3] = square_bboxes[:, 1] + max_side - 1.0 779 | return square_bboxes 780 | 781 | @staticmethod 782 | def correct_bboxes(bboxes, width, height): 783 | """Crop boxes that are too big and get coordinates 784 | with respect to cutouts. 785 | 786 | Arguments: 787 | bboxes: a float numpy array of shape [n, 5], 788 | where each row is (xmin, ymin, xmax, ymax, score). 789 | width: a float number. 790 | height: a float number. 791 | 792 | Returns: 793 | dy, dx, edy, edx: a int numpy arrays of shape [n], 794 | coordinates of the boxes with respect to the cutouts. 795 | y, x, ey, ex: a int numpy arrays of shape [n], 796 | corrected ymin, xmin, ymax, xmax. 797 | h, w: a int numpy arrays of shape [n], 798 | just heights and widths of boxes. 799 | 800 | in the following order: 801 | [dy, edy, dx, edx, y, ey, x, ex, w, h]. 802 | """ 803 | 804 | x1, y1, x2, y2 = [bboxes[:, i] for i in range(4)] 805 | w, h = x2 - x1 + 1.0, y2 - y1 + 1.0 806 | num_boxes = bboxes.shape[0] 807 | 808 | # 'e' stands for end 809 | # (x, y) -> (ex, ey) 810 | x, y, ex, ey = x1, y1, x2, y2 811 | 812 | # we need to cut out a box from the image. 813 | # (x, y, ex, ey) are corrected coordinates of the box 814 | # in the image. 815 | # (dx, dy, edx, edy) are coordinates of the box in the cutout 816 | # from the image. 817 | dx, dy = np.zeros((num_boxes,)), np.zeros((num_boxes,)) 818 | edx, edy = w.copy() - 1.0, h.copy() - 1.0 819 | 820 | # if box's bottom right corner is too far right 821 | ind = np.where(ex > width - 1.0)[0] 822 | edx[ind] = w[ind] + width - 2.0 - ex[ind] 823 | ex[ind] = width - 1.0 824 | 825 | # if box's bottom right corner is too low 826 | ind = np.where(ey > height - 1.0)[0] 827 | edy[ind] = h[ind] + height - 2.0 - ey[ind] 828 | ey[ind] = height - 1.0 829 | 830 | # if box's top left corner is too far left 831 | ind = np.where(x < 0.0)[0] 832 | dx[ind] = 0.0 - x[ind] 833 | x[ind] = 0.0 834 | 835 | # if box's top left corner is too high 836 | ind = np.where(y < 0.0)[0] 837 | dy[ind] = 0.0 - y[ind] 838 | y[ind] = 0.0 839 | 840 | return_list = [dy, edy, dx, edx, y, ey, x, ex, w, h] 841 | return_list = [i.astype('int32') for i in return_list] 842 | 843 | return return_list 844 | 845 | @staticmethod 846 | def get_image_boxes(bounding_boxes, img, size=24): 847 | """Cut out boxes from the image. 848 | 849 | Arguments: 850 | bounding_boxes: a float numpy array of shape [n, 5]. 851 | img: an instance of PIL.Image. 852 | size: an integer, size of cutouts. 853 | 854 | Returns: 855 | a float numpy array of shape [n, 3, size, size]. 856 | """ 857 | 858 | num_boxes = len(bounding_boxes) 859 | width, height = img.size 860 | 861 | [dy, edy, dx, edx, y, ey, x, ex, w, h] = MtCNNFaceDetection.correct_bboxes(bounding_boxes, width, height) 862 | img_boxes = np.zeros((num_boxes, 3, size, size), 'float32') 863 | 864 | for i in range(num_boxes): 865 | if h[i] <= 0 or w[i] <= 0: 866 | continue 867 | img_box = np.zeros((h[i], w[i], 3), 'uint8') 868 | 869 | img_array = np.asarray(img, 'uint8') 870 | img_box[dy[i]:(edy[i] + 1), dx[i]:(edx[i] + 1), :] = \ 871 | img_array[y[i]:(ey[i] + 1), x[i]:(ex[i] + 1), :] 872 | 873 | # resize 874 | img_box = Image.fromarray(img_box) 875 | img_box = img_box.resize((size, size), Image.BILINEAR) 876 | img_box = np.asarray(img_box, 'float32') 877 | 878 | img_boxes[i, :, :, :] = MtCNNFaceDetection.preprocess(img_box) 879 | 880 | return img_boxes -------------------------------------------------------------------------------- /inference_services/facedetection/face_detection_service.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | from flask import Flask, request, jsonify, make_response, redirect 24 | import logging 25 | import sys 26 | import optparse 27 | import time 28 | import cv2 as cv 29 | import asyncio 30 | import threading 31 | 32 | from detection.face_detection_ov import FaceDetectionConfig, OpenMZooFaceDetection, FaceDetectionModelTypes, MtCNNFaceDetection, MTCNNFaceDetectionConfig 33 | from detection.age_gender_detection_ov import AgeGenderConfig, MTCNNAgeGenderDetection, AgeGenderDetectionTypes, MTCNNAgeGenderConfig, AgeGenderDetection 34 | from utils.image_utils import ImageUtil 35 | 36 | app = Flask(__name__) 37 | 38 | start = int(round(time.time())) 39 | 40 | loop = asyncio.get_event_loop() 41 | 42 | thread = threading.Thread() 43 | 44 | class AppStatus: 45 | STARTED = "STARTED" 46 | FINISHED = "FINISHED" 47 | NOTSTARTED = "NOTSTARTED" 48 | STOPREQUEST = "STOPREQUESTED" 49 | 50 | 51 | def prepare_configs(): 52 | """ 53 | Set Configurations for Face, Age Gender Models 54 | :return: face config, age_gender config 55 | """ 56 | logging.getLogger(name="inference").log(logging.INFO, "Setting Configurations") 57 | 58 | if face_detection_model == FaceDetectionModelTypes.MTCNN: 59 | face_infer_cfg = MTCNNFaceDetectionConfig() 60 | else: 61 | face_infer_cfg = FaceDetectionConfig() 62 | 63 | face_infer_cfg.read_dict(json_req) 64 | 65 | logging.getLogger(name="inference").log(logging.INFO, "Configuration Set Completed...") 66 | 67 | return face_infer_cfg 68 | 69 | 70 | async def inference(): 71 | if inference_status == AppStatus.FINISHED or inference_status == AppStatus.NOTSTARTED: 72 | run_inference() 73 | else: 74 | logging.log(logging.WARN, "Inference Already Running ... ") 75 | loop.stop() 76 | return "OK" 77 | 78 | 79 | def run_inference(): 80 | """ 81 | Runs Face Detection Application with the Requested JSON 82 | :return: 83 | """ 84 | 85 | face_cfg = prepare_configs() 86 | 87 | if input_type == "video": 88 | logging.log(logging.INFO, "Video File Input Selected") 89 | capture = cv.VideoCapture(input_path) 90 | has_frame, frame = capture.read() 91 | elif input_type == "webcam": 92 | logging.log(logging.INFO, "Webcam Video Selected") 93 | capture = cv.VideoCapture(web_cam_index) 94 | has_frame, frame = capture.read() 95 | elif input_type == "image": 96 | logging.log(logging.INFO, "Single Image Inference Selected") 97 | frame = cv.imread(input_path) 98 | else: 99 | logging.log(logging.ERROR, "Invalid Input Type: {}".format(input_type)) 100 | exit(-1) 101 | 102 | face_cfg.InputHeight = frame.shape[0] 103 | face_cfg.InputWidth = frame.shape[1] 104 | 105 | logging.getLogger(name="inference").log(logging.INFO, "Input Frame H: {} W: {}".format(face_cfg.InputHeight, face_cfg.InputWidth)) 106 | 107 | if face_detection_model == FaceDetectionModelTypes.MTCNN: 108 | face_infer = MtCNNFaceDetection(face_cfg) 109 | else: 110 | face_infer = OpenMZooFaceDetection(face_cfg) 111 | 112 | face_request_order = list() 113 | face_process_order = list() 114 | 115 | for i in range(face_infer.Config.RequestCount): 116 | face_request_order.append(i) 117 | 118 | frame_order = [] 119 | frame_id = 1 120 | 121 | global inference_status 122 | inference_status = AppStatus.STARTED 123 | 124 | if save_roi_text: 125 | roi_file = open(output_dir + roi_text_filename, 'w') 126 | roi = "{};{};{};{};{}\n".format("frameid","xmin","ymin","xmax","ymax") 127 | roi_file.write(roi) 128 | 129 | if save_roi_video: 130 | fourcc = cv.VideoWriter_fourcc('X', '2', '6', '4') 131 | roi_video = cv.VideoWriter(output_dir + roi_video_filename, fourcc, 10, (face_cfg.InputWidth, face_cfg.InputHeight )) 132 | 133 | if input_type == "video" or input_type == "webcam": 134 | while has_frame: 135 | 136 | if inference_status == AppStatus.STOPREQUEST: 137 | break 138 | 139 | logging.log(logging.DEBUG, "Processing Frame {}".format(frame_id)) 140 | if len(face_request_order) > 0: 141 | req_id = face_request_order[0] 142 | face_request_order.pop(0) 143 | face_infer.infer(frame, req_id) 144 | face_process_order.append(req_id) 145 | frame_order.append(frame) 146 | 147 | if len(face_process_order) > 0: 148 | first = face_process_order[0] 149 | if face_infer.request_ready(request_id=first): 150 | detected_faces = face_infer.get_face_detection_data(first) 151 | if face_cfg.ModelType == FaceDetectionModelTypes.MTCNN: 152 | face_landmarks = face_infer.get_face_landmarks_data(first) 153 | face_process_order.pop(0) 154 | face_request_order.append(first) 155 | show_frame = frame_order[0] 156 | frame_order.pop(0) 157 | if len(detected_faces) > 0: 158 | for idx, face in enumerate(detected_faces): 159 | ImageUtil.draw_rectangle(show_frame, (face[0], face[1], face[2], face[3])) 160 | 161 | if face_cfg.ModelType == FaceDetectionModelTypes.MTCNN: 162 | for coordinate in range(0, len(face_landmarks[idx]), 2): 163 | ImageUtil.draw_ellipse(show_frame, [face_landmarks[idx][coordinate], 164 | face_landmarks[idx][coordinate + 1]]) 165 | 166 | if save_roi_text: 167 | roi = "{};{};{};{};{}\n".format(frame_id, face[0], face[1], face[2], face[3]) 168 | roi_file.write(roi) 169 | 170 | if save_only_frames and not save_roi_video and len(detected_faces) > 0: 171 | cv.imwrite(output_dir + roi_frame_filename + "_{}.png".format(frame_id), show_frame) 172 | elif save_roi_video: 173 | roi_video.write(show_frame) 174 | 175 | # Required Since 176 | face_infer.LastFaceDetections = [] 177 | face_infer.LastLandmarkDetections = [] 178 | 179 | if len(face_request_order) > 0: 180 | has_frame, frame = capture.read() 181 | frame_id += 1 182 | else: 183 | face_infer.infer(frame) 184 | faces = face_infer.get_face_detection_data() 185 | if face_cfg.ModelType == FaceDetectionModelTypes.MTCNN: 186 | landmarks = face_infer.get_face_landmarks_data() 187 | 188 | if len(faces) > 0: 189 | print("Detected {} Faces with {} Threshold".format(len(faces), face_infer.Config.FaceDetectionThreshold)) 190 | for idx, face in enumerate(faces): 191 | ImageUtil.draw_rectangle(frame, (face[0], face[1], face[2], face[3])) 192 | 193 | if face_cfg.ModelType == FaceDetectionModelTypes.MTCNN: 194 | for coordinate in range(0, len(landmarks[idx]), 2): 195 | ImageUtil.draw_ellipse(frame, [landmarks[idx][coordinate], landmarks[idx][coordinate + 1]]) 196 | 197 | if save_roi_text: 198 | roi = "{};{};{};{};{}\n".format(frame_id, face[0], face[1], face[2], face[3]) 199 | roi_file.write(roi) 200 | 201 | if save_only_frames: 202 | cv.imwrite(output_dir + roi_frame_filename + "_{}.png".format(frame_id), frame) 203 | 204 | face_infer.print_inference_performance_metrics() 205 | 206 | inference_status = AppStatus.FINISHED 207 | 208 | roi_file.close() 209 | roi_video.release() 210 | 211 | 212 | inference_status = AppStatus.NOTSTARTED 213 | 214 | input_type = "image" 215 | input_path = '' 216 | web_cam_index = 0 217 | face_detection_model = FaceDetectionModelTypes.OPENMODELZOO 218 | logfile_name = "log.txt" # "/app/log.txt" 219 | json_req = None 220 | 221 | output_dir = "./" 222 | roi_text_filename = "inference_roi.txt" 223 | roi_video_filename = "inference_roi.mp4" 224 | roi_frame_filename = "inference_frame" 225 | 226 | save_roi_video = False 227 | save_only_frames = False 228 | save_roi_text = True 229 | 230 | 231 | @app.route("/", methods=['GET', 'POST']) 232 | def start(): 233 | if request.is_json: 234 | # Parse the JSON into a Python dictionary 235 | req = request.json 236 | try: 237 | if req["log_level"] == "DEBUG": 238 | logging.basicConfig(filename=logfile_name, 239 | level=logging.DEBUG, 240 | filemode='a', 241 | format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s', 242 | datefmt='%H:%M:%S') 243 | elif req["log_level"] == "INFO": 244 | logging.basicConfig(filename=logfile_name, 245 | level=logging.INFO, 246 | filemode='a', 247 | format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s', 248 | datefmt='%H:%M:%S') 249 | elif req["log_level"] == "WARN": 250 | logging.basicConfig(filename=logfile_name, 251 | level=logging.WARN, 252 | filemode='a', 253 | format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s', 254 | datefmt='%H:%M:%S') 255 | else: 256 | logging.basicConfig(filename=logfile_name, 257 | level=logging.ERROR, 258 | filemode='a', 259 | format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s', 260 | datefmt='%H:%M:%S') 261 | 262 | logging.log(logging.WARN, "Log Level Set to: {}".format(req["log_level"])) 263 | 264 | global input_path 265 | input_path = req["input_path"] 266 | 267 | logging.log(logging.WARN, "Input Path: {}".format(req["input_path"])) 268 | 269 | global input_type 270 | input_type = req["input_type"] 271 | 272 | logging.log(logging.WARN, "Input Type: {}".format(req["input_type"])) 273 | 274 | global web_cam_index 275 | web_cam_index = int(req["web_cam_index"]) 276 | 277 | logging.log(logging.WARN, "Web Cam {}".format(req["web_cam_index"])) 278 | 279 | global face_detection_model 280 | if req['face_detection_model'] == FaceDetectionModelTypes.MTCNN: 281 | face_detection_model = FaceDetectionModelTypes.MTCNN 282 | 283 | logging.log(logging.WARN, "Face Detection Model {}".format(req["face_detection_model"])) 284 | 285 | global save_roi_video 286 | if req["save_roi_video"] == "True": 287 | save_roi_video = True 288 | 289 | global save_only_frames 290 | if req["save_only_frames"] == "True": 291 | save_only_frames = True 292 | 293 | global save_roi_text 294 | if req["save_roi"] == "False": 295 | save_roi_text = False 296 | 297 | res = make_response(jsonify({"message": "INFERENCE STARTED"}), 200) 298 | 299 | global json_req 300 | json_req = req 301 | 302 | #threading.Thread(target=run_inference()).start() 303 | # Start Async Thread 304 | logging.log(logging.WARN, "Starting Inference ...") 305 | task = loop.create_task(inference()) 306 | 307 | if not loop.is_running(): 308 | loop.run_forever() 309 | else: 310 | logging.log(logging.WARN, "Thread Loop Running ...") 311 | 312 | return res 313 | except KeyError: 314 | logging.log(logging.ERROR, "Key Not Found Error") 315 | exit(-1) 316 | except Exception as e: 317 | logging.log(logging.ERROR, e.__str__()) 318 | exit(-1) 319 | # Return a string along with an HTTP status code 320 | 321 | else: 322 | # The request body wasn't JSON so return a 400 HTTP status code 323 | return "Request was not JSON", 400 324 | 325 | 326 | @app.route("/status", methods=["GET"]) 327 | def status(): 328 | """ 329 | Get App Status 330 | :return: 331 | """ 332 | logging.log(logging.WARN, "STATUS CALLED") 333 | return jsonify(inference_status), 200 334 | 335 | 336 | @app.route("/stop_inference", methods=["POST"]) 337 | def stop_inference(): 338 | """ 339 | Get App Status 340 | :return: 341 | """ 342 | 343 | global inference_status 344 | inference_status = AppStatus.STOPREQUEST 345 | logging.log(logging.WARN, "STOPPING INFERENCE ... ") 346 | return jsonify(inference_status), 200 347 | 348 | 349 | @app.route("/logs", methods=["GET"]) 350 | def logs(): 351 | """ 352 | Show Logs 353 | :return: 354 | """ 355 | with open(logfile_name) as f: 356 | file_content = f.read() 357 | 358 | return file_content, 200 359 | 360 | 361 | @app.route("/results", methods=["GET"]) 362 | def results(): 363 | """ 364 | Get Latest Results 365 | :return: 366 | """ 367 | 368 | roifile = output_dir + roi_text_filename 369 | with open(roifile) as f: 370 | file_content = f.read() 371 | 372 | return file_content 373 | 374 | 375 | @app.route('/play_roi', methods=["GET"]) 376 | def play_roi(): 377 | return redirect(output_dir + roi_video_filename) 378 | 379 | 380 | if __name__ == '__main__': 381 | parser = optparse.OptionParser(usage="python3 /app/face_detection_service.py -p ") 382 | parser.add_option('-p', '--port', action='store', dest='port', help='The port to listen on.') 383 | 384 | (args, _) = parser.parse_args() 385 | 386 | if args.port is None: 387 | print("Missing required argument: -p/--port") 388 | sys.exit(1) 389 | 390 | app.run(host='0.0.0.0', port=int(args.port), debug=True, threaded=True) -------------------------------------------------------------------------------- /inference_services/facedetection/inference_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "omz_facedetection" : { 3 | "model_path" : "/app/models/", 4 | "model_name" : "face-detection-retail-0013", 5 | "target_device" : "CPU", 6 | "cpu_extension" : "True", 7 | "cpu_extension_path" : "/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension.so", 8 | "face_detection_threshold" : 0.6, 9 | "async" : "False", 10 | "request_count" : 1, 11 | "dynamic_batch" : "False", 12 | "batch_size" : 1, 13 | "limit_cpu_threads" : "False", 14 | "number_of_cpu_threads" : 4, 15 | "bind_cpu_threads" : "True", 16 | "cpu_stream" : "AUTO", 17 | "gpu_stream" : "AUTO" 18 | }, 19 | 20 | "mtcnn_facedetection" : { 21 | "model_path" : "/app/models/", 22 | "p_model_file_name" : "det1-0001", 23 | "r_model_file_name" : "det2-0001", 24 | "o_model_file_name" : "det3-0001", 25 | "target_device" : "CPU", 26 | "cpu_extension" : "True", 27 | "cpu_extension_path" : "/opt/intel/openvino/inference_engine/lib/intel64/libcpu_extension_sse4.so", 28 | 29 | "p_network_threshold" : 0.6, 30 | "r_network_threshold" : 0.7, 31 | "o_network_threshold" : 0.8, 32 | 33 | "minimum_face_size" : 15.0, 34 | "minimum_length" : 720, 35 | "factor_count" : 0, 36 | "factor" : 0.707, 37 | "min_detection_size" : 12, 38 | 39 | "nms_thresholds" : [0.6, 0.6, 0.6], 40 | "r_input_batch_size" : 256, 41 | "o_input_batch_size" : 256, 42 | 43 | "limit_cpu_threads" : "False", 44 | "number_of_cpu_threads" : 4, 45 | "bind_cpu_threads" : "True", 46 | "cpu_stream" : "AUTO", 47 | "gpu_stream" : "AUTO" 48 | }, 49 | 50 | "face_detection_model" : "mtcnn_facedetection", 51 | "input_type" : "video", 52 | "input_path" : "/app/videos/facedetection.mp4", 53 | "web_cam_index" : "0", 54 | "log_level" : "DEBUG", 55 | 56 | "save_roi_video" : "False", 57 | "save_only_frames" : "False", 58 | "save_roi" : "True" 59 | } -------------------------------------------------------------------------------- /inference_services/facedetection/models/age-gender-recognition-retail-0013.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/models/age-gender-recognition-retail-0013.bin -------------------------------------------------------------------------------- /inference_services/facedetection/models/age-gender-recognition-retail-0013.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 1 8 | 3 9 | 62 10 | 62 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 1 19 | 3 20 | 62 21 | 62 22 | 23 | 24 | 25 | 26 | 1 27 | 48 28 | 60 29 | 60 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 1 42 | 48 43 | 60 44 | 60 45 | 46 | 47 | 48 | 49 | 1 50 | 48 51 | 30 52 | 30 53 | 54 | 55 | 56 | 57 | 58 | 59 | 1 60 | 48 61 | 30 62 | 30 63 | 64 | 65 | 66 | 67 | 1 68 | 48 69 | 30 70 | 30 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 1 79 | 48 80 | 30 81 | 30 82 | 83 | 84 | 85 | 86 | 1 87 | 64 88 | 28 89 | 28 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 1 102 | 64 103 | 28 104 | 28 105 | 106 | 107 | 108 | 109 | 1 110 | 64 111 | 14 112 | 14 113 | 114 | 115 | 116 | 117 | 118 | 119 | 1 120 | 64 121 | 14 122 | 14 123 | 124 | 125 | 126 | 127 | 1 128 | 64 129 | 14 130 | 14 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 1 139 | 64 140 | 14 141 | 14 142 | 143 | 144 | 145 | 146 | 1 147 | 96 148 | 14 149 | 14 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 1 162 | 96 163 | 14 164 | 14 165 | 166 | 167 | 168 | 169 | 1 170 | 96 171 | 7 172 | 7 173 | 174 | 175 | 176 | 177 | 178 | 179 | 1 180 | 96 181 | 7 182 | 7 183 | 184 | 185 | 186 | 187 | 1 188 | 96 189 | 7 190 | 7 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 1 199 | 96 200 | 7 201 | 7 202 | 203 | 204 | 205 | 206 | 1 207 | 192 208 | 5 209 | 5 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | 1 221 | 192 222 | 5 223 | 5 224 | 225 | 226 | 227 | 228 | 1 229 | 192 230 | 5 231 | 5 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 1 240 | 192 241 | 5 242 | 5 243 | 244 | 245 | 246 | 247 | 1 248 | 256 249 | 3 250 | 3 251 | 252 | 253 | 254 | 255 | 256 | 257 | 258 | 259 | 260 | 261 | 1 262 | 256 263 | 3 264 | 3 265 | 266 | 267 | 268 | 269 | 1 270 | 256 271 | 3 272 | 3 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 1 281 | 256 282 | 3 283 | 3 284 | 285 | 286 | 287 | 288 | 1 289 | 256 290 | 1 291 | 1 292 | 293 | 294 | 295 | 296 | 297 | 298 | 299 | 300 | 301 | 302 | 1 303 | 256 304 | 1 305 | 1 306 | 307 | 308 | 309 | 310 | 1 311 | 256 312 | 1 313 | 1 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 1 322 | 256 323 | 1 324 | 1 325 | 326 | 327 | 328 | 329 | 1 330 | 512 331 | 1 332 | 1 333 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 1 344 | 512 345 | 1 346 | 1 347 | 348 | 349 | 350 | 351 | 1 352 | 512 353 | 1 354 | 1 355 | 356 | 357 | 358 | 359 | 360 | 361 | 362 | 1 363 | 512 364 | 1 365 | 1 366 | 367 | 368 | 369 | 370 | 1 371 | 1 372 | 1 373 | 1 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 1 386 | 256 387 | 3 388 | 3 389 | 390 | 391 | 392 | 393 | 1 394 | 256 395 | 1 396 | 1 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 1 408 | 256 409 | 1 410 | 1 411 | 412 | 413 | 414 | 415 | 1 416 | 256 417 | 1 418 | 1 419 | 420 | 421 | 422 | 423 | 424 | 425 | 426 | 1 427 | 256 428 | 1 429 | 1 430 | 431 | 432 | 433 | 434 | 1 435 | 512 436 | 1 437 | 1 438 | 439 | 440 | 441 | 442 | 443 | 444 | 445 | 446 | 447 | 448 | 1 449 | 512 450 | 1 451 | 1 452 | 453 | 454 | 455 | 456 | 1 457 | 512 458 | 1 459 | 1 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 1 468 | 512 469 | 1 470 | 1 471 | 472 | 473 | 474 | 475 | 1 476 | 2 477 | 1 478 | 1 479 | 480 | 481 | 482 | 483 | 484 | 485 | 486 | 487 | 488 | 489 | 490 | 1 491 | 2 492 | 1 493 | 1 494 | 495 | 496 | 497 | 498 | 1 499 | 2 500 | 1 501 | 1 502 | 503 | 504 | 505 | 506 | 507 | 508 | 509 | 510 | 511 | 512 | 513 | 514 | 515 | 516 | 517 | 518 | 519 | 520 | 521 | 522 | 523 | 524 | 525 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 | 544 | 545 | 546 | 547 | 548 | 549 | 550 | 551 | 552 | 553 | 554 | 555 | 556 | 557 | 558 | 559 | 560 | 561 | 562 | 563 | 564 | 565 | 566 | 567 | 568 | 569 | 570 | 571 | 572 | 573 | 574 | 575 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det1-0001.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/models/det1-0001.bin -------------------------------------------------------------------------------- /inference_services/facedetection/models/det1-0001.mapping: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det1-0001.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 1 8 | 3 9 | 12 10 | 12 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 1 19 | 3 20 | 12 21 | 12 22 | 23 | 24 | 25 | 26 | 1 27 | 10 28 | 10 29 | 10 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 1 42 | 10 43 | 10 44 | 10 45 | 46 | 47 | 48 | 49 | 1 50 | 10 51 | 10 52 | 10 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 1 64 | 10 65 | 10 66 | 10 67 | 68 | 69 | 70 | 71 | 1 72 | 10 73 | 5 74 | 5 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 1 83 | 10 84 | 5 85 | 5 86 | 87 | 88 | 89 | 90 | 1 91 | 16 92 | 3 93 | 3 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 1 106 | 16 107 | 3 108 | 3 109 | 110 | 111 | 112 | 113 | 1 114 | 16 115 | 3 116 | 3 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 1 128 | 16 129 | 3 130 | 3 131 | 132 | 133 | 134 | 135 | 1 136 | 32 137 | 1 138 | 1 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 1 151 | 32 152 | 1 153 | 1 154 | 155 | 156 | 157 | 158 | 1 159 | 32 160 | 1 161 | 1 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 1 173 | 32 174 | 1 175 | 1 176 | 177 | 178 | 179 | 180 | 1 181 | 4 182 | 1 183 | 1 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 1 196 | 32 197 | 1 198 | 1 199 | 200 | 201 | 202 | 203 | 1 204 | 2 205 | 1 206 | 1 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 1 219 | 2 220 | 1 221 | 1 222 | 223 | 224 | 225 | 226 | 1 227 | 2 228 | 1 229 | 1 230 | 231 | 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 241 | 242 | 243 | 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det2-0001.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/models/det2-0001.bin -------------------------------------------------------------------------------- /inference_services/facedetection/models/det2-0001.mapping: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det2-0001.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 1 8 | 3 9 | 24 10 | 24 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 1 19 | 3 20 | 24 21 | 24 22 | 23 | 24 | 25 | 26 | 1 27 | 28 28 | 22 29 | 22 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 1 42 | 28 43 | 22 44 | 22 45 | 46 | 47 | 48 | 49 | 1 50 | 28 51 | 22 52 | 22 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 1 64 | 28 65 | 22 66 | 22 67 | 68 | 69 | 70 | 71 | 1 72 | 28 73 | 11 74 | 11 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 1 83 | 28 84 | 11 85 | 11 86 | 87 | 88 | 89 | 90 | 1 91 | 48 92 | 9 93 | 9 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 1 106 | 48 107 | 9 108 | 9 109 | 110 | 111 | 112 | 113 | 1 114 | 48 115 | 9 116 | 9 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 1 128 | 48 129 | 9 130 | 9 131 | 132 | 133 | 134 | 135 | 1 136 | 48 137 | 4 138 | 4 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 1 147 | 48 148 | 4 149 | 4 150 | 151 | 152 | 153 | 154 | 1 155 | 64 156 | 3 157 | 3 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 1 170 | 64 171 | 3 172 | 3 173 | 174 | 175 | 176 | 177 | 1 178 | 64 179 | 3 180 | 3 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 1 192 | 64 193 | 3 194 | 3 195 | 196 | 197 | 198 | 199 | 1 200 | 128 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 1 213 | 128 214 | 215 | 216 | 217 | 218 | 1 219 | 128 220 | 221 | 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | 230 | 1 231 | 128 232 | 233 | 234 | 235 | 236 | 1 237 | 4 238 | 239 | 240 | 241 | 242 | 243 | 244 | 245 | 246 | 247 | 248 | 249 | 1 250 | 128 251 | 252 | 253 | 254 | 255 | 1 256 | 2 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 1 269 | 2 270 | 271 | 272 | 273 | 274 | 1 275 | 2 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | 297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det3-0001.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/models/det3-0001.bin -------------------------------------------------------------------------------- /inference_services/facedetection/models/det3-0001.mapping: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | -------------------------------------------------------------------------------- /inference_services/facedetection/models/det3-0001.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 1 8 | 3 9 | 48 10 | 48 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 1 19 | 3 20 | 48 21 | 48 22 | 23 | 24 | 25 | 26 | 1 27 | 32 28 | 46 29 | 46 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 1 42 | 32 43 | 46 44 | 46 45 | 46 | 47 | 48 | 49 | 1 50 | 32 51 | 46 52 | 46 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 1 64 | 32 65 | 46 66 | 46 67 | 68 | 69 | 70 | 71 | 1 72 | 32 73 | 23 74 | 23 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 1 83 | 32 84 | 23 85 | 23 86 | 87 | 88 | 89 | 90 | 1 91 | 64 92 | 21 93 | 21 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 1 106 | 64 107 | 21 108 | 21 109 | 110 | 111 | 112 | 113 | 1 114 | 64 115 | 21 116 | 21 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 1 128 | 64 129 | 21 130 | 21 131 | 132 | 133 | 134 | 135 | 1 136 | 64 137 | 10 138 | 10 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 1 147 | 64 148 | 10 149 | 10 150 | 151 | 152 | 153 | 154 | 1 155 | 64 156 | 8 157 | 8 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 1 170 | 64 171 | 8 172 | 8 173 | 174 | 175 | 176 | 177 | 1 178 | 64 179 | 8 180 | 8 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 1 192 | 64 193 | 8 194 | 8 195 | 196 | 197 | 198 | 199 | 1 200 | 64 201 | 4 202 | 4 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 1 211 | 64 212 | 4 213 | 4 214 | 215 | 216 | 217 | 218 | 1 219 | 128 220 | 3 221 | 3 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | 230 | 231 | 232 | 233 | 1 234 | 128 235 | 3 236 | 3 237 | 238 | 239 | 240 | 241 | 1 242 | 128 243 | 3 244 | 3 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | 255 | 1 256 | 128 257 | 3 258 | 3 259 | 260 | 261 | 262 | 263 | 1 264 | 256 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 1 277 | 256 278 | 279 | 280 | 281 | 282 | 1 283 | 256 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 1 295 | 256 296 | 297 | 298 | 299 | 300 | 1 301 | 4 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 1 314 | 256 315 | 316 | 317 | 318 | 319 | 1 320 | 10 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 1 333 | 256 334 | 335 | 336 | 337 | 338 | 1 339 | 2 340 | 341 | 342 | 343 | 344 | 345 | 346 | 347 | 348 | 349 | 350 | 351 | 1 352 | 2 353 | 354 | 355 | 356 | 357 | 1 358 | 2 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | 415 | -------------------------------------------------------------------------------- /inference_services/facedetection/requirements.txt: -------------------------------------------------------------------------------- 1 | Flask>=1.1 2 | opencv-python 3 | pillow -------------------------------------------------------------------------------- /inference_services/facedetection/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/inference_services/facedetection/utils/__init__.py -------------------------------------------------------------------------------- /inference_services/facedetection/utils/image_utils.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import cv2 as cv 24 | import numpy as np 25 | 26 | 27 | class ImageUtil(object): 28 | @staticmethod 29 | def crop_frame(frame, coordinate, normalized=True): 30 | """ 31 | Crop Frame 32 | :param frame: cv mat object 33 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 34 | :param normalized: if values normalized 35 | :return: 36 | """ 37 | 38 | x1 = coordinate[0] 39 | y1 = coordinate[1] 40 | x2 = coordinate[2] 41 | y2 = coordinate[3] 42 | 43 | if normalized: 44 | h = frame.shape[0] 45 | w = frame.shape[1] 46 | 47 | x1 = int(x1 * w) 48 | x2 = int(x2 * w) 49 | 50 | y1 = int(y1 * h) 51 | y2 = int(y2 * h) 52 | 53 | return frame[y1:y2, x1:x2] 54 | 55 | @staticmethod 56 | def draw_text(frame, text, coordinate, line_color=(0, 255, 124), normalized=True): 57 | """ 58 | Draw text with cv.puttext method 59 | :param frame: cv mat object 60 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 61 | :param normalized: if values normalized 62 | :param text: Text to write on image 63 | :param line_color: color of text 64 | :return: 65 | """ 66 | 67 | x1 = coordinate[0] 68 | y1 = coordinate[1] 69 | x2 = coordinate[2] 70 | y2 = coordinate[3] 71 | 72 | if normalized: 73 | h = frame.shape[0] 74 | w = frame.shape[1] 75 | 76 | x1 = int(x1 * w) 77 | x2 = int(x2 * w) 78 | 79 | y1 = int(y1 * h) 80 | y2 = int(y2 * h) 81 | 82 | font = cv.FONT_HERSHEY_SIMPLEX 83 | bottom_left_corner_of_text = (x2, y1 + 10) 84 | font_scale = 0.4 85 | font_color = line_color 86 | line_type = 1 87 | 88 | cv.putText(frame, 89 | text, 90 | bottom_left_corner_of_text, 91 | font, 92 | font_scale, 93 | font_color, 94 | line_type) 95 | 96 | @staticmethod 97 | def draw_rectangles(frame, coordinates, line_color=(0, 255, 124), normalized=True): 98 | """ 99 | Draw Rectangles with given Normalized 100 | :param frame: cv mat object 101 | :param coordinates: x,y coordinates [xmin, ymin, xmax, ymax] 102 | :param normalized: if values normalized 103 | :param line_color: color of rectangle 104 | :return: 105 | """ 106 | for coordinate in coordinates: 107 | x1 = coordinate[0] 108 | y1 = coordinate[1] 109 | x2 = coordinate[2] 110 | y2 = coordinate[3] 111 | 112 | if normalized: 113 | h = frame.shape[0] 114 | w = frame.shape[1] 115 | 116 | x1 = int(x1 * w) 117 | x2 = int(x2 * w) 118 | 119 | y1 = int(y1 * h) 120 | y2 = int(y2 * h) 121 | 122 | cv.rectangle(frame, (x1, y1), (x2, y2), line_color, 2) 123 | 124 | @staticmethod 125 | def draw_rectangle(frame, coordinate, line_color=(0, 255, 124), normalized=True): 126 | """ 127 | Draw Rectangle with given Normalized 128 | :param frame: cv mat object 129 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 130 | :param normalized: if values normalized 131 | :param line_color: color of rectangle 132 | :return: 133 | """ 134 | 135 | x1 = coordinate[0] 136 | y1 = coordinate[1] 137 | x2 = coordinate[2] 138 | y2 = coordinate[3] 139 | 140 | if normalized: 141 | h = frame.shape[0] 142 | w = frame.shape[1] 143 | 144 | x1 = int(x1 * w) 145 | x2 = int(x2 * w) 146 | 147 | y1 = int(y1 * h) 148 | y2 = int(y2 * h) 149 | 150 | cv.rectangle(frame, (x1, y1), (x2, y2), line_color, 2) 151 | 152 | @staticmethod 153 | def draw_ellipse(frame, coordinate, line_color=(124, 0, 0), radius=1, normalized=True): 154 | """ 155 | Draw Circle with given values 156 | :param frame: cv mat object 157 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 158 | :param normalized: if values normalized 159 | :param line_color: color of rectangle 160 | :param radius: radius of circle 161 | :return: 162 | """ 163 | 164 | x1 = coordinate[0] 165 | y1 = coordinate[1] 166 | 167 | if normalized: 168 | h = frame.shape[0] 169 | w = frame.shape[1] 170 | 171 | x1 = int(x1 * w) 172 | y1 = int(y1 * h) 173 | 174 | cv.circle(frame, (x1, y1), radius=radius, color=line_color, thickness=1) 175 | -------------------------------------------------------------------------------- /inference_services/flask_hello_world/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:18.04 2 | 3 | MAINTAINER ONUR DUNDAR "onur.dundar1@gmail.com" 4 | 5 | RUN apt-get update -y && \ 6 | apt-get install -y python3-pip python3-dev 7 | 8 | COPY ./requirements.txt /app/requirements.txt 9 | 10 | WORKDIR /app 11 | 12 | RUN pip3 install -r requirements.txt 13 | 14 | COPY flask_app.py /app/ 15 | 16 | EXPOSE 8000 17 | 18 | CMD ["python3", "/app/flask_app.py", "-p 8000"] -------------------------------------------------------------------------------- /inference_services/flask_hello_world/README.md: -------------------------------------------------------------------------------- 1 | # A Basic Flask App 2 | 3 | ```bash 4 | docker build -t flask-tutorial:latest . 5 | 6 | docker run -d -p 8000:8000 flask-tutorial 7 | ``` 8 | 9 | 10 | 11 | 12 | 13 | 14 | -------------------------------------------------------------------------------- /inference_services/flask_hello_world/flask_app.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | from flask import Flask 24 | import sys 25 | import optparse 26 | import time 27 | 28 | app = Flask(__name__) 29 | 30 | start = int(round(time.time())) 31 | 32 | 33 | @app.route("/") 34 | def hello_world(): 35 | return "Hello Flask" 36 | 37 | 38 | if __name__ == '__main__': 39 | parser = optparse.OptionParser(usage="python simpleapp.py -p ") 40 | parser.add_option('-p', '--port', action='store', dest='port', help='The port to listen on.') 41 | (args, _) = parser.parse_args() 42 | if args.port == None: 43 | print("Missing required argument: -p/--port") 44 | sys.exit(1) 45 | app.run(host='0.0.0.0', port=int(args.port), debug=False) -------------------------------------------------------------------------------- /inference_services/flask_hello_world/requirements.txt: -------------------------------------------------------------------------------- 1 | Flask>=1.1 -------------------------------------------------------------------------------- /models/README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Models 2 | 3 | In this face detection applications, I used models from Intel's Open Model Zoo and from `deepinsight` github repository. 4 | 5 | ## Open Model Zoo Face & Age-Gender Detection Models 6 | 7 | There are numerous PoC type of DL models by intel in the following repository: 8 | 9 | https://github.com/opencv/open_model_zoo 10 | 11 | I have used face detection and age-gender detection models respectively: 12 | 13 | - https://github.com/opencv/open_model_zoo/tree/master/intel_models/face-detection-retail-0004 14 | - https://github.com/opencv/open_model_zoo/tree/master/intel_models/age-gender-recognition-retail-0013 15 | 16 | These models can be downloaded by OpenVINO(TM) Toolkit model downloader using below commands: 17 | 18 | ```bash 19 | python3 /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py --name age-gender-recognition-retail-0013 --output_dir openvino_models/ 20 | python3 /opt/intel/openvino/deployment_tools/tools/model_downloader/downloader.py --name face-detection-retail-0004 --output_dir openvino_models/ 21 | ``` 22 | 23 | You can use the models from their downloaded directory. Each has FP16 version as well. They are ready to be used with Inference Engine no extra work required. 24 | 25 | Please also check the descriptions from the `description/face-detection-retail-0004.md` file to understand their input and outputs. 26 | 27 | ## MTCNN Face & Age-Gender Detection Models 28 | 29 | ### Face Detection Model 30 | 31 | Source of Face Detection Model: 32 | 33 | - https://github.com/YYuanAnyVision/mxnet_mtcnn_face_detection 34 | 35 | Get Models and convert them using Model Optimizer. 36 | 37 | Models are being trained using MxNet* framework therefore we will check guidance for conversion from MxNet conversion guideline: 38 | 39 | - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet.html 40 | 41 | ```bash 42 | git clone https://github.com/YYuanAnyVision/mxnet_mtcnn_face_detection.git 43 | 44 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_mxnet.py --input_model mxnet_mtcnn_face_detection/model/det1-0001.params --input_symbol mxnet_mtcnn_face_detection/model/det1-symbol.json --input_shape [1,3,12,12] --output_dir FP32/ --reverse_input_channels 45 | 46 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_mxnet.py --input_model mxnet_mtcnn_face_detection/model/det2-0001.params --input_symbol mxnet_mtcnn_face_detection/model/det2-symbol.json --input_shape [1,3,24,24] --output_dir FP32/ --reverse_input_channels 47 | 48 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_mxnet.py --input_model mxnet_mtcnn_face_detection/model/det3-0001.params --input_symbol mxnet_mtcnn_face_detection/model/det3-symbol.json --input_shape [1,3,48,48] --output_dir FP32/ --reverse_input_channels 49 | ``` 50 | 51 | You can also change FP32 to FP16 and make other corrections. 52 | 53 | ### Age Gender Model 54 | 55 | - https://github.com/deepinsight/insightface/tree/master/gender-age 56 | 57 | Pre-trained Model URL: 58 | 59 | https://www.dropbox.com/s/2xq8mcao6z14e3u/gamodel-r50.zip?dl=0 60 | 61 | You can convert models using Model Optimizer to make them ready to be used with OpenVINO(TM) 62 | 63 | Deepinsight models are being trained using MxNet* framework therefore we will check guidance for conversion from MxNet conversion guideline: 64 | 65 | - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet.html 66 | 67 | ```bash 68 | unzip gamodel-r50.zip 69 | 70 | cd gamodel-r50 71 | 72 | python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_mxnet.py --input_model model-0000.params --input_shape [1,3,112,112] --output_dir FP32 --data_type FP32 --scale 0.0399 --mean_values [127.5,127.5,127.5] 73 | ``` 74 | 75 | Important part here is the --input_shape, --mean_values and --scale parameters, you should investigate what they should be from the model descriptions page. Otherwise you will get faulty output. 76 | 77 | # Important Notes for Model Conversion 78 | 79 | Important factor while converting models to IR representations is that, all layers should be supported by Intel(R) Disribution of OpenVINO(TM) Model Optimizer otherwise certain errors will be occured. 80 | 81 | If layers not supported, you should follow some additional development process and extend OpenVINO(TM) Toolkit Model Optimizer and add layer primitive implementation to use it in the runtime. 82 | 83 | Many of the layer implementations are supported for MxNet*, Tensorflow*, Caffe*, ONNX* and Kaldi* 84 | 85 | Below are the guidelines to follow from documentation to learn about the model conversion process. 86 | 87 | - Here is the general guideline for conversion process: 88 | 89 | - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Prepare_Trained_Model.html 90 | - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html 91 | 92 | - Here you can find the list of supported layers for each framework: 93 | 94 | - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html 95 | 96 | If there are custom layers being used you need to add those layers with using defined steps in OpenVINO(TM) Toolkit documentation. 97 | 98 | - Adding Custom Layers: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odundar/face-detection-python/8ef8863d85c4ddd3c67512ff3e48fa4cf7c1d770/utils/__init__.py -------------------------------------------------------------------------------- /utils/image_utils.py: -------------------------------------------------------------------------------- 1 | # MIT License 2 | # 3 | # Copyright (c) 2019 Onur Dundar onur.dundar1@gmail.com 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import cv2 as cv 24 | import numpy as np 25 | 26 | 27 | class ImageUtil(object): 28 | @staticmethod 29 | def crop_frame(frame, coordinate, normalized=True): 30 | """ 31 | Crop Frame 32 | :param frame: cv mat object 33 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 34 | :param normalized: if values normalized 35 | :return: 36 | """ 37 | 38 | x1 = coordinate[0] 39 | y1 = coordinate[1] 40 | x2 = coordinate[2] 41 | y2 = coordinate[3] 42 | 43 | if normalized: 44 | h = frame.shape[0] 45 | w = frame.shape[1] 46 | 47 | x1 = int(x1 * w) 48 | x2 = int(x2 * w) 49 | 50 | y1 = int(y1 * h) 51 | y2 = int(y2 * h) 52 | 53 | return frame[y1:y2, x1:x2] 54 | 55 | @staticmethod 56 | def draw_text(frame, text, coordinate, line_color=(0, 255, 124), normalized=True): 57 | """ 58 | Draw text with cv.puttext method 59 | :param frame: cv mat object 60 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 61 | :param normalized: if values normalized 62 | :param text: Text to write on image 63 | :param line_color: color of text 64 | :return: 65 | """ 66 | 67 | x1 = coordinate[0] 68 | y1 = coordinate[1] 69 | x2 = coordinate[2] 70 | y2 = coordinate[3] 71 | 72 | if normalized: 73 | h = frame.shape[0] 74 | w = frame.shape[1] 75 | 76 | x1 = int(x1 * w) 77 | x2 = int(x2 * w) 78 | 79 | y1 = int(y1 * h) 80 | y2 = int(y2 * h) 81 | 82 | font = cv.FONT_HERSHEY_SIMPLEX 83 | bottom_left_corner_of_text = (x2, y1 + 10) 84 | font_scale = 0.4 85 | font_color = line_color 86 | line_type = 1 87 | 88 | cv.putText(frame, 89 | text, 90 | bottom_left_corner_of_text, 91 | font, 92 | font_scale, 93 | font_color, 94 | line_type) 95 | 96 | @staticmethod 97 | def draw_rectangles(frame, coordinates, line_color=(0, 255, 124), normalized=True): 98 | """ 99 | Draw Rectangles with given Normalized 100 | :param frame: cv mat object 101 | :param coordinates: x,y coordinates [xmin, ymin, xmax, ymax] 102 | :param normalized: if values normalized 103 | :param line_color: color of rectangle 104 | :return: 105 | """ 106 | for coordinate in coordinates: 107 | x1 = coordinate[0] 108 | y1 = coordinate[1] 109 | x2 = coordinate[2] 110 | y2 = coordinate[3] 111 | 112 | if normalized: 113 | h = frame.shape[0] 114 | w = frame.shape[1] 115 | 116 | x1 = int(x1 * w) 117 | x2 = int(x2 * w) 118 | 119 | y1 = int(y1 * h) 120 | y2 = int(y2 * h) 121 | 122 | cv.rectangle(frame, (x1, y1), (x2, y2), line_color, 2) 123 | 124 | @staticmethod 125 | def draw_rectangle(frame, coordinate, line_color=(0, 255, 124), normalized=True): 126 | """ 127 | Draw Rectangle with given Normalized 128 | :param frame: cv mat object 129 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 130 | :param normalized: if values normalized 131 | :param line_color: color of rectangle 132 | :return: 133 | """ 134 | 135 | x1 = coordinate[0] 136 | y1 = coordinate[1] 137 | x2 = coordinate[2] 138 | y2 = coordinate[3] 139 | 140 | if normalized: 141 | h = frame.shape[0] 142 | w = frame.shape[1] 143 | 144 | x1 = int(x1 * w) 145 | x2 = int(x2 * w) 146 | 147 | y1 = int(y1 * h) 148 | y2 = int(y2 * h) 149 | 150 | cv.rectangle(frame, (x1, y1), (x2, y2), line_color, 2) 151 | 152 | @staticmethod 153 | def draw_ellipse(frame, coordinate, line_color=(124, 0, 0), radius=1, normalized=True): 154 | """ 155 | Draw Circle with given values 156 | :param frame: cv mat object 157 | :param coordinate: x,y coordinates [xmin, ymin, xmax, ymax] 158 | :param normalized: if values normalized 159 | :param line_color: color of rectangle 160 | :param radius: radius of circle 161 | :return: 162 | """ 163 | 164 | x1 = coordinate[0] 165 | y1 = coordinate[1] 166 | 167 | if normalized: 168 | h = frame.shape[0] 169 | w = frame.shape[1] 170 | 171 | x1 = int(x1 * w) 172 | y1 = int(y1 * h) 173 | 174 | cv.circle(frame, (x1, y1), radius=radius, color=line_color, thickness=1) 175 | --------------------------------------------------------------------------------