├── Images
    ├── IE graphic.png
    ├── OV Overview.png
    ├── face_detection.png
    └── face_detection_overlay.png
├── README.md
├── Videos
    └── head-pose-face-detection-female-and-male.mp4
└── dx_face_detection
    ├── CMakeLists.txt
    ├── cmake
        └── CMakeLists_common.cmake
    ├── face_detection.hpp
    ├── main.cpp
    └── run_fd.sh


/Images/IE graphic.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/bc5268f8f4722a95cca2ae6f5e130ec39623d43c/Images/IE graphic.png


--------------------------------------------------------------------------------
/Images/OV Overview.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/bc5268f8f4722a95cca2ae6f5e130ec39623d43c/Images/OV Overview.png


--------------------------------------------------------------------------------
/Images/face_detection.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/bc5268f8f4722a95cca2ae6f5e130ec39623d43c/Images/face_detection.png


--------------------------------------------------------------------------------
/Images/face_detection_overlay.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/bc5268f8f4722a95cca2ae6f5e130ec39623d43c/Images/face_detection_overlay.png


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # OpenVINO with FPGA Hello World Face Detection Exercise
  2 | 
  3 | **Note:** This tutorial has been written using OpenVINO™ Toolkit for Linux with FPGA Support version 2018 R4 and is for use with this version only. Using this tutorial with any other version may not be correct.
  4 | 
  5 | # Table of Contents
  6 | 
  7 | <div class="table-of-contents"><ul>
  8 | <li><a href="#openvino-with-fpga-hello-world-face-detection-exercise">OpenVINO with FPGA Hello World Face Detection Exercise</a></li>
  9 | <li><a href="#table-of-contents">Table of Contents</a></li>
 10 | <li><a href="#introduction">Introduction</a></li>
 11 | <li><a href="#getting-started">Getting Started</a><ul>
 12 | <li><a href="#prerequisites">Prerequisites</a></li></ul></li>
 13 | <li><a href="#download-the-tutorial-from-the-git-repository">Download the Tutorial from the Git Repository</a><ul>
 14 | <li><a href="#use-git-clone-to-clone-the-entire-repository">Use Git Clone to Clone the Entire Repository</a></li></ul></li><ul></li></ul>
 15 | <li><a href="#tutorial-files">Tutorial Files</a></li>
 16 | <li><a href="#openvino-toolkit-overview-and-terminology">OpenVINO™ Toolkit Overview and Terminology</a><ul>
 17 | <li><a href="#the-inference-engine">The Inference Engine</a></li></ul><ul>
 18 | <li><a href="#face-detection-sample">Face Detection Sample</a></li></ul>
 19 | <li><a href="#build">Build</a>
 20 | <li><a href="#run-the-application">Run the Application</a></li></ul></div>
 21 | 
 22 | ## Introduction
 23 | 
 24 | The OpenVINO™ toolkit runs inference models using the CPU to process images. As an option, OpenVINO can use GPU and VPU devices, if available. You can use the inference models to process video from an optional USB camera, an existing video file, or still image files. 
 25 | 
 26 | This tutorial examines a sample application that was created with the OpenVINO™ toolkit. The tutorial steps will guide you through downloading the latest Face Detection Tutorial from GitHub, walk you through the sample code and then compile and run the code on the the available hardware. During the process, you will become familiar with key OpenVINO™ concepts.
 27 | 
 28 | This tutorial starts with a base application that reads image data and outputs the image to a window. The steps build on each other by adding deep learning models that process image data and make inferences. In the final step, you will use the completed application to detect faces, report the age and gender of the detected face, and draw a 3D axis representing the head pose for each face.
 29 | 
 30 | ## Getting Started
 31 | 
 32 | ### Prerequisites
 33 | 
 34 | To download all the files for this tutorial, you will need to access GitHub on the Internet.
 35 | 
 36 | To run the application in this tutorial, the OpenVINO™ toolkit for Linux with FPGA and its dependencies must already be installed and verified using the included demos. Installation instructions can be found here: https://software.intel.com/en-us/articles/OpenVINO-Install-Linux-FPGA
 37 | If any optional hardware is to be used, it must be installed and verified including:
 38 | * Intel® Arria® 10 GX FPGA Development Kit or the Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA. 
 39 | * GPU - normally embedded with supported Intel® CPUs; requires drivers and updated Linux kernel to run.
 40 | * VPU - USB Intel® Movidius™ Neural Compute Stick with Myriad. The **GitHub** download or clone will include the dx_face_detection zip file.
 41 | 
 42 | ### Summary of what is needed:
 43 | ### Hardware
 44 | Target and development platforms meeting the requirements described in the "System Requirements" section of the OpenVINO™ toolkit documentation which can be found here: https://software.intel.com/openvino-toolkit
 45 | 
 46 | **Note:** For this tutorial, an Intel® i7-7700 (CPU with GPU & Intel® Arria® 10 GX FPGA Development Kit) was used as both the development and target platform.
 47 | 
 48 | **Optional:**
 49 | * Intel® Movidius™ Neural Compute Stick
 50 | * GPU support
 51 | ### Software
 52 | * OpenVINO™ toolkit supported Linux operating system. This tutorial was run on 64-bit Ubuntu 16.04.3 LTS updated to kernel 4.14.20 following the OpenVINO™ toolkit installation instructions.
 53 | * The latest OpenVINO™ toolkit with FPGA support installed and verified. This tutorial was written using version 2018 R4.
 54 | * At least one utility for downloading from the GitHub repository: Subversion (svn), Git (git), or both.
 55 | 
 56 | By now you should have completed the Linux installation guide for the OpenVINO™ toolkit with FPGA support (link). Before continuing, please ensure:
 57 | * That you have run the supplied demo samples.
 58 | * If you have and intend to use an FPGA: You have installed and tested the Quartus® Programmer (link?) and are able to program bitstreams. 
 59 | * If you have and intend to use a GPU: You have installed and tested the GPU drivers.
 60 | * If you have and intend to use a Myriad: You have connected and tested the USB Intel® Movidius™ Neural Compute Stick. 
 61 | 
 62 | ### Required Connectivity
 63 | * Your development platform is connected to a network and has Internet access. 
 64 | * You have Github access.
 65 | 
 66 | ## Download the Tutorial from the Git Repository
 67 | In these steps you create a directory for the files you download from the “OpenVINO FPGA Hello World Face Detection” GitHub repository. 
 68 | ### Use Git Clone to Clone the Entire Repository
 69 | 1.	Open a command shell prompt (such as xterm).
 70 | 2. If you have not installed Git, install it now:
 71 | ```
 72 | sudo apt-get update
 73 | sudo apt-get install git
 74 | ```
 75 | 3.	Create a directory named "tutorials":
 76 | ```
 77 | mkdir tutorials
 78 | ```
 79 | 4. Go to the tutorials directory:
 80 | ```
 81 | cd tutorials
 82 | ```
 83 | 5.	Clone the repository:
 84 | ```
 85 | git clone https://github.com/fritzboyle/openvino-with-fpga-hello-world-face-detection
 86 | ```
 87 | 
 88 | 6.	Change to the face detection tutorial folder:
 89 | ```
 90 | cd openvino-with-fpga-hello-world-face-detection
 91 | ```
 92 | 
 93 | You now have the Face Detection Tutorial files. The next section shows the directory structure of the files you extracted.
 94 | 
 95 | # Tutorial Files
 96 | The "tutorial" directory contains:
 97 | * Images\ - directory of images
 98 | * Videos\ - directory of videos
 99 | * cmake\ - directory of common CMake files
100 | * dx_face_detection\  - directory of code to help run the scripts
101 | * Readme.md - This document
102 | 
103 | # OpenVINO™ Toolkit Overview and Terminology
104 | The OpenVINO™ toolkit enables the quick deployment of Convolutional Neural Networks (CNN) for heterogeneous execution on Intel® hardware while maximizing performance. Deployment is accomplished through the the Intel® Deep Learning Deployment Toolkit (Intel® DL Deployment Toolkit) included within the OpenVINO™ toolkit.
105 | ![OV Overview](https://github.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/blob/master/Images/OV%20Overview.png)
106 | 
107 | The CNN workflow is:
108 | 1.	Create and train the CNN inference model in a framework, such as Caffe*.
109 | 2.	Use the Model Optimizer on the trained model to produce an optimized Intermediate Representation (IR), stored .bin and .xml files for use with the Inference Engine.
110 | 3.	Use the Inference Engine with your application to load and run the model on your devices.
111 | 
112 | This tutorial focuses on the last step in the workflow; using the user application the Inference Engine to run models on a CPU, GPU, FPGA, and Movidius™ Neural Compute Stick.
113 | ## The Inference Engine
114 | Below is a more detailed view of the User Application and Inference Engine:
115 | ![IE Graphic](https://github.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/blob/master/Images/IE%20graphic.png)
116 | 
117 | The Inference Engine includes a plugin library for each supported device that has been optimized for the Intel® hardware device CPU, GPU, FPGA and Myriad. We will use the terms "device" and “plugin” with the assumption that one infers the other (e.g. CPU device infers the CPU plugin and vice versa). As part of loading the model, the User Application tells the Inference Engine which device to target which in turn loads the associated plugin library to later run on the associated device. The Inference Engine uses “blobs” for all data exchanges which are arrays in memory arranged according to the input and output data of the model.
118 | ![face_detection](https://github.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/blob/master/Images/face_detection.png)
119 | 
120 | ## Face Detection Sample
121 | In this Face Detection sample, the model estimates the head pose based on the face image it is given. The face detection model estimates the age, gender and head pose. After the head pose model has processed the face, the application will draw a set of axes over the face, indicating the Yaw, Pitch, and Roll orientation of the head. A sample output showing the results with the three axes appears below. The metrics reported also include the time to run the head pose model.
122 | 
123 | ![face_detection_overlay](https://github.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/blob/master/Images/face_detection_overlay.png)
124 | 
125 | In the image above, the three axes intersect in the center of the head. The blue line represents Roll, and it extends from the center of the head to the front and the back of the head. The red line represents Pitch, and is drawn from the center of the head to the left ear. The green line represents Yaw, and is drawn from the center of the head to the top of the head.
126 | 
127 | For details about the models see the Full Tutorial: https://github.com/intel-iot-devkit/inference-tutorials-generic/blob/openvino_toolkit_r3_0/face_detection_tutorial/step_4/Readme.md#introduction
128 | 
129 | In the next section, you build and run the application and see how it runs the three analysis models.
130 | 
131 | # Build
132 | 1.	Use a terminal window to access a command shell prompt.
133 | 
134 | 2.	Go the directory containing the Hello World files:
135 | ```
136 | cd dx_face_detection
137 | ```
138 | 3.	Source the variables:
139 | ```
140 | source /home/<user>/Downloads/fpga_support_files/setup_env.sh
141 | ```
142 | 4.	Create a directory to build the tutorial:
143 | ```
144 | mkdir build
145 | ```
146 | 5.	Go to the build directory:
147 | ```
148 | cd build
149 | ```
150 | 6.	Run CMake to set the build target and file locations:
151 | 
152 | ```
153 | cmake -DCMAKE_BUILD_TYPE=Release ..
154 | ```
155 | 7.	Build the executable:
156 | ```
157 | make
158 | ```
159 | **Alternative make command:** Run make across multiple pieces of hardware to speed the process
160 | ```
161 | make -j $(nproc)
162 | ```
163 | 8.	Load a bitstream that works well for object detection. The OpenVINO toolkit with support for FPGA includes bitstreams for Both Arria 10 FPGA cards. 
164 | 
165 | * For the Arria 10 GX Development Kit, use this command:
166 | ```
167 | aocl program acl0 /opt/intel/computer_vision_sdk_fpga_2018.3.343/a10_devkit_bitstreams/2-0-1_A10DK_FP11_ResNet50-101.aocx
168 | ```
169 | * For the Intel® Vision Accelerator Design with Intel® Arria® 10 FPGA (IEI Mustang-F100-A10), use this command:
170 | ```
171 | aocl program acl0 /opt/intel/computer_vision_sdk_2018.4.420/bitstreams/a10_vision_design_bitstreams/4-0_PL1_FP11_MobileNet_ResNet_VGG_Clamp.aocx
172 | ```
173 | 
174 | 
175 | # Run the Application
176 | 1.	Go to the main level directory:
177 | ```
178 | cd ..
179 | ```
180 | 2.	Turn the script into an executable file:
181 | ```
182 | chmod +x run_fd.sh
183 | ```
184 | ## Script Description
185 | This tutorial includes a script to select the media file, models and hardware. The script commands are provided under each step to provide the opportunity to explore other possibilities.
186 | `run_fd.sh` requires at least one hardware target, and supports up to 3.
187 | 
188 | ### How To Use:
189 | ```
190 | ./run_fd.sh (face detection hardware) (age/gender hardware) (head pose hardware)
191 | ```
192 | EXAMPLE: `./run_fd.sh fpga fpga gpu`
193 | 
194 | **Choose a hardware component for each argument you use**
195 | 1. cpu 
196 | 2. gpu 
197 | 3. fpga 
198 | 
199 | **Targets (in order on command line):**
200 | 1. 1st argument is required, for face detection
201 | 2. 2nd argument, optional, for age & gender recognition
202 | 3. 3rd argument, optional, requires face detection + age/gender recognition, for head pose
203 | 
204 | You will see rectangles and the head pose axes that follow the faces around the image (if the faces move), accompanied by age and gender results for the faces, and the timing statistics for processing each frame of the video.
205 | 
206 | When the video finishes, press any key to close the window and see statistics of the trial.
207 | 
208 | 
209 | ## Example 1 - Run face detection on targeted hardware (CPU):
210 | ```
211 | ./run_fd.sh cpu
212 | ```
213 | 
214 | **Full command**
215 | ```
216 | build/intel64/Release/face_detection_tutorial -i ../Videos/head-pose-face-detection-female-and-male.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-retail-0004/FP32/face-detection-retail-0004.xml -d CPU
217 | ```
218 | 
219 | 
220 | ## Example 2 - Run face detection on targeted hardware (FPGA):
221 |  ```
222 |  ./run_fd.sh fpga
223 |  ```
224 | 
225 | **Full command**
226 | ```
227 | build/intel64/Release/face_detection_tutorial -i ../Videos/head-pose-face-detection-female-and-male.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-retail-0004/FP16/face-detection-retail-0004.xml -d HETERO:FPGA,CPU
228 | ```
229 | 
230 | ## Example 3 - Run face detection on FPGA with age/gender recognition on GPU
231 | ```
232 | ./run_fd.sh fpga gpu
233 | ```
234 | 
235 | **Full command**
236 | ```
237 | build/intel64/Release/face_detection_tutorial -i ../Videos/head-pose-face-detection-female-and-male.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-retail-0004/FP16/face-detection-retail-0004.xml -m_ag /opt/intel/computer_vision_sdk/deployment_tools/intel_models/age-gender-recognition-retail-0013/FP16/age-gender-recognition-retail-0013.xml -d HETERO:FPGA,CPU -d_ag GPU
238 | ```
239 | 
240 | ## Example 4 - Run face detection on FPGA, age/gender recognition on a GPU, and head pose estimation on a CPU
241 | ```
242 | ./run_fd.sh fpga gpu cpu
243 | ```
244 | 
245 | **Full command**
246 | ```
247 | build/intel64/Release/face_detection_tutorial -i ../Videos/head-pose-face-detection-female-and-male.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-retail-0004/FP16/face-detection-retail-0004.xml -m_ag /opt/intel/computer_vision_sdk/deployment_tools/intel_models/age-gender-recognition-retail-0013/FP16/age-gender-recognition-retail-0013.xml -m_hp /opt/intel/computer_vision_sdk/deployment_tools/intel_models/head-pose-estimation-adas-0001/FP32/head-pose-estimation-adas-0001.xml -d HETERO:FPGA,CPU -d_ag GPU -d_hp CPU
248 | ```
249 | 
250 | 
251 | ## Example 5 - Run everything on cpu
252 | ```
253 | ./run_fd.sh cpu cpu cpu
254 | ```
255 | 
256 | **Full command**
257 | ```
258 | build/intel64/Release/face_detection_tutorial -i ../Videos/head-pose-face-detection-female-and-male.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-retail-0004/FP32/face-detection-retail-0004.xml -m_ag /opt/intel/computer_vision_sdk/deployment_tools/intel_models/age-gender-recognition-retail-0013/FP32/age-gender-recognition-retail-0013.xml -m_hp /opt/intel/computer_vision_sdk/deployment_tools/intel_models/head-pose-estimation-adas-0001/FP32/head-pose-estimation-adas-0001.xml -d CPU -d_ag CPU -d_hp CPU
259 | ```
260 | 
261 | **NOTE:** The FPGA plugin does NOT support the head pose model.  If specified, it will be replaced with CPU.
262 | 
263 | This completes the application tutorial.  For more information on using the OpenVINO™ toolkit in your environment, see the documentation here: https://software.intel.com/openvino-toolkit
264 | 


--------------------------------------------------------------------------------
/Videos/head-pose-face-detection-female-and-male.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/fritzboyle/openvino-with-fpga-hello-world-face-detection/bc5268f8f4722a95cca2ae6f5e130ec39623d43c/Videos/head-pose-face-detection-female-and-male.mp4


--------------------------------------------------------------------------------
/dx_face_detection/CMakeLists.txt:
--------------------------------------------------------------------------------
 1 | # Copyright (c) 2018 Intel Corporation
 2 | 
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | 
 7 | #      http://www.apache.org/licenses/LICENSE-2.0
 8 | 
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | cmake_minimum_required(VERSION 2.8)
15 | 
16 | # Common to all tutorial sections
17 | include(cmake/CMakeLists_common.cmake)
18 | 
19 | set(TARGET_NAME "face_detection_tutorial")
20 | 
21 | find_package(OpenCV)
22 | if(OpenCV_FOUND)
23 |     message(STATUS "OPENCV is enabled")
24 |     message(STATUS "OpenCV_INCLUDE_DIRS=${OpenCV_INCLUDE_DIRS}")
25 |     message(STATUS "OpenCV_LIBS=${OpenCV_LIBS}")
26 | else()
27 |     message(STATUS "OPENCV is disabled or not found, " ${TARGET_NAME} " skiped")
28 |     return()
29 | endif()
30 | 
31 | if( BUILD_SAMPLE_NAME AND NOT ${BUILD_SAMPLE_NAME} STREQUAL ${TARGET_NAME} )
32 |     message(STATUS "SAMPLE ${TARGET_NAME} SKIPPED")
33 |     return()
34 | endif()
35 | 
36 | file (GLOB MAIN_SRC
37 |         ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp
38 |         )
39 | 
40 | file (GLOB MAIN_HEADERS
41 |         ${CMAKE_CURRENT_SOURCE_DIR}/*.h
42 |         )
43 | 
44 | # Create named folders for the sources within the .vcproj
45 | # Empty name lists them directly under the .vcproj
46 | source_group("src" FILES ${MAIN_SRC})
47 | source_group("include" FILES ${MAIN_HEADERS})
48 | 
49 | include_directories (
50 | 	${OpenCV_INCLUDE_DIRS}
51 | 	#/opt/intel/computer_vision_sdk/inference_engine/src/extension
52 | 	/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/src/extension
53 | 	/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/include/cpp/
54 | )
55 | link_directories(
56 | 	${LIB_FOLDER}
57 | 	/opt/intel/computer_vision_sdk_2018.4.420/deployment_tools/inference_engine/lib/ubuntu_16.04/intel64
58 | )
59 | 
60 | # Create library file from sources.
61 | add_executable(${TARGET_NAME} ${MAIN_SRC} ${MAIN_HEADERS})
62 | 
63 | add_dependencies(${TARGET_NAME} gflags)
64 | 
65 | set_target_properties(${TARGET_NAME} PROPERTIES "CMAKE_CXX_FLAGS" "${CMAKE_CXX_FLAGS} -fPIE"
66 | COMPILE_PDB_NAME ${TARGET_NAME})
67 | 
68 | 
69 | target_link_libraries(${TARGET_NAME} format_reader ${IE_LIBRARIES} gflags)
70 | 
71 | if(UNIX)
72 |     target_link_libraries( ${TARGET_NAME} inference_engine  cpu_extension_avx2 ${LIB_DL} pthread ${OpenCV_LIBRARIES})
73 | endif()
74 | 


--------------------------------------------------------------------------------
/dx_face_detection/cmake/CMakeLists_common.cmake:
--------------------------------------------------------------------------------
  1 | # Copyright (c) 2018 Intel Corporation
  2 | 
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | 
  7 | #      http://www.apache.org/licenses/LICENSE-2.0
  8 | 
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | cmake_minimum_required (VERSION 2.8)
 16 | 
 17 | project(face_detection_tutorial)
 18 | 
 19 | # environment variable InferenceEngine_DIR is used to find CMake files in installation 
 20 | # point to common pieces in Inference Engine's samples directory
 21 | if("$ENV{InferenceEngine_DIR}" STREQUAL "")
 22 |     message(FATAL_ERROR "Environment variable 'InferenceEngine_DIR' is not defined.  Before running CMake, please be sure to source the setupvars.sh in the OpenVINO installation directory using the command:\n\tsource /opt/intel/computer_vision_sdk/bin/setupvars.sh\n")
 23 | endif()
 24 | 
 25 | message(STATUS "InferenceEngine_DIR=$ENV{InferenceEngine_DIR}")
 26 | if(NOT(UNIX))
 27 | 	get_filename_component(InferenceEngine_Samples_DIR "$ENV{InferenceEngine_DIR}/../samples" ABSOLUTE)
 28 | else()
 29 | 	set(InferenceEngine_Samples_DIR "$ENV{InferenceEngine_DIR}/../samples" )
 30 | endif()
 31 | 
 32 | list (APPEND CMAKE_MODULE_PATH ${InferenceEngine_Samples_DIR}/cmake)
 33 | message(STATUS "CMAKE_MODULE_PATH=${CMAKE_MODULE_PATH}")
 34 | 
 35 | get_filename_component(CMAKE_PREFIX_PATH "${CMAKE_SOURCE_DIR}/../share" ABSOLUTE)
 36 | 
 37 | message(STATUS "Looking for inference engine configuration file at: ${CMAKE_PREFIX_PATH}")
 38 | find_package(InferenceEngine 1.0)
 39 | 
 40 | if (NOT InferenceEngine_FOUND)
 41 |     message(FATAL_ERROR "")
 42 | endif()
 43 | 
 44 | 
 45 | if("${CMAKE_BUILD_TYPE}" STREQUAL "")
 46 |     message(STATUS "CMAKE_BUILD_TYPE not defined, 'Release' will be used")
 47 |     set(CMAKE_BUILD_TYPE "Release")
 48 | endif()
 49 | 
 50 | if (NOT(BIN_FOLDER))
 51 |     if("${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
 52 |         set (ARCH intel64)
 53 |     else()
 54 |         set (ARCH ia32)
 55 |     endif()
 56 | 
 57 |     set (BIN_FOLDER ${ARCH})
 58 | endif()
 59 | 
 60 | if (NOT (IE_MAIN_SOURCE_DIR))
 61 |     set(NEED_EXTENSIONS TRUE)
 62 | #    if (WIN32)
 63 | #        set (IE_MAIN_SOURCE_DIR ${CMAKE_SOURCE_DIR}/../bin/)
 64 | #    else()
 65 |         set (IE_MAIN_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR})
 66 | #    endif()
 67 | endif()
 68 | 
 69 | if(NOT(UNIX))
 70 |     set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 71 |     set (CMAKE_LIBRARY_PATH ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 72 |     set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 73 |     set (CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 74 |     set (CMAKE_PDB_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 75 |     set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 76 |     set (LIBRARY_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER})
 77 |     set (LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}) # compatibility issue: linux uses LIBRARY_OUTPUT_PATH, windows uses LIBRARY_OUTPUT_DIRECTORY
 78 | else ()
 79 |     set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE}/lib)
 80 |     set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE}/lib)
 81 |     set (CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE})
 82 |     set (CMAKE_PDB_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE})
 83 |     set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE})
 84 |     set (LIBRARY_OUTPUT_DIRECTORY ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${CMAKE_BUILD_TYPE}/lib)
 85 |     set (LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}/lib)
 86 | endif()
 87 | 
 88 | set(CMAKE_CXX_FLAGS "-std=c++11 ${CMAKE_CXX_FLAGS}")
 89 | if (WIN32)
 90 |     if(NOT "${CMAKE_GENERATOR}" MATCHES "(Win64|IA64)")
 91 |         message(FATAL_ERROR "Only 64-bit supported on Windows")
 92 |     endif()
 93 | 
 94 |     set_property(DIRECTORY APPEND PROPERTY COMPILE_DEFINITIONS _CRT_SECURE_NO_WARNINGS)
 95 |     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_SCL_SECURE_NO_WARNINGS -DNOMINMAX")
 96 |     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHsc") #no asynchronous structured exception handling
 97 |     set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /LARGEADDRESSAWARE")
 98 |     if (ENABLE_OMP)
 99 |         find_package(OpenMP)
100 |         if (OPENMP_FOUND)
101 |             set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
102 |             set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
103 |         endif()
104 |     endif()
105 | else()
106 |     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror -Werror=return-type ")
107 |     if (APPLE)
108 |         set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=unused-command-line-argument")
109 |     elseif(UNIX)
110 |         set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wuninitialized -Winit-self -Wmaybe-uninitialized")
111 |     endif()
112 | endif()
113 | 
114 | include(feature_defs OPTIONAL)
115 | 
116 | # Find OpenCV libray if exists
117 | find_package(OpenCV)
118 | include_directories(${OpenCV_INCLUDE_DIRS})
119 | if(OpenCV_FOUND)
120 |     add_definitions(-DUSE_OPENCV)
121 | endif()
122 | 
123 | ####################################
124 | ## to use C++11
125 | set (CMAKE_CXX_STANDARD 11)
126 | set (CMAKE_CXX_STANDARD_REQUIRED ON)
127 | set(CMAKE_CXX_FLAGS "-std=c++11 ${CMAKE_CXX_FLAGS}")
128 | ####################################
129 | 
130 | # Make sure dependencies are present
131 | set(IE_SAMPLES_GFLAGS_DIR "${InferenceEngine_Samples_DIR}/thirdparty/gflags")
132 | set(IE_SAMPLES_FORMAT_READER_DIR "${InferenceEngine_Samples_DIR}/common/format_reader")
133 | 
134 | if(NOT EXISTS "${IE_SAMPLES_GFLAGS_DIR}/CMakeLists.txt")
135 |     message(FATAL_ERROR "The required 'gflags' library was not found in the Inference Engine's samples at: ${IE_SAMPLES_GFLAGS_DIR}")
136 | endif()
137 | if(NOT EXISTS "${IE_SAMPLES_FORMAT_READER_DIR}/CMakeLists.txt")
138 |     message(FATAL_ERROR "The required 'format_reader' library was not found in the Inference Engine's samples at: ${IE_SAMPLES_GFLAGS_DIR}")
139 | endif()
140 | 
141 | 
142 | set(GFLAGS_IS_SUBPROJECT TRUE)
143 | add_subdirectory(${IE_SAMPLES_GFLAGS_DIR} ${CMAKE_CURRENT_BINARY_DIR}/gflags)
144 | add_subdirectory(${IE_SAMPLES_FORMAT_READER_DIR} ${CMAKE_CURRENT_BINARY_DIR}/format_reader)
145 | 
146 | # Properties->C/C++->General->Additional Include Directories
147 | include_directories (
148 |     ${InferenceEngine_Samples_DIR}/common/format_reader
149 |     ${InferenceEngine_Samples_DIR}
150 |     ${InferenceEngine_Samples_DIR}/../include
151 |     ${InferenceEngine_Samples_DIR}/thirdparty/gflags/include
152 |     ${InferenceEngine_Samples_DIR}/common
153 | )
154 | 
155 | if (UNIX)
156 |     SET(LIB_DL dl)
157 | endif()
158 | 
159 | #add_subdirectory(object_detection_sample)
160 | #add_subdirectory(interactive_face_detection_sample)
161 | #add_subdirectory(security_barrier_camera_sample)
162 | #add_subdirectory(object_detection_demo_ssd_async)
163 | #add_subdirectory(object_detection_sample_ssd)
164 | #add_subdirectory(classification_sample)
165 | #add_subdirectory(classification_sample_async)
166 | #add_subdirectory(hello_classification)
167 | #add_subdirectory(hello_request_classification)
168 | #add_subdirectory(segmentation_sample)
169 | #add_subdirectory(style_transfer_sample)
170 | 
171 | if (NEED_EXTENSIONS)
172 |     #add_subdirectory(${InferenceEngine_Samples_DIR}/src/extension ${CMAKE_CURRENT_BINARY_DIR}/extension)
173 | 	#add_subdirectory("/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/src/extension/" ${CMAKE_CURRENT_BINARY_DIR}/extension)
174 | endif()
175 | 
176 | #if (OpenCV_FOUND)
177 | #    add_subdirectory(validation_app)
178 | #else()
179 | #    message(STATUS "Validation app build is switched off")
180 | #endif()
181 | 


--------------------------------------------------------------------------------
/dx_face_detection/face_detection.hpp:
--------------------------------------------------------------------------------
  1 | /*
  2 | // Copyright (c) 2018 Intel Corporation
  3 | //
  4 | // Licensed under the Apache License, Version 2.0 (the "License");
  5 | // you may not use this file except in compliance with the License.
  6 | // You may obtain a copy of the License at
  7 | //
  8 | //      http://www.apache.org/licenses/LICENSE-2.0
  9 | //
 10 | // Unless required by applicable law or agreed to in writing, software
 11 | // distributed under the License is distributed on an "AS IS" BASIS,
 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 13 | // See the License for the specific language governing permissions and
 14 | // limitations under the License.
 15 | */
 16 | 
 17 | ///////////////////////////////////////////////////////////////////////////////////////////////////
 18 | #pragma once
 19 | 
 20 | #include <string>
 21 | #include <vector>
 22 | #include <gflags/gflags.h>
 23 | #include <iostream>
 24 | 
 25 | #ifdef _WIN32
 26 | #include <os/windows/w_dirent.h>
 27 | #else
 28 | #include <dirent.h>
 29 | #endif
 30 | 
 31 | /// @brief message for help argument
 32 | static const char help_message[] = "Print a usage message.";
 33 | 
 34 | /// @brief message for images argument
 35 | static const char video_message[] = "Optional. Path to an video file. Default value is \"cam\" to work with camera.";
 36 | 
 37 | /// @brief message for model argument
 38 | static const char face_detection_model_message[] = "Required. Path to an .xml file with a trained face detection model.";
 39 | static const char age_gender_model_message[] = "Optional. Path to an .xml file with a trained age gender model.";
 40 | static const char head_pose_model_message[] = "Optional. Path to an .xml file with a trained head pose model.";
 41 | 
 42 | /// @brief message for plugin argument
 43 | static const char plugin_message[] = "Plugin name. For example MKLDNNPlugin. If this parameter is pointed, " \
 44 | "the sample will look for this plugin only.";
 45 | 
 46 | /// @brief message for assigning face detection calculation to device
 47 | static const char target_device_message[] = "Specify the target device for Face Detection (CPU, GPU, FPGA, or MYRIAD. " \
 48 | "Sample will look for a suitable plugin for device specified.";
 49 | 
 50 | /// @brief message for assigning age gender calculation to device
 51 | static const char target_device_message_ag[] = "Specify the target device for Age Gender Detection (CPU, GPU, FPGA, or MYRIAD. " \
 52 | "Sample will look for a suitable plugin for device specified.";
 53 | 
 54 | /// @brief message for number of simultaneously age gender detections using dynamic batch
 55 | static const char num_batch_ag_message[] = "Specify number of maximum simultaneously processed faces for Age Gender Detection ( default is 1).";
 56 | 
 57 | /// @brief message for assigning age gender calculation to device
 58 | static const char target_device_message_hp[] = "Specify the target device for Head Pose Detection (CPU, GPU, FPGA, or MYRIAD. " \
 59 | "Sample will look for a suitable plugin for device specified.";
 60 | 
 61 | /// @brief message for number of simultaneously age gender detections using dynamic batch
 62 | static const char num_batch_hp_message[] = "Specify number of maximum simultaneously processed faces for Head Pose Detection ( default is 1).";
 63 | 
 64 | /// @brief message for performance counters
 65 | static const char performance_counter_message[] = "Enables per-layer performance report.";
 66 | 
 67 | /// @brief message for clDNN custom kernels desc
 68 | static const char custom_cldnn_message[] = "Required for clDNN (GPU)-targeted custom kernels."\
 69 | "Absolute path to the xml file with the kernels desc.";
 70 | 
 71 | /// @brief message for user library argument
 72 | static const char custom_cpu_library_message[] = "Required for MKLDNN (CPU)-targeted custom layers." \
 73 | "Absolute path to a shared library with the kernels impl.";
 74 | 
 75 | /// @brief message for probability threshold argument
 76 | static const char thresh_output_message[] = "Probability threshold for detections.";
 77 | 
 78 | /// @brief message raw output flag
 79 | static const char raw_output_message[] = "Inference results as raw values.";
 80 | 
 81 | /// @brief message no wait for keypress after input stream completed
 82 | static const char no_wait_for_keypress_message[] = "No wait for key press in the end.";
 83 | 
 84 | /// @brief message no show processed video
 85 | static const char no_show_processed_video[] = "No show processed video.";
 86 | 
 87 | 
 88 | /// \brief Define flag for showing help message <br>
 89 | DEFINE_bool(h, false, help_message);
 90 | 
 91 | /// \brief Define parameter for set image file <br>
 92 | /// It is a required parameter
 93 | DEFINE_string(i, "cam", video_message);
 94 | 
 95 | /// \brief Define parameter for face detection  model file <br>
 96 | /// It is a required parameter
 97 | DEFINE_string(m, "", face_detection_model_message);
 98 | 
 99 | /// \brief Define parameter for face detection  model file <br>
100 | /// It is a required parameter
101 | DEFINE_string(m_ag, "", age_gender_model_message);
102 | 
103 | /// \brief Define parameter for face detection  model file <br>
104 | /// It is a required parameter
105 | DEFINE_string(m_hp, "", head_pose_model_message);
106 | 
107 | /// \brief device the target device for face detection infer on <br>
108 | DEFINE_string(d, "CPU", target_device_message);
109 | 
110 | /// \brief device the target device for age gender detection on <br>
111 | DEFINE_string(d_ag, "CPU", target_device_message_ag);
112 | 
113 | /// \brief device the target device for age gender detection on <br>
114 | DEFINE_uint32(n_ag, 1, num_batch_ag_message);
115 | 
116 | /// \brief device the target device for head pose detection on <br>
117 | DEFINE_string(d_hp, "CPU", target_device_message_hp);
118 | 
119 | /// \brief device the target device for head pose detection on <br>
120 | DEFINE_uint32(n_hp, 1, num_batch_hp_message);
121 | 
122 | /// \brief Enable per-layer performance report
123 | DEFINE_bool(pc, false, performance_counter_message);
124 | 
125 | /// @brief clDNN custom kernels path <br>
126 | /// Default is ./lib
127 | DEFINE_string(c, "", custom_cldnn_message);
128 | 
129 | /// @brief Absolute path to CPU library with user layers <br>
130 | /// It is a optional parameter
131 | DEFINE_string(l, "", custom_cpu_library_message);
132 | 
133 | /// \brief Flag to output raw scoring results<br>
134 | /// It is an optional parameter
135 | DEFINE_bool(r, false, raw_output_message);
136 | 
137 | /// \brief Flag to output raw scoring results<br>
138 | /// It is an optional parameter
139 | DEFINE_double(t, 0.5, thresh_output_message);
140 | 
141 | /// \brief Flag to disable keypress exit<br>
142 | /// It is an optional parameter
143 | DEFINE_bool(no_wait, false, no_wait_for_keypress_message);
144 | 
145 | /// \brief Flag to disable processed video showing<br>
146 | /// It is an optional parameter
147 | DEFINE_bool(no_show, false, no_show_processed_video);
148 | 
149 | /**
150 | * \brief This function show a help message
151 | */
152 | static void showUsage() {
153 |     std::cout << std::endl;
154 |     std::cout << "face_detection_tutorial [OPTION]" << std::endl;
155 |     std::cout << "Options:" << std::endl;
156 |     std::cout << std::endl;
157 |     std::cout << "    -h                         " << help_message << std::endl;
158 |     std::cout << "    -i \"<path>\"                " << video_message << std::endl;
159 |     std::cout << "    -m \"<path>\"                " << face_detection_model_message<< std::endl;
160 |     std::cout << "    -m_ag \"<path>\"             " << age_gender_model_message << std::endl;
161 |     std::cout << "    -m_hp \"<path>\"             " << head_pose_model_message << std::endl;
162 |     std::cout << "      -l \"<absolute_path>\"     " << custom_cpu_library_message << std::endl;
163 |     std::cout << "          Or" << std::endl;
164 |     std::cout << "      -c \"<absolute_path>\"     " << custom_cldnn_message << std::endl;
165 |     std::cout << "    -d \"<device>\"              " << target_device_message << std::endl;
166 |     std::cout << "    -d_ag \"<device>\"           " << target_device_message_ag << std::endl;
167 |     std::cout << "    -d_hp \"<device>\"           " << target_device_message_hp << std::endl;
168 |     std::cout << "    -n_ag \"<num>\"              " << num_batch_ag_message << std::endl;
169 |     std::cout << "    -n_hp \"<num>\"              " << num_batch_hp_message << std::endl;
170 |     std::cout << "    -no_wait                   " << no_wait_for_keypress_message << std::endl;
171 |     std::cout << "    -no_show                   " << no_show_processed_video << std::endl;
172 |     std::cout << "    -pc                        " << performance_counter_message << std::endl;
173 |     std::cout << "    -r                         " << raw_output_message << std::endl;
174 |     std::cout << "    -t                         " << thresh_output_message << std::endl;
175 | }
176 | 


--------------------------------------------------------------------------------
/dx_face_detection/main.cpp:
--------------------------------------------------------------------------------
  1 | /*
  2 | // Copyright (c) 2018 Intel Corporation
  3 | //
  4 | // Licensed under the Apache License, Version 2.0 (the "License");
  5 | // you may not use this file except in compliance with the License.
  6 | // You may obtain a copy of the License at
  7 | //
  8 | //      http://www.apache.org/licenses/LICENSE-2.0
  9 | //
 10 | // Unless required by applicable law or agreed to in writing, software
 11 | // distributed under the License is distributed on an "AS IS" BASIS,
 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 13 | // See the License for the specific language governing permissions and
 14 | // limitations under the License.
 15 | */
 16 | 
 17 | /**
 18 | * \brief The entry point for the Inference Engine interactive_face_detection sample application
 19 | * \file object_detection_sample_ssd/main.cpp
 20 | * \example object_detection_sample_ssd/main.cpp
 21 | */
 22 | #include <gflags/gflags.h>
 23 | #include <functional>
 24 | #include <iostream>
 25 | #include <fstream>
 26 | #include <random>
 27 | #include <memory>
 28 | #include <chrono>
 29 | #include <vector>
 30 | #include <string>
 31 | #include <utility>
 32 | #include <algorithm>
 33 | #include <iterator>
 34 | #include <map>
 35 | 
 36 | #include <inference_engine.hpp>
 37 | 
 38 | #include <samples/common.hpp>
 39 | #include <samples/slog.hpp>
 40 | 
 41 | #include "face_detection.hpp"
 42 | //#include "mkldnn/mkldnn_extension_ptr.hpp"		// deprecated 4.20
 43 | #include <ext_list.hpp>
 44 | 
 45 | 
 46 | #include <opencv2/opencv.hpp>
 47 | #include <opencv2/videoio/videoio_c.h>				// added for 4.20, defs 
 48 | #include <ie_precision.hpp>							// added for 4.20
 49 | #include <ie_cnn_net_reader.h>						// added for 4.20
 50 | 
 51 | using namespace InferenceEngine;
 52 | 
 53 | bool ParseAndCheckCommandLine(int argc, char *argv[]) {
 54 |     // ---------------------------Parsing and validation of input args--------------------------------------
 55 |     gflags::ParseCommandLineNonHelpFlags(&argc, &argv, true);
 56 |     if (FLAGS_h) {
 57 |         showUsage();
 58 |         return false;
 59 |     }
 60 |     slog::info << "Parsing input parameters" << slog::endl;
 61 | 
 62 |     if (FLAGS_i.empty()) {
 63 |         throw std::logic_error("Parameter -i is not set");
 64 |     }
 65 | 
 66 |     if (FLAGS_m.empty()) {
 67 |         throw std::logic_error("Parameter -m is not set");
 68 |     }
 69 | 
 70 |     if (FLAGS_n_ag < 1) {
 71 |         throw std::logic_error("Parameter -n_ag cannot be 0");
 72 |     }
 73 | 
 74 |     if (FLAGS_n_hp < 1) {
 75 |         throw std::logic_error("Parameter -n_hp cannot be 0");
 76 |     }
 77 | 
 78 |     return true;
 79 | }
 80 | 
 81 | // -------------------------Generic routines for detection networks-------------------------------------------------
 82 | 
 83 | struct BaseDetection {
 84 |     ExecutableNetwork net;
 85 |     InferenceEngine::InferencePlugin * plugin = NULL;
 86 |     InferRequest::Ptr request;
 87 |     std::string & commandLineFlag;
 88 |     std::string topoName;
 89 |     const int maxBatch;
 90 | 
 91 |     BaseDetection(std::string &commandLineFlag, std::string topoName, int maxBatch)
 92 |         : commandLineFlag(commandLineFlag), topoName(topoName), maxBatch(maxBatch) {}
 93 | 
 94 |     virtual ~BaseDetection() {}
 95 | 
 96 |     ExecutableNetwork* operator ->() {
 97 |         return &net;
 98 |     }
 99 |     virtual InferenceEngine::CNNNetwork read()  = 0;
100 | 
101 |     virtual void submitRequest() {
102 |         if (!enabled() || request == nullptr) return;
103 |         request->StartAsync();
104 |     }
105 | 
106 |     virtual void wait() {
107 |         if (!enabled()|| !request) return;
108 |         request->Wait(IInferRequest::WaitMode::RESULT_READY);
109 |     }
110 |     mutable bool enablingChecked = false;
111 |     mutable bool _enabled = false;
112 | 
113 |     bool enabled() const  {
114 |         if (!enablingChecked) {
115 |             _enabled = !commandLineFlag.empty();
116 |             if (!_enabled) {
117 |                 slog::info << topoName << " DISABLED" << slog::endl;
118 |             }
119 |             enablingChecked = true;
120 |         }
121 |         return _enabled;
122 |     }
123 |     void printPerformanceCounts() {
124 |         if (!enabled()) {
125 |             return;
126 |         }
127 |         slog::info << "Performance counts for " << topoName << slog::endl << slog::endl;
128 |         ::printPerformanceCounts(request->GetPerformanceCounts(), std::cout, false);
129 |     }
130 | };
131 | 
132 | struct FaceDetectionClass : BaseDetection {
133 |     std::string input;
134 |     std::string output;
135 |     int maxProposalCount = 0;
136 |     int objectSize = 0;
137 |     int enquedFrames = 0;
138 |     float width = 0;
139 |     float height = 0;
140 |     bool resultsFetched = false;
141 |     std::vector<std::string> labels;
142 |     using BaseDetection::operator=;
143 | 
144 |     struct Result {
145 |         int label;
146 |         float confidence;
147 |         cv::Rect location;
148 |     };
149 | 
150 |     std::vector<Result> results;
151 | 
152 |     void submitRequest() override {
153 |         if (!enquedFrames) return;
154 |         enquedFrames = 0;
155 |         resultsFetched = false;
156 |         results.clear();
157 |         BaseDetection::submitRequest();
158 |     }
159 | 
160 |     void enqueue(const cv::Mat &frame) {
161 |         if (!enabled()) return;
162 | 
163 |         if (!request) {
164 |             request = net.CreateInferRequestPtr();
165 |         }
166 | 
167 |         width = frame.cols;
168 |         height = frame.rows;
169 | 
170 |         auto  inputBlob = request->GetBlob(input);
171 | 
172 |         matU8ToBlob<uint8_t >(frame, inputBlob);
173 | 		enquedFrames = 1;
174 |     }
175 | 
176 | 
177 |     FaceDetectionClass() : BaseDetection(FLAGS_m, "Face Detection", 1) {}
178 |     InferenceEngine::CNNNetwork read() override {
179 |         slog::info << "Loading network files for Face Detection" << slog::endl;
180 |         InferenceEngine::CNNNetReader netReader;
181 |         /** Read network model **/
182 |         netReader.ReadNetwork(FLAGS_m);
183 |         /** Set batch size to 1 **/
184 |         slog::info << "Batch size is set to  "<< maxBatch << slog::endl;
185 |         netReader.getNetwork().setBatchSize(maxBatch);
186 |         /** Extract model name and load it's weights **/
187 |         std::string binFileName = fileNameNoExt(FLAGS_m) + ".bin";
188 |         netReader.ReadWeights(binFileName);
189 |         /** Read labels (if any)**/
190 |         std::string labelFileName = fileNameNoExt(FLAGS_m) + ".labels";
191 | 
192 |         std::ifstream inputFile(labelFileName);
193 |         std::copy(std::istream_iterator<std::string>(inputFile),
194 |                   std::istream_iterator<std::string>(),
195 |                   std::back_inserter(labels));
196 |         // -----------------------------------------------------------------------------------------------------
197 | 
198 |         /** SSD-based network should have one input and one output **/
199 |         // ---------------------------Check inputs ------------------------------------------------------
200 |         slog::info << "Checking Face Detection inputs" << slog::endl;
201 |         InferenceEngine::InputsDataMap inputInfo(netReader.getNetwork().getInputsInfo());
202 |         if (inputInfo.size() != 1) {
203 |             throw std::logic_error("Face Detection network should have only one input");
204 |         }
205 |         auto& inputInfoFirst = inputInfo.begin()->second;
206 |         inputInfoFirst->setPrecision(Precision::U8);
207 |         inputInfoFirst->getInputData()->setLayout(Layout::NCHW);
208 |         // -----------------------------------------------------------------------------------------------------
209 | 
210 |         // ---------------------------Check outputs ------------------------------------------------------
211 |         slog::info << "Checking Face Detection outputs" << slog::endl;
212 |         InferenceEngine::OutputsDataMap outputInfo(netReader.getNetwork().getOutputsInfo());
213 |         if (outputInfo.size() != 1) {
214 |             throw std::logic_error("Face Detection network should have only one output");
215 |         }
216 |         auto& _output = outputInfo.begin()->second;
217 |         output = outputInfo.begin()->first;
218 | 
219 |         const auto outputLayer = netReader.getNetwork().getLayerByName(output.c_str());
220 |         if (outputLayer->type != "DetectionOutput") {
221 |             throw std::logic_error("Face Detection network output layer(" + outputLayer->name +
222 |                 ") should be DetectionOutput, but was " +  outputLayer->type);
223 |         }
224 | 
225 |         if (outputLayer->params.find("num_classes") == outputLayer->params.end()) {
226 |             throw std::logic_error("Face Detection network output layer (" +
227 |                 output + ") should have num_classes integer attribute");
228 |         }
229 | 
230 |         const int num_classes = outputLayer->GetParamAsInt("num_classes");
231 |         if (labels.size() != num_classes) {
232 |             if (labels.size() == (num_classes - 1))  // if network assumes default "background" class, having no label
233 |                 labels.insert(labels.begin(), "fake");
234 |             else
235 |                 labels.clear();
236 |         }
237 |         const InferenceEngine::SizeVector outputDims = _output->dims;
238 |         maxProposalCount = outputDims[1];
239 |         objectSize = outputDims[0];
240 |         if (objectSize != 7) {
241 |             throw std::logic_error("Face Detection network output layer should have 7 as a last dimension");
242 |         }
243 |         if (outputDims.size() != 4) {
244 |             throw std::logic_error("Face Detection network output dimensions not compatible shoulld be 4, but was " +
245 |                                            std::to_string(outputDims.size()));
246 |         }
247 |         _output->setPrecision(Precision::FP32);
248 |         _output->setLayout(Layout::NCHW);
249 | 
250 |         slog::info << "Loading Face Detection model to the "<< FLAGS_d << " plugin" << slog::endl;
251 |         input = inputInfo.begin()->first;
252 |         return netReader.getNetwork();
253 |     }
254 | 
255 |     void fetchResults() {
256 |         if (!enabled()) return;
257 |         results.clear();
258 |         if (resultsFetched) return;
259 |         resultsFetched = true;
260 |         const float *detections = request->GetBlob(output)->buffer().as<float *>();
261 | 
262 |         for (int i = 0; i < maxProposalCount; i++) {
263 |             float image_id = detections[i * objectSize + 0];
264 |             Result r;
265 |             r.label = static_cast<int>(detections[i * objectSize + 1]);
266 |             r.confidence = detections[i * objectSize + 2];
267 |             if (r.confidence <= FLAGS_t) {
268 |                 continue;
269 |             }
270 | 
271 |             r.location.x = detections[i * objectSize + 3] * width;
272 |             r.location.y = detections[i * objectSize + 4] * height;
273 |             r.location.width = detections[i * objectSize + 5] * width - r.location.x;
274 |             r.location.height = detections[i * objectSize + 6] * height - r.location.y;
275 | 
276 |             if ((image_id < 0) || (image_id >= maxBatch)) {  // indicates end of detections
277 |                 break;
278 |             }
279 |             if (FLAGS_r) {
280 |                 std::cout << "[" << i << "," << r.label << "] element, prob = " << r.confidence <<
281 |                           "    (" << r.location.x << "," << r.location.y << ")-(" << r.location.width << ","
282 |                           << r.location.height << ")"
283 |                           << ((r.confidence > FLAGS_t) ? " WILL BE RENDERED!" : "") << std::endl;
284 |             }
285 | 
286 |             results.push_back(r);
287 |         }
288 |     }
289 | };
290 | 
291 | struct AgeGenderDetection : BaseDetection {
292 |     std::string input;
293 |     std::string outputAge;
294 |     std::string outputGender;
295 |     int enquedFaces = 0;
296 | 
297 | 
298 |     using BaseDetection::operator=;
299 |     AgeGenderDetection() : BaseDetection(FLAGS_m_ag, "Age Gender", FLAGS_n_ag) {}
300 | 
301 |     void submitRequest() override {
302 |         if (!enquedFaces) return;
303 |         BaseDetection::submitRequest();
304 |         enquedFaces = 0;
305 |     }
306 | 
307 |     void enqueue(const cv::Mat &face) {
308 |         if (!enabled()) {
309 |             return;
310 |         }
311 |         if (enquedFaces >= maxBatch) {
312 |             slog::warn << "Number of detected faces more than maximum(" << maxBatch << ") processed by Age Gender detector" << slog::endl;
313 |             return;
314 |         }
315 |         if (!request) {
316 |             request = net.CreateInferRequestPtr();
317 |         }
318 | 
319 |         auto  inputBlob = request->GetBlob(input);
320 | 
321 |         matU8ToBlob<float>(face, inputBlob, enquedFaces);
322 |         enquedFaces++;
323 |     }
324 | 
325 |     struct Result { float age; float maleProb;};
326 |     Result operator[] (int idx) const {
327 |         auto  genderBlob = request->GetBlob(outputGender);
328 |         auto  ageBlob    = request->GetBlob(outputAge);
329 | 
330 |         return {ageBlob->buffer().as<float*>()[idx] * 100,
331 |                 genderBlob->buffer().as<float*>()[idx * 2 + 1]};
332 |     }
333 | 
334 |     CNNNetwork read() override {
335 |         slog::info << "Loading network files for AgeGender" << slog::endl;
336 |         InferenceEngine::CNNNetReader netReader;
337 |         /** Read network model **/
338 |         netReader.ReadNetwork(FLAGS_m_ag);
339 | 
340 |         /** Set batch size **/
341 |         netReader.getNetwork().setBatchSize(maxBatch);
342 |         slog::info << "Batch size is set to " << netReader.getNetwork().getBatchSize() << " for Age Gender" << slog::endl;
343 | 
344 | 
345 |         /** Extract model name and load it's weights **/
346 |         std::string binFileName = fileNameNoExt(FLAGS_m_ag) + ".bin";
347 |         netReader.ReadWeights(binFileName);
348 | 
349 |         // -----------------------------------------------------------------------------------------------------
350 | 
351 |         /** Age Gender network should have one input two outputs **/
352 |         // ---------------------------Check inputs ------------------------------------------------------
353 |         slog::info << "Checking Age Gender inputs" << slog::endl;
354 |         InferenceEngine::InputsDataMap inputInfo(netReader.getNetwork().getInputsInfo());
355 |         if (inputInfo.size() != 1) {
356 |             throw std::logic_error("Age gender topology should have only one input");
357 |         }
358 |         auto& inputInfoFirst = inputInfo.begin()->second;
359 |         inputInfoFirst->setPrecision(Precision::FP32);
360 |         inputInfoFirst->getInputData()->setLayout(Layout::NCHW);
361 |         input = inputInfo.begin()->first;
362 |         // -----------------------------------------------------------------------------------------------------
363 | 
364 |         // ---------------------------Check outputs ------------------------------------------------------
365 |         slog::info << "Checking Age Gender outputs" << slog::endl;
366 |         InferenceEngine::OutputsDataMap outputInfo(netReader.getNetwork().getOutputsInfo());
367 |         if (outputInfo.size() != 2) {
368 |             throw std::logic_error("Age Gender network should have two output layers");
369 |         }
370 |         auto it = outputInfo.begin();
371 |         auto ageOutput = (it++)->second;
372 |         auto genderOutput = (it++)->second;
373 | 
374 |         // if gender output is convolution, it can be swapped with age
375 |         if (genderOutput->getCreatorLayer().lock()->type == "Convolution") {
376 |             std::swap(ageOutput, genderOutput);
377 |         }
378 | 
379 |         if (ageOutput->getCreatorLayer().lock()->type != "Convolution") {
380 |             throw std::logic_error("In Age Gender network, age layer (" + ageOutput->getCreatorLayer().lock()->name +
381 |                 ") should be a Convolution, but was: " + ageOutput->getCreatorLayer().lock()->type);
382 |         }
383 | 
384 |         if (genderOutput->getCreatorLayer().lock()->type != "SoftMax") {
385 |             throw std::logic_error("In Age Gender network, gender layer (" + genderOutput->getCreatorLayer().lock()->name +
386 |                 ") should be a SoftMax, but was: " + genderOutput->getCreatorLayer().lock()->type);
387 |         }
388 |         slog::info << "Age layer: " << ageOutput->getCreatorLayer().lock()->name<< slog::endl;
389 |         slog::info << "Gender layer: " << genderOutput->getCreatorLayer().lock()->name<< slog::endl;
390 | 
391 |         outputAge = ageOutput->name;
392 |         outputGender = genderOutput->name;
393 | 
394 |         slog::info << "Loading Age Gender model to the "<< FLAGS_d_ag << " plugin" << slog::endl;
395 |         _enabled = true;
396 |         return netReader.getNetwork();
397 |     }
398 | };
399 | 
400 | struct HeadPoseDetection : BaseDetection {
401 |     std::string input;
402 |     std::string outputAngleR = "angle_r_fc";
403 |     std::string outputAngleP = "angle_p_fc";
404 |     std::string outputAngleY = "angle_y_fc";
405 |     int enquedFaces = 0;
406 |     cv::Mat cameraMatrix;
407 |     HeadPoseDetection() : BaseDetection(FLAGS_m_hp, "Head Pose", FLAGS_n_hp) {}
408 | 
409 |     void submitRequest() override {
410 |         if (!enquedFaces) return;
411 |         BaseDetection::submitRequest();
412 |         enquedFaces = 0;
413 |     }
414 | 
415 |     void enqueue(const cv::Mat &face) {
416 |         if (!enabled()) {
417 |             return;
418 |         }
419 |         if (enquedFaces == maxBatch) {
420 |             slog::warn << "Number of detected faces more than maximum(" << maxBatch << ") processed by Head Pose detector" << slog::endl;
421 |             return;
422 |         }
423 |         if (!request) {
424 |             request = net.CreateInferRequestPtr();
425 |         }
426 | 
427 |         auto  inputBlob = request->GetBlob(input);
428 | 
429 |         matU8ToBlob<float>(face, inputBlob, enquedFaces);
430 |         enquedFaces++;
431 |     }
432 | 
433 |     struct Results {
434 |         float angle_r;
435 |         float angle_p;
436 |         float angle_y;
437 |     };
438 | 
439 |     Results operator[] (int idx) const {
440 |         auto  angleR = request->GetBlob(outputAngleR);
441 |         auto  angleP = request->GetBlob(outputAngleP);
442 |         auto  angleY = request->GetBlob(outputAngleY);
443 | 
444 |         return {angleR->buffer().as<float*>()[idx],
445 |                 angleP->buffer().as<float*>()[idx],
446 |                 angleY->buffer().as<float*>()[idx]};
447 |     }
448 | 
449 |     CNNNetwork read() override {
450 |         slog::info << "Loading network files for Head Pose detection " << slog::endl;
451 |         InferenceEngine::CNNNetReader netReader;
452 |         /** Read network model **/
453 |         netReader.ReadNetwork(FLAGS_m_hp);
454 |         /** Set batch size to maximum currently set to one provided from command line **/
455 |         netReader.getNetwork().setBatchSize(maxBatch);
456 |         netReader.getNetwork().setBatchSize(maxBatch);
457 |         slog::info << "Batch size is set to  " << netReader.getNetwork().getBatchSize() << " for Head Pose Network" << slog::endl;
458 |         /** Extract model name and load it's weights **/
459 |         std::string binFileName = fileNameNoExt(FLAGS_m_hp) + ".bin";
460 |         netReader.ReadWeights(binFileName);
461 | 
462 | 
463 |         // ---------------------------Check inputs ------------------------------------------------------
464 |         slog::info << "Checking Head Pose Network inputs" << slog::endl;
465 |         InferenceEngine::InputsDataMap inputInfo(netReader.getNetwork().getInputsInfo());
466 |         if (inputInfo.size() != 1) {
467 |             throw std::logic_error("Head Pose topology should have only one input");
468 |         }
469 |         auto& inputInfoFirst = inputInfo.begin()->second;
470 |         inputInfoFirst->setPrecision(Precision::FP32);
471 |         inputInfoFirst->getInputData()->setLayout(Layout::NCHW);
472 |         input = inputInfo.begin()->first;
473 |         // -----------------------------------------------------------------------------------------------------
474 | 
475 |         // ---------------------------Check outputs ------------------------------------------------------
476 |         slog::info << "Checking Head Pose network outputs" << slog::endl;
477 |         InferenceEngine::OutputsDataMap outputInfo(netReader.getNetwork().getOutputsInfo());
478 |         if (outputInfo.size() != 3) {
479 |             throw std::logic_error("Head Pose network should have 3 outputs");
480 |         }
481 |         std::map<std::string, bool> layerNames = {
482 |             {outputAngleR, false},
483 |             {outputAngleP, false},
484 |             {outputAngleY, false}
485 |         };
486 | 
487 |         for (auto && output : outputInfo) {
488 |             auto layer = output.second->getCreatorLayer().lock();
489 |             if (layerNames.find(layer->name) == layerNames.end()) {
490 |                 throw std::logic_error("Head Pose network output layer unknown: " + layer->name + ", should be " +
491 |                     outputAngleR + " or " + outputAngleP + " or " + outputAngleY);
492 |             }
493 |             if (layer->type != "FullyConnected") {
494 |                 throw std::logic_error("Head Pose network output layer (" + layer->name + ") has invalid type: " +
495 |                     layer->type + ", should be FullyConnected");
496 |             }
497 |             auto fc = dynamic_cast<FullyConnectedLayer*>(layer.get());
498 |             if (fc->_out_num != 1) {
499 |                 throw std::logic_error("Head Pose network output layer (" + layer->name + ") has invalid out-size=" +
500 |                     std::to_string(fc->_out_num) + ", should be 1");
501 |             }
502 |             layerNames[layer->name] = true;
503 |         }
504 | 
505 |         slog::info << "Loading Head Pose model to the "<< FLAGS_d_hp << " plugin" << slog::endl;
506 | 
507 |         _enabled = true;
508 |         return netReader.getNetwork();
509 |     }
510 | 
511 |     void buildCameraMatrix(int cx, int cy, float focalLength) {
512 |         if (!cameraMatrix.empty()) return;
513 |         cameraMatrix = cv::Mat::zeros(3, 3, CV_32F);
514 |         cameraMatrix.at<float>(0) = focalLength;
515 |         cameraMatrix.at<float>(2) = static_cast<float>(cx);
516 |         cameraMatrix.at<float>(4) = focalLength;
517 |         cameraMatrix.at<float>(5) = static_cast<float>(cy);
518 |         cameraMatrix.at<float>(8) = 1;
519 |     }
520 | 
521 |     void drawAxes(cv::Mat& frame, cv::Point3f cpoint, Results headPose, float scale) {
522 |         double yaw   = headPose.angle_y;
523 |         double pitch = headPose.angle_p;
524 |         double roll  = headPose.angle_r;
525 | 
526 |         if (FLAGS_r) {
527 |             std::cout << "Head pose results: yaw, pitch, roll = " << yaw << ";" << pitch << ";" << roll << std::endl;
528 |         }
529 | 
530 |         pitch *= CV_PI / 180.0;
531 |         yaw   *= CV_PI / 180.0;
532 |         roll  *= CV_PI / 180.0;
533 | 
534 |         cv::Matx33f        Rx(1,           0,            0,
535 |                               0,  cos(pitch),  -sin(pitch),
536 |                               0,  sin(pitch),  cos(pitch));
537 |         cv::Matx33f Ry(cos(yaw),           0,    -sin(yaw),
538 |                               0,           1,            0,
539 |                        sin(yaw),           0,    cos(yaw));
540 |         cv::Matx33f Rz(cos(roll), -sin(roll),            0,
541 |                        sin(roll),  cos(roll),            0,
542 |                               0,           0,            1);
543 | 
544 | 
545 |         auto r = cv::Mat(Rz*Ry*Rx);
546 |         buildCameraMatrix(frame.cols / 2, frame.rows / 2, 950.0);
547 | 
548 |         cv::Mat xAxis(3, 1, CV_32F), yAxis(3, 1, CV_32F), zAxis(3, 1, CV_32F), zAxis1(3, 1, CV_32F);
549 | 
550 |         xAxis.at<float>(0) = 1 * scale;
551 |         xAxis.at<float>(1) = 0;
552 |         xAxis.at<float>(2) = 0;
553 | 
554 |         yAxis.at<float>(0) = 0;
555 |         yAxis.at<float>(1) = -1 * scale;
556 |         yAxis.at<float>(2) = 0;
557 | 
558 |         zAxis.at<float>(0) = 0;
559 |         zAxis.at<float>(1) = 0;
560 |         zAxis.at<float>(2) = -1 * scale;
561 | 
562 |         zAxis1.at<float>(0) = 0;
563 |         zAxis1.at<float>(1) = 0;
564 |         zAxis1.at<float>(2) = 1 * scale;
565 | 
566 |         cv::Mat o(3, 1, CV_32F, cv::Scalar(0));
567 |         o.at<float>(2) = cameraMatrix.at<float>(0);
568 | 
569 |         xAxis = r * xAxis + o;
570 |         yAxis = r * yAxis + o;
571 |         zAxis = r * zAxis + o;
572 |         zAxis1 = r * zAxis1 + o;
573 | 
574 |         cv::Point p1, p2;
575 | 
576 |         p2.x = static_cast<int>((xAxis.at<float>(0) / xAxis.at<float>(2) * cameraMatrix.at<float>(0)) + cpoint.x);
577 |         p2.y = static_cast<int>((xAxis.at<float>(1) / xAxis.at<float>(2) * cameraMatrix.at<float>(4)) + cpoint.y);
578 |         cv::line(frame, cv::Point(cpoint.x, cpoint.y), p2, cv::Scalar(0, 0, 255), 2);
579 | 
580 |         p2.x = static_cast<int>((yAxis.at<float>(0) / yAxis.at<float>(2) * cameraMatrix.at<float>(0)) + cpoint.x);
581 |         p2.y = static_cast<int>((yAxis.at<float>(1) / yAxis.at<float>(2) * cameraMatrix.at<float>(4)) + cpoint.y);
582 |         cv::line(frame, cv::Point(cpoint.x, cpoint.y), p2, cv::Scalar(0, 255, 0), 2);
583 | 
584 |         p1.x = static_cast<int>((zAxis1.at<float>(0) / zAxis1.at<float>(2) * cameraMatrix.at<float>(0)) + cpoint.x);
585 |         p1.y = static_cast<int>((zAxis1.at<float>(1) / zAxis1.at<float>(2) * cameraMatrix.at<float>(4)) + cpoint.y);
586 | 
587 |         p2.x = static_cast<int>((zAxis.at<float>(0) / zAxis.at<float>(2) * cameraMatrix.at<float>(0)) + cpoint.x);
588 |         p2.y = static_cast<int>((zAxis.at<float>(1) / zAxis.at<float>(2) * cameraMatrix.at<float>(4)) + cpoint.y);
589 |         cv::line(frame, p1, p2, cv::Scalar(255, 0, 0), 2);
590 | 
591 |         cv::circle(frame, p2, 3, cv::Scalar(255, 0, 0), 2);
592 |     }
593 | };
594 | 
595 | struct Load {
596 |     BaseDetection& detector;
597 |     explicit Load(BaseDetection& detector) : detector(detector) { }
598 | 
599 |     void into(InferenceEngine::InferencePlugin & plg) const {
600 |         if (detector.enabled()) {
601 |             detector.net = plg.LoadNetwork(detector.read(), {});
602 |             detector.plugin = &plg;
603 |         }
604 |     }
605 | };
606 | 
607 | int main(int argc, char *argv[]) {
608 |     try {
609 |         /** This sample covers 3 certain topologies and cannot be generalized **/
610 |         std::cout << "InferenceEngine: " << InferenceEngine::GetInferenceEngineVersion() << std::endl;
611 | 
612 |         // ---------------------------Parsing and validation of input args--------------------------------------
613 |         if (!ParseAndCheckCommandLine(argc, argv)) {
614 |             return 0;
615 |         }
616 | 
617 |         // -----------------------------Read input -----------------------------------------------------
618 |         slog::info << "Reading input" << slog::endl;
619 |         cv::VideoCapture cap;
620 |         const bool isCamera = FLAGS_i == "cam";
621 |         if (!(FLAGS_i == "cam" ? cap.open(0) : cap.open(FLAGS_i))) {
622 |             throw std::logic_error("Cannot open input file or camera: " + FLAGS_i);
623 |         }
624 |         const size_t width  = (size_t) cap.get(CV_CAP_PROP_FRAME_WIDTH);
625 |         const size_t height = (size_t) cap.get(CV_CAP_PROP_FRAME_HEIGHT);
626 | 
627 |         // read input (video) frame
628 |         cv::Mat frame;
629 |         if (!cap.read(frame)) {
630 |             throw std::logic_error("Failed to get frame from cv::VideoCapture");
631 |         }
632 | 
633 |         // ---------------------Load plugins for inference engine------------------------------------------------
634 |         std::map<std::string, InferencePlugin> pluginsForDevices;
635 |         std::vector<std::pair<std::string, std::string>> cmdOptions = {
636 |             {FLAGS_d, FLAGS_m}, {FLAGS_d_ag, FLAGS_m_ag}, {FLAGS_d_hp, FLAGS_m_hp}
637 |         };
638 | 
639 |         FaceDetectionClass FaceDetection;
640 |         AgeGenderDetection AgeGender;
641 |         HeadPoseDetection HeadPose;
642 | 
643 | 
644 |         for (auto && option : cmdOptions) {
645 |             auto deviceName = option.first;
646 |             auto networkName = option.second;
647 | 
648 |             if (deviceName == "" || networkName == "") {
649 |                 continue;
650 |             }
651 | 
652 |             if (pluginsForDevices.find(deviceName) != pluginsForDevices.end()) {
653 |                 continue;
654 |             }
655 |             slog::info << "Loading plugin " << deviceName << slog::endl;
656 |             InferencePlugin plugin = PluginDispatcher({"../../../lib/intel64", ""}).getPluginByDevice(deviceName);
657 | 
658 |             /** Printing plugin version **/
659 |             printPluginVersion(plugin, std::cout);
660 | 
661 |             /** Load extensions for the CPU plugin **/
662 |             if ((deviceName.find("CPU") != std::string::npos)) {
663 |                 plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
664 | 
665 |                 if (!FLAGS_l.empty()) {
666 |                     // CPU(MKLDNN) extensions are loaded as a shared library and passed as a pointer to base extension
667 |                     //auto extension_ptr = make_so_pointer<InferenceEngine::MKLDNNPlugin::IMKLDNNExtension>(FLAGS_l);		// 4.20
668 |                     auto extension_ptr = make_so_pointer<IExtension>(FLAGS_l);												// 4.20
669 |                     plugin.AddExtension(std::static_pointer_cast<IExtension>(extension_ptr));
670 |                 }
671 |             } else if (!FLAGS_c.empty()) {
672 |                 // Load Extensions for other plugins not CPU
673 |                 plugin.SetConfig({ { PluginConfigParams::KEY_CONFIG_FILE, FLAGS_c } });
674 |             }
675 | 
676 |             pluginsForDevices[deviceName] = plugin;
677 |         }
678 | 
679 |         /** Per layer metrics **/
680 |         if (FLAGS_pc) {
681 |             for (auto && plugin : pluginsForDevices) {
682 |                 plugin.second.SetConfig({{PluginConfigParams::KEY_PERF_COUNT, PluginConfigParams::YES}});
683 |             }
684 |         }
685 | 
686 | 
687 |         // --------------------Load networks (Generated xml/bin files)-------------------------------------------
688 | 
689 |         Load(FaceDetection).into(pluginsForDevices[FLAGS_d]);
690 |         Load(AgeGender).into(pluginsForDevices[FLAGS_d_ag]);
691 |         Load(HeadPose).into(pluginsForDevices[FLAGS_d_hp]);
692 | 
693 | 
694 |         // ----------------------------Do inference-------------------------------------------------------------
695 |         slog::info << "Start inference " << slog::endl;
696 |         typedef std::chrono::duration<double, std::ratio<1, 1000>> ms;
697 |         std::chrono::high_resolution_clock::time_point wallclockStart, wallclockEnd;
698 | 
699 |         int totalFrames = 1;  // cap.read() above
700 |         double ocv_decode_time = 0, ocv_render_time = 0;
701 | 		float fdFpsTot = 0.0; 
702 | 		float otherTotFps = 0.0; 
703 | 
704 | 		double ocv_ttl_render = 0;
705 | 		double ocv_ttl_decode = 0;
706 | 
707 | 		wallclockStart = std::chrono::high_resolution_clock::now();
708 |         /** Start inference & calc performance **/
709 |         while (true) {
710 |         	double secondDetection = 0;
711 | 
712 |             /** requesting new frame if any*/
713 |             cap.grab();
714 | 
715 |             auto t0 = std::chrono::high_resolution_clock::now();
716 |             FaceDetection.enqueue(frame);
717 |             auto t1 = std::chrono::high_resolution_clock::now();
718 |             ocv_decode_time = std::chrono::duration_cast<ms>(t1 - t0).count();
719 | 
720 |             t0 = std::chrono::high_resolution_clock::now();
721 |             // ----------------------------Run face detection inference------------------------------------------
722 |             FaceDetection.submitRequest();
723 |             FaceDetection.wait();
724 | 
725 |             t1 = std::chrono::high_resolution_clock::now();
726 |             ms detection = std::chrono::duration_cast<ms>(t1 - t0);
727 | 
728 |             // fetch all face results
729 |             FaceDetection.fetchResults();
730 | 
731 |             // track and store age and gender results for all faces
732 |             std::vector<AgeGenderDetection::Result> ageGenderResults;
733 |             int ageGenderFaceIdx = 0;
734 |             int ageGenderNumFacesInferred = 0;
735 |             int ageGenderNumFacesToInfer = AgeGender.enabled() ? FaceDetection.results.size() : 0;
736 | 
737 |             // track and store head pose results for all faces
738 |             std::vector<HeadPoseDetection::Results> headPoseResults;
739 |             int headPoseFaceIdx = 0;
740 |             int headPoseNumFacesInferred = 0;
741 |             int headPoseNumFacesToInfer = HeadPose.enabled() ? FaceDetection.results.size() : 0;
742 | 
743 | 
744 |             while((ageGenderFaceIdx < ageGenderNumFacesToInfer)
745 |         		   || (headPoseFaceIdx < headPoseNumFacesToInfer)) {
746 |             	// enqueue input batch
747 |             	while ((ageGenderFaceIdx < ageGenderNumFacesToInfer) && (AgeGender.enquedFaces < AgeGender.maxBatch)) {
748 | 					FaceDetectionClass::Result faceResult = FaceDetection.results[ageGenderFaceIdx];
749 | 					auto clippedRect = faceResult.location & cv::Rect(0, 0, width, height);
750 | 					auto face = frame(clippedRect);
751 | 					AgeGender.enqueue(face);
752 | 					ageGenderFaceIdx++;
753 |             	}
754 | 
755 |             	while ((headPoseFaceIdx < headPoseNumFacesToInfer) && (HeadPose.enquedFaces < HeadPose.maxBatch)) {
756 | 					FaceDetectionClass::Result faceResult = FaceDetection.results[headPoseFaceIdx];
757 | 					auto clippedRect = faceResult.location & cv::Rect(0, 0, width, height);
758 | 					auto face = frame(clippedRect);
759 | 					HeadPose.enqueue(face);
760 | 					headPoseFaceIdx++;
761 |             	}
762 | 
763 | 				t0 = std::chrono::high_resolution_clock::now();
764 | 
765 |             	// if faces are enqueued, then start inference
766 |             	if (AgeGender.enquedFaces > 0) {
767 | 					AgeGender.submitRequest();
768 |             	}
769 |             	if (HeadPose.enquedFaces > 0) {
770 |             		HeadPose.submitRequest();
771 |             	}
772 | 
773 |             	// if there are outstanding results, then wait for inference to complete
774 |             	if (ageGenderNumFacesInferred < ageGenderFaceIdx) {
775 | 					AgeGender.wait();
776 |             	}
777 |             	if (headPoseNumFacesInferred < headPoseFaceIdx) {
778 |             		HeadPose.wait();
779 |             	}
780 | 
781 | 				t1 = std::chrono::high_resolution_clock::now();
782 | 				secondDetection += std::chrono::duration_cast<ms>(t1 - t0).count();
783 | 
784 | 				// process results if there are any
785 | 				if (ageGenderNumFacesInferred < ageGenderFaceIdx) {
786 | 					for(int ri = 0; ri < AgeGender.maxBatch; ri++) {
787 | 						ageGenderResults.push_back(AgeGender[ri]);
788 | 						ageGenderNumFacesInferred++;
789 | 					}
790 |             	}
791 | 				if (headPoseNumFacesInferred < headPoseFaceIdx) {
792 | 					for(int ri = 0; ri < HeadPose.maxBatch; ri++) {
793 | 						headPoseResults.push_back(HeadPose[ri]);
794 | 						headPoseNumFacesInferred++;
795 | 					}
796 |             	}
797 |             }
798 | 
799 |             // ----------------------------Processing outputs-----------------------------------------------------
800 | 			ocv_ttl_render += ocv_render_time;
801 | 			ocv_ttl_decode += ocv_decode_time;
802 | 
803 |             std::ostringstream out;
804 |             out << "OpenCV cap/render time: " << std::fixed << std::setprecision(2)
805 |                 << (ocv_decode_time + ocv_render_time) << " ms";
806 |             cv::putText(frame, out.str(), cv::Point2f(0, 25), cv::FONT_HERSHEY_TRIPLEX, 0.5, cv::Scalar(255, 0, 0));
807 | 						float currFdFps = 1000.f / detection.count();
808 | 						fdFpsTot += currFdFps;
809 | 
810 |             out.str("");
811 |             out << "Face detection time  : " << std::fixed << std::setprecision(2) << detection.count()
812 |                 << " ms ("
813 |                 << currFdFps << " fps)";
814 |             cv::putText(frame, out.str(), cv::Point2f(0, 45), cv::FONT_HERSHEY_TRIPLEX, 0.5,
815 |                         cv::Scalar(255, 0, 0));
816 | 
817 |             if (HeadPose.enabled() || AgeGender.enabled()) {
818 |                 out.str("");
819 |                 out << (AgeGender.enabled() ? "Age Gender"  : "")
820 |                     << (AgeGender.enabled() && HeadPose.enabled() ? "+"  : "")
821 |                     << (HeadPose.enabled() ? "Head Pose "  : "")
822 |                     << "time: "<< std::fixed << std::setprecision(2) << secondDetection
823 |                     << " ms ";
824 |                 if (!FaceDetection.results.empty()) {
825 |                     float otherFps = 1000.f / secondDetection;
826 | 					otherTotFps += otherFps;
827 |                     out << "(" << otherFps << " fps)";
828 |                 }
829 |                 cv::putText(frame, out.str(), cv::Point2f(0, 65), cv::FONT_HERSHEY_TRIPLEX, 0.5, cv::Scalar(255, 0, 0));
830 |             }
831 | 
832 |             // render results
833 |             for(int ri = 0; ri < FaceDetection.results.size(); ri++) {
834 |             	FaceDetectionClass::Result faceResult = FaceDetection.results[ri];
835 |                 cv::Rect rect = faceResult.location;
836 | 
837 |                 out.str("");
838 | 
839 |                 if (AgeGender.enabled()) {
840 |                     out << (ageGenderResults[ri].maleProb > 0.5 ? "M" : "F");
841 |                     out << std::fixed << std::setprecision(0) << "," << ageGenderResults[ri].age;
842 |                 } else {
843 |                     out << (faceResult.label < FaceDetection.labels.size() ? FaceDetection.labels[faceResult.label] :
844 |                              std::string("label #") + std::to_string(faceResult.label))
845 |                         << ": " << std::fixed << std::setprecision(3) << faceResult.confidence;
846 |                 }
847 | 
848 |                 cv::putText(frame,
849 |                             out.str(),
850 |                             cv::Point2f(faceResult.location.x, faceResult.location.y - 15),
851 |                             cv::FONT_HERSHEY_COMPLEX_SMALL,
852 |                             0.8,
853 |                             cv::Scalar(0, 0, 255));
854 | 
855 |                 if (FLAGS_r) {
856 |                     std::cout << "Predicted gender, age = " << out.str() << std::endl;
857 |                 }
858 | 
859 |                 if (HeadPose.enabled()) {
860 |                     cv::Point3f center(rect.x + rect.width / 2, rect.y + rect.height / 2, 0);
861 |                     HeadPose.drawAxes(frame, center, headPoseResults[ri], 50);
862 |                 }
863 | 
864 |                 auto genderColor =
865 |                 		(AgeGender.enabled()) ?
866 |                               ((ageGenderResults[ri].maleProb < 0.5) ? cv::Scalar(0, 0, 255) : cv::Scalar(255, 0, 0)) :
867 |                               cv::Scalar(0, 255, 0);
868 |                 cv::rectangle(frame, faceResult.location, genderColor, 2);
869 |             }
870 | 
871 |             int keyPressed;
872 |             if (-1 != (keyPressed = cv::waitKey(1))) {
873 |             	// done processing, save time
874 |             	wallclockEnd = std::chrono::high_resolution_clock::now();
875 | 
876 |             	if ('s' == keyPressed) {
877 |             		// save screen to output file
878 |             		slog::info << "Saving screenshot" << slog::endl;
879 |             		cv::imwrite("snapshot.bmp", frame);
880 |             	} else {
881 |             		break;
882 |             	}
883 |             }
884 | 
885 |             t0 = std::chrono::high_resolution_clock::now();
886 |             if (!FLAGS_no_show)
887 |                 cv::imshow("Detection results", frame);
888 | 
889 |             t1 = std::chrono::high_resolution_clock::now();
890 |             ocv_render_time = std::chrono::duration_cast<ms>(t1 - t0).count();
891 | 
892 |             // end of file, for single frame file, like image we just keep it displayed to let user check what was shown
893 |             cv::Mat newFrame;
894 |             if (!cap.retrieve(newFrame)) {
895 |             	// done processing, save time
896 |             	wallclockEnd = std::chrono::high_resolution_clock::now();
897 | 
898 | 				if (!FLAGS_no_wait && !FLAGS_no_show) {
899 |                     slog::info << "Press 's' key to save a screenshot, press any other key to exit" << slog::endl;
900 |                     while (cv::waitKey(0) == 's') {
901 |                 		// save screen to output file
902 |                 		slog::info << "Saving screenshot of image" << slog::endl;
903 |                 		cv::imwrite("screenshot.bmp", frame);
904 |                     }
905 |                 }
906 |                 break;
907 |             }
908 |             frame = newFrame;  // shallow copy
909 | 			totalFrames++;
910 |         }
911 | 
912 | 		float avgFdFps = fdFpsTot/totalFrames;
913 | 		float avgAGHpFps = otherTotFps/totalFrames;
914 | 
915 |         // calculate total run time
916 |         ms total_wallclock_time = std::chrono::duration_cast<ms>(wallclockEnd - wallclockStart);
917 | 
918 |         // report loop time
919 | 		float avgTimePerFrameMs = total_wallclock_time.count() / (float)totalFrames;
920 | 
921 | 		std::string na(80, '=');
922 | 		std::string nb(80, '-');
923 | 		std::cout << na << std::endl;
924 | 
925 | 		slog::info << "   Total main-loop time: " << std::fixed << std::setprecision(2)
926 | 				<< total_wallclock_time.count() << " ms " <<  slog::endl;
927 | 		slog::info << "     Total number of frames: " << totalFrames <<  slog::endl;
928 | 
929 | 		std::cout << nb << std::endl;
930 | 
931 | 		// Debug - OCV Times
932 | 		slog::info << "     Total OpenCV Render Time: " << ocv_ttl_render << " (" << (ocv_ttl_render / total_wallclock_time.count()) * 100 << "%)" << slog::endl;
933 | 		slog::info << "     Total OpenCV Decode Time: " << ocv_ttl_decode <<  " (" << (ocv_ttl_decode / total_wallclock_time.count()) * 100 << "%)" << slog::endl;
934 | 
935 | 		slog::info << "     Avg OpenCV Render Time: " << std::fixed << std::setprecision(2) << ocv_ttl_render/totalFrames <<  "ms" << slog::endl;
936 | 		slog::info << "     Avg OpenCV Decode Time: " << std::fixed << std::setprecision(2) << ocv_ttl_decode/totalFrames <<  "ms" << slog::endl;
937 | 
938 | 		std::cout << nb << std::endl;
939 | 
940 | 		slog::info << "   Average time per frame:           " << std::fixed << std::setprecision(2)
941 | 					<< avgTimePerFrameMs << " ms "
942 | 					<< "(" << 1000.0f / avgTimePerFrameMs << " fps)" << slog::endl;
943 | 
944 | 		slog::info << "   Average Face Detection FPS:       " << std::fixed << std::setprecision(2)
945 | 					<< avgFdFps << " fps" << slog::endl;
946 | 
947 | 		slog::info << "   Average Age/Gender/Head Pose FPS: " << std::fixed << std::setprecision(2)
948 | 					<< avgAGHpFps << " fps" << slog::endl;
949 | 
950 | 		std::cout << nb << std::endl;
951 | 
952 |         // ---------------------------Some perf data--------------------------------------------------
953 |         if (FLAGS_pc) {
954 |             FaceDetection.printPerformanceCounts();
955 |             AgeGender.printPerformanceCounts();
956 |             HeadPose.printPerformanceCounts();
957 |         }
958 | 
959 |     } catch (const std::exception& error) {
960 |         slog::err << error.what() << slog::endl;
961 |         return 1;
962 |     } catch (...) {
963 |         slog::err << "Unknown/internal exception happened." << slog::endl;
964 |         return 1;
965 |     }
966 | 
967 |     slog::info << "Execution successful" << slog::endl;
968 |     return 0;
969 | }
970 | 


--------------------------------------------------------------------------------
/dx_face_detection/run_fd.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/inference_engine_samples/intel64/Release/lib/
 4 | 
 5 | get_params() {
 6 |   hw="CPU"
 7 |   het="HETERO:FPGA,CPU"
 8 |   solo_mode=1
 9 | 	fp="FP32"
10 | 	fp_ag="FP32"
11 | 	fp_hp="FP32"
12 | 
13 |   # Get Hardware Target Parameters 
14 |   if [[ -n $@ ]]; then
15 |     if [[ -n $1 ]]; then
16 |       hw="$1"
17 |     fi
18 |     if [[ -n $2 ]]; then
19 |       hw_ag="$2"
20 |       solo_mode=0
21 |     fi
22 |     if [[ -n $3 ]]; then
23 |       hw_hp="$3"
24 |       solo_mode=0
25 |     fi
26 |   fi
27 | 
28 |   # Expand FPGA, Capitalize Other Strings
29 |   if [[ $hw == "fpga" ]]; then
30 |     hw="$het"
31 |   else
32 |     hw=${hw^^}
33 |   fi
34 |   if [[ $hw_ag == "fpga" ]]; then
35 |     hw_ag="$het"
36 |   else
37 |     hw_ag=${hw_ag^^}
38 |   fi
39 |   if [[ $hw_hp == "fpga" ]]; then
40 | 		echo "Warning: FPGA doesn't support the head pose model - changing to CPU."
41 | 		hw_hp="CPU"
42 |   else
43 |     hw_hp=${hw_hp^^}
44 |   fi
45 | 
46 | 	# Update FP Settings
47 | 	if [[ $hw != "CPU" ]]; then
48 | 		fp="FP16"
49 | 	fi
50 | 
51 | 	if [[ $hw_ag != "CPU" ]]; then
52 | 		fp_ag="FP16"
53 | 	fi
54 | 
55 | 	if [[ $hw_hp != "CPU" ]]; then
56 | 		fp_hp="FP16"
57 | 	fi
58 | }
59 | 
60 | # Main
61 | #--------------------------------------------------------------------------------
62 | get_params $@
63 | 
64 | fd=build/intel64/Release/face_detection_tutorial
65 | vid=../Videos/head-pose-face-detection-female-and-male.mp4 
66 | moddir=/opt/intel/computer_vision_sdk/deployment_tools/intel_models
67 | mod_od=$moddir/face-detection-retail-0004/$fp/face-detection-retail-0004.xml 
68 | mod_ag=$moddir/age-gender-recognition-retail-0013/$fp_ag/age-gender-recognition-retail-0013.xml 
69 | mod_hp=$moddir/head-pose-estimation-adas-0001/$fp_hp/head-pose-estimation-adas-0001.xml
70 | 
71 | 
72 | # Execute Face Detection 
73 | if [[ -n $hw_ag && -n $hw_hp ]]; then
74 | 		# face detection + age & gender + head pose
75 | 	$fd -i $vid -m $mod_od -m_ag $mod_ag -m_hp $mod_hp -d $hw -d_ag $hw_ag -d_hp $hw_hp
76 | else
77 | 	if [[ -n $hw_ag ]]; then
78 | 		# face detection + age & gender
79 | 		$fd -i $vid -m $mod_od -m_ag $mod_ag -d $hw -d_ag $hw_ag
80 | 	else
81 | 		# Just face detection
82 | 		$fd -i $vid -m $mod_od -d $hw
83 | 	fi
84 | fi
85 | 
86 | exit
87 | 
88 | 


--------------------------------------------------------------------------------