├── .gitmodules ├── README.md ├── cpp ├── CMakeLists.txt └── main.cpp ├── data ├── custom_weight.names ├── me.jpeg └── me_cpp_pred.png └── reparameterization_yolov7.py /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "yolov7"] 2 | path = yolov7 3 | url = https://github.com/WongKinYiu/yolov7 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # yolov7_opencv_cpp 3 | Object Detection using YOLOv7 and OpenCV DNN C++ 4 | 5 | ### Get repo and install dependences 6 | ```bash 7 | git clone --recursive https://github.com/majnas/yolov7_opencv_cpp.git 8 | cd yolov7_opencv_cpp/yolov7 9 | 10 | # Install dependencies. 11 | pip install -r requirements.txt 12 | pip install onnx 13 | ``` 14 | 15 | 16 | ### Prepare custom weight 17 | Download my custom yolov7 face detection using this cmd 18 | ```shell 19 | cd cfg/training/ 20 | wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1INiC_M_ttd8xMpZ9CuSA1FTqUxZT4e1y' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1INiC_M_ttd8xMpZ9CuSA1FTqUxZT4e1y" -O custom_weight.pt && rm -rf /tmp/cookies.txt 21 | ``` 22 | 23 | Or place use your own custom yolov7 weight in following folder. 24 | 25 | ```shell 26 | └── yolov7 27 | ├── cfg 28 | │   ├── baseline 29 | │   ├── deploy 30 | │   └── training 31 | │   ├── custom_weight.pt <----- Place the custom weight here 32 | │   ├── yolov7-d6.yaml 33 | │   ├── yolov7-e6e.yaml 34 | │   ├── yolov7-e6.yaml 35 | │   ├── yolov7-tiny.yaml 36 | │   ├── yolov7-w6.yaml 37 | │   ├── yolov7x.yaml 38 | │   └── yolov7.yaml 39 | 40 | ``` 41 | 42 | 43 | * Make a copy of yolov7/cfg/deploy/yolov7.yml and rename to yolov7_custom_weight.yaml then change number of class in line number 2 (nc=1). For my custom weight there is only one class (face). 44 | ```shell 45 | cd .. 46 | cp deploy/yolov7.yaml deploy/yolov7_custom_weight.yaml 47 | # edit yolov7_custom_weight.yaml => Set nc in line 2 => nc=1 in my case which I have only one class 48 | ``` 49 | * Moving reparameterization_yolov7.py to yolov7 directory. 50 | ```shell 51 | # In root of repo directory 52 | mv reparameterization_yolov7.py ./yolov7/reparameterization_yolov7.py 53 | # Edit./yolov7/reparameterization_yolov7.py 54 | # 1 - Set number of classes 55 | # nc=1 # change this base on number of classes in your custom model 56 | # 2 - Set device 57 | # device = select_device('0', batch_size=1) # if using GPU 58 | # device = select_device('cpu', batch_size=1) # if using CPU 59 | ``` 60 | 61 | ### Reparameterization of model 62 | ```shell 63 | cd ./yolov7 64 | python reparameterization_yolov7.py 65 | ``` 66 | > :warning: **If you are using different version of yolov7 (yolov7x, yolov7-tiny, ...) use different reparameterizatioin script from [here](https://github.com/WongKinYiu/yolov7/blob/main/tools/reparameterization.ipynb) **: Be very careful here! 67 | 68 | This will create another model (custom_weight_reparameterized.pt) in cfg/deploy/custom_weight_reparameterized.pt, which is reparameterized version of custom weight. 69 | 70 | ```shell 71 | └── yolov7 72 | ├── cfg 73 | │   ├── baseline 74 | │   ├── deploy 75 | │   │   ├── custom_weight_reparameterized.pt <------------- Reparameterized custom weight 76 | │   │   ├── yolov7_custom_weight.yaml 77 | │   │   ├── yolov7-d6.yaml 78 | │   │   ├── yolov7-e6e.yaml 79 | │   │   ├── yolov7-e6.yaml 80 | │   │   ├── yolov7-tiny-silu.yaml 81 | │   │   ├── yolov7-tiny.yaml 82 | │   │   ├── yolov7-w6.yaml 83 | │   │   ├── yolov7x.yaml 84 | │   │   └── yolov7.yaml 85 | │   └── training 86 | │   ├── custom_weight.pt <------------- Custom weight 87 | │   ├── yolov7-d6.yaml 88 | │   ├── yolov7-e6e.yaml 89 | │   ├── yolov7-e6.yaml 90 | │   ├── yolov7-tiny.yaml 91 | │   ├── yolov7-w6.yaml 92 | │   ├── yolov7x.yaml 93 | │   └── yolov7.yaml 94 | 95 | ``` 96 | 97 | ### Export to ONNX 98 | To export ONNX we have to checkout to u5 branch, and export reparameterized version of custom weight to onnx and torchscript, to do this 99 | ```shell 100 | git checkout u5 101 | python export.py --weights cfg/deploy/custom_weight_reparameterized.pt --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640 102 | ``` 103 | Now we will have onnx and tochscript version of our custom_weight.pt 104 | ```shell 105 | └── yolov7 106 | ├── cfg 107 | │   ├── deploy 108 | │   │   ├── custom_weight_reparameterized.onnx <------------- onnx version 109 | │   │   ├── custom_weight_reparameterized.pt <------------- Reparameterized custom weight 110 | │   │   └── custom_weight_reparameterized.torchscript <------------- torchscript version version 111 | │   └── training 112 | │   └── custom_weight.pt <------------- Custom weight 113 | 114 | ``` 115 | 116 | ### To compile and run cpp version 117 | ```bash 118 | # In root of repo directory 119 | cd cpp 120 | mkdir build 121 | cd build 122 | cmake .. 123 | make 124 | ./app "../../data/me.jpeg" "../../yolov7/cfg/deploy/custom_weight_reparameterized.onnx" 640 640 125 | ``` 126 | 127 | :information_source: If you got following error, you must install [opencv](https://github.com/opencv/opencv) on your system. 128 | ``` 129 | CMake Error at CMakeLists.txt:7 (find_package): 130 | By not providing "FindOpenCV.cmake" in CMAKE_MODULE_PATH this project has 131 | asked CMake to find a package configuration file provided by "OpenCV", but 132 | CMake did not find one. 133 | 134 | Could not find a package configuration file provided by "OpenCV" with any 135 | of the following names: 136 | 137 | OpenCVConfig.cmake 138 | opencv-config.cmake 139 | 140 | Add the installation prefix of "OpenCV" to CMAKE_PREFIX_PATH or set 141 | "OpenCV_DIR" to a directory containing one of the above files. If "OpenCV" 142 | provides a separate development package or SDK, be sure it has been 143 | installed. 144 | ``` 145 | 146 |
147 | 148 |
149 |

150 | Figure 1: cpp prediction for me_cpp_pred.png 151 |

152 | -------------------------------------------------------------------------------- /cpp/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | # Minimum version of CMake required to build this project 2 | cmake_minimum_required(VERSION 3.0 FATAL_ERROR) 3 | 4 | # Name of the project 5 | project(app) 6 | 7 | find_package(OpenCV REQUIRED) 8 | 9 | include_directories("${OpenCV_INCLUDE_DIRS}") 10 | 11 | # Add all the source files needed to build the executable 12 | add_executable(app main.cpp) 13 | 14 | # Link the executable and the library together 15 | target_link_libraries(app "${OpenCV_LIBS}") 16 | set_property(TARGET app PROPERTY CXX_STANDARD 14) 17 | -------------------------------------------------------------------------------- /cpp/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | using namespace cv; 9 | using namespace dnn; 10 | using namespace std; 11 | 12 | struct Net_config 13 | { 14 | float confThreshold; // Confidence threshold 15 | float nmsThreshold; // Non-maximum suppression threshold 16 | string modelpath; 17 | int inpHeight; 18 | int inpWidth; 19 | }; 20 | 21 | class YOLOV7 22 | { 23 | public: 24 | YOLOV7(Net_config config); 25 | void detect(Mat& frame); 26 | private: 27 | int inpWidth; 28 | int inpHeight; 29 | vector class_names; 30 | int num_class; 31 | 32 | float confThreshold; 33 | float nmsThreshold; 34 | Net net; 35 | void drawPred(float conf, int left, int top, int right, int bottom, Mat& frame, int classid); 36 | }; 37 | 38 | YOLOV7::YOLOV7(Net_config config) 39 | { 40 | this->confThreshold = config.confThreshold; 41 | this->nmsThreshold = config.nmsThreshold; 42 | 43 | this->net = readNet(config.modelpath); 44 | ifstream ifs("../../data/custom_weight.names"); 45 | string line; 46 | while (getline(ifs, line)) this->class_names.push_back(line); 47 | this->num_class = class_names.size(); 48 | this->inpHeight = config.inpHeight; 49 | this->inpWidth = config.inpWidth; 50 | } 51 | 52 | void YOLOV7::drawPred(float conf, int left, int top, int right, int bottom, Mat& frame, int classid) // Draw the predicted bounding box 53 | { 54 | //Draw a rectangle displaying the bounding box 55 | rectangle(frame, Point(left, top), Point(right, bottom), Scalar(0, 0, 255), 2); 56 | //Get the label for the class name and its confidence 57 | string label = format("%.2f", conf); 58 | label = this->class_names[classid] + ":" + label; 59 | 60 | //Display the label at the top of the bounding box 61 | int baseLine; 62 | Size labelSize = getTextSize(label, FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine); 63 | top = max(top, labelSize.height); 64 | //rectangle(frame, Point(left, top - int(1.5 * labelSize.height)), Point(left + int(1.5 * labelSize.width), top + baseLine), Scalar(0, 255, 0), FILLED); 65 | putText(frame, label, Point(left, top), FONT_HERSHEY_SIMPLEX, 0.75, Scalar(0, 255, 0), 1); 66 | } 67 | 68 | void YOLOV7::detect(Mat& frame) 69 | { 70 | Mat blob = blobFromImage(frame, 1 / 255.0, Size(this->inpWidth, this->inpHeight), Scalar(0, 0, 0), true, false); 71 | this->net.setInput(blob); 72 | vector outs; 73 | this->net.forward(outs, this->net.getUnconnectedOutLayersNames()); 74 | int num_proposal = outs[0].size[0]; 75 | int nout = outs[0].size[1]; 76 | if (outs[0].dims > 2) 77 | { 78 | num_proposal = outs[0].size[1]; 79 | nout = outs[0].size[2]; 80 | outs[0] = outs[0].reshape(0, num_proposal); 81 | } 82 | /////generate proposals 83 | vector confidences; 84 | vector boxes; 85 | vector classIds; 86 | float ratioh = (float)frame.rows / this->inpHeight, ratiow = (float)frame.cols / this->inpWidth; 87 | int n = 0, row_ind = 0; ///cx,cy,w,h,box_score,class_score 88 | float* pdata = (float*)outs[0].data; 89 | for (n = 0; n < num_proposal; n++) ///ÌØÕ÷ͼ³ß¶È 90 | { 91 | float box_score = pdata[4]; 92 | if (box_score > this->confThreshold) 93 | { 94 | Mat scores = outs[0].row(row_ind).colRange(5, nout); 95 | Point classIdPoint; 96 | double max_class_socre; 97 | // Get the value and location of the maximum score 98 | minMaxLoc(scores, 0, &max_class_socre, 0, &classIdPoint); 99 | max_class_socre *= box_score; 100 | if (max_class_socre > this->confThreshold) 101 | { 102 | const int class_idx = classIdPoint.x; 103 | float cx = pdata[0] * ratiow; ///cx 104 | float cy = pdata[1] * ratioh; ///cy 105 | float w = pdata[2] * ratiow; ///w 106 | float h = pdata[3] * ratioh; ///h 107 | 108 | int left = int(cx - 0.5 * w); 109 | int top = int(cy - 0.5 * h); 110 | 111 | confidences.push_back((float)max_class_socre); 112 | boxes.push_back(Rect(left, top, (int)(w), (int)(h))); 113 | classIds.push_back(class_idx); 114 | } 115 | } 116 | row_ind++; 117 | pdata += nout; 118 | } 119 | 120 | // Perform non maximum suppression to eliminate redundant overlapping boxes with 121 | // lower confidences 122 | vector indices; 123 | dnn::NMSBoxes(boxes, confidences, this->confThreshold, this->nmsThreshold, indices); 124 | for (size_t i = 0; i < indices.size(); ++i) 125 | { 126 | int idx = indices[i]; 127 | Rect box = boxes[idx]; 128 | this->drawPred(confidences[idx], box.x, box.y, 129 | box.x + box.width, box.y + box.height, frame, classIds[idx]); 130 | } 131 | /* 132 | */ 133 | } 134 | 135 | int main(int argc, char* argv[]) 136 | { 137 | assert (argc == 5); 138 | string imgpath = argv[1]; 139 | string modelPath = argv[2]; 140 | int inputHeight = std::stoi(argv[3]); 141 | int inputWidth = std::stoi(argv[4]); 142 | 143 | std::cout << "imgpath: " << imgpath << std::endl; 144 | std::cout << "modelPath: " << modelPath << std::endl; 145 | std::cout << "inputHeight: " << inputHeight << std::endl; 146 | std::cout << "inputWidth: " << inputWidth << std::endl; 147 | 148 | Net_config YOLOV7_nets = { 0.3, 0.5, modelPath, inputHeight, inputWidth}; 149 | YOLOV7 net(YOLOV7_nets); 150 | Mat srcimg = imread(imgpath); 151 | net.detect(srcimg); 152 | 153 | static const string kWinName = "Deep learning object detection in OpenCV"; 154 | namedWindow(kWinName, WINDOW_NORMAL); 155 | imshow(kWinName, srcimg); 156 | waitKey(0); 157 | destroyAllWindows(); 158 | } 159 | -------------------------------------------------------------------------------- /data/custom_weight.names: -------------------------------------------------------------------------------- 1 | face 2 | -------------------------------------------------------------------------------- /data/me.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/majnas/yolov7_opencv_cpp/42096904b63f7c4b8bc18b94f4d91c6926b60869/data/me.jpeg -------------------------------------------------------------------------------- /data/me_cpp_pred.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/majnas/yolov7_opencv_cpp/42096904b63f7c4b8bc18b94f4d91c6926b60869/data/me_cpp_pred.png -------------------------------------------------------------------------------- /reparameterization_yolov7.py: -------------------------------------------------------------------------------- 1 | # import 2 | from copy import deepcopy 3 | from models.yolo import Model 4 | import torch 5 | from utils.torch_utils import select_device, is_parallel 6 | import yaml 7 | 8 | nc=1 # change this base on number of classes in your custom model 9 | 10 | device = select_device('0', batch_size=1) 11 | # model trained by cfg/training/*.yaml 12 | ckpt = torch.load('cfg/training/custom_weight.pt', map_location=device) 13 | # reparameterized model in cfg/deploy/*.yaml 14 | model = Model('cfg/deploy/yolov7_custom_weight.yaml', ch=3, nc=nc).to(device) 15 | 16 | with open('cfg/deploy/yolov7_custom_weight.yaml') as f: 17 | yml = yaml.load(f, Loader=yaml.SafeLoader) 18 | anchors = len(yml['anchors'][0]) // 2 19 | 20 | # copy intersect weights 21 | state_dict = ckpt['model'].float().state_dict() 22 | exclude = [] 23 | intersect_state_dict = {k: v for k, v in state_dict.items() if k in model.state_dict() and not any(x in k for x in exclude) and v.shape == model.state_dict()[k].shape} 24 | model.load_state_dict(intersect_state_dict, strict=False) 25 | model.names = ckpt['model'].names 26 | model.nc = ckpt['model'].nc 27 | 28 | # reparametrized YOLOR 29 | for i in range((model.nc+5)*anchors): 30 | model.state_dict()['model.105.m.0.weight'].data[i, :, :, :] *= state_dict['model.105.im.0.implicit'].data[:, i, : :].squeeze() 31 | model.state_dict()['model.105.m.1.weight'].data[i, :, :, :] *= state_dict['model.105.im.1.implicit'].data[:, i, : :].squeeze() 32 | model.state_dict()['model.105.m.2.weight'].data[i, :, :, :] *= state_dict['model.105.im.2.implicit'].data[:, i, : :].squeeze() 33 | model.state_dict()['model.105.m.0.bias'].data += state_dict['model.105.m.0.weight'].mul(state_dict['model.105.ia.0.implicit']).sum(1).squeeze() 34 | model.state_dict()['model.105.m.1.bias'].data += state_dict['model.105.m.1.weight'].mul(state_dict['model.105.ia.1.implicit']).sum(1).squeeze() 35 | model.state_dict()['model.105.m.2.bias'].data += state_dict['model.105.m.2.weight'].mul(state_dict['model.105.ia.2.implicit']).sum(1).squeeze() 36 | model.state_dict()['model.105.m.0.bias'].data *= state_dict['model.105.im.0.implicit'].data.squeeze() 37 | model.state_dict()['model.105.m.1.bias'].data *= state_dict['model.105.im.1.implicit'].data.squeeze() 38 | model.state_dict()['model.105.m.2.bias'].data *= state_dict['model.105.im.2.implicit'].data.squeeze() 39 | 40 | # model to be saved 41 | ckpt = {'model': deepcopy(model.module if is_parallel(model) else model).half(), 42 | 'optimizer': None, 43 | 'training_results': None, 44 | 'epoch': -1} 45 | 46 | # save reparameterized model 47 | torch.save(ckpt, 'cfg/deploy/custom_weight_reparameterized.pt') 48 | --------------------------------------------------------------------------------