├── .gitignore ├── CODE_OF_CONDUCT.md ├── Cargo.toml ├── LICENSE ├── README.md ├── download_v3.sh ├── download_v3_tiny.sh ├── download_v4.sh ├── download_v4_tiny.sh ├── download_v7.sh ├── download_v7_tiny.sh ├── download_v8_l.sh ├── download_v8_m.sh ├── download_v8_n.sh ├── download_v8_s.sh ├── download_v8_x.sh ├── examples ├── yolo_v3.rs ├── yolo_v3_tiny.rs ├── yolo_v4.rs ├── yolo_v4_tiny.rs ├── yolo_v4_tiny_onnx.rs ├── yolo_v7.rs ├── yolo_v7_tiny.rs ├── yolo_v8_l.rs ├── yolo_v8_m.rs ├── yolo_v8_n.rs ├── yolo_v8_s.rs └── yolo_v8_x.rs ├── images ├── dog.jpg ├── dog_yolov3.jpg ├── dog_yolov3_tiny.jpg ├── dog_yolov4.jpg ├── dog_yolov4_tiny.jpg ├── dog_yolov7.jpg ├── dog_yolov7_tiny.jpg ├── dog_yolov8_l.jpg ├── dog_yolov8_m.jpg ├── dog_yolov8_n.jpg ├── dog_yolov8_s.jpg └── dog_yolov8_x.jpg └── src ├── lib.rs ├── model.rs ├── model_classic.rs ├── model_format.rs ├── model_ultralytics.rs └── utils.rs /.gitignore: -------------------------------------------------------------------------------- 1 | /target 2 | /Cargo.lock 3 | /pretrained -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | We as members, contributors, and leaders pledge to make participation in our 6 | community a harassment-free experience for everyone, regardless of age, body 7 | size, visible or invisible disability, ethnicity, sex characteristics, gender 8 | identity and expression, level of experience, education, socio-economic status, 9 | nationality, personal appearance, race, religion, or sexual identity 10 | and orientation. 11 | 12 | We pledge to act and interact in ways that contribute to an open, welcoming, 13 | diverse, inclusive, and healthy community. 14 | 15 | ## Our Standards 16 | 17 | Examples of behavior that contributes to a positive environment for our 18 | community include: 19 | 20 | * Demonstrating empathy and kindness toward other people 21 | * Being respectful of differing opinions, viewpoints, and experiences 22 | * Giving and gracefully accepting constructive feedback 23 | * Accepting responsibility and apologizing to those affected by our mistakes, 24 | and learning from the experience 25 | * Focusing on what is best not just for us as individuals, but for the 26 | overall community 27 | 28 | Examples of unacceptable behavior include: 29 | 30 | * The use of sexualized language or imagery, and sexual attention or 31 | advances of any kind 32 | * Trolling, insulting or derogatory comments, and personal or political attacks 33 | * Public or private harassment 34 | * Publishing others' private information, such as a physical or email 35 | address, without their explicit permission 36 | * Other conduct which could reasonably be considered inappropriate in a 37 | professional setting 38 | 39 | ## Enforcement Responsibilities 40 | 41 | Community leaders are responsible for clarifying and enforcing our standards of 42 | acceptable behavior and will take appropriate and fair corrective action in 43 | response to any behavior that they deem inappropriate, threatening, offensive, 44 | or harmful. 45 | 46 | Community leaders have the right and responsibility to remove, edit, or reject 47 | comments, commits, code, wiki edits, issues, and other contributions that are 48 | not aligned to this Code of Conduct, and will communicate reasons for moderation 49 | decisions when appropriate. 50 | 51 | ## Scope 52 | 53 | This Code of Conduct applies within all community spaces, and also applies when 54 | an individual is officially representing the community in public spaces. 55 | Examples of representing our community include using an official e-mail address, 56 | posting via an official social media account, or acting as an appointed 57 | representative at an online or offline event. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported to the community leaders responsible for enforcement at 63 | sexykdi@gmail.com. 64 | All complaints will be reviewed and investigated promptly and fairly. 65 | 66 | All community leaders are obligated to respect the privacy and security of the 67 | reporter of any incident. 68 | 69 | ## Enforcement Guidelines 70 | 71 | Community leaders will follow these Community Impact Guidelines in determining 72 | the consequences for any action they deem in violation of this Code of Conduct: 73 | 74 | ### 1. Correction 75 | 76 | **Community Impact**: Use of inappropriate language or other behavior deemed 77 | unprofessional or unwelcome in the community. 78 | 79 | **Consequence**: A private, written warning from community leaders, providing 80 | clarity around the nature of the violation and an explanation of why the 81 | behavior was inappropriate. A public apology may be requested. 82 | 83 | ### 2. Warning 84 | 85 | **Community Impact**: A violation through a single incident or series 86 | of actions. 87 | 88 | **Consequence**: A warning with consequences for continued behavior. No 89 | interaction with the people involved, including unsolicited interaction with 90 | those enforcing the Code of Conduct, for a specified period of time. This 91 | includes avoiding interactions in community spaces as well as external channels 92 | like social media. Violating these terms may lead to a temporary or 93 | permanent ban. 94 | 95 | ### 3. Temporary Ban 96 | 97 | **Community Impact**: A serious violation of community standards, including 98 | sustained inappropriate behavior. 99 | 100 | **Consequence**: A temporary ban from any sort of interaction or public 101 | communication with the community for a specified period of time. No public or 102 | private interaction with the people involved, including unsolicited interaction 103 | with those enforcing the Code of Conduct, is allowed during this period. 104 | Violating these terms may lead to a permanent ban. 105 | 106 | ### 4. Permanent Ban 107 | 108 | **Community Impact**: Demonstrating a pattern of violation of community 109 | standards, including sustained inappropriate behavior, harassment of an 110 | individual, or aggression toward or disparagement of classes of individuals. 111 | 112 | **Consequence**: A permanent ban from any sort of public interaction within 113 | the community. 114 | 115 | ## Attribution 116 | 117 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 118 | version 2.0, available at 119 | https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. 120 | 121 | Community Impact Guidelines were inspired by [Mozilla's code of conduct 122 | enforcement ladder](https://github.com/mozilla/diversity). 123 | 124 | [homepage]: https://www.contributor-covenant.org 125 | 126 | For answers to common questions about this code of conduct, see the FAQ at 127 | https://www.contributor-covenant.org/faq. Translations are available at 128 | https://www.contributor-covenant.org/translations. 129 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "od_opencv" 3 | repository = "https://github.com/LdDl/object-detection-opencv-rust" 4 | readme = "README.md" 5 | keywords = [ 6 | "computer-vision", 7 | "object-detection", 8 | "yolo", 9 | "opencv", 10 | "yolo-v8", 11 | ] 12 | license = "MIT" 13 | version = "0.1.8" 14 | edition = "2021" 15 | rust-version = "1.73" 16 | authors = ["Dimitrii Lopanov "] 17 | exclude = ["/.github", "/ci", "/tools", "release.toml", "rustfmt.toml", "download_v3_tiny.sh", "download_v4_tiny.sh", "download_v7_tiny.sh", "download_v8_n.sh", "/images", "/pretrained"] 18 | description = "Object detection utilities in Rust programming language for YOLO-based neural networks in OpenCV ecosystem" 19 | homepage = "https://github.com/LdDl/object-detection-opencv-rust" 20 | documentation = "https://docs.rs/od_opencv" 21 | categories = [ 22 | "algorithms", 23 | "multimedia", 24 | "computer-vision", 25 | "mathematics", 26 | "science", 27 | ] 28 | 29 | # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html 30 | 31 | [lib] 32 | name = "od_opencv" 33 | path = "src/lib.rs" 34 | doctest = false 35 | 36 | [[example]] 37 | name = "yolo_v3_tiny" 38 | 39 | [[example]] 40 | name = "yolo_v4_tiny" 41 | 42 | [[example]] 43 | name = "yolo_v7_tiny" 44 | 45 | [[example]] 46 | name = "yolo_v8_n" 47 | 48 | [dependencies] 49 | lazy_static = "1.4.0" 50 | opencv = { version = "0.94.2", default-features = false, features = ["dnn", "imgcodecs", "imgproc"] } 51 | 52 | [patch.crates-io] 53 | cc = { git = "https://github.com/rust-lang/cc-rs.git", version = "1.0", tag = "1.0.79" } 54 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Dimitrii Lopanov 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Package](https://img.shields.io/crates/v/od_opencv.svg)](https://crates.io/crates/od_opencv) 2 | 3 | # Object detection utilities in Rust programming language for YOLO-based neural networks in OpenCV ecosystem 4 | 5 | This crate provides some basic structures and methods for solving object detections tasks via [OpenCV's DNN module](https://docs.opencv.org/4.8.0/d2/d58/tutorial_table_of_content_dnn.html). Currently implemented and tested workflows: 6 | 7 | | Network type | Darknet | ONNX | 8 | | ------------- | ------------- | ------------- | 9 | | YOLO v3 tiny | :white_check_mark: | :warning: (need to test) | 10 | | YOLO v4 tiny | :white_check_mark: | :warning: (need to test) | 11 | | YOLO v7 tiny | :white_check_mark: | :warning: (need to test) | 12 | | YOLO v3 | :white_check_mark: | :warning: (need to test) | 13 | | YOLO v4 | :white_check_mark: | :warning: (need to test) | 14 | | YOLO v7 | :white_check_mark: | :warning: (need to test) | 15 | | YOLO v8 n | :x: (is it even possible?) | :white_check_mark: | 16 | | YOLO v8 s | :x: (is it even possible?) | :white_check_mark: | 17 | | YOLO v8 m | :x: (is it even possible?) | :white_check_mark: | 18 | | YOLO v8 l | :x: (is it even possible?) | :white_check_mark: | 19 | | YOLO v8 x | :x: (is it even possible?) | :white_check_mark: | 20 | 21 | ## Table of Contents 22 | 23 | - [About](#about) 24 | - [Prerequisites](#prerequisites) 25 | - [Usage](#usage) 26 | - [References](#references) 27 | 28 | ## About 29 | 30 | _- Why?_ 31 | 32 | Well, I just tired a bit of boilerplating (model initializing, postprocessing functions and etc.) in my both private and public projects. 33 | 34 | _- When it is usefull?_ 35 | 36 | Well, there are several circumstances when you may need this crate: 37 | 38 | * You need to use YOLO as your neural network base; 39 | * You do not want use Pytorch / Tensorflow / Jax or any other DL/ML framework (someday it may happen to use pure ONNX without OpenCV features in this crate - PR's are welcome); 40 | * You need to use OpenCV's DNN module to initialize neural network; 41 | 42 | _- Why no YOLOv5?_ 43 | 44 | I think there is a difference in postprocessing stuff between v8 and v5 versions. I need more time to investigate what should be done exactly to make v5 work. 45 | 46 | _- What OpenCV's version is tested?_ 47 | 48 | I've tested it with v4.7.0. Rust bindings version: v0.66.0 49 | 50 | _- Are wrapper structures thread safe?_ 51 | 52 | I'm not sure it is intended to be used in multiple threads (PR's are welcome). But I think you should use some queue mechanism if you want to give "async" acces to provided structs. 53 | 54 | ## Prerequisites 55 | 56 | * For sure you must have OpenCV installed with DNN extra module. If you need to ulitize power of GPU/OpenVINO then you need to consider to include corresponding extra modules too. 57 | 58 | I love to use this [Makefile](https://github.com/hybridgroup/gocv/blob/release/Makefile) with little adjustment (OpenCV's version / enabling python bindings) for my needs. 59 | 60 | * Prepare neural network: train it or get pretrained one. I provide pretty simple Bash scripts to download "small" versions of YOLO 61 | * YOLO v3 tiny - [download_v3_tiny.sh](download_v3_tiny.sh); YOLO v3 - [download_v3.sh](download_v3.sh); 62 | * YOLO v4 tiny - [download_v4_tiny.sh](download_v4_tiny.sh); YOLO v4 - [download_v4.sh](download_v4.sh); 63 | * YOLO v7 tiny - [download_v7_tiny.sh](download_v7_tiny.sh); YOLO v7 - [download_v7.sh](download_v7.sh); 64 | * YOLO v8 nano (n) - [download_v8_n.sh](download_v8_n.sh). 65 | * YOLO v8 small (s) - [download_v8_s.sh](download_v8_s.sh). 66 | * YOLO v8 medium (m) - [download_v8_m.sh](download_v8_m.sh). 67 | * YOLO v8 large (l) - [download_v8_l.sh](download_v8_l.sh). 68 | * YOLO v8 extra (x) - [download_v8_x.sh](download_v8_x.sh). 69 | 70 | __Notice that "v8" script downloads Pytorch-based weights and converts it into ONNX one via `ultralytics` package for Python.__ 71 | 72 | ## Usage 73 | 74 | There are some [examples](examples), but let me guide you step-by-step 75 | 76 | 1. Add this crate to your's `Cargo.toml`: 77 | ```shell 78 | cargo add od_opencv 79 | ``` 80 | 81 | 1. Add OpenCV's bindings crate to `Cargo.toml` also: 82 | ```shell 83 | # I'm using 0.66 version 84 | cargo add opencv@0.66 85 | ``` 86 | 87 | 2. Download pretrained or use your own neural network. 88 | 89 | I will use pretrained weights from [prerequisites section](#prerequisites) 90 | 91 | 3. Import "basic" OpenCV stuff in yours `main.rs` file: 92 | 93 | ```rust 94 | use opencv::{ 95 | core::{Scalar, Vector}, 96 | imgcodecs::imread, 97 | imgcodecs::imwrite, 98 | imgproc::LINE_4, 99 | imgproc::rectangle, 100 | dnn::DNN_BACKEND_CUDA, // I will utilize my GPU to perform faster inference. Your way may vary 101 | dnn::DNN_TARGET_CUDA, 102 | }; 103 | ``` 104 | 4. Import crate 105 | ```rust 106 | use od_opencv::{ 107 | model_format::ModelFormat, 108 | // I'll use YOLOv8 by Ultralytics. 109 | // If you prefer traditional YOLO, then import it as: 110 | // model_classic::ModelYOLOClassic 111 | model_ultralytics::ModelUltralyticsV8 112 | }; 113 | ``` 114 | 115 | 5. Prepare model 116 | 117 | ```rust 118 | // Define classes (in this case we consider 80 COCO labels) 119 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 120 | 121 | // Define format for OpenCV's DNN module 122 | let mf = ModelFormat::ONNX; 123 | 124 | // Define model's input size 125 | let net_width = 640; 126 | let net_height = 640; 127 | 128 | // Initialize optional filters. 129 | // E.g.: if you do want to find only dogs and cats and you can't re-train neural network, 130 | // then you can just place `vec![15, 16]` to filter dogs and cats (15 - index of `cat` in class labels, 16 - `dog`) 131 | // let class_filters: Vec = vec![15, 16]; 132 | let class_filters: Vec = vec![]; 133 | 134 | // Initialize model itself 135 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8n.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, class_filters).unwrap(); 136 | 137 | // Read image into the OpenCV's Mat object 138 | // Define it as mutable since we are going to put bounding boxes onto it. 139 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 140 | 141 | // Feed forward image through the model 142 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 143 | 144 | // Process results 145 | for (i, bbox) in bboxes.iter().enumerate() { 146 | // Place bounding boxes onto the image 147 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 148 | // Debug output to stdin 149 | println!("Class: {}", classes_labels[class_ids[i]]); 150 | println!("\tBounding box: {:?}", bbox); 151 | println!("\tConfidences: {}", confidences[i]); 152 | } 153 | 154 | // Finally save the updated image to the file system 155 | imwrite("images/dog_yolov8_n.jpg", &frame, &Vector::new()).unwrap(); 156 | ``` 157 | 158 | 6. You are good to go 159 | ```rust 160 | cargo run 161 | ``` 162 | 163 | 7. If anything is going wrong, feel free to [open an issue](https://github.com/LdDl/object-detection-opencv-rust/issues/new) 164 | 165 | ## References 166 | * YOLO v3 paper - https://arxiv.org/abs/1804.02767, Joseph Redmon, Ali Farhadi 167 | * YOLO v4 paper - https://arxiv.org/abs/2004.10934, Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao 168 | * YOLO v7 paper - https://arxiv.org/abs/2207.02696, Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao 169 | * Original Darknet YOLO repository - https://github.com/pjreddie/darknet 170 | * Most popular fork of Darknet YOLO - https://github.com/AlexeyAB/darknet 171 | * Developers of YOLOv8 - https://github.com/ultralytics/ultralytics. If you are aware of some original papers for YOLOv8 architecture, please contact me to mention it in this README. 172 | * Rust OpenCV's bindings - https://github.com/twistedfall/opencv-rust 173 | * Go OpenCV's bindings (for ready-to-go Makefile) - https://github.com/hybridgroup/gocv 174 | -------------------------------------------------------------------------------- /download_v3.sh: -------------------------------------------------------------------------------- 1 | curl https://pjreddie.com/media/files/yolov3.weights --create-dirs -o pretrained/yolov3.weights 2 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov3.cfg --create-dirs -o pretrained/yolov3.cfg 3 | -------------------------------------------------------------------------------- /download_v3_tiny.sh: -------------------------------------------------------------------------------- 1 | curl https://pjreddie.com/media/files/yolov3-tiny.weights --create-dirs -o pretrained/yolov3-tiny.weights 2 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov3-tiny.cfg --create-dirs -o pretrained/yolov3-tiny.cfg -------------------------------------------------------------------------------- /download_v4.sh: -------------------------------------------------------------------------------- 1 | mkdir -p pretrained 2 | wget https://github.com/AlexeyAB/darknet/releases/download/yolov4/yolov4.weights -O pretrained/yolov4.weights 3 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg --create-dirs -o pretrained/yolov4.cfg 4 | # Make sure that batch size is for inference needs only 5 | sed 's/batch=64/batch=1/g' -i pretrained/yolov4.cfg 6 | sed 's/subdivision=8/subdivision=1/g' -i pretrained/yolov4.cfg 7 | -------------------------------------------------------------------------------- /download_v4_tiny.sh: -------------------------------------------------------------------------------- 1 | curl https://github.com/AlexeyAB/darknet/releases/download/yolov4/yolov4-tiny.weights --create-dirs -o pretrained/yolov4-tiny.weights 2 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4-tiny.cfg --create-dirs -o pretrained/yolov4-tiny.cfg -------------------------------------------------------------------------------- /download_v7.sh: -------------------------------------------------------------------------------- 1 | mkdir -p pretrained 2 | wget https://github.com/AlexeyAB/darknet/releases/download/yolov4/yolov7.weights -O pretrained/yolov7.weights 3 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov7.cfg --create-dirs -o pretrained/yolov7.cfg 4 | # Make sure that batch size is for inference needs only 5 | sed 's/batch=8/batch=1/g' -i pretrained/yolov7.cfg 6 | -------------------------------------------------------------------------------- /download_v7_tiny.sh: -------------------------------------------------------------------------------- 1 | curl https://github.com/AlexeyAB/darknet/releases/download/yolov4/yolov7-tiny.weights --create-dirs -o pretrained/yolov7-tiny.weights 2 | curl https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov7-tiny.cfg --create-dirs -o pretrained/yolov7-tiny.cfg 3 | # Make sure that batch size is for inference needs only 4 | sed 's/batch=64/batch=1/g' -i pretrained/yolov7-tiny.cfg -------------------------------------------------------------------------------- /download_v8_l.sh: -------------------------------------------------------------------------------- 1 | # Not working cURL? 2 | # curl https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8l.pt --create-dirs -o pretrained/yolov8l.pt 3 | mkdir -p pretrained 4 | wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8l.pt -O pretrained/yolov8l.pt 5 | printf "\n\n" 6 | RED='\033[0;31m' 7 | NC='\033[0m' # No Color 8 | printf "${RED}Make sure that you have installed 'ultralytics' for Python environment${NC}" 9 | printf "\n\n" 10 | python3 -c 'from ultralytics import YOLO; model = YOLO("pretrained/yolov8l.pt"); model.export(format="onnx", opset=12)' -------------------------------------------------------------------------------- /download_v8_m.sh: -------------------------------------------------------------------------------- 1 | # Not working cURL? 2 | # curl https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt --create-dirs -o pretrained/yolov8m.pt 3 | mkdir -p pretrained 4 | wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt -O pretrained/yolov8m.pt 5 | printf "\n\n" 6 | RED='\033[0;31m' 7 | NC='\033[0m' # No Color 8 | printf "${RED}Make sure that you have installed 'ultralytics' for Python environment${NC}" 9 | printf "\n\n" 10 | python3 -c 'from ultralytics import YOLO; model = YOLO("pretrained/yolov8m.pt"); model.export(format="onnx", opset=12)' -------------------------------------------------------------------------------- /download_v8_n.sh: -------------------------------------------------------------------------------- 1 | # Not working cURL? 2 | # curl https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt --create-dirs -o pretrained/yolov8n.pt 3 | mkdir -p pretrained 4 | wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt -O pretrained/yolov8n.pt 5 | printf "\n\n" 6 | RED='\033[0;31m' 7 | NC='\033[0m' # No Color 8 | printf "${RED}Make sure that you have installed 'ultralytics' for Python environment${NC}" 9 | printf "\n\n" 10 | python3 -c 'from ultralytics import YOLO; model = YOLO("pretrained/yolov8n.pt"); model.export(format="onnx", opset=12)' -------------------------------------------------------------------------------- /download_v8_s.sh: -------------------------------------------------------------------------------- 1 | # Not working cURL? 2 | # curl https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt --create-dirs -o pretrained/yolov8s.pt 3 | mkdir -p pretrained 4 | wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt -O pretrained/yolov8s.pt 5 | printf "\n\n" 6 | RED='\033[0;31m' 7 | NC='\033[0m' # No Color 8 | printf "${RED}Make sure that you have installed 'ultralytics' for Python environment${NC}" 9 | printf "\n\n" 10 | python3 -c 'from ultralytics import YOLO; model = YOLO("pretrained/yolov8s.pt"); model.export(format="onnx", opset=12)' -------------------------------------------------------------------------------- /download_v8_x.sh: -------------------------------------------------------------------------------- 1 | # Not working cURL? 2 | # curl https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x.pt --create-dirs -o pretrained/yolov8x.pt 3 | mkdir -p pretrained 4 | wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x.pt -O pretrained/yolov8x.pt 5 | printf "\n\n" 6 | RED='\033[0;31m' 7 | NC='\033[0m' # No Color 8 | printf "${RED}Make sure that you have installed 'ultralytics' for Python environment${NC}" 9 | printf "\n\n" 10 | python3 -c 'from ultralytics import YOLO; model = YOLO("pretrained/yolov8x.pt"); model.export(format="onnx", opset=12)' -------------------------------------------------------------------------------- /examples/yolo_v3.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov3.weights", Some("pretrained/yolov3.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov3.jpg", &frame, &Vector::new()).unwrap(); 33 | } 34 | -------------------------------------------------------------------------------- /examples/yolo_v3_tiny.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov3-tiny.weights", Some("pretrained/yolov3-tiny.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov3_tiny.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v4.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov4.weights", Some("pretrained/yolov4.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov4.jpg", &frame, &Vector::new()).unwrap(); 33 | } 34 | -------------------------------------------------------------------------------- /examples/yolo_v4_tiny.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov4-tiny.weights", Some("pretrained/yolov4-tiny.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov4_tiny.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v4_tiny_onnx.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{model_classic::ModelYOLOClassic, model_format::ModelFormat}; 2 | 3 | use opencv::{ 4 | core::{Scalar, Vector}, 5 | dnn::DNN_BACKEND_CUDA, 6 | dnn::DNN_TARGET_CUDA, 7 | imgcodecs::imread, 8 | imgcodecs::imwrite, 9 | imgproc::rectangle, 10 | imgproc::LINE_4, 11 | }; 12 | 13 | fn main() { 14 | let classes_labels: Vec<&str> = vec!["car", "motorbike", "bus", "truck"]; 15 | let mf = ModelFormat::ONNX; 16 | let net_width = 416; 17 | let net_height = 416; 18 | // let class_filters: Vec = vec![15, 16]; 19 | let class_filters: Vec = vec![]; 20 | let mut model = ModelYOLOClassic::new_from_file( 21 | "pretrained/best_fp32.onnx", 22 | None, 23 | (net_width, net_height), 24 | mf, 25 | DNN_BACKEND_CUDA, 26 | DNN_TARGET_CUDA, 27 | vec![], 28 | ) 29 | .unwrap(); 30 | let mut frame = imread( 31 | "images/dog.jpg", 32 | 1, 33 | ) 34 | .unwrap(); 35 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.75, 0.4).unwrap(); 36 | for (i, bbox) in bboxes.iter().enumerate() { 37 | rectangle( 38 | &mut frame, 39 | *bbox, 40 | Scalar::from((0.0, 255.0, 0.0)), 41 | 2, 42 | LINE_4, 43 | 0, 44 | ) 45 | .unwrap(); 46 | println!("Class: {}", classes_labels[class_ids[i]]); 47 | println!("\tBounding box: {:?}", bbox); 48 | println!("\tConfidences: {}", confidences[i]); 49 | } 50 | imwrite("images/dog_yolov4_tiny.jpg", &frame, &Vector::new()).unwrap(); 51 | } 52 | -------------------------------------------------------------------------------- /examples/yolo_v7.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov7.weights", Some("pretrained/yolov7.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, class_filters).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov7.jpg", &frame, &Vector::new()).unwrap(); 33 | } 34 | -------------------------------------------------------------------------------- /examples/yolo_v7_tiny.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_classic::ModelYOLOClassic 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::Darknet; 19 | let net_width = 416; 20 | let net_height = 416; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov7-tiny.weights", Some("pretrained/yolov7-tiny.cfg"), (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, class_filters).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov7_tiny.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v8_l.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_ultralytics::ModelUltralyticsV8 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::ONNX; 19 | let net_width = 640; 20 | let net_height = 640; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8l.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov8_l.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v8_m.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_ultralytics::ModelUltralyticsV8 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::ONNX; 19 | let net_width = 640; 20 | let net_height = 640; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8m.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov8_m.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v8_n.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_ultralytics::ModelUltralyticsV8 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::ONNX; 19 | let net_width = 640; 20 | let net_height = 640; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8n.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov8_n.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v8_s.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_ultralytics::ModelUltralyticsV8 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::ONNX; 19 | let net_width = 640; 20 | let net_height = 640; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8s.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov8_s.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /examples/yolo_v8_x.rs: -------------------------------------------------------------------------------- 1 | use od_opencv::{ 2 | model_format::ModelFormat, 3 | model_ultralytics::ModelUltralyticsV8 4 | }; 5 | 6 | use opencv::{ 7 | core::{Scalar, Vector}, 8 | imgcodecs::imread, 9 | imgcodecs::imwrite, 10 | imgproc::LINE_4, 11 | imgproc::rectangle, 12 | dnn::DNN_BACKEND_CUDA, 13 | dnn::DNN_TARGET_CUDA, 14 | }; 15 | 16 | fn main() { 17 | let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 18 | let mf = ModelFormat::ONNX; 19 | let net_width = 640; 20 | let net_height = 640; 21 | // let class_filters: Vec = vec![15, 16]; 22 | let class_filters: Vec = vec![]; 23 | let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8x.onnx", None, (net_width, net_height), mf, DNN_BACKEND_CUDA, DNN_TARGET_CUDA, vec![]).unwrap(); 24 | let mut frame = imread("images/dog.jpg", 1).unwrap(); 25 | let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 26 | for (i, bbox) in bboxes.iter().enumerate() { 27 | rectangle(&mut frame, *bbox, Scalar::from((0.0, 255.0, 0.0)), 2, LINE_4, 0).unwrap(); 28 | println!("Class: {}", classes_labels[class_ids[i]]); 29 | println!("\tBounding box: {:?}", bbox); 30 | println!("\tConfidences: {}", confidences[i]); 31 | } 32 | imwrite("images/dog_yolov8_x.jpg", &frame, &Vector::new()).unwrap(); 33 | } -------------------------------------------------------------------------------- /images/dog.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog.jpg -------------------------------------------------------------------------------- /images/dog_yolov3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov3.jpg -------------------------------------------------------------------------------- /images/dog_yolov3_tiny.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov3_tiny.jpg -------------------------------------------------------------------------------- /images/dog_yolov4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov4.jpg -------------------------------------------------------------------------------- /images/dog_yolov4_tiny.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov4_tiny.jpg -------------------------------------------------------------------------------- /images/dog_yolov7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov7.jpg -------------------------------------------------------------------------------- /images/dog_yolov7_tiny.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov7_tiny.jpg -------------------------------------------------------------------------------- /images/dog_yolov8_l.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov8_l.jpg -------------------------------------------------------------------------------- /images/dog_yolov8_m.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov8_m.jpg -------------------------------------------------------------------------------- /images/dog_yolov8_n.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov8_n.jpg -------------------------------------------------------------------------------- /images/dog_yolov8_s.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov8_s.jpg -------------------------------------------------------------------------------- /images/dog_yolov8_x.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LdDl/object-detection-opencv-rust/5ace6d34a1c080645648953e3e453008701b8b26/images/dog_yolov8_x.jpg -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | pub mod utils; 2 | pub mod model_format; 3 | pub mod model_classic; 4 | pub mod model_ultralytics; 5 | pub mod model; 6 | -------------------------------------------------------------------------------- /src/model.rs: -------------------------------------------------------------------------------- 1 | use crate::model_format::{ 2 | ModelFormat, 3 | ModelVersion 4 | }; 5 | use crate::utils::FORMAT_VERSION_VALID; 6 | use crate::model_classic::ModelYOLOClassic; 7 | use crate::model_ultralytics::ModelUltralyticsV8; 8 | use opencv::{ 9 | core::Mat, 10 | core::Rect, 11 | dnn::Net, 12 | Error 13 | }; 14 | 15 | /// Just a trait wrapper for models 16 | /// Should be used in scenarios when you uncertain about model type in compile time 17 | pub trait ModelTrait { 18 | fn forward(&mut self, image: &Mat, conf_threshold: f32, nms_threshold: f32) -> Result<(Vec, Vec, Vec), Error>; 19 | } 20 | 21 | /// Creates model from file 22 | /// 23 | /// Resulting structure is just a trait wrapper for ModelYOLOClassic::new_from_file and ModelUltralyticsV8::new_from_file 24 | /// 25 | /// List of supported combinations model format / model version: 26 | /// | | ModelFormat::Darknet | ModelFormat::ONNX | 27 | /// |------------------------|----------------------|------------------| 28 | /// | ModelVersion::V3 | + | | 29 | /// | ModelVersion::V4 | + | + | 30 | /// | ModelVersion::V7 | + | | 31 | /// | ModelVersion::V8 | | + | 32 | /// 33 | /// List of supported combinations backend / target: 34 | /// | | DNN_BACKEND_OPENCV | DNN_BACKEND_INFERENCE_ENGINE | DNN_BACKEND_HALIDE | DNN_BACKEND_CUDA | 35 | /// |------------------------|--------------------|------------------------------|--------------------|-------------------| 36 | /// | DNN_TARGET_CPU | + | + | + | | 37 | /// | DNN_TARGET_OPENCL | + | + | + | | 38 | /// | DNN_TARGET_OPENCL_FP16 | + | + | | | 39 | /// | DNN_TARGET_MYRIAD | | + | | | 40 | /// | DNN_TARGET_FPGA | | + | | | 41 | /// | DNN_TARGET_CUDA | | | | + | 42 | /// | DNN_TARGET_CUDA_FP16 | | | | + | 43 | /// | DNN_TARGET_HDDL | | + | | | 44 | /// 45 | /// Basic usage: 46 | /// 47 | /// ``` 48 | /// use opencv::dnn::{DNN_BACKEND_OPENCV, DNN_TARGET_CPU}; 49 | /// use opencv::imgcodecs::imread; 50 | /// use od_opencv::model_format::{ModelFormat, ModelVersion}; 51 | /// use od_opencv::model::new_from_file; 52 | /// let mf = ModelFormat::Darknet; 53 | /// let mv = ModelVersion::V4; 54 | /// let net_width = 416; 55 | /// let net_height = 416; 56 | /// let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 57 | /// let filter_classes: Vec = vec![16]; // 16-th class is 'Dog' in COCO's 80 classes. So in this case model will output only dogs if it founds them. Make it empty and you will get all detected objects. 58 | /// let mut model = new_from_file("pretrained/yolov4-tiny.weights", Some("pretrained/yolov4-tiny.cfg"), (net_width, net_height), mf, mv, DNN_BACKEND_OPENCV, DNN_TARGET_CPU, filter_classes).unwrap(); 59 | /// let mut frame = imread("images/dog.jpg", 1).unwrap(); 60 | /// let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 61 | /// for (i, bbox) in bboxes.iter().enumerate() { 62 | /// println!("Class: {}", classes_labels[class_ids[i]]); 63 | /// println!("\tBounding box: {:?}", bbox); 64 | /// println!("\tConfidences: {}", confidences[i]); 65 | /// } 66 | /// ``` 67 | /// 68 | pub fn new_from_file(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, model_version: ModelVersion, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 69 | if FORMAT_VERSION_VALID.get(&model_format).and_then(|map| map.get(&model_version)).is_none() { 70 | return Err(Error::new(400, format!("Combination of model format '{}' and model version '{}' is not valid", model_format, model_version))); 71 | }; 72 | match model_version { 73 | ModelVersion::V3 | ModelVersion::V4 | ModelVersion::V7 => { 74 | return Ok(Box::new(ModelYOLOClassic::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 75 | }, 76 | ModelVersion::V8 => { 77 | return Ok(Box::new(ModelUltralyticsV8::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 78 | } 79 | } 80 | } 81 | 82 | /* Shorthands to Ultralytics versions */ 83 | 84 | /// Shorthand to ModelUltralyticsV8::new_from_file 85 | pub fn new_from_file_v8(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 86 | Ok(Box::new(ModelUltralyticsV8::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 87 | } 88 | 89 | /// Shorthand to ModelUltralyticsV8::new_from_darknet_file 90 | pub fn new_from_darknet_file_v8(weight_file_path: &str, cfg_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 91 | Ok(Box::new(ModelUltralyticsV8::new_from_darknet_file(weight_file_path, cfg_file_path, net_size, backend_id, target_id, filter_classes)?)) 92 | } 93 | 94 | /// Shorthand to ModelUltralyticsV8::new_from_onnx_file 95 | pub fn new_from_onnx_file_v8(weight_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 96 | Ok(Box::new(ModelUltralyticsV8::new_from_onnx_file(weight_file_path, net_size, backend_id, target_id, filter_classes)?)) 97 | } 98 | 99 | /// Shorthand to ModelUltralyticsV8::new_from_dnn 100 | #[allow(unused_mut)] 101 | pub fn new_from_dnn_v8(mut neural_net: Net, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 102 | Ok(Box::new(ModelUltralyticsV8::new_from_dnn(neural_net, net_size, backend_id, target_id, filter_classes)?)) 103 | } 104 | 105 | /* Shorthands to Classic versions */ 106 | 107 | /// Shorthand to ModelYOLOClassic::new_from_file 108 | pub fn new_from_file_v3(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 109 | Ok(Box::new(ModelYOLOClassic::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 110 | } 111 | 112 | /// Shorthand to ModelYOLOClassic::new_from_darknet_file 113 | pub fn new_from_darknet_file_v3(weight_file_path: &str, cfg_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 114 | Ok(Box::new(ModelYOLOClassic::new_from_darknet_file(weight_file_path, cfg_file_path, net_size, backend_id, target_id, filter_classes)?)) 115 | } 116 | 117 | /// Shorthand to ModelYOLOClassic::new_from_onnx_file 118 | pub fn new_from_onnx_file_v3(weight_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 119 | Ok(Box::new(ModelYOLOClassic::new_from_onnx_file(weight_file_path, net_size, backend_id, target_id, filter_classes)?)) 120 | } 121 | 122 | /// Shorthand to ModelYOLOClassic::new_from_dnn 123 | #[allow(unused_mut)] 124 | pub fn new_from_dnn_v3(mut neural_net: Net, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 125 | Ok(Box::new(ModelYOLOClassic::new_from_dnn(neural_net, net_size, backend_id, target_id, filter_classes)?)) 126 | } 127 | 128 | /// Shorthand to ModelYOLOClassic::new_from_file 129 | pub fn new_from_file_v4(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 130 | Ok(Box::new(ModelYOLOClassic::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 131 | } 132 | 133 | /// Shorthand to ModelYOLOClassic::new_from_darknet_file 134 | pub fn new_from_darknet_file_v4(weight_file_path: &str, cfg_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 135 | Ok(Box::new(ModelYOLOClassic::new_from_darknet_file(weight_file_path, cfg_file_path, net_size, backend_id, target_id, filter_classes)?)) 136 | } 137 | 138 | /// Shorthand to ModelYOLOClassic::new_from_onnx_file 139 | pub fn new_from_onnx_file_v4(weight_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 140 | Ok(Box::new(ModelYOLOClassic::new_from_onnx_file(weight_file_path, net_size, backend_id, target_id, filter_classes)?)) 141 | } 142 | 143 | /// Shorthand to ModelYOLOClassic::new_from_dnn 144 | #[allow(unused_mut)] 145 | pub fn new_from_dnn_v4(mut neural_net: Net, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 146 | Ok(Box::new(ModelYOLOClassic::new_from_dnn(neural_net, net_size, backend_id, target_id, filter_classes)?)) 147 | } 148 | 149 | /// Shorthand to ModelYOLOClassic::new_from_file 150 | pub fn new_from_file_v7(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 151 | Ok(Box::new(ModelYOLOClassic::new_from_file(weight_file_path, cfg_file_path, net_size, model_format, backend_id, target_id, filter_classes)?)) 152 | } 153 | 154 | /// Shorthand to ModelYOLOClassic::new_from_darknet_file 155 | pub fn new_from_darknet_file_v7(weight_file_path: &str, cfg_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 156 | Ok(Box::new(ModelYOLOClassic::new_from_darknet_file(weight_file_path, cfg_file_path, net_size, backend_id, target_id, filter_classes)?)) 157 | } 158 | 159 | /// Shorthand to ModelYOLOClassic::new_from_onnx_file 160 | pub fn new_from_onnx_file_v7(weight_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 161 | Ok(Box::new(ModelYOLOClassic::new_from_onnx_file(weight_file_path, net_size, backend_id, target_id, filter_classes)?)) 162 | } 163 | 164 | /// Shorthand to ModelYOLOClassic::new_from_dnn 165 | #[allow(unused_mut)] 166 | pub fn new_from_dnn_v7(mut neural_net: Net, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result, Error> { 167 | Ok(Box::new(ModelYOLOClassic::new_from_dnn(neural_net, net_size, backend_id, target_id, filter_classes)?)) 168 | } -------------------------------------------------------------------------------- /src/model_classic.rs: -------------------------------------------------------------------------------- 1 | use opencv::{ 2 | core::Mat, core::Rect, core::Scalar, core::Size, core::Vector, core::VectorToVec, core::CV_32F, 3 | dnn::blob_from_image, dnn::nms_boxes, dnn::read_net, dnn::read_net_from_onnx, dnn::Net, 4 | imgproc::resize, imgproc::INTER_AREA, prelude::MatTraitConst, prelude::MatTraitConstManual, 5 | prelude::NetTrait, prelude::NetTraitConst, Error, 6 | }; 7 | 8 | use crate::model::ModelTrait; 9 | use crate::model_format::ModelFormat; 10 | use crate::utils::BACKEND_TARGET_VALID; 11 | 12 | const YOLO_BLOB_MEAN: (f64, f64, f64, f64) = (0.0, 0.0, 0.0, 0.0); 13 | 14 | /// Wrapper around "classic" versions of YOLO (v3, v4 and v7 are supported currently) 15 | /// Why this versions is considered to be "classic"? Well, because they are developed historically by AlexeyAB and pjreddie (see the ref. https://github.com/AlexeyAB/darknet and https://github.com/pjreddie/darknet) 16 | pub struct ModelYOLOClassic { 17 | // Underlying OpenCV's DNN Net implementation 18 | net: Net, 19 | // Input size a.k.a network size (width and height of input). It is usefull when calculating relative bounding box to size of source image or resizing image to proper (width, height) 20 | input_size: Size, 21 | // Blob's scale for OpenCV's blob. For YOLO it is just (0.0 - red channel, 0.0 - blue, 0.0 - green, 0.0 - alpha) most of time (if it is not, than it is needed to make this field adjustable) 22 | blob_mean: Scalar, 23 | // Blob's scale for OpenCV's blob. For YOLO it is just 1/255 (~ 0.0039) most of time (if it is not, than it is needed to make this field adjustable) 24 | blob_scale: f64, 25 | // Blob's name basically for OpenCV's blob. For YOLO it is just an empty string most of time (if it is not, than it is needed to make this field adjustable) 26 | blob_name: &'static str, 27 | // Layers to aggregate results from (for some models there could me multiple YOLO layers) 28 | out_layers: Vector, 29 | // Set of classes which will be used to filter detections 30 | filter_classes: Vec, 31 | } 32 | 33 | impl ModelYOLOClassic { 34 | /// Read file for specified model format, BACKEND and TARGET combo and then prepares model. 35 | /// 36 | /// List of supported combinations backend / target: 37 | /// | | DNN_BACKEND_OPENCV | DNN_BACKEND_INFERENCE_ENGINE | DNN_BACKEND_HALIDE | DNN_BACKEND_CUDA | 38 | /// |------------------------|--------------------|------------------------------|--------------------|-------------------| 39 | /// | DNN_TARGET_CPU | + | + | + | | 40 | /// | DNN_TARGET_OPENCL | + | + | + | | 41 | /// | DNN_TARGET_OPENCL_FP16 | + | + | | | 42 | /// | DNN_TARGET_MYRIAD | | + | | | 43 | /// | DNN_TARGET_FPGA | | + | | | 44 | /// | DNN_TARGET_CUDA | | | | + | 45 | /// | DNN_TARGET_CUDA_FP16 | | | | + | 46 | /// | DNN_TARGET_HDDL | | + | | | 47 | /// 48 | /// Basic usage: 49 | /// 50 | /// ``` 51 | /// use opencv::dnn::{DNN_BACKEND_OPENCV, DNN_TARGET_CPU}; 52 | /// use opencv::imgcodecs::imread; 53 | /// use od_opencv::model_format::ModelFormat; 54 | /// use od_opencv::model_classic::ModelYOLOClassic; 55 | /// let mf = ModelFormat::Darknet; 56 | /// let net_width = 416; 57 | /// let net_height = 416; 58 | /// let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 59 | /// let filter_classes: Vec = vec![16]; // 16-th class is 'Dog' in COCO's 80 classes. So in this case model will output only dogs if it founds them. Make it empty and you will get all detected objects. 60 | /// let mut model = ModelYOLOClassic::new_from_file("pretrained/yolov4-tiny.weights", Some("pretrained/yolov4-tiny.cfg"), (net_width, net_height), mf, DNN_BACKEND_OPENCV, DNN_TARGET_CPU, filter_classes).unwrap(); 61 | /// let mut frame = imread("images/dog.jpg", 1).unwrap(); 62 | /// let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 63 | /// for (i, bbox) in bboxes.iter().enumerate() { 64 | /// println!("Class: {}", classes_labels[class_ids[i]]); 65 | /// println!("\tBounding box: {:?}", bbox); 66 | /// println!("\tConfidences: {}", confidences[i]); 67 | /// } 68 | /// ``` 69 | /// 70 | pub fn new_from_file( 71 | weight_file_path: &str, 72 | cfg_file_path: Option<&str>, 73 | net_size: (i32, i32), 74 | model_format: ModelFormat, 75 | backend_id: i32, 76 | target_id: i32, 77 | filter_classes: Vec, 78 | ) -> Result { 79 | if BACKEND_TARGET_VALID 80 | .get(&backend_id) 81 | .and_then(|map| map.get(&target_id)) 82 | .is_none() 83 | { 84 | return Err(Error::new( 85 | 400, 86 | format!( 87 | "Combination of BACKEND '{}' and TARGET '{}' is not valid", 88 | backend_id, target_id 89 | ), 90 | )); 91 | }; 92 | 93 | if model_format == ModelFormat::ONNX { 94 | return ModelYOLOClassic::new_from_onnx_file( 95 | weight_file_path, 96 | net_size, 97 | backend_id, 98 | target_id, 99 | filter_classes, 100 | ); 101 | } 102 | let cfg = match cfg_file_path { 103 | Some(s) => { 104 | if s == "" { 105 | return Err(Error::new(400, "Empty configuration file path")); 106 | } 107 | Ok(s) 108 | } 109 | None => Err(Error::new( 110 | 400, 111 | "No configuration file path has been provided", 112 | )), 113 | }?; 114 | 115 | ModelYOLOClassic::new_from_darknet_file( 116 | weight_file_path, 117 | cfg, 118 | net_size, 119 | backend_id, 120 | target_id, 121 | filter_classes, 122 | ) 123 | } 124 | /// Reads file in Darknet specification and prepares model 125 | pub fn new_from_darknet_file( 126 | weight_file_path: &str, 127 | cfg_file_path: &str, 128 | net_size: (i32, i32), 129 | backend_id: i32, 130 | target_id: i32, 131 | filter_classes: Vec, 132 | ) -> Result { 133 | ModelYOLOClassic::new_from_dnn( 134 | read_net(weight_file_path, cfg_file_path, "Darknet")?, 135 | net_size, 136 | backend_id, 137 | target_id, 138 | filter_classes, 139 | ) 140 | } 141 | /// Reads file in ONNX specification and prepares model 142 | pub fn new_from_onnx_file( 143 | weight_file_path: &str, 144 | net_size: (i32, i32), 145 | backend_id: i32, 146 | target_id: i32, 147 | filter_classes: Vec, 148 | ) -> Result { 149 | ModelYOLOClassic::new_from_dnn( 150 | read_net_from_onnx(weight_file_path)?, 151 | net_size, 152 | backend_id, 153 | target_id, 154 | filter_classes, 155 | ) 156 | } 157 | /// Prepares model from OpenCV's DNN neural network 158 | pub fn new_from_dnn( 159 | mut neural_net: Net, 160 | net_size: (i32, i32), 161 | backend_id: i32, 162 | target_id: i32, 163 | filter_classes: Vec, 164 | ) -> Result { 165 | neural_net.set_preferable_backend(backend_id)?; 166 | neural_net.set_preferable_target(target_id)?; 167 | let out_layers = neural_net.get_unconnected_out_layers_names()?; 168 | Ok(Self { 169 | net: neural_net, 170 | input_size: Size::new(net_size.0, net_size.1), 171 | blob_mean: Scalar::new( 172 | YOLO_BLOB_MEAN.0, 173 | YOLO_BLOB_MEAN.1, 174 | YOLO_BLOB_MEAN.2, 175 | YOLO_BLOB_MEAN.3, 176 | ), 177 | blob_scale: 1.0 / 255.0, 178 | blob_name: "", 179 | out_layers: out_layers, 180 | filter_classes: filter_classes, 181 | }) 182 | } 183 | pub fn forward( 184 | &mut self, 185 | image: &Mat, 186 | conf_threshold: f32, 187 | nms_threshold: f32, 188 | ) -> Result<(Vec, Vec, Vec), Error> { 189 | let image_width = image.cols(); 190 | let image_height = image.rows(); 191 | let image_width_f32 = image_width as f32; 192 | let image_height_f32 = image_height as f32; 193 | let need_to_resize = 194 | image_width != self.input_size.width || image_height != self.input_size.height; 195 | let blobimg = match need_to_resize { 196 | true => { 197 | let mut resized_frame: Mat = Mat::default(); 198 | resize( 199 | &image, 200 | &mut resized_frame, 201 | self.input_size, 202 | 1.0, 203 | 1.0, 204 | INTER_AREA, 205 | )?; 206 | blob_from_image( 207 | &resized_frame, 208 | self.blob_scale, 209 | self.input_size, 210 | self.blob_mean, 211 | true, 212 | false, 213 | CV_32F, 214 | )? 215 | } 216 | false => blob_from_image( 217 | &image, 218 | self.blob_scale, 219 | self.input_size, 220 | self.blob_mean, 221 | true, 222 | false, 223 | CV_32F, 224 | )?, 225 | }; 226 | let mut detections = Vector::::new(); 227 | self.net 228 | .set_input(&blobimg, self.blob_name, 1.0, self.blob_mean)?; 229 | self.net.forward(&mut detections, &self.out_layers)?; 230 | 231 | // Collect output data 232 | let mut bboxes = Vector::::new(); 233 | let mut confidences = Vector::::new(); 234 | let mut class_ids = Vec::new(); 235 | 236 | // Specific to YOLOv3, YOLOv4, YOLOv7 reading detections vector 237 | for layer in detections { 238 | let num_boxes = layer.rows(); 239 | for index in 0..num_boxes { 240 | let pred = layer.row(index)?; 241 | let detection = pred.data_typed::()?; 242 | let (center_x, center_y, width, height, confidence) = match &detection[0..5] { 243 | &[a,b,c,d,e] => (a * image_width_f32, b * image_height_f32, c * image_width_f32, d * image_height_f32, e), 244 | _ => { 245 | return Err(Error::new(500, "Can't extract (center_x, center_y, width, height, confidence) from detection vector")) 246 | } 247 | }; 248 | let detected_classes = &detection[5..]; 249 | if confidence > conf_threshold { 250 | let mut class_index = -1; 251 | let mut score = 0.0; 252 | for (idx, &val) in detected_classes.iter().enumerate() { 253 | if val > score { 254 | class_index = idx as i32; 255 | score = val; 256 | } 257 | } 258 | if class_index > -1 && score > 0. { 259 | let class_id = class_index as usize; 260 | if self.filter_classes.len() > 0 && !self.filter_classes.contains(&class_id) 261 | { 262 | continue; 263 | } 264 | let left = center_x - width / 2.0; 265 | let top = center_y - height / 2.0; 266 | let bbox = Rect::new( 267 | left.floor() as i32, 268 | top.floor() as i32, 269 | width as i32, 270 | height as i32, 271 | ); 272 | class_ids.push(class_id); 273 | confidences.push(confidence); 274 | bboxes.push(bbox); 275 | } 276 | } 277 | } 278 | } 279 | 280 | // Run NMS on collected detections to filter duplicates and overlappings 281 | let mut indices = Vector::::new(); 282 | nms_boxes( 283 | &bboxes, 284 | &confidences, 285 | conf_threshold, 286 | nms_threshold, 287 | &mut indices, 288 | 1.0, 289 | 0, 290 | )?; 291 | 292 | let mut nms_bboxes = vec![]; 293 | let mut nms_classes_ids = vec![]; 294 | let mut nms_confidences = vec![]; 295 | 296 | let indices_vec = indices.to_vec(); 297 | let mut bboxes = bboxes.to_vec(); 298 | nms_bboxes.extend(bboxes.drain(..).enumerate().filter_map(|(idx, item)| { 299 | if indices_vec.contains(&(idx as i32)) { 300 | Some(item) 301 | } else { 302 | None 303 | } 304 | })); 305 | 306 | nms_classes_ids.extend(class_ids.drain(..).enumerate().filter_map(|(idx, item)| { 307 | if indices_vec.contains(&(idx as i32)) { 308 | Some(item) 309 | } else { 310 | None 311 | } 312 | })); 313 | 314 | nms_confidences.extend(confidences.to_vec().drain(..).enumerate().filter_map( 315 | |(idx, item)| { 316 | if indices_vec.contains(&(idx as i32)) { 317 | Some(item) 318 | } else { 319 | None 320 | } 321 | }, 322 | )); 323 | 324 | Ok((nms_bboxes, nms_classes_ids, nms_confidences)) 325 | } 326 | } 327 | 328 | impl ModelTrait for ModelYOLOClassic { 329 | fn forward( 330 | &mut self, 331 | image: &Mat, 332 | conf_threshold: f32, 333 | nms_threshold: f32, 334 | ) -> Result<(Vec, Vec, Vec), Error> { 335 | self.forward(image, conf_threshold, nms_threshold) 336 | } 337 | } 338 | -------------------------------------------------------------------------------- /src/model_format.rs: -------------------------------------------------------------------------------- 1 | use std::fmt; 2 | 3 | /// Enum that should be used for constructing object detection model 4 | /// 5 | /// Basic usage: 6 | /// ``` 7 | /// use od_opencv::model_format::ModelFormat; 8 | /// let mf = ModelFormat::Darknet; 9 | /// ``` 10 | /// 11 | #[derive(PartialEq, Eq, Hash, Debug, Clone, Copy)] 12 | pub enum ModelFormat { 13 | // Most of time use Darknet specification when you are using either https://github.com/AlexeyAB/darknet or https://github.com/pjreddie/darknet to train neural network of version v3, v4 or v7 (classic ones). 14 | Darknet, 15 | // When you are using other DL/ML/NN framework than Darknet one it is better to convert weights to ONNX specification and use it further 16 | // It is better to converte weights to ONNX specification with providing flag `opset = 12`` (So far as I tested it do not show any runtime errors) - e.g. YOLOv8 - https://github.com/ultralytics/ultralytics/issues/1097 17 | ONNX, 18 | } 19 | 20 | impl fmt::Display for ModelFormat { 21 | fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { 22 | match self { 23 | ModelFormat::Darknet => write!(f, "darknet"), 24 | ModelFormat::ONNX => write!(f, "onnx"), 25 | } 26 | } 27 | } 28 | 29 | /// Enum that should be used for constructing object detection model 30 | /// 31 | /// Basic usage: 32 | /// ``` 33 | /// use od_opencv::model_format::ModelVersion; 34 | /// let mv = ModelVersion::V3; 35 | /// ``` 36 | /// 37 | #[derive(PartialEq, Eq, Hash, Debug, Clone, Copy)] 38 | pub enum ModelVersion { 39 | V3 = 3, 40 | V4 = 4, 41 | V7 = 5, 42 | V8 = 8, 43 | } 44 | 45 | impl fmt::Display for ModelVersion { 46 | fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { 47 | match self { 48 | ModelVersion::V3 => write!(f, "v3"), 49 | ModelVersion::V4 => write!(f, "v4"), 50 | ModelVersion::V7 => write!(f, "v7"), 51 | ModelVersion::V8 => write!(f, "v8") 52 | } 53 | } 54 | } -------------------------------------------------------------------------------- /src/model_ultralytics.rs: -------------------------------------------------------------------------------- 1 | use opencv::{ 2 | prelude::NetTrait, 3 | prelude::NetTraitConst, 4 | prelude::MatTraitConst, 5 | core::VectorToVec, 6 | core::Scalar, 7 | core::Size, 8 | core::Mat, 9 | core::Vector, 10 | core::Rect, 11 | core::CV_32F, 12 | dnn::read_net, 13 | dnn::read_net_from_onnx, 14 | dnn::blob_from_image, 15 | dnn::nms_boxes, 16 | dnn::Net, 17 | imgproc::resize, 18 | imgproc::INTER_AREA, 19 | Error 20 | }; 21 | 22 | use crate::model_format::ModelFormat; 23 | use crate::model::ModelTrait; 24 | use crate::utils::{ 25 | BACKEND_TARGET_VALID, 26 | min_max_loc_partial 27 | }; 28 | 29 | const YOLO_BLOB_MEAN: (f64, f64, f64, f64) = (0.0, 0.0, 0.0, 0.0); 30 | 31 | /// Wrapper around YOLOv8 32 | /// See the ref. https://github.com/ultralytics/ultralytics 33 | pub struct ModelUltralyticsV8 { 34 | // Underlying OpenCV's DNN Net implementation 35 | net: Net, 36 | // Input size a.k.a network size (width and height of input). It is usefull when calculating relative bounding box to size of source image or resizing image to proper (width, height) 37 | input_size: Size, 38 | // Blob's scale for OpenCV's blob. For YOLO it is just (0.0 - red channel, 0.0 - blue, 0.0 - green, 0.0 - alpha) most of time (if it is not, than it is needed to make this field adjustable) 39 | blob_mean: Scalar, 40 | // Blob's scale for OpenCV's blob. For YOLO it is just 1/255 (~ 0.0039) most of time (if it is not, than it is needed to make this field adjustable) 41 | blob_scale: f64, 42 | // Blob's name basically for OpenCV's blob. For YOLO it is just an empty string most of time (if it is not, than it is needed to make this field adjustable) 43 | blob_name: &'static str, 44 | // Layers to aggregate results from (for some models there could me multiple YOLO layers) 45 | out_layers: Vector, 46 | // Set of classes which will be used to filter detections 47 | filter_classes: Vec 48 | } 49 | 50 | impl ModelUltralyticsV8 { 51 | /// Read file for specified model format, BACKEND and TARGET combo and then prepares model. 52 | /// 53 | /// List of supported combinations backend / target: 54 | /// | | DNN_BACKEND_OPENCV | DNN_BACKEND_INFERENCE_ENGINE | DNN_BACKEND_HALIDE | DNN_BACKEND_CUDA | 55 | /// |------------------------|--------------------|------------------------------|--------------------|-------------------| 56 | /// | DNN_TARGET_CPU | + | + | + | | 57 | /// | DNN_TARGET_OPENCL | + | + | + | | 58 | /// | DNN_TARGET_OPENCL_FP16 | + | + | | | 59 | /// | DNN_TARGET_MYRIAD | | + | | | 60 | /// | DNN_TARGET_FPGA | | + | | | 61 | /// | DNN_TARGET_CUDA | | | | + | 62 | /// | DNN_TARGET_CUDA_FP16 | | | | + | 63 | /// | DNN_TARGET_HDDL | | + | | | 64 | /// 65 | /// Basic usage: 66 | /// 67 | /// ``` 68 | /// use opencv::dnn::{DNN_BACKEND_OPENCV, DNN_TARGET_CPU}; 69 | /// use opencv::imgcodecs::imread; 70 | /// use od_opencv::model_format::ModelFormat; 71 | /// use od_opencv::model_ultralytics::ModelUltralyticsV8; 72 | /// let mf = ModelFormat::ONNX; 73 | /// let net_width = 640; 74 | /// let net_height = 640; 75 | /// let classes_labels: Vec<&str> = vec!["person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]; 76 | /// let filter_classes: Vec = vec![16]; // 16-th class is 'Dog' in COCO's 80 classes. So in this case model will output only dogs if it founds them. Make it empty and you will get all detected objects. 77 | /// let mut model = ModelUltralyticsV8::new_from_file("pretrained/yolov8n.onnx", None, (net_width, net_height), mf, DNN_BACKEND_OPENCV, DNN_TARGET_CPU, filter_classes).unwrap(); 78 | /// let mut frame = imread("images/dog.jpg", 1).unwrap(); 79 | /// let (bboxes, class_ids, confidences) = model.forward(&frame, 0.25, 0.4).unwrap(); 80 | /// for (i, bbox) in bboxes.iter().enumerate() { 81 | /// println!("Class: {}", classes_labels[class_ids[i]]); 82 | /// println!("\tBounding box: {:?}", bbox); 83 | /// println!("\tConfidences: {}", confidences[i]); 84 | /// } 85 | /// ``` 86 | /// 87 | pub fn new_from_file(weight_file_path: &str, cfg_file_path: Option<&str>, net_size: (i32, i32), model_format: ModelFormat, backend_id: i32, target_id: i32, filter_classes: Vec) -> Result { 88 | if BACKEND_TARGET_VALID.get(&backend_id).and_then(|map| map.get(&target_id)).is_none() { 89 | return Err(Error::new(400, format!("Combination of BACKEND '{}' and TARGET '{}' is not valid", backend_id, target_id))); 90 | }; 91 | 92 | if model_format == ModelFormat::ONNX { 93 | return ModelUltralyticsV8::new_from_onnx_file(weight_file_path, net_size, backend_id, target_id, filter_classes) 94 | } 95 | let cfg = match cfg_file_path { 96 | Some(s) => { 97 | if s == "" { 98 | return Err(Error::new(400, "Empty configuration file path")) 99 | } 100 | Ok(s) 101 | }, 102 | None => { Err(Error::new(400, "No configuration file path has been provided")) } 103 | }?; 104 | 105 | ModelUltralyticsV8::new_from_darknet_file(weight_file_path, cfg, net_size, backend_id, target_id, filter_classes) 106 | } 107 | /// Reads file in Darknet specification and prepares model 108 | pub fn new_from_darknet_file(weight_file_path: &str, cfg_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result { 109 | ModelUltralyticsV8::new_from_dnn(read_net(weight_file_path, cfg_file_path, "Darknet")?, net_size, backend_id, target_id, filter_classes) 110 | } 111 | /// Reads file in ONNX specification and prepares model 112 | pub fn new_from_onnx_file(weight_file_path: &str, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result { 113 | ModelUltralyticsV8::new_from_dnn(read_net_from_onnx(weight_file_path)?, net_size, backend_id, target_id, filter_classes) 114 | } 115 | /// Prepares model from OpenCV's DNN neural network 116 | pub fn new_from_dnn(mut neural_net: Net, net_size: (i32, i32), backend_id: i32, target_id: i32, filter_classes: Vec) -> Result { 117 | neural_net.set_preferable_backend(backend_id)?; 118 | neural_net.set_preferable_target(target_id)?; 119 | let out_layers = neural_net.get_unconnected_out_layers_names()?; 120 | Ok(Self{ 121 | net: neural_net, 122 | input_size: Size::new(net_size.0, net_size.1), 123 | blob_mean: Scalar::new(YOLO_BLOB_MEAN.0, YOLO_BLOB_MEAN.1, YOLO_BLOB_MEAN.2, YOLO_BLOB_MEAN.3), 124 | blob_scale: 1.0 / 255.0, 125 | blob_name: "", 126 | out_layers: out_layers, 127 | filter_classes: filter_classes 128 | }) 129 | } 130 | pub fn forward(&mut self, image: &Mat, conf_threshold: f32, nms_threshold: f32) -> Result<(Vec, Vec, Vec), Error>{ 131 | let image_width = image.cols(); 132 | let image_height = image.rows(); 133 | let x_factor = image_width as f32 / self.input_size.width as f32; 134 | let y_factor = image_height as f32 / self.input_size.height as f32; 135 | let need_to_resize = image_width != self.input_size.width || image_height != self.input_size.height; 136 | let blobimg = match need_to_resize { 137 | true => { 138 | let mut resized_frame: Mat = Mat::default(); 139 | resize(&image, &mut resized_frame, self.input_size, 1.0, 1.0, INTER_AREA)?; 140 | blob_from_image(&resized_frame, self.blob_scale, self.input_size, self.blob_mean, true, false, CV_32F)? 141 | }, 142 | false => { 143 | blob_from_image(&image, self.blob_scale, self.input_size, self.blob_mean, true, false, CV_32F)? 144 | } 145 | }; 146 | let mut detections = Vector::::new(); 147 | self.net.set_input(&blobimg, self.blob_name, 1.0, self.blob_mean)?; 148 | self.net.forward(&mut detections, &self.out_layers)?; 149 | 150 | // Collect output data 151 | let mut bboxes = Vector::::new(); 152 | let mut confidences = Vector::::new(); 153 | let mut class_ids = Vec::new(); 154 | 155 | // Specific to YOLOv8 reading detections vector 156 | // See the ref. https://github.com/ultralytics/ultralytics/blob/main/examples/YOLOv8-OpenCV-ONNX-Python/main.py#L65 157 | for layer in detections { 158 | let mat_size = layer.mat_size(); 159 | let cols = mat_size[1]; 160 | let rows: i32 = mat_size[2]; 161 | for i in 0..rows{ 162 | // Access elements along the second dimension 163 | let object_data: Vec<&f32> = (0..cols) 164 | .map(|j| layer.at_3d::(0, j, i).unwrap()) 165 | .collect(); 166 | // Access elements as if transposed 167 | let classes_scores: Vec<&f32> = object_data[4..].to_vec(); 168 | // Find min and max scores and locations 169 | let (_, max_score, _, max_class_index) = match min_max_loc_partial(&classes_scores) { 170 | Some((a, b, c, d)) => { 171 | (a, b, c, d) 172 | }, 173 | None => { 174 | return Err(Error::new(500, "Can't execute min_max_loc_partial due the class_scores is an empty array")); 175 | } 176 | }; 177 | if max_score >= 0.25 { 178 | if self.filter_classes.len() > 0 && !self.filter_classes.contains(&max_class_index) { 179 | continue; 180 | } 181 | // Calculate box coordinates 182 | let bbox: [i32; 4] = [ 183 | ((object_data[0] - (0.5 * object_data[2])) * x_factor).round() as i32, 184 | ((object_data[1] - (0.5 * object_data[3])) * y_factor).round() as i32, 185 | (*object_data[2] * x_factor).round() as i32, 186 | (*object_data[3] * y_factor).round() as i32, 187 | ]; 188 | let bbox_cv = Rect::new(bbox[0], bbox[1], bbox[2], bbox[3]); 189 | bboxes.push(bbox_cv); 190 | confidences.push(max_score); 191 | class_ids.push(max_class_index); 192 | } 193 | } 194 | } 195 | // Run NMS on collected detections to filter duplicates and overlappings 196 | let mut indices = Vector::::new(); 197 | nms_boxes(&bboxes, &confidences, conf_threshold, nms_threshold, &mut indices, 1.0, 0)?; 198 | 199 | let mut nms_bboxes = vec![]; 200 | let mut nms_classes_ids = vec![]; 201 | let mut nms_confidences = vec![]; 202 | 203 | let indices_vec = indices.to_vec(); 204 | let mut bboxes = bboxes.to_vec(); 205 | nms_bboxes.extend(bboxes.drain(..) 206 | .enumerate() 207 | .filter_map(|(idx, item)| if indices_vec.contains(&(idx as i32)) {Some(item)} else {None})); 208 | 209 | nms_classes_ids.extend(class_ids.drain(..) 210 | .enumerate() 211 | .filter_map(|(idx, item)| if indices_vec.contains(&(idx as i32)) {Some(item)} else {None})); 212 | 213 | nms_confidences.extend(confidences.to_vec().drain(..) 214 | .enumerate() 215 | .filter_map(|(idx, item)| if indices_vec.contains(&(idx as i32)) {Some(item)} else {None})); 216 | 217 | Ok((nms_bboxes, nms_classes_ids, nms_confidences)) 218 | } 219 | } 220 | 221 | impl ModelTrait for ModelUltralyticsV8 { 222 | fn forward(&mut self, image: &Mat, conf_threshold: f32, nms_threshold: f32) -> Result<(Vec, Vec, Vec), Error>{ 223 | self.forward(image, conf_threshold, nms_threshold) 224 | } 225 | } -------------------------------------------------------------------------------- /src/utils.rs: -------------------------------------------------------------------------------- 1 | use opencv::{ 2 | dnn::DNN_BACKEND_OPENCV, 3 | dnn::DNN_BACKEND_INFERENCE_ENGINE, 4 | dnn::DNN_BACKEND_HALIDE, 5 | dnn::DNN_BACKEND_CUDA, 6 | dnn::DNN_TARGET_CPU, 7 | dnn::DNN_TARGET_OPENCL, 8 | dnn::DNN_TARGET_OPENCL_FP16, 9 | dnn::DNN_TARGET_MYRIAD, 10 | dnn::DNN_TARGET_FPGA, 11 | dnn::DNN_TARGET_CUDA, 12 | dnn::DNN_TARGET_CUDA_FP16, 13 | dnn::DNN_TARGET_HDDL, 14 | }; 15 | use std::collections::{HashMap, HashSet}; 16 | use std::iter::FromIterator; 17 | use crate::model_format::{ 18 | ModelFormat, 19 | ModelVersion 20 | }; 21 | use lazy_static::lazy_static; 22 | 23 | // Utilize lazy_static macro to create compatibility table for OpenCV's DNN backends and targets valid combinations 24 | lazy_static!{ 25 | pub static ref BACKEND_TARGET_VALID: HashMap> = vec![ 26 | (DNN_BACKEND_OPENCV, HashSet::from_iter(vec![DNN_TARGET_CPU, DNN_TARGET_OPENCL, DNN_TARGET_OPENCL_FP16])), 27 | (DNN_BACKEND_INFERENCE_ENGINE, HashSet::from_iter(vec![DNN_TARGET_CPU, DNN_TARGET_OPENCL, DNN_TARGET_OPENCL_FP16, DNN_TARGET_MYRIAD, DNN_TARGET_FPGA, DNN_TARGET_HDDL])), 28 | (DNN_BACKEND_HALIDE, HashSet::from_iter(vec![DNN_TARGET_CPU, DNN_TARGET_OPENCL])), 29 | (DNN_BACKEND_CUDA, HashSet::from_iter(vec![DNN_TARGET_CUDA, DNN_TARGET_CUDA_FP16])), 30 | ].into_iter().collect(); 31 | } 32 | 33 | lazy_static!{ 34 | pub static ref FORMAT_VERSION_VALID: HashMap> = vec![ 35 | (ModelFormat::Darknet, HashSet::from_iter(vec![ModelVersion::V3, ModelVersion::V4, ModelVersion::V7])), 36 | (ModelFormat::ONNX, HashSet::from_iter(vec![ModelVersion::V4, ModelVersion::V8])), 37 | ].into_iter().collect(); 38 | } 39 | 40 | 41 | /// Implementation of minMaxLoc in OpenCV (see the ref. https://docs.opencv.org/4.8.0/d2/de8/group__core__array.html#gab473bf2eb6d14ff97e89b355dac20707) basically. 42 | /// Returns min, max and its indices in array. 43 | /// Notice that it is for &[&T] rather than &[T] 44 | /// Basic usage: 45 | /// ``` 46 | /// use od_opencv::utils::min_max_loc_partial; 47 | /// let x = vec![&3.0, &4.0, &1.0, &2.0, &9.0, &0.5, &8.0]; 48 | /// let (min, max, min_loc, max_loc) = min_max_loc_partial(&x).unwrap(); 49 | /// println!("Min: {} at pos {}. Max: {} at pos {}", min, max, min_loc, max_loc); 50 | /// ``` 51 | /// 52 | pub fn min_max_loc_partial(data: &[&T]) -> Option<(T, T, usize, usize)> { 53 | if data.is_empty() { 54 | return None; 55 | } 56 | 57 | let mut min_val = data[0]; 58 | let mut min_loc = 0; 59 | let mut max_val = data[0]; 60 | let mut max_loc = 0; 61 | 62 | for (i, &val) in data.iter().enumerate().skip(1) { 63 | if val < min_val { 64 | min_val = val; 65 | min_loc = i; 66 | } else if val > max_val { 67 | max_val = val; 68 | max_loc = i; 69 | } 70 | } 71 | 72 | Some((*min_val, *max_val, min_loc, max_loc)) 73 | } 74 | 75 | mod tests { 76 | #[test] 77 | fn test_min_max_loc_partial() { 78 | let x = vec![&3.0, &4.0, &1.0, &2.0, &9.0, &0.5, &8.0]; 79 | let (min, max, min_loc, max_loc) = crate::utils::min_max_loc_partial(&x).unwrap(); 80 | assert_eq!(0.5, min); 81 | assert_eq!(5, min_loc); 82 | assert_eq!(9.0, max); 83 | assert_eq!(4, max_loc); 84 | } 85 | } --------------------------------------------------------------------------------