├── COCO_labels.txt ├── James.mp4 ├── LICENSE ├── MobileNetV1.cpp ├── README.md ├── TestTensorFlow_Lite.cbp └── detect.tflite /COCO_labels.txt: -------------------------------------------------------------------------------- 1 | unlabeled 2 | person 3 | bicycle 4 | car 5 | motorcycle 6 | airplane 7 | bus 8 | train 9 | truck 10 | boat 11 | trafficlight 12 | firehydrant 13 | streetsign 14 | stopsign 15 | parkingmeter 16 | bench 17 | bird 18 | cat 19 | dog 20 | horse 21 | sheep 22 | cow 23 | elephant 24 | bear 25 | zebra 26 | giraffe 27 | hat 28 | backpack 29 | umbrella 30 | shoe 31 | eyeglasses 32 | handbag 33 | tie 34 | suitcase 35 | frisbee 36 | skis 37 | snowboard 38 | sportsball 39 | kite 40 | baseballbat 41 | baseballglove 42 | skateboard 43 | surfboard 44 | tennisracket 45 | bottle 46 | plate 47 | wineglass 48 | cup 49 | fork 50 | knife 51 | spoon 52 | bowl 53 | banana 54 | apple 55 | sandwich 56 | orange 57 | broccoli 58 | carrot 59 | hotdog 60 | pizza 61 | donut 62 | cake 63 | chair 64 | couch 65 | pottedplant 66 | bed 67 | mirror 68 | diningtable 69 | window 70 | desk 71 | toilet 72 | door 73 | tv 74 | laptop 75 | mouse 76 | remote 77 | keyboard 78 | cellphone 79 | microwave 80 | oven 81 | toaster 82 | sink 83 | refrigerator 84 | blender 85 | book 86 | clock 87 | vase 88 | scissors 89 | teddybear 90 | hairdrier 91 | toothbrush -------------------------------------------------------------------------------- /James.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Qengineering/TensorFlow_Lite_SSD_RPi_64-bits/9d4aa7988e0c9bf516ef0f9969719af33c6dcfa3/James.mp4 -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2021, Q-engineering 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | 3. Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /MobileNetV1.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include "tensorflow/lite/interpreter.h" 9 | #include "tensorflow/lite/kernels/register.h" 10 | #include "tensorflow/lite/string_util.h" 11 | #include "tensorflow/lite/model.h" 12 | #include 13 | 14 | using namespace cv; 15 | using namespace std; 16 | 17 | const size_t width = 300; 18 | const size_t height = 300; 19 | 20 | std::vector Labels; 21 | std::unique_ptr interpreter; 22 | 23 | static bool getFileContent(std::string fileName) 24 | { 25 | 26 | // Open the File 27 | std::ifstream in(fileName.c_str()); 28 | // Check if object is valid 29 | if(!in.is_open()) return false; 30 | 31 | std::string str; 32 | // Read the next line from File untill it reaches the end. 33 | while (std::getline(in, str)) 34 | { 35 | // Line contains string of length > 0 then save it in vector 36 | if(str.size()>0) Labels.push_back(str); 37 | } 38 | // Close The File 39 | in.close(); 40 | return true; 41 | } 42 | 43 | void detect_from_video(Mat &src) 44 | { 45 | Mat image; 46 | int cam_width =src.cols; 47 | int cam_height=src.rows; 48 | 49 | // copy image to input as input tensor 50 | cv::resize(src, image, Size(300,300)); 51 | memcpy(interpreter->typed_input_tensor(0), image.data, image.total() * image.elemSize()); 52 | 53 | interpreter->SetAllowFp16PrecisionForFp32(true); 54 | interpreter->SetNumThreads(4); //quad core 55 | 56 | // cout << "tensors size: " << interpreter->tensors_size() << "\n"; 57 | // cout << "nodes size: " << interpreter->nodes_size() << "\n"; 58 | // cout << "inputs: " << interpreter->inputs().size() << "\n"; 59 | // cout << "input(0) name: " << interpreter->GetInputName(0) << "\n"; 60 | // cout << "outputs: " << interpreter->outputs().size() << "\n"; 61 | 62 | interpreter->Invoke(); // run your model 63 | 64 | const float* detection_locations = interpreter->tensor(interpreter->outputs()[0])->data.f; 65 | const float* detection_classes=interpreter->tensor(interpreter->outputs()[1])->data.f; 66 | const float* detection_scores = interpreter->tensor(interpreter->outputs()[2])->data.f; 67 | const int num_detections = *interpreter->tensor(interpreter->outputs()[3])->data.f; 68 | 69 | //there are ALWAYS 10 detections no matter how many objects are detectable 70 | // cout << "number of detections: " << num_detections << "\n"; 71 | 72 | const float confidence_threshold = 0.5; 73 | for(int i = 0; i < num_detections; i++){ 74 | if(detection_scores[i] > confidence_threshold){ 75 | int det_index = (int)detection_classes[i]+1; 76 | float y1=detection_locations[4*i ]*cam_height; 77 | float x1=detection_locations[4*i+1]*cam_width; 78 | float y2=detection_locations[4*i+2]*cam_height; 79 | float x2=detection_locations[4*i+3]*cam_width; 80 | 81 | Rect rec((int)x1, (int)y1, (int)(x2 - x1), (int)(y2 - y1)); 82 | rectangle(src,rec, Scalar(0, 0, 255), 1, 8, 0); 83 | putText(src, format("%s", Labels[det_index].c_str()), Point(x1, y1-5) ,FONT_HERSHEY_SIMPLEX,0.5, Scalar(0, 0, 255), 1, 8, 0); 84 | } 85 | } 86 | } 87 | 88 | int main(int argc,char ** argv) 89 | { 90 | float f; 91 | float FPS[16]; 92 | int i, Fcnt=0; 93 | Mat frame; 94 | chrono::steady_clock::time_point Tbegin, Tend; 95 | 96 | for(i=0;i<16;i++) FPS[i]=0.0; 97 | 98 | // Load model 99 | std::unique_ptr model = tflite::FlatBufferModel::BuildFromFile("detect.tflite"); 100 | 101 | // Build the interpreter 102 | tflite::ops::builtin::BuiltinOpResolver resolver; 103 | tflite::InterpreterBuilder(*model.get(), resolver)(&interpreter); 104 | 105 | interpreter->AllocateTensors(); 106 | 107 | // Get the names 108 | bool result = getFileContent("COCO_labels.txt"); 109 | if(!result) 110 | { 111 | cout << "loading labels failed"; 112 | exit(-1); 113 | } 114 | 115 | VideoCapture cap("James.mp4"); 116 | if (!cap.isOpened()) { 117 | cerr << "ERROR: Unable to open the camera" << endl; 118 | return 0; 119 | } 120 | 121 | cout << "Start grabbing, press ESC on Live window to terminate" << endl; 122 | while(1){ 123 | // frame=imread("Traffic.jpg"); //need to refresh frame before dnn class detection 124 | cap >> frame; 125 | if (frame.empty()) { 126 | cerr << "ERROR: Unable to grab from the camera" << endl; 127 | break; 128 | } 129 | 130 | Tbegin = chrono::steady_clock::now(); 131 | 132 | detect_from_video(frame); 133 | 134 | Tend = chrono::steady_clock::now(); 135 | //calculate frame rate 136 | f = chrono::duration_cast (Tend - Tbegin).count(); 137 | if(f>0.0) FPS[((Fcnt++)&0x0F)]=1000.0/f; 138 | for(f=0.0, i=0;i<16;i++){ f+=FPS[i]; } 139 | putText(frame, format("FPS %0.2f", f/16),Point(10,20),FONT_HERSHEY_SIMPLEX,0.6, Scalar(0, 0, 255)); 140 | 141 | //show output 142 | // cout << "FPS" << f/16 << endl; 143 | imshow("RPi 4 - 1,9 GHz - 2 Mb RAM", frame); 144 | 145 | char esc = waitKey(5); 146 | if(esc == 27) break; 147 | } 148 | 149 | cout << "Closing the camera" << endl; 150 | destroyAllWindows(); 151 | cout << "Bye!" << endl; 152 | 153 | return 0; 154 | } 155 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![output image](https://qengineering.eu/images/SDcard16GB_tiny.jpg) Find this example on our [SD-image](https://github.com/Qengineering/RPi-image) 2 | # TensorFlow_Lite_SSD_RPi_64-bits 3 | ![output image]( https://qengineering.eu/images/James_24.jpg )
4 | ## TensorFlow Lite SSD running at 24 FPS on a bare Raspberry Pi 4 64-OS
5 | [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)

6 | 7 | A fast C++ implementation of TensorFlow Lite on a bare Raspberry Pi 4 64-bit OS. 8 | Once overclocked to 1925 MHz, the app runs a whopping 24 FPS! 9 | Without any hardware accelerator, just you and your Pi. 10 | 11 | https://arxiv.org/abs/1611.10012
12 | Training set: COCO
13 | Size: 300x300
14 | Frame rate V1 Lite : 28 FPS (RPi 4 @ 1925 MHz - 64 bits Bullseye OS)
15 | Frame rate V1 Lite : 17 FPS (RPi 4 @ 2000 MHz - 32 bits OS) see [32-OS](https://github.com/Qengineering/TensorFlow_Lite_SSD_RPi_32-bits)
16 |
17 | Special made for a Raspberry Pi 4 see [Q-engineering deep learning examples](https://qengineering.eu/deep-learning-examples-on-raspberry-32-64-os.html)
18 |
19 | To extract and run the network in Code::Blocks
20 | $ mkdir *MyDir*
21 | $ cd *MyDir*
22 | $ wget https://github.com/Qengineering/TensorFlow_Lite_SSD_RPi_64-bits/archive/refs/heads/master.zip
23 | $ unzip -j master.zip
24 | Remove master.zip and README.md as they are no longer needed.
25 | $ rm master.zip
26 | $ rm README.md

27 | Your *MyDir* folder must now look like this:
28 | James.mp4
29 | COCO_labels.txt
30 | detect.tflite
31 | TestTensorFlow_Lite.cpb
32 | MobileNetV1.cpp
33 |
34 | Run TestTensorFlow_Lite.cpb with Code::Blocks. More info or
35 | if you want to connect a camera to the app, follow the instructions at [Hands-On](https://qengineering.eu/deep-learning-examples-on-raspberry-32-64-os.html#HandsOn).
36 | I fact you can run this example on any aarch64 Linux system.

37 | See the movie at: https://vimeo.com/393889226 38 | 39 | ------------ 40 | 41 | [![paypal](https://qengineering.eu/images/TipJarSmall4.png)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=CPZTM5BB3FCYL) 42 | 43 | 44 | -------------------------------------------------------------------------------- /TestTensorFlow_Lite.cbp: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 60 | 61 | -------------------------------------------------------------------------------- /detect.tflite: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Qengineering/TensorFlow_Lite_SSD_RPi_64-bits/9d4aa7988e0c9bf516ef0f9969719af33c6dcfa3/detect.tflite --------------------------------------------------------------------------------