├── README.md ├── app ├── hog_svm_test │ ├── compile.sh │ ├── dma_simple.c │ ├── dma_simple.h │ ├── frame.png │ ├── json11.cpp │ ├── json11.hpp │ ├── main.cpp │ └── weights.json └── realtime_webcam │ ├── compile.sh │ ├── dma_simple.c │ ├── dma_simple.h │ ├── json11.cpp │ ├── json11.hpp │ ├── main.cpp │ └── weights.json ├── cpp ├── frametest │ ├── compile_frametest.sh │ ├── feature.h │ ├── frame.png │ ├── frametest.cpp │ └── weights.h ├── hogtest │ ├── hogtest.cpp │ ├── input.png │ ├── myhog.py │ └── weights.h └── realtimetest │ ├── compile_realtimetest.sh │ ├── feature.h │ ├── realtimetest.cpp │ └── weights.h ├── data.zip ├── hls └── hog_svm │ ├── consts.h │ ├── frame.png │ ├── main.cpp │ ├── main_tb.cpp │ ├── solution1 │ ├── directives.tcl │ ├── impl │ │ └── ip │ │ │ └── xilinx_com_hls_hog_svm_1_0.zip │ └── script.tcl │ ├── vivado_hls.app │ └── weight.h ├── overlay ├── bingen.sh ├── design_1_wrapper.bit ├── dma0.dts ├── fclk0-zynqmp.dts ├── fpga-load.dts ├── fpga.bif ├── fpga.bin ├── hog_svm.dts ├── init.sh ├── udmabuf0.dts └── udmabuf1.dts ├── python ├── README.md ├── estimatetestor.py ├── myestimtaor.py ├── myhog.py ├── train.py ├── vdtools.py └── vdtools.pyc ├── util ├── README.md └── ap_fixed_convert │ ├── main.cpp │ ├── main_tb.cpp │ ├── solution1 │ ├── directives.tcl │ └── script.tcl │ ├── vivado_hls.app │ └── weights.h └── vivado ├── Ultra96_constraints_180318.xdc ├── circuit.png ├── create_bd.tcl └── create_project.tcl /README.md: -------------------------------------------------------------------------------- 1 | # ImageDetectionHW2 2 | **The 'heart2019' tag is stable** https://github.com/lp6m/ImageDetectionHW2/tree/heart2019 3 | HOG/BGRHSV + SVM FPGA Implementation of Object Detection 4 | This system detects red traffic signal from USB webcam captured image in real time. 5 | System runs on Debian/Ubuntu OS on Ultra96(rev1) development board. HW Object Detection Accelerator is implemented in FPGA. 6 | HW Accelerator is developed by HLS, which calculates the probability of being a red traffic signal for 891(32pix\*64pix) regions in (240pix\*320pix) BGR image. 7 | Detail of Algorithm is described in `hls/README.md` 8 | ![result](https://github.com/lp6m/ImageDetectionHW2/blob/image/result.png?raw=true) 9 | - HLS IP Latency: min:153838 max:196936 (not fully pipelined) 10 | - Achieved FPS: pver 142fps (including DMA data transmission time, more than 200 times faster than SW) 11 | 12 | ## Application Example 13 | This system is equipped with the miniature robot car. 14 | Youtube Link: 15 | [![](https://img.youtube.com/vi/qpykwwZaatw/0.jpg)](https://www.youtube.com/watch?v=qpykwwZaatw) 16 | [![](https://img.youtube.com/vi/zwXOoCu3Y1Q/0.jpg)](https://www.youtube.com/watch?v=zwXOoCu3Y1Q) 17 | 18 | # Usage 19 | ### Create SD Card for Ultra96 20 | First create SD Card for Ultra96 following this instruction: [ZynqMP-FPGA-Linux](https://github.com/ikwzm/ZynqMP-FPGA-Linux). 21 | You must install udmabuf device driver following the [instruction](https://github.com/ikwzm/ZynqMP-FPGA-Linux/blob/master/doc/build/device-drivers.md). 22 | Japanese instruction is [here](https://qiita.com/ikwzm/items/975ab6997905700dd2e0). 23 | ### Install OpenCV 24 | After you finish setup of Debian OS on Ultra96, install OpenCV from source code like [this](https://gist.github.com/okanon/c09669f3ff3351c864742bc2754b01ea). In my environment, I use OpenCV-3.4.1 version. It takes a while. 25 | ### Copy files to SD Card 26 | Copy `overlay/` and `app/` directory of this repository to rootfs partition of your SD card. 27 | Steps after this should be done on Ultra96 board. 28 | ### FPGA Configuration and Device Tree Overlay 29 | Connect Ultra96 board by USB-serial port or ssh connection. 30 | ``` 31 | fpga@debian-fpga$ sudo -s 32 | root@debian-fpga$ cd ~/overlay 33 | root@debian-fpga$ cp fpga.bin /lib/firmware/ 34 | root@debian-fpga:~/overlay# ./init.sh 35 | hog_svm.dtb: Warning (unit_address_vs_reg): /fragment@0/__overlay__/hog_svm_0: node has a reg or ranges property, but no unit name 36 | dma0.dtb: Warning (unit_address_vs_reg): /fragment@0/__overlay__/axi_dma_0: node has a reg or ranges property, but no unit name 37 | ``` 38 | FPGA configuration is done by this command. 39 | ### Run demo application on Ultra96. 40 | #### 1. detect traffic signal from one frame image. 41 | ``` 42 | root@debian-fpga:~# cd ~/app/hog_svm_test 43 | root@debian-fpga:~/app/hog_svm_test# sh compile.sh 44 | root@debian-fpga:~/app/hog_svm_test# ./a.out 45 | 1.6268558502 0.8357384801 46 | 2.9449763298 0.9500254989 47 | 2.1004710197 0.8909489512 48 | elapsed:6.0000000000[milisec] 49 | fps:166.6666666667[fps] 50 | ``` 51 | `result.png` is generated as output. This file should be like [this](https://github.com/lp6m/ImageDetectionHW2/blob/image/frame.png?raw=true). 52 | 53 | #### 2. detect traffic signal from captured image in real-time. 54 | First plug USB webcam to Ultra96. 55 | ``` 56 | root@debian-fpga:~# cd ~/app/realtime_webcam 57 | root@debian-fpga:~/app/realtime_webcam# sh compile.sh 58 | root@debian-fpga:~/app/realtime_webcam# ./a.out 59 | ``` 60 | If red traffic signal is detected, the probability is printed to standard output. 61 | To avoid false positive, detection region is used. 62 | # Re-train and use updated SVM weight 63 | In this project, trained parameters of SVM used in HLS IP are saved in BRAM, and you can update parameter from SW via AXI Bram Controller. 64 | ### Run re-train in `python/train.py` 65 | Modify `self.load_saved` in `python/vdtools.py` to `False`, and run `python/train.py` 66 | ### Extract trained value from cached SVM model. 67 | Run `python/myestimator.py`. This code extracts trained SVM paramter. Output file is genrated to `output/weights.h`. 68 | ### Convert float point value to 32bit unsigned int expression of fixed point. 69 | HLS IP uses 32bit fixed point (ap_fixed<32,10>). When updating the SVM weight from SW, the weight is expressed as a 32-bit unsigned int value. Therefore, it is necessary to convert the 32-bit fixed point to the 32-bit unsigned int of the same bit string. 70 | Copy `python/output/weights.h` to `util/ap_fixed_convert/weight.h`. And run *ap_fixed_convert* on Vivado HLS, copy json string output to `app/hog_svm_test/weights.json`. 71 | 72 | # File Description 73 | ### data 74 | Training data. This files are used `python/vdtools.py` 75 | ### python 76 | python and scikit-learn library are used to train, verify accuracy and export trained weight to cpp header file. 77 | detail description is in `python/README.md` 78 | ### hls 79 | HOG + SVM Vivado HLS project. 80 | The detail description of algorithm used is in `hls/README.md` 81 | ### vivado 82 | You can create project by running `create_project.tcl` from Vivado 2018.2 83 | ![circuit](vivado/circuit.png) 84 | ### cpp 85 | Some SW-level applications to verify porting from python to C++. 86 | detail description is in `cpp/README.md` 87 | ### overlay 88 | FPGA bistream is configured and IPs are registered as `generic-uio` device by Linux Device Tree Overlay function. 89 | This directory contains bitstream, device tree file, and initialize shell script. 90 | ### app 91 | These application only works on Ultra96, containing interface layer to call FPGA IP. 92 | ### util 93 | Utility program. 94 | 95 | # Reference 96 | [Hua Luo, Jian & Hong Lin, Chang. (2018). Pure FPGA Implementation of an HOG Based Real-Time Pedestrian Detection System. Sensors. 18. 1174. 10.3390/s18041174.](https://www.ncbi.nlm.nih.gov/pubmed/29649146) 97 | 98 | ## Known issue 99 | This SVM Classifier tends to detect false positive. This may be because of the limit of Linear SVM model. 100 | -------------------------------------------------------------------------------- /app/hog_svm_test/compile.sh: -------------------------------------------------------------------------------- 1 | g++ -std=c++11 -O3 -fpermissive -Wno-pointer-arith json11.cpp dma_simple.c main.cpp -I/usr/local/include/opencv2 -I/usr/local/include/opencv -L/usr/local/lib -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_imgproc -L/usr/local/lib 2 | -------------------------------------------------------------------------------- /app/hog_svm_test/dma_simple.c: -------------------------------------------------------------------------------- 1 | #include "dma_simple.h" 2 | 3 | int udmabuf_open(struct udmabuf* udmabuf, const char* name){ 4 | char file_name[1024]; 5 | int fd; 6 | unsigned char attr[1024]; 7 | 8 | strcpy(udmabuf->name, name); 9 | udmabuf->file = -1; 10 | sprintf(file_name, "/sys/class/udmabuf/%s/phys_addr", name); 11 | if ((fd = open(file_name, O_RDONLY)) == -1){ 12 | printf("Can not open %s\n", file_name); 13 | return(-1); 14 | } 15 | read(fd, (void*)attr, 1024); 16 | sscanf(attr, "%x", &udmabuf->phys_addr); 17 | close(fd); 18 | 19 | sprintf(file_name, "/sys/class/udmabuf/%s/size", name); 20 | if((fd = open(file_name, O_RDONLY)) == -1){ 21 | printf("Can not open %s\n", file_name); 22 | return(-1); 23 | } 24 | read(fd, (void*)attr, 1024); 25 | sscanf(attr, "%d", &udmabuf->buf_size); 26 | close(fd); 27 | sprintf(file_name, "/dev/%s", name); 28 | if((udmabuf->file = open(file_name, O_RDWR|O_SYNC)) == -1){ 29 | printf("Can not open %s\n", file_name); 30 | return(-1); 31 | } 32 | udmabuf->buf = mmap(NULL, udmabuf->buf_size, PROT_READ|PROT_WRITE, MAP_SHARED, udmabuf->file, 0); 33 | udmabuf->debug_vma = 0; 34 | udmabuf->sync_mode = 1; 35 | 36 | return 0; 37 | } 38 | 39 | int udmabuf_close(struct udmabuf* udmabuf){ 40 | if(udmabuf->file < 0) return -1; 41 | 42 | close(udmabuf->file); 43 | udmabuf->file = -1; 44 | return 0; 45 | } 46 | -------------------------------------------------------------------------------- /app/hog_svm_test/dma_simple.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | #define DMA_INTAKE_DMACR (0x0000) 11 | #define DMA_INTAKE_DMASR (0x0004) 12 | #define DMA_INTAKE_SA (0x0018) 13 | #define DMA_INTAKE_LENGTH (0x0028) 14 | 15 | #define DMA_OUTLET_DMACR (0x0030) 16 | #define DMA_OUTLET_DMASR (0x0034) 17 | #define DMA_OUTLET_DA (0x0048) 18 | #define DMA_OUTLET_LENGTH (0x0058) 19 | 20 | #define DMA_CR_RS (1u<<0) 21 | #define DMA_CR_RESET (1u<<2) 22 | 23 | #define DMA_SR_HALTED (1u<<0) 24 | #define DMA_SR_IDLE (1u<<1) 25 | #define DMA_SR_IOC_Irq (1u<<12) 26 | #define DMA_SR_ERR_Irq (1u<<14) 27 | 28 | static inline uint32_t regs_read32(void* addr){ 29 | volatile uint32_t* regs_addr = (uint32_t*)(addr); 30 | return *regs_addr; 31 | } 32 | 33 | static inline void regs_write32(void* addr, uint32_t data){ 34 | volatile uint32_t* regs_addr = (uint32_t*)(addr); 35 | *regs_addr = data; 36 | } 37 | 38 | static inline void dma_reset(void* regs){ 39 | regs_write32(regs + DMA_INTAKE_DMACR, DMA_CR_RESET); 40 | while(regs_read32(regs + DMA_INTAKE_DMACR) & DMA_CR_RESET); 41 | regs_write32(regs + DMA_OUTLET_DMASR, DMA_CR_RESET); 42 | while(regs_read32(regs + DMA_OUTLET_DMACR) & DMA_CR_RESET); 43 | } 44 | 45 | static inline void dma_setup(void* regs, unsigned long src_addr, unsigned long dst_addr){ 46 | regs_write32(regs + DMA_OUTLET_DMACR, DMA_CR_RS); 47 | regs_write32(regs + DMA_OUTLET_DA, dst_addr); 48 | regs_write32(regs + DMA_INTAKE_DMACR, DMA_CR_RS); 49 | regs_write32(regs + DMA_INTAKE_SA, src_addr); 50 | } 51 | 52 | static inline void dma_intake_start(void* regs, unsigned int xfer_size){ 53 | regs_write32(regs + DMA_INTAKE_LENGTH, xfer_size); 54 | } 55 | 56 | static inline void dma_outlet_start(void* regs, unsigned int xfer_size){ 57 | regs_write32(regs + DMA_OUTLET_LENGTH, xfer_size); 58 | } 59 | 60 | static inline void dma_wait_irq(void* regs){ 61 | while(~regs_read32(regs + DMA_INTAKE_DMASR) & DMA_SR_IOC_Irq); 62 | while(~regs_read32(regs + DMA_OUTLET_DMASR) & DMA_SR_IOC_Irq); 63 | } 64 | 65 | static inline void dma_clear_status(void* regs){ 66 | regs_write32(regs + DMA_INTAKE_DMASR, DMA_SR_IOC_Irq | DMA_SR_ERR_Irq); 67 | regs_write32(regs + DMA_OUTLET_DMASR, DMA_SR_IOC_Irq | DMA_SR_ERR_Irq); 68 | } 69 | 70 | struct udmabuf{ 71 | char name[128]; 72 | int file; 73 | unsigned char* buf; 74 | unsigned int buf_size; 75 | unsigned long phys_addr; 76 | unsigned long debug_vma; 77 | unsigned long sync_mode; 78 | }; 79 | 80 | int udmabuf_open(struct udmabuf* udmabuf, const char* name); 81 | 82 | int udmabuf_close(struct udmabuf* udmabuf); 83 | -------------------------------------------------------------------------------- /app/hog_svm_test/frame.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/app/hog_svm_test/frame.png -------------------------------------------------------------------------------- /app/hog_svm_test/json11.hpp: -------------------------------------------------------------------------------- 1 | /* json11 2 | * 3 | * json11 is a tiny JSON library for C++11, providing JSON parsing and serialization. 4 | * 5 | * The core object provided by the library is json11::Json. A Json object represents any JSON 6 | * value: null, bool, number (int or double), string (std::string), array (std::vector), or 7 | * object (std::map). 8 | * 9 | * Json objects act like values: they can be assigned, copied, moved, compared for equality or 10 | * order, etc. There are also helper methods Json::dump, to serialize a Json to a string, and 11 | * Json::parse (static) to parse a std::string as a Json object. 12 | * 13 | * Internally, the various types of Json object are represented by the JsonValue class 14 | * hierarchy. 15 | * 16 | * A note on numbers - JSON specifies the syntax of number formatting but not its semantics, 17 | * so some JSON implementations distinguish between integers and floating-point numbers, while 18 | * some don't. In json11, we choose the latter. Because some JSON implementations (namely 19 | * Javascript itself) treat all numbers as the same type, distinguishing the two leads 20 | * to JSON that will be *silently* changed by a round-trip through those implementations. 21 | * Dangerous! To avoid that risk, json11 stores all numbers as double internally, but also 22 | * provides integer helpers. 23 | * 24 | * Fortunately, double-precision IEEE754 ('double') can precisely store any integer in the 25 | * range +/-2^53, which includes every 'int' on most systems. (Timestamps often use int64 26 | * or long long to avoid the Y2038K problem; a double storing microseconds since some epoch 27 | * will be exact for +/- 275 years.) 28 | */ 29 | 30 | /* Copyright (c) 2013 Dropbox, Inc. 31 | * 32 | * Permission is hereby granted, free of charge, to any person obtaining a copy 33 | * of this software and associated documentation files (the "Software"), to deal 34 | * in the Software without restriction, including without limitation the rights 35 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 36 | * copies of the Software, and to permit persons to whom the Software is 37 | * furnished to do so, subject to the following conditions: 38 | * 39 | * The above copyright notice and this permission notice shall be included in 40 | * all copies or substantial portions of the Software. 41 | * 42 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 43 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 44 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 45 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 46 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 47 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 48 | * THE SOFTWARE. 49 | */ 50 | 51 | #pragma once 52 | 53 | #include 54 | #include 55 | #include 56 | #include 57 | #include 58 | 59 | #ifdef _MSC_VER 60 | #if _MSC_VER <= 1800 // VS 2013 61 | #ifndef noexcept 62 | #define noexcept throw() 63 | #endif 64 | 65 | #ifndef snprintf 66 | #define snprintf _snprintf_s 67 | #endif 68 | #endif 69 | #endif 70 | 71 | namespace json11 { 72 | 73 | enum JsonParse { 74 | STANDARD, COMMENTS 75 | }; 76 | 77 | class JsonValue; 78 | 79 | class Json final { 80 | public: 81 | // Types 82 | enum Type { 83 | NUL, NUMBER, BOOL, STRING, ARRAY, OBJECT 84 | }; 85 | 86 | // Array and object typedefs 87 | typedef std::vector array; 88 | typedef std::map object; 89 | 90 | // Constructors for the various types of JSON value. 91 | Json() noexcept; // NUL 92 | Json(std::nullptr_t) noexcept; // NUL 93 | Json(double value); // NUMBER 94 | Json(int value); // NUMBER 95 | Json(bool value); // BOOL 96 | Json(const std::string &value); // STRING 97 | Json(std::string &&value); // STRING 98 | Json(const char * value); // STRING 99 | Json(const array &values); // ARRAY 100 | Json(array &&values); // ARRAY 101 | Json(const object &values); // OBJECT 102 | Json(object &&values); // OBJECT 103 | 104 | // Implicit constructor: anything with a to_json() function. 105 | template 106 | Json(const T & t) : Json(t.to_json()) {} 107 | 108 | // Implicit constructor: map-like objects (std::map, std::unordered_map, etc) 109 | template ().begin()->first)>::value 111 | && std::is_constructible().begin()->second)>::value, 112 | int>::type = 0> 113 | Json(const M & m) : Json(object(m.begin(), m.end())) {} 114 | 115 | // Implicit constructor: vector-like objects (std::list, std::vector, std::set, etc) 116 | template ().begin())>::value, 118 | int>::type = 0> 119 | Json(const V & v) : Json(array(v.begin(), v.end())) {} 120 | 121 | // This prevents Json(some_pointer) from accidentally producing a bool. Use 122 | // Json(bool(some_pointer)) if that behavior is desired. 123 | Json(void *) = delete; 124 | 125 | // Accessors 126 | Type type() const; 127 | 128 | bool is_null() const { return type() == NUL; } 129 | bool is_number() const { return type() == NUMBER; } 130 | bool is_bool() const { return type() == BOOL; } 131 | bool is_string() const { return type() == STRING; } 132 | bool is_array() const { return type() == ARRAY; } 133 | bool is_object() const { return type() == OBJECT; } 134 | 135 | // Return the enclosed value if this is a number, 0 otherwise. Note that json11 does not 136 | // distinguish between integer and non-integer numbers - number_value() and int_value() 137 | // can both be applied to a NUMBER-typed object. 138 | double number_value() const; 139 | int int_value() const; 140 | 141 | // Return the enclosed value if this is a boolean, false otherwise. 142 | bool bool_value() const; 143 | // Return the enclosed string if this is a string, "" otherwise. 144 | const std::string &string_value() const; 145 | // Return the enclosed std::vector if this is an array, or an empty vector otherwise. 146 | const array &array_items() const; 147 | // Return the enclosed std::map if this is an object, or an empty map otherwise. 148 | const object &object_items() const; 149 | 150 | // Return a reference to arr[i] if this is an array, Json() otherwise. 151 | const Json & operator[](size_t i) const; 152 | // Return a reference to obj[key] if this is an object, Json() otherwise. 153 | const Json & operator[](const std::string &key) const; 154 | 155 | // Serialize. 156 | void dump(std::string &out) const; 157 | std::string dump() const { 158 | std::string out; 159 | dump(out); 160 | return out; 161 | } 162 | 163 | // Parse. If parse fails, return Json() and assign an error message to err. 164 | static Json parse(const std::string & in, 165 | std::string & err, 166 | JsonParse strategy = JsonParse::STANDARD); 167 | static Json parse(const char * in, 168 | std::string & err, 169 | JsonParse strategy = JsonParse::STANDARD) { 170 | if (in) { 171 | return parse(std::string(in), err, strategy); 172 | } else { 173 | err = "null input"; 174 | return nullptr; 175 | } 176 | } 177 | // Parse multiple objects, concatenated or separated by whitespace 178 | static std::vector parse_multi( 179 | const std::string & in, 180 | std::string::size_type & parser_stop_pos, 181 | std::string & err, 182 | JsonParse strategy = JsonParse::STANDARD); 183 | 184 | static inline std::vector parse_multi( 185 | const std::string & in, 186 | std::string & err, 187 | JsonParse strategy = JsonParse::STANDARD) { 188 | std::string::size_type parser_stop_pos; 189 | return parse_multi(in, parser_stop_pos, err, strategy); 190 | } 191 | 192 | bool operator== (const Json &rhs) const; 193 | bool operator< (const Json &rhs) const; 194 | bool operator!= (const Json &rhs) const { return !(*this == rhs); } 195 | bool operator<= (const Json &rhs) const { return !(rhs < *this); } 196 | bool operator> (const Json &rhs) const { return (rhs < *this); } 197 | bool operator>= (const Json &rhs) const { return !(*this < rhs); } 198 | 199 | /* has_shape(types, err) 200 | * 201 | * Return true if this is a JSON object and, for each item in types, has a field of 202 | * the given type. If not, return false and set err to a descriptive message. 203 | */ 204 | typedef std::initializer_list> shape; 205 | bool has_shape(const shape & types, std::string & err) const; 206 | 207 | private: 208 | std::shared_ptr m_ptr; 209 | }; 210 | 211 | // Internal class hierarchy - JsonValue objects are not exposed to users of this API. 212 | class JsonValue { 213 | protected: 214 | friend class Json; 215 | friend class JsonInt; 216 | friend class JsonDouble; 217 | virtual Json::Type type() const = 0; 218 | virtual bool equals(const JsonValue * other) const = 0; 219 | virtual bool less(const JsonValue * other) const = 0; 220 | virtual void dump(std::string &out) const = 0; 221 | virtual double number_value() const; 222 | virtual int int_value() const; 223 | virtual bool bool_value() const; 224 | virtual const std::string &string_value() const; 225 | virtual const Json::array &array_items() const; 226 | virtual const Json &operator[](size_t i) const; 227 | virtual const Json::object &object_items() const; 228 | virtual const Json &operator[](const std::string &key) const; 229 | virtual ~JsonValue() {} 230 | }; 231 | 232 | } // namespace json11 233 | -------------------------------------------------------------------------------- /app/hog_svm_test/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | #include 11 | #include 12 | #include 13 | #include 14 | 15 | #include "json11.hpp" 16 | #include "dma_simple.h" 17 | using namespace std; 18 | using namespace cv; 19 | 20 | void *dma_regs; 21 | void *hls_regs; 22 | struct udmabuf intake_buf; 23 | struct udmabuf outlet_buf; 24 | float BIAS; 25 | float THRESH; 26 | 27 | unsigned int *assignToPhysicalUInt(unsigned long address,unsigned int size){ 28 | int devmem = open("/dev/mem", O_RDWR | O_SYNC); 29 | off_t PageOffset = (off_t) address % getpagesize(); 30 | off_t PageAddress = (off_t) (address - PageOffset); 31 | return (unsigned int *) mmap(0, size*sizeof(unsigned int), PROT_READ|PROT_WRITE, MAP_SHARED, devmem, PageAddress); 32 | } 33 | 34 | int hw_setup(){ 35 | int uio1_fd = open("/dev/uio1", O_RDWR); 36 | hls_regs = mmap(NULL, 0x10000, PROT_READ|PROT_WRITE, MAP_SHARED, uio1_fd, 0); 37 | 38 | int uio2_fd = open("/dev/uio2", O_RDWR); 39 | dma_regs = mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, uio2_fd, 0); 40 | 41 | if(udmabuf_open(&intake_buf, "udmabuf0") == -1){ 42 | printf("udmabuf_open failed\n"); 43 | return -1; 44 | } 45 | if(udmabuf_open(&outlet_buf, "udmabuf1") == -1){ 46 | printf("udmabuf_open failed\n"); 47 | return -1; 48 | } 49 | dma_reset(dma_regs); 50 | 51 | regs_write32(hls_regs, 0x01); //start 52 | regs_write32(hls_regs, 0x80); //enable autorestart 53 | 54 | return 1; 55 | } 56 | 57 | void writebram(unsigned int* target, string array_name, json11::Json json){ 58 | int cnt = 0; 59 | for(auto &k: json[array_name].array_items()){ 60 | target[cnt++] = k.number_value(); 61 | } 62 | } 63 | int param_setup(){ 64 | unsigned int* bgrhsv_w1 = assignToPhysicalUInt(0xA0000000, 24*4); 65 | unsigned int* bgrhsv_w2 = assignToPhysicalUInt(0xA0004000, 24*4); 66 | unsigned int* bgrhsv_w3 = assignToPhysicalUInt(0xA0008000, 24*4); 67 | unsigned int* bgrhsv_w4 = assignToPhysicalUInt(0xA000C000, 24*4); 68 | unsigned int* hog_w1 = assignToPhysicalUInt(0xA0010000, 63*4); 69 | unsigned int* hog_w2 = assignToPhysicalUInt(0xA0014000, 63*4); 70 | unsigned int* hog_w3 = assignToPhysicalUInt(0xA0018000, 63*4); 71 | 72 | ifstream ifs("weights.json"); 73 | if(ifs.fail()){ 74 | cerr << "json file load failed" << endl; 75 | return -1; 76 | } 77 | std::istreambuf_iterator it(ifs); 78 | std::istreambuf_iterator last; 79 | std::string str(it, last); 80 | string err; 81 | auto json = json11::Json::parse(str, err); 82 | if(err != ""){ 83 | cerr << "json file parse error: " << err << endl; 84 | return -1; 85 | } 86 | 87 | BIAS = json["bias"].number_value(); 88 | THRESH = json["thresh"].number_value(); 89 | writebram(hog_w1, "hog_w1", json); 90 | writebram(hog_w2, "hog_w2", json); 91 | writebram(hog_w3, "hog_w3", json); 92 | writebram(bgrhsv_w1, "bgrhsv_w1", json); 93 | writebram(bgrhsv_w2, "bgrhsv_w2", json); 94 | writebram(bgrhsv_w3, "bgrhsv_w3", json); 95 | writebram(bgrhsv_w4, "bgrhsv_w4", json); 96 | 97 | return 1; 98 | } 99 | 100 | cv::Mat predict(cv::Mat bgr_img){ 101 | std::chrono::system_clock::time_point t1, t2; 102 | t1 = std::chrono::system_clock::now(); 103 | cv::Mat result_frame = bgr_img.clone(); 104 | cv::Mat bgra_img; 105 | cv::cvtColor(bgr_img, bgra_img, CV_BGR2BGRA); 106 | cv::Mat bgra_quarter_img(bgra_img, cv::Rect(0, 0, 320, 240)); 107 | //copy image pixel value to dma input buffer 108 | for(int y = 0; y < 240; y++) { 109 | memcpy(intake_buf.buf + 320 * y * 4, bgra_quarter_img.data + bgra_quarter_img.step * y, bgra_quarter_img.cols * 4 * sizeof(unsigned char)); 110 | } 111 | 112 | dma_setup(dma_regs, intake_buf.phys_addr, outlet_buf.phys_addr); 113 | dma_outlet_start(dma_regs, 27*33*4); 114 | dma_intake_start(dma_regs, 320*240*4); 115 | dma_wait_irq(dma_regs); 116 | dma_clear_status(dma_regs); 117 | for(int y = 0; y < 27; y++){ 118 | for(int x = 0; x < 33; x++){ 119 | int index = y * 33 + x; 120 | float rst = ((float*) (outlet_buf.buf))[index] + BIAS; 121 | if(rst >= THRESH){ 122 | //sigmoid function 123 | float proba = 1.0/(1.0+exp(-rst)); 124 | cout << fixed << setprecision(10) << rst << " " << proba << endl; 125 | int yy = y * 8; 126 | int xx = x * 8; 127 | rectangle(result_frame, Point(xx, yy), Point(xx + 64, yy + 32), Scalar(0,0,200), 2); //x,y //Scaler = B,G,R 128 | cv::putText(result_frame, to_string(proba), cv::Point(xx+5,yy+5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(255,0,0), 1, CV_AA); 129 | } 130 | } 131 | } 132 | 133 | t2 = std::chrono::system_clock::now(); 134 | double elapsed = (double)std::chrono::duration_cast(t2-t1).count(); 135 | cout << "elapsed:" << elapsed << "[milisec]" << endl; 136 | cout << "fps:" << 1000.0/elapsed << "[fps]" << endl; 137 | return result_frame; 138 | } 139 | 140 | int main(){ 141 | if(param_setup() < 0){ 142 | cout << "param setup failed" << endl; 143 | return -1; 144 | } 145 | if(hw_setup() < 0){ 146 | cout << "hw_setup() failed" << endl; 147 | return -1; 148 | } 149 | cv::Mat frame = cv::imread("frame.png"); 150 | cv::Mat result_frame = predict(frame); 151 | cv::imwrite("result.png", result_frame); 152 | return 0; 153 | } 154 | -------------------------------------------------------------------------------- /app/hog_svm_test/weights.json: -------------------------------------------------------------------------------- 1 | { 2 | "thresh" : 1.3862943611, 3 | "bias" : -2.1975765, 4 | "bgrhsv_w1" : [286925, 414125, 4294792783, 4294530074, 180523, 302453, 121256, 4294838112, 4294852325, 175511, 57948, 220327, 188681, 246448, 4294175583, 62839, 213119, 192516, 158200, 4294599129, 4294870722, 195913, 21862, 108219, 135707, 4294560836, 4294531582, 471793, 100014, 4294687414, 4294795984, 66431, 136769, 4294626978, 74939, 4294520268, 4294803702, 4294833774, 1358169, 228843, 4294533169, 4294711845, 4294635639, 4294561320, 4294548122, 4294877153, 4294580325, 4294701642, 315921, 4294787473, 4294823948, 931766, 69555, 4294609375, 4294504419, 4294420190, 280053, 4294545252, 117506, 4294416930, 224873, 4294892981, 33826, 179021, 4294919630, 4294754903, 124001, 187118, 932180, 99537, 628398, 35208, 4293709472, 4294617393, 4294545899, 4294858360, 4293714819, 4294509769, 1283148, 4294901810, 681494, 184406, 475595, 105753, 4294544366, 4294691033, 4294686162, 4294130703, 4294366668, 4294703790, 4294894858, 4294914286, 651540, 188299, 385667, 73682] 5 | ,"bgrhsv_w2" : [4294678382, 4294426082, 33900, 4294786573, 4294798260, 4294465271, 4294517259, 181256, 300716, 4294576336, 4294779851, 4294392855, 4294501450, 4294559575, 1747455, 1895189, 4294343273, 4294494832, 4293427624, 4293396760, 557902, 231784, 4294216869, 4294397969, 370238, 4294689001, 4294732968, 4293838387, 285698, 4294924877, 4292752297, 4294311841, 1373130, 527198, 447725, 88187, 316414, 133618, 4294912849, 336151, 4294763669, 4294699908, 4294049825, 4293331584, 823228, 226144, 459479, 4294963093, 368041, 4294110388, 520032, 716422, 4293988786, 4293440987, 4294441252, 4294108101, 354961, 98951, 133964, 4294626653, 330929, 45665, 4294777303, 4294750516, 4294768541, 4294572619, 4294612355, 4294583897, 1410668, 1060654, 943010, 574817, 4294563529, 4294621839, 749760, 4294735436, 4294558413, 4294577201, 1507408, 608711, 1990093, 922274, 1535905, 642773, 4294579386, 4294841981, 128733, 160141, 4294488597, 4294652830, 24372, 77325, 303718, 294846, 71547, 278933] 6 | ,"bgrhsv_w3" : [136128, 4294683817, 360450, 824960, 4294829132, 4294484903, 78219, 558577, 4294953960, 4294683371, 26876, 4294689487, 4294585981, 4294597802, 657838, 175524, 4294797448, 4294450399, 4293758241, 4293295348, 4294749523, 560172, 4294399572, 4294458180, 4294539494, 4294467598, 6848, 533868, 4294052521, 4294123324, 4293517802, 4292904425, 4294275118, 23086, 4294311795, 4294322181, 186208, 323675, 439545, 1009885, 4294276375, 4294211295, 4294829568, 4294063271, 434514, 37201, 542138, 8651, 4293960369, 341051, 656091, 7056, 4292601118, 4293503497, 197622, 4294515689, 115372, 154966, 106308, 213954, 61908, 504986, 4294661555, 178876, 4294199586, 4294713840, 4294502092, 4294531736, 236442, 949123, 175692, 710489, 145860, 4294823904, 4294953657, 84629, 4294579148, 4294668689, 236680, 1522779, 970197, 1739651, 890460, 1415223, 4294920959, 4294751862, 319014, 4294912025, 4294638246, 4294419507, 4294530293, 30753, 4294946745, 374105, 4294819087, 180199] 7 | ,"bgrhsv_w4" : [403704, 4294801894, 868547, 316093, 115595, 4294499655, 4294661153, 618745, 4294959983, 4294505215, 264048, 4294678169, 488273, 469491, 774405, 206550, 387303, 174489, 4294145055, 4294080680, 179321, 4294677598, 392911, 22020, 327094, 755533, 440055, 73898, 17878, 41040, 4294492078, 4293955327, 4294578226, 219646, 4294846005, 326587, 4294945083, 1145589, 4294730853, 4294798050, 4294823835, 602937, 4294863616, 4294438105, 4294802318, 736541, 4294860055, 1114370, 4294945265, 236557, 400947, 4294794605, 4294662560, 4294360523, 4294804086, 995708, 79256, 689381, 228611, 1236042, 4294564002, 4294623144, 700676, 364280, 4294086319, 4293948378, 4294932167, 459088, 262850, 374235, 223124, 523079, 143209, 46147, 474402, 8173, 4294640515, 4294622236, 4294896322, 289868, 292561, 548621, 497756, 574074, 4294835465, 4294695249, 802842, 620113, 4294612231, 4294192971, 4294427941, 4294618022, 80447, 4294828207, 157610, 4294792798] 8 | ,"hog_w1" : [535309, 4293005267, 1402609, 4293041238, 4294306044, 4294445583, 3351008, 4292520672, 1012001, 871316, 70594, 202348, 4294153533, 4292716739, 3076009, 749431, 1517115, 4293920630, 435200, 4294356531, 4699442, 3508577, 4277500, 4292252235, 4292245271, 295506, 4293593549, 4289819238, 4293805393, 4293440057, 4294557208, 4291144504, 1465615, 4998613, 2884052, 531450, 3848662, 4294403727, 4292579865, 4292561336, 487626, 677523, 4291863723, 391728, 1326555, 2502535, 2168643, 34821, 1763602, 4290987194, 591466, 2011220, 1501187, 4294170042, 4292777586, 57088, 125006, 4292118314, 4293746914, 1245292, 4566514, 4289637780, 519179, 4293515881, 405758, 4292490713, 4293693879, 4293747840, 1224580, 4294313136, 2525635, 4445466, 953641, 4292925704, 4294211398, 4292235420, 288479, 4294906231, 3559886, 4289165495, 4293793757, 1078188, 4393351, 1235044, 4293138825, 4294184806, 4294422484, 3681230, 344712, 4293824920, 4292272383, 4293652092, 4885701, 49470, 4292582390, 1093057, 527765, 2736123, 1074189, 3877691, 414689, 4294310962, 4293406974, 649369, 1457623, 4294770504, 4292611076, 6025879, 4294748524, 912863, 1918596, 4292258316, 4293798268, 4294934847, 4294341057, 4294920153, 4291889023, 4293867324, 128412, 4127868, 4293567494, 832332, 3056900, 4294804250, 4294631683, 1197861, 4293396700, 4292846950, 1238821, 3440306, 779631, 4294223017, 2017061, 4293733195, 212887, 1556647, 4294386898, 4294886721, 4294413077, 404240, 1772765, 856047, 820263, 349217, 4294539302, 1312191, 4294128386, 4294607018, 4294738996, 2901963, 133066, 4292206186, 2091738, 4293998541, 4291544306, 1750836, 4292019185, 1030596, 1187554, 3681579, 4294449500, 4293210564, 1240103, 4293388127, 829861, 562889, 1779854, 4289428213, 4398200, 4294167808, 4658810, 4293192198, 1162426, 4293593232, 117845, 4293729413, 4294684496, 4294656958, 737383, 2859885, 1043086, 1465339, 4290757268, 4294039726, 4293481430, 2560099, 4294765710, 990129, 1717358, 3916676, 912265, 4290873841, 4293620286, 4294276654, 4294490870, 4293493745, 653789, 439495, 4292196903, 4294952344, 4235220, 416617, 4290897675, 2476391, 1445530, 4294509798, 4294076629, 6888573, 4293422447, 1010112, 4293387382, 4292546549, 718258, 290822, 1269663, 519313, 5221375, 611997, 4287869637, 4290619128, 4290760957, 455221, 4290452115, 5011140, 4291910045, 5741630, 330635, 9983, 4289466062, 23062, 4289290867, 4292644336, 1672612, 482817, 4294035212, 4293088324, 2703730, 1001528, 4290377013, 4290968144, 3219596, 467878, 4294737188, 348047, 4293967923, 4294174124, 4291398680, 4293887631, 77063, 4294385984, 6214884, 961048] 9 | ,"hog_w2" : [93835, 4293342059, 4294531268, 4292038590, 10805383, 4293759014, 645676, 1309227, 13581, 4293266446, 3031018, 601713, 4293551746, 4293895330, 4292238468, 805111, 4294286524, 121735, 4292807381, 4293473740, 2727095, 971130, 2057365, 8529918, 4861036, 612012, 4291073383, 754264, 1052528, 4884032, 4291423882, 4293291081, 2820368, 2088045, 4293500019, 2154234, 1424912, 1392018, 1283110, 4293986703, 4294486473, 3768367, 4294604033, 4294922791, 4292978179, 4294496424, 4294936587, 2315338, 1299850, 4294743778, 4293413163, 4292136525, 4294572948, 935642, 4294441732, 4293636743, 1289055, 3852683, 4294925812, 6657322, 2342630, 4294684726, 2498375, 4293446354, 4292872631, 4294438083, 4293015100, 4294147721, 3042360, 1705251, 4294376796, 61344, 4293639080, 4294008331, 427264, 877418, 1432024, 93662, 252826, 2335749, 4291899531, 4291769264, 223558, 2542304, 4293848510, 2208596, 797028, 4291987681, 4294272582, 332661, 25330, 4293950859, 2648866, 2371695, 2179449, 935402, 4292457429, 1305322, 4291262854, 4401446, 1070351, 577245, 4293008202, 4293765849, 135507, 1776482, 4294779938, 4293737083, 622171, 4293856777, 279197, 254734, 1113806, 630281, 839723, 4293936725, 4291449056, 1393231, 501479, 4294055579, 4293201395, 1208671, 4292901311, 1312390, 454305, 4294502573, 4294118524, 1775475, 721032, 1469306, 278848, 6143929, 4294610535, 4289529775, 648405, 1422505, 4294374616, 4293813980, 4294726813, 33552, 999510, 4291541326, 2028982, 1021456, 4294009244, 1108565, 2932174, 2220695, 4290950697, 2476670, 1070787, 935288, 4290360483, 4290518976, 5384820, 4293216842, 2461671, 4291326399, 4294553775, 4292382069, 1302762, 4292540478, 18026, 4293061374, 3438488, 4292571662, 62041, 2463772, 1194992, 4291863400, 391556, 647275, 4532957, 4289142757, 4294402821, 4294612341, 2674712, 4292074073, 759478, 4293847313, 521803, 4294330342, 1267877, 1334609, 4292818905, 4292803422, 388266, 1006662, 6534351, 4289956787, 4669485, 1538580, 4294492578, 4294509527, 40091, 4291557289, 4294146081, 4293063911, 4294883107, 4294276546, 1870904, 1538610, 4294608821, 585632, 2854841, 4294107121, 4293280517, 4294116389, 1202717, 4293502613, 4293079915, 4291814762, 3682184, 4292243268, 671004, 4294465232, 2184287, 4035836, 3030619, 4294423413, 317758, 2283875, 4289943065, 626827, 4293106887, 6866259, 4289653641, 4114956, 4289365260, 4293124734, 4288570873, 4294136002, 4293290114, 4293338974, 548332, 255816, 1505268, 1122866, 2261413, 1344533, 3293154, 4293774144, 1698442, 984079, 4291165143, 4292864904, 4292036243, 4293881221, 4294022829, 4292988710, 1920647, 4294896043] 10 | ,"hog_w3" : [4288671221, 4293277452, 4293753831, 826774, 4293946600, 4290259719, 4291727443, 4293994659, 393577, 4289938510, 4293175415, 4293667656, 4292637791, 4615737, 287022, 1935141, 4292149312, 4293255182, 4293549308, 4294557896, 4292678228, 4294884589, 3170764, 4349959, 1810957, 4723497, 1169103, 5865362, 76205, 2629854, 1878429, 7867789, 4294808249, 10009090, 4292134700, 2040181, 4293405275, 4290241233, 3097180, 4294799887, 4293188933, 4294164846, 3245244, 4294194678, 4294719090, 2463483, 953484, 4293555969, 17833, 54375, 4631450, 4294940994, 4294814344, 4294471119, 4294371023, 4294327529, 4289671201, 4293951002, 4293033818, 6377944, 2425460, 1360672, 469512, 1797291, 4294745704, 4294745340, 4290928522, 2288995, 2915604, 4294227675, 2860540, 4294517113, 4294519449, 4293600403, 369671, 222744, 4293708509, 4292689120, 5096731, 4294652781, 4292217965, 1072005, 4294033148, 4292335022, 4220137, 4293725667, 4294196200, 4294555469, 4294941425, 4294601618, 4294045130, 270302, 4292049351, 4291411024, 4293693218, 5073016, 4291094484, 4489388, 4291490242, 9262732, 4294028258, 4293749907, 4292677901, 450238, 2139340, 1605037, 4293712086, 3357119, 4291625455, 4294584780, 1017873, 1465073, 4292621616, 4294722166, 3874530, 5116058, 4284425932, 4291786252, 4289287370, 1984084, 3566816, 5866708, 4294350086, 318196, 4294355941, 4294501973, 4294816112, 962694, 4290257315, 4292106529, 4294820463, 2969813, 530991, 4290925477, 4294910851, 390810, 6515811, 4294534493, 4291785315, 1343217, 1056256, 1213310, 1307446, 157171, 4291691148, 4289699149, 465844, 11044, 4292996706, 4291026338, 3658635, 647187, 2367832, 4282365451, 1935707, 4290352817, 5217993, 3381887, 6224912, 4293187021, 1001931, 4293584094, 1089522, 4293313459, 2244379, 4290256419, 1136020, 4294935603, 1907829, 4292389216, 4293580919, 364757, 2131366, 3879182, 4294274945, 4291700164, 2118247, 251943, 1449082, 393411, 4288152058, 4289721973, 1204548, 4294810895, 4286326905, 4291082636, 1433209, 1460694, 1070349, 4293226656, 6656792, 176541, 4292417449, 1354748, 1210588, 732139, 4294393538, 4294863870, 4294928486, 4294478485, 4290745000, 4294098953, 4293215231, 1599444, 4294264925, 4293951443, 4294455804, 743307, 263527, 4294596838, 1469140, 1491638, 1393939, 1381611, 3780483, 694923, 4289030942, 4289779056, 1531386, 2660291, 3081849, 4291980098, 5161734, 2230192, 1513958, 4425298, 1694744, 2474751, 4292397831, 4294225998, 4294718011, 4294127357, 4291874224, 4293631955, 4294569586, 82024, 4286408384, 4290555389, 4293135058, 318684, 4282881483, 4291793422, 4293789646, 351694, 4286676998, 4293848505, 908723, 2560265, 661082, 1012789, 708343, 2094701] 11 | } 12 | -------------------------------------------------------------------------------- /app/realtime_webcam/compile.sh: -------------------------------------------------------------------------------- 1 | g++ -std=c++11 -O3 -fpermissive -Wno-pointer-arith json11.cpp dma_simple.c main.cpp -I/usr/local/include/opencv2 -I/usr/local/include/opencv -L/usr/local/lib -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_videoio -lopencv_imgproc -L/usr/local/lib 2 | -------------------------------------------------------------------------------- /app/realtime_webcam/dma_simple.c: -------------------------------------------------------------------------------- 1 | #include "dma_simple.h" 2 | 3 | int udmabuf_open(struct udmabuf* udmabuf, const char* name){ 4 | char file_name[1024]; 5 | int fd; 6 | unsigned char attr[1024]; 7 | 8 | strcpy(udmabuf->name, name); 9 | udmabuf->file = -1; 10 | sprintf(file_name, "/sys/class/udmabuf/%s/phys_addr", name); 11 | if ((fd = open(file_name, O_RDONLY)) == -1){ 12 | printf("Can not open %s\n", file_name); 13 | return(-1); 14 | } 15 | read(fd, (void*)attr, 1024); 16 | sscanf(attr, "%x", &udmabuf->phys_addr); 17 | close(fd); 18 | 19 | sprintf(file_name, "/sys/class/udmabuf/%s/size", name); 20 | if((fd = open(file_name, O_RDONLY)) == -1){ 21 | printf("Can not open %s\n", file_name); 22 | return(-1); 23 | } 24 | read(fd, (void*)attr, 1024); 25 | sscanf(attr, "%d", &udmabuf->buf_size); 26 | close(fd); 27 | sprintf(file_name, "/dev/%s", name); 28 | if((udmabuf->file = open(file_name, O_RDWR|O_SYNC)) == -1){ 29 | printf("Can not open %s\n", file_name); 30 | return(-1); 31 | } 32 | udmabuf->buf = mmap(NULL, udmabuf->buf_size, PROT_READ|PROT_WRITE, MAP_SHARED, udmabuf->file, 0); 33 | udmabuf->debug_vma = 0; 34 | udmabuf->sync_mode = 1; 35 | 36 | return 0; 37 | } 38 | 39 | int udmabuf_close(struct udmabuf* udmabuf){ 40 | if(udmabuf->file < 0) return -1; 41 | 42 | close(udmabuf->file); 43 | udmabuf->file = -1; 44 | return 0; 45 | } 46 | -------------------------------------------------------------------------------- /app/realtime_webcam/dma_simple.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | #define DMA_INTAKE_DMACR (0x0000) 11 | #define DMA_INTAKE_DMASR (0x0004) 12 | #define DMA_INTAKE_SA (0x0018) 13 | #define DMA_INTAKE_LENGTH (0x0028) 14 | 15 | #define DMA_OUTLET_DMACR (0x0030) 16 | #define DMA_OUTLET_DMASR (0x0034) 17 | #define DMA_OUTLET_DA (0x0048) 18 | #define DMA_OUTLET_LENGTH (0x0058) 19 | 20 | #define DMA_CR_RS (1u<<0) 21 | #define DMA_CR_RESET (1u<<2) 22 | 23 | #define DMA_SR_HALTED (1u<<0) 24 | #define DMA_SR_IDLE (1u<<1) 25 | #define DMA_SR_IOC_Irq (1u<<12) 26 | #define DMA_SR_ERR_Irq (1u<<14) 27 | 28 | static inline uint32_t regs_read32(void* addr){ 29 | volatile uint32_t* regs_addr = (uint32_t*)(addr); 30 | return *regs_addr; 31 | } 32 | 33 | static inline void regs_write32(void* addr, uint32_t data){ 34 | volatile uint32_t* regs_addr = (uint32_t*)(addr); 35 | *regs_addr = data; 36 | } 37 | 38 | static inline void dma_reset(void* regs){ 39 | regs_write32(regs + DMA_INTAKE_DMACR, DMA_CR_RESET); 40 | while(regs_read32(regs + DMA_INTAKE_DMACR) & DMA_CR_RESET); 41 | regs_write32(regs + DMA_OUTLET_DMASR, DMA_CR_RESET); 42 | while(regs_read32(regs + DMA_OUTLET_DMACR) & DMA_CR_RESET); 43 | } 44 | 45 | static inline void dma_setup(void* regs, unsigned long src_addr, unsigned long dst_addr){ 46 | regs_write32(regs + DMA_OUTLET_DMACR, DMA_CR_RS); 47 | regs_write32(regs + DMA_OUTLET_DA, dst_addr); 48 | regs_write32(regs + DMA_INTAKE_DMACR, DMA_CR_RS); 49 | regs_write32(regs + DMA_INTAKE_SA, src_addr); 50 | } 51 | 52 | static inline void dma_intake_start(void* regs, unsigned int xfer_size){ 53 | regs_write32(regs + DMA_INTAKE_LENGTH, xfer_size); 54 | } 55 | 56 | static inline void dma_outlet_start(void* regs, unsigned int xfer_size){ 57 | regs_write32(regs + DMA_OUTLET_LENGTH, xfer_size); 58 | } 59 | 60 | static inline void dma_wait_irq(void* regs){ 61 | while(~regs_read32(regs + DMA_INTAKE_DMASR) & DMA_SR_IOC_Irq); 62 | while(~regs_read32(regs + DMA_OUTLET_DMASR) & DMA_SR_IOC_Irq); 63 | } 64 | 65 | static inline void dma_clear_status(void* regs){ 66 | regs_write32(regs + DMA_INTAKE_DMASR, DMA_SR_IOC_Irq | DMA_SR_ERR_Irq); 67 | regs_write32(regs + DMA_OUTLET_DMASR, DMA_SR_IOC_Irq | DMA_SR_ERR_Irq); 68 | } 69 | 70 | struct udmabuf{ 71 | char name[128]; 72 | int file; 73 | unsigned char* buf; 74 | unsigned int buf_size; 75 | unsigned long phys_addr; 76 | unsigned long debug_vma; 77 | unsigned long sync_mode; 78 | }; 79 | 80 | int udmabuf_open(struct udmabuf* udmabuf, const char* name); 81 | 82 | int udmabuf_close(struct udmabuf* udmabuf); 83 | -------------------------------------------------------------------------------- /app/realtime_webcam/json11.hpp: -------------------------------------------------------------------------------- 1 | /* json11 2 | * 3 | * json11 is a tiny JSON library for C++11, providing JSON parsing and serialization. 4 | * 5 | * The core object provided by the library is json11::Json. A Json object represents any JSON 6 | * value: null, bool, number (int or double), string (std::string), array (std::vector), or 7 | * object (std::map). 8 | * 9 | * Json objects act like values: they can be assigned, copied, moved, compared for equality or 10 | * order, etc. There are also helper methods Json::dump, to serialize a Json to a string, and 11 | * Json::parse (static) to parse a std::string as a Json object. 12 | * 13 | * Internally, the various types of Json object are represented by the JsonValue class 14 | * hierarchy. 15 | * 16 | * A note on numbers - JSON specifies the syntax of number formatting but not its semantics, 17 | * so some JSON implementations distinguish between integers and floating-point numbers, while 18 | * some don't. In json11, we choose the latter. Because some JSON implementations (namely 19 | * Javascript itself) treat all numbers as the same type, distinguishing the two leads 20 | * to JSON that will be *silently* changed by a round-trip through those implementations. 21 | * Dangerous! To avoid that risk, json11 stores all numbers as double internally, but also 22 | * provides integer helpers. 23 | * 24 | * Fortunately, double-precision IEEE754 ('double') can precisely store any integer in the 25 | * range +/-2^53, which includes every 'int' on most systems. (Timestamps often use int64 26 | * or long long to avoid the Y2038K problem; a double storing microseconds since some epoch 27 | * will be exact for +/- 275 years.) 28 | */ 29 | 30 | /* Copyright (c) 2013 Dropbox, Inc. 31 | * 32 | * Permission is hereby granted, free of charge, to any person obtaining a copy 33 | * of this software and associated documentation files (the "Software"), to deal 34 | * in the Software without restriction, including without limitation the rights 35 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 36 | * copies of the Software, and to permit persons to whom the Software is 37 | * furnished to do so, subject to the following conditions: 38 | * 39 | * The above copyright notice and this permission notice shall be included in 40 | * all copies or substantial portions of the Software. 41 | * 42 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 43 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 44 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 45 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 46 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 47 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 48 | * THE SOFTWARE. 49 | */ 50 | 51 | #pragma once 52 | 53 | #include 54 | #include 55 | #include 56 | #include 57 | #include 58 | 59 | #ifdef _MSC_VER 60 | #if _MSC_VER <= 1800 // VS 2013 61 | #ifndef noexcept 62 | #define noexcept throw() 63 | #endif 64 | 65 | #ifndef snprintf 66 | #define snprintf _snprintf_s 67 | #endif 68 | #endif 69 | #endif 70 | 71 | namespace json11 { 72 | 73 | enum JsonParse { 74 | STANDARD, COMMENTS 75 | }; 76 | 77 | class JsonValue; 78 | 79 | class Json final { 80 | public: 81 | // Types 82 | enum Type { 83 | NUL, NUMBER, BOOL, STRING, ARRAY, OBJECT 84 | }; 85 | 86 | // Array and object typedefs 87 | typedef std::vector array; 88 | typedef std::map object; 89 | 90 | // Constructors for the various types of JSON value. 91 | Json() noexcept; // NUL 92 | Json(std::nullptr_t) noexcept; // NUL 93 | Json(double value); // NUMBER 94 | Json(int value); // NUMBER 95 | Json(bool value); // BOOL 96 | Json(const std::string &value); // STRING 97 | Json(std::string &&value); // STRING 98 | Json(const char * value); // STRING 99 | Json(const array &values); // ARRAY 100 | Json(array &&values); // ARRAY 101 | Json(const object &values); // OBJECT 102 | Json(object &&values); // OBJECT 103 | 104 | // Implicit constructor: anything with a to_json() function. 105 | template 106 | Json(const T & t) : Json(t.to_json()) {} 107 | 108 | // Implicit constructor: map-like objects (std::map, std::unordered_map, etc) 109 | template ().begin()->first)>::value 111 | && std::is_constructible().begin()->second)>::value, 112 | int>::type = 0> 113 | Json(const M & m) : Json(object(m.begin(), m.end())) {} 114 | 115 | // Implicit constructor: vector-like objects (std::list, std::vector, std::set, etc) 116 | template ().begin())>::value, 118 | int>::type = 0> 119 | Json(const V & v) : Json(array(v.begin(), v.end())) {} 120 | 121 | // This prevents Json(some_pointer) from accidentally producing a bool. Use 122 | // Json(bool(some_pointer)) if that behavior is desired. 123 | Json(void *) = delete; 124 | 125 | // Accessors 126 | Type type() const; 127 | 128 | bool is_null() const { return type() == NUL; } 129 | bool is_number() const { return type() == NUMBER; } 130 | bool is_bool() const { return type() == BOOL; } 131 | bool is_string() const { return type() == STRING; } 132 | bool is_array() const { return type() == ARRAY; } 133 | bool is_object() const { return type() == OBJECT; } 134 | 135 | // Return the enclosed value if this is a number, 0 otherwise. Note that json11 does not 136 | // distinguish between integer and non-integer numbers - number_value() and int_value() 137 | // can both be applied to a NUMBER-typed object. 138 | double number_value() const; 139 | int int_value() const; 140 | 141 | // Return the enclosed value if this is a boolean, false otherwise. 142 | bool bool_value() const; 143 | // Return the enclosed string if this is a string, "" otherwise. 144 | const std::string &string_value() const; 145 | // Return the enclosed std::vector if this is an array, or an empty vector otherwise. 146 | const array &array_items() const; 147 | // Return the enclosed std::map if this is an object, or an empty map otherwise. 148 | const object &object_items() const; 149 | 150 | // Return a reference to arr[i] if this is an array, Json() otherwise. 151 | const Json & operator[](size_t i) const; 152 | // Return a reference to obj[key] if this is an object, Json() otherwise. 153 | const Json & operator[](const std::string &key) const; 154 | 155 | // Serialize. 156 | void dump(std::string &out) const; 157 | std::string dump() const { 158 | std::string out; 159 | dump(out); 160 | return out; 161 | } 162 | 163 | // Parse. If parse fails, return Json() and assign an error message to err. 164 | static Json parse(const std::string & in, 165 | std::string & err, 166 | JsonParse strategy = JsonParse::STANDARD); 167 | static Json parse(const char * in, 168 | std::string & err, 169 | JsonParse strategy = JsonParse::STANDARD) { 170 | if (in) { 171 | return parse(std::string(in), err, strategy); 172 | } else { 173 | err = "null input"; 174 | return nullptr; 175 | } 176 | } 177 | // Parse multiple objects, concatenated or separated by whitespace 178 | static std::vector parse_multi( 179 | const std::string & in, 180 | std::string::size_type & parser_stop_pos, 181 | std::string & err, 182 | JsonParse strategy = JsonParse::STANDARD); 183 | 184 | static inline std::vector parse_multi( 185 | const std::string & in, 186 | std::string & err, 187 | JsonParse strategy = JsonParse::STANDARD) { 188 | std::string::size_type parser_stop_pos; 189 | return parse_multi(in, parser_stop_pos, err, strategy); 190 | } 191 | 192 | bool operator== (const Json &rhs) const; 193 | bool operator< (const Json &rhs) const; 194 | bool operator!= (const Json &rhs) const { return !(*this == rhs); } 195 | bool operator<= (const Json &rhs) const { return !(rhs < *this); } 196 | bool operator> (const Json &rhs) const { return (rhs < *this); } 197 | bool operator>= (const Json &rhs) const { return !(*this < rhs); } 198 | 199 | /* has_shape(types, err) 200 | * 201 | * Return true if this is a JSON object and, for each item in types, has a field of 202 | * the given type. If not, return false and set err to a descriptive message. 203 | */ 204 | typedef std::initializer_list> shape; 205 | bool has_shape(const shape & types, std::string & err) const; 206 | 207 | private: 208 | std::shared_ptr m_ptr; 209 | }; 210 | 211 | // Internal class hierarchy - JsonValue objects are not exposed to users of this API. 212 | class JsonValue { 213 | protected: 214 | friend class Json; 215 | friend class JsonInt; 216 | friend class JsonDouble; 217 | virtual Json::Type type() const = 0; 218 | virtual bool equals(const JsonValue * other) const = 0; 219 | virtual bool less(const JsonValue * other) const = 0; 220 | virtual void dump(std::string &out) const = 0; 221 | virtual double number_value() const; 222 | virtual int int_value() const; 223 | virtual bool bool_value() const; 224 | virtual const std::string &string_value() const; 225 | virtual const Json::array &array_items() const; 226 | virtual const Json &operator[](size_t i) const; 227 | virtual const Json::object &object_items() const; 228 | virtual const Json &operator[](const std::string &key) const; 229 | virtual ~JsonValue() {} 230 | }; 231 | 232 | } // namespace json11 233 | -------------------------------------------------------------------------------- /app/realtime_webcam/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | #include 11 | #include 12 | #include 13 | #include 14 | 15 | #include "json11.hpp" 16 | #include "dma_simple.h" 17 | using namespace std; 18 | using namespace cv; 19 | 20 | void *dma_regs; 21 | void *hls_regs; 22 | struct udmabuf intake_buf; 23 | struct udmabuf outlet_buf; 24 | 25 | float THRESH; 26 | float BIAS; 27 | bool LOG_MODE; 28 | 29 | struct window_rect{ 30 | int sy; 31 | int sx; 32 | int ey; 33 | int ex; 34 | }; 35 | 36 | window_rect shukai_waku, cross_waku; 37 | 38 | unsigned int *assignToPhysicalUInt(unsigned long address,unsigned int size){ 39 | int devmem = open("/dev/mem", O_RDWR | O_SYNC); 40 | off_t PageOffset = (off_t) address % getpagesize(); 41 | off_t PageAddress = (off_t) (address - PageOffset); 42 | return (unsigned int *) mmap(0, size*sizeof(unsigned int), PROT_READ|PROT_WRITE, MAP_SHARED, devmem, PageAddress); 43 | } 44 | 45 | int hw_setup(){ 46 | int uio1_fd = open("/dev/uio1", O_RDWR); 47 | hls_regs = mmap(NULL, 0x10000, PROT_READ|PROT_WRITE, MAP_SHARED, uio1_fd, 0); 48 | 49 | int uio2_fd = open("/dev/uio2", O_RDWR); 50 | dma_regs = mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, uio2_fd, 0); 51 | 52 | if(udmabuf_open(&intake_buf, "udmabuf0") == -1){ 53 | printf("udmabuf_open failed\n"); 54 | return -1; 55 | } 56 | if(udmabuf_open(&outlet_buf, "udmabuf1") == -1){ 57 | printf("udmabuf_open failed\n"); 58 | return -1; 59 | } 60 | dma_reset(dma_regs); 61 | regs_write32(hls_regs, 0x81); 62 | return 1; 63 | } 64 | 65 | void writebram(unsigned int* target, string array_name, json11::Json json){ 66 | int cnt = 0; 67 | for(auto &k: json[array_name].array_items()){ 68 | target[cnt++] = k.number_value(); 69 | } 70 | } 71 | int json_setup(){ 72 | unsigned int* bgrhsv_w1 = assignToPhysicalUInt(0xA0000000, 24*4); 73 | unsigned int* bgrhsv_w2 = assignToPhysicalUInt(0xA0004000, 24*4); 74 | unsigned int* bgrhsv_w3 = assignToPhysicalUInt(0xA0008000, 24*4); 75 | unsigned int* bgrhsv_w4 = assignToPhysicalUInt(0xA000C000, 24*4); 76 | unsigned int* hog_w1 = assignToPhysicalUInt(0xA0010000, 63*4); 77 | unsigned int* hog_w2 = assignToPhysicalUInt(0xA0014000, 63*4); 78 | unsigned int* hog_w3 = assignToPhysicalUInt(0xA0018000, 63*4); 79 | 80 | ifstream ifs("weights.json"); 81 | if(ifs.fail()){ 82 | cerr << "json file load failed" << endl; 83 | return -1; 84 | } 85 | std::istreambuf_iterator it(ifs); 86 | std::istreambuf_iterator last; 87 | std::string str(it, last); 88 | string err; 89 | auto json = json11::Json::parse(str, err); 90 | if(err != ""){ 91 | cerr << "json file parse error: " << err << endl; 92 | return -1; 93 | } 94 | 95 | writebram(hog_w1, "hog_w1", json); 96 | writebram(hog_w2, "hog_w2", json); 97 | writebram(hog_w3, "hog_w3", json); 98 | writebram(bgrhsv_w1, "bgrhsv_w1", json); 99 | writebram(bgrhsv_w2, "bgrhsv_w2", json); 100 | writebram(bgrhsv_w3, "bgrhsv_w3", json); 101 | writebram(bgrhsv_w4, "bgrhsv_w4", json); 102 | 103 | THRESH = json["thresh"].number_value(); 104 | BIAS = json["bias"].number_value(); 105 | LOG_MODE = (json["log_mode"].int_value() == 1); 106 | 107 | shukai_waku.sy = json["shukai_sy"].int_value(); 108 | shukai_waku.sx = json["shukai_sx"].int_value(); 109 | shukai_waku.ey = json["shukai_ey"].int_value(); 110 | shukai_waku.ex = json["shukai_ex"].int_value(); 111 | 112 | cross_waku.sy = json["cross_sy"].int_value(); 113 | cross_waku.sx = json["cross_sx"].int_value(); 114 | cross_waku.ey = json["cross_ey"].int_value(); 115 | cross_waku.ex = json["cross_ex"].int_value(); 116 | 117 | return 1; 118 | } 119 | 120 | cv::Mat getShrinkFrame(cv::Mat original_img, int window_height){ 121 | //shrink whole frame 122 | float scaling = 32.0/window_height; 123 | // cv::Mat resized_img(cv::Size(640.0*scaling, 480.0*scaling), CV_8UC3); 124 | cv::Mat resized_img; 125 | resize(original_img, resized_img, cv::Size(), scaling, scaling); 126 | //pase scaled image to 320x240 canvas 127 | cv::Mat scaled_frame(cv::Size(320, 240), CV_8UC3); 128 | cv::Mat roi_dst = scaled_frame(cv::Rect(0, 0, 640*scaling, 480*scaling)); 129 | resized_img.copyTo(roi_dst); 130 | return scaled_frame; 131 | } 132 | 133 | //input: 320x240pix 134 | //output: ((sy,sx), window_height) 135 | vector,int>> predictRectFrame(cv::Mat inputimg, int window_height, window_rect waku, int sy = 0, int sx = 0){ 136 | vector,int>> rst; 137 | 138 | cv::Mat bgra_img; 139 | cv::cvtColor(inputimg, bgra_img, CV_BGR2BGRA); 140 | //copy image pixel value to input buffer 141 | for(int y = 0; y < 240; y++) { 142 | memcpy(intake_buf.buf + 320 * y * 4, bgra_img.data + bgra_img.step * y, bgra_img.cols * 4 * sizeof(unsigned char)); 143 | } 144 | 145 | dma_setup(dma_regs, intake_buf.phys_addr, outlet_buf.phys_addr); 146 | dma_outlet_start(dma_regs, 27*33*4); 147 | dma_intake_start(dma_regs, 320*240*4); 148 | dma_wait_irq(dma_regs); 149 | dma_clear_status(dma_regs); 150 | 151 | for(int y = 0; y < 27; y++){ 152 | for(int x = 0; x < 33; x++){ 153 | int index = y * 33 + x; 154 | float hwrst = ((float*) (outlet_buf.buf))[index]; 155 | hwrst += BIAS; 156 | // float proba = 1.0/(1.0+exp(-rst)); 157 | if(hwrst > THRESH){ 158 | int detected_y = (int)((float)y * 8 * window_height / 32.0) + sy; 159 | int detected_x = (int)((float)x * 8 * window_height / 32.0) + sx; 160 | //inside window rect 161 | if(waku.sy <= detected_y && waku.sx <= detected_x && detected_y + window_height <= waku.ey && detected_x + window_height*2 <= waku.ex){ 162 | float proba = 1.0/(1.0+exp(-hwrst)); 163 | if(LOG_MODE) cout << window_height << " " << proba << " " << detected_y << " " << detected_x << endl; 164 | rst.push_back(make_pair(make_pair(detected_y, detected_x), window_height)); 165 | } 166 | } 167 | } 168 | } 169 | return rst; 170 | } 171 | 172 | vector,int>> processRectFrame(cv::Mat original_img, int window_height, window_rect waku, int sy = 0, int sx = 0){ 173 | if(window_height <= 32){ 174 | //crop and predict 175 | Mat cropped_img = original_img(cv::Rect(sx, sy, 320, 240)); 176 | return predictRectFrame(cropped_img, window_height, waku, sy, sx); 177 | }else if(window_height <= 64){ 178 | cv::Mat resized_img; 179 | float scaling = 32.0/window_height; 180 | resize(original_img, resized_img, cv::Size(), scaling, scaling); 181 | Mat cropped_img = resized_img(cv::Rect(0, 0, 320, 240)); 182 | return predictRectFrame(cropped_img, window_height, waku); 183 | }else { 184 | Mat shrinked_img = getShrinkFrame(original_img, window_height); 185 | return predictRectFrame(shrinked_img, window_height, waku); 186 | } 187 | } 188 | 189 | int main(int argc, char *argv[]){ 190 | int mode = atoi(argv[1]); 191 | 192 | if(json_setup() < 0){ 193 | cout << "json setup failed" << endl; 194 | return -1; 195 | } 196 | if(hw_setup() < 0){ 197 | cout << "hw_setup() failed" << endl; 198 | return -1; 199 | } 200 | cout << "hw_setup() finished" << endl; 201 | 202 | cv::VideoCapture cap(0); 203 | if(!cap.isOpened()){ 204 | cout << "failed" << endl; 205 | return -1; 206 | } 207 | 208 | std::chrono::system_clock::time_point t1, t2; 209 | while(1){ 210 | t1 = std::chrono::system_clock::now(); 211 | 212 | cv::Mat frame; 213 | cap >> frame; // get a new frame from camera 214 | cv::Mat res = frame.clone(); 215 | 216 | window_rect waku; 217 | if(mode == 0) waku = shukai_waku; 218 | else waku = cross_waku; 219 | 220 | auto candidates = processRectFrame(frame, 32, waku, 240, 160); 221 | auto candidates2 = processRectFrame(frame, 80, waku); 222 | auto candidates3 = processRectFrame(frame, 60, waku); 223 | 224 | t2 = std::chrono::system_clock::now(); 225 | double elapsed = (double)std::chrono::duration_cast(t2-t1).count(); 226 | if(LOG_MODE) cout << "elapsed:" << elapsed << "[milisec]" << endl; 227 | if(LOG_MODE) cout << "fps:" << 1000.0/elapsed << "[fps]" << endl; 228 | } 229 | return 0; 230 | } 231 | -------------------------------------------------------------------------------- /app/realtime_webcam/weights.json: -------------------------------------------------------------------------------- 1 | { 2 | "thresh" : 1.3862943611, 3 | "bias" : -2.1975765, 4 | "bgrhsv_w1" : [286925, 414125, 4294792783, 4294530074, 180523, 302453, 121256, 4294838112, 4294852325, 175511, 57948, 220327, 188681, 246448, 4294175583, 62839, 213119, 192516, 158200, 4294599129, 4294870722, 195913, 21862, 108219, 135707, 4294560836, 4294531582, 471793, 100014, 4294687414, 4294795984, 66431, 136769, 4294626978, 74939, 4294520268, 4294803702, 4294833774, 1358169, 228843, 4294533169, 4294711845, 4294635639, 4294561320, 4294548122, 4294877153, 4294580325, 4294701642, 315921, 4294787473, 4294823948, 931766, 69555, 4294609375, 4294504419, 4294420190, 280053, 4294545252, 117506, 4294416930, 224873, 4294892981, 33826, 179021, 4294919630, 4294754903, 124001, 187118, 932180, 99537, 628398, 35208, 4293709472, 4294617393, 4294545899, 4294858360, 4293714819, 4294509769, 1283148, 4294901810, 681494, 184406, 475595, 105753, 4294544366, 4294691033, 4294686162, 4294130703, 4294366668, 4294703790, 4294894858, 4294914286, 651540, 188299, 385667, 73682], 5 | "bgrhsv_w2" : [4294678382, 4294426082, 33900, 4294786573, 4294798260, 4294465271, 4294517259, 181256, 300716, 4294576336, 4294779851, 4294392855, 4294501450, 4294559575, 1747455, 1895189, 4294343273, 4294494832, 4293427624, 4293396760, 557902, 231784, 4294216869, 4294397969, 370238, 4294689001, 4294732968, 4293838387, 285698, 4294924877, 4292752297, 4294311841, 1373130, 527198, 447725, 88187, 316414, 133618, 4294912849, 336151, 4294763669, 4294699908, 4294049825, 4293331584, 823228, 226144, 459479, 4294963093, 368041, 4294110388, 520032, 716422, 4293988786, 4293440987, 4294441252, 4294108101, 354961, 98951, 133964, 4294626653, 330929, 45665, 4294777303, 4294750516, 4294768541, 4294572619, 4294612355, 4294583897, 1410668, 1060654, 943010, 574817, 4294563529, 4294621839, 749760, 4294735436, 4294558413, 4294577201, 1507408, 608711, 1990093, 922274, 1535905, 642773, 4294579386, 4294841981, 128733, 160141, 4294488597, 4294652830, 24372, 77325, 303718, 294846, 71547, 278933], 6 | "bgrhsv_w3" : [136128, 4294683817, 360450, 824960, 4294829132, 4294484903, 78219, 558577, 4294953960, 4294683371, 26876, 4294689487, 4294585981, 4294597802, 657838, 175524, 4294797448, 4294450399, 4293758241, 4293295348, 4294749523, 560172, 4294399572, 4294458180, 4294539494, 4294467598, 6848, 533868, 4294052521, 4294123324, 4293517802, 4292904425, 4294275118, 23086, 4294311795, 4294322181, 186208, 323675, 439545, 1009885, 4294276375, 4294211295, 4294829568, 4294063271, 434514, 37201, 542138, 8651, 4293960369, 341051, 656091, 7056, 4292601118, 4293503497, 197622, 4294515689, 115372, 154966, 106308, 213954, 61908, 504986, 4294661555, 178876, 4294199586, 4294713840, 4294502092, 4294531736, 236442, 949123, 175692, 710489, 145860, 4294823904, 4294953657, 84629, 4294579148, 4294668689, 236680, 1522779, 970197, 1739651, 890460, 1415223, 4294920959, 4294751862, 319014, 4294912025, 4294638246, 4294419507, 4294530293, 30753, 4294946745, 374105, 4294819087, 180199], 7 | "bgrhsv_w4" : [403704, 4294801894, 868547, 316093, 115595, 4294499655, 4294661153, 618745, 4294959983, 4294505215, 264048, 4294678169, 488273, 469491, 774405, 206550, 387303, 174489, 4294145055, 4294080680, 179321, 4294677598, 392911, 22020, 327094, 755533, 440055, 73898, 17878, 41040, 4294492078, 4293955327, 4294578226, 219646, 4294846005, 326587, 4294945083, 1145589, 4294730853, 4294798050, 4294823835, 602937, 4294863616, 4294438105, 4294802318, 736541, 4294860055, 1114370, 4294945265, 236557, 400947, 4294794605, 4294662560, 4294360523, 4294804086, 995708, 79256, 689381, 228611, 1236042, 4294564002, 4294623144, 700676, 364280, 4294086319, 4293948378, 4294932167, 459088, 262850, 374235, 223124, 523079, 143209, 46147, 474402, 8173, 4294640515, 4294622236, 4294896322, 289868, 292561, 548621, 497756, 574074, 4294835465, 4294695249, 802842, 620113, 4294612231, 4294192971, 4294427941, 4294618022, 80447, 4294828207, 157610, 4294792798], 8 | "hog_w1" : [535309, 4293005267, 1402609, 4293041238, 4294306044, 4294445583, 3351008, 4292520672, 1012001, 871316, 70594, 202348, 4294153533, 4292716739, 3076009, 749431, 1517115, 4293920630, 435200, 4294356531, 4699442, 3508577, 4277500, 4292252235, 4292245271, 295506, 4293593549, 4289819238, 4293805393, 4293440057, 4294557208, 4291144504, 1465615, 4998613, 2884052, 531450, 3848662, 4294403727, 4292579865, 4292561336, 487626, 677523, 4291863723, 391728, 1326555, 2502535, 2168643, 34821, 1763602, 4290987194, 591466, 2011220, 1501187, 4294170042, 4292777586, 57088, 125006, 4292118314, 4293746914, 1245292, 4566514, 4289637780, 519179, 4293515881, 405758, 4292490713, 4293693879, 4293747840, 1224580, 4294313136, 2525635, 4445466, 953641, 4292925704, 4294211398, 4292235420, 288479, 4294906231, 3559886, 4289165495, 4293793757, 1078188, 4393351, 1235044, 4293138825, 4294184806, 4294422484, 3681230, 344712, 4293824920, 4292272383, 4293652092, 4885701, 49470, 4292582390, 1093057, 527765, 2736123, 1074189, 3877691, 414689, 4294310962, 4293406974, 649369, 1457623, 4294770504, 4292611076, 6025879, 4294748524, 912863, 1918596, 4292258316, 4293798268, 4294934847, 4294341057, 4294920153, 4291889023, 4293867324, 128412, 4127868, 4293567494, 832332, 3056900, 4294804250, 4294631683, 1197861, 4293396700, 4292846950, 1238821, 3440306, 779631, 4294223017, 2017061, 4293733195, 212887, 1556647, 4294386898, 4294886721, 4294413077, 404240, 1772765, 856047, 820263, 349217, 4294539302, 1312191, 4294128386, 4294607018, 4294738996, 2901963, 133066, 4292206186, 2091738, 4293998541, 4291544306, 1750836, 4292019185, 1030596, 1187554, 3681579, 4294449500, 4293210564, 1240103, 4293388127, 829861, 562889, 1779854, 4289428213, 4398200, 4294167808, 4658810, 4293192198, 1162426, 4293593232, 117845, 4293729413, 4294684496, 4294656958, 737383, 2859885, 1043086, 1465339, 4290757268, 4294039726, 4293481430, 2560099, 4294765710, 990129, 1717358, 3916676, 912265, 4290873841, 4293620286, 4294276654, 4294490870, 4293493745, 653789, 439495, 4292196903, 4294952344, 4235220, 416617, 4290897675, 2476391, 1445530, 4294509798, 4294076629, 6888573, 4293422447, 1010112, 4293387382, 4292546549, 718258, 290822, 1269663, 519313, 5221375, 611997, 4287869637, 4290619128, 4290760957, 455221, 4290452115, 5011140, 4291910045, 5741630, 330635, 9983, 4289466062, 23062, 4289290867, 4292644336, 1672612, 482817, 4294035212, 4293088324, 2703730, 1001528, 4290377013, 4290968144, 3219596, 467878, 4294737188, 348047, 4293967923, 4294174124, 4291398680, 4293887631, 77063, 4294385984, 6214884, 961048], 9 | "hog_w2" : [93835, 4293342059, 4294531268, 4292038590, 10805383, 4293759014, 645676, 1309227, 13581, 4293266446, 3031018, 601713, 4293551746, 4293895330, 4292238468, 805111, 4294286524, 121735, 4292807381, 4293473740, 2727095, 971130, 2057365, 8529918, 4861036, 612012, 4291073383, 754264, 1052528, 4884032, 4291423882, 4293291081, 2820368, 2088045, 4293500019, 2154234, 1424912, 1392018, 1283110, 4293986703, 4294486473, 3768367, 4294604033, 4294922791, 4292978179, 4294496424, 4294936587, 2315338, 1299850, 4294743778, 4293413163, 4292136525, 4294572948, 935642, 4294441732, 4293636743, 1289055, 3852683, 4294925812, 6657322, 2342630, 4294684726, 2498375, 4293446354, 4292872631, 4294438083, 4293015100, 4294147721, 3042360, 1705251, 4294376796, 61344, 4293639080, 4294008331, 427264, 877418, 1432024, 93662, 252826, 2335749, 4291899531, 4291769264, 223558, 2542304, 4293848510, 2208596, 797028, 4291987681, 4294272582, 332661, 25330, 4293950859, 2648866, 2371695, 2179449, 935402, 4292457429, 1305322, 4291262854, 4401446, 1070351, 577245, 4293008202, 4293765849, 135507, 1776482, 4294779938, 4293737083, 622171, 4293856777, 279197, 254734, 1113806, 630281, 839723, 4293936725, 4291449056, 1393231, 501479, 4294055579, 4293201395, 1208671, 4292901311, 1312390, 454305, 4294502573, 4294118524, 1775475, 721032, 1469306, 278848, 6143929, 4294610535, 4289529775, 648405, 1422505, 4294374616, 4293813980, 4294726813, 33552, 999510, 4291541326, 2028982, 1021456, 4294009244, 1108565, 2932174, 2220695, 4290950697, 2476670, 1070787, 935288, 4290360483, 4290518976, 5384820, 4293216842, 2461671, 4291326399, 4294553775, 4292382069, 1302762, 4292540478, 18026, 4293061374, 3438488, 4292571662, 62041, 2463772, 1194992, 4291863400, 391556, 647275, 4532957, 4289142757, 4294402821, 4294612341, 2674712, 4292074073, 759478, 4293847313, 521803, 4294330342, 1267877, 1334609, 4292818905, 4292803422, 388266, 1006662, 6534351, 4289956787, 4669485, 1538580, 4294492578, 4294509527, 40091, 4291557289, 4294146081, 4293063911, 4294883107, 4294276546, 1870904, 1538610, 4294608821, 585632, 2854841, 4294107121, 4293280517, 4294116389, 1202717, 4293502613, 4293079915, 4291814762, 3682184, 4292243268, 671004, 4294465232, 2184287, 4035836, 3030619, 4294423413, 317758, 2283875, 4289943065, 626827, 4293106887, 6866259, 4289653641, 4114956, 4289365260, 4293124734, 4288570873, 4294136002, 4293290114, 4293338974, 548332, 255816, 1505268, 1122866, 2261413, 1344533, 3293154, 4293774144, 1698442, 984079, 4291165143, 4292864904, 4292036243, 4293881221, 4294022829, 4292988710, 1920647, 4294896043], 10 | "hog_w3" : [4288671221, 4293277452, 4293753831, 826774, 4293946600, 4290259719, 4291727443, 4293994659, 393577, 4289938510, 4293175415, 4293667656, 4292637791, 4615737, 287022, 1935141, 4292149312, 4293255182, 4293549308, 4294557896, 4292678228, 4294884589, 3170764, 4349959, 1810957, 4723497, 1169103, 5865362, 76205, 2629854, 1878429, 7867789, 4294808249, 10009090, 4292134700, 2040181, 4293405275, 4290241233, 3097180, 4294799887, 4293188933, 4294164846, 3245244, 4294194678, 4294719090, 2463483, 953484, 4293555969, 17833, 54375, 4631450, 4294940994, 4294814344, 4294471119, 4294371023, 4294327529, 4289671201, 4293951002, 4293033818, 6377944, 2425460, 1360672, 469512, 1797291, 4294745704, 4294745340, 4290928522, 2288995, 2915604, 4294227675, 2860540, 4294517113, 4294519449, 4293600403, 369671, 222744, 4293708509, 4292689120, 5096731, 4294652781, 4292217965, 1072005, 4294033148, 4292335022, 4220137, 4293725667, 4294196200, 4294555469, 4294941425, 4294601618, 4294045130, 270302, 4292049351, 4291411024, 4293693218, 5073016, 4291094484, 4489388, 4291490242, 9262732, 4294028258, 4293749907, 4292677901, 450238, 2139340, 1605037, 4293712086, 3357119, 4291625455, 4294584780, 1017873, 1465073, 4292621616, 4294722166, 3874530, 5116058, 4284425932, 4291786252, 4289287370, 1984084, 3566816, 5866708, 4294350086, 318196, 4294355941, 4294501973, 4294816112, 962694, 4290257315, 4292106529, 4294820463, 2969813, 530991, 4290925477, 4294910851, 390810, 6515811, 4294534493, 4291785315, 1343217, 1056256, 1213310, 1307446, 157171, 4291691148, 4289699149, 465844, 11044, 4292996706, 4291026338, 3658635, 647187, 2367832, 4282365451, 1935707, 4290352817, 5217993, 3381887, 6224912, 4293187021, 1001931, 4293584094, 1089522, 4293313459, 2244379, 4290256419, 1136020, 4294935603, 1907829, 4292389216, 4293580919, 364757, 2131366, 3879182, 4294274945, 4291700164, 2118247, 251943, 1449082, 393411, 4288152058, 4289721973, 1204548, 4294810895, 4286326905, 4291082636, 1433209, 1460694, 1070349, 4293226656, 6656792, 176541, 4292417449, 1354748, 1210588, 732139, 4294393538, 4294863870, 4294928486, 4294478485, 4290745000, 4294098953, 4293215231, 1599444, 4294264925, 4293951443, 4294455804, 743307, 263527, 4294596838, 1469140, 1491638, 1393939, 1381611, 3780483, 694923, 4289030942, 4289779056, 1531386, 2660291, 3081849, 4291980098, 5161734, 2230192, 1513958, 4425298, 1694744, 2474751, 4292397831, 4294225998, 4294718011, 4294127357, 4291874224, 4293631955, 4294569586, 82024, 4286408384, 4290555389, 4293135058, 318684, 4282881483, 4291793422, 4293789646, 351694, 4286676998, 4293848505, 908723, 2560265, 661082, 1012789, 708343, 2094701], 11 | "shukai_sy" : 60, 12 | "shukai_sx" : 110, 13 | "shukai_ey" : 250, 14 | "shukai_ex" : 530, 15 | 16 | "cross_sy" : 280, 17 | "cross_sx" : 160, 18 | "cross_ey" : 350, 19 | "cross_ex" : 480, 20 | "log_mode" : 1 21 | } -------------------------------------------------------------------------------- /cpp/frametest/compile_frametest.sh: -------------------------------------------------------------------------------- 1 | g++ -std=c++11 frametest.cpp -I/usr/local/include/opencv2 -I/usr/local/include/opencv -L/usr/local/lib -lopencv_core -lopencv_imgcodecs -lopencv_highgui -L/usr/local/lib -lopencv_shape -lopencv_stitching -lopencv_objdetect -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_features2d -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_video -lopencv_photo -lopencv_ml -lopencv_imgproc -lopencv_flann -lopencv_core 2 | -------------------------------------------------------------------------------- /cpp/frametest/feature.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | using namespace std; 11 | using namespace cv; 12 | 13 | #define HOG_WIDTH 64 14 | #define HOG_HEIGHT 32 15 | #define CELL_SIZE_ROW 8 16 | #define CELL_SIZE_COL 8 17 | #define BLOCK_SIZE_ROW 2 18 | #define BLOCK_SIZE_COL 2 19 | #define N_CELLS_ROW HOG_HEIGHT/CELL_SIZE_ROW //4 20 | #define N_CELLS_COL HOG_WIDTH/CELL_SIZE_COL //8 21 | #define N_BLOCK_ROW (N_CELLS_ROW - 1) //3 22 | #define N_BLOCK_COL (N_CELLS_COL - 1) //7 23 | #define HISTOGRAMSIZE N_BLOCK_ROW * N_BLOCK_COL * BLOCK_SIZE_ROW * BLOCK_SIZE_COL * 9 //3*7*2*2*9 24 | #define FEATURESIZE (HISTOGRAMSIZE + 192*2) 25 | 26 | 27 | inline int approx_distance(int dx, int dy){ 28 | int min, max; //uint 29 | if(dx < 0) dx = -dx; 30 | if(dy < 0) dy = -dy; 31 | 32 | if(dx < dy){ 33 | min = dx; 34 | max = dy; 35 | }else{ 36 | min = dy; 37 | max = dx; 38 | } 39 | 40 | //coefficients equivalent to (123/128 * max) and (51/128*min) 41 | return ((( max << 8 ) + ( max << 3 ) - ( max << 4 ) - ( max << 1 ) + 42 | ( min << 7 ) - ( min << 5 ) + ( min << 3 ) - ( min << 1 )) >> 8 ); 43 | } 44 | 45 | void lite_hog(cv::Mat img, double* dst, bool approx_mode){ 46 | int descriptor[HISTOGRAMSIZE]={0}; 47 | int sum_of_block[N_BLOCK_ROW * N_BLOCK_COL]={0}; 48 | for(int i = 0; i < HISTOGRAMSIZE; i++) descriptor[i] = 0; 49 | for(int i = 0; i < N_BLOCK_ROW * N_BLOCK_COL; i++) sum_of_block[i] = 0; 50 | int lut0[256] = {0,0,0,1,1,1,2,2,2,3,3,3,4,4,5,5,5,6,6,6,7,7,7,8,8,9,9,9,10,10,10,11,11,11,12,12,13,13,13,14,14,14,15,15,15,16,16,17,17,17,18,18,18,19,19,19,20,20,21,21,21,22,22,22,23,23,23,24,24,25,25,25,26,26,26,27,27,27,28,28,29,29,29,30,30,30,31,31,31,32,32,33,33,33,34,34,34,35,35,35,36,36,37,37,37,38,38,38,39,39,39,40,40,41,41,41,42,42,42,43,43,43,44,44,45,45,45,46,46,46,47,47,47,48,48,49,49,49,50,50,50,51,51,51,52,52,53,53,53,54,54,54,55,55,55,56,56,57,57,57,58,58,58,59,59,59,60,60,61,61,61,62,62,62,63,63,63,64,64,65,65,65,66,66,66,67,67,67,68,68,69,69,69,70,70,70,71,71,71,72,72,73,73,73,74,74,74,75,75,75,76,76,77,77,77,78,78,78,79,79,79,80,80,81,81,81,82,82,82,83,83,83,84,84,85,85,85,86,86,86,87,87,87,88,88,89,89,89,90,90,90,91,91,91,92,92}; 51 | int lut1[256] = {0,0,1,2,3,4,5,5,6,7,8,9,10,10,11,12,13,14,15,15,16,17,18,19,20,20,21,22,23,24,25,26,26,27,28,29,30,31,31,32,33,34,35,36,36,37,38,39,40,41,41,42,43,44,45,46,46,47,48,49,50,51,52,52,53,54,55,56,57,57,58,59,60,61,62,62,63,64,65,66,67,67,68,69,70,71,72,72,73,74,75,76,77,78,78,79,80,81,82,83,83,84,85,86,87,88,88,89,90,91,92,93,93,94,95,96,97,98,98,99,100,101,102,103,104,104,105,106,107,108,109,109,110,111,112,113,114,114,115,116,117,118,119,119,120,121,122,123,124,124,125,126,127,128,129,130,130,131,132,133,134,135,135,136,137,138,139,140,140,141,142,143,144,145,145,146,147,148,149,150,150,151,152,153,154,155,156,156,157,158,159,160,161,161,162,163,164,165,166,166,167,168,169,170,171,171,172,173,174,175,176,177,177,178,179,180,181,182,182,183,184,185,186,187,187,188,189,190,191,192,192,193,194,195,196,197,197,198,199,200,201,202,203,203,204,205,206,207,208,208,209,210,211,212,213,213}; 52 | int lut2[256] = {0,1,3,5,6,8,10,12,13,15,17,19,20,22,24,25,27,29,31,32,34,36,38,39,41,43,44,46,48,50,51,53,55,57,58,60,62,63,65,67,69,70,72,74,76,77,79,81,83,84,86,88,89,91,93,95,96,98,100,102,103,105,107,108,110,112,114,115,117,119,121,122,124,126,127,129,131,133,134,136,138,140,141,143,145,147,148,150,152,153,155,157,159,160,162,164,166,167,169,171,172,174,176,178,179,181,183,185,186,188,190,191,193,195,197,198,200,202,204,205,207,209,210,212,214,216,217,219,221,223,224,226,228,230,231,233,235,236,238,240,242,243,245,247,249,250,252,254,255,257,259,261,262,264,266,268,269,271,273,274,276,278,280,281,283,285,287,288,290,292,294,295,297,299,300,302,304,306,307,309,311,313,314,316,318,319,321,323,325,326,328,330,332,333,335,337,338,340,342,344,345,347,349,351,352,354,356,358,359,361,363,364,366,368,370,371,373,375,377,378,380,382,383,385,387,389,390,392,394,396,397,399,401,402,404,406,408,409,411,413,415,416,418,420,421,423,425,427,428,430,432,434,435,437,439,441}; 53 | int lut3[256] = {0,5,11,17,22,28,34,39,45,51,56,62,68,73,79,85,90,96,102,107,113,119,124,130,136,141,147,153,158,164,170,175,181,187,192,198,204,209,215,221,226,232,238,243,249,255,260,266,272,277,283,289,294,300,306,311,317,323,328,334,340,345,351,357,362,368,374,379,385,391,396,402,408,413,419,425,430,436,442,447,453,459,464,470,476,481,487,493,498,504,510,515,521,527,532,538,544,549,555,561,566,572,578,584,589,595,601,606,612,618,623,629,635,640,646,652,657,663,669,674,680,686,691,697,703,708,714,720,725,731,737,742,748,754,759,765,771,776,782,788,793,799,805,810,816,822,827,833,839,844,850,856,861,867,873,878,884,890,895,901,907,912,918,924,929,935,941,946,952,958,963,969,975,980,986,992,997,1003,1009,1014,1020,1026,1031,1037,1043,1048,1054,1060,1065,1071,1077,1082,1088,1094,1099,1105,1111,1116,1122,1128,1133,1139,1145,1150,1156,1162,1168,1173,1179,1185,1190,1196,1202,1207,1213,1219,1224,1230,1236,1241,1247,1253,1258,1264,1270,1275,1281,1287,1292,1298,1304,1309,1315,1321,1326,1332,1338,1343,1349,1355,1360,1366,1372,1377,1383,1389,1394,1400,1406,1411,1417,1423,1428,1434,1440,1445}; 54 | 55 | for(int y = 0; y < N_BLOCK_ROW; y++){//3 56 | for(int x = 0; x < N_BLOCK_COL; x++){//7 57 | for(int yy = 0; yy < CELL_SIZE_ROW*BLOCK_SIZE_ROW; yy++){//16 58 | for(int xx = 0; xx < CELL_SIZE_COL*BLOCK_SIZE_COL; xx++){//16 59 | int ny = CELL_SIZE_ROW * y + yy; 60 | int nx = CELL_SIZE_COL * x + xx; 61 | //compute gradient 62 | int mag = 0; 63 | int bin_index; 64 | if(ny == 0 || nx == 0 || ny == 31 || nx == 63){ 65 | mag = 0; 66 | bin_index = 0; 67 | }else{ 68 | int Gx = int(img.ptr(ny)[nx+1]) - int(img.ptr(ny)[nx-1]); 69 | int Gy = int(img.ptr(ny+1)[nx]) - int(img.ptr(ny-1)[nx]); 70 | mag = (approx_mode ? (approx_distance(Gx, Gy)) : sqrt((double)(Gx*Gx + Gy * Gy))); 71 | if((Gx >= 0 && Gy >= 0) || (Gx <= 0 && Gy <= 0)){ 72 | if (abs(Gy) < lut0[abs(Gx)]) {bin_index = 0;} 73 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 1;} 74 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 2;} 75 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 3;} 76 | else {bin_index = 4;} 77 | } else{ 78 | if (abs(Gy) < lut0[abs(Gx)]){bin_index = 4;} 79 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 5;} 80 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 6;} 81 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 7;} 82 | else {bin_index = 8;} 83 | } 84 | } 85 | int cell_index = (yy / 8) * 2 + (xx / 8); 86 | int block_index = y * N_BLOCK_COL + x; 87 | int descrip_index = block_index * 36 + cell_index * 9 + bin_index; 88 | descriptor[descrip_index] += mag; 89 | sum_of_block[block_index] += mag; 90 | } 91 | } 92 | } 93 | } 94 | 95 | for(int by = 0; by < N_BLOCK_ROW; by++){//3 96 | for(int bx = 0; bx < N_BLOCK_COL; bx++){//7 97 | int blkIdx = by * N_BLOCK_COL + bx; 98 | for (int i = 0; i < 36; i++) { 99 | int index = blkIdx * 36 + i; 100 | if(sum_of_block[blkIdx] == 0){ 101 | dst[index] = 0; 102 | }else{ 103 | dst[index] = (double)descriptor[index]/(double)sum_of_block[blkIdx]; 104 | } 105 | } 106 | } 107 | } 108 | } 109 | 110 | 111 | // void ravel(cv::Mat img, int* dst){ 112 | void ravel(cv::Mat img, double* dst){ 113 | vector rst; 114 | for(int y = 0; y < img.rows; y++){ 115 | for(int x = 0; x < img.cols; x++){ 116 | cv::Vec pix = img.ptr(y)[x]; 117 | dst[3 * (y * img.cols + x)] = double(pix[0])/256.0; 118 | dst[3 * (y * img.cols + x) + 1] = double(pix[1])/256.0; 119 | dst[3 * (y * img.cols + x) + 2] = double(pix[2])/256.0; 120 | } 121 | } 122 | } 123 | 124 | 125 | //Input: 126 | //img: BGR image 127 | //dst: pointer to save extracted feature value 128 | void getFeature(cv::Mat img, double* dst, bool approx_mode){ 129 | Mat resized_img, gray; 130 | cv::resize(img, resized_img, cv::Size(64 ,32), INTER_NEAREST); 131 | 132 | cv::Mat resized_hls; 133 | cv::cvtColor(resized_img, resized_hls, CV_BGR2HSV); 134 | 135 | cv::Size spatial_size(8, 8); 136 | Mat spatial_rgb, spatial_hls; 137 | cv::resize(resized_img, spatial_rgb, spatial_size, INTER_LINEAR); 138 | cv::resize(resized_hls, spatial_hls, spatial_size, INTER_LINEAR); 139 | cv::cvtColor(img, gray, CV_BGR2GRAY); 140 | 141 | ravel(spatial_hls, dst); 142 | ravel(spatial_rgb, dst + 192); 143 | lite_hog(gray, dst+192*2, approx_mode); 144 | } -------------------------------------------------------------------------------- /cpp/frametest/frame.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/cpp/frametest/frame.png -------------------------------------------------------------------------------- /cpp/frametest/frametest.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | 7 | 8 | #include 9 | #include 10 | #include 11 | #include 12 | 13 | #include "feature.h" 14 | #include "weights.h" 15 | 16 | using namespace std; 17 | using namespace cv; 18 | 19 | double predict(int y, int x, double* dst){ 20 | double dotprodcut = 0; 21 | for(int i = 0; i < FEATURESIZE; i++) dotprodcut += dst[i] * unscaled_weight[i]; 22 | double svm_output = dotprodcut + unscaled_bias; 23 | double proba = 1.0/(1.0+exp(-1*svm_output)); 24 | return proba; 25 | } 26 | 27 | int main(void){ 28 | 29 | cv::Mat img = cv::imread("frame.png"); 30 | cv::Mat frame_copy = img.clone(); 31 | std::chrono::system_clock::time_point t1, t2; 32 | t1 = std::chrono::system_clock::now(); 33 | 34 | cv::Mat gray; 35 | // cv::cvtColor(img, gray, CV_BGR2GRAY); 36 | double max_proba = -1; 37 | int max_sy, max_sx; 38 | cv::Mat max_proba_img; 39 | int hitcnt = 0; 40 | int cnt = 0; 41 | for(int y = 0; y <= 240 - 32; y+=8){ 42 | for(int x = 0; x <= 320 - 64; x+=8){ 43 | cnt++; 44 | double dst[FEATURESIZE]; 45 | for(int i = 0; i < FEATURESIZE; i++) dst[i] = 0; 46 | // Mat window_gray_img = gray(cv::Rect(x, y, 64, 32)); 47 | Mat window_bgr_img = img(cv::Rect(x, y, 64, 32)); 48 | getFeature(window_bgr_img, dst, true); 49 | double proba = predict(y, x, dst); 50 | // cout << rst << endl; 51 | if(max_proba < proba){ 52 | max_proba = proba; 53 | max_sx = x; 54 | max_sy = y; 55 | max_proba_img = window_bgr_img.clone(); 56 | } 57 | //cout << proba << endl; 58 | if(proba > 0.80){ 59 | //cout << hitcnt << " " << y << " " << x << " " << proba << endl; 60 | rectangle(frame_copy, Point(x, y), Point(x + 64, y + 32), Scalar(0,0,200), 2); //x,y //Scaler = B,G,R 61 | cv::putText(frame_copy, to_string(proba), cv::Point(x+5,y+5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(255,0,0), 1, CV_AA); 62 | 63 | hitcnt++; 64 | 65 | cv::imwrite("hit/hit_" + to_string(hitcnt) + ".png", window_bgr_img); 66 | } 67 | } 68 | } 69 | cout << cnt << endl; 70 | // getFeature(img, dst, false); 71 | // double rst = predict(dst); 72 | // cout << fixed << setprecision(10) << max_proba << endl; 73 | // cout << fixed << max_sy << " " << max_sx << endl; 74 | // cv::imshow("result", frame_copy); 75 | // cv::imwrite("result.png", frame_copy); 76 | t2 = std::chrono::system_clock::now(); 77 | double elapsed = (double)std::chrono::duration_cast(t2-t1).count(); 78 | 79 | cout << "elapsed:" << elapsed << "[milisec]" << endl; 80 | cout << "fps:" << 1000.0/elapsed << "[fps]" << endl; 81 | // cv::waitKey(0); 82 | } 83 | -------------------------------------------------------------------------------- /cpp/frametest/weights.h: -------------------------------------------------------------------------------- 1 | double unscaled_weight[1140] = { 2 | -0.060135617, -0.0057954398, 0.045693831, 0.016413542, -0.058851423, 0.041404777, 0.095144363, 0.026578408, -0.087437889, -0.067619527, 3 | -0.024569344, -0.06627456, 0.2208882, -0.11555182, -0.11210355, 0.012134506, 0.0073816193, 0.02014449, 0.013669401, -0.034089829, 4 | 0.018874543, -0.16775158, -0.061615645, -0.0094228159, 0.014176663, -0.020922399, -0.0002247474, -0.15501695, 0.034271714, -0.020733171, 5 | -0.075410699, -0.031585536, 0.0027388054, 0.24269586, -0.052392697, -0.063675106, 0.022049254, -0.094687538, 0.0081115005, -0.020468343, 6 | 0.0094195257, 0.13601282, -0.087035301, 0.26586309, 0.10081141, -0.089664147, -0.081292588, 0.094105199, -0.098959813, 0.012429139, 7 | -0.11499898, 0.30675831, -0.38633805, -0.087812077, -0.15027921, -0.17136537, -0.0061138071, 0.019405064, -0.33082818, 0.028331688, 8 | 0.090760739, -0.085195202, -0.075823232, -0.074736315, -0.089295404, 0.10387837, -0.065125323, 0.13618468, 0.14969925, -0.030108807, 9 | 0.053016641, 0.062173862, 0.025986987, -0.07308242, -0.032996054, 0.44607121, -0.30489175, -0.14105135, 0.017821448, -0.47546276, 10 | 0.11685057, 0.040354345, -0.21530378, 0.097384141, 0.16632161, -0.12007745, 0.006137299, -0.067712798, -0.048504531, 0.22217895, 11 | 0.049767602, 0.26928249, 0.33303365, 0.027215983, 0.012851815, 0.011090834, 0.18718466, 0.11326366, -0.074230666, 0.019707286, 12 | -0.34903343, -0.13006236, 0.026575514, -0.46369279, -0.12458107, 0.20798129, -0.10197877, 0.0026801587, 0.050638248, -0.072497724, 13 | 0.041808197, 0.012400063, -0.069311902, 0.15147505, 0.017371309, 0.31829167, 0.28718122, -0.048210545, -0.0072576521, 0.012216123, 14 | -0.022380964, 0.066214183, 0.003424697, 0.10587493, -0.32617923, -0.14700754, -0.018866309, -0.28001023, -0.079662353, 0.17034666, 15 | -0.0076032748, 0.082642996, 0.12740345, 0.084443232, 0.061418434, -0.061118061, -0.041095958, 0.069190215, 0.0067587872, 0.08600845, 16 | 0.21424869, 0.12094091, -0.081178912, -0.031897812, 0.072095734, 0.10119302, -0.062292027, -0.03319446, -0.22837144, -0.051515788, 17 | 0.076026888, -0.23622804, 0.07048171, 0.018197457, -0.17569889, 0.2456675, -0.068465514, 0.14577799, 0.22237007, 0.10034971, 18 | 0.076866278, 0.091166362, 0.04315965, 0.025400063, 0.095350066, 0.099975616, -0.081606623, -0.035478411, 0.2069061, -0.018539502, 19 | 0.070563727, 0.29284157, -0.082546928, 0.10632353, 0.16980832, -0.16099992, -0.0048784595, -0.0095195711, 0.066546515, -0.031592207, 20 | 0.088973383, -0.11510229, 0.026395684, 0.15694182, -0.039984733, 0.052622537, 0.051549395, -0.038927671, 0.082586441, 0.26376159, 21 | -0.099207183, 0.0089791167, 0.071718994, 0.064503939, 0.049915915, 0.055734367, 0.058150857, 0.064533208, -0.073816283, -0.055448193, 22 | -0.07229727, -0.035064126, -0.047196167, -0.028713163, -0.042975881, -0.065431776, -0.084916735, -0.017618273, -0.034819028, 0.048360994, 23 | -0.097129977, -0.11122496, 0.041568736, -0.079507798, -0.069853619, 0.013194104, 0.042868621, 0.040634134, -0.018644412, -0.0039340655, 24 | 0.021400148, -0.0079983548, 0.028292341, 0.0056756612, 0.016937914, -0.023523476, -0.070046004, -0.077116125, 0.074123942, -0.024569983, 25 | 0.05172077, 0.044054233, -0.0059379858, 0.18340005, -0.24203939, -0.24768132, 0.14076899, -0.071780535, -0.09996613, 0.15101906, 26 | -0.11041869, -0.09815847, -0.077346101, -0.046618739, -0.067633451, 0.080608553, -0.070320048, -0.03893961, 0.086197884, 0.053513054, 27 | -0.044465559, 0.071199187, -0.18226779, -0.33370621, 0.0019465214, 0.0077182065, -0.075175455, 0.20019577, -0.070052181, -0.077025977, 28 | 0.20142407, -0.049255343, -0.080671093, 0.078112826, -0.048614712, -0.025710276, 0.034636939, -0.097567002, -0.12218862, 0.13725474, 29 | 0.095986339, 0.073690979, 0.32334103, 0.077334259, -0.034708884, 0.17198275, 0.056617119, -0.24650847, 0.059405688, 0.080569042, 30 | -0.023552449, 0.3157757, -0.049622295, -0.04694741, 0.41691862, -0.085572701, -0.096671728, 0.060310682, -0.072593932, -0.096998959, 31 | -0.062874926, -0.1035657, -0.11729924, 0.087523887, -0.10569357, -0.12462082, 0.010957146, 0.031505559, -0.17373285, 0.0050550884, 32 | 0.03361039, -0.3404327, 0.029395012, 0.089217806, -0.060157034, 0.21102903, -0.039471127, -0.061019776, 0.35085927, -0.058006675, 33 | -0.11456846, 0.051004388, 0.022822894, -0.023201339, -0.0068807958, -0.08735551, -0.088614547, -0.046650871, -0.051226604, -0.13139576, 34 | -0.089534008, 0.028357528, -0.15173333, 0.028837835, -0.20314345, -0.50544272, 0.054633277, -0.0055018313, -0.15247298, 0.078817873, 35 | 0.029866847, -0.063360154, 0.23003345, 0.00078410297, -0.060846131, 0.0028142183, -0.036997184, -0.094242207, -0.084144355, 0.037710992, 36 | -0.016576658, -0.10779956, 0.14839454, 0.012394632, 0.05448209, 0.25556456, 0.095196168, 0.20576199, 0.015041904, -0.12703546, 37 | 0.16729519, -0.075982298, -0.24046561, 0.067984485, 0.037242005, -0.05703905, 0.1019084, -0.060293562, -0.13187027, -0.020311943, 38 | 0.080877661, 0.041762805, 0.012762562, 0.12903533, 0.08106361, 0.066770251, 0.073078431, 0.0056474465, -0.040541959, -0.0054410243, 39 | -0.033948896, -0.049546994, -0.00022436104, -0.075669782, 0.0083089351, -0.060536387, -0.16373889, 0.063258835, 0.015281203, -0.064744215, 40 | 0.064432176, -0.03306017, -0.090470313, -0.0037354867, -0.35628605, -0.31723602, 0.11909434, 0.26828443, -0.00078841494, -0.38156945, 41 | -0.49419935, -0.88879534, 0.45275036, 0.047676167, 0.63220562, -0.14846863, 0.32486898, 0.027885187, 0.94127472, -0.33675227, 42 | 0.064137457, 0.58508165, -0.32647172, -0.24829488, 0.26972277, -0.69145909, -0.25876666, 0.69509991, 0.4037626, -0.6370146, 43 | 0.98162768, -0.025106112, -0.18832362, 0.1672017, -0.035666887, 0.27526755, 0.98384672, -0.50747695, -0.003760215, 0.45541522, 44 | -0.5585181, 0.25701913, -0.16183018, 0.10374813, -0.087122587, 0.40343766, -0.36883378, -0.081259759, 0.95012746, -0.72364728, 45 | -0.87658884, 0.62984685, 0.23406394, -0.40177671, -0.18915054, 0.11823205, -0.13497879, 0.46497862, -0.192562, 0.13649086, 46 | 0.47857159, -0.57159943, -0.1556469, -0.52531559, -1.1442471, -0.39161618, -0.11208798, 0.79157913, 0.034546319, 0.14706447, 47 | 0.24867948, 0.26378595, 0.19061987, 0.97217715, 0.086423045, 0.4042792, -0.59510323, -1.2386396, 0.36873645, 0.6490909, 48 | -0.20647917, 0.24350999, 0.69333016, 0.19490234, 1.1449012, -0.38053561, 0.46310943, 0.91551664, -0.069686592, -0.59808565, 49 | -0.46342137, 0.158642, -0.39723732, -0.47622094, -0.19930833, -0.14492569, 0.11454941, 0.011674376, -0.1861944, 0.064081006, 50 | 0.53241694, -0.0054759592, 0.15043603, 0.17323742, -0.083523774, -0.25824045, -0.55869312, 0.13993466, 1.0391054, 0.20708796, 51 | 0.018232401, 0.17974352, -0.56099621, -0.0091533217, 0.79787385, 0.10424282, -0.44138978, 0.078206729, 0.29721484, -0.064392584, 52 | 0.13553823, 0.26650536, 0.040570009, -0.13694993, 0.75585689, -0.32148036, -0.0052724929, 0.23556664, -0.23043319, -0.12166779, 53 | 0.32672508, -0.043271061, -0.24523792, -0.09085898, 0.3088755, 0.84012745, -0.13213067, -0.17679964, 0.076349265, -0.03634353, 54 | -0.27169228, -0.40541379, -0.26494376, -0.11884931, 0.2185186, 0.39512794, 0.072717117, 0.43627201, -0.28414061, -0.086069092, 55 | 0.1110596, 0.84718159, -0.32000967, -0.99566885, -0.0441068, -0.24436503, 0.33564579, -0.16491565, 0.1020262, -0.80421341, 56 | 0.031782211, 0.25180233, 0.47267213, 1.1721252, -0.06532109, 0.31437444, 0.41546063, 0.55608725, 0.1837418, 0.24635306, 57 | -0.42262117, 0.066454046, -0.021011395, -0.05352981, 0.055685919, -0.17689983, -0.21368124, 0.37897596, -0.62666347, -0.15387333, 58 | 0.03660154, 0.89552889, 0.30703572, 0.0054022621, -0.18443497, 0.1856669, -1.0001259, -0.55264684, 0.022059506, 0.41295707, 59 | 1.5034193, -0.73607218, 0.18666054, -0.83123966, -0.030461499, -0.014661977, 0.060309653, -0.48469268, -0.64976144, -0.16205418, 60 | -0.31389536, 0.31808545, 0.1950983, 0.39640251, 0.77843246, -0.079123361, 0.11800785, -0.035723147, -0.077139098, 0.25816343, 61 | 0.32354955, 0.32889905, -0.29439932, 0.35570851, -0.33424168, -0.026530224, 0.6755207, 0.45843613, -0.15789671, 0.1658087, 62 | -1.1017868, 0.89650351, -0.22954675, -0.24350046, -0.37779956, -0.54273152, 0.081709816, -0.24776463, 0.27432617, -1.365597, 63 | -0.54966142, -0.26263329, -1.4774131, -0.21450403, -0.87087689, -0.053930217, -0.78178804, 1.299377, 0.17615701, -0.069472118, 64 | 1.1904555, -0.050161087, -0.0036415223, 0.19373445, 0.16175336, -0.044959099, 0.024744358, 0.98055107, -0.50911825, -0.48670208, 65 | -1.3667961, 0.35244499, 0.67547635, 0.65431733, -0.037364327, -0.038149865, -0.55923397, 0.12196405, 0.14219757, -0.12418675, 66 | -0.4567762, 1.8185091, 0.23359701, -0.53531694, 0.63030772, -0.29743613, 0.26412697, 0.56982846, -0.63559352, -0.39737215, 67 | 0.22325785, -0.74040839, -0.65804861, -0.31517078, -0.23894115, 0.020620429, -0.042327487, -0.062556244, 0.0066368059, 0.19282915, 68 | 0.22607052, 0.93131532, 0.45099527, 0.13267173, 2.1787398, 0.065072117, -0.24562835, -0.17747741, 0.62439932, 1.0975483, 69 | -0.018493087, 0.38702695, -0.40011818, 0.23143872, 0.44158637, -0.61500341, -0.27611845, 1.2966113, -0.1939877, -0.1117886, 70 | -0.011483012, 0.19559564, -0.17646146, -0.13520051, -0.23098717, -0.1324596, 0.011399135, 0.62604599, -0.37839075, -0.25730446, 71 | 0.35907056, 0.56515773, 0.29494222, -0.0076186182, 0.12654192, 0.84036363, -0.028360462, -0.1870663, 0.27303282, 0.21514311, 72 | -0.2303612, -0.36606286, 0.38862597, -0.13700299, 0.25711256, 0.51367461, -0.43655661, 0.73049667, 0.33891319, 0.31605389, 73 | 0.32908352, -0.46599634, -0.21892778, 0.21154124, 0.94270983, -0.10265328, -0.18497555, 0.16611285, 0.017939857, 0.10120931, 74 | -0.14908316, 0.097154704, 0.57311627, -0.66707884, -0.49862741, -0.13404251, -0.19257854, -0.14827768, -0.53525397, 0.4888086, 75 | 0.033704089, 0.22076222, 0.30930584, -0.088527393, 0.47273434, -0.26233062, 0.042947313, -0.78235855, -0.38148974, -0.068184797, 76 | 0.40592323, -0.65737194, 0.18576291, 0.16061569, 0.31900274, -0.15451525, -0.10052602, -0.015456316, 0.44983382, 1.4612943, 77 | 0.17578508, -0.17981669, 0.10287496, 0.058901074, 0.19006627, 0.29506949, -0.46961307, -0.18971922, -0.054759415, 0.17778768, 78 | -0.097020925, 0.44917938, 0.023700301, 0.26753414, 0.19427756, 0.17030396, -0.0088988566, 0.021900862, -1.3733445, -0.15965934, 79 | -0.4627457, 0.086282055, 0.20288655, -0.86989341, -0.42176656, 0.096852924, -0.10501101, -0.068006894, -0.20324846, -0.091520928, 80 | 0.61018761, 0.24748878, -0.24401933, -0.4516992, -0.4193946, 0.32843033, 0.10515609, -0.21918803, -0.26802576, 0.5271809, 81 | 0.090310411, 1.0085832, -0.17618316, 0.0097166103, 0.09308949, 0.0098074179, -0.11954438, 0.17842319, 0.31071036, 0.63986373, 82 | -1.0796895, -0.85036219, -0.39493146, -0.58460099, -0.45772657, -1.162359, -0.82454993, -0.20609036, -0.63906787, -0.95910345, 83 | 0.012708205, 0.41098276, 0.88266648, 0.28954448, 0.6737733, 0.67579923, 0.19998924, 0.015447355, 0.20091239, -0.72959952, 84 | -0.086891327, 0.25655318, -0.20683647, -0.56227185, 0.025543171, 0.32266058, 0.084314556, 1.0032744, -0.046329233, -0.11432419, 85 | -0.37443717, -0.20328957, -0.41479811, 0.17213616, -0.10023693, -0.30080046, -1.0583221, -0.56752432, -0.27048311, 0.45310139, 86 | -0.18042397, -0.69442872, -0.68019183, 0.060848034, -0.40224798, 1.358583, -0.072071326, -0.067369866, 0.21870928, 0.72634729, 87 | 0.34179642, 0.44788299, 0.04873005, 0.015144522, 0.9077817, -0.14978403, -0.020191758, 0.026462054, 0.2216188, -0.18695989, 88 | 0.088706873, 0.42077891, -0.4191944, -0.8785035, -1.4882633, 0.21039137, 0.59521808, 0.42056525, -0.29475477, 0.33775373, 89 | 0.75214033, 0.24854402, 1.3904045, -0.30485415, -0.29491475, 0.1346705, -0.035948049, -0.30094726, -0.4800212, 0.42589988, 90 | -0.2317294, -0.31854752, -1.5078233, -0.3382337, 0.30592648, 0.76163652, -1.1870473, -0.48522768, 0.22870649, -0.16309247, 91 | -0.22528821, -0.042190452, 0.13708076, 1.055269, 1.1985923, 1.2824879, 0.52225023, -0.14709748, -1.1109281, -0.19255232, 92 | -0.39334986, -0.26066996, 0.6598281, 0.44290436, 0.17800764, -0.38264229, -0.58305087, -0.73800375, -1.0014849, 1.034972, 93 | -0.46158508, -0.12637337, 0.82976176, 0.64108275, 1.944417, -1.1193663, -0.10945977, 0.058455966, -0.63828672, -0.6673701, 94 | -0.70730893, 0.20348789, 0.1012614, 0.13947098, 0.0332865, -0.36240611, -0.074024367, -0.34746846, -0.15733946, 1.2487439, 95 | 0.33747726, 0.24308943, -0.0057647031, 0.56555161, 0.67434167, 0.10088683, 0.8883937, -0.18731089, -0.2744646, 0.17609208, 96 | -0.82984469, 0.70376197, -0.84186467, 0.38813648, 0.6185403, -0.061989287, -0.17076365, -0.25016817, 0.14651057, 0.15663918, 97 | 0.14477473, -0.37042581, -0.5727373, -0.22194095, 0.061461184, -0.042919164, -0.99564308, 0.76191547, -0.25164804, 0.62857499, 98 | 0.045846822, 0.096232792, -0.46860609, -0.0085367112, 0.019068989, 1.113625, 1.9706783, -0.16275578, 0.77381412, 0.062079073, 99 | 0.90192478, -0.29846436, -0.38175043, -0.11705119, -0.23472752, -0.95151094, -0.55648705, -0.24711977, -0.37847214, -0.65382812, 100 | 0.16015243, -0.017593383, -0.054238873, -0.4882071, 1.0528135, -0.28290995, 0.32789963, -0.31529872, -0.12969364, -0.74517397, 101 | 0.61871797, 0.028015511, -1.1753765, -0.80780098, -0.052342662, 0.57466979, 0.45031528, 1.1578884, 0.26048148, -0.0067441782, 102 | 0.32892812, 0.63737881, -0.13451255, 0.1872742, 0.10361034, 0.24119859, 0.93525243, -1.1309768, -0.061778545, 0.0020555274, 103 | -0.090712462, -0.017604833, -0.91762442, 0.051235305, -0.22902313, 0.50147349, -0.64185563, 1.3593889, -0.050604172, -1.0527692, 104 | -0.96829347, -0.070859199, 0.25759182, -0.83655189, -0.62944983, -2.3800076, 0.70422535, -0.10630678, -0.86673173, -0.014763052, 105 | 1.2635276, 0.25123897, 0.048214172, 0.17932358, -1.0315034, -0.40817134, -0.3252153, -0.13154578, 0.20412687, -0.93575768, 106 | -0.2767996, 0.030497214, 0.85208179, 0.51234637, 1.088986, 0.27097953, 0.30552677, -0.19718352, -0.18311434, 0.34689242, 107 | -1.0520858, -0.93125179, -2.9569899, 0.5248053, -0.25797235, -0.76633766, -0.38707186, 0.59839951, 0.045898331, -0.55004767, 108 | -0.42547579, 0.63107999, 1.3677458, 0.23236951, 0.44788387, 0.46034632, 0.48762295, 0.30590682, -0.038059495, 0.41426364, 109 | 0.061508599, -0.16567374, -0.022857681, 0.41256005, 0.16467914, -0.15588478, 0.091723617, 0.21010443, 0.28991467, 1.3706905, 110 | 0.37200217, 0.059796672, -0.40075562, -0.10205151, 0.31096087, 0.49200834, -0.91923485, -0.98004874, -0.1930938, 0.61270648, 111 | 0.038662338, -0.41327564, -0.35870347, 0.04555703, 0.26374439, -1.2482992, -1.8092711, 0.12049751, -0.64772112, -0.23011708, 112 | -0.99753729, -0.064456055, 0.37197866, 0.2737506, 0.61109388, 0.44618125, 0.57090875, 0.10161303, 0.08439814, 0.19360304, 113 | 0.0951574, 0.72928679, 0.26855731, 0.24202178, 0.86022675, 0.48056215, -0.25633564, -0.015808993, -0.49415717, -0.30119794, 114 | -0.15255799, -0.14173975, -0.73347561, -0.51544755, 0.86042285, -0.1712932, -0.32820265, -0.99832144, -0.51673697, 0.091211012, 115 | 0.18406492, -0.95238815, 0.40699132, 0.2335278, -0.5641072, -0.79639213, -1.9764537, -3.2774365, -1.6896813, -0.23660068 116 | }; 117 | double unscaled_bias = -1.9456217; -------------------------------------------------------------------------------- /cpp/hogtest/hogtest.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | using namespace std; 12 | using namespace cv; 13 | 14 | #define HOG_WIDTH 64 15 | #define HOG_HEIGHT 32 16 | #define CELL_SIZE_ROW 8 17 | #define CELL_SIZE_COL 8 18 | #define BLOCK_SIZE_ROW 2 19 | #define BLOCK_SIZE_COL 2 20 | #define N_CELLS_ROW HOG_HEIGHT/CELL_SIZE_ROW //4 21 | #define N_CELLS_COL HOG_WIDTH/CELL_SIZE_COL //8 22 | #define N_BLOCK_ROW (N_CELLS_ROW - 1) //3 23 | #define N_BLOCK_COL (N_CELLS_COL - 1) //7 24 | #define HISTOGRAMSIZE N_BLOCK_ROW * N_BLOCK_COL * BLOCK_SIZE_ROW * BLOCK_SIZE_COL * 9 //3*7*2*2*9 25 | 26 | void lite_hog(cv::Mat img, double* dst, bool approx_mode); 27 | 28 | int main(void){ 29 | cv::Mat img = cv::imread("input.png"); 30 | cv::Mat gray; 31 | cv::cvtColor(img, gray, CV_BGR2GRAY); 32 | double dst[HISTOGRAMSIZE]; 33 | for(int i = 0; i < HISTOGRAMSIZE; i++) dst[i] = 0; 34 | lite_hog(gray, dst, false); 35 | /*for(int i = 0; i < HISTOGRAMSIZE; i++){ 36 | cout << fixed << setprecision(9) << dst[i] << " "; 37 | if(i%9 == 8) cout << endl; 38 | }*/ 39 | } 40 | 41 | inline int approx_distance(int dx, int dy){ 42 | int min, max; //uint 43 | if(dx < 0) dx = -dx; 44 | if(dy < 0) dy = -dy; 45 | 46 | if(dx < dy){ 47 | min = dx; 48 | max = dy; 49 | }else{ 50 | min = dy; 51 | max = dx; 52 | } 53 | 54 | //coefficients equivalent to (123/128 * max) and (51/128*min) 55 | return ((( max << 8 ) + ( max << 3 ) - ( max << 4 ) - ( max << 1 ) + 56 | ( min << 7 ) - ( min << 5 ) + ( min << 3 ) - ( min << 1 )) >> 8 ); 57 | } 58 | 59 | void lite_hog(cv::Mat img, double* dst, bool approx_mode){ 60 | double descriptor[HISTOGRAMSIZE]={0}; 61 | double sum_of_block[N_BLOCK_ROW * N_BLOCK_COL]={0}; 62 | for(int i = 0; i < HISTOGRAMSIZE; i++) descriptor[i] = 0; 63 | for(int i = 0; i < N_BLOCK_ROW * N_BLOCK_COL; i++) sum_of_block[i] = 0; 64 | int lut0[256] = {0,0,0,1,1,1,2,2,2,3,3,3,4,4,5,5,5,6,6,6,7,7,7,8,8,9,9,9,10,10,10,11,11,11,12,12,13,13,13,14,14,14,15,15,15,16,16,17,17,17,18,18,18,19,19,19,20,20,21,21,21,22,22,22,23,23,23,24,24,25,25,25,26,26,26,27,27,27,28,28,29,29,29,30,30,30,31,31,31,32,32,33,33,33,34,34,34,35,35,35,36,36,37,37,37,38,38,38,39,39,39,40,40,41,41,41,42,42,42,43,43,43,44,44,45,45,45,46,46,46,47,47,47,48,48,49,49,49,50,50,50,51,51,51,52,52,53,53,53,54,54,54,55,55,55,56,56,57,57,57,58,58,58,59,59,59,60,60,61,61,61,62,62,62,63,63,63,64,64,65,65,65,66,66,66,67,67,67,68,68,69,69,69,70,70,70,71,71,71,72,72,73,73,73,74,74,74,75,75,75,76,76,77,77,77,78,78,78,79,79,79,80,80,81,81,81,82,82,82,83,83,83,84,84,85,85,85,86,86,86,87,87,87,88,88,89,89,89,90,90,90,91,91,91,92,92}; 65 | int lut1[256] = {0,0,1,2,3,4,5,5,6,7,8,9,10,10,11,12,13,14,15,15,16,17,18,19,20,20,21,22,23,24,25,26,26,27,28,29,30,31,31,32,33,34,35,36,36,37,38,39,40,41,41,42,43,44,45,46,46,47,48,49,50,51,52,52,53,54,55,56,57,57,58,59,60,61,62,62,63,64,65,66,67,67,68,69,70,71,72,72,73,74,75,76,77,78,78,79,80,81,82,83,83,84,85,86,87,88,88,89,90,91,92,93,93,94,95,96,97,98,98,99,100,101,102,103,104,104,105,106,107,108,109,109,110,111,112,113,114,114,115,116,117,118,119,119,120,121,122,123,124,124,125,126,127,128,129,130,130,131,132,133,134,135,135,136,137,138,139,140,140,141,142,143,144,145,145,146,147,148,149,150,150,151,152,153,154,155,156,156,157,158,159,160,161,161,162,163,164,165,166,166,167,168,169,170,171,171,172,173,174,175,176,177,177,178,179,180,181,182,182,183,184,185,186,187,187,188,189,190,191,192,192,193,194,195,196,197,197,198,199,200,201,202,203,203,204,205,206,207,208,208,209,210,211,212,213,213}; 66 | int lut2[256] = {0,1,3,5,6,8,10,12,13,15,17,19,20,22,24,25,27,29,31,32,34,36,38,39,41,43,44,46,48,50,51,53,55,57,58,60,62,63,65,67,69,70,72,74,76,77,79,81,83,84,86,88,89,91,93,95,96,98,100,102,103,105,107,108,110,112,114,115,117,119,121,122,124,126,127,129,131,133,134,136,138,140,141,143,145,147,148,150,152,153,155,157,159,160,162,164,166,167,169,171,172,174,176,178,179,181,183,185,186,188,190,191,193,195,197,198,200,202,204,205,207,209,210,212,214,216,217,219,221,223,224,226,228,230,231,233,235,236,238,240,242,243,245,247,249,250,252,254,255,257,259,261,262,264,266,268,269,271,273,274,276,278,280,281,283,285,287,288,290,292,294,295,297,299,300,302,304,306,307,309,311,313,314,316,318,319,321,323,325,326,328,330,332,333,335,337,338,340,342,344,345,347,349,351,352,354,356,358,359,361,363,364,366,368,370,371,373,375,377,378,380,382,383,385,387,389,390,392,394,396,397,399,401,402,404,406,408,409,411,413,415,416,418,420,421,423,425,427,428,430,432,434,435,437,439,441}; 67 | int lut3[256] = {0,5,11,17,22,28,34,39,45,51,56,62,68,73,79,85,90,96,102,107,113,119,124,130,136,141,147,153,158,164,170,175,181,187,192,198,204,209,215,221,226,232,238,243,249,255,260,266,272,277,283,289,294,300,306,311,317,323,328,334,340,345,351,357,362,368,374,379,385,391,396,402,408,413,419,425,430,436,442,447,453,459,464,470,476,481,487,493,498,504,510,515,521,527,532,538,544,549,555,561,566,572,578,584,589,595,601,606,612,618,623,629,635,640,646,652,657,663,669,674,680,686,691,697,703,708,714,720,725,731,737,742,748,754,759,765,771,776,782,788,793,799,805,810,816,822,827,833,839,844,850,856,861,867,873,878,884,890,895,901,907,912,918,924,929,935,941,946,952,958,963,969,975,980,986,992,997,1003,1009,1014,1020,1026,1031,1037,1043,1048,1054,1060,1065,1071,1077,1082,1088,1094,1099,1105,1111,1116,1122,1128,1133,1139,1145,1150,1156,1162,1168,1173,1179,1185,1190,1196,1202,1207,1213,1219,1224,1230,1236,1241,1247,1253,1258,1264,1270,1275,1281,1287,1292,1298,1304,1309,1315,1321,1326,1332,1338,1343,1349,1355,1360,1366,1372,1377,1383,1389,1394,1400,1406,1411,1417,1423,1428,1434,1440,1445}; 68 | 69 | for(int y = 0; y < N_BLOCK_ROW; y++){//3 70 | for(int x = 0; x < N_BLOCK_COL; x++){//7 71 | for(int yy = 0; yy < CELL_SIZE_ROW*BLOCK_SIZE_ROW; yy++){//16 72 | for(int xx = 0; xx < CELL_SIZE_COL*BLOCK_SIZE_COL; xx++){//16 73 | int ny = CELL_SIZE_ROW * y + yy; 74 | int nx = CELL_SIZE_COL * x + xx; 75 | //compute gradient 76 | int mag = 0; 77 | int bin_index; 78 | if(ny == 0 || nx == 0 || ny == 31 || nx == 63){ 79 | mag = 0; 80 | bin_index = 0; 81 | }else{ 82 | int Gx = int(img.ptr(ny)[nx+1]) - int(img.ptr(ny)[nx-1]); 83 | int Gy = int(img.ptr(ny+1)[nx]) - int(img.ptr(ny-1)[nx]); 84 | mag = (approx_mode ? (approx_distance(Gx, Gy)) : sqrt((double)(Gx*Gx + Gy * Gy))); 85 | if((Gx >= 0 && Gy >= 0) || (Gx <= 0 && Gy <= 0)){ 86 | if (abs(Gy) < lut0[abs(Gx)]) {bin_index = 0;} 87 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 1;} 88 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 2;} 89 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 3;} 90 | else {bin_index = 4;} 91 | } else{ 92 | if (abs(Gy) < lut0[abs(Gx)]){bin_index = 4;} 93 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 5;} 94 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 6;} 95 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 7;} 96 | else {bin_index = 8;} 97 | } 98 | } 99 | int cell_index = (yy / 8) * 2 + (xx / 8); 100 | int block_index = y * N_BLOCK_COL + x; 101 | int descrip_index = block_index * 36 + cell_index * 9 + bin_index; 102 | descriptor[descrip_index] += mag; 103 | sum_of_block[block_index] += mag; 104 | } 105 | } 106 | } 107 | } 108 | // int normalized[HISTOGRAMSIZE]={0}; 109 | // for(int i = 0; i < HISTOGRAMSIZE; i++) normalized[i] = 0; 110 | double allsum = 0; 111 | int average; 112 | unsigned int blkSize = 36; 113 | for(int by = 0; by < N_BLOCK_ROW; by++){//3 114 | for(int bx = 0; bx < N_BLOCK_COL; bx++){//7 115 | int blkIdx = by * N_BLOCK_COL + bx; 116 | for (int i = 0; i < 36; i++) { 117 | int index = blkIdx * 36 + i; 118 | if(sum_of_block[blkIdx] == 0){ 119 | dst[index] = 0; 120 | }else{ 121 | dst[index] = sqrt((double)descriptor[index]/(double)sum_of_block[blkIdx]); 122 | } 123 | allsum += dst[index]; 124 | } 125 | } 126 | } 127 | // cout << allsum << endl; 128 | for(int i = 0; i < HISTOGRAMSIZE; i++){ 129 | cout << fixed << setprecision(10) << descriptor[i] << endl; 130 | } 131 | 132 | } -------------------------------------------------------------------------------- /cpp/hogtest/input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/cpp/hogtest/input.png -------------------------------------------------------------------------------- /cpp/hogtest/myhog.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | from skimage.feature import hog 3 | import numpy as np 4 | import math 5 | 6 | def _hog_channel_gradient(channel): 7 | g_row = np.empty(channel.shape, dtype=np.int16) 8 | g_row[0, :] = 0 9 | g_row[-1, :] = 0 10 | g_row[1:-1, :] = channel[2:, :] - channel[:-2, :] 11 | g_col = np.empty(channel.shape, dtype=np.int16) 12 | g_col[:, 0] = 0 13 | g_col[:, -1] = 0 14 | g_col[:, 1:-1] = channel[:, 2:] - channel[:, :-2] 15 | return g_row, g_col 16 | 17 | def approx_distance(dx, dy): 18 | min = 0 19 | max = 0 20 | if(dx < 0): 21 | dx = -dx; 22 | if(dy < 0): 23 | dy = -dy; 24 | 25 | if(dx < dy): 26 | min = dx; 27 | max = dy; 28 | else: 29 | min = dy; 30 | max = dx; 31 | 32 | 33 | return ((( max << 8 ) + ( max << 3 ) - ( max << 4 ) - ( max << 1 ) + 34 | ( min << 7 ) - ( min << 5 ) + ( min << 3 ) - ( min << 1 )) >> 8 ) 35 | 36 | lut0 = [0,0,0,1,1,1,2,2,2,3,3,3,4,4,5,5,5,6,6,6,7,7,7,8,8,9,9,9,10,10,10,11,11,11,12,12,13,13,13,14,14,14,15,15,15,16,16,17,17,17,18,18,18,19,19,19,20,20,21,21,21,22,22,22,23,23,23,24,24,25,25,25,26,26,26,27,27,27,28,28,29,29,29,30,30,30,31,31,31,32,32,33,33,33,34,34,34,35,35,35,36,36,37,37,37,38,38,38,39,39,39,40,40,41,41,41,42,42,42,43,43,43,44,44,45,45,45,46,46,46,47,47,47,48,48,49,49,49,50,50,50,51,51,51,52,52,53,53,53,54,54,54,55,55,55,56,56,57,57,57,58,58,58,59,59,59,60,60,61,61,61,62,62,62,63,63,63,64,64,65,65,65,66,66,66,67,67,67,68,68,69,69,69,70,70,70,71,71,71,72,72,73,73,73,74,74,74,75,75,75,76,76,77,77,77,78,78,78,79,79,79,80,80,81,81,81,82,82,82,83,83,83,84,84,85,85,85,86,86,86,87,87,87,88,88,89,89,89,90,90,90,91,91,91,92,92] 37 | lut1 = [0,0,1,2,3,4,5,5,6,7,8,9,10,10,11,12,13,14,15,15,16,17,18,19,20,20,21,22,23,24,25,26,26,27,28,29,30,31,31,32,33,34,35,36,36,37,38,39,40,41,41,42,43,44,45,46,46,47,48,49,50,51,52,52,53,54,55,56,57,57,58,59,60,61,62,62,63,64,65,66,67,67,68,69,70,71,72,72,73,74,75,76,77,78,78,79,80,81,82,83,83,84,85,86,87,88,88,89,90,91,92,93,93,94,95,96,97,98,98,99,100,101,102,103,104,104,105,106,107,108,109,109,110,111,112,113,114,114,115,116,117,118,119,119,120,121,122,123,124,124,125,126,127,128,129,130,130,131,132,133,134,135,135,136,137,138,139,140,140,141,142,143,144,145,145,146,147,148,149,150,150,151,152,153,154,155,156,156,157,158,159,160,161,161,162,163,164,165,166,166,167,168,169,170,171,171,172,173,174,175,176,177,177,178,179,180,181,182,182,183,184,185,186,187,187,188,189,190,191,192,192,193,194,195,196,197,197,198,199,200,201,202,203,203,204,205,206,207,208,208,209,210,211,212,213,213] 38 | lut2 = [0,1,3,5,6,8,10,12,13,15,17,19,20,22,24,25,27,29,31,32,34,36,38,39,41,43,44,46,48,50,51,53,55,57,58,60,62,63,65,67,69,70,72,74,76,77,79,81,83,84,86,88,89,91,93,95,96,98,100,102,103,105,107,108,110,112,114,115,117,119,121,122,124,126,127,129,131,133,134,136,138,140,141,143,145,147,148,150,152,153,155,157,159,160,162,164,166,167,169,171,172,174,176,178,179,181,183,185,186,188,190,191,193,195,197,198,200,202,204,205,207,209,210,212,214,216,217,219,221,223,224,226,228,230,231,233,235,236,238,240,242,243,245,247,249,250,252,254,255,257,259,261,262,264,266,268,269,271,273,274,276,278,280,281,283,285,287,288,290,292,294,295,297,299,300,302,304,306,307,309,311,313,314,316,318,319,321,323,325,326,328,330,332,333,335,337,338,340,342,344,345,347,349,351,352,354,356,358,359,361,363,364,366,368,370,371,373,375,377,378,380,382,383,385,387,389,390,392,394,396,397,399,401,402,404,406,408,409,411,413,415,416,418,420,421,423,425,427,428,430,432,434,435,437,439,441] 39 | lut3 = [0,5,11,17,22,28,34,39,45,51,56,62,68,73,79,85,90,96,102,107,113,119,124,130,136,141,147,153,158,164,170,175,181,187,192,198,204,209,215,221,226,232,238,243,249,255,260,266,272,277,283,289,294,300,306,311,317,323,328,334,340,345,351,357,362,368,374,379,385,391,396,402,408,413,419,425,430,436,442,447,453,459,464,470,476,481,487,493,498,504,510,515,521,527,532,538,544,549,555,561,566,572,578,584,589,595,601,606,612,618,623,629,635,640,646,652,657,663,669,674,680,686,691,697,703,708,714,720,725,731,737,742,748,754,759,765,771,776,782,788,793,799,805,810,816,822,827,833,839,844,850,856,861,867,873,878,884,890,895,901,907,912,918,924,929,935,941,946,952,958,963,969,975,980,986,992,997,1003,1009,1014,1020,1026,1031,1037,1043,1048,1054,1060,1065,1071,1077,1082,1088,1094,1099,1105,1111,1116,1122,1128,1133,1139,1145,1150,1156,1162,1168,1173,1179,1185,1190,1196,1202,1207,1213,1219,1224,1230,1236,1241,1247,1253,1258,1264,1270,1275,1281,1287,1292,1298,1304,1309,1315,1321,1326,1332,1338,1343,1349,1355,1360,1366,1372,1377,1383,1389,1394,1400,1406,1411,1417,1423,1428,1434,1440,1445] 40 | 41 | def myhog(image): 42 | sum_of_block = [0] * 21 43 | descriptor = [0] * 21 * 4 * 9 44 | dst = [0] * 21 * 4 * 9 45 | gxlist = [] 46 | 47 | binarray = [0] * (32*64) 48 | magarray = [0] * (32*64) 49 | 50 | image = image.astype(np.int16) 51 | # gy, gx = _hog_channel_gradient(image) 52 | # print(gx) 53 | for y in range(32): 54 | for x in range(64): 55 | mag = 0 56 | bin_index = 0 57 | if y == 0 or y == 31 or x == 0 or x == 63: 58 | bin_index = 0 59 | mag = 0 60 | else: 61 | Gx = image[y][x+1] - image[y][x-1] 62 | Gy = image[y+1][x] - image[y-1][x] 63 | 64 | # mag = math.sqrt(Gx*Gx+Gy*Gy) 65 | # print(Gx*Gx+Gy*Gy) 66 | a = np.array([image[y][x+1], image[y+1][x]]) 67 | b = np.array([image[y][x-1], image[y-1][x]]) 68 | 69 | mag = np.linalg.norm(a-b) 70 | # print(abs(Gx)) 71 | # print(abs(Gy)) 72 | if((Gx >= 0 and Gy >= 0) or (Gx <= 0 and Gy <= 0)): 73 | if (abs(Gy) < lut0[abs(Gx)]): 74 | bin_index = 0 75 | elif (abs(Gy) < lut1[abs(Gx)]): 76 | bin_index = 1 77 | elif (abs(Gy) < lut2[abs(Gx)]): 78 | bin_index = 2 79 | elif (abs(Gy) < lut3[abs(Gx)]): 80 | bin_index = 3 81 | else: 82 | bin_index = 4 83 | else: 84 | if (abs(Gy) < lut0[abs(Gx)]): 85 | bin_index = 4 86 | elif (abs(Gy) < lut1[abs(Gx)]): 87 | bin_index = 5 88 | elif (abs(Gy) < lut2[abs(Gx)]): 89 | bin_index = 6 90 | elif (abs(Gy) < lut3[abs(Gx)]): 91 | bin_index = 7 92 | else: 93 | bin_index = 8 94 | binarray[y*64+x] = bin_index 95 | magarray[y*64+x] = mag 96 | 97 | for y in range(3): 98 | for x in range(7): 99 | for yy in range(16): 100 | for xx in range(16): 101 | ny = 8 * y + yy 102 | nx = 8 * x + xx 103 | bin_index = binarray[ny * 64 + nx] 104 | cell_index = (yy // 8) * 2 + (xx // 8); 105 | block_index = y * 7 + x; 106 | descrip_index = block_index * 36 + cell_index * 9 + bin_index; 107 | 108 | descriptor[descrip_index] += magarray[ny * 64 + nx]; 109 | sum_of_block[block_index] += magarray[ny * 64 + nx]; 110 | 111 | average = 0 112 | blkSize = 36 113 | for by in range(3): 114 | for bx in range(7): 115 | blkIdx = by * 7 + bx 116 | for i in range(36): 117 | index = blkIdx * 36 + i 118 | if sum_of_block[blkIdx] == 0 : 119 | dst[index] = 0 120 | else: 121 | dst[index] = np.sqrt(descriptor[index] / sum_of_block[blkIdx]) 122 | 123 | 124 | return np.array(dst) 125 | 126 | np.set_printoptions(linewidth=100) 127 | img = cv2.imread("input.png") 128 | gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 129 | print(myhog(gray)) -------------------------------------------------------------------------------- /cpp/hogtest/weights.h: -------------------------------------------------------------------------------- 1 | double unscaled_weight[756] = { 2 | -0.27702861, 0.60882872, 0.77489772, -0.57971451, 1.1351337, -0.61708925, -0.037168136, 3 | 0.53348561, 0.65899799, -0.2304792, 0.15729401, -0.38586537, 1.0745021, 0.37144029, 4 | 0.26018033, -0.050108985, 0.50318494, 0.19848787, -0.81186583, 0.090699661, -0.69320837, 5 | 0.3581912, -0.22366082, -0.00046446369, 0.020652174, 0.066666796, 0.72009897, -0.11970606, 6 | 0.16936678, 0.49433372, -0.41093125, -0.25566695, 0.41232198, -0.22603478, -0.30878159, 7 | 0.5752175, 0.025700067, 0.55524942, -0.14299036, 1.1606084, 0.65674398, 0.26819078, 8 | -0.62530728, 0.047398622, -0.1328025, -0.96653118, -0.99550293, 0.029158756, 0.60217548, 9 | 0.1142082, 0.15795023, 0.5264729, 0.58868957, 0.74495914, -0.21356984, 0.18473744, 10 | 0.78924964, -0.4198517, -0.26577343, 0.23037056, -0.22418226, -0.4438898, 0.46172914, 11 | -0.21188348, -0.024238361, 0.5134554, -0.023279752, 0.36486243, 0.97045258, 0.22966675, 12 | -0.098204999, 0.26555553, -0.86914489, -0.68238714, 0.026812079, 0.41011881, 0.057557923, 13 | 0.26800468, 0.24119578, 0.3414049, 0.6902127, -0.28180391, -0.55422329, 0.37444749, 14 | 0.10904583, -0.46401356, 0.49501922, -0.22026527, -0.79844701, 0.24561922, -0.31483978, 15 | -0.20492422, 0.38120321, -0.0136952, 0.13366862, 0.82982646, 0.19849744, -0.43043436, 16 | 0.28016748, 0.38495892, -0.055314627, 0.37917025, -0.12441661, 0.24382124, 0.48965262, 17 | 0.31991869, -0.080944209, 0.42775918, -0.24671259, -0.012490305, 0.59119726, 0.54870899, 18 | -0.14582407, 0.43749451, -0.1957042, -0.47959878, 0.7652754, -0.77971012, 0.068167356, 19 | 0.22010137, -0.28520843, 0.12596349, 0.43234721, -0.18261237, -0.91296345, 0.59792026, 20 | 0.52231893, 0.10873035, 0.23189917, 0.33637669, 0.18480004, 0.37732502, 0.20249588, 21 | -0.0086822237, 0.42405191, 0.14519455, 0.58410151, -0.48353986, 0.33736452, 0.15643579, 22 | 0.56669449, 0.30621269, 0.11954056, 0.35308405, -0.41700489, 0.43662467, 0.01164503, 23 | -0.79628251, 0.20602757, 0.73133087, -0.21961537, -0.84576873, 0.71220664, 0.016172465, 24 | 0.080416432, -0.0014595037, 0.38547646, 0.50455541, -0.070570844, 0.92615698, -0.40507349, 25 | 0.57797506, 0.28772145, 1.1379891, -0.16899795, 0.26234811, -0.047884443, 0.31540193, 26 | -0.09030899, -0.1386055, 0.061466535, 0.30002874, 0.3748508, 1.0799594, -0.178928, 27 | 0.28508135, 0.31442179, 0.69212965, -0.34006707, -0.086685334, -0.27969151, -0.20775641, 28 | -0.11083242, 0.21389093, 0.46324219, -0.51753321, 0.51174946, -0.47594278, 0.56434511, 29 | -0.76210136, -0.55888976, -0.73619471, 0.088034475, -0.75506386, -0.23101256, -0.19797806, 30 | -0.3629989, 0.10285923, 0.22232903, 0.19699354, 1.1235911, 0.04239838, 0.18618405, 31 | 0.20667088, 0.22342196, -0.17847417, -0.084844314, 0.82994545, 0.86637083, 0.79220726, 32 | 0.34035208, 0.87545105, 0.69480061, 0.96528777, 0.51438682, 0.57698045, -0.84012493, 33 | -0.39109452, -0.49328681, 0.075806932, -1.0200621, -0.37038699, -0.13468987, -0.3954281, 34 | -0.095477797, -2.6873869, 1.3384302, 0.42602405, 0.19305884, 0.059088147, 0.51419599, 35 | -0.026306977, -0.59619403, 0.21839868, 0.51190524, 0.86711813, 1.0035063, 0.65906713, 36 | 0.8361062, 0.33776363, 0.86298273, 0.478098, 0.71702999, 0.55562994, -0.42534472, 37 | -0.1631459, -0.6406547, 0.20347042, 0.69544903, 1.7594445, 0.27672169, 0.41676805, 38 | -0.026565011, 0.8750027, 0.21319598, 0.45462325, -0.17950526, -0.18613147, -0.53005273, 39 | -0.52325662, 0.68785114, 0.063024414, 0.64865514, 1.022538, -0.40963255, -0.077097115, 40 | 0.17893047, -0.052994053, -0.46129413, 0.51845943, 0.23440801, -0.15624654, -0.0067885032, 41 | 0.11268131, 0.97924908, 0.17584953, 0.51239629, 0.74171771, 0.09752271, 0.20597678, 42 | 0.39502654, 0.40717249, 0.16875216, -0.3626527, -0.079479627, 0.60823399, -0.1211186, 43 | 0.36074692, -0.081084726, 0.21061735, 0.53397405, -0.70642196, -0.19027283, 0.40136058, 44 | -0.15184119, -0.45998149, 0.33319134, 0.32961565, -0.11731105, 0.14184209, -0.61769355, 45 | 0.74055756, 0.98278503, 0.41457567, -0.25431528, 0.16394702, 0.20218627, 0.089478814, 46 | -0.16113657, -0.34105227, -0.23109162, 0.23156054, 0.61959063, 0.076230109, 0.09632748, 47 | -0.24881096, -0.20862664, 0.039224784, 0.54311276, -0.033572687, 0.055324484, -0.19684684, 48 | -0.40374573, 0.27092075, 0.375982, -0.072703436, 0.36800475, -0.34038849, 0.70672804, 49 | 1.2860142, 0.65728211, -0.42856792, 0.025150893, 0.63502534, 0.1422673, 0.49842448, 50 | -0.38521487, 0.30287861, 0.44065762, 0.28560239, -0.011097929, 0.46351148, -1.0033978, 51 | -0.82056151, -0.77426304, 0.060839325, 0.069293353, 0.34755052, 0.56671999, 0.34718724, 52 | 0.45202454, -0.25124857, -0.30847623, 0.47957693, -0.35061047, 0.21305469, -0.14469981, 53 | 0.33012084, -0.31366777, 0.1174494, 0.90244197, 0.26415254, 0.26486262, 0.21520949, 54 | 0.52106953, 0.89206695, 0.7112813, 0.23528755, 0.27022265, 0.37931878, 0.96805914, 55 | -0.57568438, 0.15180537, 0.32653507, 0.59538149, 0.34913614, 0.0013011896, 0.22434735, 56 | -0.25821259, -0.39824002, 1.0617368, -0.0052754369, 0.16161773, 0.22708221, 0.5766827, 57 | -0.047858989, 0.06390307, -0.44883223, 0.81818747, 0.0087344773, 0.29438361, 0.26548172, 58 | -0.80206375, -0.17507186, 0.19135255, 0.77115767, 0.14376092, 1.3076891, -0.22560989, 59 | 0.03874272, 0.33688028, 0.024086497, -0.11127855, 0.015224355, -0.059854957, 0.74046974, 60 | 0.050317658, 0.71731345, -0.35621582, 0.88144578, 0.34302006, 0.49848935, -0.08149386, 61 | -0.18364422, -0.61092334, -0.039563839, -0.64805246, -0.73209886, -0.1612037, -0.1824376, 62 | -0.10095392, -0.54863707, -0.13006024, 0.84035913, -0.1074547, 0.095771397, 0.065293077, 63 | 0.50295166, 0.76087729, -0.05915887, 0.0038541875, 0.17693993, -0.10783917, -1.395354, 64 | -0.18819719, -0.91640697, 0.10561303, -0.87915558, -1.2311914, -0.8063915, -0.53489989, 65 | 0.40550872, 0.31034574, 0.13466076, 0.15388423, 0.42449091, -0.62185287, -0.1069148, 66 | -0.24735412, 0.091001952, 0.078088304, -1.4804614, -0.47730564, -0.85410218, -0.73027194, 67 | 0.28144448, -0.7827631, -1.0725579, -0.87395905, 0.019227432, -0.36397524, 0.039364162, 68 | -1.189656, 0.42376989, 0.029999473, 0.97409512, 0.28772906, 0.11587147, 0.20056804, 69 | -0.26722339, -0.42421664, -0.38231162, 0.48156817, -0.16695242, 0.43342902, -0.28646052, 70 | -0.034916129, 0.25049893, -1.0733647, -0.48631756, -0.49391604, -0.50135032, -0.040424458, 71 | 1.0258481, -0.62519932, 0.21244732, 0.33072464, 0.013948886, 0.57503137, -0.71442171, 72 | 0.32876287, -0.52405946, 0.26531818, -0.032614129, -0.049290842, 0.29821244, 0.55137431, 73 | -0.081307618, -2.3686619, -0.86137179, -0.98107756, -0.97629564, -0.40670394, -0.8485431, 74 | 0.6214841, -0.26896073, -0.1465191, 0.079095399, 1.5062982, 0.084564116, 0.20235011, 75 | 0.37424373, 0.029640964, 0.16488969, -0.060275283, -0.0043007629, -0.16993278, -0.37384607, 76 | 0.12352943, 0.85481896, -0.035967501, -0.053911601, -0.40461548, -0.9585113, -0.30067559, 77 | 1.140472, -0.49255279, 0.38920071, 1.8597437, -0.19081892, -0.29259476, -0.23485883, 78 | 0.10457873, 1.5402637, -0.12268229, 0.67514796, -0.45597952, 0.53631412, -0.25938066, 79 | 0.11603961, -0.16532986, -0.28776586, -0.30216771, -0.13983241, -0.23488818, 0.26841173, 80 | 0.67579952, 0.2075759, 0.10079492, 0.2883067, -0.076254561, -0.056714445, -0.12717646, 81 | -0.25804559, 0.10767514, 0.094400795, -0.58775883, 0.35738747, 0.30955558, 0.11727025, 82 | 1.5156538, -0.33047703, 0.9563396, -0.36071855, 0.71598153, -0.0018377865, 0.26591552, 83 | 0.21674342, 0.059564024, 0.070069545, -0.79517899, 0.36173561, -0.98804456, 0.023132508, 84 | -0.37981175, 0.6817025, 0.064290062, -0.032293471, -0.091723149, 0.16920939, 0.39532108, 85 | 0.92078631, 1.3558296, 0.28348178, 0.86695562, 0.26121262, 0.4401242, 0.99947631, 86 | -0.15200249, 0.75073983, 0.15087219, 0.44446802, -0.48152652, 0.23903341, 0.44827476, 87 | 0.63191367, 0.73132405, -0.15659082, 0.52343435, -0.39743259, 0.80255174, 0.085470138, 88 | 0.36742239, -0.083730254, -0.49241272, -0.11393205, 0.79072976, 0.3507508, -0.77723122, 89 | 0.63394345, -0.87268678, 0.61178516, 0.34102065, 0.39300976, 1.2152561, -0.010222636, 90 | 1.0280432, 0.51924918, 0.80043923, -0.075764857, 0.4653466, -0.028802604, 1.5028069, 91 | 0.13099841, 0.42172255, 0.69861489, 0.18687582, 0.34924764, 0.32517003, 0.68760069, 92 | 0.076066524, -0.11743453, 0.41831777, 1.3341999, 0.19940946, -0.10386014, 0.96476401, 93 | -0.52698667, 0.41861975, -0.59498568, -0.22380025, -1.7230965, -0.16611817, 0.40480869, 94 | -0.52719617, 0.09220452, 0.73069318, 1.077759, -0.13308773, 0.95736562, -0.1212927, 95 | -0.28634779, 0.40505793, 0.46279891, 0.30133646, 0.18380982, 0.32466724, 1.297843, 96 | 0.48267698, 0.95107169, 0.94341752, 1.2225781, 0.63840273, 0.4601062, 0.37791788, 97 | 0.64735413, -0.12596486, -0.1281331, -1.9312925, -0.26744321, 0.12684679, -0.099555644, 98 | 0.26884079, 0.31270983, 0.66459285, 0.4119147, 0.31075679, 1.5324757, 1.1849294, 99 | 0.23011497, 0.39824328, -0.029211714, 0.33448819, 0.74007796, 0.36431529, -0.76619006, 100 | 0.19175927, 0.21671878, 0.29059059, 0.13743621, -0.018456065, -0.29732754, 0.29029415, 101 | 0.63100797, 0.22476678, 0.80325931, -0.15971009, 1.548191, 0.38844737, 1.0856287, 102 | 1.0905757, 1.0661546, 0.18293959, 0.16563473, 1.2804299, 0.48590165, -0.72035689, 103 | 0.23467517, -0.42360673, -0.77477641, -0.075293888, -0.71484734, -1.4952735, 0.1655765, 104 | 0.36127044, 0.054662283, -0.5566214, 0.32951196, 0.36611205, 0.2143739, 0.68814351, 105 | 0.33994408, 0.98631133, 0.20109481, 1.0035688, -0.14433747, 0.56124644, 0.62519471, 106 | 0.8047775, 0.62384013, 1.2266169, 0.5431755, -1.4693138, -0.080789101, -0.47382196, 107 | -0.68793498, -0.20996721, 0.12546701, -0.30741481, -0.97833445, 0.64061856, 0.45745884, 108 | -0.27755774, -0.85114379, -0.11401231, -0.58619251, -0.10015404, -0.51423913, 0.012935767, 109 | 1.4442867, -0.14993207, -1.356337, -0.21873557, -1.7991753, -1.2349689, -0.6561743 110 | }; 111 | double unscaled_bias = -14.786592; 112 | -------------------------------------------------------------------------------- /cpp/realtimetest/compile_realtimetest.sh: -------------------------------------------------------------------------------- 1 | g++ -std=c++11 realtimetest.cpp -O3 -I/usr/local/include/opencv2 -I/usr/local/include/opencv -L/usr/local/lib -lopencv_core -lopencv_imgcodecs -lopencv_highgui -L/usr/local/lib -lopencv_shape -lopencv_stitching -lopencv_objdetect -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_features2d -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_video -lopencv_photo -lopencv_ml -lopencv_imgproc -lopencv_flann -lopencv_core 2 | -------------------------------------------------------------------------------- /cpp/realtimetest/feature.h: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | using namespace std; 11 | using namespace cv; 12 | 13 | #define HOG_WIDTH 64 14 | #define HOG_HEIGHT 32 15 | #define CELL_SIZE_ROW 8 16 | #define CELL_SIZE_COL 8 17 | #define BLOCK_SIZE_ROW 2 18 | #define BLOCK_SIZE_COL 2 19 | #define N_CELLS_ROW HOG_HEIGHT/CELL_SIZE_ROW //4 20 | #define N_CELLS_COL HOG_WIDTH/CELL_SIZE_COL //8 21 | #define N_BLOCK_ROW (N_CELLS_ROW - 1) //3 22 | #define N_BLOCK_COL (N_CELLS_COL - 1) //7 23 | #define HISTOGRAMSIZE N_BLOCK_ROW * N_BLOCK_COL * BLOCK_SIZE_ROW * BLOCK_SIZE_COL * 9 //3*7*2*2*9 24 | #define FEATURESIZE (HISTOGRAMSIZE + 192*2) 25 | 26 | 27 | inline int approx_distance(int dx, int dy){ 28 | int min, max; //uint 29 | if(dx < 0) dx = -dx; 30 | if(dy < 0) dy = -dy; 31 | 32 | if(dx < dy){ 33 | min = dx; 34 | max = dy; 35 | }else{ 36 | min = dy; 37 | max = dx; 38 | } 39 | 40 | //coefficients equivalent to (123/128 * max) and (51/128*min) 41 | return ((( max << 8 ) + ( max << 3 ) - ( max << 4 ) - ( max << 1 ) + 42 | ( min << 7 ) - ( min << 5 ) + ( min << 3 ) - ( min << 1 )) >> 8 ); 43 | } 44 | 45 | void lite_hog(cv::Mat img, double* dst, bool approx_mode){ 46 | int descriptor[HISTOGRAMSIZE]={0}; 47 | int sum_of_block[N_BLOCK_ROW * N_BLOCK_COL]={0}; 48 | for(int i = 0; i < HISTOGRAMSIZE; i++) descriptor[i] = 0; 49 | for(int i = 0; i < N_BLOCK_ROW * N_BLOCK_COL; i++) sum_of_block[i] = 0; 50 | int lut0[256] = {0,0,0,1,1,1,2,2,2,3,3,3,4,4,5,5,5,6,6,6,7,7,7,8,8,9,9,9,10,10,10,11,11,11,12,12,13,13,13,14,14,14,15,15,15,16,16,17,17,17,18,18,18,19,19,19,20,20,21,21,21,22,22,22,23,23,23,24,24,25,25,25,26,26,26,27,27,27,28,28,29,29,29,30,30,30,31,31,31,32,32,33,33,33,34,34,34,35,35,35,36,36,37,37,37,38,38,38,39,39,39,40,40,41,41,41,42,42,42,43,43,43,44,44,45,45,45,46,46,46,47,47,47,48,48,49,49,49,50,50,50,51,51,51,52,52,53,53,53,54,54,54,55,55,55,56,56,57,57,57,58,58,58,59,59,59,60,60,61,61,61,62,62,62,63,63,63,64,64,65,65,65,66,66,66,67,67,67,68,68,69,69,69,70,70,70,71,71,71,72,72,73,73,73,74,74,74,75,75,75,76,76,77,77,77,78,78,78,79,79,79,80,80,81,81,81,82,82,82,83,83,83,84,84,85,85,85,86,86,86,87,87,87,88,88,89,89,89,90,90,90,91,91,91,92,92}; 51 | int lut1[256] = {0,0,1,2,3,4,5,5,6,7,8,9,10,10,11,12,13,14,15,15,16,17,18,19,20,20,21,22,23,24,25,26,26,27,28,29,30,31,31,32,33,34,35,36,36,37,38,39,40,41,41,42,43,44,45,46,46,47,48,49,50,51,52,52,53,54,55,56,57,57,58,59,60,61,62,62,63,64,65,66,67,67,68,69,70,71,72,72,73,74,75,76,77,78,78,79,80,81,82,83,83,84,85,86,87,88,88,89,90,91,92,93,93,94,95,96,97,98,98,99,100,101,102,103,104,104,105,106,107,108,109,109,110,111,112,113,114,114,115,116,117,118,119,119,120,121,122,123,124,124,125,126,127,128,129,130,130,131,132,133,134,135,135,136,137,138,139,140,140,141,142,143,144,145,145,146,147,148,149,150,150,151,152,153,154,155,156,156,157,158,159,160,161,161,162,163,164,165,166,166,167,168,169,170,171,171,172,173,174,175,176,177,177,178,179,180,181,182,182,183,184,185,186,187,187,188,189,190,191,192,192,193,194,195,196,197,197,198,199,200,201,202,203,203,204,205,206,207,208,208,209,210,211,212,213,213}; 52 | int lut2[256] = {0,1,3,5,6,8,10,12,13,15,17,19,20,22,24,25,27,29,31,32,34,36,38,39,41,43,44,46,48,50,51,53,55,57,58,60,62,63,65,67,69,70,72,74,76,77,79,81,83,84,86,88,89,91,93,95,96,98,100,102,103,105,107,108,110,112,114,115,117,119,121,122,124,126,127,129,131,133,134,136,138,140,141,143,145,147,148,150,152,153,155,157,159,160,162,164,166,167,169,171,172,174,176,178,179,181,183,185,186,188,190,191,193,195,197,198,200,202,204,205,207,209,210,212,214,216,217,219,221,223,224,226,228,230,231,233,235,236,238,240,242,243,245,247,249,250,252,254,255,257,259,261,262,264,266,268,269,271,273,274,276,278,280,281,283,285,287,288,290,292,294,295,297,299,300,302,304,306,307,309,311,313,314,316,318,319,321,323,325,326,328,330,332,333,335,337,338,340,342,344,345,347,349,351,352,354,356,358,359,361,363,364,366,368,370,371,373,375,377,378,380,382,383,385,387,389,390,392,394,396,397,399,401,402,404,406,408,409,411,413,415,416,418,420,421,423,425,427,428,430,432,434,435,437,439,441}; 53 | int lut3[256] = {0,5,11,17,22,28,34,39,45,51,56,62,68,73,79,85,90,96,102,107,113,119,124,130,136,141,147,153,158,164,170,175,181,187,192,198,204,209,215,221,226,232,238,243,249,255,260,266,272,277,283,289,294,300,306,311,317,323,328,334,340,345,351,357,362,368,374,379,385,391,396,402,408,413,419,425,430,436,442,447,453,459,464,470,476,481,487,493,498,504,510,515,521,527,532,538,544,549,555,561,566,572,578,584,589,595,601,606,612,618,623,629,635,640,646,652,657,663,669,674,680,686,691,697,703,708,714,720,725,731,737,742,748,754,759,765,771,776,782,788,793,799,805,810,816,822,827,833,839,844,850,856,861,867,873,878,884,890,895,901,907,912,918,924,929,935,941,946,952,958,963,969,975,980,986,992,997,1003,1009,1014,1020,1026,1031,1037,1043,1048,1054,1060,1065,1071,1077,1082,1088,1094,1099,1105,1111,1116,1122,1128,1133,1139,1145,1150,1156,1162,1168,1173,1179,1185,1190,1196,1202,1207,1213,1219,1224,1230,1236,1241,1247,1253,1258,1264,1270,1275,1281,1287,1292,1298,1304,1309,1315,1321,1326,1332,1338,1343,1349,1355,1360,1366,1372,1377,1383,1389,1394,1400,1406,1411,1417,1423,1428,1434,1440,1445}; 54 | 55 | for(int y = 0; y < N_BLOCK_ROW; y++){//3 56 | for(int x = 0; x < N_BLOCK_COL; x++){//7 57 | for(int yy = 0; yy < CELL_SIZE_ROW*BLOCK_SIZE_ROW; yy++){//16 58 | for(int xx = 0; xx < CELL_SIZE_COL*BLOCK_SIZE_COL; xx++){//16 59 | int ny = CELL_SIZE_ROW * y + yy; 60 | int nx = CELL_SIZE_COL * x + xx; 61 | //compute gradient 62 | int mag = 0; 63 | int bin_index; 64 | if(ny == 0 || nx == 0 || ny == 31 || nx == 63){ 65 | mag = 0; 66 | bin_index = 0; 67 | }else{ 68 | int Gx = int(img.ptr(ny)[nx+1]) - int(img.ptr(ny)[nx-1]); 69 | int Gy = int(img.ptr(ny+1)[nx]) - int(img.ptr(ny-1)[nx]); 70 | mag = (approx_mode ? (approx_distance(Gx, Gy)) : sqrt((double)(Gx*Gx + Gy * Gy))); 71 | if((Gx >= 0 && Gy >= 0) || (Gx <= 0 && Gy <= 0)){ 72 | if (abs(Gy) < lut0[abs(Gx)]) {bin_index = 0;} 73 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 1;} 74 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 2;} 75 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 3;} 76 | else {bin_index = 4;} 77 | } else{ 78 | if (abs(Gy) < lut0[abs(Gx)]){bin_index = 4;} 79 | else if (abs(Gy) < lut1[abs(Gx)]) {bin_index = 5;} 80 | else if (abs(Gy) < lut2[abs(Gx)]) {bin_index = 6;} 81 | else if (abs(Gy) < lut3[abs(Gx)]) {bin_index = 7;} 82 | else {bin_index = 8;} 83 | } 84 | } 85 | int cell_index = (yy / 8) * 2 + (xx / 8); 86 | int block_index = y * N_BLOCK_COL + x; 87 | int descrip_index = block_index * 36 + cell_index * 9 + bin_index; 88 | descriptor[descrip_index] += mag; 89 | sum_of_block[block_index] += mag; 90 | } 91 | } 92 | } 93 | } 94 | 95 | for(int by = 0; by < N_BLOCK_ROW; by++){//3 96 | for(int bx = 0; bx < N_BLOCK_COL; bx++){//7 97 | int blkIdx = by * N_BLOCK_COL + bx; 98 | for (int i = 0; i < 36; i++) { 99 | int index = blkIdx * 36 + i; 100 | if(sum_of_block[blkIdx] == 0){ 101 | dst[index] = 0; 102 | }else{ 103 | dst[index] = (double)descriptor[index]/(double)sum_of_block[blkIdx]; 104 | } 105 | } 106 | } 107 | } 108 | } 109 | 110 | 111 | // void ravel(cv::Mat img, int* dst){ 112 | void ravel(cv::Mat img, double* dst){ 113 | vector rst; 114 | for(int y = 0; y < img.rows; y++){ 115 | for(int x = 0; x < img.cols; x++){ 116 | cv::Vec pix = img.ptr(y)[x]; 117 | dst[3 * (y * img.cols + x)] = double(pix[0])/256.0; 118 | dst[3 * (y * img.cols + x) + 1] = double(pix[1])/256.0; 119 | dst[3 * (y * img.cols + x) + 2] = double(pix[2])/256.0; 120 | } 121 | } 122 | } 123 | 124 | 125 | //Input: 126 | //img: BGR image 127 | //dst: pointer to save extracted feature value 128 | void getFeature(cv::Mat img, double* dst, bool approx_mode){ 129 | Mat resized_img, gray; 130 | cv::resize(img, resized_img, cv::Size(64 ,32), INTER_NEAREST); 131 | 132 | cv::Mat resized_hls; 133 | cv::cvtColor(resized_img, resized_hls, CV_BGR2HSV); 134 | 135 | cv::Size spatial_size(8, 8); 136 | Mat spatial_rgb, spatial_hls; 137 | cv::resize(resized_img, spatial_rgb, spatial_size, INTER_LINEAR); 138 | cv::resize(resized_hls, spatial_hls, spatial_size, INTER_LINEAR); 139 | cv::cvtColor(img, gray, CV_BGR2GRAY); 140 | 141 | ravel(spatial_hls, dst); 142 | ravel(spatial_rgb, dst + 192); 143 | lite_hog(gray, dst+192*2, approx_mode); 144 | } -------------------------------------------------------------------------------- /cpp/realtimetest/realtimetest.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | #include "feature.h" 12 | #include "weights.h" 13 | 14 | using namespace std; 15 | using namespace cv; 16 | 17 | float thresh = 0.85; 18 | 19 | double predict(int y, int x, double* dst){ 20 | double dotprodcut = 0; 21 | for(int i = 0; i < FEATURESIZE; i++) dotprodcut += dst[i] * unscaled_weight[i]; 22 | double svm_output = dotprodcut + unscaled_bias; 23 | double proba = 1.0/(1.0+exp(-1*svm_output)); 24 | return proba; 25 | } 26 | 27 | cv::Mat getShrinkFrame(cv::Mat original_img, int window_height){ 28 | //shrink whole frame 29 | float scaling = 32.0/window_height; 30 | // cv::Mat resized_img(cv::Size(640.0*scaling, 480.0*scaling), CV_8UC3); 31 | cv::Mat resized_img; 32 | resize(original_img, resized_img, cv::Size(), scaling, scaling); 33 | //pase scaled image to 320x240 canvas 34 | cv::Mat scaled_frame(cv::Size(320, 240), CV_8UC3); 35 | cv::Mat roi_dst = scaled_frame(cv::Rect(0, 0, 640*scaling, 480*scaling)); 36 | resized_img.copyTo(roi_dst); 37 | return scaled_frame; 38 | } 39 | 40 | //input: 320x240pix 41 | //output: ((sy,sx), window_height) 42 | vector,int>> predictRectFrame(cv::Mat inputimg, int window_height, int sy = 0, int sx = 0){ 43 | vector,int>> rst; 44 | // float scaling = 32.0 / window_height; 45 | cv::Mat frame_copy = inputimg.clone(); 46 | for(int y = 0; y <= 240 - 32; y+=8){ 47 | for(int x = 0; x <= 320 - 64; x+=8){ 48 | double dst[FEATURESIZE]; 49 | for(int i = 0; i < FEATURESIZE; i++) dst[i] = 0; 50 | // Mat window_gray_img = gray(cv::Rect(x, y, 64, 32)); 51 | Mat window_bgr_img = inputimg(cv::Rect(x, y, 64, 32)); 52 | getFeature(window_bgr_img, dst, true); 53 | double proba = predict(y, x, dst); 54 | 55 | if(proba > thresh){ 56 | cout << window_height << " " << proba << endl; 57 | int detected_y = (int)((float)y * window_height / 32.0) + sy; 58 | int detected_x = (int)((float)x * window_height / 32.0) + sx; 59 | rst.push_back(make_pair(make_pair(detected_y, detected_x), window_height)); 60 | // cout << y << " " << x << " " << proba << endl; 61 | // rectangle(frame_copy, Point(x, y), Point(x + 64, y + 32), Scalar(0,0,200), 2); //x,y //Scaler = B,G,R 62 | // cv::putText(frame_copy, to_string(proba), cv::Point(x+5,y+5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(255,0,0), 1, CV_AA); 63 | } 64 | } 65 | } 66 | return rst; 67 | } 68 | 69 | vector,int>> processRectFrame(cv::Mat original_img, int window_height, int sy = 0, int sx = 0){ 70 | if(window_height <= 32){ 71 | //crop and predict 72 | Mat cropped_img = original_img(cv::Rect(sx, sy, 320, 240)); 73 | return predictRectFrame(cropped_img, window_height, sy, sx); 74 | }else if(window_height <= 64){ 75 | cv::Mat resized_img; 76 | float scaling = 32.0/window_height; 77 | resize(original_img, resized_img, cv::Size(), scaling, scaling); 78 | Mat cropped_img = resized_img(cv::Rect(0, 0, 320, 240)); 79 | return predictRectFrame(cropped_img, window_height); 80 | }else { 81 | Mat shrinked_img = getShrinkFrame(original_img, window_height); 82 | return predictRectFrame(shrinked_img, window_height); 83 | } 84 | } 85 | 86 | void putRectangle(vector,int>> window_candidates, Mat frame_image){ 87 | for(int i = 0; i < window_candidates.size(); i++){ 88 | int sy = window_candidates[i].first.first; 89 | int sx = window_candidates[i].first.second; 90 | int window_height = window_candidates[i].second; 91 | rectangle(frame_image, Point(sx, sy), Point(sx + window_height*2, sy + window_height), Scalar(0,0,200), 2); //x,y //Scaler = B,G,R 92 | } 93 | } 94 | 95 | int main(void){ 96 | /*cv::Mat img = cv::imread("frame.png"); 97 | cv::Mat res = processFrame(img, 80); 98 | cv::imshow("res", res); 99 | cv::waitKey(0);*/ 100 | 101 | cv::VideoCapture cap(1); 102 | if(!cap.isOpened()){ 103 | cout << "failed" << endl; 104 | return -1; 105 | } 106 | // std::chrono::system_clock::time_point t1, t2, t3, t4, t5, t6, t7; 107 | while(1){ 108 | // t1 = std::chrono::system_clock::now(); 109 | cv::Mat frame; 110 | cap >> frame; // get a new frame from camera 111 | cv::Mat res = frame.clone(); 112 | //auto candidates = processRectFrame(frame, 32, 240, 160); 113 | auto candidates2 = processRectFrame(frame, 80); 114 | //auto candidates3 = processRectFrame(frame, 70); 115 | auto candidates4 = processRectFrame(frame, 60); 116 | auto candidates5 = processRectFrame(frame, 50); 117 | 118 | //putRectangle(candidates, res); 119 | putRectangle(candidates2, res); 120 | //putRectangle(candidates3, res); 121 | putRectangle(candidates4, res); 122 | putRectangle(candidates5, res); 123 | cv::imshow("result", res); 124 | // cv::imwrite("result.png", frame_copy); 125 | cv::waitKey(1); 126 | } 127 | } 128 | -------------------------------------------------------------------------------- /data.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/data.zip -------------------------------------------------------------------------------- /hls/hog_svm/consts.h: -------------------------------------------------------------------------------- 1 | struct hogweight{ 2 | ap_uint<128> weightval[9]; 3 | }; 4 | 5 | struct pixweight{ 6 | ap_uint<128> weight[3]; 7 | }; 8 | 9 | typedef ap_fixed<32, 10> ap_fixed_point; 10 | typedef ap_fixed<64, 20> accum_fixed; 11 | 12 | -------------------------------------------------------------------------------- /hls/hog_svm/frame.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/hls/hog_svm/frame.png -------------------------------------------------------------------------------- /hls/hog_svm/main_tb.cpp: -------------------------------------------------------------------------------- 1 | #include "/opt/Xilinx/Vivado/2018.2/include/gmp.h" 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include "weight.h" 9 | #include 10 | #include 11 | #include 12 | #include "consts.h" 13 | 14 | using namespace std; 15 | using namespace cv; 16 | 17 | void hog_svm(hls::stream >& instream, hls::stream >& outstream, 18 | pixweight bgrhsv_w1[8], pixweight bgrhsv_w2[8], pixweight bgrhsv_w3[8], pixweight bgrhsv_w4[8], 19 | hogweight hog_w1[7], hogweight hog_w2[7], hogweight hog_w3[7]); 20 | 21 | string tostr(double val){ 22 | std::stringstream ss; 23 | ss << val; 24 | std::string str = ss.str(); 25 | return str; 26 | } 27 | 28 | pixweight bound_bgrhsv_w1[8], bound_bgrhsv_w2[8], bound_bgrhsv_w3[8], bound_bgrhsv_w4[8]; 29 | hogweight bound_hog_w1[7], bound_hog_w2[7], bound_hog_w3[7]; 30 | 31 | void prepare_bound_input(){ 32 | for(int i = 0; i < 7; i++){ 33 | for(int j = 0; j < 9; j++){ 34 | int index = (i*9+j)*4; 35 | ap_uint<128> tmp = 0; 36 | tmp = tmp | ((ap_uint<128>)unbound_hog_w1[index]); 37 | tmp = tmp | ((ap_uint<128>)unbound_hog_w1[index+1] << 32); 38 | tmp = tmp | ((ap_uint<128>)unbound_hog_w1[index+2] << 64); 39 | tmp = tmp | ((ap_uint<128>)unbound_hog_w1[index+3] << 96); 40 | bound_hog_w1[i].weightval[j] = tmp; 41 | } 42 | } 43 | for(int i = 0; i < 7; i++){ 44 | for(int j = 0; j < 9; j++){ 45 | int index = (i*9+j)*4; 46 | ap_uint<128> tmp = 0; 47 | tmp = tmp | ((ap_uint<128>)unbound_hog_w2[index]); 48 | tmp = tmp | ((ap_uint<128>)unbound_hog_w2[index+1] << 32); 49 | tmp = tmp | ((ap_uint<128>)unbound_hog_w2[index+2] << 64); 50 | tmp = tmp | ((ap_uint<128>)unbound_hog_w2[index+3] << 96); 51 | bound_hog_w2[i].weightval[j] = tmp; 52 | } 53 | } 54 | for(int i = 0; i < 7; i++){ 55 | for(int j = 0; j < 9; j++){ 56 | int index = (i*9+j)*4; 57 | ap_uint<128> tmp = 0; 58 | tmp = tmp | ((ap_uint<128>)unbound_hog_w3[index]); 59 | tmp = tmp | ((ap_uint<128>)unbound_hog_w3[index+1] << 32); 60 | tmp = tmp | ((ap_uint<128>)unbound_hog_w3[index+2] << 64); 61 | tmp = tmp | ((ap_uint<128>)unbound_hog_w3[index+3] << 96); 62 | bound_hog_w3[i].weightval[j] = tmp; 63 | } 64 | } 65 | for(int i = 0; i < 8; i++){ 66 | for(int j = 0; j < 3; j++){ 67 | int index = (i*3+j)*4; 68 | ap_uint<128> tmp = 0; 69 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w1[index]); 70 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w1[index+1] << 32); 71 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w1[index+2] << 64); 72 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w1[index+3] << 96); 73 | bound_bgrhsv_w1[i].weight[j] = tmp; 74 | } 75 | } 76 | for(int i = 0; i < 8; i++){ 77 | for(int j = 0; j < 3; j++){ 78 | int index = (i*3+j)*4; 79 | ap_uint<128> tmp = 0; 80 | //bbgr ubgr bhsv uhsv 81 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w2[index]);//uhsv 82 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w2[index+1] << 32);//bhsv 83 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w2[index+2] << 64);//ubgr 84 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w2[index+3] << 96);//bbgr 85 | bound_bgrhsv_w2[i].weight[j] = tmp; 86 | } 87 | } 88 | 89 | for(int i = 0; i < 8; i++){ 90 | for(int j = 0; j < 3; j++){ 91 | int index = (i*3+j)*4; 92 | ap_uint<128> tmp = 0; 93 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w3[index]); 94 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w3[index+1] << 32); 95 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w3[index+2] << 64); 96 | tmp = tmp | ((ap_uint<128>)unbound_bgrhsv_w3[index+3] << 96); 97 | bound_bgrhsv_w3[i].weight[j] = tmp; 98 | } 99 | } 100 | } 101 | int main(){ 102 | prepare_bound_input(); 103 | 104 | cv::Mat img = cv::imread("frame.png"); 105 | cv::Mat bgra_img; 106 | cv::cvtColor(img, bgra_img, CV_BGR2BGRA); 107 | cv::Mat frame_copy = img.clone(); 108 | hls::stream > instream; 109 | 110 | //Input Value Preparation 111 | for(int y = 0; y < 240; y++){ 112 | for(int x = 0; x < 320; x++){ 113 | cv::Vec pix = bgra_img.ptr(y)[x]; 114 | int pix_int[3] = {pix[0], pix[1], pix[2]}; 115 | ap_axiu<32,1,1,1> in; 116 | in.data = (pix_int[0]| (pix_int[1] << 8) | pix_int[2] << 16); 117 | instream << in; 118 | } 119 | } 120 | 121 | //Execute HW 122 | hls::stream> resultstream; 123 | double thresh = 0.75; 124 | hog_svm(instream, resultstream, bound_bgrhsv_w1, bound_bgrhsv_w2, bound_bgrhsv_w3, bound_bgrhsv_w4, bound_hog_w1, bound_hog_w2, bound_hog_w3); 125 | int cnt = 0; 126 | while(!resultstream.empty()){ 127 | int y = (cnt / 33) * 8; 128 | int x = (cnt % 33) * 8; 129 | ap_uint<32> data = resultstream.read().data; 130 | union{ 131 | int ival; 132 | float oval; 133 | }converter; 134 | converter.ival = data; 135 | float rst = converter.oval + unscaled_bias; 136 | float proba = 1.0/(1.0 + exp(-1 * rst)); 137 | if(proba > 0.75){ 138 | cout << fixed << setprecision(10) << rst << " " << proba*100.0 << endl; 139 | rectangle(frame_copy, Point(x, y), Point(x + 64, y + 32), Scalar(0,0,200), 2); //x,y //Scaler = B,G,R 140 | cv::putText(frame_copy, tostr(proba * 100.0), cv::Point(x,y-5), 5, 0.5, cv::Scalar(255.0,0,0), 1, CV_AA, false); 141 | } 142 | cnt++; 143 | } 144 | imwrite("result.png", frame_copy); 145 | //imshow("result", frame_copy); 146 | bool err = false; 147 | if(!err) return 0; 148 | else return -1; 149 | } 150 | -------------------------------------------------------------------------------- /hls/hog_svm/solution1/directives.tcl: -------------------------------------------------------------------------------- 1 | ############################################################ 2 | ## This file is generated automatically by Vivado HLS. 3 | ## Please DO NOT edit it. 4 | ## Copyright (C) 1986-2018 Xilinx, Inc. All Rights Reserved. 5 | ############################################################ 6 | -------------------------------------------------------------------------------- /hls/hog_svm/solution1/impl/ip/xilinx_com_hls_hog_svm_1_0.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/hls/hog_svm/solution1/impl/ip/xilinx_com_hls_hog_svm_1_0.zip -------------------------------------------------------------------------------- /hls/hog_svm/solution1/script.tcl: -------------------------------------------------------------------------------- 1 | ############################################################ 2 | ## This file is generated automatically by Vivado HLS. 3 | ## Please DO NOT edit it. 4 | ## Copyright (C) 1986-2018 Xilinx, Inc. All Rights Reserved. 5 | ############################################################ 6 | open_project hog_svm 7 | set_top hog_svm 8 | add_files hog_svm/blue.png 9 | add_files hog_svm/consts.h 10 | add_files hog_svm/frame.png 11 | add_files hog_svm/main.cpp 12 | add_files -tb hog_svm/main_tb.cpp -cflags "-Wno-unknown-pragmas -Wno-unknown-pragmas" 13 | add_files -tb hog_svm/weight.h -cflags "-Wno-unknown-pragmas -Wno-unknown-pragmas" 14 | open_solution "solution1" 15 | set_part {xczu3eg-sbva484-1-e} -tool vivado 16 | create_clock -period 8 -name default 17 | #source "./hog_svm/solution1/directives.tcl" 18 | csim_design 19 | csynth_design 20 | cosim_design -trace_level all -rtl vhdl 21 | export_design -rtl verilog -format ip_catalog 22 | -------------------------------------------------------------------------------- /hls/hog_svm/vivado_hls.app: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /hls/hog_svm/weight.h: -------------------------------------------------------------------------------- 1 | ap_uint<32> unbound_bgrhsv_w1[] = {284367, 77590, 4294105380, 4294115066, 300269, 88665, 4294916528, 45030, 343231, 86125, 238194, 29297, 4294848816, 117452, 4294624404, 524630, 4294875950, 176605, 4294758042, 4294799152, 4294809453, 208655, 4294705263, 96047, 211710, 4294788039, 4294381119, 516093, 180261, 4294889626, 4294749805, 4294897514, 151422, 4294836846, 130885, 4294743727, 59536, 60082, 399664, 4294913077, 4294856780, 29664, 400517, 4294759989, 4294910719, 37753, 4294933649, 4294877508, 223482, 4294718917, 4294925129, 1493883, 4294842589, 4294577460, 4294704378, 4294401872, 59663, 4294475678, 38876, 4294425079, 351314, 4294822068, 524059, 272799, 127924, 4294746878, 4294687435, 4294928737, 616538, 4294967032, 459891, 4294907461, 4293952032, 4294575477, 4294811251, 204795, 4293809481, 4294443412, 983324, 114203, 571642, 8353, 433397, 4294953245, 4294720245, 4294775615, 46244, 4293905309, 4294584397, 4294751512, 4294933589, 4294633404, 532639, 4294837027, 381818, 4294760726}; 2 | ap_uint<32> unbound_bgrhsv_w2[] = {4294740003, 4294736161, 682346, 133592, 4294715138, 4294766246, 4294809625, 4294782980, 22029, 4294934962, 4294732790, 4294736659, 4294737784, 4294784545, 1386309, 223458, 4294558810, 4294784861, 4294130032, 4293862430, 275124, 127409, 4294553059, 4294624155, 270090, 4294692440, 4294628400, 4294401585, 163395, 4294677982, 4293350753, 4294420221, 921038, 57931, 314759, 4294791406, 243509, 118974, 108248, 486235, 4294686195, 4294653679, 4294554808, 4294179962, 392043, 146211, 390960, 112551, 359977, 4294558234, 332914, 527582, 4294276427, 4293946341, 4294406357, 4294409631, 71023, 4294656939, 24182, 4294511472, 293808, 4294799941, 4294765758, 4294781154, 4294927934, 4294357839, 4294645973, 35987, 906943, 562371, 696566, 329639, 173787, 4294596757, 4294152835, 4294716448, 158938, 4294610590, 781826, 599444, 1496670, 802664, 1382729, 636434, 4294645547, 4294438668, 378199, 4294474475, 4294557885, 4294369279, 370960, 312314, 475047, 315799, 396590, 231345}; 3 | ap_uint<32> unbound_bgrhsv_w3[] = {4294748507, 4294545314, 4294342426, 685647, 4294849544, 4294567517, 291145, 326868, 4294908304, 4294659563, 4294869432, 4294588964, 4294700952, 4294871527, 641456, 430349, 4294626627, 4294811664, 4292788551, 4293722950, 41216, 490789, 4294478949, 4294774732, 4294844706, 4294718885, 76337, 357662, 4294581995, 4294556202, 4293826920, 4293366367, 4294891431, 4294905760, 4294790107, 4294558381, 4294946211, 316321, 1025153, 1079786, 4294363543, 4294379059, 4294588603, 4294737571, 322241, 114403, 279175, 247500, 4294169538, 79401, 584601, 702836, 4293107921, 4293667207, 407836, 4294807665, 540514, 205208, 520786, 344020, 4294854294, 301184, 4294940992, 164579, 4294416876, 4294884163, 4294847181, 4294688767, 519947, 786985, 455131, 643272, 4294799672, 86603, 4294652740, 1260, 4294575907, 4294920526, 609477, 1561462, 1009104, 1573858, 905946, 1498212, 89424, 4294857050, 252107, 48155, 4294931209, 4294684463, 4294607233, 1698, 103520, 407701, 4294931128, 262047}; 4 | ap_uint<32> unbound_bgrhsv_w4[] = {244660, 4294775212, 570880, 4294712259, 205990, 4294740460, 4294526791, 232692, 179928, 4294753682, 149106, 4294724533, 382230, 4294834742, 1085519, 4294435549, 349499, 4294811780, 4294545573, 4293694754, 385103, 4294625227, 319607, 4294631102, 48857, 433133, 295474, 123213, 4294834150, 63632, 4294349405, 4294022658, 4294788740, 201536, 4294745305, 83719, 4294936242, 651979, 4294853257, 4294570502, 4294815829, 424066, 106505, 4294348387, 4294889808, 616724, 4294835978, 669039, 4294947217, 4294710267, 150183, 4294605265, 4294746952, 4294222619, 4294651602, 156523, 196112, 589312, 128793, 560984, 4294707665, 4294474834, 610469, 527877, 4294311572, 4293914017, 4294861588, 263697, 186855, 350754, 163439, 364702, 4294941143, 4294836254, 3437, 295661, 4294776702, 4294583667, 4294715907, 3342, 296996, 348777, 265732, 296544, 4294604315, 4294659347, 511040, 239897, 4294431956, 4294567995, 4294545243, 4294618649, 4294905271, 60029, 4294818336, 4294910319}; 5 | ap_uint<32> unbound_hog_w1[] = {4293807146, 4293042323, 212202, 4292740320, 4293820545, 4293921295, 1181014, 4294761972, 715156, 2178492, 4293515611, 4294609111, 645042, 4290101743, 1951716, 121829, 719728, 4294688131, 827273, 664067, 4663463, 4294276289, 5743584, 4292229995, 4292144297, 2803467, 170147, 339885, 4294333083, 4293369336, 4294390992, 781650, 1134406, 2747981, 1095107, 3414565, 2365757, 4294392602, 4291810646, 4293916837, 214482, 136736, 4291276208, 251975, 1190909, 1291854, 2658021, 4293859535, 334533, 4293503963, 303853, 1230292, 559963, 4294053879, 4294084990, 4294621216, 1999355, 4293161053, 869737, 2281223, 3505299, 4289612865, 4294507282, 4294382884, 636172, 4292134452, 4294510304, 4293624876, 1747746, 4294660362, 864592, 2176826, 29509, 415866, 4292926655, 4292721004, 4294895098, 469460, 3198154, 4290997174, 4294294904, 1052497, 2939376, 1944427, 4293822944, 4294680438, 4293293415, 670531, 498250, 4294408059, 4292036781, 184276, 2000621, 663217, 4294417800, 1939993, 1170731, 1897161, 4292980012, 104602, 4294604166, 538470, 4292279494, 459648, 1379830, 2163043, 4294855602, 2810206, 4294617139, 1287317, 4294927603, 4293248847, 180333, 436222, 204925, 1930971, 4293698519, 4294758029, 4294349019, 3500805, 4292320076, 995216, 755433, 140072, 4294337272, 774847, 4293545148, 4292730961, 975350, 587353, 2309866, 4294380636, 1405402, 4294676662, 606977, 4293225970, 1307703, 4294036237, 4294890623, 4293717146, 1896021, 1244830, 740720, 1373287, 4294426341, 572873, 4292858552, 4293524667, 4293766983, 3146681, 4294856174, 507064, 1740710, 1419529, 4292190896, 58206, 4293762001, 4293870331, 4294779587, 674779, 4294682486, 4293917964, 1191388, 4293526509, 234978, 250422, 1392635, 4293690314, 3659972, 4294701357, 2505020, 4294358726, 712134, 302370, 4294757614, 4294711112, 82650, 148012, 795992, 2385346, 1037338, 482695, 4291269980, 4293448271, 4293683811, 948983, 4293807586, 437724, 2449924, 3446176, 4293807549, 4292100769, 4294206670, 649925, 4294043205, 4293789652, 4294819799, 537069, 4291946058, 573971, 2992148, 4294708777, 4291920705, 487956, 2455660, 4294690327, 4291832544, 2318645, 534224, 927572, 4293951385, 4293207598, 318745, 1575044, 2707382, 703987, 5114646, 1380227, 4293326999, 4291495921, 4293479047, 1245836, 4292662086, 2705513, 4291404540, 5770871, 1489876, 4293695921, 4291086556, 413171, 4290207507, 4294565593, 544067, 4294497793, 4292447941, 4292532672, 2730248, 935702, 4293384421, 4294065619, 2862156, 1113006, 4292978284, 4294539538, 4294717227, 10420, 4293032330, 1899, 960155, 183281, 141167, 2715519}; 6 | ap_uint<32> unbound_hog_w2[] = {548049, 4294374335, 4293328519, 4292756342, 6843796, 4294680983, 4294666936, 4294508487, 218598, 4293315031, 1491977, 1502899, 4293038901, 4293659835, 4294215938, 4293227991, 4294917542, 690171, 4294199600, 4293894459, 1780256, 4294837981, 3606782, 4294911366, 4413488, 699530, 4293488564, 1688942, 4294318518, 3034304, 4293628320, 4293541751, 2737179, 1819740, 4293547017, 4294858094, 4294210617, 1129862, 892593, 4294407318, 4292910123, 2197063, 4294804313, 1209939, 4293906880, 4294940734, 4293222639, 1360871, 230654, 4292322613, 4293709203, 4294548685, 4294340471, 322091, 4294235481, 4294018539, 4294886942, 3446426, 382350, 5778210, 2671157, 774712, 1740937, 4294245932, 153197, 4294514327, 4293657828, 255001, 2421365, 1247553, 4294283974, 61823, 4294422377, 4292510740, 624013, 2355161, 4294733391, 4292934833, 658635, 2763767, 4292507159, 4293673574, 512895, 1017914, 4292406152, 4294907344, 601507, 4294247874, 4294540571, 4294663822, 426873, 4294297861, 2527687, 4293129321, 1151384, 944200, 4292714255, 1521115, 4293488856, 2839270, 2029674, 4294463181, 4293358517, 4294478031, 4294370231, 1497179, 4294414002, 64464, 4294585497, 415533, 180711, 1300390, 467906, 993916, 1854738, 4294926269, 4292102493, 4294805451, 4294084240, 4294481427, 4291726690, 4293585168, 4291046231, 577908, 4294308209, 20470, 4294572960, 1362945, 4294527930, 374763, 558737, 4247759, 1483563, 4290998779, 1037045, 1467632, 4294880192, 220260, 214892, 4294050246, 1420963, 4294243052, 934764, 666344, 4294823842, 4294012458, 2106644, 747640, 4293337185, 1924901, 4294777177, 1840909, 4292424592, 4291372696, 3822690, 4293926603, 4294776994, 4291238234, 762834, 4291464888, 1575564, 4292665120, 714474, 4294112861, 2874771, 4292488332, 410598, 1145397, 494385, 4294498825, 396602, 863517, 2042440, 4291210030, 4294596188, 4294080429, 1459193, 4293418580, 280436, 4294486781, 348298, 882723, 1366985, 496682, 4293374496, 4294495876, 4293130390, 4294052611, 2982623, 4293877766, 2766887, 4294838650, 4294929616, 4293656640, 4293514566, 4293657480, 122687, 4294528489, 4294854490, 61203, 327993, 1521690, 66797, 4294910664, 3674102, 4293630383, 4294946977, 4291322372, 1463048, 4292837297, 4294660711, 4292999772, 2656324, 4292199132, 4294857193, 4294897057, 1333241, 3121591, 2525307, 1414158, 4292430826, 1905937, 4292963404, 487155, 4294757862, 4068318, 4291488592, 3306166, 4288348235, 2292, 4290391453, 4294491806, 4293864339, 4294632786, 4293749352, 282360, 2613887, 106774, 1386711, 2255056, 650973, 934288, 1323055, 1717782, 4289113998, 4293927774, 4292394151, 428793, 4290149889, 4294338083, 1252, 60858}; 7 | ap_uint<32> unbound_hog_w3[] = {4291698990, 4290239894, 4294242719, 1513911, 4293691328, 4292241795, 4290814216, 4294663075, 892823, 4292877253, 4293118852, 4294003436, 4294346694, 8437644, 4291478977, 4292720625, 4293306129, 4292869564, 4293536586, 716416, 4294861080, 4293784062, 1078958, 3488605, 2755513, 3590274, 1256526, 5629451, 80810, 4599774, 4292347387, 6883380, 950053, 3327988, 4292149525, 1380342, 4293841645, 4292309271, 325145, 4294159704, 4293949233, 909942, 2596030, 4293431631, 150519, 3753476, 1159578, 4294062948, 2197507, 2300121, 1683673, 4292734693, 246816, 4294772798, 4293347537, 4293874521, 4290418160, 701781, 4294056142, 4079476, 2165760, 2433943, 4294224644, 1489148, 4292962622, 368027, 4290931153, 4294585855, 2476807, 1017084, 217659, 4294128902, 4292408825, 4294130220, 210198, 4294126012, 4293763559, 4294013237, 3244218, 741391, 4294483522, 2658678, 4294173420, 4294916115, 2866711, 3077097, 4293167573, 4294868203, 4294871617, 447923, 4294079710, 107235, 4289418402, 4292654824, 971611, 2796005, 4293997010, 2442088, 4291854399, 5425751, 364979, 4292289247, 4294080918, 4293600961, 1628368, 1457051, 4291951861, 2103572, 4292124803, 4292318607, 1333799, 1432929, 4291702711, 1269129, 4206987, 3657538, 4286949980, 762363, 4291903786, 114356, 1623976, 5007179, 4292884075, 4294420837, 4294464462, 4294443102, 4294125473, 1303043, 4290923312, 4290511123, 1415805, 2817604, 4294701299, 4294167081, 564309, 4293433386, 3184199, 781102, 4293462413, 1863041, 1698653, 623232, 978339, 4294887097, 4294119205, 4291205578, 342776, 124209, 4294202183, 4292352261, 1962591, 2521438, 945244, 4284788551, 690643, 4292267242, 4687850, 4294907558, 2079786, 4292489227, 1647349, 4293599468, 693238, 4292910136, 2886642, 4289965494, 1558933, 4294226764, 1484500, 4292085557, 4294004880, 189065, 2519244, 944918, 4294436959, 4292497729, 2109817, 737415, 4294843431, 4294653274, 4289458747, 4292888724, 790629, 1016593, 4288638579, 4293008961, 97487, 761587, 772655, 4292987462, 4607235, 1031310, 4291412852, 527419, 1602956, 4294272432, 4293458300, 1096688, 86150, 327941, 4292421550, 4294559292, 4294643665, 1874218, 4292650150, 4293991219, 905615, 4294297880, 1901773, 865587, 436390, 4294499510, 1609502, 1653445, 769392, 4293778621, 4292811009, 4290975431, 855009, 2964928, 3287357, 4292099275, 478261, 3292066, 1284535, 4070636, 2861620, 3023810, 4294127903, 201268, 4292093607, 2270376, 4291766679, 4293499562, 4294387978, 597865, 4288751920, 4292083519, 682822, 1077288, 4285496940, 4291822738, 4293157597, 2030071, 4288577730, 914262, 4291069141, 1405994, 4294729400, 955078, 4290557397, 4294517661}; 8 | 9 | float unscaled_bias = -1.8334937; 10 | -------------------------------------------------------------------------------- /overlay/bingen.sh: -------------------------------------------------------------------------------- 1 | bootgen -image fpga.bif -arch zynqmp -w -o fpga.bin 2 | -------------------------------------------------------------------------------- /overlay/design_1_wrapper.bit: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/overlay/design_1_wrapper.bit -------------------------------------------------------------------------------- /overlay/dma0.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/;/plugin/; 2 | / { 3 | fragment@0 { 4 | target-path = "/amba_pl@0"; 5 | __overlay__ { 6 | #address-cells = <2>; 7 | #size-cells = <2>; 8 | axi_dma_0 { 9 | compatible = "generic-uio"; 10 | reg = <0x0 0xA001C000 0x0 0x1000>; 11 | }; 12 | }; 13 | }; 14 | }; 15 | 16 | 17 | -------------------------------------------------------------------------------- /overlay/fclk0-zynqmp.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/;/plugin/; 2 | / { 3 | fragment@0 { 4 | target-path = "/amba"; 5 | __overlay__ { 6 | fclk0 { 7 | compatible = "ikwzm,fclkcfg-0.10.a"; 8 | clocks = <&clk 0x47>; 9 | insert-rate = "100000000"; 10 | insert-enable = <1>; 11 | remove-rate = "1000000"; 12 | remove-enable = <0>; 13 | }; 14 | }; 15 | }; 16 | }; 17 | -------------------------------------------------------------------------------- /overlay/fpga-load.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/; 2 | / { 3 | fragment@0 { 4 | target-path = "/fpga-full"; 5 | __overlay__ { 6 | firmware-name = "fpga.bin"; 7 | }; 8 | }; 9 | }; -------------------------------------------------------------------------------- /overlay/fpga.bif: -------------------------------------------------------------------------------- 1 | all: 2 | { 3 | [destination_device = pl] design_1_wrapper.bit 4 | } -------------------------------------------------------------------------------- /overlay/fpga.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/overlay/fpga.bin -------------------------------------------------------------------------------- /overlay/hog_svm.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/;/plugin/; 2 | / { 3 | fragment@0 { 4 | target-path = "/amba_pl@0"; 5 | __overlay__ { 6 | #address-cells = <2>; 7 | #size-cells = <2>; 8 | hog_svm_0 { 9 | compatible = "generic-uio"; 10 | reg = <0x0 0xA0020000 0x0 0x10000>; 11 | }; 12 | }; 13 | }; 14 | }; 15 | 16 | 17 | -------------------------------------------------------------------------------- /overlay/init.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | mkdir /config/device-tree/overlays/fpga 4 | dtc -I dts -O dtb -o fpga-load.dtb fpga-load.dts 5 | cp fpga-load.dtb /config/device-tree/overlays/fpga/dtbo 6 | 7 | mkdir /config/device-tree/overlays/fclk0 8 | dtc -I dts -O dtb -o fclk0-zynqmp.dtb fclk0-zynqmp.dts 9 | cp fclk0-zynqmp.dtb /config/device-tree/overlays/fclk0/dtbo 10 | 11 | mkdir /config/device-tree/overlays/udmabuf0 12 | dtc -I dts -O dtb -o udmabuf0.dtb udmabuf0.dts 13 | cp udmabuf0.dtb /config/device-tree/overlays/udmabuf0/dtbo 14 | 15 | mkdir /config/device-tree/overlays/udmabuf1 16 | dtc -I dts -O dtb -o udmabuf1.dtb udmabuf1.dts 17 | cp udmabuf1.dtb /config/device-tree/overlays/udmabuf1/dtbo 18 | 19 | mkdir /config/device-tree/overlays/hog_svm 20 | dtc -I dts -O dtb -o hog_svm.dtb hog_svm.dts 21 | cp hog_svm.dtb /config/device-tree/overlays/hog_svm/dtbo 22 | 23 | mkdir /config/device-tree/overlays/dma0 24 | dtc -I dts -O dtb -o dma0.dtb dma0.dts 25 | cp dma0.dtb /config/device-tree/overlays/dma0/dtbo 26 | -------------------------------------------------------------------------------- /overlay/udmabuf0.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/;/plugin/; 2 | / { 3 | fragment@0 { 4 | target-path = "/amba_pl@0"; 5 | #address-celss = <2>; 6 | #size-cells = <2>; 7 | __overlay__ { 8 | udmabuf0 { 9 | compatible = "ikwzm,udmabuf-0.10.a"; 10 | device-name="udmabuf0"; 11 | size = <0x00050000>; 12 | }; 13 | }; 14 | }; 15 | }; 16 | -------------------------------------------------------------------------------- /overlay/udmabuf1.dts: -------------------------------------------------------------------------------- 1 | /dts-v1/;/plugin/; 2 | / { 3 | fragment@0 { 4 | target-path = "/amba_pl@0"; 5 | #address-celss = <2>; 6 | #size-cells = <2>; 7 | __overlay__ { 8 | udmabuf1 { 9 | compatible = "ikwzm,udmabuf-0.10.a"; 10 | device-name="udmabuf1"; 11 | size = <0x00050000>; 12 | }; 13 | }; 14 | }; 15 | }; 16 | -------------------------------------------------------------------------------- /python/README.md: -------------------------------------------------------------------------------- 1 | ### Python 2 | Python3 and Scikit-learn are used to train object, and extracted trained classifier parameter. 3 | 4 | #### vdtools.py 5 | MainClass, this refers [https://github.com/t-lanigan/vehicle-detection-and-tracking](https://github.com/t-lanigan/vehicle-detection-and-tracking) repository. 6 | This file is module, so this file is not executed alone. 7 | 8 | #### myhog.py 9 | This file is module, so this file is not executed alone. 10 | In this repo, scikit-learn HOG function library is not used. This module extracts L1 normalized HOG feature. 11 | 12 | #### train.py 13 | Train the object. 14 | Set `self.load_saved = True` in vdtools.py to train. 15 | Trained data is saved in `./cache` directory. 16 | 17 | #### estimatetestor.py 18 | You can check model performance on Tkinter GUI 19 | 20 | |key |behavior | 21 | |-----------------|--------------------------| 22 | |<- | move to previous image | 23 | |-> | move to next image | 24 | |mouse scroll up | make a rectangle bigger | 25 | |mouse scroll down| make a rectanle smaller | 26 | |drag mouse | move a rectangle | 27 | 28 | 29 | #### myestimator.py 30 | Extract SVM parameter from cached trained model in `./cache` directory, and estimation is conducted without scikit-learn. 31 | Extracted parameter is exported in C++ code form to `./output/weights.h` 32 | The trained SVM weight is inverse-scaled. In `vdtools.py`, scaling is applied to extracted feature before input to SVM. 33 | Perform inverse-scaling on weights and biases to remove the necessity to perform scaling on extracted feature on HW. 34 | 35 | ``` 36 | double unscaled_weight[1140] = { 37 | 0.00026532167, 0.00010114468, 0.00025412301, 0.00089063402, 38 | ... 39 | } 40 | double unscaled_bias = -6.6535004; 41 | ``` 42 | 43 | -------------------------------------------------------------------------------- /python/estimatetestor.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from tkinter import * 3 | from PIL import Image 4 | import os 5 | import glob 6 | import cv2 7 | import numpy as np 8 | import vdtools 9 | 10 | parentdir = '../video' 11 | inputdir = 'bkc2_resize' 12 | # outputdir = 'output' 13 | imgname_prefix = 'image_' 14 | # csvlogfilename = 'crop.csv' 15 | 16 | 17 | _canvas_width = 640 18 | _canvas_height = 640 19 | _crop_width = 64 20 | _crop_height = 32 21 | 22 | #---------------------------------------------------------------------- 23 | class MainWindow(): 24 | 25 | #---------------- 26 | def __init__(self, main): 27 | # warning : image must be smaller than canvas size. 28 | # create image canvas 29 | self.canvas = Canvas(main, width=_canvas_width, height=_canvas_height) 30 | self.canvas.grid(row=0, column=0, columnspan=2, rowspan=4) 31 | 32 | # load image list 33 | # image file name must be 'prefix + number(0000-9999) .png' 34 | self.file_list = [] 35 | for i in range(10000): 36 | filepath = './' + parentdir + '/' + inputdir + '/' + imgname_prefix + ('%04d' % i) + '.png' 37 | # outputpath = './' + parentdir + '/' + outputdir + '/' + imgname_prefix + ('%04d' % i) + '.png' 38 | if os.path.exists(filepath): 39 | self.file_list.append((i, filepath)) 40 | 41 | # load cropping frame information if csv file exists 42 | # self.crop_frame_dic = {} 43 | # self.csvfilepath = './' + parentdir + '/' + csvlogfilename 44 | # if os.path.exists(self.csvfilepath): 45 | # with open(self.csvfilepath) as f: 46 | # for line in f: 47 | # elements = line.split(',') 48 | # #filename, sx, sy, ex, ey 49 | # self.crop_frame_dic[elements[0]] = (int(elements[1]), int(elements[2]), int(elements[3]), int(elements[4])) 50 | 51 | # load first image 52 | self.now_image_index = 0 53 | self.nowimage = PhotoImage(file=self.file_list[0][1]) 54 | self.image_on_canvas = self.canvas.create_image(0, 0, anchor=NW, image=self.nowimage) 55 | self.cvimage = cv2.imread(self.file_list[0][1]) 56 | 57 | # set callback function for key and mouse events 58 | self.canvas.bind('', self.right_key_pressed) 59 | self.canvas.bind('', self.left_key_pressed) 60 | self.canvas.bind('', self.space_key_pressed) 61 | self.canvas.bind("", self.mouse_lclick_moving) 62 | self.canvas.bind("", self.mouse_lbuton_pressed) 63 | self.canvas.bind("", self.mouse_lbutton_released) 64 | # mouse event with Windows OS 65 | root.bind("", self.mouse_wheel_moving) 66 | # mouse event with Linux OS 67 | root.bind("", self.mouse_wheel_moving) 68 | root.bind("", self.mouse_wheel_moving) 69 | 70 | self.canvas.focus_set() #need to recive key events 71 | 72 | # set initial value of crop frame 73 | self.cropframe_width = 128 74 | self.cropframe_height = self.cropframe_width * _crop_height / _crop_width 75 | self.cropframe_centerposx = _canvas_width/2 76 | self.cropframe_centerposy = _canvas_height/2 77 | 78 | # draw crop frame 79 | sx, sy, ex, ey = self.__getCropFrameCoordinate() 80 | self.crop_rectangle = self.canvas.create_rectangle(sx, sy, ex, ey, outline="blue", width=3) 81 | 82 | # filename label 83 | self.filenamelabel = Label(self.canvas, bg="black",fg="green",text=self.file_list[0][1],borderwidth=0,font=("",20)) 84 | self.filenamelabel.place(anchor=NW) 85 | 86 | #result label 87 | self.resultlabel = Label(self.canvas, bg="black",fg="red",text="",borderwidth=0,font=("",20)) 88 | self.resultlabel.place(relx=0.0, rely=1.0,anchor=SW) 89 | self.resultlabel2 = Label(self.canvas, bg="black",fg="green",text="",borderwidth=0,font=("",20)) 90 | self.resultlabel2.place(relx=1.0, rely=1.0,anchor=SE) 91 | 92 | self.__imageRefresh() 93 | 94 | #new vdtool instance 95 | self.finder = vdtools.WindowFinder("doll_3264.json") 96 | #---------------- 97 | def right_key_pressed(self, event): 98 | # open next image 99 | self.now_image_index += 1 100 | if self.now_image_index >= len(self.file_list): 101 | self.now_image_index = 0 102 | 103 | self.__imageRefresh() 104 | 105 | def left_key_pressed(self, event): 106 | # open previous image 107 | self.now_image_index -= 1 108 | if self.now_image_index < 0: 109 | self.now_image_index = len(self.file_list) - 1 110 | 111 | self.__imageRefresh() 112 | 113 | def space_key_pressed(self, event): 114 | pass 115 | 116 | def mouse_lclick_moving(self, event): 117 | self.cropframe_centerposx = event.x 118 | self.cropframe_centerposy = event.y 119 | self.__cropRectangleRefresh() 120 | 121 | def mouse_lbuton_pressed(self, event): 122 | pass 123 | def mouse_lbutton_released(self, event): 124 | sx, sy, ex, ey = self.__getCropFrameCoordinate() 125 | svm_proba, rf_proba = self.__doEstimate(sx, sy, ex, ey) 126 | self.resultlabel["text"] = str(svm_proba) 127 | self.resultlabel2["text"] = str(rf_proba) 128 | 129 | 130 | def mouse_wheel_moving(self, event): 131 | delta = 2 132 | delta_width = delta 133 | delta_height = delta * _crop_height / _crop_width 134 | # respond to Linux or Windows wheel event 135 | if event.num == 5 or event.delta == -120: 136 | # mouse wheel down 137 | self.cropframe_width = max(0, self.cropframe_width - delta_width) 138 | self.cropframe_height = max(0, self.cropframe_height - delta_height) 139 | if event.num == 4 or event.delta == 120: 140 | # mouse wheel up 141 | self.cropframe_width += delta_width 142 | self.cropframe_height += delta_height 143 | self.__cropRectangleRefresh() 144 | 145 | #---------------- 146 | def __imageRefresh(self): 147 | #change image 148 | self.nowimage = PhotoImage(file=self.file_list[self.now_image_index][1]) 149 | self.canvas.itemconfig(self.image_on_canvas,image=self.nowimage) 150 | 151 | #check whether cropped image coordinate information already exists 152 | #ignore wheter cropped image file exists 153 | inputfilepath = self.file_list[self.now_image_index][1] 154 | imagefilename = os.path.basename(inputfilepath) 155 | 156 | #update filename label 157 | self.filenamelabel["text"] = inputfilepath 158 | self.resultlabel["text"] = ""; 159 | self.resultlabel2["text"] = ""; 160 | self.cvimage = cv2.imread(inputfilepath) 161 | def __cropRectangleRefresh(self): 162 | sx, sy, ex, ey = self.__getCropFrameCoordinate() 163 | self.canvas.coords(self.crop_rectangle, sx, sy, ex, ey) 164 | 165 | def __saveCropImage(self): 166 | pass 167 | # im = Image.open(self.file_list[self.now_image_index][1]) 168 | # sx, sy, ex, ey = self.__getCropFrameCoordinate() 169 | # im_crop = im.crop((sx, sy, ex, ey)) 170 | # im_crop_resized = im_crop.resize((_crop_width, _crop_height)) 171 | # im_crop_resized.save(self.file_list[self.now_image_index][2], quality=95) 172 | 173 | def __addCropDic(self): 174 | pass 175 | # originalfilename = os.path.basename(self.file_list[self.now_image_index][1]) 176 | # sx, sy, ex, ey = self.__getCropFrameCoordinate() 177 | # self.crop_frame_dic[originalfilename] = (sx, sy, ex, ey) 178 | 179 | def __saveCropInfoToCSV(self): 180 | pass 181 | # with open(self.csvfilepath, mode='w') as f: 182 | # for filename, coordinate in self.crop_frame_dic.items(): 183 | # f.write(','.join([filename, str(coordinate[0]), str(coordinate[1]), str(coordinate[2]), str(coordinate[3])]) + "\n") 184 | # print("csv saved") 185 | 186 | def __getCropFrameCoordinate(self): 187 | sx = self.cropframe_centerposx - self.cropframe_width/2 188 | sy = self.cropframe_centerposy - self.cropframe_height/2 189 | ex = self.cropframe_centerposx + self.cropframe_width/2 190 | ey = self.cropframe_centerposy + self.cropframe_height/2 191 | return sx, sy, ex, ey 192 | 193 | def __doEstimate(self, sx, sy, ex, ey): 194 | test_crop = self.cvimage[int(sy):int(ey), int(sx):int(ex)] 195 | # test_crop = cv2.imread("../data/fpt/not_red_shukai/image_0023_1.png") 196 | svm_proba, rf_proba = self.finder.predictoneimage(test_crop) 197 | return svm_proba, rf_proba 198 | 199 | 200 | #---------------------------------------------------------------------- 201 | 202 | root = Tk() 203 | MainWindow(root) 204 | root.mainloop() -------------------------------------------------------------------------------- /python/myestimtaor.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | import graphviz 3 | import numpy as np 4 | from skimage.feature import hog 5 | from sklearn.svm import LinearSVC 6 | from sklearn.ensemble import AdaBoostClassifier, RandomForestClassifier 7 | from sklearn.metrics import recall_score 8 | from sklearn.preprocessing import StandardScaler 9 | from sklearn.model_selection import train_test_split 10 | from sklearn.utils import shuffle 11 | import cv2 12 | import vdtools 13 | 14 | #load trained models and scaler 15 | clf = pickle.load(open( './cache/clf.p', 'rb')) 16 | weights = clf.coef_[0] 17 | bias = clf.intercept_ 18 | scaler_mean = pickle.load(open( "./cache/scaler_mean.p", "rb" )) 19 | scaler_std = pickle.load(open( "./cache/scaler_std.p", "rb" )) 20 | 21 | #get geatures 22 | img = cv2.imread("input.png") 23 | feature = vdtools.WindowFinder().singleimgfeatures(img) 24 | 25 | #1. estimate in same way as scikit-learn 26 | scaled_feature = (feature - scaler_mean) / (scaler_std) 27 | rst = np.dot(scaled_feature, weights) + bias 28 | sigmoided_rst = 1/(1+np.exp(-1*rst)) 29 | print("estimate in same way as scikit-learn...") 30 | print("rst", rst, "sigmoided_rst", sigmoided_rst) 31 | 32 | #2. estimate by using unscaled weight 33 | unscaled_weight = weights / scaler_std 34 | unscaled_bias = bias + (- 1) * np.sum((scaler_mean * weights) / scaler_std) 35 | rst2 = np.dot(feature, unscaled_weight) + unscaled_bias 36 | sigmoided_rst2 = 1/(1+np.exp(-1*rst2)) 37 | print("estimate by using unscaled_weight") 38 | print("rst2", rst2, "sigmoided_rst2", sigmoided_rst2) 39 | 40 | #3. print unscaled weights and bias 41 | weight_num = len(unscaled_weight) 42 | code = "" 43 | unscaled_weight_strarray = list(map(lambda val: "{:.8g}".format(val), unscaled_weight)) 44 | 45 | code += "double unscaled_weight[{}] = {{\n".format(weight_num) + "{}\n}};\n".format(",\n".join(map(lambda list: " " * 31 + ", ".join(list), np.array(unscaled_weight_strarray).reshape(-1, 10)))) 46 | code += "double unscaled_bias = {:.8g};\n".format(unscaled_bias[0]) 47 | 48 | hsv_weight = unscaled_weight[0:192].reshape(8,8,3) 49 | bgr_weight = unscaled_weight[192:384].reshape(8,8,3) 50 | hog_weight = unscaled_weight[384:].reshape(3,7,4,9); 51 | 52 | code += "const weight WeightData[WINDOW_BLOCKNUM_H][WINDOW_BLOCKNUM_W] = {\n" 53 | for by in range(3): 54 | code += "{" 55 | for bx in range(7): 56 | code += "{" 57 | for part in range(4): 58 | code += "{" 59 | for bin_index in range(9): 60 | code += "{:.8g}".format(hog_weight[by][bx][part][bin_index]) 61 | if bin_index != 8: 62 | code += ", " 63 | code += "}" 64 | if part != 3: 65 | code += ", " 66 | code += "}" 67 | if bx != 6: 68 | code += ", \n" 69 | code += "}" 70 | if by != 2: 71 | code += ", \n" 72 | code += "\n};" 73 | 74 | code += "\n" 75 | 76 | code += "const pixweight WeightData[4][8] = {\n" 77 | for y in range(4): 78 | code += "{" 79 | for x in range(8): 80 | code += "{" 81 | code += "{" + "{:.8g}".format(hsv_weight[2*y][x][0]) + ", " + "{:.8g}".format(hsv_weight[2*y][x][1]) + ", " + "{:.8g}".format(hsv_weight[2*y][x][2]) + "}, " 82 | code += "{" + "{:.8g}".format(hsv_weight[2*y+1][x][0]) + ", " + "{:.8g}".format(hsv_weight[2*y+1][x][1]) + ", " + "{:.8g}".format(hsv_weight[2*y+1][x][2]) + "}, " 83 | code += "{" + "{:.8g}".format(bgr_weight[2*y][x][0]) + ", " + "{:.8g}".format(bgr_weight[2*y][x][1]) + ", " + "{:.8g}".format(bgr_weight[2*y][x][2]) + "}, " 84 | code += "{" + "{:.8g}".format(bgr_weight[2*y+1][x][0]) + ", " + "{:.8g}".format(bgr_weight[2*y+1][x][1]) + ", " + "{:.8g}".format(bgr_weight[2*y+1][x][2]) + "}" 85 | code += "}" 86 | if(x != 7): 87 | code += ", \n" 88 | code += "}\n" 89 | if y != 3: 90 | code += ", \n" 91 | code += "\n};" 92 | with open('./output/weights.h', mode='w') as f: 93 | f.write(code) 94 | -------------------------------------------------------------------------------- /python/myhog.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | from skimage.feature import hog 3 | import numpy as np 4 | import math 5 | 6 | INPUT_WIDTH = 128 7 | INPUT_HEIGHT = 64 8 | CELL_SIZE = 8 9 | BLOCK_SIZE = 2 10 | CELL_NUM_WIDTH = (INPUT_WIDTH//CELL_SIZE) 11 | CELL_NUM_HEIGHT = (INPUT_HEIGHT//CELL_SIZE) 12 | BLOCK_NUM_WIDTH = (CELL_NUM_WIDTH-1) 13 | BLOCK_NUM_HEIGHT = (CELL_NUM_HEIGHT-1) 14 | 15 | 16 | def _hog_channel_gradient(channel): 17 | g_row = np.empty(channel.shape, dtype=np.int16) 18 | g_row[0, :] = 0 19 | g_row[-1, :] = 0 20 | g_row[1:-1, :] = channel[2:, :] - channel[:-2, :] 21 | g_col = np.empty(channel.shape, dtype=np.int16) 22 | g_col[:, 0] = 0 23 | g_col[:, -1] = 0 24 | g_col[:, 1:-1] = channel[:, 2:] - channel[:, :-2] 25 | return g_row, g_col 26 | 27 | def approx_distance(dx, dy): 28 | min = 0 29 | max = 0 30 | if(dx < 0): 31 | dx = -dx; 32 | if(dy < 0): 33 | dy = -dy; 34 | 35 | if(dx < dy): 36 | min = dx; 37 | max = dy; 38 | else: 39 | min = dy; 40 | max = dx; 41 | 42 | 43 | return ((( max << 8 ) + ( max << 3 ) - ( max << 4 ) - ( max << 1 ) + 44 | ( min << 7 ) - ( min << 5 ) + ( min << 3 ) - ( min << 1 )) >> 8 ) 45 | 46 | lut0 = [0,0,0,1,1,1,2,2,2,3,3,3,4,4,5,5,5,6,6,6,7,7,7,8,8,9,9,9,10,10,10,11,11,11,12,12,13,13,13,14,14,14,15,15,15,16,16,17,17,17,18,18,18,19,19,19,20,20,21,21,21,22,22,22,23,23,23,24,24,25,25,25,26,26,26,27,27,27,28,28,29,29,29,30,30,30,31,31,31,32,32,33,33,33,34,34,34,35,35,35,36,36,37,37,37,38,38,38,39,39,39,40,40,41,41,41,42,42,42,43,43,43,44,44,45,45,45,46,46,46,47,47,47,48,48,49,49,49,50,50,50,51,51,51,52,52,53,53,53,54,54,54,55,55,55,56,56,57,57,57,58,58,58,59,59,59,60,60,61,61,61,62,62,62,63,63,63,64,64,65,65,65,66,66,66,67,67,67,68,68,69,69,69,70,70,70,71,71,71,72,72,73,73,73,74,74,74,75,75,75,76,76,77,77,77,78,78,78,79,79,79,80,80,81,81,81,82,82,82,83,83,83,84,84,85,85,85,86,86,86,87,87,87,88,88,89,89,89,90,90,90,91,91,91,92,92] 47 | lut1 = [0,0,1,2,3,4,5,5,6,7,8,9,10,10,11,12,13,14,15,15,16,17,18,19,20,20,21,22,23,24,25,26,26,27,28,29,30,31,31,32,33,34,35,36,36,37,38,39,40,41,41,42,43,44,45,46,46,47,48,49,50,51,52,52,53,54,55,56,57,57,58,59,60,61,62,62,63,64,65,66,67,67,68,69,70,71,72,72,73,74,75,76,77,78,78,79,80,81,82,83,83,84,85,86,87,88,88,89,90,91,92,93,93,94,95,96,97,98,98,99,100,101,102,103,104,104,105,106,107,108,109,109,110,111,112,113,114,114,115,116,117,118,119,119,120,121,122,123,124,124,125,126,127,128,129,130,130,131,132,133,134,135,135,136,137,138,139,140,140,141,142,143,144,145,145,146,147,148,149,150,150,151,152,153,154,155,156,156,157,158,159,160,161,161,162,163,164,165,166,166,167,168,169,170,171,171,172,173,174,175,176,177,177,178,179,180,181,182,182,183,184,185,186,187,187,188,189,190,191,192,192,193,194,195,196,197,197,198,199,200,201,202,203,203,204,205,206,207,208,208,209,210,211,212,213,213] 48 | lut2 = [0,1,3,5,6,8,10,12,13,15,17,19,20,22,24,25,27,29,31,32,34,36,38,39,41,43,44,46,48,50,51,53,55,57,58,60,62,63,65,67,69,70,72,74,76,77,79,81,83,84,86,88,89,91,93,95,96,98,100,102,103,105,107,108,110,112,114,115,117,119,121,122,124,126,127,129,131,133,134,136,138,140,141,143,145,147,148,150,152,153,155,157,159,160,162,164,166,167,169,171,172,174,176,178,179,181,183,185,186,188,190,191,193,195,197,198,200,202,204,205,207,209,210,212,214,216,217,219,221,223,224,226,228,230,231,233,235,236,238,240,242,243,245,247,249,250,252,254,255,257,259,261,262,264,266,268,269,271,273,274,276,278,280,281,283,285,287,288,290,292,294,295,297,299,300,302,304,306,307,309,311,313,314,316,318,319,321,323,325,326,328,330,332,333,335,337,338,340,342,344,345,347,349,351,352,354,356,358,359,361,363,364,366,368,370,371,373,375,377,378,380,382,383,385,387,389,390,392,394,396,397,399,401,402,404,406,408,409,411,413,415,416,418,420,421,423,425,427,428,430,432,434,435,437,439,441] 49 | lut3 = [0,5,11,17,22,28,34,39,45,51,56,62,68,73,79,85,90,96,102,107,113,119,124,130,136,141,147,153,158,164,170,175,181,187,192,198,204,209,215,221,226,232,238,243,249,255,260,266,272,277,283,289,294,300,306,311,317,323,328,334,340,345,351,357,362,368,374,379,385,391,396,402,408,413,419,425,430,436,442,447,453,459,464,470,476,481,487,493,498,504,510,515,521,527,532,538,544,549,555,561,566,572,578,584,589,595,601,606,612,618,623,629,635,640,646,652,657,663,669,674,680,686,691,697,703,708,714,720,725,731,737,742,748,754,759,765,771,776,782,788,793,799,805,810,816,822,827,833,839,844,850,856,861,867,873,878,884,890,895,901,907,912,918,924,929,935,941,946,952,958,963,969,975,980,986,992,997,1003,1009,1014,1020,1026,1031,1037,1043,1048,1054,1060,1065,1071,1077,1082,1088,1094,1099,1105,1111,1116,1122,1128,1133,1139,1145,1150,1156,1162,1168,1173,1179,1185,1190,1196,1202,1207,1213,1219,1224,1230,1236,1241,1247,1253,1258,1264,1270,1275,1281,1287,1292,1298,1304,1309,1315,1321,1326,1332,1338,1343,1349,1355,1360,1366,1372,1377,1383,1389,1394,1400,1406,1411,1417,1423,1428,1434,1440,1445] 50 | 51 | def myhog(input_image): 52 | sum_of_block = np.zeros(BLOCK_NUM_WIDTH * BLOCK_NUM_HEIGHT, dtype = 'float64') 53 | descriptor = np.zeros(BLOCK_NUM_WIDTH * BLOCK_NUM_HEIGHT * BLOCK_SIZE * BLOCK_SIZE * 9, dtype = 'float64') 54 | dst = np.zeros(BLOCK_NUM_WIDTH * BLOCK_NUM_HEIGHT * BLOCK_SIZE * BLOCK_SIZE * 9, dtype = 'float64') 55 | gxlist = [] 56 | 57 | binarray = np.zeros(INPUT_WIDTH*INPUT_HEIGHT, dtype = int) 58 | magarray = np.zeros(INPUT_WIDTH*INPUT_HEIGHT, dtype = 'float64') 59 | image = cv2.resize(input_image, (INPUT_WIDTH, INPUT_HEIGHT)) 60 | image = image.astype(np.int16) 61 | 62 | for y in range(INPUT_HEIGHT): 63 | for x in range(INPUT_WIDTH): 64 | mag = 0 65 | bin_index = 0 66 | if y == 0 or y == (INPUT_HEIGHT-1) or x == 0 or x == (INPUT_WIDTH-1): 67 | bin_index = 0 68 | mag = 0 69 | else: 70 | Gx = image[y][x+1] - image[y][x-1] 71 | Gy = image[y+1][x] - image[y-1][x] 72 | a = np.array([image[y][x+1], image[y+1][x]]) 73 | b = np.array([image[y][x-1], image[y-1][x]]) 74 | 75 | # mag = np.linalg.norm(a-b) 76 | mag = approx_distance(Gx, Gy) 77 | #bin_index = np.int((np.rad2deg(np.arctan2(Gx, Gy)) % 180) // 20) scikit-learn mode 78 | if((Gx >= 0 and Gy >= 0) or (Gx <= 0 and Gy <= 0)): 79 | if (abs(Gy) < lut0[abs(Gx)]): 80 | bin_index = 0 81 | elif (abs(Gy) < lut1[abs(Gx)]): 82 | bin_index = 1 83 | elif (abs(Gy) < lut2[abs(Gx)]): 84 | bin_index = 2 85 | elif (abs(Gy) < lut3[abs(Gx)]): 86 | bin_index = 3 87 | else: 88 | bin_index = 4 89 | else: 90 | if (abs(Gy) < lut0[abs(Gx)]): 91 | bin_index = 4 92 | elif (abs(Gy) < lut1[abs(Gx)]): 93 | bin_index = 5 94 | elif (abs(Gy) < lut2[abs(Gx)]): 95 | bin_index = 6 96 | elif (abs(Gy) < lut3[abs(Gx)]): 97 | bin_index = 7 98 | else: 99 | bin_index = 8 100 | binarray[y*INPUT_WIDTH+x] = bin_index 101 | magarray[y*INPUT_WIDTH+x] = mag 102 | 103 | for y in range(BLOCK_NUM_HEIGHT): 104 | for x in range(BLOCK_NUM_WIDTH): 105 | for yy in range(CELL_SIZE*BLOCK_SIZE): 106 | for xx in range(CELL_SIZE*BLOCK_SIZE): 107 | ny = CELL_SIZE * y + yy 108 | nx = CELL_SIZE * x + xx 109 | bin_index = binarray[ny * INPUT_WIDTH + nx] 110 | cell_index = (yy // CELL_SIZE) * BLOCK_SIZE + (xx // CELL_SIZE); 111 | block_index = y * BLOCK_NUM_WIDTH + x; 112 | descrip_index = block_index * BLOCK_SIZE * BLOCK_SIZE * 9 + cell_index * 9 + bin_index; 113 | descriptor[descrip_index] += magarray[ny * INPUT_WIDTH + nx]; 114 | sum_of_block[block_index] += magarray[ny * INPUT_WIDTH + nx]; 115 | 116 | average = 0 117 | # blkSize = 36 118 | for by in range(BLOCK_NUM_HEIGHT): 119 | for bx in range(BLOCK_NUM_WIDTH): 120 | blkIdx = by * BLOCK_NUM_WIDTH + bx 121 | for i in range(BLOCK_SIZE*BLOCK_SIZE*9): 122 | index = blkIdx * (BLOCK_SIZE*BLOCK_SIZE*9) + i 123 | if sum_of_block[blkIdx] == 0 : 124 | dst[index] = 0 125 | else: 126 | # dst[index] = np.sqrt(descriptor[index] / sum_of_block[blkIdx]) 127 | dst[index] = descriptor[index] / (sum_of_block[blkIdx]) 128 | 129 | 130 | return np.array(dst) 131 | 132 | # np.set_printoptions(linewidth=100) 133 | # img = cv2.imread("input.png") 134 | # gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 135 | # print(myhog(gray)) -------------------------------------------------------------------------------- /python/train.py: -------------------------------------------------------------------------------- 1 | import vdtools 2 | import numpy as np 3 | import pickle 4 | import cv2 5 | import glob 6 | import matplotlib.image as mpimg 7 | 8 | finder = vdtools.WindowFinder() -------------------------------------------------------------------------------- /python/vdtools.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import glob 3 | import json 4 | import math 5 | import numpy as np 6 | import pickle 7 | import time 8 | 9 | import myhog 10 | 11 | import matplotlib.image as mpimg 12 | import matplotlib.pyplot as plt 13 | from scipy.ndimage.measurements import label 14 | from skimage.feature import hog 15 | from sklearn.svm import LinearSVC 16 | from sklearn.ensemble import AdaBoostClassifier, RandomForestClassifier 17 | from sklearn.metrics import recall_score 18 | from sklearn.preprocessing import StandardScaler 19 | from sklearn.model_selection import train_test_split 20 | from sklearn.utils import shuffle 21 | from sklearn import svm 22 | from sklearn.externals import joblib 23 | from sklearn.model_selection import GridSearchCV 24 | from sklearn.metrics import make_scorer 25 | from sklearn.metrics import f1_score 26 | 27 | class WindowFinder(object): 28 | """Finds windows in an image that contain a car.""" 29 | def __init__(self, cfgfilepath): 30 | 31 | 32 | jsonobject = None 33 | with open(cfgfilepath) as f: 34 | jobject = json.loads(f.read()) 35 | self.spatial_size = (jobject["spatial_size"], jobject["spatial_size"]) 36 | self.spatial_feat = jobject["spatial_feat"]# Spatial features on or off 37 | self.hog_feat = jobject["hog_feat"]# HOG features on or off 38 | 39 | ### Hyperparameters, if changed ->(load_saved = False) If 40 | ### the classifier is changes load_feaures can be True 41 | self.load_saved = True #Histogram features on or off 42 | self.load_features = False # Loads saved features (to train new classifier) 43 | 44 | # The locations of all the data. 45 | self.negative_data_folders = jobject["negative_data_folders"] 46 | self.positive_data_folders = jobject["positive_data_folders"] 47 | 48 | self.svm_clf_name = jobject["svm_clf_name"] 49 | self.rf_clf_name = jobject["rf_clf_name"] 50 | self.svm_enable = jobject["svm_enable"] 51 | self.rf_enable = jobject["rf_enable"] 52 | 53 | self.scaler_name = jobject["scaler_name"] 54 | self.scaler_mean_name = jobject["scaler_mean_name"] 55 | self.scaler_std_name = jobject["scaler_std_name"] 56 | 57 | ######Classifiers 58 | self.pred_thresh = 0.65 #Increase to decrease likelihood of detection. 59 | 60 | ###### Variable for Classifier and Feature Scaler ########## 61 | #SVM 62 | tuned_parameters = [{'C': [0.1, 0.5, 1.0, 5.0, 10.0, 50.0, 100.0]}] 63 | self.svm_grid_search = GridSearchCV(svm.LinearSVC(max_iter = 1000000), tuned_parameters, cv=5) 64 | #RF 65 | forest_grid_param = { 66 | 'n_estimators': [100], 67 | 'max_features': [2, 4, 8, 'auto'], 68 | 'max_depth': [25], 69 | 'min_samples_leaf': [2, 4, 8] 70 | } 71 | f1_scoring = make_scorer(f1_score, pos_label=1) 72 | self.forest_grid_search = GridSearchCV(RandomForestClassifier(random_state=0, n_jobs=4), forest_grid_param, scoring=f1_scoring, cv=4) 73 | 74 | self.trained_svm, self.trained_rf, self.scaler = self.__get_classifier_and_scaler() 75 | 76 | def __get_classifier_and_scaler(self): 77 | """ 78 | Gets the classifier and scaler needed for the rest of the operations. Loads from cache if 79 | load_saved is set to true. 80 | """ 81 | if self.load_saved: 82 | print('Loading saved classifier and scaler...') 83 | svm_clf = pickle.load( open( "./cache/" + self.svm_clf_name, "rb")) if self.svm_enable else None 84 | rf_clf = pickle.load( open( "./cache/" + self.rf_clf_name, "rb")) if self.rf_enable else None 85 | scaler = pickle.load(open( "./cache/" + self.scaler_name, "rb")) 86 | 87 | np.set_printoptions(suppress=True) 88 | np.set_printoptions(precision=6, floatmode='fixed') 89 | else: 90 | # Split up data into randomized training and test sets 91 | print('Training...') 92 | rand_state = np.random.randint(0, 100) 93 | 94 | notred_features, red_features, filenames = self.__get_features() 95 | scaled_X, y, scaler, mean, std = self.__get_scaled_X_y(notred_features, red_features) 96 | 97 | test_size = 0.05 98 | X_train, X_test, y_train, y_test = train_test_split( 99 | scaled_X, y, test_size=test_size, random_state=rand_state) 100 | # Check the training time for the SVC 101 | svm_clf = None 102 | if self.svm_enable: 103 | t=time.time() 104 | gscv = self.svm_grid_search 105 | gscv.fit(X_train, y_train) 106 | t2 = time.time() 107 | print(round(t2-t, 2), 'Seconds to train CLF...') 108 | # Extract best estimator 109 | svm_clf = gscv.best_estimator_ 110 | print('Grid Search is finished, search result of C =', svm_clf.C) 111 | self.__test_classifier(svm_clf, X_test, y_test, scaled_X, filenames, rand_state) 112 | 113 | rf_clf = None 114 | if self.rf_enable: 115 | t=time.time() 116 | gscv = self.forest_grid_search 117 | print(X_train, y_train) 118 | gscv.fit(X_train, y_train) 119 | # Extract best estimator 120 | rf_clf = gscv.best_estimator_ 121 | print('Best parameters: {}'.format(gscv.best_params_)) 122 | t2 = time.time() 123 | print(round(t2-t, 2), 'Seconds to train CLF...') 124 | self.__test_classifier(rf_clf, X_test, y_test, scaled_X, filenames, rand_state) 125 | 126 | 127 | 128 | print('Pickling classifier and scaler...') 129 | if self.svm_enable: 130 | pickle.dump(svm_clf, open( "./cache/" + self.svm_clf_name, "wb" )) 131 | if self.rf_enable: 132 | pickle.dump(rf_clf, open( "./cache/" + self.rf_clf_name, "wb" )) 133 | 134 | pickle.dump(scaler, open( "./cache/" + self.scaler_name, "wb" ) ) 135 | pickle.dump(mean, open( "./cache/" + self.scaler_mean_name, "wb")) 136 | pickle.dump(std, open( "./cache/" + self.scaler_std_name, "wb")) 137 | 138 | return svm_clf, rf_clf, scaler 139 | 140 | def __test_classifier(self, clf, X_test, y_test, scaled_X, filenames, rand_state): 141 | # Check the score of the Classifier 142 | if isinstance(clf, LinearSVC): 143 | preds = 1/(1+(np.exp(-1*clf.decision_function(X_test)))) 144 | elif isinstance(clf, RandomForestClassifier): 145 | preds = list(map(lambda x: x[1] / (x[0] + x[1]), clf.predict_proba(X_test))) 146 | else: 147 | pass 148 | #get filename 149 | test_filenames = shuffle(filenames, random_state=rand_state)[:len(preds)] 150 | print(len(test_filenames), len(preds)) 151 | for i, proba in enumerate(preds): 152 | correct = False 153 | ans = -1 154 | ans_proba = 0 155 | if proba < 0.5: 156 | ans = 0 157 | ans_proba = (1.0-proba)*100.0 158 | else: 159 | ans = 1 160 | ans_proba = proba * 100.0 161 | if ans == y_test[i]: 162 | correct = True 163 | if correct == False: 164 | print('\033[31mproba = {}, predict = {}, answer = {}, testcase = {}\033[0m'.format(round(ans_proba, 3), ans, int(y_test[i]), test_filenames[i])) 165 | else: 166 | print('proba = {}, predict = {}, answer = {}, testcase = {}'.format(round(ans_proba, 3), ans, int(y_test[i]), test_filenames[i])) 167 | 168 | def __get_features(self): 169 | """ 170 | Gets features either by loading them from cache, or by extracting them from the data. 171 | """ 172 | if self.load_features: 173 | print('Loading saved features...') 174 | notred_features, red_features, filenames = pickle.load( open( "./cache/features.p", "rb" ) ) 175 | 176 | else: 177 | # print("Extracting features from %s samples..." % self.sample_size) 178 | print("Extracting features...") 179 | 180 | notreds = [] 181 | reds = [] 182 | filenames = [] 183 | 184 | for folder in self.negative_data_folders: 185 | image_paths =glob.glob(folder+'/*') 186 | for path in image_paths: 187 | notreds.append(path) 188 | for folder in self.positive_data_folders: 189 | image_paths =glob.glob(folder+'/*') 190 | for path in image_paths: 191 | reds.append(path) 192 | 193 | #same input data num for RF 194 | #sample_size = min(len(notreds), len(reds)) 195 | #notreds = notreds[0:sample_size] 196 | #reds = reds[0:sample_size] 197 | 198 | filenames.extend(notreds) 199 | filenames.extend(reds) 200 | print("netative input data num : ", len(notreds)) 201 | print("positive input data num : ", len(reds)) 202 | start = time.clock() 203 | notred_features = self.__extract_features(notreds) 204 | red_features = self.__extract_features(reds) 205 | 206 | 207 | end = time.clock() 208 | print("Running time : %s seconds" % (end - start)) 209 | 210 | print('Pickling features...') 211 | pickle.dump((notred_features, red_features, filenames), open( "./cache/features.p", "wb" )) 212 | 213 | return notred_features, red_features, filenames 214 | 215 | def __extract_features(self, imgs): 216 | """ 217 | Extract features from image files. 218 | """ 219 | 220 | # Create a list to append feature vectors to 221 | features = [] 222 | # Iterate through the list of images 223 | for file in imgs: 224 | # Read in each one by one 225 | image = cv2.imread(file) 226 | # Get features for one image 227 | file_features = self.__single_img_features(image) 228 | #Append to master list 229 | features.append(file_features) 230 | # Return list of feature vectors 231 | return features 232 | 233 | 234 | def __single_img_features(self, img): 235 | img = cv2.resize(img, (128, 64), cv2.INTER_LINEAR) 236 | 237 | """ 238 | Define a function to extract features from a single image window 239 | This function is very similar to extract_features() 240 | just for a single image rather than list of images 241 | Define a function to extract features from a single image window 242 | This function is very similar to extract_features() 243 | just for a single image rather than list of images 244 | """ 245 | #1) Define an empty list to receive features 246 | img_features = [] 247 | #2) Apply color conversion if other than 'RGB' 248 | 249 | # hls = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)# convert it to HLS 250 | gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 251 | 252 | #3) Compute spatial features if flag is set 253 | if self.spatial_feat == True: 254 | # spatial_hls = self.__bin_spatial(hls) 255 | spatial_rgb = self.__bin_spatial(img) 256 | spatial_hsv = cv2.cvtColor(spatial_rgb, cv2.COLOR_BGR2HSV) 257 | img_features.append(spatial_hsv.ravel()/256) 258 | img_features.append(spatial_rgb.ravel()/256) 259 | 260 | #7) Compute HOG features if flag is set 261 | if self.hog_feat == True: 262 | 263 | hog_features = self.__get_hog_features(gray) 264 | #8) Append features to list 265 | img_features.append(hog_features) 266 | 267 | #9) Return concatenated array of img_features 268 | return np.concatenate(img_features) 269 | 270 | def singleimgfeatures(self, img): 271 | return self.__single_img_features(img) 272 | 273 | def __get_scaled_X_y(self, notred_features, red_features): 274 | X = np.vstack((notred_features, red_features)).astype(np.float64) 275 | # Fit a per-column scaler 276 | X_scaler = StandardScaler().fit(X) 277 | # Apply the scaler to X 278 | scaled_X = X_scaler.transform(X) 279 | 280 | # Define the labels vector 281 | y = np.hstack((np.zeros(len(notred_features)), np.ones(len(red_features)))) 282 | 283 | # Pickle scaler parameter 284 | mean = np.nanmean(np.array(X), axis=0) 285 | std = np.nanstd(np.array(X), axis=0) 286 | 287 | return scaled_X, y, X_scaler, mean, std 288 | 289 | # Define a function to return HOG features and visualization 290 | def __get_hog_features(self, img, vis=False, feature_vec=True): 291 | return myhog.myhog(img) 292 | 293 | 294 | # Define a function to compute binned color features 295 | def __bin_spatial(self, img): 296 | features = cv2.resize(img, self.spatial_size, cv2.INTER_LINEAR) 297 | return features 298 | 299 | # Define a function to extract features from a list of images 300 | # Have this function call bin_spatial() and color_hist() 301 | def predictoneimage(self, img): 302 | # test_image = cv2.resize(img, (128, 64), cv2.INTER_LINEAR) 303 | test_image = cv2.resize(img, (128, 64), cv2.INTER_LINEAR) 304 | features = self.__single_img_features(test_image) 305 | test_features = self.scaler.transform(np.array(features).reshape(1, -1)) 306 | # prediction = self.trained_svm.predict_proba(test_features)[:,1] 307 | svm_prediction = 0 308 | rf_prediction = 0 309 | if self.svm_enable: 310 | bias = self.trained_svm.intercept_ 311 | dot = np.dot(self.trained_svm.coef_[0], test_features[0]) 312 | rst = dot + bias 313 | sigmoided_rst = 1/(1+np.exp(-1*rst)) 314 | print(dot.shape) 315 | print("rst:", rst, "sigmoid:", sigmoided_rst) 316 | svm_prediction = 1/(1+(np.exp(-1*self.trained_svm.decision_function(test_features)))) 317 | if self.rf_enable: 318 | rf_prediction = self.trained_rf.predict_proba(test_features)[:,1] 319 | return svm_prediction, rf_prediction 320 | -------------------------------------------------------------------------------- /python/vdtools.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/python/vdtools.pyc -------------------------------------------------------------------------------- /util/README.md: -------------------------------------------------------------------------------- 1 | # util 2 | ## ap_fixed_convert 3 | This utility works on Vivado HLS. 4 | It expresses the trained weight in 32-bit fixed point and convert it to the value interpreted as 32-bit unsigned int notation. 5 | - main.cpp 6 | dummy code. This Vivado HLS project is just conversion utility. 7 | - main_tb.cpp 8 | simple conversion code 9 | - weight.h 10 | declare trained weight array genarated by `python/myestimator.py` 11 | 12 | The generated JSON file is like the following. JSON file is used in `sw/` project to set trained parameter to BRAM. 13 | ``` 14 | {"bgrhsv_w1" : [286925, 414125, ...], 15 | "bgrhsv_w2" : [4294678382, 4294426082, ...], 16 | "bgrhsv_w3" : [136128, 4294683817, ...], 17 | "hog_w1" : [535309, 4293005267, ...], 18 | "hog_w2" : [93835, 4293342059, ...], 19 | "hog_w3" : [4288671221, 4293277452, ...} 20 | ``` -------------------------------------------------------------------------------- /util/ap_fixed_convert/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | using namespace std; 4 | 5 | void dummy(){ 6 | 7 | } 8 | -------------------------------------------------------------------------------- /util/ap_fixed_convert/main_tb.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include "weights.h" 4 | #include 5 | #include 6 | #include 7 | 8 | using namespace std; 9 | 10 | #define INPUT_WIDTH 128 11 | #define INPUT_HEIGHT 64 12 | #define CELL_SIZE 8 13 | #define BLOCK_SIZE 2 14 | #define CELL_NUM_WIDTH (INPUT_WIDTH/CELL_SIZE) 15 | #define CELL_NUM_HEIGHT (INPUT_HEIGHT/CELL_SIZE) 16 | #define BLOCK_NUM_WIDTH (CELL_NUM_WIDTH-1) 17 | #define BLOCK_NUM_HEIGHT (CELL_NUM_HEIGHT-1) 18 | 19 | unsigned int convFixedToUint(ap_fixed<32, 10> val){ 20 | unsigned int res = 0; 21 | string str = val.to_string(2, false); 22 | str = str.substr(2, str.length() - 2); 23 | unsigned int tmp = 1; 24 | for(int i = str.length() - 1; i >= 0; i--){ 25 | if(str[i] == '.') continue; 26 | 27 | if(str[i] == '1'){ 28 | res += tmp; 29 | } 30 | tmp = tmp << 1; 31 | } 32 | return res; 33 | } 34 | 35 | float reverseUIntRepresentToFloat(unsigned int val){ 36 | float res = 0; 37 | for(int i = 31; i >= 0; i--){ 38 | unsigned int tmp = 1 << i; 39 | //cout << tmp << endl; 40 | if(val > tmp){ 41 | if(i == 31) res -= 512; 42 | else if(i >= 22) res += (1 << (i - 22)); 43 | else res += pow(0.5, (22 - i)); 44 | val -= tmp; 45 | } 46 | } 47 | return res; 48 | } 49 | 50 | string to_string(unsigned int val){ 51 | stringstream ss; 52 | ss << val; 53 | return ss.str(); 54 | } 55 | 56 | string toJson(string name, vector arr, bool commadisable = false){ 57 | string rst = ""; 58 | rst += "\"" + name + "\" : ["; 59 | for(int i = 0; i < 63; i++){ 60 | rst += to_string(arr[i]); 61 | if(i != 62) rst += ", "; 62 | } 63 | if(commadisable) rst += "]\n"; 64 | else rst += "],\n"; 65 | return rst; 66 | } 67 | 68 | struct bgr{ 69 | double u_b; 70 | double u_g; 71 | double u_r; 72 | 73 | double b_b; 74 | double b_g; 75 | double b_r; 76 | }; 77 | 78 | struct hsv{ 79 | ap_fixed<32,10> u_h; 80 | ap_fixed<32,10> u_s; 81 | ap_fixed<32,10> u_v; 82 | 83 | ap_fixed<32,10> b_h; 84 | ap_fixed<32,10> b_s; 85 | ap_fixed<32,10> b_v; 86 | }; 87 | 88 | struct hogblock{ 89 | ap_fixed<32,10> ul[9]; 90 | ap_fixed<32,10> ur[9]; 91 | ap_fixed<32,10> bl[9]; 92 | ap_fixed<32,10> br[9]; 93 | }; 94 | bgr bgrweight[32]; 95 | hsv hsvweight[32]; 96 | hogblock hogweight[BLOCK_NUM_HEIGHT*BLOCK_NUM_WIDTH]; 97 | 98 | string joinstr(vector arr){ 99 | string rst = ""; 100 | for(int i = 0; i < arr.size(); i++){ 101 | rst += arr[i]; 102 | if(i != arr.size() - 1) rst += ", "; 103 | } 104 | return rst; 105 | } 106 | 107 | int main(){ 108 | int cnt = 0; 109 | for(int y = 0; y < 8; y++){ 110 | for(int x = 0; x < 8; x++){ 111 | if(y % 2 == 0){ 112 | hsvweight[(y/2)*8+x].u_h = unscaled_weight[cnt++]; 113 | hsvweight[(y/2)*8+x].u_s = unscaled_weight[cnt++]; 114 | hsvweight[(y/2)*8+x].u_v = unscaled_weight[cnt++]; 115 | }else{ 116 | hsvweight[(y/2)*8+x].b_h = unscaled_weight[cnt++]; 117 | hsvweight[(y/2)*8+x].b_s = unscaled_weight[cnt++]; 118 | hsvweight[(y/2)*8+x].b_v = unscaled_weight[cnt++]; 119 | } 120 | 121 | } 122 | } 123 | 124 | for(int y = 0; y < 8; y++){ 125 | for(int x = 0; x < 8; x++){ 126 | if(y % 2 == 0){ 127 | bgrweight[(y/2)*8+x].u_b = unscaled_weight[cnt++]; 128 | bgrweight[(y/2)*8+x].u_g = unscaled_weight[cnt++]; 129 | bgrweight[(y/2)*8+x].u_r = unscaled_weight[cnt++]; 130 | }else{ 131 | bgrweight[(y/2)*8+x].b_b = unscaled_weight[cnt++]; 132 | bgrweight[(y/2)*8+x].b_g = unscaled_weight[cnt++]; 133 | bgrweight[(y/2)*8+x].b_r = unscaled_weight[cnt++]; 134 | } 135 | 136 | } 137 | } 138 | 139 | for(int y = 0; y < BLOCK_NUM_HEIGHT; y++){ 140 | for(int x = 0; x < BLOCK_NUM_WIDTH; x++){ 141 | int index = y * BLOCK_NUM_WIDTH + x; 142 | //ul, ur, bl, br 143 | for(int blocki = 0; blocki < 4; blocki++){ 144 | for(int bini = 0; bini < 9; bini++){ 145 | if(blocki == 0) hogweight[index].ul[bini] = unscaled_weight[cnt++]; 146 | if(blocki == 1) hogweight[index].ur[bini] = unscaled_weight[cnt++]; 147 | if(blocki == 2) hogweight[index].bl[bini] = unscaled_weight[cnt++]; 148 | if(blocki == 3) hogweight[index].br[bini] = unscaled_weight[cnt++]; 149 | } 150 | } 151 | } 152 | } 153 | 154 | vector strs; 155 | 156 | string jsonstr = ""; 157 | jsonstr += "{"; 158 | //bgrhsv_w1 159 | strs.clear(); 160 | jsonstr += "\"bgrhsv_w1\" : ["; 161 | for(int i = 0; i < 8; i++){ 162 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_b))); 163 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_b))); 164 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_h))); 165 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_h))); 166 | 167 | 168 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_g))); 169 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_g))); 170 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_s))); 171 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_s))); 172 | 173 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_r))); 174 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_r))); 175 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_v))); 176 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_v))); 177 | } 178 | jsonstr += joinstr(strs); 179 | jsonstr += "],\n"; 180 | 181 | //bgrhsv_w2 182 | strs.clear(); 183 | jsonstr += "\"bgrhsv_w2\" : ["; 184 | for(int i = 8; i < 16; i++){ 185 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_b))); 186 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_b))); 187 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_h))); 188 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_h))); 189 | 190 | 191 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_g))); 192 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_g))); 193 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_s))); 194 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_s))); 195 | 196 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_r))); 197 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_r))); 198 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_v))); 199 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_v))); 200 | } 201 | jsonstr += joinstr(strs); 202 | jsonstr += "],\n"; 203 | 204 | //bgrhsv_w3 205 | strs.clear(); 206 | jsonstr += "\"bgrhsv_w3\" : ["; 207 | for(int i = 16; i < 24; i++){ 208 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_b))); 209 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_b))); 210 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_h))); 211 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_h))); 212 | 213 | 214 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_g))); 215 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_g))); 216 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_s))); 217 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_s))); 218 | 219 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_r))); 220 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_r))); 221 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_v))); 222 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_v))); 223 | } 224 | jsonstr += joinstr(strs); 225 | jsonstr += "],\n"; 226 | 227 | //bgrhsv_w4 228 | strs.clear(); 229 | jsonstr += "\"bgrhsv_w4\" : ["; 230 | for(int i = 24; i < 32; i++){ 231 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_b))); 232 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_b))); 233 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_h))); 234 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_h))); 235 | 236 | 237 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_g))); 238 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_g))); 239 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_s))); 240 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_s))); 241 | 242 | strs.push_back(to_string(convFixedToUint(bgrweight[i].b_r))); 243 | strs.push_back(to_string(convFixedToUint(bgrweight[i].u_r))); 244 | strs.push_back(to_string(convFixedToUint(hsvweight[i].b_v))); 245 | strs.push_back(to_string(convFixedToUint(hsvweight[i].u_v))); 246 | } 247 | jsonstr += joinstr(strs); 248 | jsonstr += "],\n"; 249 | 250 | //hog_w1 251 | for(int k = 0; k < BLOCK_NUM_HEIGHT; k++){ 252 | strs.clear(); 253 | jsonstr += "\"hog_w" + to_string(k+1) + "\" : ["; 254 | for(int i = BLOCK_NUM_WIDTH * k; i < BLOCK_NUM_WIDTH + BLOCK_NUM_WIDTH * k; i++){ 255 | for(int j = 0; j < 9; j++){ 256 | strs.push_back(to_string(convFixedToUint(hogweight[i].br[j]))); 257 | strs.push_back(to_string(convFixedToUint(hogweight[i].bl[j]))); 258 | strs.push_back(to_string(convFixedToUint(hogweight[i].ur[j]))); 259 | strs.push_back(to_string(convFixedToUint(hogweight[i].ul[j]))); 260 | } 261 | } 262 | jsonstr += joinstr(strs); 263 | jsonstr += "],\n"; 264 | } 265 | 266 | //end 267 | jsonstr += "}"; 268 | 269 | cout << jsonstr << endl; 270 | } 271 | -------------------------------------------------------------------------------- /util/ap_fixed_convert/solution1/directives.tcl: -------------------------------------------------------------------------------- 1 | ############################################################ 2 | ## This file is generated automatically by Vivado HLS. 3 | ## Please DO NOT edit it. 4 | ## Copyright (C) 1986-2017 Xilinx, Inc. All Rights Reserved. 5 | ############################################################ 6 | -------------------------------------------------------------------------------- /util/ap_fixed_convert/solution1/script.tcl: -------------------------------------------------------------------------------- 1 | ############################################################ 2 | ## This file is generated automatically by Vivado HLS. 3 | ## Please DO NOT edit it. 4 | ## Copyright (C) 1986-2017 Xilinx, Inc. All Rights Reserved. 5 | ############################################################ 6 | open_project ap_fixed_convert 7 | set_top main 8 | add_files ap_fixed_convert/main.cpp 9 | add_files -tb ap_fixed_convert/main_tb.cpp 10 | add_files -tb ap_fixed_convert/weights.h 11 | open_solution "solution1" 12 | set_part {xqku115-rlf1924-2-i} -tool vivado 13 | create_clock -period 10 -name default 14 | #source "./ap_fixed_convert/solution1/directives.tcl" 15 | csim_design -compiler gcc 16 | csynth_design 17 | cosim_design 18 | export_design -format ip_catalog 19 | -------------------------------------------------------------------------------- /util/ap_fixed_convert/vivado_hls.app: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | -------------------------------------------------------------------------------- /vivado/circuit.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lp6m/ImageDetectionHW2/5144ebbbbe3d8ea7b5e5a604f39be626e0b81f54/vivado/circuit.png -------------------------------------------------------------------------------- /vivado/create_project.tcl: -------------------------------------------------------------------------------- 1 | # 2 | # create_project.tcl Tcl script for creating project 3 | # 4 | set project_name "ultra96_design" 5 | set root_directory [file join [file dirname [info script]] ".."] 6 | set project_directory [file join $root_directory $project_name] 7 | set board_part [get_board_parts -quiet -latest_file_version "*ultra96v1*"] 8 | set device_part "xczu3eg-sbva484-1-e" 9 | set design_bd_tcl_file [file normalize [file join $root_directory "./vivado/create_bd.tcl" ]] 10 | set design_pin_xdc_file [file normalize [file join $root_directory "./vivado/Ultra96_constraints_180318.xdc" ]] 11 | lappend ip_repo_path_list [file join $root_directory "hls"] 12 | 13 | # 14 | # Create project 15 | # 16 | create_project -force $project_name $project_directory 17 | # 18 | # Set project properties 19 | # 20 | if {[info exists board_part ] && [string equal $board_part "" ] == 0} { 21 | set_property "board_part" $board_part [current_project] 22 | } elseif {[info exists device_part] && [string equal $device_part "" ] == 0} { 23 | set_property "part" $device_part [current_project] 24 | } else { 25 | puts "ERROR: Please set board_part or device_part." 26 | return 1 27 | } 28 | set_property "default_lib" "xil_defaultlib" [current_project] 29 | set_property "simulator_language" "Mixed" [current_project] 30 | set_property "target_language" "Verilog" [current_project] 31 | # 32 | # Create fileset "sources_1" 33 | # 34 | if {[string equal [get_filesets -quiet sources_1] ""]} { 35 | create_fileset -srcset sources_1 36 | } 37 | # 38 | # Create fileset "constrs_1" 39 | # 40 | if {[string equal [get_filesets -quiet constrs_1] ""]} { 41 | create_fileset -constrset constrs_1 42 | } 43 | # 44 | # Create fileset "sim_1" 45 | # 46 | if {[string equal [get_filesets -quiet sim_1] ""]} { 47 | create_fileset -simset sim_1 48 | } 49 | # 50 | # Create run "synth_1" and set property 51 | # 52 | set synth_1_flow "Vivado Synthesis 2018" 53 | set synth_1_strategy "Vivado Synthesis Defaults" 54 | if {[string equal [get_runs -quiet synth_1] ""]} { 55 | create_run -name synth_1 -flow $synth_1_flow -strategy $synth_1_strategy -constrset constrs_1 56 | } else { 57 | set_property flow $synth_1_flow [get_runs synth_1] 58 | set_property strategy $synth_1_strategy [get_runs synth_1] 59 | } 60 | current_run -synthesis [get_runs synth_1] 61 | # 62 | # Create run "impl_1" and set property 63 | # 64 | set impl_1_flow "Vivado Implementation 2018" 65 | set impl_1_strategy "Vivado Implementation Defaults" 66 | if {[string equal [get_runs -quiet impl_1] ""]} { 67 | create_run -name impl_1 -flow $impl_1_flow -strategy $impl_1_strategy -constrset constrs_1 -parent_run synth_1 68 | } else { 69 | set_property flow $impl_1_flow [get_runs impl_1] 70 | set_property strategy $impl_1_strategy [get_runs impl_1] 71 | } 72 | current_run -implementation [get_runs impl_1] 73 | # 74 | # Set IP Repository 75 | # 76 | if {[info exists ip_repo_path_list] && [llength $ip_repo_path_list] > 0 } { 77 | set_property ip_repo_paths $ip_repo_path_list [current_fileset] 78 | update_ip_catalog 79 | } 80 | # 81 | # Create block design 82 | # 83 | if {[info exists design_bd_tcl_file]} { 84 | 85 | # 86 | # Set -dir argument value of create_bd_design 87 | # 88 | set ::origin_dir_loc [file join $project_directory $project_name.srcs] 89 | # 90 | # Read block design file 91 | # 92 | source $design_bd_tcl_file 93 | # 94 | # Save block design 95 | # 96 | regenerate_bd_layout 97 | save_bd_design 98 | # 99 | # Generate wrapper files 100 | # 101 | set design_bd_name [get_bd_designs] 102 | make_wrapper -files [get_files $design_bd_name.bd] -top -import 103 | } 104 | # 105 | # Import timing files 106 | # 107 | if {[info exists design_timing_xdc_file]} { 108 | add_files -fileset constrs_1 -norecurse $design_timing_xdc_file 109 | } 110 | # 111 | # Import pin files 112 | # 113 | if {[info exists design_pin_xdc_file]} { 114 | add_files -fileset constrs_1 -norecurse $design_pin_xdc_file 115 | } 116 | --------------------------------------------------------------------------------