├── LICENSE ├── README.md ├── cpp ├── CMakeLists.txt ├── build-linux_RK3588.sh ├── include │ ├── GrallocOps.h │ ├── RgaApi.h │ ├── RgaMutex.h │ ├── RgaSingleton.h │ ├── RgaUtils.h │ ├── RockchipRga.h │ ├── drm_func.h │ ├── drmrga.h │ ├── im2d.h │ ├── im2d.hpp │ ├── im2d_hardware.h │ ├── im2d_type.h │ ├── im2d_version.h │ ├── postprocess.h │ ├── preprocess.h │ ├── rga.h │ ├── rga_func.h │ ├── rknn_api.h │ ├── rknn_matmul_api.h │ └── yolov5.hpp ├── lib │ ├── librga.so │ ├── librknn_api.so │ └── librknnrt.so └── src │ ├── main.cc │ ├── postprocess.cc │ └── preprocess.cc ├── font └── NotoSansCJK-Regular.otf ├── imgs ├── double_yellow.jpg ├── new.jpg ├── out.jpg ├── result.jpg ├── test.jpg ├── 京A25016_32.jpg └── 沪A5L320.jpg ├── python ├── plate_detection_rec_rknnlite.py ├── plate_rec_rknnlite.py ├── postprocess_numpy.py └── utils.py └── weights ├── car_plate_detect_1x3x80x80x16-640-640_rm_transpose_rk3588.rknn ├── plate.txt └── plate_recognition_color-168-48_rm_transpose_rk3588.rknn /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # RK3588 车牌检测与识别 RKNPU推理 2 | 3 | ## 介绍 4 | 基于[yolov5-face-RK3588](https://github.com/YaoQ/yolov5-face-RK3588),在RK3588上运行[car recognition](https://github.com/we0091234/Car_recognition)检测模型,实现车辆+车牌检测,并输出车牌的4个landmark点,用于车牌识别之前的矫正. 5 | 6 | 7 | ## 环境依赖 8 | - RK3588 Debian11 9 | - rknntoolit 1.6.0 10 | - rknnnpu 1.6.0 11 | - opencv4+ 12 | 13 | ## 模型 14 | python 使用的模型尺寸 15 | ``` 16 | 输入： 1 3 640 640 17 | 输出： 18 | 1 3 80 80 16 19 | 1 3 40 40 16 20 | 1 3 20 20 16 21 | ``` 22 | 23 | ## 后处理之前的模型输出结构说明 24 | 16位分别对应: [x, y, w, h , conf, x1, y1, x2, y2, x3, y3, x4, y4, cls0, cls1, cls2] 25 | 其中检测标签: 26 | - cls0: 单层车牌 27 | - cls1: 双层车牌 28 | - cls2: 车 29 | 30 | 31 | ## python推理测试 32 | 33 | 实现了车牌检测, 车牌矫正,双层和单层车牌的预处理以及车牌识别 34 | 35 | ```bash 36 | # python 37 | cd python 38 | python plate_detection_rec_rknnlite.py --input ../imgs/double_yellow.jpg 39 | 40 | ``` 41 | 42 | ![result](imgs/result.jpg) 43 | 44 | ## C++代码推理测试 45 | 46 | ```bash 47 | cd cpp 48 | bash build-linux_RK3588.sh # 在RK3588上直接编译 49 | cd build 50 | ./rknn_yolov5_demo ../../weights/car_plate_detect_1x3x80x80x16-640-640_rm_transpose_rk3588.rknn ../../imgs/沪A5L320.jpg 51 | 52 | # 运行结果: 53 | post process config: box_conf_threshold = 0.25, nms_threshold = 0.45 54 | Loading mode... 55 | sdk version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11) driver version: 0.9.2 56 | model input num: 1, output num: 3 57 | index=0, name=input, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, w_stride = 640, size_with_stride=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 58 | index=0, name=output, n_dims=5, dims=[1, 3, 80, 80, 16], n_elems=307200, size=307200, w_stride = 0, size_with_stride=307200, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=63, scale=0.176689 59 | index=1, name=975, n_dims=5, dims=[1, 3, 40, 40, 16], n_elems=76800, size=76800, w_stride = 0, size_with_stride=76800, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=44, scale=0.161176 60 | index=2, name=987, n_dims=5, dims=[1, 3, 20, 20, 16], n_elems=19200, size=19200, w_stride = 0, size_with_stride=19200, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=84, scale=0.329176 61 | model is NHWC input fmt 62 | model input height=640, width=640, channel=3 63 | Read ../../imgs/沪A5L320.jpg ... 64 | img width = 1920, img height = 1080 65 | resize image with letterbox 66 | once run use 35.621000 ms 67 | loadLabelName ../../weights/plate.txt 68 | car @ (465 279 1059 735) 0.875814 69 | Single-L @ (762 672 882 720) 0.867393 70 | save detect result to ./out.jpg 71 | loop count = 10 , average run 29.781800 ms 72 | ``` 73 | 74 | ![detection result](./imgs/out.jpg) 75 | 76 | ## 模型转换说明 77 | 在导出onnx模型时，由于输出的尺度不一样，特别说明一下，截断得到3输出，尺寸是(1, 3, 80, 80, 16) 78 | 79 | [Car_recognition/yolo.py](https://github.com/we0091234/Car_recognition/blob/master/models/yolo.py) 80 | 81 | ``` 82 | 45 def forward(self, x): 83 | 46 # x = x.copy() # for profiling 84 | 47 z = [] # inference output 85 | 48 if self.export_cat: 86 | 49 for i in range(self.nl): 87 | 50 x[i] = self.m[i](x[i]) # conv 88 | 51 bs, _, ny, nx = x[i].shape # x(bs,255,20,20) to x(bs,3,20,20,85) 89 | 52 90 | 53 x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous() 91 | 54 return x 92 | 93 | ``` 94 | 备注： 95 | 如果需要导出其他的尺度，需要修改52行代码，将view的尺寸修改为相应的要求来适配：(bs, self.na, self.no, ny, nx) 96 | 97 | 98 | # 参考 99 | - https://github.com/we0091234/Car_recognition 100 | -------------------------------------------------------------------------------- /cpp/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 3.4.1) 2 | 3 | project(rknn_yolov5_demo) 4 | 5 | set(CMAKE_CXX_STANDARD 11) 6 | set(CMAKE_CXX_STANDARD_REQUIRED ON) 7 | 8 | # skip 3rd-party lib dependencies 9 | set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--allow-shlib-undefined") 10 | 11 | # install target and libraries 12 | set(CMAKE_INSTALL_PREFIX ${CMAKE_SOURCE_DIR}/install/rknn_yolov5_demo_${CMAKE_SYSTEM_NAME}) 13 | 14 | set(CMAKE_SKIP_INSTALL_RPATH FALSE) 15 | set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE) 16 | set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib") 17 | 18 | # rknn , rga 19 | link_directories(${CMAKE_CURRENT_SOURCE_DIR}/lib) 20 | include_directories(${CMake_CURRENT_SOURCE_DIR}/include) 21 | 22 | # opencv 23 | find_package(OpenCV REQUIRED) 24 | 25 | 26 | # rknn_yolov5_demo 27 | include_directories( ${CMAKE_SOURCE_DIR}/include) 28 | 29 | add_executable(rknn_yolov5_demo 30 | src/main.cc 31 | src/postprocess.cc 32 | src/preprocess.cc 33 | ) 34 | 35 | target_link_libraries(rknn_yolov5_demo 36 | ${OpenCV_LIBS} 37 | rknnrt 38 | rknn_api 39 | rga 40 | ) 41 | 42 | 43 | # install target and libraries 44 | set(CMAKE_INSTALL_PREFIX ${CMAKE_SOURCE_DIR}/install) 45 | install(TARGETS rknn_yolov5_demo DESTINATION . ) 46 | install(DIRECTORY model DESTINATION .) 47 | -------------------------------------------------------------------------------- /cpp/build-linux_RK3588.sh: -------------------------------------------------------------------------------- 1 | set -e 2 | 3 | TARGET_SOC="rk3588" 4 | GCC_COMPILER=aarch64-linux 5 | 6 | export LD_LIBRARY_PATH=${TOOL_CHAIN}/lib64:$LD_LIBRARY_PATH 7 | export CC=gcc 8 | export CXX=g++ 9 | 10 | ROOT_PWD=$( cd "$( dirname $0 )" && cd -P "$( dirname "$SOURCE" )" && pwd ) 11 | 12 | # build 13 | BUILD_DIR=${ROOT_PWD}/build/ 14 | 15 | if [[ ! -d "${BUILD_DIR}" ]]; then 16 | mkdir -p ${BUILD_DIR} 17 | fi 18 | 19 | cd ${BUILD_DIR} 20 | cmake .. -DCMAKE_SYSTEM_NAME=Linux -DTARGET_SOC=${TARGET_SOC} 21 | make -j4 22 | -------------------------------------------------------------------------------- /cpp/include/GrallocOps.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _rk_graphic_buffer_h_ 20 | #define _rk_graphic_buffer_h_ 21 | 22 | #ifdef ANDROID 23 | 24 | #include 25 | #include 26 | #include 27 | 28 | #include 29 | 30 | #include 31 | 32 | #include 33 | #include 34 | #include 35 | #include 36 | #include 37 | #include 38 | 39 | #include 40 | #include 41 | 42 | #include 43 | #include 44 | #include 45 | #include 46 | #include 47 | 48 | #include "drmrga.h" 49 | #include "rga.h" 50 | 51 | // ------------------------------------------------------------------------------- 52 | int RkRgaGetHandleFd(buffer_handle_t handle, int *fd); 53 | int RkRgaGetHandleAttributes(buffer_handle_t handle, 54 | std::vector *attrs); 55 | int RkRgaGetHandleMapAddress(buffer_handle_t handle, 56 | void **buf); 57 | #endif //Android 58 | 59 | #endif //_rk_graphic_buffer_h_ 60 | -------------------------------------------------------------------------------- /cpp/include/RgaApi.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | #ifndef _rockchip_rga_c_h_ 19 | #define _rockchip_rga_c_h_ 20 | 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include 31 | #include 32 | 33 | #include "drmrga.h" 34 | #include "rga.h" 35 | 36 | #ifdef __cplusplus 37 | extern "C"{ 38 | #endif 39 | 40 | /* 41 | * Compatible with the old version of C interface.The new 42 | * version of the C interface no longer requires users to 43 | * initialize rga, so RgaInit and RgaDeInit are just for 44 | * compatibility with the old C interface, so please do 45 | * not use ctx, because it is usually a NULL. 46 | */ 47 | #define RgaInit(ctx) ({ \ 48 | int ret = 0; \ 49 | ret = c_RkRgaInit(); \ 50 | c_RkRgaGetContext(ctx); \ 51 | ret;\ 52 | }) 53 | #define RgaDeInit(ctx) { \ 54 | (void)ctx; /* unused */ \ 55 | c_RkRgaDeInit(); \ 56 | } 57 | #define RgaBlit(...) c_RkRgaBlit(__VA_ARGS__) 58 | #define RgaCollorFill(...) c_RkRgaColorFill(__VA_ARGS__) 59 | #define RgaFlush() c_RkRgaFlush() 60 | 61 | int c_RkRgaInit(); 62 | void c_RkRgaDeInit(); 63 | void c_RkRgaGetContext(void **ctx); 64 | int c_RkRgaBlit(rga_info_t *src, rga_info_t *dst, rga_info_t *src1); 65 | int c_RkRgaColorFill(rga_info_t *dst); 66 | int c_RkRgaFlush(); 67 | 68 | #ifndef ANDROID /* linux */ 69 | int c_RkRgaGetAllocBuffer(bo_t *bo_info, int width, int height, int bpp); 70 | int c_RkRgaGetAllocBufferCache(bo_t *bo_info, int width, int height, int bpp); 71 | int c_RkRgaGetMmap(bo_t *bo_info); 72 | int c_RkRgaUnmap(bo_t *bo_info); 73 | int c_RkRgaFree(bo_t *bo_info); 74 | int c_RkRgaGetBufferFd(bo_t *bo_info, int *fd); 75 | #endif /* #ifndef ANDROID */ 76 | 77 | #ifdef __cplusplus 78 | } 79 | #endif 80 | 81 | #endif /* #ifndef _rockchip_rga_c_h_ */ 82 | -------------------------------------------------------------------------------- /cpp/include/RgaMutex.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2020 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * PutinLee 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | 20 | #ifndef _LIBS_RGA_MUTEX_H 21 | #define _LIBS_RGA_MUTEX_H 22 | 23 | #ifndef ANDROID 24 | #include 25 | #include 26 | #include 27 | 28 | #include 29 | 30 | 31 | // Enable thread safety attributes only with clang. 32 | // The attributes can be safely erased when compiling with other compilers. 33 | #if defined(__clang__) && (!defined(SWIG)) 34 | #define THREAD_ANNOTATION_ATTRIBUTE__(x) __attribute__((x)) 35 | #else 36 | #define THREAD_ANNOTATION_ATTRIBUTE__(x) // no-op 37 | #endif 38 | 39 | #define CAPABILITY(x) THREAD_ANNOTATION_ATTRIBUTE__(capability(x)) 40 | 41 | #define SCOPED_CAPABILITY THREAD_ANNOTATION_ATTRIBUTE__(scoped_lockable) 42 | 43 | #define GUARDED_BY(x) THREAD_ANNOTATION_ATTRIBUTE__(guarded_by(x)) 44 | 45 | #define PT_GUARDED_BY(x) THREAD_ANNOTATION_ATTRIBUTE__(pt_guarded_by(x)) 46 | 47 | #define ACQUIRED_BEFORE(...) THREAD_ANNOTATION_ATTRIBUTE__(acquired_before(__VA_ARGS__)) 48 | 49 | #define ACQUIRED_AFTER(...) THREAD_ANNOTATION_ATTRIBUTE__(acquired_after(__VA_ARGS__)) 50 | 51 | #define REQUIRES(...) THREAD_ANNOTATION_ATTRIBUTE__(requires_capability(__VA_ARGS__)) 52 | 53 | #define REQUIRES_SHARED(...) THREAD_ANNOTATION_ATTRIBUTE__(requires_shared_capability(__VA_ARGS__)) 54 | 55 | #define ACQUIRE(...) THREAD_ANNOTATION_ATTRIBUTE__(acquire_capability(__VA_ARGS__)) 56 | 57 | #define ACQUIRE_SHARED(...) THREAD_ANNOTATION_ATTRIBUTE__(acquire_shared_capability(__VA_ARGS__)) 58 | 59 | #define RELEASE(...) THREAD_ANNOTATION_ATTRIBUTE__(release_capability(__VA_ARGS__)) 60 | 61 | #define RELEASE_SHARED(...) THREAD_ANNOTATION_ATTRIBUTE__(release_shared_capability(__VA_ARGS__)) 62 | 63 | #define TRY_ACQUIRE(...) THREAD_ANNOTATION_ATTRIBUTE__(try_acquire_capability(__VA_ARGS__)) 64 | 65 | #define TRY_ACQUIRE_SHARED(...) \ 66 | THREAD_ANNOTATION_ATTRIBUTE__(try_acquire_shared_capability(__VA_ARGS__)) 67 | 68 | #define EXCLUDES(...) THREAD_ANNOTATION_ATTRIBUTE__(locks_excluded(__VA_ARGS__)) 69 | 70 | #define ASSERT_CAPABILITY(x) THREAD_ANNOTATION_ATTRIBUTE__(assert_capability(x)) 71 | 72 | #define ASSERT_SHARED_CAPABILITY(x) THREAD_ANNOTATION_ATTRIBUTE__(assert_shared_capability(x)) 73 | 74 | #define RETURN_CAPABILITY(x) THREAD_ANNOTATION_ATTRIBUTE__(lock_returned(x)) 75 | 76 | #define NO_THREAD_SAFETY_ANALYSIS THREAD_ANNOTATION_ATTRIBUTE__(no_thread_safety_analysis) 77 | 78 | class Condition; 79 | 80 | /* 81 | * NOTE: This class is for code that builds on Win32. Its usage is 82 | * deprecated for code which doesn't build for Win32. New code which 83 | * doesn't build for Win32 should use std::mutex and std::lock_guard instead. 84 | * 85 | * Simple mutex class. The implementation is system-dependent. 86 | * 87 | * The mutex must be unlocked by the thread that locked it. They are not 88 | * recursive, i.e. the same thread can't lock it multiple times. 89 | */ 90 | class CAPABILITY("mutex") Mutex { 91 | public: 92 | enum { 93 | PRIVATE = 0, 94 | SHARED = 1 95 | }; 96 | 97 | Mutex(); 98 | explicit Mutex(const char* name); 99 | explicit Mutex(int type, const char* name = nullptr); 100 | ~Mutex(); 101 | 102 | // lock or unlock the mutex 103 | int32_t lock() ACQUIRE(); 104 | void unlock() RELEASE(); 105 | 106 | // lock if possible; returns 0 on success, error otherwise 107 | int32_t tryLock() TRY_ACQUIRE(0); 108 | 109 | int32_t timedLock(int64_t timeoutNs) TRY_ACQUIRE(0); 110 | 111 | // Manages the mutex automatically. It'll be locked when Autolock is 112 | // constructed and released when Autolock goes out of scope. 113 | class SCOPED_CAPABILITY Autolock { 114 | public: 115 | inline explicit Autolock(Mutex& mutex) ACQUIRE(mutex) : mLock(mutex) { 116 | mLock.lock(); 117 | } 118 | inline explicit Autolock(Mutex* mutex) ACQUIRE(mutex) : mLock(*mutex) { 119 | mLock.lock(); 120 | } 121 | inline ~Autolock() RELEASE() { 122 | mLock.unlock(); 123 | } 124 | 125 | private: 126 | Mutex& mLock; 127 | // Cannot be copied or moved - declarations only 128 | Autolock(const Autolock&); 129 | Autolock& operator=(const Autolock&); 130 | }; 131 | 132 | private: 133 | friend class Condition; 134 | 135 | // A mutex cannot be copied 136 | Mutex(const Mutex&); 137 | Mutex& operator=(const Mutex&); 138 | 139 | pthread_mutex_t mMutex; 140 | }; 141 | 142 | // --------------------------------------------------------------------------- 143 | inline Mutex::Mutex() { 144 | pthread_mutex_init(&mMutex, nullptr); 145 | } 146 | inline Mutex::Mutex(__attribute__((unused)) const char* name) { 147 | pthread_mutex_init(&mMutex, nullptr); 148 | } 149 | inline Mutex::Mutex(int type, __attribute__((unused)) const char* name) { 150 | if (type == SHARED) { 151 | pthread_mutexattr_t attr; 152 | pthread_mutexattr_init(&attr); 153 | pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED); 154 | pthread_mutex_init(&mMutex, &attr); 155 | pthread_mutexattr_destroy(&attr); 156 | } else { 157 | pthread_mutex_init(&mMutex, nullptr); 158 | } 159 | } 160 | inline Mutex::~Mutex() { 161 | pthread_mutex_destroy(&mMutex); 162 | } 163 | inline int32_t Mutex::lock() { 164 | return -pthread_mutex_lock(&mMutex); 165 | } 166 | inline void Mutex::unlock() { 167 | pthread_mutex_unlock(&mMutex); 168 | } 169 | inline int32_t Mutex::tryLock() { 170 | return -pthread_mutex_trylock(&mMutex); 171 | } 172 | inline int32_t Mutex::timedLock(int64_t timeoutNs) { 173 | timespec now; 174 | clock_gettime(CLOCK_REALTIME, &now); 175 | timeoutNs += now.tv_sec*1000000000 + now.tv_nsec; 176 | const struct timespec ts = { 177 | /* .tv_sec = */ static_cast(timeoutNs / 1000000000), 178 | /* .tv_nsec = */ static_cast(timeoutNs % 1000000000), 179 | }; 180 | return -pthread_mutex_timedlock(&mMutex, &ts); 181 | } 182 | 183 | // --------------------------------------------------------------------------- 184 | 185 | /* 186 | * Automatic mutex. Declare one of these at the top of a function. 187 | * When the function returns, it will go out of scope, and release the 188 | * mutex. 189 | */ 190 | 191 | typedef Mutex::Autolock AutoMutex; 192 | #endif // __ANDROID_VNDK__ 193 | #endif // _LIBS_RGA_MUTEX_H 194 | -------------------------------------------------------------------------------- /cpp/include/RgaSingleton.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _LIBS_RGA_SINGLETON_H 20 | #define _LIBS_RGA_SINGLETON_H 21 | 22 | #ifndef ANDROID 23 | #include "RgaMutex.h" 24 | 25 | #if defined(__clang__) 26 | #pragma clang diagnostic push 27 | #pragma clang diagnostic ignored "-Wundefined-var-template" 28 | #endif 29 | 30 | template 31 | class Singleton { 32 | public: 33 | static TYPE& getInstance() { 34 | Mutex::Autolock _l(sLock); 35 | TYPE* instance = sInstance; 36 | if (instance == nullptr) { 37 | instance = new TYPE(); 38 | sInstance = instance; 39 | } 40 | return *instance; 41 | } 42 | 43 | static bool hasInstance() { 44 | Mutex::Autolock _l(sLock); 45 | return sInstance != nullptr; 46 | } 47 | 48 | protected: 49 | ~Singleton() { } 50 | Singleton() { } 51 | 52 | private: 53 | Singleton(const Singleton&); 54 | Singleton& operator = (const Singleton&); 55 | static Mutex sLock; 56 | static TYPE* sInstance; 57 | }; 58 | 59 | #if defined(__clang__) 60 | #pragma clang diagnostic pop 61 | #endif 62 | 63 | #define RGA_SINGLETON_STATIC_INSTANCE(TYPE) \ 64 | template<> ::Mutex \ 65 | (::Singleton< TYPE >::sLock)(::Mutex::PRIVATE); \ 66 | template<> TYPE* ::Singleton< TYPE >::sInstance(nullptr); /* NOLINT */ \ 67 | template class ::Singleton< TYPE >; 68 | 69 | #endif //ANDROID 70 | #endif //_LIBS_RGA_SINGLETON_H 71 | -------------------------------------------------------------------------------- /cpp/include/RgaUtils.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _rga_utils_h_ 20 | #define _rga_utils_h_ 21 | 22 | // ------------------------------------------------------------------------------- 23 | float get_bpp_from_format(int format); 24 | int get_perPixel_stride_from_format(int format); 25 | int get_buf_from_file(void *buf, int f, int sw, int sh, int index); 26 | int output_buf_data_to_file(void *buf, int f, int sw, int sh, int index); 27 | const char *translate_format_str(int format); 28 | int get_buf_from_file_FBC(void *buf, int f, int sw, int sh, int index); 29 | int output_buf_data_to_file_FBC(void *buf, int f, int sw, int sh, int index); 30 | #endif 31 | 32 | -------------------------------------------------------------------------------- /cpp/include/RockchipRga.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _rockchip_rga_h_ 20 | #define _rockchip_rga_h_ 21 | 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | 33 | #include "drmrga.h" 34 | #include "GrallocOps.h" 35 | #include "RgaUtils.h" 36 | #include "rga.h" 37 | 38 | ////////////////////////////////////////////////////////////////////////////////// 39 | #ifndef ANDROID 40 | #include "RgaSingleton.h" 41 | #endif 42 | 43 | #ifdef ANDROID 44 | #include 45 | #include 46 | #include 47 | 48 | namespace android { 49 | #endif 50 | 51 | class RockchipRga :public Singleton { 52 | public: 53 | 54 | static inline RockchipRga& get() { 55 | return getInstance(); 56 | } 57 | 58 | int RkRgaInit(); 59 | void RkRgaDeInit(); 60 | void RkRgaGetContext(void **ctx); 61 | #ifndef ANDROID /* LINUX */ 62 | int RkRgaAllocBuffer(int drm_fd /* input */, bo_t *bo_info, 63 | int width, int height, int bpp, int flags); 64 | int RkRgaFreeBuffer(int drm_fd /* input */, bo_t *bo_info); 65 | int RkRgaGetAllocBuffer(bo_t *bo_info, int width, int height, int bpp); 66 | int RkRgaGetAllocBufferExt(bo_t *bo_info, int width, int height, int bpp, int flags); 67 | int RkRgaGetAllocBufferCache(bo_t *bo_info, int width, int height, int bpp); 68 | int RkRgaGetMmap(bo_t *bo_info); 69 | int RkRgaUnmap(bo_t *bo_info); 70 | int RkRgaFree(bo_t *bo_info); 71 | int RkRgaGetBufferFd(bo_t *bo_info, int *fd); 72 | #else 73 | int RkRgaGetBufferFd(buffer_handle_t handle, int *fd); 74 | int RkRgaGetHandleMapCpuAddress(buffer_handle_t handle, void **buf); 75 | #endif 76 | int RkRgaBlit(rga_info *src, rga_info *dst, rga_info *src1); 77 | int RkRgaCollorFill(rga_info *dst); 78 | int RkRgaCollorPalette(rga_info *src, rga_info *dst, rga_info *lut); 79 | int RkRgaFlush(); 80 | 81 | 82 | void RkRgaSetLogOnceFlag(int log) { 83 | mLogOnce = log; 84 | } 85 | void RkRgaSetAlwaysLogFlag(bool log) { 86 | mLogAlways = log; 87 | } 88 | void RkRgaLogOutRgaReq(struct rga_req rgaReg); 89 | int RkRgaLogOutUserPara(rga_info *rgaInfo); 90 | inline bool RkRgaIsReady() { 91 | return mSupportRga; 92 | } 93 | 94 | RockchipRga(); 95 | ~RockchipRga(); 96 | private: 97 | bool mSupportRga; 98 | int mLogOnce; 99 | int mLogAlways; 100 | void * mContext; 101 | 102 | friend class Singleton; 103 | }; 104 | 105 | #ifdef ANDROID 106 | }; // namespace android 107 | #endif 108 | 109 | #endif 110 | 111 | -------------------------------------------------------------------------------- /cpp/include/drm_func.h: -------------------------------------------------------------------------------- 1 | #ifndef __DRM_FUNC_H__ 2 | #define __DRM_FUNC_H__ 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include // open function 8 | #include // close function 9 | #include 10 | #include 11 | 12 | 13 | #include 14 | #include "libdrm/drm_fourcc.h" 15 | #include "xf86drm.h" 16 | 17 | #ifdef __cplusplus 18 | extern "C" { 19 | #endif 20 | 21 | typedef int (* FUNC_DRM_IOCTL)(int fd, unsigned long request, void *arg); 22 | 23 | typedef struct _drm_context{ 24 | void *drm_handle; 25 | FUNC_DRM_IOCTL io_func; 26 | } drm_context; 27 | 28 | /* memory type definitions. */ 29 | enum drm_rockchip_gem_mem_type 30 | { 31 | /* Physically Continuous memory and used as default. */ 32 | ROCKCHIP_BO_CONTIG = 1 << 0, 33 | /* cachable mapping. */ 34 | ROCKCHIP_BO_CACHABLE = 1 << 1, 35 | /* write-combine mapping. */ 36 | ROCKCHIP_BO_WC = 1 << 2, 37 | ROCKCHIP_BO_SECURE = 1 << 3, 38 | ROCKCHIP_BO_MASK = ROCKCHIP_BO_CONTIG | ROCKCHIP_BO_CACHABLE | 39 | ROCKCHIP_BO_WC | ROCKCHIP_BO_SECURE 40 | }; 41 | 42 | int drm_init(drm_context *drm_ctx); 43 | 44 | void* drm_buf_alloc(drm_context *drm_ctx,int drm_fd, int TexWidth, int TexHeight,int bpp,int *fd,unsigned int *handle,size_t *actual_size); 45 | 46 | int drm_buf_destroy(drm_context *drm_ctx,int drm_fd,int buf_fd, int handle,void *drm_buf,size_t size); 47 | 48 | void drm_deinit(drm_context *drm_ctx, int drm_fd); 49 | 50 | #ifdef __cplusplus 51 | } 52 | #endif 53 | #endif /*__DRM_FUNC_H__*/ -------------------------------------------------------------------------------- /cpp/include/drmrga.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _rk_drm_rga_ 20 | #define _rk_drm_rga_ 21 | 22 | #include 23 | #include 24 | #include 25 | 26 | #include "rga.h" 27 | 28 | #ifdef ANDROID 29 | #define DRMRGA_HARDWARE_MODULE_ID "librga" 30 | 31 | #include 32 | #include 33 | #include 34 | #include 35 | 36 | #ifdef ANDROID_12 37 | #include 38 | #endif 39 | 40 | #endif 41 | 42 | #ifndef ANDROID /* LINUX */ 43 | /* flip source image horizontally (around the vertical axis) */ 44 | #define HAL_TRANSFORM_FLIP_H 0x01 45 | /* flip source image vertically (around the horizontal axis)*/ 46 | #define HAL_TRANSFORM_FLIP_V 0x02 47 | /* rotate source image 90 degrees clockwise */ 48 | #define HAL_TRANSFORM_ROT_90 0x04 49 | /* rotate source image 180 degrees */ 50 | #define HAL_TRANSFORM_ROT_180 0x03 51 | /* rotate source image 270 degrees clockwise */ 52 | #define HAL_TRANSFORM_ROT_270 0x07 53 | #endif 54 | 55 | #define HAL_TRANSFORM_FLIP_H_V 0x08 56 | 57 | /*****************************************************************************/ 58 | 59 | /* for compatibility */ 60 | #define DRM_RGA_MODULE_API_VERSION HWC_MODULE_API_VERSION_0_1 61 | #define DRM_RGA_DEVICE_API_VERSION HWC_DEVICE_API_VERSION_0_1 62 | #define DRM_RGA_API_VERSION HWC_DEVICE_API_VERSION 63 | 64 | #define DRM_RGA_TRANSFORM_ROT_MASK 0x0000000F 65 | #define DRM_RGA_TRANSFORM_ROT_0 0x00000000 66 | #define DRM_RGA_TRANSFORM_ROT_90 HAL_TRANSFORM_ROT_90 67 | #define DRM_RGA_TRANSFORM_ROT_180 HAL_TRANSFORM_ROT_180 68 | #define DRM_RGA_TRANSFORM_ROT_270 HAL_TRANSFORM_ROT_270 69 | 70 | #define DRM_RGA_TRANSFORM_FLIP_MASK 0x00000003 71 | #define DRM_RGA_TRANSFORM_FLIP_H HAL_TRANSFORM_FLIP_H 72 | #define DRM_RGA_TRANSFORM_FLIP_V HAL_TRANSFORM_FLIP_V 73 | 74 | enum { 75 | AWIDTH = 0, 76 | AHEIGHT, 77 | ASTRIDE, 78 | AFORMAT, 79 | ASIZE, 80 | ATYPE, 81 | }; 82 | /*****************************************************************************/ 83 | 84 | #ifndef ANDROID 85 | /* memory type definitions. */ 86 | enum drm_rockchip_gem_mem_type { 87 | /* Physically Continuous memory and used as default. */ 88 | ROCKCHIP_BO_CONTIG = 1 << 0, 89 | /* cachable mapping. */ 90 | ROCKCHIP_BO_CACHABLE = 1 << 1, 91 | /* write-combine mapping. */ 92 | ROCKCHIP_BO_WC = 1 << 2, 93 | ROCKCHIP_BO_SECURE = 1 << 3, 94 | ROCKCHIP_BO_MASK = ROCKCHIP_BO_CONTIG | ROCKCHIP_BO_CACHABLE | 95 | ROCKCHIP_BO_WC | ROCKCHIP_BO_SECURE 96 | }; 97 | 98 | typedef struct bo { 99 | int fd; 100 | void *ptr; 101 | size_t size; 102 | size_t offset; 103 | size_t pitch; 104 | unsigned handle; 105 | } bo_t; 106 | #endif 107 | 108 | /* 109 | @value size: user not need care about.For avoid read/write out of memory 110 | */ 111 | typedef struct rga_rect { 112 | int xoffset; 113 | int yoffset; 114 | int width; 115 | int height; 116 | int wstride; 117 | int hstride; 118 | int format; 119 | int size; 120 | } rga_rect_t; 121 | 122 | typedef struct rga_nn { 123 | int nn_flag; 124 | int scale_r; 125 | int scale_g; 126 | int scale_b; 127 | int offset_r; 128 | int offset_g; 129 | int offset_b; 130 | } rga_nn_t; 131 | 132 | typedef struct rga_dither { 133 | int enable; 134 | int mode; 135 | int lut0_l; 136 | int lut0_h; 137 | int lut1_l; 138 | int lut1_h; 139 | } rga_dither_t; 140 | 141 | /* 142 | @value fd: use fd to share memory, it can be ion shard fd,and dma fd. 143 | @value virAddr:userspace address 144 | @value phyAddr:use phy address 145 | @value hnd: use buffer_handle_t 146 | */ 147 | typedef struct rga_info { 148 | int fd; 149 | void *virAddr; 150 | void *phyAddr; 151 | #ifndef ANDROID /* LINUX */ 152 | unsigned hnd; 153 | #else /* Android */ 154 | buffer_handle_t hnd; 155 | #endif 156 | int format; 157 | rga_rect_t rect; 158 | unsigned int blend; 159 | int bufferSize; 160 | int rotation; 161 | int color; 162 | int testLog; 163 | int mmuFlag; 164 | int colorkey_en; 165 | int colorkey_mode; 166 | int colorkey_max; 167 | int colorkey_min; 168 | int scale_mode; 169 | int color_space_mode; 170 | int sync_mode; 171 | rga_nn_t nn; 172 | rga_dither_t dither; 173 | int rop_code; 174 | int rd_mode; 175 | unsigned short is_10b_compact; 176 | unsigned short is_10b_endian; 177 | 178 | int in_fence_fd; 179 | int out_fence_fd; 180 | 181 | int core; 182 | int priority; 183 | 184 | unsigned short enable; 185 | 186 | int handle; 187 | 188 | struct rga_mosaic_info mosaic_info; 189 | 190 | struct rga_osd_info osd_info; 191 | 192 | struct rga_pre_intr_info pre_intr; 193 | 194 | int mpi_mode; 195 | int ctx_id; 196 | 197 | char reserve[402]; 198 | } rga_info_t; 199 | 200 | 201 | typedef struct drm_rga { 202 | rga_rect_t src; 203 | rga_rect_t dst; 204 | } drm_rga_t; 205 | 206 | /* 207 | @fun rga_set_rect:For use to set the rects esayly 208 | 209 | @param rect:The rect user want to set,like setting the src rect: 210 | drm_rga_t rects; 211 | rga_set_rect(rects.src,0,0,1920,1080,1920,NV12); 212 | mean to set the src rect to the value. 213 | */ 214 | static inline int rga_set_rect(rga_rect_t *rect, 215 | int x, int y, int w, int h, int sw, int sh, int f) { 216 | if (!rect) 217 | return -EINVAL; 218 | 219 | rect->xoffset = x; 220 | rect->yoffset = y; 221 | rect->width = w; 222 | rect->height = h; 223 | rect->wstride = sw; 224 | rect->hstride = sh; 225 | rect->format = f; 226 | 227 | return 0; 228 | } 229 | 230 | #ifndef ANDROID /* LINUX */ 231 | static inline void rga_set_rotation(rga_info_t *info, int angle) { 232 | if (angle == 90) 233 | info->rotation = HAL_TRANSFORM_ROT_90; 234 | else if (angle == 180) 235 | info->rotation = HAL_TRANSFORM_ROT_180; 236 | else if (angle == 270) 237 | info->rotation = HAL_TRANSFORM_ROT_270; 238 | } 239 | #endif 240 | /*****************************************************************************/ 241 | 242 | #endif 243 | -------------------------------------------------------------------------------- /cpp/include/im2d.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2020 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * PutinLee 5 | * Cerf Yu 6 | * 7 | * Licensed under the Apache License, Version 2.0 (the "License"); 8 | * you may not use this file except in compliance with the License. 9 | * You may obtain a copy of the License at 10 | * 11 | * http://www.apache.org/licenses/LICENSE-2.0 12 | * 13 | * Unless required by applicable law or agreed to in writing, software 14 | * distributed under the License is distributed on an "AS IS" BASIS, 15 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | * See the License for the specific language governing permissions and 17 | * limitations under the License. 18 | */ 19 | 20 | #ifndef _im2d_h_ 21 | #define _im2d_h_ 22 | 23 | #ifdef __cplusplus 24 | extern "C" { 25 | #endif 26 | 27 | #include "im2d_version.h" 28 | #include "im2d_type.h" 29 | 30 | #ifndef IM_API 31 | #define IM_API /* define API export as needed */ 32 | #endif 33 | 34 | #define RGA_GET_MIN(n1, n2) ((n1) < (n2) ? (n1) : (n2)) 35 | 36 | /* 37 | * @return error message string 38 | */ 39 | #define imStrError(...) \ 40 | ({ \ 41 | const char* im2d_api_err; \ 42 | int __args[] = {__VA_ARGS__}; \ 43 | int __argc = sizeof(__args)/sizeof(int); \ 44 | if (__argc == 0) { \ 45 | im2d_api_err = imStrError_t(IM_STATUS_INVALID_PARAM); \ 46 | } else if (__argc == 1){ \ 47 | im2d_api_err = imStrError_t((IM_STATUS)__args[0]); \ 48 | } else { \ 49 | im2d_api_err = ("Fatal error, imStrError() too many parameters\n"); \ 50 | printf("Fatal error, imStrError() too many parameters\n"); \ 51 | } \ 52 | im2d_api_err; \ 53 | }) 54 | IM_API const char* imStrError_t(IM_STATUS status); 55 | 56 | /* 57 | * Import external buffers into RGA driver. 58 | * 59 | * @param fd/va/pa 60 | * Select dma_fd/virtual_address/physical_address by buffer type 61 | * @param param 62 | * Configure buffer parameters 63 | * 64 | * @return rga_buffer_handle_t 65 | */ 66 | IM_API rga_buffer_handle_t importbuffer_fd(int fd, im_handle_param_t *param); 67 | IM_API rga_buffer_handle_t importbuffer_virtualaddr(void *va, im_handle_param_t *param); 68 | IM_API rga_buffer_handle_t importbuffer_physicaladdr(uint64_t pa, im_handle_param_t *param); 69 | 70 | /* 71 | * Import external buffers into RGA driver. 72 | * 73 | * @param handle 74 | * rga buffer handle 75 | * 76 | * @return success or else negative error code. 77 | */ 78 | IM_API IM_STATUS releasebuffer_handle(rga_buffer_handle_t handle); 79 | 80 | /* 81 | * Wrap image Parameters. 82 | * 83 | * @param handle 84 | * RGA buffer handle. 85 | * @param width 86 | * Width of image manipulation area. 87 | * @param height 88 | * Height of image manipulation area. 89 | * @param wstride 90 | * Width pixel stride, default (width = wstride). 91 | * @param hstride 92 | * Height pixel stride, default (height = hstride). 93 | * @param format 94 | * Image format. 95 | * 96 | * @return rga_buffer_t 97 | */ 98 | #define wrapbuffer_handle(handle, width, height, format, ...) \ 99 | ({ \ 100 | rga_buffer_t im2d_api_buffer; \ 101 | int __args[] = {__VA_ARGS__}; \ 102 | int __argc = sizeof(__args)/sizeof(int); \ 103 | if (__argc == 0) { \ 104 | im2d_api_buffer = wrapbuffer_handle_t(handle, width, height, width, height, format); \ 105 | } else if (__argc == 2){ \ 106 | im2d_api_buffer = wrapbuffer_handle_t(handle, width, height, __args[0], __args[1], format); \ 107 | } else { \ 108 | printf("invalid parameter\n"); \ 109 | } \ 110 | im2d_api_buffer; \ 111 | }) 112 | IM_API rga_buffer_t wrapbuffer_handle_t(rga_buffer_handle_t handle, int width, int height, int wstride, int hstride, int format); 113 | 114 | /* For legarcy. */ 115 | #define wrapbuffer_virtualaddr(vir_addr, width, height, format, ...) \ 116 | ({ \ 117 | rga_buffer_t im2d_api_buffer; \ 118 | int __args[] = {__VA_ARGS__}; \ 119 | int __argc = sizeof(__args)/sizeof(int); \ 120 | if (__argc == 0) { \ 121 | im2d_api_buffer = wrapbuffer_virtualaddr_t(vir_addr, width, height, width, height, format); \ 122 | } else if (__argc == 2){ \ 123 | im2d_api_buffer = wrapbuffer_virtualaddr_t(vir_addr, width, height, __args[0], __args[1], format); \ 124 | } else { \ 125 | printf("invalid parameter\n"); \ 126 | } \ 127 | im2d_api_buffer; \ 128 | }) 129 | 130 | #define wrapbuffer_physicaladdr(phy_addr, width, height, format, ...) \ 131 | ({ \ 132 | rga_buffer_t im2d_api_buffer; \ 133 | int __args[] = {__VA_ARGS__}; \ 134 | int __argc = sizeof(__args)/sizeof(int); \ 135 | if (__argc == 0) { \ 136 | im2d_api_buffer = wrapbuffer_physicaladdr_t(phy_addr, width, height, width, height, format); \ 137 | } else if (__argc == 2){ \ 138 | im2d_api_buffer = wrapbuffer_physicaladdr_t(phy_addr, width, height, __args[0], __args[1], format); \ 139 | } else { \ 140 | printf("invalid parameter\n"); \ 141 | } \ 142 | im2d_api_buffer; \ 143 | }) 144 | 145 | #define wrapbuffer_fd(fd, width, height, format, ...) \ 146 | ({ \ 147 | rga_buffer_t im2d_api_buffer; \ 148 | int __args[] = {__VA_ARGS__}; \ 149 | int __argc = sizeof(__args)/sizeof(int); \ 150 | if (__argc == 0) { \ 151 | im2d_api_buffer = wrapbuffer_fd_t(fd, width, height, width, height, format); \ 152 | } else if (__argc == 2){ \ 153 | im2d_api_buffer = wrapbuffer_fd_t(fd, width, height, __args[0], __args[1], format); \ 154 | } else { \ 155 | printf("invalid parameter\n"); \ 156 | } \ 157 | im2d_api_buffer; \ 158 | }) 159 | IM_API rga_buffer_t wrapbuffer_virtualaddr_t(void* vir_addr, int width, int height, int wstride, int hstride, int format); 160 | IM_API rga_buffer_t wrapbuffer_physicaladdr_t(void* phy_addr, int width, int height, int wstride, int hstride, int format); 161 | IM_API rga_buffer_t wrapbuffer_fd_t(int fd, int width, int height, int wstride, int hstride, int format); 162 | 163 | /* 164 | * Query RGA basic information, supported resolution, supported format, etc. 165 | * 166 | * @param name 167 | * RGA_VENDOR 168 | * RGA_VERSION 169 | * RGA_MAX_INPUT 170 | * RGA_MAX_OUTPUT 171 | * RGA_INPUT_FORMAT 172 | * RGA_OUTPUT_FORMAT 173 | * RGA_EXPECTED 174 | * RGA_ALL 175 | * 176 | * @returns a string describing properties of RGA. 177 | */ 178 | IM_API const char* querystring(int name); 179 | 180 | /* 181 | * check RGA basic information, supported resolution, supported format, etc. 182 | * 183 | * @param src 184 | * @param dst 185 | * @param src_rect 186 | * @param dst_rect 187 | * @param mode_usage 188 | * 189 | * @returns no error or else negative error code. 190 | */ 191 | #define imcheck(src, dst, src_rect, dst_rect, ...) \ 192 | ({ \ 193 | IM_STATUS __ret = IM_STATUS_NOERROR; \ 194 | rga_buffer_t __pat; \ 195 | im_rect __pat_rect; \ 196 | memset(&__pat, 0, sizeof(rga_buffer_t)); \ 197 | memset(&__pat_rect, 0, sizeof(im_rect)); \ 198 | int __args[] = {__VA_ARGS__}; \ 199 | int __argc = sizeof(__args)/sizeof(int); \ 200 | if (__argc == 0) { \ 201 | rga_check_perpare((rga_buffer_t *)(&src), (rga_buffer_t *)(&dst), (rga_buffer_t *)(&__pat), \ 202 | (im_rect *)(&src_rect), (im_rect *)(&dst_rect), (im_rect *)(&__pat_rect), 0); \ 203 | __ret = imcheck_t(src, dst, __pat, src_rect, dst_rect, __pat_rect, 0); \ 204 | } else if (__argc == 1){ \ 205 | rga_check_perpare((rga_buffer_t *)(&src), (rga_buffer_t *)(&dst), (rga_buffer_t *)(&__pat), \ 206 | (im_rect *)(&src_rect), (im_rect *)(&dst_rect), (im_rect *)(&__pat_rect), __args[0]); \ 207 | __ret = imcheck_t(src, dst, __pat, src_rect, dst_rect, __pat_rect, __args[0]); \ 208 | } else { \ 209 | __ret = IM_STATUS_FAILED; \ 210 | printf("check failed\n"); \ 211 | } \ 212 | __ret; \ 213 | }) 214 | #define imcheck_composite(src, dst, pat, src_rect, dst_rect, pat_rect, ...) \ 215 | ({ \ 216 | IM_STATUS __ret = IM_STATUS_NOERROR; \ 217 | int __args[] = {__VA_ARGS__}; \ 218 | int __argc = sizeof(__args)/sizeof(int); \ 219 | if (__argc == 0) { \ 220 | rga_check_perpare((rga_buffer_t *)(&src), (rga_buffer_t *)(&dst), (rga_buffer_t *)(&pat), \ 221 | (im_rect *)(&src_rect), (im_rect *)(&dst_rect), (im_rect *)(&pat_rect), 0); \ 222 | __ret = imcheck_t(src, dst, pat, src_rect, dst_rect, pat_rect, 0); \ 223 | } else if (__argc == 1){ \ 224 | rga_check_perpare((rga_buffer_t *)(&src), (rga_buffer_t *)(&dst), (rga_buffer_t *)(&pat), \ 225 | (im_rect *)(&src_rect), (im_rect *)(&dst_rect), (im_rect *)(&pat_rect), __args[0]); \ 226 | __ret = imcheck_t(src, dst, pat, src_rect, dst_rect, pat_rect, __args[0]); \ 227 | } else { \ 228 | __ret = IM_STATUS_FAILED; \ 229 | printf("check failed\n"); \ 230 | } \ 231 | __ret; \ 232 | }) 233 | IM_API void rga_check_perpare(rga_buffer_t *src, rga_buffer_t *dst, rga_buffer_t *pat, 234 | im_rect *src_rect, im_rect *dst_rect, im_rect *pat_rect, int mode_usage); 235 | IM_API IM_STATUS imcheck_t(const rga_buffer_t src, const rga_buffer_t dst, const rga_buffer_t pat, 236 | const im_rect src_rect, const im_rect dst_rect, const im_rect pat_rect, const int mode_usage); 237 | 238 | /* 239 | * Resize 240 | * 241 | * @param src 242 | * @param dst 243 | * @param fx 244 | * @param fy 245 | * @param interpolation 246 | * @param sync 247 | * wait until operation complete 248 | * 249 | * @returns success or else negative error code. 250 | */ 251 | #define imresize(src, dst, ...) \ 252 | ({ \ 253 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 254 | double __args[] = {__VA_ARGS__}; \ 255 | int __argc = sizeof(__args)/sizeof(double); \ 256 | if (__argc == 0) { \ 257 | __ret = imresize_t(src, dst, 0, 0, INTER_LINEAR, 1); \ 258 | } else if (__argc == 2){ \ 259 | __ret = imresize_t(src, dst, __args[RGA_GET_MIN(__argc, 0)], __args[RGA_GET_MIN(__argc, 1)], INTER_LINEAR, 1); \ 260 | } else if (__argc == 3){ \ 261 | __ret = imresize_t(src, dst, __args[RGA_GET_MIN(__argc, 0)], __args[RGA_GET_MIN(__argc, 1)], (int)__args[RGA_GET_MIN(__argc, 2)], 1); \ 262 | } else if (__argc == 4){ \ 263 | __ret = imresize_t(src, dst, __args[RGA_GET_MIN(__argc, 0)], __args[RGA_GET_MIN(__argc, 1)], (int)__args[RGA_GET_MIN(__argc, 2)], (int)__args[RGA_GET_MIN(__argc, 3)]); \ 264 | } else { \ 265 | __ret = IM_STATUS_INVALID_PARAM; \ 266 | printf("invalid parameter\n"); \ 267 | } \ 268 | __ret; \ 269 | }) 270 | 271 | #define impyramid(src, dst, direction) \ 272 | imresize_t(src, \ 273 | dst, \ 274 | direction == IM_UP_SCALE ? 0.5 : 2, \ 275 | direction == IM_UP_SCALE ? 0.5 : 2, \ 276 | INTER_LINEAR, 1) 277 | 278 | IM_API IM_STATUS imresize_t(const rga_buffer_t src, rga_buffer_t dst, double fx, double fy, int interpolation, int sync); 279 | 280 | /* 281 | * Crop 282 | * 283 | * @param src 284 | * @param dst 285 | * @param rect 286 | * @param sync 287 | * wait until operation complete 288 | * 289 | * @returns success or else negative error code. 290 | */ 291 | #define imcrop(src, dst, rect, ...) \ 292 | ({ \ 293 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 294 | int __args[] = {__VA_ARGS__}; \ 295 | int __argc = sizeof(__args)/sizeof(int); \ 296 | if (__argc == 0) { \ 297 | __ret = imcrop_t(src, dst, rect, 1); \ 298 | } else if (__argc == 1){ \ 299 | __ret = imcrop_t(src, dst, rect, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 300 | } else { \ 301 | __ret = IM_STATUS_INVALID_PARAM; \ 302 | printf("invalid parameter\n"); \ 303 | } \ 304 | __ret; \ 305 | }) 306 | 307 | IM_API IM_STATUS imcrop_t(const rga_buffer_t src, rga_buffer_t dst, im_rect rect, int sync); 308 | 309 | /* 310 | * rotation 311 | * 312 | * @param src 313 | * @param dst 314 | * @param rotation 315 | * IM_HAL_TRANSFORM_ROT_90 316 | * IM_HAL_TRANSFORM_ROT_180 317 | * IM_HAL_TRANSFORM_ROT_270 318 | * @param sync 319 | * wait until operation complete 320 | * 321 | * @returns success or else negative error code. 322 | */ 323 | #define imrotate(src, dst, rotation, ...) \ 324 | ({ \ 325 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 326 | int __args[] = {__VA_ARGS__}; \ 327 | int __argc = sizeof(__args)/sizeof(int); \ 328 | if (__argc == 0) { \ 329 | __ret = imrotate_t(src, dst, rotation, 1); \ 330 | } else if (__argc == 1){ \ 331 | __ret = imrotate_t(src, dst, rotation, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 332 | } else { \ 333 | __ret = IM_STATUS_INVALID_PARAM; \ 334 | printf("invalid parameter\n"); \ 335 | } \ 336 | __ret; \ 337 | }) 338 | 339 | IM_API IM_STATUS imrotate_t(const rga_buffer_t src, rga_buffer_t dst, int rotation, int sync); 340 | 341 | /* 342 | * flip 343 | * 344 | * @param src 345 | * @param dst 346 | * @param mode 347 | * IM_HAL_TRANSFORM_FLIP_H 348 | * IM_HAL_TRANSFORM_FLIP_V 349 | * @param sync 350 | * wait until operation complete 351 | * 352 | * @returns success or else negative error code. 353 | */ 354 | #define imflip(src, dst, mode, ...) \ 355 | ({ \ 356 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 357 | int __args[] = {__VA_ARGS__}; \ 358 | int __argc = sizeof(__args)/sizeof(int); \ 359 | if (__argc == 0) { \ 360 | __ret = imflip_t(src, dst, mode, 1); \ 361 | } else if (__argc == 1){ \ 362 | __ret = imflip_t(src, dst, mode, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 363 | } else { \ 364 | __ret = IM_STATUS_INVALID_PARAM; \ 365 | printf("invalid parameter\n"); \ 366 | } \ 367 | __ret; \ 368 | }) 369 | 370 | IM_API IM_STATUS imflip_t (const rga_buffer_t src, rga_buffer_t dst, int mode, int sync); 371 | 372 | /* 373 | * fill/reset/draw 374 | * 375 | * @param src 376 | * @param dst 377 | * @param rect 378 | * @param color 379 | * @param sync 380 | * wait until operation complete 381 | * 382 | * @returns success or else negative error code. 383 | */ 384 | #define imfill(buf, rect, color, ...) \ 385 | ({ \ 386 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 387 | int __args[] = {__VA_ARGS__}; \ 388 | int __argc = sizeof(__args)/sizeof(int); \ 389 | if (__argc == 0) { \ 390 | __ret = imfill_t(buf, rect, color, 1); \ 391 | } else if (__argc == 1){ \ 392 | __ret = imfill_t(buf, rect, color, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 393 | } else { \ 394 | __ret = IM_STATUS_INVALID_PARAM; \ 395 | printf("invalid parameter\n"); \ 396 | } \ 397 | __ret; \ 398 | }) 399 | 400 | #define imreset(buf, rect, color, ...) \ 401 | ({ \ 402 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 403 | int __args[] = {__VA_ARGS__}; \ 404 | int __argc = sizeof(__args)/sizeof(int); \ 405 | if (__argc == 0) { \ 406 | __ret = imfill_t(buf, rect, color, 1); \ 407 | } else if (__argc == 1){ \ 408 | __ret = imfill_t(buf, rect, color, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 409 | } else { \ 410 | __ret = IM_STATUS_INVALID_PARAM; \ 411 | printf("invalid parameter\n"); \ 412 | } \ 413 | __ret; \ 414 | }) 415 | 416 | #define imdraw(buf, rect, color, ...) \ 417 | ({ \ 418 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 419 | int __args[] = {__VA_ARGS__}; \ 420 | int __argc = sizeof(__args)/sizeof(int); \ 421 | if (__argc == 0) { \ 422 | __ret = imfill_t(buf, rect, color, 1); \ 423 | } else if (__argc == 1){ \ 424 | __ret = imfill_t(buf, rect, color, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 425 | } else { \ 426 | __ret = IM_STATUS_INVALID_PARAM; \ 427 | printf("invalid parameter\n"); \ 428 | } \ 429 | __ret; \ 430 | }) 431 | IM_API IM_STATUS imfill_t(rga_buffer_t dst, im_rect rect, int color, int sync); 432 | 433 | /* 434 | * palette 435 | * 436 | * @param src 437 | * @param dst 438 | * @param lut 439 | * @param sync 440 | * wait until operation complete 441 | * 442 | * @returns success or else negative error code. 443 | */ 444 | #define impalette(src, dst, lut, ...) \ 445 | ({ \ 446 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 447 | int __args[] = {__VA_ARGS__}; \ 448 | int __argc = sizeof(__args)/sizeof(int); \ 449 | if (__argc == 0) { \ 450 | __ret = impalette_t(src, dst, lut, 1); \ 451 | } else if (__argc == 1){ \ 452 | __ret = impalette_t(src, dst, lut, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 453 | } else { \ 454 | __ret = IM_STATUS_INVALID_PARAM; \ 455 | printf("invalid parameter\n"); \ 456 | } \ 457 | __ret; \ 458 | }) 459 | IM_API IM_STATUS impalette_t(rga_buffer_t src, rga_buffer_t dst, rga_buffer_t lut, int sync); 460 | 461 | /* 462 | * translate 463 | * 464 | * @param src 465 | * @param dst 466 | * @param x 467 | * @param y 468 | * @param sync 469 | * wait until operation complete 470 | * 471 | * @returns success or else negative error code. 472 | */ 473 | #define imtranslate(src, dst, x, y, ...) \ 474 | ({ \ 475 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 476 | int __args[] = {__VA_ARGS__}; \ 477 | int __argc = sizeof(__args)/sizeof(int); \ 478 | if (__argc == 0) { \ 479 | __ret = imtranslate_t(src, dst, x, y, 1); \ 480 | } else if (__argc == 1){ \ 481 | __ret = imtranslate_t(src, dst, x, y, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 482 | } else { \ 483 | __ret = IM_STATUS_INVALID_PARAM; \ 484 | printf("invalid parameter\n"); \ 485 | } \ 486 | __ret; \ 487 | }) 488 | IM_API IM_STATUS imtranslate_t(const rga_buffer_t src, rga_buffer_t dst, int x, int y, int sync); 489 | 490 | /* 491 | * copy 492 | * 493 | * @param src 494 | * @param dst 495 | * @param sync 496 | * wait until operation complete 497 | * 498 | * @returns success or else negative error code. 499 | */ 500 | #define imcopy(src, dst, ...) \ 501 | ({ \ 502 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 503 | int __args[] = {__VA_ARGS__}; \ 504 | int __argc = sizeof(__args)/sizeof(int); \ 505 | if (__argc == 0) { \ 506 | __ret = imcopy_t(src, dst, 1); \ 507 | } else if (__argc == 1){ \ 508 | __ret = imcopy_t(src, dst, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 509 | } else { \ 510 | __ret = IM_STATUS_INVALID_PARAM; \ 511 | printf("invalid parameter\n"); \ 512 | } \ 513 | __ret; \ 514 | }) 515 | 516 | IM_API IM_STATUS imcopy_t(const rga_buffer_t src, rga_buffer_t dst, int sync); 517 | 518 | /* 519 | * blend (SRC + DST -> DST or SRCA + SRCB -> DST) 520 | * 521 | * @param srcA 522 | * @param srcB can be NULL. 523 | * @param dst 524 | * @param mode 525 | * IM_ALPHA_BLEND_MODE 526 | * @param sync 527 | * wait until operation complete 528 | * 529 | * @returns success or else negative error code. 530 | */ 531 | #define imblend(srcA, dst, ...) \ 532 | ({ \ 533 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 534 | rga_buffer_t srcB; \ 535 | memset(&srcB, 0x00, sizeof(rga_buffer_t)); \ 536 | int __args[] = {__VA_ARGS__}; \ 537 | int __argc = sizeof(__args)/sizeof(int); \ 538 | if (__argc == 0) { \ 539 | __ret = imblend_t(srcA, srcB, dst, IM_ALPHA_BLEND_SRC_OVER, 1); \ 540 | } else if (__argc == 1){ \ 541 | __ret = imblend_t(srcA, srcB, dst, (int)__args[RGA_GET_MIN(__argc, 0)], 1); \ 542 | } else if (__argc == 2){ \ 543 | __ret = imblend_t(srcA, srcB, dst, (int)__args[RGA_GET_MIN(__argc, 0)], (int)__args[RGA_GET_MIN(__argc, 1)]); \ 544 | } else { \ 545 | __ret = IM_STATUS_INVALID_PARAM; \ 546 | printf("invalid parameter\n"); \ 547 | } \ 548 | __ret; \ 549 | }) 550 | #define imcomposite(srcA, srcB, dst, ...) \ 551 | ({ \ 552 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 553 | int __args[] = {__VA_ARGS__}; \ 554 | int __argc = sizeof(__args)/sizeof(int); \ 555 | if (__argc == 0) { \ 556 | __ret = imblend_t(srcA, srcB, dst, IM_ALPHA_BLEND_SRC_OVER, 1); \ 557 | } else if (__argc == 1){ \ 558 | __ret = imblend_t(srcA, srcB, dst, (int)__args[RGA_GET_MIN(__argc, 0)], 1); \ 559 | } else if (__argc == 2){ \ 560 | __ret = imblend_t(srcA, srcB, dst, (int)__args[RGA_GET_MIN(__argc, 0)], (int)__args[RGA_GET_MIN(__argc, 1)]); \ 561 | } else { \ 562 | __ret = IM_STATUS_INVALID_PARAM; \ 563 | printf("invalid parameter\n"); \ 564 | } \ 565 | __ret; \ 566 | }) 567 | IM_API IM_STATUS imblend_t(const rga_buffer_t srcA, const rga_buffer_t srcB, rga_buffer_t dst, int mode, int sync); 568 | 569 | /* 570 | * color key 571 | * 572 | * @param src 573 | * @param dst 574 | * @param colorkey_range 575 | * max color 576 | * min color 577 | * @param sync 578 | * wait until operation complete 579 | * 580 | * @returns success or else negative error code. 581 | */ 582 | #define imcolorkey(src, dst, range, ...) \ 583 | ({ \ 584 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 585 | int __args[] = {__VA_ARGS__}; \ 586 | int __argc = sizeof(__args)/sizeof(int); \ 587 | if (__argc == 0) { \ 588 | __ret = imcolorkey_t(src, dst, range, IM_ALPHA_COLORKEY_NORMAL, 1); \ 589 | } else if (__argc == 1){ \ 590 | __ret = imcolorkey_t(src, dst, range, (int)__args[RGA_GET_MIN(__argc, 0)], 1); \ 591 | } else if (__argc == 2){ \ 592 | __ret = imcolorkey_t(src, dst, range, (int)__args[RGA_GET_MIN(__argc, 0)], (int)__args[RGA_GET_MIN(__argc, 1)]); \ 593 | } else { \ 594 | __ret = IM_STATUS_INVALID_PARAM; \ 595 | printf("invalid parameter\n"); \ 596 | } \ 597 | __ret; \ 598 | }) 599 | IM_API IM_STATUS imcolorkey_t(const rga_buffer_t src, rga_buffer_t dst, im_colorkey_range range, int mode, int sync); 600 | 601 | /* 602 | * format convert 603 | * 604 | * @param src 605 | * @param dst 606 | * @param sfmt 607 | * @param dfmt 608 | * @param mode 609 | * color space mode: IM_COLOR_SPACE_MODE 610 | * @param sync 611 | * wait until operation complete 612 | * 613 | * @returns success or else negative error code. 614 | */ 615 | #define imcvtcolor(src, dst, sfmt, dfmt, ...) \ 616 | ({ \ 617 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 618 | int __args[] = {__VA_ARGS__}; \ 619 | int __argc = sizeof(__args)/sizeof(int); \ 620 | if (__argc == 0) { \ 621 | __ret = imcvtcolor_t(src, dst, sfmt, dfmt, IM_COLOR_SPACE_DEFAULT, 1); \ 622 | } else if (__argc == 1){ \ 623 | __ret = imcvtcolor_t(src, dst, sfmt, dfmt, (int)__args[RGA_GET_MIN(__argc, 0)], 1); \ 624 | } else if (__argc == 2){ \ 625 | __ret = imcvtcolor_t(src, dst, sfmt, dfmt, (int)__args[RGA_GET_MIN(__argc, 0)], (int)__args[RGA_GET_MIN(__argc, 1)]); \ 626 | } else { \ 627 | __ret = IM_STATUS_INVALID_PARAM; \ 628 | printf("invalid parameter\n"); \ 629 | } \ 630 | __ret; \ 631 | }) 632 | 633 | IM_API IM_STATUS imcvtcolor_t(rga_buffer_t src, rga_buffer_t dst, int sfmt, int dfmt, int mode, int sync); 634 | 635 | /* 636 | * nn quantize 637 | * 638 | * @param src 639 | * @param dst 640 | * @param nninfo 641 | * @param sync 642 | * wait until operation complete 643 | * 644 | * @returns success or else negative error code. 645 | */ 646 | #define imquantize(src, dst, nn_info, ...) \ 647 | ({ \ 648 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 649 | int __args[] = {__VA_ARGS__}; \ 650 | int __argc = sizeof(__args)/sizeof(int); \ 651 | if (__argc == 0) { \ 652 | __ret = imquantize_t(src, dst, nn_info, 1); \ 653 | } else if (__argc == 1){ \ 654 | __ret = imquantize_t(src, dst, nn_info, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 655 | } else { \ 656 | __ret = IM_STATUS_INVALID_PARAM; \ 657 | printf("invalid parameter\n"); \ 658 | } \ 659 | __ret; \ 660 | }) 661 | 662 | IM_API IM_STATUS imquantize_t(const rga_buffer_t src, rga_buffer_t dst, im_nn_t nn_info, int sync); 663 | 664 | /* 665 | * ROP 666 | * 667 | * @param src 668 | * @param dst 669 | * @param rop_code 670 | * @param sync 671 | * wait until operation complete 672 | * 673 | * @returns success or else negative error code. 674 | */ 675 | #define imrop(src, dst, rop_code, ...) \ 676 | ({ \ 677 | IM_STATUS __ret = IM_STATUS_SUCCESS; \ 678 | int __args[] = {__VA_ARGS__}; \ 679 | int __argc = sizeof(__args)/sizeof(int); \ 680 | if (__argc == 0) { \ 681 | __ret = imrop_t(src, dst, rop_code, 1); \ 682 | } else if (__argc == 1){ \ 683 | __ret = imrop_t(src, dst, rop_code, (int)__args[RGA_GET_MIN(__argc, 0)]); \ 684 | } else { \ 685 | __ret = IM_STATUS_INVALID_PARAM; \ 686 | printf("invalid parameter\n"); \ 687 | } \ 688 | __ret; \ 689 | }) 690 | IM_API IM_STATUS imrop_t(const rga_buffer_t src, rga_buffer_t dst, int rop_code, int sync); 691 | 692 | /* 693 | * MOSAIC 694 | * 695 | * @param src 696 | * @param dst 697 | * @param mosaic_mode 698 | * @param sync 699 | * wait until operation complete 700 | * 701 | * @returns success or else negative error code. 702 | */ 703 | IM_API IM_STATUS immosaic(const rga_buffer_t image, im_rect rect, int mosaic_mode, int sync); 704 | 705 | /* 706 | * OSD 707 | * 708 | * @param osd 709 | * osd block 710 | * @param dst 711 | * background image 712 | * @param osd_rect 713 | * @param osd_config 714 | * osd mode config 715 | * @param sync 716 | * wait until operation complete 717 | * 718 | * @returns success or else negative error code. 719 | */ 720 | IM_API IM_STATUS imosd(const rga_buffer_t osd,const rga_buffer_t dst, 721 | const im_rect osd_rect, im_osd_t *osd_config, int sync); 722 | 723 | /* 724 | * process 725 | * 726 | * @param src 727 | * @param dst 728 | * @param usage 729 | * @param ... 730 | * wait until operation complete 731 | * 732 | * @returns success or else negative error code. 733 | */ 734 | IM_API IM_STATUS improcess(rga_buffer_t src, rga_buffer_t dst, rga_buffer_t pat, 735 | im_rect srect, im_rect drect, im_rect prect, int usage); 736 | 737 | /* 738 | * block until all execution is complete 739 | * 740 | * @returns success or else negative error code. 741 | */ 742 | IM_API IM_STATUS imsync(int out_fence_fd); 743 | 744 | /* 745 | * config 746 | * 747 | * @param name 748 | * enum IM_CONFIG_NAME 749 | * @param value 750 | * 751 | * @returns success or else negative error code. 752 | */ 753 | IM_API IM_STATUS imconfig(IM_CONFIG_NAME name, uint64_t value); 754 | 755 | IM_API im_ctx_id_t imbegin(uint32_t flags); 756 | 757 | IM_API IM_STATUS imcancel(im_ctx_id_t id); 758 | 759 | #ifdef __cplusplus 760 | } 761 | #endif 762 | #endif /* _im2d_h_ */ 763 | 764 | -------------------------------------------------------------------------------- /cpp/include/im2d.hpp: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2020 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * PutinLee 5 | * Cerf Yu 6 | * 7 | * Licensed under the Apache License, Version 2.0 (the "License"); 8 | * you may not use this file except in compliance with the License. 9 | * You may obtain a copy of the License at 10 | * 11 | * http://www.apache.org/licenses/LICENSE-2.0 12 | * 13 | * Unless required by applicable law or agreed to in writing, software 14 | * distributed under the License is distributed on an "AS IS" BASIS, 15 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | * See the License for the specific language governing permissions and 17 | * limitations under the License. 18 | */ 19 | #ifndef _im2d_hpp_ 20 | #define _im2d_hpp_ 21 | 22 | #include "im2d.h" 23 | #include "RgaUtils.h" 24 | 25 | #ifdef ANDROID 26 | 27 | #include 28 | 29 | using namespace android; 30 | #endif 31 | 32 | /* 33 | * Import external buffers into RGA driver. 34 | * 35 | * @param fd/va/pa 36 | * Select dma_fd/virtual_address/physical_address by buffer type 37 | * @param size 38 | * Describes the size of the image buffer 39 | * 40 | * @return rga_buffer_handle_t 41 | */ 42 | IM_API rga_buffer_handle_t importbuffer_fd(int fd, int size); 43 | IM_API rga_buffer_handle_t importbuffer_virtualaddr(void *va, int size); 44 | IM_API rga_buffer_handle_t importbuffer_physicaladdr(uint64_t pa, int size); 45 | 46 | /* 47 | * Import external buffers into RGA driver. 48 | * 49 | * @param fd/va/pa 50 | * Select dma_fd/virtual_address/physical_address by buffer type 51 | * @param width 52 | * Describes the pixel width stride of the image buffer 53 | * @param height 54 | * Describes the pixel height stride of the image buffer 55 | * @param format 56 | * Describes the pixel format of the image buffer 57 | * 58 | * @return rga_buffer_handle_t 59 | */ 60 | IM_API rga_buffer_handle_t importbuffer_fd(int fd, int width, int height, int format); 61 | IM_API rga_buffer_handle_t importbuffer_virtualaddr(void *va, int width, int height, int format); 62 | IM_API rga_buffer_handle_t importbuffer_physicaladdr(uint64_t pa, int width, int height, int format); 63 | 64 | #undef wrapbuffer_handle 65 | IM_API rga_buffer_t wrapbuffer_handle(rga_buffer_handle_t handle, 66 | int width, int height, 67 | int wstride, int hstride, 68 | int format); 69 | IM_API rga_buffer_t wrapbuffer_handle(rga_buffer_handle_t handle, 70 | int width, int height, 71 | int format); 72 | 73 | #if ANDROID 74 | IM_API rga_buffer_handle_t importbuffer_GraphicBuffer_handle(buffer_handle_t hnd); 75 | IM_API rga_buffer_handle_t importbuffer_GraphicBuffer(sp buf); 76 | 77 | IM_API rga_buffer_t wrapbuffer_handle(buffer_handle_t hnd); 78 | IM_API rga_buffer_t wrapbuffer_GraphicBuffer(sp buf); 79 | 80 | #if USE_AHARDWAREBUFFER 81 | #include 82 | IM_API rga_buffer_handle_t importbuffer_AHardwareBuffer(AHardwareBuffer *buf); 83 | IM_API rga_buffer_t wrapbuffer_AHardwareBuffer(AHardwareBuffer *buf); 84 | 85 | #endif /* USE_AHARDWAREBUFFER */ 86 | #endif /* ANDROID */ 87 | 88 | /* 89 | * Resize 90 | * 91 | * @param src 92 | * @param dst 93 | * @param fx 94 | * @param fy 95 | * @param interpolation 96 | * @param sync 97 | * wait until operation complete 98 | * 99 | * @returns success or else negative error code. 100 | */ 101 | #undef imresize 102 | IM_API IM_STATUS imresize(const rga_buffer_t src, rga_buffer_t dst, double fx = 0, double fy = 0, int interpolation = 0, int sync = 1, int *release_fence_fd = NULL); 103 | 104 | /* 105 | * Crop 106 | * 107 | * @param src 108 | * @param dst 109 | * @param rect 110 | * @param sync 111 | * wait until operation complete 112 | * 113 | * @returns success or else negative error code. 114 | */ 115 | #undef imcrop 116 | IM_API IM_STATUS imcrop(const rga_buffer_t src, rga_buffer_t dst, im_rect rect, int sync = 1, int *release_fence_fd = NULL); 117 | 118 | /* 119 | * rotation 120 | * 121 | * @param src 122 | * @param dst 123 | * @param rotation 124 | * IM_HAL_TRANSFORM_ROT_90 125 | * IM_HAL_TRANSFORM_ROT_180 126 | * IM_HAL_TRANSFORM_ROT_270 127 | * @param sync 128 | * wait until operation complete 129 | * 130 | * @returns success or else negative error code. 131 | */ 132 | #undef imrotate 133 | IM_API IM_STATUS imrotate(const rga_buffer_t src, rga_buffer_t dst, int rotation, int sync = 1, int *release_fence_fd = NULL); 134 | 135 | /* 136 | * flip 137 | * 138 | * @param src 139 | * @param dst 140 | * @param mode 141 | * IM_HAL_TRANSFORM_FLIP_H 142 | * IM_HAL_TRANSFORM_FLIP_V 143 | * @param sync 144 | * wait until operation complete 145 | * 146 | * @returns success or else negative error code. 147 | */ 148 | #undef imflip 149 | IM_API IM_STATUS imflip(const rga_buffer_t src, rga_buffer_t dst, int mode, int sync = 1, int *release_fence_fd = NULL); 150 | 151 | /* 152 | * fill/reset/draw 153 | * 154 | * @param src 155 | * @param dst 156 | * @param rect 157 | * @param color 158 | * @param sync 159 | * wait until operation complete 160 | * 161 | * @returns success or else negative error code. 162 | */ 163 | #undef imfill 164 | IM_API IM_STATUS imfill(rga_buffer_t dst, im_rect rect, int color, int sync = 1, int *release_fence_fd = NULL); 165 | 166 | /* 167 | * palette 168 | * 169 | * @param src 170 | * @param dst 171 | * @param lut 172 | * @param sync 173 | * wait until operation complete 174 | * 175 | * @returns success or else negative error code. 176 | */ 177 | #undef impalette 178 | IM_API IM_STATUS impalette(rga_buffer_t src, rga_buffer_t dst, rga_buffer_t lut, int sync = 1, int *release_fence_fd = NULL); 179 | 180 | /* 181 | * translate 182 | * 183 | * @param src 184 | * @param dst 185 | * @param x 186 | * @param y 187 | * @param sync 188 | * wait until operation complete 189 | * 190 | * @returns success or else negative error code. 191 | */ 192 | #undef imtranslate 193 | IM_API IM_STATUS imtranslate(const rga_buffer_t src, rga_buffer_t dst, int x, int y, int sync = 1, int *release_fence_fd = NULL); 194 | 195 | /* 196 | * copy 197 | * 198 | * @param src 199 | * @param dst 200 | * @param sync 201 | * wait until operation complete 202 | * 203 | * @returns success or else negative error code. 204 | */ 205 | #undef imcopy 206 | IM_API IM_STATUS imcopy(const rga_buffer_t src, rga_buffer_t dst, int sync = 1, int *release_fence_fd = NULL); 207 | 208 | /* 209 | * blend (SRC + DST -> DST or SRCA + SRCB -> DST) 210 | * 211 | * @param srcA 212 | * @param srcB can be NULL. 213 | * @param dst 214 | * @param mode 215 | * IM_ALPHA_BLEND_MODE 216 | * @param sync 217 | * wait until operation complete 218 | * 219 | * @returns success or else negative error code. 220 | */ 221 | #undef imblend 222 | IM_API IM_STATUS imblend(const rga_buffer_t src, rga_buffer_t dst, int mode = IM_ALPHA_BLEND_SRC_OVER, int sync = 1, int *release_fence_fd = NULL); 223 | #undef imcomposite 224 | IM_API IM_STATUS imcomposite(const rga_buffer_t srcA, const rga_buffer_t srcB, rga_buffer_t dst, int mode = IM_ALPHA_BLEND_SRC_OVER, int sync = 1, int *release_fence_fd = NULL); 225 | 226 | /* 227 | * color key 228 | * 229 | * @param src 230 | * @param dst 231 | * @param colorkey_range 232 | * max color 233 | * min color 234 | * @param sync 235 | * wait until operation complete 236 | * 237 | * @returns success or else negative error code. 238 | */ 239 | #undef imcolorkey 240 | IM_API IM_STATUS imcolorkey(const rga_buffer_t src, rga_buffer_t dst, im_colorkey_range range, int mode = IM_ALPHA_COLORKEY_NORMAL, int sync = 1, int *release_fence_fd = NULL); 241 | 242 | /* 243 | * format convert 244 | * 245 | * @param src 246 | * @param dst 247 | * @param sfmt 248 | * @param dfmt 249 | * @param mode 250 | * color space mode: IM_COLOR_SPACE_MODE 251 | * @param sync 252 | * wait until operation complete 253 | * 254 | * @returns success or else negative error code. 255 | */ 256 | #undef imcvtcolor 257 | IM_API IM_STATUS imcvtcolor(rga_buffer_t src, rga_buffer_t dst, int sfmt, int dfmt, int mode = IM_COLOR_SPACE_DEFAULT, int sync = 1, int *release_fence_fd = NULL); 258 | 259 | /* 260 | * nn quantize 261 | * 262 | * @param src 263 | * @param dst 264 | * @param nninfo 265 | * @param sync 266 | * wait until operation complete 267 | * 268 | * @returns success or else negative error code. 269 | */ 270 | #undef imquantize 271 | IM_API IM_STATUS imquantize(const rga_buffer_t src, rga_buffer_t dst, im_nn_t nn_info, int sync = 1, int *release_fence_fd = NULL); 272 | 273 | /* 274 | * ROP 275 | * 276 | * @param src 277 | * @param dst 278 | * @param rop_code 279 | * @param sync 280 | * wait until operation complete 281 | * 282 | * @returns success or else negative error code. 283 | */ 284 | #undef imrop 285 | IM_API IM_STATUS imrop(const rga_buffer_t src, rga_buffer_t dst, int rop_code, int sync = 1, int *release_fence_fd = NULL); 286 | 287 | /* 288 | * MOSAIC 289 | * 290 | * @param src 291 | * @param dst 292 | * @param mosaic_mode 293 | * @param sync 294 | * wait until operation complete 295 | * 296 | * @returns success or else negative error code. 297 | */ 298 | IM_API IM_STATUS immosaic(const rga_buffer_t image, im_rect rect, int mosaic_mode, int sync = 1, int *release_fence_fd = NULL); 299 | 300 | /* 301 | * OSD 302 | * 303 | * @param osd 304 | * osd block 305 | * @param dst 306 | * background image 307 | * @param osd_rect 308 | * @param osd_config 309 | * osd mode config 310 | * @param sync 311 | * wait until operation complete 312 | * 313 | * @returns success or else negative error code. 314 | */ 315 | IM_API IM_STATUS imosd(const rga_buffer_t osd,const rga_buffer_t dst, 316 | const im_rect osd_rect, im_osd_t *osd_config, 317 | int sync = 1, int *release_fence_fd = NULL); 318 | 319 | /* 320 | * process 321 | * 322 | * @param src 323 | * @param dst 324 | * @param pat 325 | * @param srect 326 | * @param drect 327 | * @param prect 328 | * @param acquire_fence_fd 329 | * @param release_fence_fd 330 | * @param opt 331 | * @param usage 332 | * 333 | * @returns success or else negative error code. 334 | */ 335 | IM_API IM_STATUS improcess(rga_buffer_t src, rga_buffer_t dst, rga_buffer_t pat, 336 | im_rect srect, im_rect drect, im_rect prect, 337 | int acquire_fence_fd, int *release_fence_fd, im_opt_t *opt_ptr, int usage); 338 | IM_API IM_STATUS improcess(rga_buffer_t src, rga_buffer_t dst, rga_buffer_t pat, 339 | im_rect srect, im_rect drect, im_rect prect, 340 | int acquire_fence_fd, int *release_fence_fd, im_opt_t *opt, int usage, im_ctx_id_t ctx_id); 341 | 342 | #endif /* _im2d_hpp_ */ 343 | 344 | -------------------------------------------------------------------------------- /cpp/include/im2d_hardware.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2021 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Cerf Yu 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _RGA_IM2D_HARDWARE_H_ 20 | #define _RGA_IM2D_HARDWARE_H_ 21 | 22 | #include "rga.h" 23 | 24 | typedef enum { 25 | IM_RGA_HW_VERSION_RGA_V_ERR_INDEX = 0x0, 26 | IM_RGA_HW_VERSION_RGA_1_INDEX, 27 | IM_RGA_HW_VERSION_RGA_1_PLUS_INDEX, 28 | IM_RGA_HW_VERSION_RGA_2_INDEX, 29 | IM_RGA_HW_VERSION_RGA_2_LITE0_INDEX, 30 | IM_RGA_HW_VERSION_RGA_2_LITE1_INDEX, 31 | IM_RGA_HW_VERSION_RGA_2_ENHANCE_INDEX, 32 | IM_RGA_HW_VERSION_RGA_3_INDEX, 33 | IM_RGA_HW_VERSION_MASK_INDEX, 34 | } IM_RGA_HW_VERSION_INDEX; 35 | 36 | typedef enum { 37 | IM_RGA_HW_VERSION_RGA_V_ERR = 1 << IM_RGA_HW_VERSION_RGA_V_ERR_INDEX, 38 | IM_RGA_HW_VERSION_RGA_1 = 1 << IM_RGA_HW_VERSION_RGA_1_INDEX, 39 | IM_RGA_HW_VERSION_RGA_1_PLUS = 1 << IM_RGA_HW_VERSION_RGA_1_PLUS_INDEX, 40 | IM_RGA_HW_VERSION_RGA_2 = 1 << IM_RGA_HW_VERSION_RGA_2_INDEX, 41 | IM_RGA_HW_VERSION_RGA_2_LITE0 = 1 << IM_RGA_HW_VERSION_RGA_2_LITE0_INDEX, 42 | IM_RGA_HW_VERSION_RGA_2_LITE1 = 1 << IM_RGA_HW_VERSION_RGA_2_LITE1_INDEX, 43 | IM_RGA_HW_VERSION_RGA_2_ENHANCE = 1 << IM_RGA_HW_VERSION_RGA_2_ENHANCE_INDEX, 44 | IM_RGA_HW_VERSION_RGA_3 = 1 << IM_RGA_HW_VERSION_RGA_3_INDEX, 45 | IM_RGA_HW_VERSION_MASK = ~((~(unsigned int)0x0 << IM_RGA_HW_VERSION_MASK_INDEX) | 1), 46 | }IM_RGA_HW_VERSION; 47 | 48 | typedef enum { 49 | IM_RGA_SUPPORT_FORMAT_ERROR_INDEX = 0, 50 | IM_RGA_SUPPORT_FORMAT_RGB_INDEX, 51 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER_INDEX, 52 | IM_RGA_SUPPORT_FORMAT_BPP_INDEX, 53 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT_INDEX, 54 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT_INDEX, 55 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT_INDEX, 56 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_10_BIT_INDEX, 57 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT_INDEX, 58 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT_INDEX, 59 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT_INDEX, 60 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_10_BIT_INDEX, 61 | IM_RGA_SUPPORT_FORMAT_YUYV_420_INDEX, 62 | IM_RGA_SUPPORT_FORMAT_YUYV_422_INDEX, 63 | IM_RGA_SUPPORT_FORMAT_YUV_400_INDEX, 64 | IM_RGA_SUPPORT_FORMAT_Y4_INDEX, 65 | IM_RGA_SUPPORT_FORMAT_RGBA2BPP_INDEX, 66 | IM_RGA_SUPPORT_FORMAT_MASK_INDEX, 67 | } IM_RGA_SUPPORT_FORMAT_INDEX; 68 | 69 | typedef enum { 70 | IM_RGA_SUPPORT_FORMAT_ERROR = 1 << IM_RGA_SUPPORT_FORMAT_ERROR_INDEX, 71 | IM_RGA_SUPPORT_FORMAT_RGB = 1 << IM_RGA_SUPPORT_FORMAT_RGB_INDEX, 72 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER = 1 << IM_RGA_SUPPORT_FORMAT_RGB_OTHER_INDEX, 73 | IM_RGA_SUPPORT_FORMAT_BPP = 1 << IM_RGA_SUPPORT_FORMAT_BPP_INDEX, 74 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT_INDEX, 75 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT_INDEX, 76 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT_INDEX, 77 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_10_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_10_BIT_INDEX, 78 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT_INDEX, 79 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT_INDEX, 80 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT_INDEX, 81 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_10_BIT = 1 << IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_10_BIT_INDEX, 82 | IM_RGA_SUPPORT_FORMAT_YUYV_420 = 1 << IM_RGA_SUPPORT_FORMAT_YUYV_420_INDEX, 83 | IM_RGA_SUPPORT_FORMAT_YUYV_422 = 1 << IM_RGA_SUPPORT_FORMAT_YUYV_422_INDEX, 84 | IM_RGA_SUPPORT_FORMAT_YUV_400 = 1 << IM_RGA_SUPPORT_FORMAT_YUV_400_INDEX, 85 | IM_RGA_SUPPORT_FORMAT_Y4 = 1 << IM_RGA_SUPPORT_FORMAT_Y4_INDEX, 86 | IM_RGA_SUPPORT_FORMAT_RGBA2BPP = 1 << IM_RGA_SUPPORT_FORMAT_RGBA2BPP_INDEX, 87 | IM_RGA_SUPPORT_FORMAT_MASK = ~((~(unsigned int)0x0 << IM_RGA_SUPPORT_FORMAT_MASK_INDEX) | 1), 88 | } IM_RGA_SUPPORT_FORMAT; 89 | 90 | typedef enum { 91 | IM_RGA_SUPPORT_FEATURE_ERROR_INDEX = 0, 92 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL_INDEX, 93 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE_INDEX, 94 | IM_RGA_SUPPORT_FEATURE_ROP_INDEX, 95 | IM_RGA_SUPPORT_FEATURE_QUANTIZE_INDEX, 96 | IM_RGA_SUPPORT_FEATURE_SRC1_R2Y_CSC_INDEX, 97 | IM_RGA_SUPPORT_FEATURE_DST_FULL_CSC_INDEX, 98 | IM_RGA_SUPPORT_FEATURE_FBC_INDEX, 99 | IM_RGA_SUPPORT_FEATURE_BLEND_YUV_INDEX, 100 | IM_RGA_SUPPORT_FEATURE_BT2020_INDEX, 101 | IM_RGA_SUPPORT_FEATURE_MOSAIC_INDEX, 102 | IM_RGA_SUPPORT_FEATURE_OSD_INDEX, 103 | IM_RGA_SUPPORT_FEATURE_PRE_INTR_INDEX, 104 | IM_RGA_SUPPORT_FEATURE_MASK_INDEX, 105 | } IM_RGA_SUPPORT_FEATURE_INDEX; 106 | 107 | typedef enum { 108 | IM_RGA_SUPPORT_FEATURE_ERROR = 1 << IM_RGA_SUPPORT_FEATURE_ERROR_INDEX, 109 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL = 1 << IM_RGA_SUPPORT_FEATURE_COLOR_FILL_INDEX, 110 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE = 1 << IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE_INDEX, 111 | IM_RGA_SUPPORT_FEATURE_ROP = 1 << IM_RGA_SUPPORT_FEATURE_ROP_INDEX, 112 | IM_RGA_SUPPORT_FEATURE_QUANTIZE = 1 << IM_RGA_SUPPORT_FEATURE_QUANTIZE_INDEX, 113 | IM_RGA_SUPPORT_FEATURE_SRC1_R2Y_CSC = 1 << IM_RGA_SUPPORT_FEATURE_SRC1_R2Y_CSC_INDEX, 114 | IM_RGA_SUPPORT_FEATURE_DST_FULL_CSC = 1 << IM_RGA_SUPPORT_FEATURE_DST_FULL_CSC_INDEX, 115 | IM_RGA_SUPPORT_FEATURE_FBC = 1 << IM_RGA_SUPPORT_FEATURE_FBC_INDEX, 116 | IM_RGA_SUPPORT_FEATURE_BLEND_YUV = 1 << IM_RGA_SUPPORT_FEATURE_BLEND_YUV_INDEX, 117 | IM_RGA_SUPPORT_FEATURE_BT2020 = 1 << IM_RGA_SUPPORT_FEATURE_BT2020_INDEX, 118 | IM_RGA_SUPPORT_FEATURE_MOSAIC = 1 << IM_RGA_SUPPORT_FEATURE_MOSAIC_INDEX, 119 | IM_RGA_SUPPORT_FEATURE_OSD = 1 << IM_RGA_SUPPORT_FEATURE_OSD_INDEX, 120 | IM_RGA_SUPPORT_FEATURE_PRE_INTR = 1 << IM_RGA_SUPPORT_FEATURE_PRE_INTR_INDEX, 121 | IM_RGA_SUPPORT_FEATURE_MASK = ~((~(unsigned int)0x0 << IM_RGA_SUPPORT_FEATURE_MASK_INDEX) | 1), 122 | } IM_RGA_SUPPORT_FEATURE; 123 | 124 | typedef struct { 125 | unsigned int version; 126 | unsigned int input_resolution; 127 | unsigned int output_resolution; 128 | unsigned int byte_stride; 129 | unsigned int scale_limit; 130 | unsigned int performance; 131 | unsigned int input_format; 132 | unsigned int output_format; 133 | unsigned int feature; 134 | char reserved[24]; 135 | } rga_info_table_entry; 136 | 137 | typedef struct { 138 | struct rga_version_t user; 139 | struct rga_version_t driver; 140 | } rga_dirver_bind_table_entry; 141 | 142 | const rga_info_table_entry hw_info_table[] = { 143 | { IM_RGA_HW_VERSION_RGA_V_ERR , 0, 0, 0, 0, 0, 0, 0, 0, {0} }, 144 | { IM_RGA_HW_VERSION_RGA_1 , 8192, 2048, 4, 8, 1, 145 | /* input format */ 146 | IM_RGA_SUPPORT_FORMAT_RGB | 147 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 148 | IM_RGA_SUPPORT_FORMAT_BPP | 149 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 150 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 151 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 152 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 153 | /* output format */ 154 | IM_RGA_SUPPORT_FORMAT_RGB | 155 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 156 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 157 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 158 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 159 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 160 | /* feature */ 161 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 162 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE | 163 | IM_RGA_SUPPORT_FEATURE_ROP, 164 | /* reserved */ 165 | {0} }, 166 | { IM_RGA_HW_VERSION_RGA_1_PLUS , 8192, 2048, 4, 8, 1, 167 | /* input format */ 168 | IM_RGA_SUPPORT_FORMAT_RGB | 169 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 170 | IM_RGA_SUPPORT_FORMAT_BPP | 171 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 172 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 173 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 174 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 175 | /* output format */ 176 | IM_RGA_SUPPORT_FORMAT_RGB | 177 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 178 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 179 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 180 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 181 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 182 | /* feature */ 183 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 184 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE, 185 | /* reserved */ 186 | {0} }, 187 | { IM_RGA_HW_VERSION_RGA_2 , 8192, 4096, 4, 16, 2, 188 | /* input format */ 189 | IM_RGA_SUPPORT_FORMAT_RGB | 190 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 191 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 192 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 193 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 194 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 195 | /* output format */ 196 | IM_RGA_SUPPORT_FORMAT_RGB | 197 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 198 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 199 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 200 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 201 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 202 | /* feature */ 203 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 204 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE | 205 | IM_RGA_SUPPORT_FEATURE_ROP, 206 | /* reserved */ 207 | {0} }, 208 | { IM_RGA_HW_VERSION_RGA_2_LITE0 , 8192, 4096, 4, 8, 2, 209 | /* input format */ 210 | IM_RGA_SUPPORT_FORMAT_RGB | 211 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 212 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 213 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 214 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 215 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 216 | /* output format */ 217 | IM_RGA_SUPPORT_FORMAT_RGB | 218 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 219 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 220 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 221 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 222 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 223 | /* feature */ 224 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 225 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE | 226 | IM_RGA_SUPPORT_FEATURE_ROP, 227 | /* reserved */ 228 | {0} }, 229 | { IM_RGA_HW_VERSION_RGA_2_LITE1 , 8192, 4096, 4, 8, 2, 230 | /* input format */ 231 | IM_RGA_SUPPORT_FORMAT_RGB | 232 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 233 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 234 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 235 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 236 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT | 237 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT | 238 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_10_BIT | 239 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT | 240 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_10_BIT, 241 | /* output format */ 242 | IM_RGA_SUPPORT_FORMAT_RGB | 243 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 244 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 245 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 246 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 247 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT, 248 | /* feature */ 249 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 250 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE, 251 | /* reserved */ 252 | {0} }, 253 | { IM_RGA_HW_VERSION_RGA_2_ENHANCE , 8192, 4096, 4, 16, 2, 254 | /* input format */ 255 | IM_RGA_SUPPORT_FORMAT_RGB | 256 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 257 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 258 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 259 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 260 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT | 261 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT | 262 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_10_BIT | 263 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT | 264 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_10_BIT, 265 | /* output format */ 266 | IM_RGA_SUPPORT_FORMAT_RGB | 267 | IM_RGA_SUPPORT_FORMAT_RGB_OTHER | 268 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 269 | IM_RGA_SUPPORT_FORMAT_YUV_420_PLANNER_8_BIT | 270 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 271 | IM_RGA_SUPPORT_FORMAT_YUV_422_PLANNER_8_BIT | 272 | IM_RGA_SUPPORT_FORMAT_YUYV_420 | 273 | IM_RGA_SUPPORT_FORMAT_YUYV_422, 274 | /* feature */ 275 | IM_RGA_SUPPORT_FEATURE_COLOR_FILL | 276 | IM_RGA_SUPPORT_FEATURE_COLOR_PALETTE | 277 | IM_RGA_SUPPORT_FEATURE_ROP, 278 | /* reserved */ 279 | {0} }, 280 | { IM_RGA_HW_VERSION_RGA_3 , 8176, 8128, 16, 8, 4, 281 | /* input format */ 282 | IM_RGA_SUPPORT_FORMAT_RGB | 283 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 284 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 285 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT | 286 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT | 287 | IM_RGA_SUPPORT_FORMAT_YUYV_422, 288 | /* output format */ 289 | IM_RGA_SUPPORT_FORMAT_RGB | 290 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_8_BIT | 291 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_8_BIT | 292 | IM_RGA_SUPPORT_FORMAT_YUV_420_SEMI_PLANNER_10_BIT | 293 | IM_RGA_SUPPORT_FORMAT_YUV_422_SEMI_PLANNER_10_BIT | 294 | IM_RGA_SUPPORT_FORMAT_YUYV_422, 295 | /* feature */ 296 | IM_RGA_SUPPORT_FEATURE_FBC | 297 | IM_RGA_SUPPORT_FEATURE_BLEND_YUV | 298 | IM_RGA_SUPPORT_FEATURE_BT2020, 299 | /* reserved */ 300 | {0} }, 301 | }; 302 | 303 | /* The range of the version is [min, max), that is version >= min, version < max. */ 304 | const rga_dirver_bind_table_entry driver_bind_table[] = { 305 | { { 0, 0, 0, "0.0.0" }, {0, 0, 0, "0.0.0" } }, 306 | { { 1, 0, 3, "1.0.3" }, {0, 0, 0, "0.0.0" } }, 307 | { { 1, 6, 0, "1.6.0" }, {1, 1, 5, "1.1.5" } }, 308 | { { 1, 7, 2, "1.7.2" }, {1, 2, 0, "1.2.0" } }, 309 | { { 1, 7, 3, "1.7.3" }, {1, 2, 4, "1.2.4" } }, 310 | }; 311 | 312 | #endif 313 | -------------------------------------------------------------------------------- /cpp/include/im2d_type.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2022 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Cerf Yu 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _RGA_IM2D_TYPE_H_ 20 | #define _RGA_IM2D_TYPE_H_ 21 | 22 | #include 23 | 24 | typedef enum { 25 | /* Rotation */ 26 | IM_HAL_TRANSFORM_ROT_90 = 1 << 0, 27 | IM_HAL_TRANSFORM_ROT_180 = 1 << 1, 28 | IM_HAL_TRANSFORM_ROT_270 = 1 << 2, 29 | IM_HAL_TRANSFORM_FLIP_H = 1 << 3, 30 | IM_HAL_TRANSFORM_FLIP_V = 1 << 4, 31 | IM_HAL_TRANSFORM_FLIP_H_V = 1 << 5, 32 | IM_HAL_TRANSFORM_MASK = 0x3f, 33 | 34 | /* 35 | * Blend 36 | * Additional blend usage, can be used with both source and target configs. 37 | * If none of the below is set, the default "SRC over DST" is applied. 38 | */ 39 | IM_ALPHA_BLEND_SRC_OVER = 1 << 6, /* Default, Porter-Duff "SRC over DST" */ 40 | IM_ALPHA_BLEND_SRC = 1 << 7, /* Porter-Duff "SRC" */ 41 | IM_ALPHA_BLEND_DST = 1 << 8, /* Porter-Duff "DST" */ 42 | IM_ALPHA_BLEND_SRC_IN = 1 << 9, /* Porter-Duff "SRC in DST" */ 43 | IM_ALPHA_BLEND_DST_IN = 1 << 10, /* Porter-Duff "DST in SRC" */ 44 | IM_ALPHA_BLEND_SRC_OUT = 1 << 11, /* Porter-Duff "SRC out DST" */ 45 | IM_ALPHA_BLEND_DST_OUT = 1 << 12, /* Porter-Duff "DST out SRC" */ 46 | IM_ALPHA_BLEND_DST_OVER = 1 << 13, /* Porter-Duff "DST over SRC" */ 47 | IM_ALPHA_BLEND_SRC_ATOP = 1 << 14, /* Porter-Duff "SRC ATOP" */ 48 | IM_ALPHA_BLEND_DST_ATOP = 1 << 15, /* Porter-Duff "DST ATOP" */ 49 | IM_ALPHA_BLEND_XOR = 1 << 16, /* Xor */ 50 | IM_ALPHA_BLEND_MASK = 0x1ffc0, 51 | 52 | IM_ALPHA_COLORKEY_NORMAL = 1 << 17, 53 | IM_ALPHA_COLORKEY_INVERTED = 1 << 18, 54 | IM_ALPHA_COLORKEY_MASK = 0x60000, 55 | 56 | IM_SYNC = 1 << 19, 57 | IM_CROP = 1 << 20, /* Unused */ 58 | IM_COLOR_FILL = 1 << 21, 59 | IM_COLOR_PALETTE = 1 << 22, 60 | IM_NN_QUANTIZE = 1 << 23, 61 | IM_ROP = 1 << 24, 62 | IM_ALPHA_BLEND_PRE_MUL = 1 << 25, 63 | IM_ASYNC = 1 << 26, 64 | IM_MOSAIC = 1 << 27, 65 | IM_OSD = 1 << 28, 66 | IM_PRE_INTR = 1 << 29, 67 | } IM_USAGE; 68 | 69 | typedef enum { 70 | IM_RASTER_MODE = 1 << 0, 71 | IM_FBC_MODE = 1 << 1, 72 | IM_TILE_MODE = 1 << 2, 73 | } IM_RD_MODE; 74 | 75 | typedef enum { 76 | IM_SCHEDULER_RGA3_CORE0 = 1 << 0, 77 | IM_SCHEDULER_RGA3_CORE1 = 1 << 1, 78 | IM_SCHEDULER_RGA2_CORE0 = 1 << 2, 79 | IM_SCHEDULER_RGA3_DEFAULT = IM_SCHEDULER_RGA3_CORE0, 80 | IM_SCHEDULER_RGA2_DEFAULT = IM_SCHEDULER_RGA2_CORE0, 81 | IM_SCHEDULER_MASK = 0x7, 82 | IM_SCHEDULER_DEFAULT = 0, 83 | } IM_SCHEDULER_CORE; 84 | 85 | typedef enum { 86 | IM_ROP_AND = 0x88, 87 | IM_ROP_OR = 0xee, 88 | IM_ROP_NOT_DST = 0x55, 89 | IM_ROP_NOT_SRC = 0x33, 90 | IM_ROP_XOR = 0xf6, 91 | IM_ROP_NOT_XOR = 0xf9, 92 | } IM_ROP_CODE; 93 | 94 | typedef enum { 95 | IM_MOSAIC_8 = 0x0, 96 | IM_MOSAIC_16 = 0x1, 97 | IM_MOSAIC_32 = 0x2, 98 | IM_MOSAIC_64 = 0x3, 99 | IM_MOSAIC_128 = 0x4, 100 | } IM_MOSAIC_MODE; 101 | 102 | /* Status codes, returned by any blit function */ 103 | typedef enum { 104 | IM_YUV_TO_RGB_BT601_LIMIT = 1 << 0, 105 | IM_YUV_TO_RGB_BT601_FULL = 2 << 0, 106 | IM_YUV_TO_RGB_BT709_LIMIT = 3 << 0, 107 | IM_YUV_TO_RGB_MASK = 3 << 0, 108 | IM_RGB_TO_YUV_BT601_FULL = 1 << 2, 109 | IM_RGB_TO_YUV_BT601_LIMIT = 2 << 2, 110 | IM_RGB_TO_YUV_BT709_LIMIT = 3 << 2, 111 | IM_RGB_TO_YUV_MASK = 3 << 2, 112 | IM_RGB_TO_Y4 = 1 << 4, 113 | IM_RGB_TO_Y4_DITHER = 2 << 4, 114 | IM_RGB_TO_Y1_DITHER = 3 << 4, 115 | IM_Y4_MASK = 3 << 4, 116 | IM_RGB_FULL = 1 << 8, 117 | IM_RGB_CLIP = 2 << 8, 118 | IM_YUV_BT601_LIMIT_RANGE = 3 << 8, 119 | IM_YUV_BT601_FULL_RANGE = 4 << 8, 120 | IM_YUV_BT709_LIMIT_RANGE = 5 << 8, 121 | IM_YUV_BT709_FULL_RANGE = 6 << 8, 122 | IM_FULL_CSC_MASK = 0xf << 8, 123 | IM_COLOR_SPACE_DEFAULT = 0, 124 | } IM_COLOR_SPACE_MODE; 125 | 126 | typedef enum { 127 | IM_UP_SCALE, 128 | IM_DOWN_SCALE, 129 | } IM_SCALE; 130 | 131 | typedef enum { 132 | INTER_NEAREST, 133 | INTER_LINEAR, 134 | INTER_CUBIC, 135 | } IM_SCALE_MODE; 136 | 137 | typedef enum { 138 | IM_CONFIG_SCHEDULER_CORE, 139 | IM_CONFIG_PRIORITY, 140 | IM_CONFIG_CHECK, 141 | } IM_CONFIG_NAME; 142 | 143 | typedef enum { 144 | IM_OSD_MODE_STATISTICS = 0x1 << 0, 145 | IM_OSD_MODE_AUTO_INVERT = 0x1 << 1, 146 | } IM_OSD_MODE; 147 | 148 | typedef enum { 149 | IM_OSD_INVERT_CHANNEL_NONE = 0x0, 150 | IM_OSD_INVERT_CHANNEL_Y_G = 0x1 << 0, 151 | IM_OSD_INVERT_CHANNEL_C_RB = 0x1 << 1, 152 | IM_OSD_INVERT_CHANNEL_ALPHA = 0x1 << 2, 153 | IM_OSD_INVERT_CHANNEL_COLOR = IM_OSD_INVERT_CHANNEL_Y_G | 154 | IM_OSD_INVERT_CHANNEL_C_RB, 155 | IM_OSD_INVERT_CHANNEL_BOTH = IM_OSD_INVERT_CHANNEL_COLOR | 156 | IM_OSD_INVERT_CHANNEL_ALPHA, 157 | } IM_OSD_INVERT_CHANNEL; 158 | 159 | typedef enum { 160 | IM_OSD_FLAGS_INTERNAL = 0, 161 | IM_OSD_FLAGS_EXTERNAL, 162 | } IM_OSD_FLAGS_MODE; 163 | 164 | typedef enum { 165 | IM_OSD_INVERT_USE_FACTOR, 166 | IM_OSD_INVERT_USE_SWAP, 167 | } IM_OSD_INVERT_MODE; 168 | 169 | typedef enum { 170 | IM_OSD_BACKGROUND_DEFAULT_BRIGHT = 0, 171 | IM_OSD_BACKGROUND_DEFAULT_DARK, 172 | } IM_OSD_BACKGROUND_DEFAULT; 173 | 174 | typedef enum { 175 | IM_OSD_BLOCK_MODE_NORMAL = 0, 176 | IM_OSD_BLOCK_MODE_DIFFERENT, 177 | } IM_OSD_BLOCK_WIDTH_MODE; 178 | 179 | typedef enum { 180 | IM_OSD_MODE_HORIZONTAL, 181 | IM_OSD_MODE_VERTICAL, 182 | } IM_OSD_DIRECTION; 183 | 184 | typedef enum { 185 | IM_OSD_COLOR_PIXEL, 186 | IM_OSD_COLOR_EXTERNAL, 187 | } IM_OSD_COLOR_MODE; 188 | 189 | typedef enum { 190 | IM_INTR_READ_INTR = 1 << 0, 191 | IM_INTR_READ_HOLD = 1 << 1, 192 | IM_INTR_WRITE_INTR = 1 << 2, 193 | } IM_PRE_INTR_FLAGS; 194 | 195 | typedef enum { 196 | IM_CONTEXT_NONE = 0x0, 197 | IM_CONTEXT_SRC_FIX_ENABLE = 0x1 << 0, // Enable kernel to modify the image parameters of the channel. 198 | IM_CONTEXT_SRC_CACHE_INFO = 0x1 << 1, // It will replace the parameters in ctx with the modified parameters. 199 | IM_CONTEXT_SRC1_FIX_ENABLE = 0x1 << 2, 200 | IM_CONTEXT_SRC1_CACHE_INFO = 0x1 << 3, 201 | IM_CONTEXT_DST_FIX_ENABLE = 0x1 << 4, 202 | IM_CONTEXT_DST_CACHE_INFO = 0x1 << 5, 203 | } IM_CONTEXT_FLAGS; 204 | 205 | /* Get RGA basic information index */ 206 | typedef enum { 207 | RGA_VENDOR = 0, 208 | RGA_VERSION, 209 | RGA_MAX_INPUT, 210 | RGA_MAX_OUTPUT, 211 | RGA_BYTE_STRIDE, 212 | RGA_SCALE_LIMIT, 213 | RGA_INPUT_FORMAT, 214 | RGA_OUTPUT_FORMAT, 215 | RGA_FEATURE, 216 | RGA_EXPECTED, 217 | RGA_ALL, 218 | } IM_INFORMATION; 219 | 220 | /* Status codes, returned by any blit function */ 221 | typedef enum { 222 | IM_STATUS_NOERROR = 2, 223 | IM_STATUS_SUCCESS = 1, 224 | IM_STATUS_NOT_SUPPORTED = -1, 225 | IM_STATUS_OUT_OF_MEMORY = -2, 226 | IM_STATUS_INVALID_PARAM = -3, 227 | IM_STATUS_ILLEGAL_PARAM = -4, 228 | IM_STATUS_ERROR_VERSION = -5, 229 | IM_STATUS_FAILED = 0, 230 | } IM_STATUS; 231 | 232 | typedef uint32_t im_api_version_t; 233 | typedef uint32_t im_ctx_id_t; 234 | typedef uint32_t rga_buffer_handle_t; 235 | 236 | /* Rectangle definition */ 237 | typedef struct { 238 | int x; /* upper-left x */ 239 | int y; /* upper-left y */ 240 | int width; /* width */ 241 | int height; /* height */ 242 | } im_rect; 243 | 244 | typedef struct { 245 | int max; /* The Maximum value of the color key */ 246 | int min; /* The minimum value of the color key */ 247 | } im_colorkey_range; 248 | 249 | 250 | typedef struct im_nn { 251 | int scale_r; /* scaling factor on R channal */ 252 | int scale_g; /* scaling factor on G channal */ 253 | int scale_b; /* scaling factor on B channal */ 254 | int offset_r; /* offset on R channal */ 255 | int offset_g; /* offset on G channal */ 256 | int offset_b; /* offset on B channal */ 257 | } im_nn_t; 258 | 259 | /* im_info definition */ 260 | typedef struct { 261 | void* vir_addr; /* virtual address */ 262 | void* phy_addr; /* physical address */ 263 | int fd; /* shared fd */ 264 | 265 | int width; /* width */ 266 | int height; /* height */ 267 | int wstride; /* wstride */ 268 | int hstride; /* hstride */ 269 | int format; /* format */ 270 | 271 | int color_space_mode; /* color_space_mode */ 272 | int global_alpha; /* global_alpha */ 273 | int rd_mode; 274 | 275 | /* legarcy */ 276 | int color; /* color, used by color fill */ 277 | im_colorkey_range colorkey_range; /* range value of color key */ 278 | im_nn_t nn; 279 | int rop_code; 280 | 281 | rga_buffer_handle_t handle; /* buffer handle */ 282 | } rga_buffer_t; 283 | 284 | typedef struct im_color { 285 | union { 286 | struct { 287 | uint8_t red; 288 | uint8_t green; 289 | uint8_t blue; 290 | uint8_t alpha; 291 | }; 292 | uint32_t value; 293 | }; 294 | } im_color_t; 295 | 296 | typedef struct im_osd_invert_factor { 297 | uint8_t alpha_max; 298 | uint8_t alpha_min; 299 | uint8_t yg_max; 300 | uint8_t yg_min; 301 | uint8_t crb_max; 302 | uint8_t crb_min; 303 | } im_osd_invert_factor_t; 304 | 305 | typedef struct im_osd_bpp2 { 306 | uint8_t ac_swap; // ac swap flag 307 | // 0: CA 308 | // 1: AC 309 | uint8_t endian_swap; // rgba2bpp endian swap 310 | // 0: Big endian 311 | // 1: Little endian 312 | im_color_t color0; 313 | im_color_t color1; 314 | } im_osd_bpp2_t; 315 | 316 | typedef struct im_osd_block { 317 | int width_mode; // normal or different 318 | // IM_OSD_BLOCK_MODE_NORMAL 319 | // IM_OSD_BLOCK_MODE_DIFFERENT 320 | union { 321 | int width; // normal_mode block width 322 | int width_index; // different_mode block width index in RAM 323 | }; 324 | 325 | int block_count; // block count 326 | 327 | int background_config; // background config is bright or dark 328 | // IM_OSD_BACKGROUND_DEFAULT_BRIGHT 329 | // IM_OSD_BACKGROUND_DEFAULT_DARK 330 | 331 | int direction; // osd block direction 332 | // IM_OSD_MODE_HORIZONTAL 333 | // IM_OSD_MODE_VERTICAL 334 | 335 | int color_mode; // using src1 color or config color 336 | // IM_OSD_COLOR_PIXEL 337 | // IM_OSD_COLOR_EXTERNAL 338 | im_color_t normal_color; // config color: normal 339 | im_color_t invert_color; // config color: invert 340 | } im_osd_block_t; 341 | 342 | typedef struct im_osd_invert { 343 | int invert_channel; // invert channel config: 344 | // IM_OSD_INVERT_CHANNEL_NONE 345 | // IM_OSD_INVERT_CHANNEL_Y_G 346 | // IM_OSD_INVERT_CHANNEL_C_RB 347 | // IM_OSD_INVERT_CHANNEL_ALPHA 348 | // IM_OSD_INVERT_CHANNEL_COLOR 349 | // IM_OSD_INVERT_CHANNEL_BOTH 350 | int flags_mode; // use external or inertnal RAM invert flags 351 | // IM_OSD_FLAGS_EXTERNAL 352 | // IM_OSD_FLAGS_INTERNAL 353 | int flags_index; // flags index when using internal RAM invert flags 354 | 355 | uint64_t invert_flags; // external invert flags 356 | uint64_t current_flags; // current flags 357 | 358 | int invert_mode; // invert use swap or factor 359 | // IM_OSD_INVERT_USE_FACTOR 360 | // IM_OSD_INVERT_USE_SWAP 361 | im_osd_invert_factor_t factor; 362 | 363 | int threash; 364 | } im_osd_invert_t; 365 | 366 | typedef struct im_osd { 367 | int osd_mode; // osd mode: statistics or auto_invert 368 | // IM_OSD_MODE_STATISTICS 369 | // IM_OSD_MODE_AUTO_INVERT 370 | im_osd_block_t block_parm; // osd block info config 371 | 372 | im_osd_invert_t invert_config; 373 | 374 | im_osd_bpp2_t bpp2_info; 375 | } im_osd_t; 376 | 377 | typedef struct im_intr_config { 378 | uint32_t flags; 379 | 380 | int read_threshold; 381 | int write_start; 382 | int write_step; 383 | } im_intr_config_t; 384 | 385 | typedef struct im_opt { 386 | im_api_version_t version; 387 | 388 | int color; /* color, used by color fill */ 389 | im_colorkey_range colorkey_range; /* range value of color key */ 390 | im_nn_t nn; 391 | int rop_code; 392 | 393 | int priority; 394 | int core; 395 | 396 | int mosaic_mode; 397 | 398 | im_osd_t osd_config; 399 | 400 | im_intr_config_t intr_config; 401 | 402 | char reserve[128]; 403 | } im_opt_t; 404 | 405 | typedef struct im_context { 406 | int priority; 407 | IM_SCHEDULER_CORE core; 408 | int check_mode; 409 | } im_context_t; 410 | 411 | typedef struct im_handle_param { 412 | uint32_t width; 413 | uint32_t height; 414 | uint32_t format; 415 | }im_handle_param_t; 416 | 417 | #endif /* _RGA_IM2D_TYPE_H_ */ 418 | -------------------------------------------------------------------------------- /cpp/include/im2d_version.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2022 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Cerf Yu 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _RGA_IM2D_VERSION_H_ 20 | #define _RGA_IM2D_VERSION_H_ 21 | 22 | #define RGA_VERSION_STR_HELPER(x) #x 23 | #define RGA_VERSION_STR(x) RGA_VERSION_STR_HELPER(x) 24 | 25 | /* RGA im2d api verison */ 26 | #define RGA_API_MAJOR_VERSION 1 27 | #define RGA_API_MINOR_VERSION 8 28 | #define RGA_API_REVISION_VERSION 0 29 | #define RGA_API_BUILD_VERSION 0 30 | 31 | #define RGA_API_VERSION \ 32 | RGA_VERSION_STR(RGA_API_MAJOR_VERSION) "." \ 33 | RGA_VERSION_STR(RGA_API_MINOR_VERSION) "." \ 34 | RGA_VERSION_STR(RGA_API_REVISION_VERSION) "_[" \ 35 | RGA_VERSION_STR(RGA_API_BUILD_VERSION) "]" 36 | #define RGA_API_FULL_VERSION "rga_api version " RGA_API_VERSION 37 | 38 | #define RGA_SET_CURRENT_API_VERISON (\ 39 | (RGA_API_MAJOR_VERSION & 0xff) << 24 | \ 40 | (RGA_API_MINOR_VERSION & 0xff) << 16 | \ 41 | (RGA_API_REVISION_VERSION & 0xff) << 8 | \ 42 | (RGA_API_BUILD_VERSION & 0xff)\ 43 | ) 44 | 45 | #endif /* _RGA_IM2D_VERSION_H_ */ 46 | -------------------------------------------------------------------------------- /cpp/include/postprocess.h: -------------------------------------------------------------------------------- 1 | #ifndef _RKNN_ZERO_COPY_DEMO_POSTPROCESS_H_ 2 | #define _RKNN_ZERO_COPY_DEMO_POSTPROCESS_H_ 3 | 4 | #include 5 | #include 6 | 7 | 8 | #define OBJ_CLASS_NUM 11 //8个点，外加3个类，所以配置成11，以对齐普通的检测 9 | 10 | #define OBJ_NAME_MAX_SIZE 20 11 | #define OBJ_NUMB_MAX_SIZE 64 12 | #define NMS_THRESH 0.45 13 | #define BOX_THRESH 0.25 14 | #define PROP_BOX_SIZE (5+OBJ_CLASS_NUM) 15 | 16 | typedef struct _BOX_RECT 17 | { 18 | int left; 19 | int right; 20 | int top; 21 | int bottom; 22 | } BOX_RECT; 23 | 24 | typedef struct __detect_result_t 25 | { 26 | char name[OBJ_NAME_MAX_SIZE]; 27 | BOX_RECT box; 28 | int landmarks[8]; 29 | float prop; 30 | int class_id; 31 | } detect_result_t; 32 | 33 | typedef struct _detect_result_group_t 34 | { 35 | int id; 36 | int count; 37 | detect_result_t results[OBJ_NUMB_MAX_SIZE]; 38 | } detect_result_group_t; 39 | 40 | int post_process(int8_t *input0, int8_t *input1, int8_t *input2, int model_in_h, int model_in_w, 41 | float conf_threshold, float nms_threshold, BOX_RECT pads, float scale_w, float scale_h, 42 | std::vector &qnt_zps, std::vector &qnt_scales, 43 | detect_result_group_t *group); 44 | 45 | void deinitPostProcess(); 46 | #endif //_RKNN_ZERO_COPY_DEMO_POSTPROCESS_H_ 47 | -------------------------------------------------------------------------------- /cpp/include/preprocess.h: -------------------------------------------------------------------------------- 1 | #ifndef _RKNN_YOLOV5_DEMO_PREPROCESS_H_ 2 | #define _RKNN_YOLOV5_DEMO_PREPROCESS_H_ 3 | 4 | #include 5 | #include "im2d.h" 6 | #include "rga.h" 7 | #include "opencv2/core/core.hpp" 8 | #include "opencv2/imgcodecs.hpp" 9 | #include "opencv2/imgproc.hpp" 10 | #include "postprocess.h" 11 | 12 | void letterbox(const cv::Mat &image, cv::Mat &padded_image, BOX_RECT &pads, const float scale, const cv::Size &target_size, const cv::Scalar &pad_color = cv::Scalar(128, 128, 128)); 13 | 14 | int resize_rga(rga_buffer_t &src, rga_buffer_t &dst, const cv::Mat &image, cv::Mat &resized_image, const cv::Size &target_size); 15 | 16 | #endif //_RKNN_YOLOV5_DEMO_PREPROCESS_H_ 17 | -------------------------------------------------------------------------------- /cpp/include/rga.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2016 Rockchip Electronics Co., Ltd. 3 | * Authors: 4 | * Zhiqin Wei 5 | * 6 | * Licensed under the Apache License, Version 2.0 (the "License"); 7 | * you may not use this file except in compliance with the License. 8 | * You may obtain a copy of the License at 9 | * 10 | * http://www.apache.org/licenses/LICENSE-2.0 11 | * 12 | * Unless required by applicable law or agreed to in writing, software 13 | * distributed under the License is distributed on an "AS IS" BASIS, 14 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | * See the License for the specific language governing permissions and 16 | * limitations under the License. 17 | */ 18 | 19 | #ifndef _RGA_DRIVER_H_ 20 | #define _RGA_DRIVER_H_ 21 | 22 | #include 23 | #include 24 | 25 | #ifdef __cplusplus 26 | extern "C" 27 | { 28 | #endif 29 | 30 | /* Use 'r' as magic number */ 31 | #define RGA_IOC_MAGIC 'r' 32 | #define RGA_IOW(nr, type) _IOW(RGA_IOC_MAGIC, nr, type) 33 | #define RGA_IOR(nr, type) _IOR(RGA_IOC_MAGIC, nr, type) 34 | #define RGA_IOWR(nr, type) _IOWR(RGA_IOC_MAGIC, nr, type) 35 | 36 | #define RGA_IOC_GET_DRVIER_VERSION RGA_IOR(0x1, struct rga_version_t) 37 | #define RGA_IOC_GET_HW_VERSION RGA_IOR(0x2, struct rga_hw_versions_t) 38 | #define RGA_IOC_IMPORT_BUFFER RGA_IOWR(0x3, struct rga_buffer_pool) 39 | #define RGA_IOC_RELEASE_BUFFER RGA_IOW(0x4, struct rga_buffer_pool) 40 | #define RGA_START_CONFIG RGA_IOR(0x5, uint32_t) 41 | #define RGA_END_CONFIG RGA_IOWR(0x6, struct rga_user_ctx_t) 42 | #define RGA_CMD_CONFIG RGA_IOWR(0x7, struct rga_user_ctx_t) 43 | #define RGA_CANCEL_CONFIG RGA_IOWR(0x8, uint32_t) 44 | 45 | #define RGA_BLIT_SYNC 0x5017 46 | #define RGA_BLIT_ASYNC 0x5018 47 | #define RGA_FLUSH 0x5019 48 | #define RGA_GET_RESULT 0x501a 49 | #define RGA_GET_VERSION 0x501b 50 | 51 | #define RGA2_BLIT_SYNC 0x6017 52 | #define RGA2_BLIT_ASYNC 0x6018 53 | #define RGA2_FLUSH 0x6019 54 | #define RGA2_GET_RESULT 0x601a 55 | #define RGA2_GET_VERSION 0x601b 56 | 57 | #define RGA_REG_CTRL_LEN 0x8 /* 8 */ 58 | #define RGA_REG_CMD_LEN 0x1c /* 28 */ 59 | #define RGA_CMD_BUF_SIZE 0x700 /* 16*28*4 */ 60 | 61 | #ifndef ENABLE 62 | #define ENABLE 1 63 | #endif 64 | 65 | #ifndef DISABLE 66 | #define DISABLE 0 67 | #endif 68 | 69 | enum rga_memory_type { 70 | RGA_DMA_BUFFER = 0, 71 | RGA_VIRTUAL_ADDRESS, 72 | RGA_PHYSICAL_ADDRESS 73 | }; 74 | 75 | /* RGA process mode enum */ 76 | enum { 77 | bitblt_mode = 0x0, 78 | color_palette_mode = 0x1, 79 | color_fill_mode = 0x2, 80 | line_point_drawing_mode = 0x3, 81 | blur_sharp_filter_mode = 0x4, 82 | pre_scaling_mode = 0x5, 83 | update_palette_table_mode = 0x6, 84 | update_patten_buff_mode = 0x7, 85 | }; 86 | 87 | 88 | enum { 89 | rop_enable_mask = 0x2, 90 | dither_enable_mask = 0x8, 91 | fading_enable_mask = 0x10, 92 | PD_enbale_mask = 0x20, 93 | }; 94 | 95 | enum { 96 | yuv2rgb_mode0 = 0x0, /* BT.601 MPEG */ 97 | yuv2rgb_mode1 = 0x1, /* BT.601 JPEG */ 98 | yuv2rgb_mode2 = 0x2, /* BT.709 */ 99 | 100 | rgb2yuv_601_full = 0x1 << 8, 101 | rgb2yuv_709_full = 0x2 << 8, 102 | yuv2yuv_601_limit_2_709_limit = 0x3 << 8, 103 | yuv2yuv_601_limit_2_709_full = 0x4 << 8, 104 | yuv2yuv_709_limit_2_601_limit = 0x5 << 8, 105 | yuv2yuv_709_limit_2_601_full = 0x6 << 8, //not support 106 | yuv2yuv_601_full_2_709_limit = 0x7 << 8, 107 | yuv2yuv_601_full_2_709_full = 0x8 << 8, //not support 108 | yuv2yuv_709_full_2_601_limit = 0x9 << 8, //not support 109 | yuv2yuv_709_full_2_601_full = 0xa << 8, //not support 110 | full_csc_mask = 0xf00, 111 | }; 112 | 113 | /* RGA rotate mode */ 114 | enum { 115 | rotate_mode0 = 0x0, /* no rotate */ 116 | rotate_mode1 = 0x1, /* rotate */ 117 | rotate_mode2 = 0x2, /* x_mirror */ 118 | rotate_mode3 = 0x3, /* y_mirror */ 119 | }; 120 | 121 | enum { 122 | color_palette_mode0 = 0x0, /* 1K */ 123 | color_palette_mode1 = 0x1, /* 2K */ 124 | color_palette_mode2 = 0x2, /* 4K */ 125 | color_palette_mode3 = 0x3, /* 8K */ 126 | }; 127 | 128 | enum { 129 | BB_BYPASS = 0x0, /* no rotate */ 130 | BB_ROTATE = 0x1, /* rotate */ 131 | BB_X_MIRROR = 0x2, /* x_mirror */ 132 | BB_Y_MIRROR = 0x3 /* y_mirror */ 133 | }; 134 | 135 | enum { 136 | nearby = 0x0, /* no rotate */ 137 | bilinear = 0x1, /* rotate */ 138 | bicubic = 0x2, /* x_mirror */ 139 | }; 140 | 141 | #define RGA_SCHED_PRIORITY_DEFAULT 0 142 | #define RGA_SCHED_PRIORITY_MAX 6 143 | 144 | enum { 145 | RGA3_SCHEDULER_CORE0 = 1 << 0, 146 | RGA3_SCHEDULER_CORE1 = 1 << 1, 147 | RGA2_SCHEDULER_CORE0 = 1 << 2, 148 | }; 149 | 150 | /* 151 | // Alpha Red Green Blue 152 | { 4, 32, {{32,24, 8, 0, 16, 8, 24,16 }}, GGL_RGBA }, // RK_FORMAT_RGBA_8888 153 | { 4, 24, {{ 0, 0, 8, 0, 16, 8, 24,16 }}, GGL_RGB }, // RK_FORMAT_RGBX_8888 154 | { 3, 24, {{ 0, 0, 8, 0, 16, 8, 24,16 }}, GGL_RGB }, // RK_FORMAT_RGB_888 155 | { 4, 32, {{32,24, 24,16, 16, 8, 8, 0 }}, GGL_BGRA }, // RK_FORMAT_BGRA_8888 156 | { 2, 16, {{ 0, 0, 16,11, 11, 5, 5, 0 }}, GGL_RGB }, // RK_FORMAT_RGB_565 157 | { 2, 16, {{ 1, 0, 16,11, 11, 6, 6, 1 }}, GGL_RGBA }, // RK_FORMAT_RGBA_5551 158 | { 2, 16, {{ 4, 0, 16,12, 12, 8, 8, 4 }}, GGL_RGBA }, // RK_FORMAT_RGBA_4444 159 | { 3, 24, {{ 0, 0, 24,16, 16, 8, 8, 0 }}, GGL_BGR }, // RK_FORMAT_BGB_888 160 | 161 | */ 162 | /* In order to be compatible with RK_FORMAT_XX and HAL_PIXEL_FORMAT_XX, 163 | * RK_FORMAT_XX is shifted to the left by 8 bits to distinguish. */ 164 | typedef enum _Rga_SURF_FORMAT { 165 | RK_FORMAT_RGBA_8888 = 0x0 << 8, 166 | RK_FORMAT_RGBX_8888 = 0x1 << 8, 167 | RK_FORMAT_RGB_888 = 0x2 << 8, 168 | RK_FORMAT_BGRA_8888 = 0x3 << 8, 169 | RK_FORMAT_RGB_565 = 0x4 << 8, 170 | RK_FORMAT_RGBA_5551 = 0x5 << 8, 171 | RK_FORMAT_RGBA_4444 = 0x6 << 8, 172 | RK_FORMAT_BGR_888 = 0x7 << 8, 173 | 174 | RK_FORMAT_YCbCr_422_SP = 0x8 << 8, 175 | RK_FORMAT_YCbCr_422_P = 0x9 << 8, 176 | RK_FORMAT_YCbCr_420_SP = 0xa << 8, 177 | RK_FORMAT_YCbCr_420_P = 0xb << 8, 178 | 179 | RK_FORMAT_YCrCb_422_SP = 0xc << 8, 180 | RK_FORMAT_YCrCb_422_P = 0xd << 8, 181 | RK_FORMAT_YCrCb_420_SP = 0xe << 8, 182 | RK_FORMAT_YCrCb_420_P = 0xf << 8, 183 | 184 | RK_FORMAT_BPP1 = 0x10 << 8, 185 | RK_FORMAT_BPP2 = 0x11 << 8, 186 | RK_FORMAT_BPP4 = 0x12 << 8, 187 | RK_FORMAT_BPP8 = 0x13 << 8, 188 | 189 | RK_FORMAT_Y4 = 0x14 << 8, 190 | RK_FORMAT_YCbCr_400 = 0x15 << 8, 191 | 192 | RK_FORMAT_BGRX_8888 = 0x16 << 8, 193 | 194 | RK_FORMAT_YVYU_422 = 0x18 << 8, 195 | RK_FORMAT_YVYU_420 = 0x19 << 8, 196 | RK_FORMAT_VYUY_422 = 0x1a << 8, 197 | RK_FORMAT_VYUY_420 = 0x1b << 8, 198 | RK_FORMAT_YUYV_422 = 0x1c << 8, 199 | RK_FORMAT_YUYV_420 = 0x1d << 8, 200 | RK_FORMAT_UYVY_422 = 0x1e << 8, 201 | RK_FORMAT_UYVY_420 = 0x1f << 8, 202 | 203 | RK_FORMAT_YCbCr_420_SP_10B = 0x20 << 8, 204 | RK_FORMAT_YCrCb_420_SP_10B = 0x21 << 8, 205 | RK_FORMAT_YCbCr_422_SP_10B = 0x22 << 8, 206 | RK_FORMAT_YCrCb_422_SP_10B = 0x23 << 8, 207 | /* For compatibility with misspellings */ 208 | RK_FORMAT_YCbCr_422_10b_SP = RK_FORMAT_YCbCr_422_SP_10B, 209 | RK_FORMAT_YCrCb_422_10b_SP = RK_FORMAT_YCrCb_422_SP_10B, 210 | 211 | RK_FORMAT_BGR_565 = 0x24 << 8, 212 | RK_FORMAT_BGRA_5551 = 0x25 << 8, 213 | RK_FORMAT_BGRA_4444 = 0x26 << 8, 214 | 215 | RK_FORMAT_ARGB_8888 = 0x28 << 8, 216 | RK_FORMAT_XRGB_8888 = 0x29 << 8, 217 | RK_FORMAT_ARGB_5551 = 0x2a << 8, 218 | RK_FORMAT_ARGB_4444 = 0x2b << 8, 219 | RK_FORMAT_ABGR_8888 = 0x2c << 8, 220 | RK_FORMAT_XBGR_8888 = 0x2d << 8, 221 | RK_FORMAT_ABGR_5551 = 0x2e << 8, 222 | RK_FORMAT_ABGR_4444 = 0x2f << 8, 223 | 224 | RK_FORMAT_RGBA2BPP = 0x30 << 8, 225 | 226 | RK_FORMAT_UNKNOWN = 0x100 << 8, 227 | } RgaSURF_FORMAT; 228 | 229 | /* RGA3 rd_mode */ 230 | enum 231 | { 232 | raster_mode = 0x1 << 0, 233 | fbc_mode = 0x1 << 1, 234 | tile_mode = 0x1 << 2, 235 | }; 236 | 237 | typedef struct rga_img_info_t { 238 | uint64_t yrgb_addr; /* yrgb mem addr */ 239 | uint64_t uv_addr; /* cb/cr mem addr */ 240 | uint64_t v_addr; /* cr mem addr */ 241 | 242 | uint32_t format; //definition by RK_FORMAT 243 | uint16_t act_w; 244 | uint16_t act_h; 245 | uint16_t x_offset; 246 | uint16_t y_offset; 247 | 248 | uint16_t vir_w; 249 | uint16_t vir_h; 250 | 251 | uint16_t endian_mode; //for BPP 252 | uint16_t alpha_swap; 253 | 254 | //used by RGA3 255 | uint16_t rotate_mode; 256 | uint16_t rd_mode; 257 | 258 | uint16_t is_10b_compact; 259 | uint16_t is_10b_endian; 260 | 261 | uint16_t enable; 262 | } 263 | rga_img_info_t; 264 | 265 | typedef struct POINT { 266 | uint16_t x; 267 | uint16_t y; 268 | } 269 | POINT; 270 | 271 | typedef struct RECT { 272 | uint16_t xmin; 273 | uint16_t xmax; // width - 1 274 | uint16_t ymin; 275 | uint16_t ymax; // height - 1 276 | } RECT; 277 | 278 | typedef struct MMU { 279 | uint8_t mmu_en; 280 | uint64_t base_addr; 281 | uint32_t mmu_flag; /* [0] mmu enable [1] src_flush [2] dst_flush [3] CMD_flush [4~5] page size*/ 282 | } MMU; 283 | 284 | typedef struct COLOR_FILL { 285 | int16_t gr_x_a; 286 | int16_t gr_y_a; 287 | int16_t gr_x_b; 288 | int16_t gr_y_b; 289 | int16_t gr_x_g; 290 | int16_t gr_y_g; 291 | int16_t gr_x_r; 292 | int16_t gr_y_r; 293 | //u8 cp_gr_saturation; 294 | } 295 | COLOR_FILL; 296 | 297 | typedef struct FADING { 298 | uint8_t b; 299 | uint8_t g; 300 | uint8_t r; 301 | uint8_t res; 302 | } 303 | FADING; 304 | 305 | typedef struct line_draw_t { 306 | POINT start_point; /* LineDraw_start_point */ 307 | POINT end_point; /* LineDraw_end_point */ 308 | uint32_t color; /* LineDraw_color */ 309 | uint32_t flag; /* (enum) LineDrawing mode sel */ 310 | uint32_t line_width; /* range 1~16 */ 311 | } 312 | line_draw_t; 313 | 314 | /* color space convert coefficient. */ 315 | typedef struct csc_coe_t { 316 | int16_t r_v; 317 | int16_t g_y; 318 | int16_t b_u; 319 | int32_t off; 320 | } csc_coe_t; 321 | 322 | typedef struct full_csc_t { 323 | uint8_t flag; 324 | csc_coe_t coe_y; 325 | csc_coe_t coe_u; 326 | csc_coe_t coe_v; 327 | } full_csc_t; 328 | 329 | struct rga_mosaic_info { 330 | uint8_t enable; 331 | uint8_t mode; 332 | }; 333 | 334 | struct rga_pre_intr_info { 335 | uint8_t enable; 336 | 337 | uint8_t read_intr_en; 338 | uint8_t write_intr_en; 339 | uint8_t read_hold_en; 340 | uint32_t read_threshold; 341 | uint32_t write_start; 342 | uint32_t write_step; 343 | }; 344 | 345 | /* MAX(min, (max - channel_value)) */ 346 | struct rga_osd_invert_factor { 347 | uint8_t alpha_max; 348 | uint8_t alpha_min; 349 | uint8_t yg_max; 350 | uint8_t yg_min; 351 | uint8_t crb_max; 352 | uint8_t crb_min; 353 | }; 354 | 355 | struct rga_color { 356 | union { 357 | struct { 358 | uint8_t red; 359 | uint8_t green; 360 | uint8_t blue; 361 | uint8_t alpha; 362 | }; 363 | uint32_t value; 364 | }; 365 | }; 366 | 367 | struct rga_osd_bpp2 { 368 | uint8_t ac_swap; // ac swap flag 369 | // 0: CA 370 | // 1: AC 371 | uint8_t endian_swap; // rgba2bpp endian swap 372 | // 0: Big endian 373 | // 1: Little endian 374 | struct rga_color color0; 375 | struct rga_color color1; 376 | }; 377 | 378 | struct rga_osd_mode_ctrl { 379 | uint8_t mode; // OSD cal mode: 380 | // 0b'1: statistics mode 381 | // 1b'1: auto inversion overlap mode 382 | uint8_t direction_mode; // horizontal or vertical 383 | // 0: horizontal 384 | // 1: vertical 385 | uint8_t width_mode; // using @fix_width or LUT width 386 | // 0: fix width 387 | // 1: LUT width 388 | uint16_t block_fix_width; // OSD block fixed width 389 | // real width = (fix_width + 1) * 2 390 | uint8_t block_num; // OSD block num 391 | uint16_t flags_index; // auto invert flags index 392 | 393 | /* invertion config */ 394 | uint8_t color_mode; // selete color 395 | // 0: src1 color 396 | // 1: config data color 397 | uint8_t invert_flags_mode; // invert flag selete 398 | // 0: use RAM flag 399 | // 1: usr last result 400 | uint8_t default_color_sel; // default color mode 401 | // 0: default is bright 402 | // 1: default is dark 403 | uint8_t invert_enable; // invert channel enable 404 | // 1 << 0: aplha enable 405 | // 1 << 1: Y/G disable 406 | // 1 << 2: C/RB disable 407 | uint8_t invert_mode; // invert cal mode 408 | // 0: normal(max-data) 409 | // 1: swap 410 | uint8_t invert_thresh; // if luma > thresh, osd_flag to be 1 411 | uint8_t unfix_index; // OSD width config index 412 | }; 413 | 414 | struct rga_osd_info { 415 | uint8_t enable; 416 | 417 | struct rga_osd_mode_ctrl mode_ctrl; 418 | struct rga_osd_invert_factor cal_factor; 419 | struct rga_osd_bpp2 bpp2_info; 420 | 421 | union { 422 | struct { 423 | uint32_t last_flags1; 424 | uint32_t last_flags0; 425 | }; 426 | uint64_t last_flags; 427 | }; 428 | 429 | union { 430 | struct { 431 | uint32_t cur_flags1; 432 | uint32_t cur_flags0; 433 | }; 434 | uint64_t cur_flags; 435 | }; 436 | }; 437 | 438 | #define RGA_VERSION_SIZE 16 439 | #define RGA_HW_SIZE 5 440 | 441 | struct rga_version_t { 442 | uint32_t major; 443 | uint32_t minor; 444 | uint32_t revision; 445 | uint8_t str[RGA_VERSION_SIZE]; 446 | }; 447 | 448 | struct rga_hw_versions_t { 449 | struct rga_version_t version[RGA_HW_SIZE]; 450 | uint32_t size; 451 | }; 452 | 453 | struct rga_memory_parm { 454 | uint32_t width; 455 | uint32_t height; 456 | uint32_t format; 457 | 458 | uint32_t size; 459 | }; 460 | 461 | struct rga_external_buffer { 462 | uint64_t memory; 463 | uint32_t type; 464 | 465 | uint32_t handle; 466 | struct rga_memory_parm memory_info; 467 | 468 | uint8_t reserve[252]; 469 | }; 470 | 471 | struct rga_buffer_pool { 472 | uint64_t buffers; 473 | uint32_t size; 474 | }; 475 | 476 | struct rga_req { 477 | uint8_t render_mode; /* (enum) process mode sel */ 478 | 479 | rga_img_info_t src; /* src image info */ 480 | rga_img_info_t dst; /* dst image info */ 481 | rga_img_info_t pat; /* patten image info */ 482 | 483 | uint64_t rop_mask_addr; /* rop4 mask addr */ 484 | uint64_t LUT_addr; /* LUT addr */ 485 | 486 | RECT clip; /* dst clip window default value is dst_vir */ 487 | /* value from [0, w-1] / [0, h-1]*/ 488 | 489 | int32_t sina; /* dst angle default value 0 16.16 scan from table */ 490 | int32_t cosa; /* dst angle default value 0 16.16 scan from table */ 491 | 492 | uint16_t alpha_rop_flag; /* alpha rop process flag */ 493 | /* ([0] = 1 alpha_rop_enable) */ 494 | /* ([1] = 1 rop enable) */ 495 | /* ([2] = 1 fading_enable) */ 496 | /* ([3] = 1 PD_enable) */ 497 | /* ([4] = 1 alpha cal_mode_sel) */ 498 | /* ([5] = 1 dither_enable) */ 499 | /* ([6] = 1 gradient fill mode sel) */ 500 | /* ([7] = 1 AA_enable) */ 501 | /* ([8] = 1 nn_quantize) */ 502 | /* ([9] = 1 Real color mode) */ 503 | 504 | uint8_t scale_mode; /* 0 nearst / 1 bilnear / 2 bicubic */ 505 | 506 | uint32_t color_key_max; /* color key max */ 507 | uint32_t color_key_min; /* color key min */ 508 | 509 | uint32_t fg_color; /* foreground color */ 510 | uint32_t bg_color; /* background color */ 511 | 512 | COLOR_FILL gr_color; /* color fill use gradient */ 513 | 514 | line_draw_t line_draw_info; 515 | 516 | FADING fading; 517 | 518 | uint8_t PD_mode; /* porter duff alpha mode sel */ 519 | 520 | uint8_t alpha_global_value; /* global alpha value */ 521 | 522 | uint16_t rop_code; /* rop2/3/4 code scan from rop code table*/ 523 | 524 | uint8_t bsfilter_flag; /* [2] 0 blur 1 sharp / [1:0] filter_type*/ 525 | 526 | uint8_t palette_mode; /* (enum) color palatte 0/1bpp, 1/2bpp 2/4bpp 3/8bpp*/ 527 | 528 | uint8_t yuv2rgb_mode; /* (enum) BT.601 MPEG / BT.601 JPEG / BT.709 */ 529 | 530 | uint8_t endian_mode; /* 0/big endian 1/little endian*/ 531 | 532 | uint8_t rotate_mode; /* (enum) rotate mode */ 533 | /* 0x0, no rotate */ 534 | /* 0x1, rotate */ 535 | /* 0x2, x_mirror */ 536 | /* 0x3, y_mirror */ 537 | 538 | uint8_t color_fill_mode; /* 0 solid color / 1 patten color */ 539 | 540 | MMU mmu_info; /* mmu information */ 541 | 542 | uint8_t alpha_rop_mode; /* ([0~1] alpha mode) */ 543 | /* ([2~3] rop mode) */ 544 | /* ([4] zero mode en) */ 545 | /* ([5] dst alpha mode) (RGA1) */ 546 | 547 | uint8_t src_trans_mode; 548 | 549 | uint8_t dither_mode; 550 | 551 | full_csc_t full_csc; /* full color space convert */ 552 | 553 | int32_t in_fence_fd; 554 | uint8_t core; 555 | uint8_t priority; 556 | int32_t out_fence_fd; 557 | 558 | uint8_t handle_flag; 559 | 560 | /* RGA2 1106 add */ 561 | struct rga_mosaic_info mosaic_info; 562 | 563 | uint8_t uvhds_mode; 564 | uint8_t uvvds_mode; 565 | 566 | struct rga_osd_info osd_info; 567 | 568 | struct rga_pre_intr_info pre_intr_info; 569 | 570 | uint8_t reservr[59]; 571 | }; 572 | 573 | struct rga_user_ctx_t { 574 | uint64_t cmd_ptr; 575 | uint32_t cmd_num; 576 | uint32_t id; 577 | uint32_t sync_mode; 578 | uint32_t out_fence_fd; 579 | 580 | uint32_t mpi_config_flags; 581 | 582 | uint8_t reservr[124]; 583 | }; 584 | 585 | #ifdef __cplusplus 586 | } 587 | #endif 588 | 589 | #endif /*_RK29_IPP_DRIVER_H_*/ 590 | -------------------------------------------------------------------------------- /cpp/include/rga_func.h: -------------------------------------------------------------------------------- 1 | #ifndef __RGA_FUNC_H__ 2 | #define __RGA_FUNC_H__ 3 | 4 | #include 5 | #include "RgaApi.h" 6 | 7 | #ifdef __cplusplus 8 | extern "C" { 9 | #endif 10 | 11 | typedef int(* FUNC_RGA_INIT)(); 12 | typedef void(* FUNC_RGA_DEINIT)(); 13 | typedef int(* FUNC_RGA_BLIT)(rga_info_t *, rga_info_t *, rga_info_t *); 14 | 15 | typedef struct _rga_context{ 16 | void *rga_handle; 17 | FUNC_RGA_INIT init_func; 18 | FUNC_RGA_DEINIT deinit_func; 19 | FUNC_RGA_BLIT blit_func; 20 | } rga_context; 21 | 22 | int RGA_init(rga_context* rga_ctx); 23 | 24 | void img_resize_fast(rga_context *rga_ctx, int src_fd, int src_w, int src_h, uint64_t dst_phys, int dst_w, int dst_h); 25 | 26 | void img_resize_slow(rga_context *rga_ctx, void *src_virt, int src_w, int src_h, void *dst_virt, int dst_w, int dst_h); 27 | 28 | int RGA_deinit(rga_context* rga_ctx); 29 | 30 | #ifdef __cplusplus 31 | } 32 | #endif 33 | #endif/*__RGA_FUNC_H__*/ 34 | -------------------------------------------------------------------------------- /cpp/include/rknn_matmul_api.h: -------------------------------------------------------------------------------- 1 | /**************************************************************************** 2 | * 3 | * Copyright (c) 2017 - 2018 by Rockchip Corp. All rights reserved. 4 | * 5 | * The material in this file is confidential and contains trade secrets 6 | * of Rockchip Corporation. This is proprietary information owned by 7 | * Rockchip Corporation. No part of this work may be disclosed, 8 | * reproduced, copied, transmitted, or used in any way for any purpose, 9 | * without the express written permission of Rockchip Corporation. 10 | * 11 | *****************************************************************************/ 12 | 13 | #ifndef _RKNN_MATMUL_API_H 14 | #define _RKNN_MATMUL_API_H 15 | 16 | #ifdef __cplusplus 17 | extern "C" { 18 | #endif 19 | 20 | #include "rknn_api.h" 21 | 22 | typedef rknn_context rknn_matmul_ctx; 23 | 24 | typedef struct _rknn_matmul_tensor_attr 25 | { 26 | char name[RKNN_MAX_NAME_LEN]; 27 | 28 | // indicate A(M, K) or B(K, N) or C(M, N) 29 | uint32_t n_dims; 30 | uint32_t dims[RKNN_MAX_DIMS]; 31 | 32 | // matmul tensor size 33 | uint32_t size; 34 | 35 | // matmul tensor data type 36 | // int8 : A, B 37 | // int32: C 38 | rknn_tensor_type type; 39 | } rknn_matmul_tensor_attr; 40 | 41 | typedef struct _rknn_matmul_io_attr 42 | { 43 | // indicate A(M, K) or B(K, N) or C(M, N) 44 | rknn_matmul_tensor_attr A; 45 | rknn_matmul_tensor_attr B; 46 | rknn_matmul_tensor_attr C; 47 | } rknn_matmul_io_attr; 48 | 49 | /* 50 | matmul information struct 51 | */ 52 | typedef struct rknn_matmul_info_t 53 | { 54 | int32_t M; 55 | int32_t K; // limit: rk356x: int8 type must be aligned with 32byte, float16 type must be aligned with 16byte; 56 | // rk3588: int8 type must be aligned with 32byte, float16 type must be aligned with 32byte; 57 | int32_t N; // limit: rk356x: int8 type must be aligned with 16byte, float16 type must be aligned with 8byte; 58 | // rk3588: int8 type must be aligned with 32byte, float16 type must be aligned with 16byte; 59 | 60 | // matmul data type 61 | // int8: int8(A) x int8(B) -> int32(C) 62 | // float16: float16(A) x float16(B) -> float32(C) 63 | rknn_tensor_type type; 64 | 65 | // matmul native layout for B 66 | // 0: normal layout 67 | // 1: native layout 68 | int32_t native_layout; 69 | 70 | // matmul perf layout for A and C 71 | // 0: normal layout 72 | // 1: perf layout 73 | int32_t perf_layout; 74 | } rknn_matmul_info; 75 | 76 | /* rknn_matmul_create 77 | 78 | params: 79 | rknn_matmul_ctx *ctx the handle of context. 80 | rknn_matmul_info *info the matmal information. 81 | rknn_matmul_io_attr *io_attr inputs/output attribute 82 | return: 83 | int error code 84 | */ 85 | int rknn_matmul_create(rknn_matmul_ctx* ctx, rknn_matmul_info* info, rknn_matmul_io_attr* io_attr); 86 | 87 | /* rknn_matmul_set_io_mem 88 | 89 | params: 90 | rknn_matmul_ctx ctx the handle of context. 91 | rknn_tensor_mem *mem the pointer of tensor memory information. 92 | rknn_matmul_tensor_attr *attr the attribute of input or output tensor buffer. 93 | return: 94 | int error code. 95 | 96 | formula: 97 | C = A * B, 98 | 99 | limit: 100 | K <= 4096 101 | K limit: rk356x: int8 type must be aligned with 32byte, float16 type must be aligned with 16byte; 102 | rk3588: int8 type must be aligned with 32byte, float16 type must be aligned with 32byte; 103 | N limit: rk356x: int8 type must be aligned with 16byte, float16 type must be aligned with 8byte; 104 | rk3588: int8 type must be aligned with 32byte, float16 type must be aligned with 16byte; 105 | 106 | A shape: M x K 107 | normal layout: (M, K) 108 | [M1K1, M1K2, ..., M1Kk, 109 | M2K1, M2K2, ..., M2Kk, 110 | ... 111 | MmK1, MmK2, ..., MmKk] 112 | for rk356x： 113 | int8: 114 | perf layout: (K / 8, M, 8) 115 | [K1M1, K2M1, ..., K8M1, 116 | K9M2, K10M2, ..., K16M2, 117 | ... 118 | K(k-7)Mm, K(k-6)Mm, ..., KkMm] 119 | float16: 120 | perf layout: (K / 4, M, 4) 121 | [K1M1, K2M1, ..., K4M1, 122 | K9M2, K10M2, ..., K8M2, 123 | ... 124 | K(k-3)Mm, K(k-2)Mm, ..., KkMm] 125 | for rk3588： 126 | int8: 127 | perf layout: (K / 16, M, 16) 128 | [K1M1, K2M1, ..., K16M1, 129 | K9M2, K10M2, ..., K32M2, 130 | ... 131 | K(k-15)Mm, K(k-14)Mm, ..., KkMm] 132 | float16: 133 | perf layout: (K / 8, M, 8) 134 | [K1M1, K2M1, ..., K8M1, 135 | K9M2, K10M2, ..., K16M2, 136 | ... 137 | K(k-7)Mm, K(k-6)Mm, ..., KkMm] 138 | B shape: K x N 139 | normal layout: (K, N) 140 | [K1N1, K1N2, ..., K1Nn, 141 | K2N1, K2N2, ..., K2Nn, 142 | ... 143 | KkN1, KkN2, ..., KkNn] 144 | for rk356x： 145 | int8: 146 | native layout: (N / 16, K / 32, 16, 32) 147 | [K1N1, K2N1, ..., K32N1, 148 | K1N2, K2N2, ..., K32N2, 149 | ... 150 | K1N16, K2N16, ..., K32N16, 151 | K33N1, K34N1, ..., K64N1, 152 | K33N2, K34N2, ..., K64N2, 153 | ... 154 | K(k-31)N16, K(k-30)N16, ..., KkN16, 155 | K1N17, K2N17, ..., K32N17, 156 | K1N18, K2N18, ..., K32N18, 157 | ... 158 | K(k-31)Nn, K(k-30)Nn, ..., KkNn] 159 | float16: 160 | native layout: (N / 8, K / 16, 8, 16) 161 | [K1N1, K2N1, ..., K16N1, 162 | K1N2, K2N2, ..., K16N2, 163 | ... 164 | K1N8, K2N8, ..., K16N8, 165 | K17N1, K18N1, ..., K32N1, 166 | K17N2, K18N2, ..., K32N2, 167 | ... 168 | K(k-15)N8, K(k-30)N8, ..., KkN8, 169 | K1N9, K2N9, ..., K16N9, 170 | K1N10, K2N10, ..., K16N10, 171 | ... 172 | K(k-15)Nn, K(k-14)Nn, ..., KkNn] 173 | for rk3588： 174 | int8: 175 | native layout: (N / 32, K / 32, 32, 32) 176 | [K1N1, K2N1, ..., K32N1, 177 | K1N2, K2N2, ..., K32N2, 178 | ... 179 | K1N32, K2N32, ..., K32N32, 180 | K33N1, K34N1, ..., K64N1, 181 | K33N2, K34N2, ..., K64N2, 182 | ... 183 | K(k-31)N32, K(k-30)N32, ..., KkN32, 184 | K1N33, K2N33, ..., K32N33, 185 | K1N34, K2N34, ..., K32N34, 186 | ... 187 | K(k-31)Nn, K(k-30)Nn, ..., KkNn] 188 | float16: 189 | native layout: (N / 16, K / 32, 16, 32) 190 | [K1N1, K2N1, ..., K32N1, 191 | K1N2, K2N2, ..., K32N2, 192 | ... 193 | K1N16, K2N16, ..., K32N16, 194 | K33N1, K34N1, ..., K64N1, 195 | K33N2, K34N2, ..., K64N2, 196 | ... 197 | K(k-31)N16, K(k-30)N16, ..., KkN16, 198 | K1N17, K2N17, ..., K32N17, 199 | K1N18, K2N18, ..., K32N18, 200 | ... 201 | K(k-31)Nn, K(k-30)Nn, ..., KkNn] 202 | C shape: M x N 203 | normal layout: (M, N) 204 | [M1N1, M1N2, ..., M1Nn, 205 | M2N1, M2N2, ..., M2Nn, 206 | ... 207 | MmN1, MmN2, ..., MmNn] 208 | perf layout: (N / 4, M, 4) 209 | [N1M1, N2M1, ..., N4M1, 210 | N5M2, N6M2, ..., N8M2, 211 | ... 212 | N(n-3)Mm, N(n-2)Mm, ..., NnMm] 213 | */ 214 | int rknn_matmul_set_io_mem(rknn_matmul_ctx ctx, rknn_tensor_mem* mem, rknn_matmul_tensor_attr* attr); 215 | 216 | /* rknn_matmul_set_core_mask 217 | 218 | set rknn core mask.(only support rk3588 in current) 219 | 220 | RKNN_NPU_CORE_AUTO: auto mode, default value 221 | RKNN_NPU_CORE_0: core 0 mode 222 | RKNN_NPU_CORE_1: core 1 mode 223 | RKNN_NPU_CORE_2: core 2 mode 224 | RKNN_NPU_CORE_0_1: combine core 0/1 mode 225 | RKNN_NPU_CORE_0_1_2: combine core 0/1/2 mode 226 | 227 | input: 228 | rknn_matmul_ctx context the handle of context. 229 | rknn_core_mask core_mask the core mask. 230 | return: 231 | int error code. 232 | */ 233 | int rknn_matmul_set_core_mask(rknn_matmul_ctx context, rknn_core_mask core_mask); 234 | 235 | /* rknn_matmul_run 236 | 237 | run the matmul in blocking mode 238 | 239 | params: 240 | rknn_matmul_ctx ctx the handle of context. 241 | return: 242 | int error code. 243 | */ 244 | int rknn_matmul_run(rknn_matmul_ctx ctx); 245 | 246 | /* rknn_matmul_destroy 247 | 248 | destroy the matmul context 249 | 250 | params: 251 | rknn_matmul_ctx ctx the handle of context. 252 | return: 253 | int error code. 254 | */ 255 | int rknn_matmul_destroy(rknn_matmul_ctx ctx); 256 | 257 | #ifdef __cplusplus 258 | } // extern "C" 259 | #endif 260 | 261 | #endif // _RKNN_MATMUL_API_H -------------------------------------------------------------------------------- /cpp/include/yolov5.hpp: -------------------------------------------------------------------------------- 1 | //===----------------------------------------------------------------------===// 2 | // 3 | // Copyright (C) 2022 Sophgo Technologies Inc. All rights reserved. 4 | // 5 | // SOPHON-DEMO is licensed under the 2-Clause BSD License except for the 6 | // third-party components. 7 | // 8 | //===----------------------------------------------------------------------===// 9 | 10 | #ifndef YOLOV5_H 11 | #define YOLOV5_H 12 | 13 | #include 14 | #include 15 | #include "opencv2/opencv.hpp" 16 | #include "bmnn_utils.h" 17 | #include "utils.hpp" 18 | #include "bm_wrapper.hpp" 19 | // Define USE_OPENCV for enabling OPENCV related funtions in bm_wrapper.hpp 20 | #define USE_OPENCV 1 21 | #define DEBUG 0 22 | 23 | struct YoloV5Box { 24 | int x, y, width, height; 25 | float score; 26 | int class_id; 27 | }; 28 | 29 | using YoloV5BoxVec = std::vector; 30 | 31 | class YoloV5 { 32 | std::shared_ptr m_bmContext; 33 | std::shared_ptr m_bmNetwork; 34 | std::vector m_resized_imgs; 35 | std::vector m_converto_imgs; 36 | 37 | //configuration 38 | float m_confThreshold= 0.5; 39 | float m_nmsThreshold = 0.5; 40 | 41 | std::vector m_class_names; 42 | int m_class_num = 80; // default is coco names 43 | int m_net_h, m_net_w; 44 | int max_batch; 45 | int output_num; 46 | int min_dim; 47 | bmcv_convert_to_attr converto_attr; 48 | 49 | TimeStamp *m_ts; 50 | 51 | private: 52 | int pre_process(const std::vector& images); 53 | int post_process(const std::vector& images, std::vector& boxes); 54 | int argmax(float* data, int dsize); 55 | static float get_aspect_scaled_ratio(int src_w, int src_h, int dst_w, int dst_h, bool *alignWidth); 56 | static float sigmoid(float x); 57 | void NMS(YoloV5BoxVec &dets, float nmsConfidence); 58 | 59 | public: 60 | YoloV5(std::shared_ptr context); 61 | virtual ~YoloV5(); 62 | int Init(float confThresh=0.5, float nmsThresh=0.5, const std::string& coco_names_file=""); 63 | void enableProfile(TimeStamp *ts); 64 | int batch_size(); 65 | int Detect(const std::vector& images, std::vector& boxes); 66 | void drawPred(int classId, float conf, int left, int top, int right, int bottom, cv::Mat& frame); 67 | void draw_bmcv(bm_handle_t &handle, int classId, float conf, int left, int top, int right, int bottom, bm_image& frame, bool put_text_flag=false); 68 | }; 69 | 70 | #endif //!YOLOV5_H 71 | -------------------------------------------------------------------------------- /cpp/lib/librga.so: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/cpp/lib/librga.so -------------------------------------------------------------------------------- /cpp/lib/librknn_api.so: -------------------------------------------------------------------------------- 1 | librknnrt.so -------------------------------------------------------------------------------- /cpp/lib/librknnrt.so: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/cpp/lib/librknnrt.so -------------------------------------------------------------------------------- /cpp/src/main.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 by Rockchip Electronics Co., Ltd. All Rights Reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | /*------------------------------------------- 16 | Includes 17 | -------------------------------------------*/ 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | #define _BASETSD_H 25 | 26 | #include "RgaUtils.h" 27 | 28 | #include "postprocess.h" 29 | 30 | #include "rknn_api.h" 31 | #include "preprocess.h" 32 | 33 | #define PERF_WITH_POST 1 34 | /*------------------------------------------- 35 | Functions 36 | -------------------------------------------*/ 37 | 38 | static void dump_tensor_attr(rknn_tensor_attr *attr) 39 | { 40 | std::string shape_str = attr->n_dims < 1 ? "" : std::to_string(attr->dims[0]); 41 | for (int i = 1; i < attr->n_dims; ++i) 42 | { 43 | shape_str += ", " + std::to_string(attr->dims[i]); 44 | } 45 | 46 | printf(" index=%d, name=%s, n_dims=%d, dims=[%s], n_elems=%d, size=%d, w_stride = %d, size_with_stride=%d, fmt=%s, " 47 | "type=%s, qnt_type=%s, " 48 | "zp=%d, scale=%f\n", 49 | attr->index, attr->name, attr->n_dims, shape_str.c_str(), attr->n_elems, attr->size, attr->w_stride, 50 | attr->size_with_stride, get_format_string(attr->fmt), get_type_string(attr->type), 51 | get_qnt_type_string(attr->qnt_type), attr->zp, attr->scale); 52 | } 53 | 54 | double __get_us(struct timeval t) { return (t.tv_sec * 1000000 + t.tv_usec); } 55 | 56 | static unsigned char *load_data(FILE *fp, size_t ofst, size_t sz) 57 | { 58 | unsigned char *data; 59 | int ret; 60 | 61 | data = NULL; 62 | 63 | if (NULL == fp) 64 | { 65 | return NULL; 66 | } 67 | 68 | ret = fseek(fp, ofst, SEEK_SET); 69 | if (ret != 0) 70 | { 71 | printf("blob seek failure.\n"); 72 | return NULL; 73 | } 74 | 75 | data = (unsigned char *)malloc(sz); 76 | if (data == NULL) 77 | { 78 | printf("buffer malloc failure.\n"); 79 | return NULL; 80 | } 81 | ret = fread(data, 1, sz, fp); 82 | return data; 83 | } 84 | 85 | static unsigned char *load_model(const char *filename, int *model_size) 86 | { 87 | FILE *fp; 88 | unsigned char *data; 89 | 90 | fp = fopen(filename, "rb"); 91 | if (NULL == fp) 92 | { 93 | printf("Open file %s failed.\n", filename); 94 | return NULL; 95 | } 96 | 97 | fseek(fp, 0, SEEK_END); 98 | int size = ftell(fp); 99 | 100 | data = load_data(fp, 0, size); 101 | 102 | fclose(fp); 103 | 104 | *model_size = size; 105 | return data; 106 | } 107 | 108 | static int saveFloat(const char *file_name, float *output, int element_size) 109 | { 110 | FILE *fp; 111 | fp = fopen(file_name, "w"); 112 | for (int i = 0; i < element_size; i++) 113 | { 114 | fprintf(fp, "%.6f\n", output[i]); 115 | } 116 | fclose(fp); 117 | return 0; 118 | } 119 | 120 | /*------------------------------------------- 121 | Main Functions 122 | -------------------------------------------*/ 123 | int main(int argc, char **argv) 124 | { 125 | if (argc < 3) 126 | { 127 | printf("Usage: %s \n", argv[0]); 128 | return -1; 129 | } 130 | int ret; 131 | rknn_context ctx; 132 | size_t actual_size = 0; 133 | int img_width = 0; 134 | int img_height = 0; 135 | int img_channel = 0; 136 | const float nms_threshold = NMS_THRESH; // 默认的NMS阈值 137 | const float box_conf_threshold = BOX_THRESH; // 默认的置信度阈值 138 | struct timeval start_time, stop_time; 139 | char *model_name = (char *)argv[1]; 140 | char *input_path = argv[2]; 141 | std::string option = "letterbox"; 142 | std::string out_path = "./out.jpg"; 143 | if (argc >= 4) 144 | { 145 | option = argv[3]; 146 | } 147 | if (argc >= 5) 148 | { 149 | out_path = argv[4]; 150 | } 151 | 152 | // init rga context 153 | rga_buffer_t src; 154 | rga_buffer_t dst; 155 | memset(&src, 0, sizeof(src)); 156 | memset(&dst, 0, sizeof(dst)); 157 | 158 | printf("post process config: box_conf_threshold = %.2f, nms_threshold = %.2f\n", box_conf_threshold, nms_threshold); 159 | 160 | /* Create the neural network */ 161 | printf("Loading mode...\n"); 162 | int model_data_size = 0; 163 | unsigned char *model_data = load_model(model_name, &model_data_size); 164 | ret = rknn_init(&ctx, model_data, model_data_size, 0, NULL); 165 | if (ret < 0) 166 | { 167 | printf("rknn_init error ret=%d\n", ret); 168 | return -1; 169 | } 170 | 171 | rknn_sdk_version version; 172 | ret = rknn_query(ctx, RKNN_QUERY_SDK_VERSION, &version, sizeof(rknn_sdk_version)); 173 | if (ret < 0) 174 | { 175 | printf("rknn_init error ret=%d\n", ret); 176 | return -1; 177 | } 178 | printf("sdk version: %s driver version: %s\n", version.api_version, version.drv_version); 179 | 180 | rknn_input_output_num io_num; 181 | ret = rknn_query(ctx, RKNN_QUERY_IN_OUT_NUM, &io_num, sizeof(io_num)); 182 | if (ret < 0) 183 | { 184 | printf("rknn_init error ret=%d\n", ret); 185 | return -1; 186 | } 187 | printf("model input num: %d, output num: %d\n", io_num.n_input, io_num.n_output); 188 | 189 | rknn_tensor_attr input_attrs[io_num.n_input]; 190 | memset(input_attrs, 0, sizeof(input_attrs)); 191 | for (int i = 0; i < io_num.n_input; i++) 192 | { 193 | input_attrs[i].index = i; 194 | ret = rknn_query(ctx, RKNN_QUERY_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr)); 195 | if (ret < 0) 196 | { 197 | printf("rknn_init error ret=%d\n", ret); 198 | return -1; 199 | } 200 | dump_tensor_attr(&(input_attrs[i])); 201 | } 202 | 203 | rknn_tensor_attr output_attrs[io_num.n_output]; 204 | memset(output_attrs, 0, sizeof(output_attrs)); 205 | for (int i = 0; i < io_num.n_output; i++) 206 | { 207 | output_attrs[i].index = i; 208 | ret = rknn_query(ctx, RKNN_QUERY_OUTPUT_ATTR, &(output_attrs[i]), sizeof(rknn_tensor_attr)); 209 | dump_tensor_attr(&(output_attrs[i])); 210 | } 211 | 212 | int channel = 3; 213 | int width = 0; 214 | int height = 0; 215 | if (input_attrs[0].fmt == RKNN_TENSOR_NCHW) 216 | { 217 | printf("model is NCHW input fmt\n"); 218 | channel = input_attrs[0].dims[1]; 219 | height = input_attrs[0].dims[2]; 220 | width = input_attrs[0].dims[3]; 221 | } 222 | else 223 | { 224 | printf("model is NHWC input fmt\n"); 225 | height = input_attrs[0].dims[1]; 226 | width = input_attrs[0].dims[2]; 227 | channel = input_attrs[0].dims[3]; 228 | } 229 | 230 | printf("model input height=%d, width=%d, channel=%d\n", height, width, channel); 231 | 232 | rknn_input inputs[1]; 233 | memset(inputs, 0, sizeof(inputs)); 234 | inputs[0].index = 0; 235 | inputs[0].type = RKNN_TENSOR_UINT8; 236 | inputs[0].size = width * height * channel; 237 | inputs[0].fmt = RKNN_TENSOR_NHWC; 238 | inputs[0].pass_through = 0; 239 | 240 | // 读取图片 241 | printf("Read %s ...\n", input_path); 242 | cv::Mat orig_img = cv::imread(input_path, 1); 243 | if (!orig_img.data) 244 | { 245 | printf("cv::imread %s fail!\n", input_path); 246 | return -1; 247 | } 248 | cv::Mat img; 249 | cv::cvtColor(orig_img, img, cv::COLOR_BGR2RGB); 250 | img_width = img.cols; 251 | img_height = img.rows; 252 | printf("img width = %d, img height = %d\n", img_width, img_height); 253 | 254 | // 指定目标大小和预处理方式,默认使用LetterBox的预处理 255 | BOX_RECT pads; 256 | memset(&pads, 0, sizeof(BOX_RECT)); 257 | cv::Size target_size(width, height); 258 | cv::Mat resized_img(target_size.height, target_size.width, CV_8UC3); 259 | // 计算缩放比例 260 | float scale_w = (float)target_size.width / img.cols; 261 | float scale_h = (float)target_size.height / img.rows; 262 | 263 | if (img_width != width || img_height != height) 264 | { 265 | // 直接缩放采用RGA加速 266 | if (option == "resize") 267 | { 268 | printf("resize image by rga\n"); 269 | ret = resize_rga(src, dst, img, resized_img, target_size); 270 | if (ret != 0) 271 | { 272 | fprintf(stderr, "resize with rga error\n"); 273 | return -1; 274 | } 275 | // 保存预处理图片 276 | cv::imwrite("resize_input.jpg", resized_img); 277 | } 278 | else if (option == "letterbox") 279 | { 280 | printf("resize image with letterbox\n"); 281 | float min_scale = std::min(scale_w, scale_h); 282 | scale_w = min_scale; 283 | scale_h = min_scale; 284 | letterbox(img, resized_img, pads, min_scale, target_size); 285 | // 保存预处理图片 286 | cv::imwrite("letterbox_input.jpg", resized_img); 287 | } 288 | else 289 | { 290 | fprintf(stderr, "Invalid resize option. Use 'resize' or 'letterbox'.\n"); 291 | return -1; 292 | } 293 | inputs[0].buf = resized_img.data; 294 | } 295 | else 296 | { 297 | inputs[0].buf = img.data; 298 | } 299 | 300 | gettimeofday(&start_time, NULL); 301 | rknn_inputs_set(ctx, io_num.n_input, inputs); 302 | 303 | rknn_output outputs[io_num.n_output]; 304 | memset(outputs, 0, sizeof(outputs)); 305 | for (int i = 0; i < io_num.n_output; i++) 306 | { 307 | outputs[i].index = i; 308 | outputs[i].want_float = 0; 309 | } 310 | 311 | // 执行推理 312 | ret = rknn_run(ctx, NULL); 313 | ret = rknn_outputs_get(ctx, io_num.n_output, outputs, NULL); 314 | gettimeofday(&stop_time, NULL); 315 | printf("once run use %f ms\n", (__get_us(stop_time) - __get_us(start_time)) / 1000); 316 | 317 | // 后处理 318 | detect_result_group_t detect_result_group; 319 | std::vector out_scales; 320 | std::vector out_zps; 321 | for (int i = 0; i < io_num.n_output; ++i) 322 | { 323 | out_scales.push_back(output_attrs[i].scale); 324 | out_zps.push_back(output_attrs[i].zp); 325 | } 326 | post_process((int8_t *)outputs[0].buf, (int8_t *)outputs[1].buf, (int8_t *)outputs[2].buf, height, width, 327 | box_conf_threshold, nms_threshold, pads, scale_w, scale_h, out_zps, out_scales, &detect_result_group); 328 | 329 | // 画框和概率 330 | char text[256]; 331 | for (int i = 0; i < detect_result_group.count; i++) 332 | { 333 | detect_result_t *det_result = &(detect_result_group.results[i]); 334 | sprintf(text, "%s %.1f%%", det_result->name, det_result->prop * 100); 335 | printf("%s @ (%d %d %d %d) %f\n", det_result->name, det_result->box.left, det_result->box.top, 336 | det_result->box.right, det_result->box.bottom, det_result->prop); 337 | int x1 = det_result->box.left; 338 | int y1 = det_result->box.top; 339 | int x2 = det_result->box.right; 340 | int y2 = det_result->box.bottom; 341 | rectangle(orig_img, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(256, 128, 0, 128), 3); 342 | putText(orig_img, text, cv::Point(x1, y1), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(255, 255, 255), 2); 343 | 344 | // 过滤车辆的landmark点 345 | if (det_result->class_id==2){ 346 | continue; 347 | } 348 | 349 | // 定义四个不同的颜色 350 | cv::Scalar colors[4] = { 351 | cv::Scalar(0, 255, 0, 255), // 绿色 352 | cv::Scalar(255, 0, 0, 255), // 红色 353 | cv::Scalar(0, 0, 255, 255), // 蓝色 354 | cv::Scalar(255, 255, 0, 255) // 黄色 355 | }; 356 | 357 | for(int i =0; i < 4 ; i ++) 358 | { 359 | int x = int(det_result->landmarks[i*2]); 360 | int y = int(det_result->landmarks[i*2+1]); 361 | circle(orig_img, cv::Point(x, y), 2, colors[i], 2); 362 | } 363 | } 364 | printf("save detect result to %s\n", out_path.c_str()); 365 | imwrite(out_path, orig_img); 366 | ret = rknn_outputs_release(ctx, io_num.n_output, outputs); 367 | 368 | // 耗时统计 369 | int test_count = 10; 370 | gettimeofday(&start_time, NULL); 371 | for (int i = 0; i < test_count; ++i) 372 | { 373 | rknn_inputs_set(ctx, io_num.n_input, inputs); 374 | ret = rknn_run(ctx, NULL); 375 | ret = rknn_outputs_get(ctx, io_num.n_output, outputs, NULL); 376 | #if PERF_WITH_POST 377 | post_process((int8_t *)outputs[0].buf, (int8_t *)outputs[1].buf, (int8_t *)outputs[2].buf, height, width, 378 | box_conf_threshold, nms_threshold, pads, scale_w, scale_h, out_zps, out_scales, &detect_result_group); 379 | #endif 380 | ret = rknn_outputs_release(ctx, io_num.n_output, outputs); 381 | } 382 | gettimeofday(&stop_time, NULL); 383 | printf("loop count = %d , average run %f ms\n", test_count, 384 | (__get_us(stop_time) - __get_us(start_time)) / 1000.0 / test_count); 385 | 386 | deinitPostProcess(); 387 | 388 | // release 389 | ret = rknn_destroy(ctx); 390 | 391 | if (model_data) 392 | { 393 | free(model_data); 394 | } 395 | 396 | return 0; 397 | } 398 | -------------------------------------------------------------------------------- /cpp/src/postprocess.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 by Rockchip Electronics Co., Ltd. All Rights Reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | #include "postprocess.h" 16 | 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | 28 | #define LABEL_NALE_TXT_PATH "../../weights/plate.txt" 29 | const int anchor0[6] = {4,5, 8,10, 13,16}; 30 | const int anchor1[6] = {23,29, 43,55, 73,105}; 31 | const int anchor2[6] = {146,217, 231,300, 335,433}; 32 | 33 | static char *labels[OBJ_CLASS_NUM]; 34 | 35 | inline static int clamp(float val, int min, int max) { return val > min ? (val < max ? val : max) : min; } 36 | 37 | char *readLine(FILE *fp, char *buffer, int *len) 38 | { 39 | int ch; 40 | int i = 0; 41 | size_t buff_len = 0; 42 | 43 | buffer = (char *)malloc(buff_len + 1); 44 | if (!buffer) 45 | return NULL; // Out of memory 46 | 47 | while ((ch = fgetc(fp)) != '\n' && ch != EOF) 48 | { 49 | buff_len++; 50 | void *tmp = realloc(buffer, buff_len + 1); 51 | if (tmp == NULL) 52 | { 53 | free(buffer); 54 | return NULL; // Out of memory 55 | } 56 | buffer = (char *)tmp; 57 | 58 | buffer[i] = (char)ch; 59 | i++; 60 | } 61 | buffer[i] = '\0'; 62 | 63 | *len = buff_len; 64 | 65 | // Detect end 66 | if (ch == EOF && (i == 0 || ferror(fp))) 67 | { 68 | free(buffer); 69 | return NULL; 70 | } 71 | return buffer; 72 | } 73 | 74 | int readLines(const char *fileName, char *lines[], int max_line) 75 | { 76 | FILE *file = fopen(fileName, "r"); 77 | char *s; 78 | int i = 0; 79 | int n = 0; 80 | 81 | if (file == NULL) 82 | { 83 | printf("Open %s fail!\n", fileName); 84 | return -1; 85 | } 86 | 87 | while ((s = readLine(file, s, &n)) != NULL) 88 | { 89 | lines[i++] = s; 90 | if (i >= max_line) 91 | break; 92 | } 93 | fclose(file); 94 | return i; 95 | } 96 | 97 | int loadLabelName(const char *locationFilename, char *label[]) 98 | { 99 | printf("loadLabelName %s\n", locationFilename); 100 | readLines(locationFilename, label, OBJ_CLASS_NUM); 101 | return 0; 102 | } 103 | 104 | static float CalculateOverlap(float xmin0, float ymin0, float xmax0, float ymax0, float xmin1, float ymin1, float xmax1, 105 | float ymax1) 106 | { 107 | float w = fmax(0.f, fmin(xmax0, xmax1) - fmax(xmin0, xmin1) + 1.0); 108 | float h = fmax(0.f, fmin(ymax0, ymax1) - fmax(ymin0, ymin1) + 1.0); 109 | float i = w * h; 110 | float u = (xmax0 - xmin0 + 1.0) * (ymax0 - ymin0 + 1.0) + (xmax1 - xmin1 + 1.0) * (ymax1 - ymin1 + 1.0) - i; 111 | return u <= 0.f ? 0.f : (i / u); 112 | } 113 | 114 | static int nms(int validCount, std::vector &outputLocations, std::vector classIds, std::vector &order, 115 | int filterId, float threshold) 116 | { 117 | for (int i = 0; i < validCount; ++i) 118 | { 119 | if (order[i] == -1 || classIds[i] != filterId) 120 | { 121 | continue; 122 | } 123 | int n = order[i]; 124 | for (int j = i + 1; j < validCount; ++j) 125 | { 126 | int m = order[j]; 127 | if (m == -1 || classIds[i] != filterId) 128 | { 129 | continue; 130 | } 131 | float xmin0 = outputLocations[n * 4 + 0]; 132 | float ymin0 = outputLocations[n * 4 + 1]; 133 | float xmax0 = outputLocations[n * 4 + 0] + outputLocations[n * 4 + 2]; 134 | float ymax0 = outputLocations[n * 4 + 1] + outputLocations[n * 4 + 3]; 135 | 136 | float xmin1 = outputLocations[m * 4 + 0]; 137 | float ymin1 = outputLocations[m * 4 + 1]; 138 | float xmax1 = outputLocations[m * 4 + 0] + outputLocations[m * 4 + 2]; 139 | float ymax1 = outputLocations[m * 4 + 1] + outputLocations[m * 4 + 3]; 140 | 141 | float iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1); 142 | 143 | if (iou > threshold) 144 | { 145 | order[j] = -1; 146 | } 147 | } 148 | } 149 | return 0; 150 | } 151 | 152 | static int quick_sort_indice_inverse(std::vector &input, int left, int right, std::vector &indices) 153 | { 154 | float key; 155 | int key_index; 156 | int low = left; 157 | int high = right; 158 | if (left < right) 159 | { 160 | key_index = indices[left]; 161 | key = input[left]; 162 | while (low < high) 163 | { 164 | while (low < high && input[high] <= key) 165 | { 166 | high--; 167 | } 168 | input[low] = input[high]; 169 | indices[low] = indices[high]; 170 | while (low < high && input[low] >= key) 171 | { 172 | low++; 173 | } 174 | input[high] = input[low]; 175 | indices[high] = indices[low]; 176 | } 177 | input[low] = key; 178 | indices[low] = key_index; 179 | quick_sort_indice_inverse(input, left, low - 1, indices); 180 | quick_sort_indice_inverse(input, low + 1, right, indices); 181 | } 182 | return low; 183 | } 184 | 185 | static float sigmoid(float x) { return 1.0 / (1.0 + expf(-x)); } 186 | 187 | static float unsigmoid(float y) { return -1.0 * logf((1.0 / y) - 1.0); } 188 | 189 | inline static int32_t __clip(float val, float min, float max) 190 | { 191 | float f = val <= min ? min : (val >= max ? max : val); 192 | return f; 193 | } 194 | 195 | static int8_t qnt_f32_to_affine(float f32, int32_t zp, float scale) 196 | { 197 | float dst_val = (f32 / scale) + zp; 198 | int8_t res = (int8_t)__clip(dst_val, -128, 127); 199 | return res; 200 | } 201 | 202 | static float deqnt_affine_to_f32(int8_t qnt, int32_t zp, float scale) { return ((float)qnt - (float)zp) * scale; } 203 | 204 | static int process(int8_t *input, int *anchor, int grid_h, int grid_w, int height, int width, int stride, 205 | std::vector &boxes, std::vector &landmarks, std::vector &objProbs, std::vector &classId, float threshold, 206 | int32_t zp, float scale) 207 | { 208 | int validCount = 0; 209 | int grid_len = grid_h * grid_w; 210 | int8_t thres_i8 = qnt_f32_to_affine(threshold, zp, scale); 211 | float landmark_x[4]; 212 | float landmark_y[4]; 213 | /* 214 | [1, 3, 80, 80, 16] 215 | [1, 3, 40, 40, 16] 216 | [1, 3, 20, 20, 16] 217 | */ 218 | /* 16 结构 219 | x y w h conf, pt1x pt1y pt2x pt2y pt3x pt3y pt4x pt4y, cls0, cls1, cls2 220 | 0 3 4 5 13 14 15 221 | */ 222 | for (int a = 0; a < 3; a++) 223 | { 224 | for (int i = 0; i < grid_h; i++) 225 | { 226 | for (int j = 0; j < grid_w; j++) 227 | { 228 | int8_t box_confidence = input[PROP_BOX_SIZE *(a * grid_len + i * grid_w + j ) + 4]; 229 | if (box_confidence >= thres_i8) 230 | { 231 | int offset = PROP_BOX_SIZE * (a * grid_len + i * grid_w + j ); 232 | int8_t *in_ptr = input + offset; 233 | float box_x = sigmoid(deqnt_affine_to_f32(*in_ptr, zp, scale)) * 2.0 - 0.5; 234 | float box_y = sigmoid(deqnt_affine_to_f32(in_ptr[1], zp, scale)) * 2.0 - 0.5; 235 | float box_w = sigmoid(deqnt_affine_to_f32(in_ptr[2], zp, scale)) * 2.0; 236 | float box_h = sigmoid(deqnt_affine_to_f32(in_ptr[3], zp, scale)) * 2.0; 237 | 238 | box_x = (box_x + j) * (float)stride; 239 | box_y = (box_y + i) * (float)stride; 240 | box_w = box_w * box_w * (float)anchor[a * 2]; 241 | box_h = box_h * box_h * (float)anchor[a * 2 + 1]; 242 | box_x -= (box_w / 2.0); 243 | box_y -= (box_h / 2.0); 244 | 245 | //calculate landmarks 246 | for (int z = 0; z < 4; z ++){ 247 | landmark_x[z] = deqnt_affine_to_f32(in_ptr[5 + z *2], zp, scale) ; 248 | landmark_y[z] = deqnt_affine_to_f32(in_ptr[6 + z *2], zp, scale) ; 249 | landmark_x[z] = (landmark_x[z])* (float)anchor[a * 2] + (float)j * (float)stride; 250 | landmark_y[z] = (landmark_y[z])* (float)anchor[a * 2 + 1] + (float)i * (float)stride; 251 | } 252 | 253 | int8_t maxClassProbs = in_ptr[13]; //cls0的位置 254 | int maxClassId = 0; 255 | for (int k = 1; k < OBJ_CLASS_NUM; ++k) 256 | { 257 | int8_t prob = in_ptr[13 + k]; 258 | if (prob > maxClassProbs) 259 | { 260 | maxClassId = k; 261 | maxClassProbs = prob; 262 | } 263 | } 264 | if (maxClassProbs > thres_i8) 265 | { 266 | objProbs.push_back(sigmoid((deqnt_affine_to_f32(maxClassProbs, zp, scale))) * sigmoid((deqnt_affine_to_f32(box_confidence, zp, scale)))); 267 | 268 | classId.push_back(maxClassId); 269 | validCount++; 270 | boxes.push_back(box_x); 271 | boxes.push_back(box_y); 272 | boxes.push_back(box_w); 273 | boxes.push_back(box_h); 274 | 275 | for (int m = 0; m < 4; m++){ 276 | landmarks.push_back(landmark_x[m]); 277 | landmarks.push_back(landmark_y[m]); 278 | } 279 | } 280 | } 281 | } 282 | } 283 | } 284 | return validCount; 285 | } 286 | 287 | int post_process(int8_t *input0, int8_t *input1, int8_t *input2, int model_in_h, int model_in_w, float conf_threshold, 288 | float nms_threshold, BOX_RECT pads, float scale_w, float scale_h, std::vector &qnt_zps, 289 | std::vector &qnt_scales, detect_result_group_t *group) 290 | { 291 | static int init = -1; 292 | if (init == -1) 293 | { 294 | int ret = 0; 295 | ret = loadLabelName(LABEL_NALE_TXT_PATH, labels); 296 | if (ret < 0) 297 | { 298 | return -1; 299 | } 300 | 301 | init = 0; 302 | } 303 | memset(group, 0, sizeof(detect_result_group_t)); 304 | 305 | std::vector filterBoxes; 306 | std::vector landmarks; 307 | std::vector objProbs; 308 | std::vector classId; 309 | 310 | // stride 8 311 | int stride0 = 8; 312 | int grid_h0 = model_in_h / stride0; 313 | int grid_w0 = model_in_w / stride0; 314 | int validCount0 = 0; 315 | validCount0 = process(input0, (int *)anchor0, grid_h0, grid_w0, model_in_h, model_in_w, stride0, filterBoxes, landmarks, objProbs, 316 | classId, conf_threshold, qnt_zps[0], qnt_scales[0]); 317 | 318 | // stride 16 319 | int stride1 = 16; 320 | int grid_h1 = model_in_h / stride1; 321 | int grid_w1 = model_in_w / stride1; 322 | int validCount1 = 0; 323 | validCount1 = process(input1, (int *)anchor1, grid_h1, grid_w1, model_in_h, model_in_w, stride1, filterBoxes, landmarks, objProbs, 324 | classId, conf_threshold, qnt_zps[1], qnt_scales[1]); 325 | 326 | // stride 32 327 | int stride2 = 32; 328 | int grid_h2 = model_in_h / stride2; 329 | int grid_w2 = model_in_w / stride2; 330 | int validCount2 = 0; 331 | validCount2 = process(input2, (int *)anchor2, grid_h2, grid_w2, model_in_h, model_in_w, stride2, filterBoxes, landmarks, objProbs, 332 | classId, conf_threshold, qnt_zps[2], qnt_scales[2]); 333 | 334 | int validCount = validCount0 + validCount1 + validCount2; 335 | 336 | // no object detect 337 | if (validCount <= 0) 338 | { 339 | return 0; 340 | } 341 | 342 | std::vector indexArray; 343 | for (int i = 0; i < validCount; ++i) 344 | { 345 | indexArray.push_back(i); 346 | } 347 | 348 | quick_sort_indice_inverse(objProbs, 0, validCount - 1, indexArray); 349 | 350 | std::set class_set(std::begin(classId), std::end(classId)); 351 | 352 | for (auto c : class_set) 353 | { 354 | nms(validCount, filterBoxes, classId, indexArray, c, nms_threshold); 355 | } 356 | 357 | int last_count = 0; 358 | group->count = 0; 359 | /* box valid detect target */ 360 | for (int i = 0; i < validCount; ++i) 361 | { 362 | if (indexArray[i] == -1 || last_count >= OBJ_NUMB_MAX_SIZE) 363 | { 364 | continue; 365 | } 366 | int n = indexArray[i]; 367 | 368 | float x1 = filterBoxes[n * 4 + 0] - pads.left; 369 | float y1 = filterBoxes[n * 4 + 1] - pads.top; 370 | float x2 = x1 + filterBoxes[n * 4 + 2]; 371 | float y2 = y1 + filterBoxes[n * 4 + 3]; 372 | int id = classId[n]; 373 | float obj_conf = objProbs[i]; 374 | 375 | group->results[last_count].box.left = (int)(clamp(x1, 0, model_in_w) / scale_w); 376 | group->results[last_count].box.top = (int)(clamp(y1, 0, model_in_h) / scale_h); 377 | group->results[last_count].box.right = (int)(clamp(x2, 0, model_in_w) / scale_w); 378 | group->results[last_count].box.bottom = (int)(clamp(y2, 0, model_in_h) / scale_h); 379 | group->results[last_count].prop = obj_conf; 380 | group->results[last_count].class_id = id; 381 | 382 | for(int i =0; i < 4; i ++){ 383 | float x = landmarks[n * 8 + i * 2] - pads.left; 384 | float y = landmarks[n * 8 + i * 2 + 1] - pads.top; 385 | group->results[last_count].landmarks[i * 2] = (int) (clamp(x, 0, model_in_w) / scale_w) ; 386 | group->results[last_count].landmarks[i * 2 + 1] = (int) (clamp(y, 0, model_in_h) / scale_h) ; 387 | } 388 | 389 | char *label = labels[id]; 390 | strncpy(group->results[last_count].name, label, OBJ_NAME_MAX_SIZE); 391 | 392 | //printf("result %2d: (%4d, %4d, %4d, %4d), %s\n", i, group->results[last_count].box.left, 393 | //group->results[last_count].box.top, 394 | // group->results[last_count].box.right, group->results[last_count].box.bottom, label); 395 | last_count++; 396 | } 397 | group->count = last_count; 398 | 399 | return 0; 400 | } 401 | 402 | void deinitPostProcess() 403 | { 404 | for (int i = 0; i < OBJ_CLASS_NUM; i++) 405 | { 406 | if (labels[i] != nullptr) 407 | { 408 | free(labels[i]); 409 | labels[i] = nullptr; 410 | } 411 | } 412 | } 413 | -------------------------------------------------------------------------------- /cpp/src/preprocess.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2023 by Rockchip Electronics Co., Ltd. All Rights Reserved. 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // http://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | #include "preprocess.h" 16 | 17 | void letterbox(const cv::Mat &image, cv::Mat &padded_image, BOX_RECT &pads, const float scale, const cv::Size &target_size, const cv::Scalar &pad_color) 18 | { 19 | // 调整图像大小 20 | cv::Mat resized_image; 21 | cv::resize(image, resized_image, cv::Size(), scale, scale); 22 | 23 | // 计算填充大小 24 | int pad_width = target_size.width - resized_image.cols; 25 | int pad_height = target_size.height - resized_image.rows; 26 | 27 | pads.left = pad_width / 2; 28 | pads.right = pad_width - pads.left; 29 | pads.top = pad_height / 2; 30 | pads.bottom = pad_height - pads.top; 31 | 32 | // 在图像周围添加填充 33 | cv::copyMakeBorder(resized_image, padded_image, pads.top, pads.bottom, pads.left, pads.right, cv::BORDER_CONSTANT, pad_color); 34 | } 35 | 36 | int resize_rga(rga_buffer_t &src, rga_buffer_t &dst, const cv::Mat &image, cv::Mat &resized_image, const cv::Size &target_size) 37 | { 38 | im_rect src_rect; 39 | im_rect dst_rect; 40 | memset(&src_rect, 0, sizeof(src_rect)); 41 | memset(&dst_rect, 0, sizeof(dst_rect)); 42 | size_t img_width = image.cols; 43 | size_t img_height = image.rows; 44 | if (image.type() != CV_8UC3) 45 | { 46 | printf("source image type is %d!\n", image.type()); 47 | return -1; 48 | } 49 | size_t target_width = target_size.width; 50 | size_t target_height = target_size.height; 51 | src = wrapbuffer_virtualaddr((void *)image.data, img_width, img_height, RK_FORMAT_RGB_888); 52 | dst = wrapbuffer_virtualaddr((void *)resized_image.data, target_width, target_height, RK_FORMAT_RGB_888); 53 | int ret = imcheck(src, dst, src_rect, dst_rect); 54 | if (IM_STATUS_NOERROR != ret) 55 | { 56 | fprintf(stderr, "rga check error! %s", imStrError((IM_STATUS)ret)); 57 | return -1; 58 | } 59 | IM_STATUS STATUS = imresize(src, dst); 60 | return 0; 61 | } 62 | -------------------------------------------------------------------------------- /font/NotoSansCJK-Regular.otf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/font/NotoSansCJK-Regular.otf -------------------------------------------------------------------------------- /imgs/double_yellow.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/double_yellow.jpg -------------------------------------------------------------------------------- /imgs/out.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/out.jpg -------------------------------------------------------------------------------- /imgs/result.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/result.jpg -------------------------------------------------------------------------------- /imgs/test.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/test.jpg -------------------------------------------------------------------------------- /imgs/京A25016_32.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/京A25016_32.jpg -------------------------------------------------------------------------------- /imgs/沪A5L320.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/imgs/沪A5L320.jpg -------------------------------------------------------------------------------- /python/plate_detection_rec_rknnlite.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | import argparse 4 | import platform 5 | from utils import COLORS, PLATE_CLASSES 6 | from plate_rec_rknnlite import PLATE_REC 7 | from rknnlite.api import RKNNLite 8 | from postprocess_numpy import PostProcess 9 | from PIL import Image, ImageDraw, ImageFont 10 | 11 | class YOLOv5: 12 | def __init__(self, args): 13 | # load model 14 | self.rknn_lite = RKNNLite() 15 | ret = self.rknn_lite.load_rknn(args.det_model) 16 | if ret != 0: 17 | print('load rknnlite model failed!') 18 | exit(ret) 19 | 20 | self.rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) 21 | self.batch_size = 1 22 | self.net_h = 640 23 | self.net_w = 640 24 | self.agnostic = True 25 | self.multi_label = True 26 | self.conf_thresh = args.conf_thresh 27 | self.nms_thresh = args.nms_thresh 28 | self.max_det = 1000 29 | 30 | self.postprocess = PostProcess( 31 | conf_thresh=self.conf_thresh, 32 | nms_thresh=self.nms_thresh, 33 | agnostic=self.agnostic, 34 | multi_label=self.multi_label, 35 | max_det=self.max_det, 36 | ) 37 | 38 | def preprocess(self, ori_img): 39 | """ 40 | pre-processing 41 | Args: 42 | img: numpy.ndarray -- (h,w,3) 43 | 44 | Returns: (3,h,w) numpy.ndarray after pre-processing 45 | 46 | """ 47 | letterbox_img, ratio, (tx1, ty1) = self.letterbox( 48 | ori_img, 49 | new_shape=(self.net_h, self.net_w), 50 | color=(114, 114, 114), 51 | auto=False, 52 | scaleFill=False, 53 | scaleup=True, 54 | stride=32 55 | ) 56 | img = letterbox_img.transpose((2, 0, 1))[::-1] # HWC to CHW, BGR to RGB 57 | return img, ratio, (tx1, ty1) 58 | 59 | def letterbox(self, im, new_shape=(640, 640), color=(114, 114, 114), auto=False, scaleFill=False, scaleup=True, stride=32): 60 | # Resize and pad image while meeting stride-multiple constraints 61 | shape = im.shape[:2] # current shape [height, width] 62 | if isinstance(new_shape, int): 63 | new_shape = (new_shape, new_shape) 64 | 65 | # Scale ratio (new / old) 66 | r = min(new_shape[0] / shape[0], new_shape[1] / shape[1]) 67 | if not scaleup: # only scale down, do not scale up (for better val mAP) 68 | r = min(r, 1.0) 69 | 70 | # Compute padding 71 | ratio = r, r # width, height ratios 72 | new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r)) 73 | dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1] # wh padding 74 | if auto: # minimum rectangle 75 | dw, dh = np.mod(dw, stride), np.mod(dh, stride) # wh padding 76 | elif scaleFill: # stretch 77 | dw, dh = 0.0, 0.0 78 | new_unpad = (new_shape[1], new_shape[0]) 79 | ratio = new_shape[1] / shape[1], new_shape[0] / shape[0] # width, height ratios 80 | 81 | dw /= 2 # divide padding into 2 sides 82 | dh /= 2 83 | 84 | if shape[::-1] != new_unpad: # resize 85 | im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) 86 | top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1)) 87 | left, right = int(round(dw - 0.1)), int(round(dw + 0.1)) 88 | im = cv2.copyMakeBorder(im, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color) # add border 89 | return im, ratio, (dw, dh) 90 | 91 | def predict(self, input_img): 92 | return self.rknn_lite.inference(inputs=[input_img], data_format=['nchw']) 93 | 94 | def __call__(self, img_list): 95 | img_num = len(img_list) 96 | ori_size_list = [] 97 | preprocessed_img_list = [] 98 | ratio_list = [] 99 | txy_list = [] 100 | for ori_img in img_list: 101 | ori_h, ori_w = ori_img.shape[:2] 102 | ori_size_list.append((ori_w, ori_h)) 103 | preprocessed_img, ratio, (tx1, ty1) = self.preprocess(ori_img) 104 | preprocessed_img_list.append(preprocessed_img) 105 | ratio_list.append(ratio) 106 | txy_list.append([tx1, ty1]) 107 | 108 | if img_num == self.batch_size: 109 | input_img = np.stack(preprocessed_img_list) 110 | else: 111 | input_img = np.zeros(self.input_shape, dtype='float32') 112 | input_img[:img_num] = np.stack(preprocessed_img_list) 113 | 114 | outputs = self.predict(input_img) 115 | results = self.postprocess(outputs, ori_size_list, ratio_list, txy_list) 116 | det = results[0] 117 | 118 | return det 119 | 120 | def extract_and_align_plate(img, det): 121 | # 提取车牌的4个顶点坐标 122 | landmarks = det[5:13].reshape(-1, 2).astype(np.int32).tolist() 123 | pts = np.array(landmarks, dtype=np.float32) 124 | 125 | # 计算车牌的宽高 126 | width = int(np.linalg.norm(pts[1] - pts[0])) # 计算左上和右上点之间的距离作为宽度 127 | height = int(np.linalg.norm(pts[2] - pts[1])) # 计算右上和右下点之间的距离作为高度 128 | dst_pts = np.array([ 129 | [0, 0], 130 | [width - 1, 0], 131 | [width - 1, height - 1], 132 | [0, height - 1] 133 | ], dtype=np.float32) 134 | 135 | # 计算透视变换矩阵 136 | M = cv2.getPerspectiveTransform(pts, dst_pts) 137 | 138 | # 应用透视变换 139 | aligned_plate = cv2.warpPerspective(img, M, (width, height)) 140 | 141 | return aligned_plate 142 | 143 | def process_dual_layer_plate(img, det): 144 | aligned_plate = extract_and_align_plate(img, det) 145 | 146 | height, width = aligned_plate.shape[:2] 147 | top_part = aligned_plate[0:height *5//12, :] 148 | bottom_part = aligned_plate[height * 1//3:, :] 149 | top_resized = cv2.resize(top_part, (width, height//2)) 150 | bottom_resized = cv2.resize(bottom_part, (width, height//2)) 151 | aligned_plate = np.hstack((top_resized, bottom_resized)) 152 | return aligned_plate 153 | 154 | def draw_results(img, det, plate_rec): 155 | x1, y1, x2, y2 = det[:4].astype(np.int32).tolist() 156 | cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 3) 157 | landmarks = det[5:13].reshape(-1, 2).astype(np.int32) 158 | for (x, y) in landmarks: 159 | cv2.circle(img, (x, y), 2, (0, 0, 255), -1) 160 | 161 | font = ImageFont.truetype("../font/NotoSansCJK-Regular.otf", 20, encoding="utf-8") 162 | img_pil = Image.fromarray(img) 163 | draw = ImageDraw.Draw(img_pil) 164 | draw.text((x1,y1 -30), plate_rec, font = font, fill = (0,0,255)) 165 | 166 | return np.array(img_pil) 167 | 168 | def argsparser(): 169 | parser = argparse.ArgumentParser(prog=__file__) 170 | parser.add_argument('--input', type=str, default='../imgs/京A25016_32.jpg', help='path of input') 171 | parser.add_argument('--det_model', type=str, default='../weights/car_plate_detect_1x3x80x80x16-640-640_rm_transpose_rk3588.rknn', help='path of model') 172 | parser.add_argument('--rec_model', type=str, default='../weights/plate_recognition_color-168-48_rm_transpose_rk3588.rknn', help='path of model') 173 | parser.add_argument('--conf_thresh', type=float, default=0.5, help='confidence threshold') 174 | parser.add_argument('--nms_thresh', type=float, default=0.5, help='nms threshold') 175 | args = parser.parse_args() 176 | return args 177 | 178 | if __name__ == '__main__': 179 | args = argsparser() 180 | 181 | ## 初始化 182 | plate_det = YOLOv5(args) 183 | plate_rec = PLATE_REC(args) 184 | 185 | ## 读取图片 186 | img = cv2.imread(args.input) 187 | drawimage = img.copy() 188 | # 车牌检测 189 | det_results = plate_det([img]) 190 | 191 | for det in det_results: 192 | x1, y1, x2, y2 = det[:4].astype(np.int32).tolist() 193 | class_id = int(det[13]) 194 | if class_id == 2: 195 | continue 196 | 197 | # 提取车牌 198 | if class_id == 0: 199 | ## 处理单层车牌 200 | aligned_plate = extract_and_align_plate(img, det) 201 | else: 202 | # 处理双层车牌 203 | aligned_plate = process_dual_layer_plate(img, det) 204 | 205 | # 车牌识别 206 | pred_numer, pred_color = plate_rec([aligned_plate]) 207 | result = str(pred_numer + ", " + pred_color) 208 | print(result) 209 | # 车牌识别 210 | drawimage = draw_results(drawimage, det, result) 211 | 212 | cv2.imwrite("result.jpg", drawimage) 213 | # 保存识别结果 214 | print("save result to result.jpg") -------------------------------------------------------------------------------- /python/plate_rec_rknnlite.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | import argparse 4 | import platform 5 | from utils import COLORS, PLATE_CLASSES 6 | from rknnlite.api import RKNNLite 7 | 8 | class PLATE_REC: 9 | def __init__(self, args): 10 | # load model 11 | self.rknn_lite = RKNNLite() 12 | ret = self.rknn_lite.load_rknn(args.rec_model) 13 | if ret != 0: 14 | print('load rknnlite model failed!') 15 | exit(ret) 16 | 17 | self.rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) 18 | self.palteStr=r"#京沪津渝冀晋蒙辽吉黑苏浙皖闽赣鲁豫鄂湘粤桂琼川贵云藏陕甘青宁新学警港澳挂使领民深危0123456789ABCDEFGHJKLMNPQRSTUVWXYZ险品" 19 | self.color=['黑色','蓝色','绿色','白色','黄色'] 20 | self.batch_size = 1 21 | self.net_h = 48 22 | self.net_w = 168 23 | 24 | def preprocess(self, ori_img): 25 | """ 26 | pre-processing 27 | Args: 28 | img: numpy.ndarray -- (h,w,3) 29 | 30 | Returns: (3,h,w) numpy.ndarray after pre-processing 31 | 32 | """ 33 | im = cv2.resize(ori_img, (self.net_w, self.net_h), interpolation=cv2.INTER_LINEAR) 34 | img = im.transpose((2, 0, 1)) # HWC to CHW 35 | return img 36 | 37 | def decodePlate(self, preds): 38 | # 将 preds 转换为 Python 列表 39 | preds = preds.flatten().tolist() 40 | 41 | # 使用列表推导式和 zip 函数来过滤重复和零值 42 | newPreds = [preds[i] for i in range(len(preds)) if preds[i] != 0 and (i == 0 or preds[i] != preds[i-1])] 43 | 44 | # 使用 ''.join() 来拼接字符串 45 | plate = ''.join([self.palteStr[i] for i in newPreds]) 46 | 47 | return plate 48 | 49 | def predict(self, input_img): 50 | return self.rknn_lite.inference(inputs=[input_img], data_format=['nchw']) 51 | 52 | def __call__(self, img_list): 53 | img_num = len(img_list) 54 | ori_size_list = [] 55 | preprocessed_img_list = [] 56 | for ori_img in img_list: 57 | ori_h, ori_w = ori_img.shape[:2] 58 | ori_size_list.append((ori_w, ori_h)) 59 | preprocessed_img = self.preprocess(ori_img) 60 | preprocessed_img_list.append(preprocessed_img) 61 | 62 | if img_num == self.batch_size: 63 | input_img = np.stack(preprocessed_img_list) 64 | else: 65 | input_img = np.zeros(self.input_shape, dtype='float32') 66 | input_img[:img_num] = np.stack(preprocessed_img_list) 67 | 68 | preds, preds_color = self.predict(input_img) 69 | 70 | max_values = np.argmax(preds, axis=-1) 71 | preds_number = self.decodePlate(max_values) 72 | preds_color=preds_color.argmax() 73 | color=self.color[preds_color.item()] 74 | 75 | return preds_number, color 76 | 77 | def argsparser(): 78 | parser = argparse.ArgumentParser(prog=__file__) 79 | parser.add_argument('--input', type=str, default='../imgs/test.jpg', help='path of input') 80 | parser.add_argument('--rec_model', type=str, default='../weights/plate_recognition_color-168-48_rm_transpose_rk3588.rknn', help='path of model') 81 | args = parser.parse_args() 82 | return args 83 | 84 | if __name__ == '__main__': 85 | args = argsparser() 86 | plate_rec_result =PLATE_REC(args) 87 | preds_number, color = plate_rec_result([cv2.imread(args.input)]) 88 | print(preds_number, color, args.input ) 89 | -------------------------------------------------------------------------------- /python/postprocess_numpy.py: -------------------------------------------------------------------------------- 1 | #----------------------------------------------------------------------===# 2 | # 3 | # Copyright (C) 2022 Sophgo Technologies Inc. All rights reserved. 4 | # 5 | # SOPHON-DEMO is licensed under the 2-Clause BSD License except for the 6 | # third-party components. 7 | # 8 | #===----------------------------------------------------------------------===# 9 | import numpy as np 10 | import cv2 11 | # import scipy.special 12 | from utils import softmax 13 | 14 | class PostProcess: 15 | def __init__(self, conf_thresh=0.1, nms_thresh=0.5, agnostic=False, multi_label=False, max_det=1000): 16 | self.conf_thresh = conf_thresh 17 | self.nms_thresh = nms_thresh 18 | self.agnostic_nms = agnostic 19 | self.multi_label = multi_label 20 | self.max_det = max_det 21 | self.nms = pseudo_torch_nms() 22 | 23 | self.nl = 3 24 | anchors = [ [4,5, 8,10, 13,16] ,[23,29, 43,55, 73,105] ,[146,217, 231,300, 335,433] ] 25 | self.anchor_grid = np.asarray(anchors, dtype=np.float32).reshape(self.nl, 1, -1, 1, 1, 2) 26 | self.grid = [np.zeros(1)] * self.nl 27 | self.stride = np.array([8., 16., 32.]) 28 | 29 | @staticmethod 30 | def _make_grid(nx=20, ny=20): 31 | xv, yv = np.meshgrid(np.arange(nx), np.arange(ny)) 32 | return np.stack((xv, yv), 2).reshape((1, 1, ny, nx, 2)).astype(np.float32) 33 | 34 | def decode_for_3outputs(self, outputs): 35 | z = [] 36 | for i, feat in enumerate(outputs): 37 | bs, _, ny, nx, nc = feat.shape 38 | if self.grid[i].shape[2:4] != feat.shape[2:4]: 39 | self.grid[i] = self._make_grid(nx, ny) 40 | 41 | y = 1 / (1 + np.exp(-feat)) # sigmoid 42 | #x,y 43 | y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + 44 | self.grid[i]) * int(self.stride[i]) 45 | y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i] # wh 46 | # landmarks,(x,y) 47 | y[..., 5:7] = feat[..., 5:7] * self.anchor_grid[i] + self.grid[i] * int(self.stride[i]) # decode landmark without sigmod 48 | y[..., 7:9] = feat[..., 7:9] * self.anchor_grid[i] + self.grid[i] * int(self.stride[i]) 49 | y[..., 9:11] = feat[..., 9:11] * self.anchor_grid[i] + self.grid[i] * int(self.stride[i]) 50 | y[..., 11:13] = feat[..., 11:13] * self.anchor_grid[i] + self.grid[i] * int(self.stride[i]) 51 | 52 | z.append(y.reshape(bs, -1, nc)) 53 | 54 | z = np.concatenate(z, axis=1) 55 | return z 56 | 57 | 58 | def __call__(self, preds_batch, org_size_batch, ratios_batch, txy_batch): 59 | """ 60 | post-processing 61 | :param preds_batch: list of predictions in a batch 62 | :param org_size_batch: list of (org_img_w, org_img_h) in a batch 63 | :param ratios_batch: list of (ratio_x, ratio_y) in a batch when resize-and-center-padding 64 | :param txy_batch: list of (tx, ty) in a batch when resize-and-center-padding 65 | :return: 66 | """ 67 | if isinstance(preds_batch, list) and len(preds_batch) == 3: 68 | # 3 output 69 | dets = self.decode_for_3outputs(preds_batch) 70 | elif isinstance(preds_batch, list) and len(preds_batch) == 1: 71 | # 1 output 72 | dets = np.concatenate(preds_batch) 73 | else: 74 | print('preds_batch type: '.format(type(preds_batch))) 75 | raise NotImplementedError 76 | 77 | outs = self.nms.non_max_suppression( 78 | dets, 79 | conf_thres=self.conf_thresh, 80 | iou_thres=self.nms_thresh, 81 | classes=None, 82 | agnostic=self.agnostic_nms, 83 | multi_label=self.multi_label, 84 | labels=(), 85 | max_det=self.max_det, 86 | 87 | ) 88 | # Rescale boxes from img_size to im0 size 89 | for det, (org_w, org_h), ratio, (tx1, ty1) in zip(outs, org_size_batch, ratios_batch, txy_batch): 90 | if len(det): 91 | # Rescale boxes from img_size to im0 size 92 | coords = det[:, :4] 93 | coords[:, [0,2]] -= tx1 # x padding 94 | coords[:, [1,3]] -= ty1 # y padding 95 | coords[:, [0,2]] /= ratio[0] 96 | coords[:, [1,3]] /= ratio[1] 97 | 98 | coords[:, [0, 2]] = coords[:, [0, 2]].clip(0, org_w - 1) # x1, x2 99 | coords[:, [1, 3]] = coords[:, [1, 3]].clip(0, org_h - 1) # y1, y2 100 | det[:,:4] = coords.round() 101 | 102 | # Rescale landmarks from image size 103 | landmarks = det[:, 5:13] 104 | landmarks[:, [0, 2, 4, 6]] -= tx1 # x padding 105 | landmarks[:, [1, 3, 5, 7]] -= ty1 # y padding 106 | landmarks[:, [0, 2, 4, 6]] /= ratio[0] # x padding 107 | landmarks[:, [1, 3, 5, 7]] /= ratio[1] # y padding 108 | landmarks[:, [0, 2, 4, 6]] = landmarks[:, [0, 2, 4, 6]].clip(0, org_w -1) 109 | landmarks[:, [1, 3, 5, 7]] = landmarks[:, [1, 3, 5, 7]].clip(0, org_h - 1) 110 | det[:, 5:13] = landmarks.round() 111 | 112 | return outs 113 | 114 | 115 | # numpy multiclass nms implementation from original yolov5 repo torch implementation 116 | class pseudo_torch_nms: 117 | def nms_boxes(self, boxes, scores, iou_thres): 118 | x = boxes[:, 0] 119 | y = boxes[:, 1] 120 | w = boxes[:, 2] - boxes[:, 0] 121 | h = boxes[:, 3] - boxes[:, 1] 122 | 123 | areas = w * h 124 | order = scores.argsort()[::-1] 125 | 126 | keep = [] 127 | while order.size > 0: 128 | i = order[0] 129 | keep.append(i) 130 | 131 | xx1 = np.maximum(x[i], x[order[1:]]) 132 | yy1 = np.maximum(y[i], y[order[1:]]) 133 | xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]]) 134 | yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]]) 135 | 136 | w1 = np.maximum(0.0, xx2 - xx1 + 0.00001) 137 | h1 = np.maximum(0.0, yy2 - yy1 + 0.00001) 138 | inter = w1 * h1 139 | 140 | ovr = inter / (areas[i] + areas[order[1:]] - inter) 141 | inds = np.where(ovr <= iou_thres)[0] 142 | order = order[inds + 1] 143 | keep = np.array(keep) 144 | return keep 145 | 146 | def xywh2xyxy(self, x): 147 | # Convert nx4 boxes from [x, y, w, h] to [x1, y1, x2, y2] where xy1=top-left, xy2=bottom-right 148 | y = x.copy() if isinstance(x, np.ndarray) else np.copy(x) 149 | y[:, 0] = x[:, 0] - x[:, 2] / 2 # top left x 150 | y[:, 1] = x[:, 1] - x[:, 3] / 2 # top left y 151 | y[:, 2] = x[:, 0] + x[:, 2] / 2 # bottom right x 152 | y[:, 3] = x[:, 1] + x[:, 3] / 2 # bottom right y 153 | return y 154 | 155 | def box_area(self, box): 156 | # box = xyxy(4,n) 157 | return (box[2] - box[0]) * (box[3] - box[1]) 158 | 159 | def box_iou(self, box1, box2): 160 | # https://github.com/pytorch/vision/blob/master/torchvision/ops/boxes.py 161 | """ 162 | Return intersection-over-union (Jaccard index) of boxes. 163 | Both sets of boxes are expected to be in (x1, y1, x2, y2) format. 164 | Arguments: 165 | box1 (Tensor[N, 4]) 166 | box2 (Tensor[M, 4]) 167 | Returns: 168 | iou (Tensor[N, M]): the NxM matrix containing the pairwise 169 | IoU values for every element in boxes1 and boxes2 170 | """ 171 | 172 | # inter(N,M) = (rb(N,M,2) - lt(N,M,2)).clamp(0).prod(2) 173 | (a1, a2), (b1, b2) = box1[:, None].chunk(2, 2), box2.chunk(2, 1) 174 | # inter = (torch.min(a2, b2) - torch.max(a1, b1)).clamp(0).prod(2) 175 | inter = (np.min([a2, b2], 0) - np.max([a1, b1], 0)).clip(min=0).prod(2) 176 | 177 | # IoU = inter / (area1 + area2 - inter) 178 | return inter / (self.box_area(box1.T)[:, None] + self.box_area(box2.T) - inter) 179 | 180 | def nms(self, pred, conf_thres=0.25, iou_thres=0.5, agnostic=False, max_det=1000): 181 | return self.non_max_suppression(pred, conf_thres, iou_thres, 182 | classes=None, 183 | agnostic=agnostic, 184 | multi_label=True, 185 | labels=(), 186 | max_det=max_det) 187 | 188 | def non_max_suppression(self, 189 | prediction, 190 | conf_thres=0.25, 191 | iou_thres=0.5, 192 | classes=None, 193 | agnostic=False, 194 | multi_label=False, 195 | labels=(), 196 | max_det=300): 197 | """Non-Maximum Suppression (NMS) on inference results to reject overlapping bounding boxes 198 | 199 | Returns: 200 | list of detections, on (n,6) tensor per image [xyxy, conf, landmarks, cls] 201 | """ 202 | 203 | bs = prediction.shape[0] # batch size 204 | nc = prediction.shape[2] - 5 - 8 # number of classes, and 8 landmark points 205 | xc = prediction[..., 4] > conf_thres # candidates 206 | # Checks 207 | # assert 0 <= conf_thres <= 1, f'Invalid Confidence threshold {conf_thres}, valid values are between 0.0 and 1.0' 208 | # assert 0 <= iou_thres <= 1, f'Invalid IoU {iou_thres}, valid values are between 0.0 and 1.0' 209 | 210 | # Settings 211 | max_wh = 4096 # (pixels) maximum box width and height 212 | max_nms = 1000 # maximum number of boxes into torchvision.ops.nms() 213 | multi_label &= nc > 1 # multiple labels per box (adds 0.5ms/img) 214 | output = [np.zeros((0, 16))] * prediction.shape[0] # add 8 landmarks + 3 labels 215 | for xi, x in enumerate(prediction): # image index, image inference 216 | # Apply constraints 217 | # x[((x[..., 2:4] < min_wh) | (x[..., 2:4] > max_wh)).any(1), 4] = 0 # width-height 218 | x = x[xc[xi]] # confidence 219 | 220 | # Cat apriori labels if autolabelling 221 | if labels and len(labels[xi]): 222 | l = labels[xi] 223 | v = np.zeros((len(l), nc + 13)) 224 | v[:, :4] = l[:, 1:5] # box 225 | v[:, 4] = 1.0 # conf 226 | v[range(len(l)), l[:, 0].long() + 13] = 1.0 # cls 227 | x = np.concatenate((x, v), 0) 228 | 229 | # If none remain process next image 230 | if not x.shape[0]: 231 | continue 232 | 233 | # Compute conf 234 | x[:, 13:] *= x[:, 4:5] # conf = obj_conf * cls_conf 235 | 236 | # Box (center x, center y, width, height) to (x1, y1, x2, y2) 237 | box = self.xywh2xyxy(x[:, :4]) 238 | 239 | # Detections matrix nx6 (xyxy, conf, landmarks, cls) 240 | if multi_label: 241 | i, j = np.where(x[:, 13:] > conf_thres) 242 | x = np.concatenate([box[i], x[i, j + 13, None], x[i, 5:13].astype(np.float32), j.astype(np.float32)[:, None]], 1) 243 | else: # best class only 244 | conf = x[:, 13:].max(1, keepdims=True) 245 | j_argmax = x[:, 13:].argmax(1) 246 | j = j_argmax if j_argmax.shape == x[:, 13:].shape else \ 247 | np.expand_dims(j_argmax, 1) # for argmax(axis, keepdims=True) 248 | x = np.concatenate([box, conf, x[:, 5:13], j.astype(np.float32)], 1)[conf.reshape(-1) > conf_thres] 249 | 250 | # Check shape 251 | n = x.shape[0] # number of boxes 252 | if not n: # no boxes 253 | continue 254 | elif n > max_nms: # excess boxes 255 | x_argsort = np.argsort(x[:, 4])[:max_nms] # sort by confidence 256 | x = x[x_argsort] 257 | # Batched NMS 258 | c = x[:, 13:] * (0 if agnostic else max_wh) # classes 259 | boxes, scores = x[:, :4], x[:, 4] # boxes (offset by class), scores 260 | 261 | ############################################# 262 | # i = torchvision.ops.nms(boxes, scores, iou_thres) # NMS 263 | i = self.nms_boxes(boxes, scores, iou_thres) 264 | ############################################ 265 | 266 | if i.shape[0] > max_det: # limit detections 267 | i = i[:max_det] 268 | 269 | output[xi] = x[i] 270 | 271 | return output 272 | 273 | -------------------------------------------------------------------------------- /python/utils.py: -------------------------------------------------------------------------------- 1 | #===----------------------------------------------------------------------===# 2 | # 3 | # Copyright (C) 2022 Sophgo Technologies Inc. All rights reserved. 4 | # 5 | # SOPHON-DEMO is licensed under the 2-Clause BSD License except for the 6 | # third-party components. 7 | # 8 | #===----------------------------------------------------------------------===# 9 | import os 10 | import sys 11 | import numpy as np 12 | import cv2 13 | 14 | COCO_CLASSES = ('background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 15 | 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 16 | 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 17 | 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 18 | 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 19 | 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 20 | 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 21 | 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 22 | 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 23 | 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 24 | 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 25 | 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 26 | 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 27 | 'scissors', 'teddy bear', 'hair drier', 'toothbrush') 28 | 29 | PLATE_CLASSES = ("single", "double", "car") 30 | 31 | COLORS = [ 32 | [0, 94, 255], 33 | [0, 37, 255], 34 | [0, 255, 94], 35 | [255, 226, 0], 36 | [0, 18, 255], 37 | [255, 151, 0], 38 | [170, 0, 255], 39 | [0, 255, 56], 40 | [255, 0, 75], 41 | [56, 0, 255], 42 | [226, 255, 0], 43 | [0, 75, 255], 44 | [0, 255, 169], 45 | [255, 0, 207], 46 | [75, 255, 0], 47 | [207, 0, 255], 48 | [37, 0, 255], 49 | [0, 207, 255], 50 | [94, 0, 255], 51 | [0, 255, 113], 52 | [255, 18, 0], 53 | [255, 0, 56], 54 | [18, 0, 255], 55 | [0, 255, 226], 56 | [170, 255, 0], 57 | [255, 0, 245], 58 | [151, 255, 0], 59 | [132, 255, 0], 60 | [75, 0, 255], 61 | [151, 0, 255], 62 | [0, 151, 255], 63 | [132, 0, 255], 64 | [0, 255, 245], 65 | [255, 132, 0], 66 | [226, 0, 255], 67 | [255, 37, 0], 68 | [207, 255, 0], 69 | [0, 255, 207], 70 | [94, 255, 0], 71 | [0, 226, 255], 72 | [56, 255, 0], 73 | [255, 94, 0], 74 | [255, 113, 0], 75 | [0, 132, 255], 76 | [255, 0, 132], 77 | [255, 170, 0], 78 | [255, 0, 188], 79 | [113, 255, 0], 80 | [245, 0, 255], 81 | [113, 0, 255], 82 | [255, 188, 0], 83 | [0, 113, 255], 84 | [255, 0, 0], 85 | [0, 56, 255], 86 | [255, 0, 113], 87 | [0, 255, 188], 88 | [255, 0, 94], 89 | [255, 0, 18], 90 | [18, 255, 0], 91 | [0, 255, 132], 92 | [0, 188, 255], 93 | [0, 245, 255], 94 | [0, 169, 255], 95 | [37, 255, 0], 96 | [255, 0, 151], 97 | [188, 0, 255], 98 | [0, 255, 37], 99 | [0, 255, 0], 100 | [255, 0, 170], 101 | [255, 0, 37], 102 | [255, 75, 0], 103 | [0, 0, 255], 104 | [255, 207, 0], 105 | [255, 0, 226], 106 | [255, 245, 0], 107 | [188, 255, 0], 108 | [0, 255, 18], 109 | [0, 255, 75], 110 | [0, 255, 151], 111 | [255, 56, 0], 112 | [245, 255, 0], 113 | ] 114 | 115 | def is_img(file_name): 116 | """judge the file is available image or not 117 | Args: 118 | file_name (str): input file name 119 | Returns: 120 | (bool) : whether the file is available image 121 | """ 122 | fmt = os.path.splitext(file_name)[-1] 123 | if isinstance(fmt, str) and fmt.lower() in ['.jpg','.png','.jpeg','.bmp','.jpeg','.webp']: 124 | return True 125 | else: 126 | return False 127 | 128 | # scipy: https://zhuanlan.zhihu.com/p/375344904 129 | def logsumexp(a, axis=None, b=None, keepdims=False, return_sign=False): 130 | """Compute the log of the sum of exponentials of input elements. 131 | Parameters 132 | ---------- 133 | a : array_like 134 | Input array. 135 | axis : None or int or tuple of ints, optional 136 | Axis or axes over which the sum is taken. By default `axis` is None, 137 | and all elements are summed. 138 | .. versionadded:: 0.11.0 139 | keepdims : bool, optional 140 | If this is set to True, the axes which are reduced are left in the 141 | result as dimensions with size one. With this option, the result 142 | will broadcast correctly against the original array. 143 | .. versionadded:: 0.15.0 144 | b : array-like, optional 145 | Scaling factor for exp(`a`) must be of the same shape as `a` or 146 | broadcastable to `a`. These values may be negative in order to 147 | implement subtraction. 148 | .. versionadded:: 0.12.0 149 | return_sign : bool, optional 150 | If this is set to True, the result will be a pair containing sign 151 | information; if False, results that are negative will be returned 152 | as NaN. Default is False (no sign information). 153 | .. versionadded:: 0.16.0 154 | Returns 155 | ------- 156 | res : ndarray 157 | The result, ``np.log(np.sum(np.exp(a)))`` calculated in a numerically 158 | more stable way. If `b` is given then ``np.log(np.sum(b*np.exp(a)))`` 159 | is returned. 160 | sgn : ndarray 161 | If return_sign is True, this will be an array of floating-point 162 | numbers matching res and +1, 0, or -1 depending on the sign 163 | of the result. If False, only one result is returned. 164 | 165 | """ 166 | # a = _asarray_validated(a, check_finite=False) 167 | if b is not None: 168 | a, b = np.broadcast_arrays(a, b) 169 | if np.any(b == 0): 170 | a = a + 0. # promote to at least float 171 | a[b == 0] = -np.inf 172 | 173 | a_max = np.amax(a, axis=axis, keepdims=True) 174 | 175 | if a_max.ndim > 0: 176 | a_max[~np.isfinite(a_max)] = 0 177 | elif not np.isfinite(a_max): 178 | a_max = 0 179 | 180 | if b is not None: 181 | b = np.asarray(b) 182 | tmp = b * np.exp(a - a_max) 183 | else: 184 | tmp = np.exp(a - a_max) 185 | 186 | # suppress warnings about log of zero 187 | with np.errstate(divide='ignore'): 188 | s = np.sum(tmp, axis=axis, keepdims=keepdims) 189 | if return_sign: 190 | sgn = np.sign(s) 191 | s *= sgn # /= makes more sense but we need zero -> zero 192 | out = np.log(s) 193 | 194 | if not keepdims: 195 | a_max = np.squeeze(a_max, axis=axis) 196 | out += a_max 197 | 198 | if return_sign: 199 | return out, sgn 200 | else: 201 | return out 202 | 203 | # scipy.special.softmax: https://zhuanlan.zhihu.com/p/375344904 204 | def softmax(x, axis=None): 205 | r""" 206 | Softmax function 207 | The softmax function transforms each element of a collection by 208 | computing the exponential of each element divided by the sum of the 209 | exponentials of all the elements. That is, if `x` is a one-dimensional 210 | numpy array:: 211 | softmax(x) = np.exp(x)/sum(np.exp(x)) 212 | Parameters 213 | ---------- 214 | x : array_like 215 | Input array. 216 | axis : int or tuple of ints, optional 217 | Axis to compute values along. Default is None and softmax will be 218 | computed over the entire array `x`. 219 | Returns 220 | ------- 221 | s : ndarray 222 | An array the same shape as `x`. The result will sum to 1 along the 223 | specified axis. 224 | 225 | """ 226 | # compute in log space for numerical stability 227 | return np.exp(x - logsumexp(x, axis=axis, keepdims=True)) 228 | -------------------------------------------------------------------------------- /weights/car_plate_detect_1x3x80x80x16-640-640_rm_transpose_rk3588.rknn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/weights/car_plate_detect_1x3x80x80x16-640-640_rm_transpose_rk3588.rknn -------------------------------------------------------------------------------- /weights/plate.txt: -------------------------------------------------------------------------------- 1 | Single-Layer plate 2 | Dual-Layer plate 3 | car 4 | -------------------------------------------------------------------------------- /weights/plate_recognition_color-168-48_rm_transpose_rk3588.rknn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/YaoQ/plate-detection-RK3588/7572329bdc0783845c1c1a34b59360351760932b/weights/plate_recognition_color-168-48_rm_transpose_rk3588.rknn --------------------------------------------------------------------------------