├── FPGA.md ├── LICENSE ├── README.md ├── demo.py ├── experiments ├── multi_platform │ ├── test.sh │ └── train.sh ├── transfer_platform │ ├── raw_test.sh │ ├── raw_train.sh │ ├── transfer_test.sh │ └── transfer_train.sh └── unseen_structure │ ├── test.sh │ └── train.sh ├── png ├── latency-measurement-pipeline.png └── latency-prediction-pipeline.png ├── predictor ├── dataset.py ├── feature │ ├── __init__.py │ ├── feature_utils.py │ ├── graph_feature.py │ ├── node_feature.py │ ├── onnx_flops.py │ ├── onnx_shape_infer.py │ ├── onnx_to_networkx.py │ └── op_attribute.py ├── main.py ├── model.py └── tocsv.py └── test_onnx └── resnet18-v1-7-no-weight.onnx /FPGA.md: -------------------------------------------------------------------------------- 1 | ### Latency Measurement on FPGA 2 | 3 | ##### Environment 4 | * Hardware: **Avnet Ultra96-V2 Board** 5 | * Software: **Vitis-AI 1.4.0** 6 | * Datatype: **int8** 7 | 8 | ##### Step 9 | * Change onnx model to caffe model 10 | * Quantize model by `vai_q_caffe` 11 | * Compile and get xmodel by `vai_c_caffe` 12 | * Run xmodel on DPU 13 | 14 | ##### Note 15 | The following errors will occur during network measurement on FPGA, so the `gt.txt` file does not contain a large number of latency items. 16 | 17 | | stage | related networks | error | reason | 18 | | :-----: | :-------:| :------: | :------: | 19 | | compilation | EfficientNets & MobileNetV3s | ValueError: Unsupported op: type: sigmoid, name: xxxxx | not support Sigmoid | 20 | | compilation | GoogleNets | [UNILOG][FATAL][XCOM_DATA_OUTRANGE][Data value is out of range!] | Unknown | 21 | | compilation | MobileNetV2s | ValueError: Unsupported group convolution: group=xxxxx | not support group != input channel for Depthwise Conv| 22 | | inference | NASBench201s & MNasNets | Check failed: round_mode == "DPU_ROUND"(STD_ROUND vs. DPU_ROUND) TODO | not support STD_ROUND | 23 | | inference | ResNet18s & SqueezeNets | Check failed: handle != NULL cannot open library! lib=libvart_op_imp_conv2d-fix.so;error=libvart_op_imp_conv2d-fix.so: cannot open shared object file: No such file or directory;op=xir::Op{name = xxxxx, type = conv2d-fix} | not support kernel size >= 9 x 9 for Conv | 24 | | inference | Vggs & AlexNets | Check failed: handle != NULL cannot open library! lib=libvart_op_imp_conv2d-fix.so;error=libvart_op_imp_conv2d-fix.so: cannot open shared object file: No such file or directory;op=xir::Op{name = xxxxx(TransferMatMulToConv2d), type = conv2d-fix} | not support input channel or output channel > 3072 for FC| -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # NNLQP 2 | 3 | NNLQP: A Multi-Platform Neural Network Latency Query and Prediction System with An Evolving Database 4 | 5 | ## Installation 6 | 7 | #### Environment 8 | * `Ubuntu 16.04` 9 | * `python 3.6.4` 10 | 11 | #### Install pytorch onnx networkx 12 | ```shell 13 | pip3 install networkx==2.5.1 14 | pip3 install onnx==1.7.0 15 | pip3 install torch==1.5.0 16 | ``` 17 | 18 | #### Install torch-geometric 19 | 20 | 21 | You should first specific CUDA version (`cpu`, `cu92`, `cu101`, `cu102`, `cu110`, `cu111`, `cu113`), and visit the page https://data.pyg.org/whl/torch-1.5.0+${CUDA}.html to look up for pre-built `torch-scatter` and `torch-sparse` packages. 22 | 23 | If `CUDA==cu102` (cuda-10.2), we can visit the page https://data.pyg.org/whl/torch-1.5.0+cu102.html, and found latest packages, and then install them by: 24 | 25 | ```shell 26 | pip3 install https://data.pyg.org/whl/torch-1.5.0%2Bcu102/torch_scatter-2.0.5-cp36-cp36m-linux_x86_64.whl 27 | pip3 install https://data.pyg.org/whl/torch-1.5.0%2Bcu102/torch_sparse-0.6.7-cp36-cp36m-linux_x86_64.whl 28 | pip3 install torch-geometric 29 | ``` 30 | 31 | For more installation details, please refer [torch-geometric official documentation](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html) 32 | 33 | #### Run Demo 34 | Clone the repository 35 | ```shell 36 | git clone https://github.com/ModelTC/NNLQP.git 37 | ``` 38 | 39 | Given an onnx model as input, the demo can predict model latencies on 9 platforms: 40 | ```shell 41 | cd NNLQP 42 | python3 demo.py --onnx_file test_onnx/resnet18-v1-7-no-weight.onnx 43 | ``` 44 | 45 | Output: 46 | ```text 47 | Read Onnx: test_onnx/resnet18-v1-7-no-weight.onnx 48 | Model inference cost: 44.1431999206543 ms 49 | Latency prediction for platform cpu-openppl-fp32 : 386.098876953125 ms 50 | Latency prediction for platform hi3559A-nnie11-int8 : 62.860267639160156 ms 51 | Latency prediction for platform gpu-T4-trt7.1-fp32 : 7.964828014373779 ms 52 | Latency prediction for platform gpu-T4-trt7.1-int8 : 1.4778786897659302 ms 53 | Latency prediction for platform gpu-P4-trt7.1-fp32 : 9.099410057067871 ms 54 | Latency prediction for platform gpu-P4-trt7.1-int8 : 3.7260262966156006 ms 55 | Latency prediction for platform hi3519A-nnie12-int8 : 61.44997787475586 ms 56 | Latency prediction for platform atlas300-acl-fp16 : 9.21658992767334 ms 57 | Latency prediction for platform mul270-neuware-int8 : 18.250690460205078 ms 58 | ``` 59 | 60 | ## Dataset 61 | 62 | #### Download 63 | ```shell 64 | cd NNLQP 65 | mkdir dataset 66 | wget https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/dataset.tar.gz -O dataset.tar.gz 67 | tar -xzvf dataset.tar.gz -C dataset 68 | wget https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/dataset2.tar.gz -O dataset2.tar.gz 69 | tar -xzvf dataset2.tar.gz -C dataset 70 | ``` 71 | #### Format 72 | The latency dataset is saved to directory `nnlqp/dataset`: 73 | ```text 74 | └── dataset 75 | ├── multi_platform 76 | │ ├── gt.txt 77 | │ └── onnx 78 | │ ├── ... 79 | │ ├── ... 80 | ├── unseen_structure 81 | | ├── gt.txt 82 | | └── onnx 83 | | ├── ... 84 | | ├── ... 85 | ├── fpga 86 | | ├── gt.txt 87 | | └── onnx 88 | | ├── ... 89 | | ├── ... 90 | ├── vit 91 | | ├── gt.txt 92 | | └── onnx 93 | | ├── ... 94 | | ├── ... 95 | ``` 96 | 97 | `onnx` is the directory of onnx models, which are removed weights. 98 | `gt.txt` is the latency ground-truth file, each line of the file is format as: 99 | 100 | ```text 101 | ${GraphID} ${OnnxPath} ${BatchSize} ${Latency}(ms) ${ModelType} ${PlatformId} [${Stage}](train/test) 102 | ``` 103 | 104 | The `gt.txt` file can like: 105 | ```text 106 | 553312 onnx/nnmeter_alexnet/nnmeter_alexnet_transform_0130.onnx 8 90.88183 alexnet 2 train 107 | 559608 onnx/nnmeter_alexnet/nnmeter_alexnet_transform_0157.onnx 8 11.1463 alexnet 23 train 108 | 549409 onnx/nnmeter_alexnet/nnmeter_alexnet_transform_0197.onnx 8 64.11547 alexnet 16 test 109 | ... 110 | ``` 111 | #### Overview 112 | The datasets contains two parts: `unseen strucutre` and `multi platform`. 113 | We will constantly add new structures and platforms to the dataset. 114 | ##### Unseen Structure 115 | * Platforms 116 | * `gpu-gtx1660-trt7.1-fp32` 117 | * Onnx models (2000 * 10) 118 | * [x] ResNets 119 | * [x] VGGs 120 | * [x] EfficientNets 121 | * [x] MobileNetV2s 122 | * [x] MobileNetV3s 123 | * [x] MnasNets 124 | * [x] AlexNets 125 | * [x] SqueezeNets 126 | * [x] GoogleNets 127 | * [x] NasBench201s 128 | * [x] VisionTransformers (inside `dataset2.tar.gz`) 129 | * Input sizes 130 | * `1 x 3 x 32 x 32` (only for NasBenc201s) 131 | * `1 x 3 x 224 x224` 132 | * Latency samples 133 | * 20000 134 | 135 | ##### Multi Platform 136 | * Platforms 137 | * [x] cpu-openppl-fp32 (CPU) 138 | * [x] hi3559A-nnie11-int8 (NPU) 139 | * [x] gpu-T4-trt7.1-fp32 (GPU) 140 | * [x] gpu-T4-trt7.1-int8 (GPU) 141 | * [x] gpu-P4-trt7.1-fp32 (GPU) 142 | * [x] gpu-P4-trt7.1-int8 (GPU) 143 | * [x] hi3519A-nnie12-int8 (NPU) 144 | * [x] atlas300-acl-fp16 (NPU) 145 | * [x] mul270-neuware-int8 (NPU) 146 | * [ ] hexagonDSP-snpe-int8 (DSP) 147 | * [x] Xilinx-Ultra96-VitisAI-int8 (FPGA) (inside `dataset2.tar.gz`, see [FPGA readme](FPGA.md)) 148 | * Onnx models (200 * 10) 149 | * ResNets 150 | * VGGs 151 | * EfficientNets 152 | * MobileNetV2s 153 | * MobileNetV3s 154 | * MnasNets 155 | * AlexNets 156 | * SqueezeNets 157 | * GoogleNets 158 | * NasBench201s 159 | * Input sizes 160 | * `8 x 3 x 32 x 32` (only for NasBenc201s) 161 | * `8 x 3 x 224 x224` 162 | * Latency samples 163 | * 10597 164 | 165 | ## Experiments 166 | #### GNN Prediction Pipeline 167 | ![avatar](png/latency-prediction-pipeline.png) 168 | 169 | #### Unseen Structure 170 | We use a certain model type for test, and use other types for training. The user can specify the test model type by specifying the `TEST_MODEL_TYPE` in the script. 171 | 172 | Before experiments, please make sure that you download the dataset, and the directory `nnlqp/dataset/unseen_structure/` are valid. 173 | 174 | ```shell 175 | cd NNLQP/experiments/unseen_structure/ 176 | ``` 177 | 178 | * Get our pre-trained models and test 179 | ```shell 180 | wegt https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/unseen_structure_ckpt.tar.gz -O unseen_structure_ckpt.tar.gz 181 | tar -xzvf unseen_structure_ckpt.tar.gz 182 | bash test.sh 183 | ``` 184 | 185 | * Train from the beginning 186 | ```shell 187 | bash train.sh 188 | ``` 189 | 190 | #### Multi Platform 191 | 192 | Before experiments, please make sure that you download the dataset, and the directory `nnlqp/dataset/multi_platform/` are valid. 193 | 194 | ```shell 195 | cd NNLQP/experiments/multi_platform/ 196 | ``` 197 | 198 | * Get our pre-trained models and test 199 | ```shell 200 | wget https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/multi_platform_ckpt.tar.gz -O multi_platform_ckpt.tar.gz 201 | tar -xzvf multi_platform_ckpt.tar.gz 202 | bash test.sh 203 | ``` 204 | 205 | * Train from the beginning 206 | Users can change `--multi_plt=1,3,4` to train a predictor that predicts latency for platforms 1, 3, 4 at the same time. The platform id should have appeared in the file `gt.txt` 207 | 208 | ```shell 209 | bash train.sh 210 | ``` 211 | 212 | #### Transfer Platform 213 | 214 | If there are latency samples on 4 platforms A, B, C, D, we can first train a pre-trained model that predicts latency on platforms A, B, C. Then we can adopt the pre-trained model to initialize weights, and use a small count of latency samples to train a predictor fo platform D. The knowledge of the model structure learned by the predictor for platforms A, B, C can be transferred to platform D. 215 | 216 | For example, the `multi platforms` dataset involved 9 platforms, we can train a pre-trained model on 8 platforms 2, 9, 12, 13, 14, 16, 18, 23, and then use a certain number of samples to train predictor of platform 10. Users can set `TRAIN_NUM` in `raw_train.sh` or `transfer_train.sh` to control the samples numbers. In this example, we use `32`, `100`, `200`, `300`, `-1 (all)`. `raw_train.sh` is for training a predictor without the pre-trained model and transfer learning, while `transfer_train.sh` uses the pre-trained model and transfer learning. 217 | 218 | ```shell 219 | cd NNLQP/experiments/transfer_platform/ 220 | ``` 221 | 222 | * Users can test the results of trained predictors: 223 | ```shell 224 | wget https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/transfer_platform_ckpt.tar.gz -O transfer_platform_ckpt.tar.gz 225 | tar -xzvf transfer_platform_ckpt.tar.gz 226 | bash raw_test.sh 227 | bash transfer_test.sh 228 | ``` 229 | 230 | * Or just train from the beginning: 231 | ```shell 232 | bash raw_train.sh 233 | bash transfer_train.sh 234 | ``` 235 | 236 | #### Custom Predictor 237 | 238 | First, users should prepare the latency dataset with [the same format as our dataset](#format). 239 | 240 | Then you can train your own latency predictor in two different ways: 241 | 242 | * Train from the beginning: 243 | * refer the script `nnlqp/unseen_structure/train.sh` 244 | 245 | * Transfer from our pre-trained model (more suitable for a small count of latency samples) 246 | * refer the script `nnlqp/transfer_platform/transfer_train.sh` 247 | 248 | 249 | ## Latency Measurement 250 | 251 | #### How to convert ONNX into hardware format 252 | To facilitate the latency measurement of multiple platforms, we developed a unified model deployment tool. Its pipeline is shown as below: 253 | 254 | ![avatar](png/latency-measurement-pipeline.png) 255 | Given the ONNX, hardware, software, batch size, data type as input, the tool does: 256 | 257 | 1. Converts ONNX to our unified graph intermediate representation (graph IR), which describes operator attributes and operator connection relationships; 258 | 259 | 2. Does some common passes to transform and verify the graph IR, such as inferring shapes and folding constant; 260 | 261 | 3. According to the query software and hardware, does some platform-aware passes so that transformed graph IR conforms to the inference constraints. For example: 262 | 263 | * For `data_type=int8`, it is necessary to do the quantization pass; 264 | * For `software=openppl`, we need to call some operator-altering passes, such as transforming Add, Sub, Mul, and Div to Eltwise; 265 | * For `software=trt7.1`, we need to call the operator fusion pass. 266 | * In addition, we provide a large number of operators with the default CPU implementation. For some operators that are not supported by the platform, the tool will split the graph IR and make these operators run on the CPU. The strategy can make sure that most models can be deployed on multiple platforms smoothly; 267 | 268 | 4. Adopts platform-aware compilation tools to compile the graph IR into executable engines that can run on hardware. 269 | 270 | 5. The tool manages multiple hardware devices. It can acquire access to a specific device based on the latency measurement request, and upload the engine and inference environment to the target machine by the RPC. Then it does the model inference and return latency results. 271 | 272 | #### Device Setup 273 | Our latency measurement tool supports the following platforms, and the platform count is continuously growing. 274 | 275 | | type | hardware | software |manufacturer| processing power | power | 276 | | :--: | :---------:| :------: | :------: | :----: | :----:| 277 | | GPU | GTX-1660 | TensorRT-7.1|Nvidia | FP16: 8616 GFLOPS, FP32: 4308 GFLOPS, FP64: 135 GFLOPS | 120W 278 | | CPU | Intel-Xeon-Gold-6246| OpenPPL |Intel | FP32: 211.2 GFLOPS, FP64: 105.6 GFLOPS (single core) | 165W| 279 | | NPU | Hi3559AV100 | NNIE11 |Hisilicon | INT8: 4TOPS (2 NNIE cores) | 3W 280 | | GPU | Tesla-T4 | TensorRT-7.1|Nvidia | INT4: 260 TOPS, INT8: 130 TOPS, FP16 / FP32 mix：65 TFLOPS, FP32: 8.1 TFLOPS | 70W 281 | | GPU | Tesla-P4 | TensorRT-7.1|Nvidia | INT8: 22 TOPS, FP32: 5.5 TFLOPS | 50/75W 282 | | NPU | Hi3519AV100 | NNIE12 |Hisilicon | INT8: 1.7 TOPS | 2.2W 283 | | NPU | Atlas300 | ACL |Huawei | INT8: 22 TOPS, FP: 11 TFLOPS | 8W 284 | | NPU | MLU270 | Neuware |Cambricon | INT4: 256 TOPS, INT8: 128 TOPS, INT16: 64 TOPS | 70W 285 | | *DSP | HexagonDSP | SNPE |Qualcomm | INT8: 26 TOPS | 5W 286 | | FPGA| Ultra96-V2|VitisAI|Xilinx | INT8: 675 TOPS | - 287 | 288 | (* means to be supported soon) 289 | 290 | The latency measurement and query database are currently not open-sourced. Shortly, we will provide external services for everyone to use. 291 | 292 | 293 | ## Why and how does our NNLQP benefit the ML model production？ 294 | 295 | #### User Case 296 | 297 | Let’s take the face unlock model for mobile phones as an example. For developers of deep learning application algorithms on a mobile phone, they intend to run on multiple devices such as Apple and Samsung and so on. These mobile phones also have different chips, such as Apple A, Qualcomm Snapdragon, MTK. Even for Qualcomm chips, there are four different types ARM/GPU/DSP/NPU. If the inference latency of the face unlocking model is required to be within 0.1s, we need to design a model to meet the latency requirements and decide what kind of device can be used. The accurate and true latency feedback is essential for model design and deployment. Therefore, the developer can use our system to alleviate the cost of model latency acquisition from two aspects: latency query and latency prediction. More specifically, we have following cases which can help with machine learning model design and improvement. 298 | 299 | * NNLQ can free researchers from the tedious process of collecting the latency of DNN models from various hardware devices: While adapting to different hardware, we need to deploy models on different platforms and get true latency feedback.  For example, we can use TensorRT to get the latency feedback on NVIDIA GPU, while we cannot use TensorRT to transform model into hardware format for other hardware devices. Most hardware has different deploying format. Therefore, the proposed model latency query system NNLQ can perform automatic model deployment and latency measurement on multiple platforms. ONNX model and target platform are provided as query input, and NNLQ returns the realistic model latency on the target hardware through the three steps: model transformation, device acquisition and latency measurement. 300 | 301 | * In the model design, our model latency acquisition system can provide some **high-level decision**: 302 | 303 | * Which operators are not suitable for ability: for example, hard swish is not supported on openppl. Therefore we should avoid using this operation. 304 | * On the choice of backbone to achieve better latency-accuracy tradeoff: For example, RegNetX-200M ResNet18 have similar ImageNet accuracy which is 68.7 and 70.3, but the latency of RegNetX-200M is 150% of ResNet18 on P4 int8, therefore, we should choose ResNet18 compared with RegNetX-200M. 305 | * In the choice of hardware for inference speedup: Given the same model-ResNet 18 + data type int8 + batch size 1, the latency on P4 is 2 times of the latency on T4. If these two devices are available, changing deployed device from P4 to T4 can bring 50% speedup. Besides, atlas300 is faster than mlu270 under the same setting. 306 | * In the choice of data type for possible accuracy degradation: for the vision transformer models, the speed up brought by int8 compared with FP32 is less than 5%. To avoid the potential accuracy degradation, you can choose to fp32 data type directly. 307 | 308 | * In the network architecture search process, our model latency prediction can help to improve search cost and find models with higher task accuracy. If the current model is not able to be deployed on some required hardware, we need to redesign a model which is general across different platforms. Hardware-aware Network Architecture Search(NAS) is an effective method to improve the model performance. In hardware-aware NAS, models are selected with the help of hardware feedback, therefore, if the hardware feedback takes long time to be acquired, this could increase the search cost and hinder the use of hardware-aware NAS. 309 | 310 | * NAS needs to test a large number of models, and true latency measurement is very slow. Our latency prediction is able to predict true model latency with 1000 times improved efficiency as shown in Table 2. 311 | 312 | * Developers can further use the data from our evolving database to reduce the cost of latency predictor training. Because the prediction process brings a possible gap in the true latency and predicted latency. Improving the performance of the latency prediction allows us to simulate the true latency feedback as accurately as possible. The comparison of time cost is as follows in the table. 313 | 314 | | | measurement| prediction| test models | time cost | 315 | | :--: | :---------:| :-------: | :----: | :--: | 316 | | measurement | 1k | 0 | 1k | (1m + 0) * T | 317 | | without transfer | 1k | 10k | 10k | (1m+10k) * T 318 | | with transfer | 50 | 10k | 10k | (50k+10k) * T | 319 | 320 | (k=1,000, m=1,000,000, T=once prediction cost, 1000T=once true latency test cost) 321 | 322 | If the training cost of the predictor is high, we may not achieve the purpose of improving efficiency, but if we use historical information with our evolving database, we can get the highly accurate latency predictor with less cost, while getting more model speed. 323 | 324 | #### Extend to a new type of architecture 325 | 326 | * Generate the required ONNX model, we can produce 2k to server as the training and validation samples for latency prediction with 1k for both sides. 327 | * Using NNLQ to get the true speed of the model, we don’t need to convert to a hardware-specific format by ourselves, reducing the cost of speed measurement 328 | * In fact, we are able to predict latency with our pre-trained predictor if this hardware is already trained once. However, to improve accuracy, we also need to select some samples to finetune this predictor. 329 | * Select the samples required to train the latency predictor. With the help of historical latency information or trained hardware predictors, we are able to perform fast finetuning to obtain a high-precision latency predictor model. In our extension experiments with Vision Transformer, we find that only 50 samples are needed to get a high-precision speed predictor and are compared with the results of 1000 samples. 330 | 331 | | | MAPE| RMSPE| ErrBound(0.1) | time cost | 332 | | :--: | :---------:| :-------: | :----: | :--: | 333 | |1000 samples without pretrain | 0.01022 | 0.01367 | 0.999 | (1m + 1k) * T | 334 | |50 samples with pretrain | 0.00977 | 0.01315 | 0.999 | (50k + 1k) * T | 335 | 336 | (k=1,000, m=1,000,000, T=once prediction cost, 1000T=once true latency test cost) 337 | 338 | #### How does this help to NAS 339 | 340 | * From a theoretical analysis perspective, we have a much higher possibility for find models which meets the latency requirements. 341 | 342 | * Given the latency requirement of the model (e.g., 5ms), the improvement of the latency predictor acc can increase the probability of finding a model that meets the latency requirements. For example, with the true latency feedback, we are only able to test 1k models, but with the latency predictor, we are allowed to get the latency of 1w models. With the help of an accurate latency predictor, we guarantee that the prediction error is less than 10% for 95% of models. Therefore, we can get more models that meet the latency requirements, which will lead to an increased probability of finding a higher-precision model. 343 | 344 | 345 | * With the help of an accurate latency predictor and accurate accuracy predictor, we are able to find models with higher task accuracy. 346 | 347 | * In Section 8.7, we give an example that how does latency prediction helps find more accurate models compared with FLOPs, lookup table, predict latency and true latency. With more accurate latency feedback, we are able to produce models with 1.2% higher task accuracy. 348 | 349 | ## License 350 | NNLQP is licensed under the [Apache-2.0](LICENSE) license. -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- 1 | from genericpath import exists 2 | import sys 3 | import os 4 | import time 5 | import torch 6 | import logging 7 | import argparse 8 | import numpy as np 9 | 10 | from torch_geometric.data import Data, Batch 11 | from predictor.feature.graph_feature import extract_graph_feature 12 | 13 | 14 | def download_file(url, file_name): 15 | if not os.path.exists(file_name): 16 | import urllib.request as urllib2 17 | try: 18 | print("download from {} to {}...".format(url, file_name)) 19 | if sys.version_info >= (3,): 20 | urllib2.urlretrieve(url, file_name) 21 | else: 22 | f = urllib2.urlopen(url) 23 | data = f.read() 24 | with open(file_name, "wb") as code: 25 | code.write(data) 26 | except Exception as err: 27 | if os.path.exists(file_name): 28 | os.remove(file_name) 29 | raise Exception("download {} failed due to {}!".format(file_name, repr(err))) 30 | 31 | 32 | class Demo(object): 33 | 34 | def __init__(self): 35 | self.args = self.init_args() 36 | print("Loading args: \n{}".format(self.args)) 37 | self.model = self.init_model() 38 | # print("Loading model: \n{}".format(self.model)) 39 | 40 | def init_args(self): 41 | parser = argparse.ArgumentParser() 42 | parser.add_argument('--cpu', action='store_true') 43 | parser.add_argument('--onnx_file', required=True, type=str) 44 | args = parser.parse_args() 45 | 46 | args.cpu = True if args.cpu else False 47 | args.batch_size = 8 48 | args.multi_plt = "2,9,10,12,13,14,16,18,23" 49 | args.multi_names = [ 50 | 'cpu-openppl-fp32', 'hi3559A-nnie11-int8', 'gpu-T4-trt7.1-fp32', 51 | 'gpu-T4-trt7.1-int8', 'gpu-P4-trt7.1-fp32', 'gpu-P4-trt7.1-int8', 52 | 'hi3519A-nnie12-int8', 'atlas300-acl-fp16', 'mul270-neuware-int8', 53 | ] 54 | 55 | args.multi_plt = {int(x): k for k, x in enumerate(args.multi_plt.split(','))} if args.multi_plt else {} 56 | ckpt_dir = os.path.join("experiments", "multi_platform") 57 | args.resume = os.path.join(ckpt_dir, "checkpoints", "plt_2_9_10_12_13_14_16_18_23", "ckpt_best.pth") 58 | 59 | # download ckpt from the Internet 60 | if not os.path.exists(args.resume): 61 | fname = os.path.join(ckpt_dir, "multi_platform_ckpt.tar.gz") 62 | download_file("https://github.com/ModelTC/NNLQP/releases/download/v1.0-data/multi_platform_ckpt.tar.gz", fname) 63 | os.system("tar -xzvf {} -C ./experiments/multi_platform/".format(fname)) 64 | return args 65 | 66 | def init_model(self): 67 | from predictor.model import Net 68 | model = Net(multi_plt=self.args.multi_plt) 69 | if self.args.resume: 70 | print("Loading checkpoint: {}".format(self.args.resume)) 71 | ckpt = torch.load(self.args.resume) 72 | start_epoch, best_acc = ckpt['epoch'], ckpt['best_acc'] 73 | model.load_state_dict(ckpt['state_dict'], strict = True) 74 | print("loaded checkpoint: {} (epoch {} best {:.2f})". \ 75 | format(self.args.resume, start_epoch, best_acc)) 76 | return model 77 | 78 | def inference_once(self, onnx_file, batch_size, device): 79 | torch.manual_seed(1234) 80 | torch.cuda.manual_seed_all(1234) 81 | self.model = self.model.to(device) 82 | self.model.eval() 83 | t0 = time.time() 84 | 85 | # extract feature, to torch data 86 | A, F, S = extract_graph_feature(onnx_file, batch_size) 87 | 88 | F = np.array(F, dtype=np.float32) 89 | x = torch.from_numpy(F).type(torch.float) 90 | E = torch.from_numpy(np.array(np.where(A > 0))).type(torch.long) 91 | S = torch.from_numpy(S).type(torch.float) 92 | 93 | batch = Batch() 94 | batch = batch.from_data_list([Data(x = x, edge_index=E)]) 95 | batch = batch.to(device) 96 | static_feature = S.to(device).view(1, -1) 97 | 98 | preds = self.model(batch, static_feature) 99 | preds = preds.view(-1).data.cpu().numpy() 100 | t1 = time.time() 101 | print("Model inference cost: {} ms".format((t1 - t0) * 1000)) 102 | 103 | if self.args.multi_plt: 104 | for k, v in self.args.multi_plt.items(): 105 | print("Latency prediction for platform {} : {} ms".format(self.args.multi_names[v], preds[v])) 106 | else: 107 | print("Latency prediction: {} ms".format(preds[0])) 108 | 109 | def run(self): 110 | device = torch.device('cuda' if not self.args.cpu and torch.cuda.is_available() else 'cpu') 111 | self.inference_once(self.args.onnx_file, self.args.batch_size, device) 112 | 113 | 114 | if __name__ == "__main__": 115 | x = Demo() 116 | x.run() -------------------------------------------------------------------------------- /experiments/multi_platform/test.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | 7 | PID="plt_2_9_10_12_13_14_16_18_23" 8 | 9 | python $BASE_DIR/predictor/main.py \ 10 | --gpu 0 \ 11 | --lr 0.001 \ 12 | --steps 500 \ 13 | --epochs 500 \ 14 | --batch_size 16 \ 15 | --data_root "$DATASET_DIR/data" \ 16 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 17 | --train_test_stage \ 18 | --multi_plt 2,9,10,12,13,14,16,18,23 \ 19 | --onnx_dir "${DATASET_DIR}" \ 20 | --log "log/$PID.log" \ 21 | --model_dir "checkpoints/$PID" \ 22 | --gnn_layer "SAGEConv" \ 23 | --ckpt_save_freq 1000 \ 24 | --test_freq 1 \ 25 | --print_freq 50 \ 26 | --resume "checkpoints/$PID/ckpt_best.pth" \ 27 | --only_test \ 28 | #--override_data 29 | -------------------------------------------------------------------------------- /experiments/multi_platform/train.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | 7 | PID="plt_2_9_10_12_13_14_16_18_23" 8 | 9 | python $BASE_DIR/predictor/main.py \ 10 | --gpu 0 \ 11 | --lr 0.001 \ 12 | --steps 500 \ 13 | --epochs 500 \ 14 | --batch_size 16 \ 15 | --data_root "$DATASET_DIR/data" \ 16 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 17 | --train_test_stage \ 18 | --multi_plt 2,9,10,12,13,14,16,18,23 \ 19 | --onnx_dir "${DATASET_DIR}" \ 20 | --log "log/$PID.log" \ 21 | --model_dir "checkpoints/$PID" \ 22 | --gnn_layer "SAGEConv" \ 23 | --ckpt_save_freq 1000 \ 24 | --test_freq 1 \ 25 | --print_freq 50 \ 26 | #--resume "checkpoints/$PID/ckpt_best.pth" \ 27 | #--only_test \ 28 | #--override_data 29 | -------------------------------------------------------------------------------- /experiments/transfer_platform/raw_test.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | PID=10 7 | 8 | for TRAIN_NUM in 32 100 200 300 -1; do 9 | BASE=plt_${PID}_${TRAIN_NUM}_raw 10 | 11 | python $BASE_DIR/predictor/main.py \ 12 | --gpu 0 \ 13 | --lr 0.001 \ 14 | --steps 500 \ 15 | --epochs 500 \ 16 | --batch_size 16 \ 17 | --data_root "$DATASET_DIR/data" \ 18 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 19 | --train_test_stage \ 20 | --train_num ${TRAIN_NUM} \ 21 | --multi_plt ${PID} \ 22 | --onnx_dir "${DATASET_DIR}" \ 23 | --log "log/$BASE.log" \ 24 | --model_dir "checkpoints/$BASE" \ 25 | --gnn_layer "SAGEConv" \ 26 | --ckpt_save_freq 1000 \ 27 | --test_freq 1 \ 28 | --print_freq 50 \ 29 | --resume "checkpoints/$BASE/ckpt_best.pth" \ 30 | --only_test \ 31 | #--override_data 32 | done 33 | -------------------------------------------------------------------------------- /experiments/transfer_platform/raw_train.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | PID=10 7 | 8 | for TRAIN_NUM in 32 100 200 300 -1; do 9 | BASE=plt_${PID}_${TRAIN_NUM}_raw 10 | 11 | python $BASE_DIR/predictor/main.py \ 12 | --gpu 0 \ 13 | --lr 0.001 \ 14 | --steps 500 \ 15 | --epochs 500 \ 16 | --batch_size 16 \ 17 | --data_root "$DATASET_DIR/data" \ 18 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 19 | --train_test_stage \ 20 | --train_num ${TRAIN_NUM} \ 21 | --multi_plt ${PID} \ 22 | --onnx_dir "${DATASET_DIR}" \ 23 | --log "log/$BASE.log" \ 24 | --model_dir "checkpoints/$BASE" \ 25 | --gnn_layer "SAGEConv" \ 26 | --ckpt_save_freq 1000 \ 27 | --test_freq 1 \ 28 | --print_freq 50 \ 29 | #--resume "checkpoints/$BASE/ckpt_best.pth" \ 30 | #--only_test \ 31 | #--override_data 32 | done 33 | -------------------------------------------------------------------------------- /experiments/transfer_platform/transfer_test.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | PID=10 7 | 8 | for TRAIN_NUM in 32 100 200 300 -1; do 9 | BASE=plt_${PID}_${TRAIN_NUM}_transfer 10 | 11 | 12 | python $BASE_DIR/predictor/main.py \ 13 | --gpu 0 \ 14 | --lr 0.001 \ 15 | --steps 500 \ 16 | --epochs 500 \ 17 | --batch_size 16 \ 18 | --data_root "$DATASET_DIR/data" \ 19 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 20 | --train_test_stage \ 21 | --train_num ${TRAIN_NUM} \ 22 | --multi_plt ${PID} \ 23 | --onnx_dir "${DATASET_DIR}" \ 24 | --log "log/$BASE.log" \ 25 | --model_dir "checkpoints/$BASE" \ 26 | --gnn_layer "SAGEConv" \ 27 | --ckpt_save_freq 1000 \ 28 | --test_freq 1 \ 29 | --print_freq 50 \ 30 | --resume "checkpoints/$BASE/ckpt_best.pth" \ 31 | --only_test \ 32 | #--override_data 33 | done 34 | -------------------------------------------------------------------------------- /experiments/transfer_platform/transfer_train.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/multi_platform" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | PID=10 7 | PRETRAIN="./pretrain/plt_2_9_12_13_14_16_18_23/ckpt_best.pth" 8 | 9 | for TRAIN_NUM in 32 100 200 300 -1; do 10 | BASE=plt_${PID}_${TRAIN_NUM}_transfer 11 | 12 | 13 | python $BASE_DIR/predictor/main.py \ 14 | --gpu 0 \ 15 | --lr 0.001 \ 16 | --steps 500 \ 17 | --epochs 500 \ 18 | --batch_size 16 \ 19 | --data_root "$DATASET_DIR/data" \ 20 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 21 | --train_test_stage \ 22 | --train_num ${TRAIN_NUM} \ 23 | --multi_plt ${PID} \ 24 | --onnx_dir "${DATASET_DIR}" \ 25 | --log "log/$BASE.log" \ 26 | --model_dir "checkpoints/$BASE" \ 27 | --gnn_layer "SAGEConv" \ 28 | --ckpt_save_freq 1000 \ 29 | --test_freq 1 \ 30 | --print_freq 50 \ 31 | --pretrain ${PRETRAIN} \ 32 | #--resume "checkpoints/$BASE/ckpt_best.pth" \ 33 | #--only_test \ 34 | #--override_data 35 | done 36 | -------------------------------------------------------------------------------- /experiments/unseen_structure/test.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/unseen_structure" 3 | 4 | rm test.log 5 | mkdir log 6 | mkdir checkpoints 7 | 8 | for TEST_MODEL_TYPE in 'resnet18' 'vgg16' 'efficientb0' 'mobilenetv2' 'mobilenetv3' 'mnasnet' 'alexnet' 'squeezenet' 'googlenet' 'nasbench201';do 9 | 10 | python $BASE_DIR/predictor/main.py \ 11 | --gpu 0 \ 12 | --lr 0.001 \ 13 | --steps 500 \ 14 | --epochs 50 \ 15 | --batch_size 16 \ 16 | --data_root "$DATASET_DIR/data" \ 17 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 18 | --test_model_type ${TEST_MODEL_TYPE}\ 19 | --norm_sf \ 20 | --onnx_dir "${DATASET_DIR}" \ 21 | --log "log/$TEST_MODEL_TYPE.log" \ 22 | --model_dir "checkpoints/$TEST_MODEL_TYPE" \ 23 | --gnn_layer "SAGEConv" \ 24 | --ckpt_save_freq 1000 \ 25 | --test_freq 1 \ 26 | --print_freq 50 \ 27 | --resume "checkpoints/$TEST_MODEL_TYPE/ckpt_best.pth" \ 28 | --only_test \ 29 | #--override_data 30 | 31 | cat log/$TEST_MODEL_TYPE.log | tail -n 11 >> test.log 32 | done 33 | 34 | python3 $BASE_DIR/predictor/tocsv.py test.log 1 35 | -------------------------------------------------------------------------------- /experiments/unseen_structure/train.sh: -------------------------------------------------------------------------------- 1 | BASE_DIR="../.." 2 | DATASET_DIR="$BASE_DIR/dataset/unseen_structure" 3 | 4 | mkdir log 5 | mkdir checkpoints 6 | 7 | for TEST_MODEL_TYPE in 'resnet18' 'vgg16' 'efficientb0' 'mobilenetv2' 'mobilenetv3' 'mnasnet' 'alexnet' 'squeezenet' 'googlenet' 'nasbench201';do 8 | 9 | python $BASE_DIR/predictor/main.py \ 10 | --gpu 0 \ 11 | --lr 0.001 \ 12 | --steps 500 \ 13 | --epochs 50 \ 14 | --batch_size 16 \ 15 | --data_root "$DATASET_DIR/data" \ 16 | --all_latency_file "${DATASET_DIR}/gt.txt" \ 17 | --test_model_type ${TEST_MODEL_TYPE}\ 18 | --norm_sf \ 19 | --onnx_dir "${DATASET_DIR}" \ 20 | --log "log/$TEST_MODEL_TYPE.log" \ 21 | --model_dir "checkpoints/$TEST_MODEL_TYPE" \ 22 | --gnn_layer "SAGEConv" \ 23 | --ckpt_save_freq 1000 \ 24 | --test_freq 1 \ 25 | --print_freq 50 \ 26 | #--resume "checkpoints/$TEST_MODEL_TYPE/ckpt_best.pth" \ 27 | #--only_test \ 28 | #--override_data 29 | 30 | done 31 | -------------------------------------------------------------------------------- /png/latency-measurement-pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelTC/NNLQP/6da64f035baa2cc5ba356071bee28b1c5b71f988/png/latency-measurement-pipeline.png -------------------------------------------------------------------------------- /png/latency-prediction-pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelTC/NNLQP/6da64f035baa2cc5ba356071bee28b1c5b71f988/png/latency-prediction-pipeline.png -------------------------------------------------------------------------------- /predictor/dataset.py: -------------------------------------------------------------------------------- 1 | import os 2 | import onnx 3 | import time 4 | import torch 5 | import random 6 | import numpy as np 7 | from torch_geometric.data import Data, Dataset 8 | from feature.graph_feature import extract_graph_feature 9 | 10 | 11 | def get_torch_data(onnx_file, batch_size, cost_time): 12 | adjacent, node_features, static_features = extract_graph_feature(onnx_file, batch_size) 13 | edge_index = torch.from_numpy(np.array(np.where(adjacent > 0))).type(torch.long) 14 | node_features = np.array(node_features, dtype=np.float32) 15 | x = torch.from_numpy(node_features).type(torch.float) 16 | sf = torch.from_numpy(static_features).type(torch.float) 17 | y = torch.FloatTensor([cost_time]) 18 | data = Data( 19 | x = x, 20 | edge_index = edge_index, 21 | y = y, 22 | ) 23 | return data, sf 24 | 25 | 26 | class GraphLatencyDataset(Dataset): 27 | # specific a platform 28 | def __init__(self, root, onnx_dir, latency_file, override_data=False, transform=None, pre_transform=None, 29 | model_types=None, train_test_stage=None, platforms=None, sample_num=-1): 30 | super(GraphLatencyDataset, self).__init__(root, transform, pre_transform) 31 | self.onnx_dir = onnx_dir 32 | self.latency_file = latency_file 33 | self.latency_ids = [] 34 | self.override_data = override_data 35 | self.model_types = model_types 36 | self.train_test_stage = train_test_stage 37 | self.platforms = platforms 38 | 39 | # Load the gnn model for block 40 | self.device = None 41 | print("Extract input data from onnx...") 42 | self.custom_process() 43 | print("Done.") 44 | 45 | if sample_num > 0: 46 | random.seed(1234) 47 | random.shuffle(self.latency_ids) 48 | self.latency_ids = self.latency_ids[:sample_num] 49 | random.seed(1234) 50 | random.shuffle(self.latency_ids) 51 | 52 | @property 53 | def raw_file_names(self): 54 | return [] 55 | 56 | @property 57 | def processed_file_names(self): 58 | return [] 59 | 60 | def download(self): 61 | pass 62 | 63 | def process(self): 64 | pass 65 | 66 | def custom_process(self): 67 | with open(self.latency_file) as f: 68 | for line in f.readlines(): 69 | 70 | line = line.rstrip() 71 | items = line.split(" ") 72 | speed_id = str(items[0]) 73 | graph_id = str(items[1]) 74 | batch_size = int(items[2]) 75 | cost_time = float(items[3]) 76 | plt_id = int(items[5]) 77 | 78 | if self.model_types and items[4] not in self.model_types: 79 | continue 80 | 81 | if self.platforms and plt_id not in self.platforms: 82 | continue 83 | 84 | if self.train_test_stage and items[6] != self.train_test_stage: 85 | continue 86 | 87 | onnx_file = os.path.join(self.onnx_dir, graph_id) 88 | if os.path.exists(onnx_file): 89 | data_file = os.path.join(self.processed_dir, '{}_{}_data.pt'.format(speed_id, plt_id)) 90 | sf_file = os.path.join(self.processed_dir, '{}_{}_sf.pt'.format(speed_id, plt_id)) 91 | graph_name = "{}_{}_{}".format(graph_id, batch_size, plt_id) 92 | self.latency_ids.append((data_file, sf_file, graph_name, plt_id)) 93 | 94 | if (not self.override_data) and os.path.exists(data_file) and os.path.exists(sf_file): 95 | continue 96 | 97 | if len(self.latency_ids) % 1000 == 0: 98 | print(len(self.latency_ids)) 99 | 100 | try: 101 | GG = onnx.load(onnx_file) 102 | data, sf = get_torch_data(GG, batch_size, cost_time) 103 | 104 | if self.pre_filter is not None and not self.pre_filter(data): 105 | continue 106 | if self.pre_transform is not None: 107 | data = self.pre_transform(data) 108 | 109 | torch.save(data, data_file) 110 | torch.save(sf, sf_file) 111 | except Exception as e: 112 | self.latency_ids.pop() 113 | print("Error", e) 114 | 115 | def len(self): 116 | return len(self.latency_ids) 117 | 118 | def get(self, idx): 119 | data_file, sf_file, graph_name, plt_id = self.latency_ids[idx] 120 | data = torch.load(data_file) 121 | sf = torch.load(sf_file) 122 | return data, sf, graph_name, plt_id -------------------------------------------------------------------------------- /predictor/feature/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelTC/NNLQP/6da64f035baa2cc5ba356071bee28b1c5b71f988/predictor/feature/__init__.py -------------------------------------------------------------------------------- /predictor/feature/feature_utils.py: -------------------------------------------------------------------------------- 1 | # define the op and its attributes to be encoded 2 | OPS = { 3 | "Conv": { 4 | "code": 1, 5 | "attrs": [ 6 | "kernel_shape", 7 | "strides", 8 | "pads", 9 | "dilations", 10 | "group", 11 | "bias", 12 | ], 13 | }, 14 | "Relu": { 15 | "code": 2, 16 | "attrs": [], 17 | }, 18 | "Add": { 19 | "code": 3, 20 | "attrs": [], 21 | }, 22 | "Sigmoid": { 23 | "code": 4, 24 | "attrs": [], 25 | }, 26 | "Reshape": { 27 | "code": 5, 28 | "attrs": [], 29 | }, 30 | "MaxPool": { 31 | "code": 6, 32 | "attrs": [ 33 | "kernel_shape", 34 | "strides", 35 | "pads", 36 | #"dilations", 37 | ], 38 | }, 39 | "Split": { 40 | "code": 7, 41 | "attrs": [], 42 | }, 43 | "GlobalAveragePool": { 44 | "code": 8, 45 | "attrs": [], 46 | }, 47 | "Gemm": { 48 | "code": 9, 49 | "attrs": [ 50 | "bias", 51 | ], 52 | }, 53 | "Transpose": { 54 | "code": 10, 55 | "attrs": [ 56 | "perm", 57 | ] 58 | }, 59 | "Upsample": { 60 | "code": 11, 61 | "attrs": [], 62 | }, 63 | "BatchNormalization": { 64 | "code": 12, 65 | "attrs": [], 66 | }, 67 | "Mul": { 68 | "code": 13, 69 | "attrs": [], 70 | }, 71 | "Concat": { 72 | "code": 14, 73 | "attrs": [], 74 | }, 75 | "Flatten": { 76 | "code": 15, 77 | "attrs": [], 78 | }, 79 | "AveragePool": { 80 | "code": 16, 81 | "attrs": [ 82 | "kernel_shape", 83 | "strides", 84 | "pads", 85 | ], 86 | }, 87 | "Cast": { 88 | "code": 17, 89 | "attrs": [], 90 | }, 91 | "Matmul": { 92 | "code": 18, 93 | "attrs": [], 94 | }, 95 | "ReduceMean": { 96 | "code": 19, 97 | "attrs": [], 98 | }, 99 | "Pow": { 100 | "code": 20, 101 | "attrs": [], 102 | }, 103 | "Slice": { 104 | "code": 21, 105 | "attrs": [], 106 | }, 107 | "Div": { 108 | "code": 22, 109 | "attrs": [], 110 | }, 111 | "Sub": { 112 | "code": 23, 113 | "attrs": [], 114 | }, 115 | "Sqrt": { 116 | "code": 24, 117 | "attrs": [], 118 | }, 119 | "Clip": { 120 | "code": 25, 121 | "attrs": [], 122 | }, 123 | "Softmax": { 124 | "code": 26, 125 | "attrs": [], 126 | }, 127 | "Tanh": { 128 | "code": 27, 129 | "attrs": [], 130 | }, 131 | "ConvTranspose": { 132 | "code": 28, 133 | "attrs": [ 134 | "kernel_shape", 135 | "strides", 136 | "pads", 137 | "dilations", 138 | "group", 139 | "bias", 140 | "output_padding", 141 | ], 142 | }, 143 | } 144 | 145 | # define the value type of attr value, and its feature length 146 | ATTRS = { 147 | "kernel_shape" : ("tuple", 1, 2.53, 56), 148 | "strides" : ("tuple", 1, 1.16, 56), 149 | "pads" : ("tuple", 1, 0.75, 6), 150 | "dilations" : ("tuple", 1, 1.00, 1), 151 | "group" : ("int" , 15.08, 6144), 152 | "bias" : ("bool" , 0.77, 1), 153 | "perm" : ("tuple", 8, 1.78, 4), 154 | "output_padding": ("tuple", 1, 0.00, 1e-5), 155 | } 156 | 157 | # define the fixed length of feature op_code, attrs, output shape 158 | FEATURE_LENGTH = { 159 | "op_code": 32, 160 | "attrs": 8, 161 | "output_shape": 4, 162 | } 163 | -------------------------------------------------------------------------------- /predictor/feature/graph_feature.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import networkx as nx 3 | from .onnx_to_networkx import onnx2nx 4 | from .onnx_shape_infer import custom_shape_infer 5 | from .onnx_flops import calculate_onnx_flops 6 | from .node_feature import extract_node_features 7 | 8 | 9 | def modify_onnx_batch_size(onnx_G, batch_size): 10 | # initializer names, in case of names of input include initializers 11 | init_names = set() 12 | for init in onnx_G.graph.initializer: 13 | init_names.add(init.name) 14 | 15 | def _modify(node, dim_idx, value): 16 | dims = node.type.tensor_type.shape.dim 17 | if len(dims) > dim_idx: 18 | value = value[node.name] if isinstance(value, dict) else value 19 | dims[dim_idx].dim_value = value 20 | 21 | # modify input 22 | for inp in onnx_G.graph.input: 23 | if inp.name in init_names: 24 | continue 25 | _modify(inp, 0, batch_size) 26 | 27 | # modify output 28 | for out in onnx_G.graph.output: 29 | _modify(out, 0, batch_size) 30 | 31 | return 32 | 33 | 34 | def parse_from_onnx(onnx_path, batch_size): 35 | pG = onnx2nx(onnx_path) 36 | nx_G, onnx_G = pG.data, pG.onnx_G 37 | 38 | # first we should change the batch_size of input in ONNX model 39 | modify_onnx_batch_size(onnx_G, batch_size) 40 | status, newG, output_shapes = custom_shape_infer(onnx_G) 41 | # if failed modify the batch to original batch size 42 | assert status is True, "Onnx shape infer error!" 43 | 44 | flops, params, macs, node_flops = calculate_onnx_flops(onnx_G, True) 45 | return nx_G, output_shapes, flops, params, macs, node_flops, newG 46 | 47 | 48 | def extract_graph_feature_from_networkx(nx_G, batch_size, output_shapes, flops, params, macs, undirected=True): 49 | # static features: flops, params, memory_access (GB) + batch_size 50 | static_features = np.array([batch_size, flops / 1e9, params / 1e9, macs / 1e9], dtype="float32") 51 | 52 | # node features 53 | node_features = extract_node_features(nx_G, output_shapes, batch_size) 54 | 55 | # get features conducted by idx 56 | features = [] 57 | name2id = {} 58 | id2name = {} 59 | for idx, node in enumerate(nx.topological_sort(nx_G)): 60 | features.append(node_features[node]) 61 | name2id[node] = idx 62 | id2name[idx] = node 63 | 64 | # get graph adjacent matrix 65 | node_num = nx_G.number_of_nodes() 66 | adjacent = np.zeros((node_num, node_num), dtype="float32") 67 | 68 | for node in nx_G.nodes(): 69 | idx = name2id[node] 70 | for child in nx_G.successors(node): 71 | conn_idx = name2id[child] 72 | adjacent[idx][conn_idx] = 1 73 | if undirected: 74 | adjacent[conn_idx][idx] = 1 75 | 76 | # test connect relationship 77 | # xs, ys = np.where(adjacent > 0) 78 | # for i in range(len(xs)): 79 | # print("Conn:", id2name[xs[i]], id2name[ys[i]]) 80 | 81 | # feature in features may be a tuple (block_adjacent, block_features, block_static_features) 82 | return adjacent, features, static_features 83 | 84 | 85 | def extract_graph_feature(onnx_path, batch_size, return_onnx=False): 86 | nx_G, output_shapes, flops, params, macs, node_flops, onnx_G = parse_from_onnx(onnx_path, batch_size) 87 | adjacent, features, static_features = extract_graph_feature_from_networkx( 88 | nx_G, batch_size, output_shapes, flops, params, macs 89 | ) 90 | 91 | if return_onnx: 92 | return adjacent, np.array(features), static_features, onnx_G 93 | else: 94 | return adjacent, np.array(features), static_features 95 | -------------------------------------------------------------------------------- /predictor/feature/node_feature.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from .feature_utils import OPS, ATTRS, FEATURE_LENGTH 3 | 4 | 5 | # int -> one hot embedding 6 | def embed_op_code(op_type): 7 | length = FEATURE_LENGTH["op_code"] 8 | if op_type not in OPS: 9 | return np.zeros(length, dtype="float32") 10 | op_code = OPS[op_type]["code"] - 1 11 | if op_code >= length: 12 | raise Exception("op code of {}: {} greater than one-hot length {}!".format( 13 | op_type, op_code, length)) 14 | return np.eye(length, dtype="float32")[op_code] 15 | 16 | 17 | class EmbedValue: 18 | # int value embedding 19 | @staticmethod 20 | def embed_int(x, center=0, scale=1): 21 | x = np.array([int(x)], dtype="float32") 22 | return (x - center) / np.abs(scale) 23 | 24 | # float value embedding 25 | @staticmethod 26 | def embed_float(x, center=0, scale=1): 27 | x = np.array([float(x)], dtype="float32") 28 | return (x - center) / np.abs(scale) 29 | 30 | # bool value embedding 31 | @staticmethod 32 | def embed_bool(x, center=0, scale=1): 33 | x = np.array([int(bool(x))], dtype="float32") 34 | return (x - center) / np.abs(scale) 35 | 36 | # tuple value embedding 37 | @staticmethod 38 | def embed_tuple(x, length, center=0, scale=1): 39 | x = np.array(x, dtype="float32").reshape(-1) 40 | if x.size > length: 41 | x = x[:length] 42 | if x.size < length: 43 | x = np.concatenate([x, np.zeros(length - x.size, dtype="float32")]) 44 | if not isinstance(center, list): 45 | center = [center] * x.size 46 | if not isinstance(scale, list): 47 | scale = [scale] * x.size 48 | center = np.array(center, dtype="float32") 49 | scale = np.array(scale, dtype="float32") 50 | return (x - center) / np.abs(scale) 51 | 52 | 53 | # attrs embedding 54 | def embed_attrs(op_type, attrs): 55 | length = FEATURE_LENGTH["attrs"] 56 | if op_type not in OPS: 57 | return np.zeros(length, dtype="float32") 58 | 59 | feats = [] 60 | for name in OPS[op_type]["attrs"]: 61 | assert name in attrs, "attr {} for {} need to be encoded but not included!".format(name, op_type) 62 | assert name in ATTRS, "attr {} for {} does not defined in ATTRS!".format(name, op_type) 63 | 64 | attr_value = attrs[name] 65 | attr_def = ATTRS[name] 66 | feat = getattr(EmbedValue, "embed_" + attr_def[0])(attr_value, *attr_def[1:]) 67 | feats.append(feat) 68 | 69 | # concat attr features 70 | feats = np.concatenate(feats) if len(feats) > 0 else np.zeros(length, dtype="float32") 71 | feat_len = feats.size 72 | if feat_len > length: 73 | raise Exception("tuple length {} is grater than the embed length {}".format( 74 | feat_len, length)) 75 | if feat_len < length: 76 | feats = np.concatenate([feats, np.zeros(length - feat_len, dtype="float32")]) 77 | return feats 78 | 79 | 80 | # networkx_G -> op_code_embeddings & attrs_embeddings 81 | # output_shapes -> output_shape_embeddings 82 | def extract_node_features(networkx_G, output_shapes, batch_size): 83 | embeddings = {} 84 | 85 | for node in networkx_G.nodes.data(): 86 | attrs = node[1]["attr"].attributes 87 | node_name = node[0] 88 | op_type = attrs["type"] 89 | 90 | # one hot op_code embedding 91 | op_code_embedding = embed_op_code(op_type) 92 | 93 | # fixed length embedding for attrs, need normalize? 94 | attrs_embedding = embed_attrs(op_type, attrs) 95 | 96 | # fixed length embedding for output shape, need normalize? 97 | assert node_name in output_shapes, "could not find output shape for node {}".format(node_name) 98 | output_shape_embedding = EmbedValue.embed_tuple( 99 | output_shapes[node_name], 100 | FEATURE_LENGTH["output_shape"], 101 | [12.25, 286.96, 32.62, 31.80], 102 | [1024, 2496, 256, 256], 103 | #[11.02, 228.78, 48.94, 63.10], 104 | #[28.38, 642.21, 78.54, 93.87], 105 | ) 106 | 107 | # concat to the final node feature 108 | embeddings[node_name] = np.concatenate([ 109 | op_code_embedding, 110 | attrs_embedding, 111 | output_shape_embedding, 112 | ]) 113 | # print(op_type, len(embeddings[node_name]), embeddings[node_name]) 114 | 115 | return embeddings 116 | -------------------------------------------------------------------------------- /predictor/feature/onnx_flops.py: -------------------------------------------------------------------------------- 1 | import onnx 2 | import logging 3 | import numpy as np 4 | from .onnx_shape_infer import custom_shape_infer, onnx_node_attributes_to_dict 5 | 6 | 7 | def flops_conv(node, attrs, maps): 8 | assert(node.op_type == 'Conv') 9 | input_shape = maps[node.input[1]] 10 | output_shape = maps[node.output[0]] 11 | 12 | Co_Ho_Wo = np.prod(output_shape[1:]) 13 | ci_k2 = np.prod(input_shape[1:]) # cin // group * k * k 14 | # group = attrs['group'] if 'group' in attrs else 1 15 | bias = 1 if len(node.input) == 3 else 0 16 | 17 | # flops: Co * Ho * Wo * (2 * Ci * k * k / group - 1 + bias) 18 | # params: Co * (Ci * k * k + bias) 19 | flops = Co_Ho_Wo * (2 * ci_k2 - 1 + bias) 20 | params = input_shape[0] * (np.prod(input_shape[1:]) + bias) 21 | return flops, params 22 | 23 | 24 | def flops_gemm(node, attrs, maps): 25 | assert(node.op_type == 'Gemm') 26 | input_shape = maps[node.input[1]] 27 | if 'transB' in attrs and attrs['transB'] != 0: 28 | Co, Ci = input_shape[-2:] 29 | else: 30 | Ci, Co = input_shape[-2:] 31 | bias = 1 if len(node.input) == 3 else 0 32 | 33 | # flops: Co * (2 * Ci - 1 +bias) 34 | # params: Co * (Ci + bias) 35 | flops = Co * (2 * Ci - 1 + bias) 36 | params = Co * (Ci + bias) 37 | return flops, params 38 | 39 | 40 | def flops_matmal(node, attrs, maps): 41 | assert(node.op_type == 'MatMul') 42 | # (N x K) * (K x M) = (N x M) 43 | N, K = maps[node.input[0]][-2:] 44 | N, M = maps[node.output[0]][-2:] 45 | 46 | # flops: N * M * (2 * K - 1) 47 | # params: 0, because there not existed fixed weight 48 | flops = N * M * (2 * K - 1) 49 | params = 0 50 | return flops, params 51 | 52 | 53 | def flops_avgpool(node, attrs, maps): 54 | assert node.op_type == 'GlobalAveragePool' or node.op_type == 'AveragePool' 55 | if node.op_type == 'GlobalAveragePool': 56 | kernel_shape = maps[node.input[0]][3] / maps[node.output[0]][3] 57 | else: 58 | kernel_shape = attrs['kernel_shape'] 59 | output = maps[node.output[0]] 60 | flops = np.prod(output[1:]) * np.prod(kernel_shape) 61 | params = 0 62 | return flops, params 63 | 64 | 65 | def flops_zero(node, attrs, maps): 66 | return 0, 0 67 | 68 | 69 | def flops_add(node, attrs, maps): 70 | flops = np.prod(maps[node.output[0]]) 71 | return flops, 0 72 | 73 | 74 | def flops_maxpool(node, attrs, maps): 75 | return 0, 0 76 | 77 | 78 | def flops_softmax(node, attrs, maps): 79 | x = np.array(maps[node.input[0]]) 80 | nfeatures = x.size // x[0] 81 | 82 | total_exp = nfeatures 83 | total_add = nfeatures - 1 84 | total_div = nfeatures 85 | flops = 1 * (total_exp + total_add + total_div) 86 | return flops, 0 87 | 88 | 89 | def flops_bn(node, attrs, maps): 90 | x = maps[node.input[0]] 91 | if len(node.input) > 1: 92 | flops = 4 * np.prod(x) 93 | return flops, 4*x[1] 94 | else: 95 | flops = 2 * np.prod(x) 96 | return flops, 2*x[1] 97 | 98 | 99 | def flops_upsample(node, attrs, maps): 100 | if attrs['mode'] not in (b"nearest", b"linear", b"bilinear", b"bicubic", b"trilinear"): 101 | logging.warning("mode %s is not implemented yet, take it a zero op" % attrs['mode']) 102 | return flops_zero(node, attrs, maps) 103 | 104 | if attrs['mode'] == b"nearest": 105 | return flops_zero(node, attrs, maps) 106 | 107 | y = maps[node.output[0]] 108 | if attrs['mode'] == b"linear": 109 | flops = np.prod(y[1:]) * 5 # 2 muls + 3 add 110 | elif attrs['mode'] == b"bilinear": 111 | # https://en.wikipedia.org/wiki/Bilinear_interpolation 112 | total_ops = np.prod(y[1:]) * 11 # 6 muls + 5 adds 113 | elif attrs['mode'] == b"bicubic": 114 | # https://en.wikipedia.org/wiki/Bicubic_interpolation 115 | # Product matrix [4x4] x [4x4] x [4x4] 116 | ops_solve_A = 224 # 128 muls + 96 adds 117 | ops_solve_p = 35 # 16 muls + 12 adds + 4 muls + 3 adds 118 | flops = np.prod(y[1:]) * (ops_solve_A + ops_solve_p) 119 | elif attrs['mode'] == b"trilinear": 120 | # https://en.wikipedia.org/wiki/Trilinear_interpolation 121 | # can viewed as 2 bilinear + 1 linear 122 | flops = np.prod(y[1:]) * (13 * 2 + 5) 123 | return flops, 0 124 | 125 | 126 | def calculate_onnx_flops(onnx_G, each_node=False): 127 | flops = 0.0 128 | params = 0.0 129 | memory_access = 0.0 130 | node_flops = {} 131 | 132 | status, G, output_shapes = custom_shape_infer(onnx_G) 133 | 134 | flops_funcs = { 135 | "Conv": flops_conv, 136 | "Relu": flops_zero, 137 | "Add": flops_add, 138 | "Sigmoid": flops_add, 139 | "Reshape": flops_zero, 140 | "MaxPool": flops_zero, 141 | "Split": flops_zero, 142 | "GlobalAveragePool": flops_avgpool, 143 | "Gemm": flops_gemm, 144 | "Transpose": flops_zero, 145 | "Upsample": flops_upsample, 146 | "BatchNormalization": flops_bn, 147 | "Mul": flops_add, 148 | "Concat": flops_zero, 149 | "Flatten": flops_zero, 150 | "AveragePool": flops_avgpool, 151 | "Cast": flops_zero, 152 | "Matmul": flops_matmal, 153 | "ReduceMean": flops_add, 154 | "Pow": flops_add, 155 | "Slice": flops_zero, 156 | "Div": flops_add, 157 | "Sub": flops_add, 158 | "Sqrt": flops_add, 159 | "Clip": flops_zero, 160 | "Softmax": flops_softmax, 161 | "Tanh": flops_add, 162 | "ConvTranspose": flops_conv 163 | } 164 | 165 | # caculate flops node by node 166 | if status is True: 167 | for node in G.graph.node: 168 | cur_access = 0 169 | cur_params = 0 170 | cur_flops = 0 171 | 172 | for output in node.output: 173 | # memory_access = feature_map_sizes + params 174 | cur_access += np.prod(output_shapes[output]) 175 | 176 | if node.op_type in flops_funcs: 177 | attrs = onnx_node_attributes_to_dict(node.attribute) 178 | cur_flops, cur_params = flops_funcs[node.op_type](node, attrs, output_shapes) 179 | flops += cur_flops 180 | params += cur_params 181 | cur_access += cur_params 182 | memory_access += cur_access 183 | 184 | for output in node.output: 185 | node_flops[output] = (cur_flops, cur_params, cur_access) 186 | 187 | if each_node is True: 188 | return flops, params, memory_access, node_flops 189 | else: 190 | return flops, params, memory_access 191 | 192 | 193 | if __name__ == '__main__': 194 | import sys 195 | onnx_path = sys.argv[1] 196 | onnx_G = onnx.load(onnx_path) 197 | print(calculate_onnx_flops(onnx_G, each_node=False)) 198 | -------------------------------------------------------------------------------- /predictor/feature/onnx_shape_infer.py: -------------------------------------------------------------------------------- 1 | import onnx 2 | from onnx.shape_inference import infer_shapes 3 | 4 | 5 | def onnx_node_attributes_to_dict(args): 6 | def onnx_attribute_to_dict(onnx_attr): 7 | if onnx_attr.HasField('t'): 8 | return onnx.numpy_helper.to_array(getattr(onnx_attr, 't')) 9 | 10 | for attr_type in ['f', 'i', 's']: 11 | if onnx_attr.HasField(attr_type): 12 | return getattr(onnx_attr, attr_type) 13 | 14 | for attr_type in ['floats', 'ints', 'strings']: 15 | if getattr(onnx_attr, attr_type): 16 | return list(getattr(onnx_attr, attr_type)) 17 | return {arg.name: onnx_attribute_to_dict(arg) for arg in args} 18 | 19 | 20 | def get_1D_const_node(name, data_type, vals): 21 | node = onnx.helper.make_node( 22 | 'Constant', 23 | inputs=[], 24 | outputs=[name], 25 | value=onnx.helper.make_tensor( 26 | name=name + '_value', 27 | data_type=data_type, 28 | dims=(len(vals), ), 29 | vals=vals, 30 | ) 31 | ) 32 | return node 33 | 34 | 35 | def custom_shape_infer(onnx_G): 36 | 37 | vals = [x for x in onnx_G.graph.value_info] 38 | for x in vals: 39 | onnx_G.graph.value_info.remove(x) 40 | 41 | def parse_dim(val): 42 | shape = list([x.dim_value for x in val.type.tensor_type.shape.dim]) 43 | shape = shape if len(shape) > 0 else None 44 | return shape 45 | 46 | output_shapes = {} 47 | init_dtypes = {} 48 | placehoders = [] 49 | 50 | # parse initializer 51 | for init in onnx_G.graph.initializer: 52 | output_shapes[init.name] = list(init.dims) 53 | init_dtypes[init.name] = init.data_type 54 | 55 | # parse input 56 | for val in onnx_G.graph.input: 57 | output_shapes[val.name] = parse_dim(val) 58 | dtype = val.type.tensor_type.elem_type 59 | 60 | # init dtype may != input dtype, correct it in case of shape inference error 61 | if val.name in init_dtypes and dtype != init_dtypes[val.name]: 62 | print(" - Warning: {}: init type:{} vs input type:{}".format(val.name, init_dtypes[val.name], dtype)) 63 | val.type.tensor_type.elem_type = init_dtypes[val.name] 64 | 65 | # placehoders for the whole graph 66 | if val.name not in init_dtypes: 67 | placehoders.append(val) 68 | 69 | # parse output 70 | output_vals = {} 71 | for val in onnx_G.graph.output: 72 | output_shapes[val.name] = parse_dim(val) 73 | output_vals[val.name] = val 74 | 75 | # parse node, handle error conditions 76 | vers = ['1.7.0', '1.8.1'] 77 | if onnx.__version__ not in vers: 78 | print(" - Warning: onnx version should in {}, but with {}, infer shape may cause error" \ 79 | .format(vers, onnx.__version__)) 80 | 81 | # infer the shapes by onnx api 82 | if onnx.__version__ == '1.8.1': 83 | G = infer_shapes(onnx_G, strict_mode=True) 84 | # args strict_mode is not introcuded for onnx == 1.7.0 85 | else: 86 | G = infer_shapes(onnx_G) 87 | 88 | for val in G.graph.value_info: 89 | output_shapes[val.name] = parse_dim(val) 90 | 91 | miss_ops = {} 92 | for node in G.graph.node: 93 | 94 | # constant output could be lost when it is =[] 95 | if node.op_type == "Constant": 96 | out = node.output[0] 97 | if out not in output_shapes or output_shapes[out] is None: 98 | for attr in node.attribute: 99 | if attr.name == "value": 100 | output_shapes[out] = list(attr.t.dims) 101 | 102 | # check if the shapes inferred are complete 103 | for output in node.output: 104 | if output not in output_shapes or output_shapes[output] is None: 105 | if node.op_type not in miss_ops: 106 | miss_ops[node.op_type] = (node.name, node.output[0]) 107 | 108 | for k, v in miss_ops.items(): 109 | print(" - Warning: Miss shape infer, op=[{}], name=[{}], output=[{}]!".format(k, v[0], v[1])) 110 | return False if miss_ops else True, G, output_shapes 111 | 112 | 113 | if __name__ == '__main__': 114 | import sys 115 | path = sys.argv[1] 116 | onnx_G = onnx.load(path) 117 | status, newG, output_shapes = custom_shape_infer(onnx_G) 118 | print(status) 119 | -------------------------------------------------------------------------------- /predictor/feature/onnx_to_networkx.py: -------------------------------------------------------------------------------- 1 | import os 2 | import onnx 3 | import copy 4 | import logging 5 | import networkx as nx 6 | 7 | from onnx.numpy_helper import to_array 8 | from .op_attribute import * 9 | 10 | 11 | # construct networkx graph from onnx file 12 | def onnx2nx(onnx_path): 13 | global WARNINGS 14 | WARNINGS.clear() 15 | 16 | if isinstance(onnx_path, str): 17 | assert(os.path.exists(onnx_path)) 18 | switch_print("Read Onnx: {}".format(onnx_path)) 19 | onnx_G = onnx.load(onnx_path) 20 | else: 21 | assert isinstance(onnx_path, onnx.onnx_ml_pb2.ModelProto), "onnx2nx input should be str or ModelProto" 22 | onnx_G = onnx_path 23 | 24 | nx_G = nx.DiGraph() 25 | switcher = Switcher(onnx_G) 26 | 27 | all_G_edges = [] 28 | all_G_nodes = [] 29 | onnx_nodes = onnx_G.graph.node 30 | 31 | for node in onnx_nodes: 32 | G_nodes, G_edges = switcher.parse_node(node) 33 | 34 | # collect the node and edges 35 | if G_nodes is not None: 36 | all_G_nodes.extend(G_nodes) 37 | all_G_edges.extend(G_edges) 38 | 39 | # add the node & edges to networkx graph 40 | nx_G.add_nodes_from(all_G_nodes) 41 | nx_G.add_edges_from(all_G_edges) 42 | 43 | input_sizes = {} 44 | output_sizes = switcher.output_sizes 45 | 46 | # for input nodes 47 | zero_indegree = [v for v, d in nx_G.in_degree() if d == 0] 48 | for node in zero_indegree: 49 | if nx_G.out_degree()[node] > 0 and node in switcher.input_sizes: 50 | nx_G.add_nodes_from([(node, {'attr': AttrInput(name=node)})]) 51 | input_sizes[node] = switcher.input_sizes[node] 52 | else: 53 | # zero input, zero output, delete the node 54 | nx_G.remove_node(node) 55 | 56 | if len(WARNINGS) > 0: 57 | path_name = onnx_path.split('/')[-1] if isinstance(onnx_path, str) else "ModelProto" 58 | switch_print("[{}] onnx -> networkx warnings: ".format(path_name)) 59 | for w in WARNINGS: 60 | switch_print(" -- {}".format(w)) 61 | 62 | return PGraph(nx_G, input_sizes, output_sizes, switcher.opsets, onnx_G) 63 | 64 | 65 | class PGraph: 66 | def __init__(self, G, input_sizes, output_sizes, opsets, onnx_G): 67 | self.data = G 68 | self.input_sizes = input_sizes 69 | self.output_sizes = output_sizes 70 | self.opsets = opsets 71 | self.onnx_G = onnx_G 72 | 73 | 74 | def switch_print(info): 75 | logger = logging.getLogger('GPDB') 76 | print(info) 77 | logger.info(info) 78 | 79 | 80 | class Switcher: 81 | def __init__(self, onnx_G): 82 | self.onnx_G = onnx_G 83 | self.input_sizes = {} 84 | self.output_sizes = {} 85 | 86 | # opset info 87 | opset_cnt = len(self.onnx_G.opset_import) 88 | if opset_cnt <= 0: 89 | self.opsets = [9] 90 | else: 91 | self.opsets = [self.onnx_G.opset_import[x].version for x in range(opset_cnt)] 92 | 93 | # input and params info 94 | for inp in self.onnx_G.graph.input: 95 | self.input_sizes[inp.name] = tuple([x.dim_value for x in inp.type.tensor_type.shape.dim]) 96 | # some weight is not regarded as input 97 | for init in self.onnx_G.graph.initializer: 98 | self.input_sizes[init.name] = tuple(init.dims) 99 | for out in self.onnx_G.graph.output: 100 | self.output_sizes[out.name] = tuple([x.dim_value for x in out.type.tensor_type.shape.dim]) 101 | 102 | def parse_node(self, node): 103 | try: 104 | parse_func = getattr(self, "parse" + node.op_type) 105 | except Exception as e: 106 | raise Exception("{}, Operator [{}] Not Supported!".format(e, node.op_type)) 107 | return parse_func(node) 108 | 109 | # parse onnx tensor value 110 | def parse_tensor(self, tensor, dims_only=False): 111 | dims = tuple(tensor.dims) 112 | if dims_only: 113 | return dims 114 | 115 | val = to_array(tensor) 116 | if len(dims) > 0: 117 | val = tuple(val) 118 | return val 119 | 120 | # parse onnx attribute of op node 121 | def parse_attrs(self, node_attrs, dims_only=False): 122 | attrs = {'opsets': self.opsets} 123 | for attr in node_attrs: 124 | if attr.type == onnx.AttributeProto.AttributeType.INTS: 125 | attrs[attr.name] = tuple(attr.ints) 126 | elif attr.type == onnx.AttributeProto.AttributeType.INT: 127 | attrs[attr.name] = attr.i 128 | elif attr.type == onnx.AttributeProto.AttributeType.FLOATS: 129 | attrs[attr.name] = tuple(attr.floats) 130 | elif attr.type == onnx.AttributeProto.AttributeType.FLOAT: 131 | attrs[attr.name] = attr.f 132 | elif attr.type == onnx.AttributeProto.AttributeType.TENSOR: 133 | attrs[attr.name] = self.parse_tensor(attr.t, dims_only) 134 | elif attr.type == onnx.AttributeProto.AttributeType.STRING: 135 | attrs[attr.name] = str(attr.s) 136 | elif attr.type == onnx.AttributeProto.AttributeType.STRINGS: 137 | attrs[attr.name] = tuple([str(x) for x in attr.strings]) 138 | else: 139 | raise Exception("ATTR Type [{}] Not Supported!".format(attr.type)) 140 | return attrs 141 | 142 | # Construct the node and edges of networkx graph 143 | def get_networkx_node_edges(self, node, G_attr, conn_input_ids, conn_output_ids): 144 | G_nodes = [] 145 | if len(conn_output_ids) == 1: 146 | idx = conn_output_ids[0] 147 | G_nodes.append((node.output[idx], {'attr': G_attr})) 148 | 149 | else: 150 | # multi output 151 | for idx in conn_output_ids: 152 | G_attr_cur = copy.deepcopy(G_attr) 153 | G_attr_cur.attributes['output_num'] = len(conn_output_ids) 154 | G_attr_cur.attributes['output_idx'] = idx 155 | G_nodes.append((node.output[idx], {'attr': G_attr_cur})) 156 | 157 | G_edges = [] 158 | for idx in conn_input_ids: 159 | for idy in conn_output_ids: 160 | G_edges.append((node.input[idx], node.output[idy])) 161 | 162 | return G_nodes, G_edges 163 | 164 | # assert the number of input and output 165 | def assert_node_input_output(self, node, input_nums, output_nums, 166 | input_low_bound=None, output_low_bound=None): 167 | in_num, out_num = len(node.input), len(node.output) 168 | 169 | if input_nums is not None and in_num not in input_nums: 170 | raise Exception("Input num of <{}> = {}, which are not in {}!". \ 171 | format(node.op_type, in_num, input_nums)) 172 | 173 | if output_nums is not None and out_num not in output_nums: 174 | raise Exception("Output num of <{}> = {}, which are not in {}!". \ 175 | format(node.op_type, out_num, output_nums)) 176 | 177 | if input_low_bound is not None and in_num < input_low_bound: 178 | raise Exception("Input num of <{}> = {}, which are not >= {}!". \ 179 | format(node.op_type, in_num, input_low_bound)) 180 | 181 | if output_low_bound is not None and out_num < output_low_bound: 182 | raise Exception("Output num of <{}> = {}, which are not >= {}!". \ 183 | format(node.op_type, out_num, output_low_bound)) 184 | 185 | # parse the general node without additional attributes 186 | def parse_general_node(self, node, AttrOp, conn_input_ids, conn_output_ids): 187 | attrs = self.parse_attrs(node.attribute) 188 | G_attr = AttrOp(name = node.output[0], **attrs) 189 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, conn_output_ids) 190 | 191 | # ------------------------------------------------------------------------------------ 192 | # For each onnx op type 193 | # ------------------------------------------------------------------------------------ 194 | 195 | def parseAbs(self, node): 196 | self.assert_node_input_output(node, [1], [1]) 197 | return self.parse_general_node(node, AttrAbs, [0], [0]) 198 | 199 | def parseAcos(self, node): 200 | self.assert_node_input_output(node, [1], [1]) 201 | return self.parse_general_node(node, AttrAcos, [0], [0]) 202 | 203 | def parseAcosh(self, node): 204 | self.assert_node_input_output(node, [1], [1]) 205 | return self.parse_general_node(node, AttrAcosh, [0], [0]) 206 | 207 | def parseAdd(self, node): 208 | self.assert_node_input_output(node, [2], [1]) 209 | return self.parse_general_node(node, AttrAdd, [0, 1], [0]) 210 | 211 | def parseAnd(self, node): 212 | self.assert_node_input_output(node, [2], [1]) 213 | return self.parse_general_node(node, AttrAnd, [0, 1], [0]) 214 | 215 | def parseArgMax(self, node): 216 | self.assert_node_input_output(node, [1], [1]) 217 | return self.parse_general_node(node, AttrArgMax, [0], [0]) 218 | 219 | def parseArgMin(self, node): 220 | self.assert_node_input_output(node, [1], [1]) 221 | return self.parse_general_node(node, AttrArgMin, [0], [0]) 222 | 223 | def parseAsin(self, node): 224 | self.assert_node_input_output(node, [1], [1]) 225 | return self.parse_general_node(node, AttrAsin, [0], [0]) 226 | 227 | def parseAsinh(self, node): 228 | self.assert_node_input_output(node, [1], [1]) 229 | return self.parse_general_node(node, AttrAsinh, [0], [0]) 230 | 231 | def parseAtan(self, node): 232 | self.assert_node_input_output(node, [1], [1]) 233 | return self.parse_general_node(node, AttrAtan, [0], [0]) 234 | 235 | def parseAtanh(self, node): 236 | self.assert_node_input_output(node, [1], [1]) 237 | return self.parse_general_node(node, AttrAtanh, [0], [0]) 238 | 239 | def parseAveragePool(self, node): 240 | self.assert_node_input_output(node, [1], [1]) 241 | return self.parse_general_node(node, AttrAveragePool, [0], [0]) 242 | 243 | def parseBatchNormalization(self, node): 244 | self.assert_node_input_output(node, [5], [1, 2, 3, 4, 5]) 245 | attrs = self.parse_attrs(node.attribute) 246 | 247 | # Get the size of scale, B, mean and var 248 | # without tuple: Error: can't pickle repeated message fields, convert to list first 249 | conn_input_ids = [0, 1, 2, 3, 4] 250 | scale_size = B_size = mean_size = var_size = -1 251 | if node.input[1] in self.input_sizes: 252 | scale_size = self.input_sizes[node.input[1]] 253 | conn_input_ids.remove(1) 254 | if node.input[2] in self.input_sizes: 255 | B_size = self.input_sizes[node.input[2]] 256 | conn_input_ids.remove(2) 257 | if node.input[3] in self.input_sizes: 258 | mean_size = self.input_sizes[node.input[3]] 259 | conn_input_ids.remove(3) 260 | if node.input[4] in self.input_sizes: 261 | var_size = self.input_sizes[node.input[4]] 262 | conn_input_ids.remove(4) 263 | 264 | G_attr = AttrBatchNormalization(scale_size, B_size, mean_size, var_size, 265 | name = node.output[0], **attrs) 266 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, range(len(node.output))) 267 | 268 | def parseBitShift(self, node): 269 | self.assert_node_input_output(node, [2], [1]) 270 | return self.parse_general_node(node, AttrBitShift, [0, 1], [0]) 271 | 272 | def parseCast(self, node): 273 | self.assert_node_input_output(node, [1], [1]) 274 | return self.parse_general_node(node, AttrCast, [0], [0]) 275 | 276 | def parseCeil(self, node): 277 | self.assert_node_input_output(node, [1], [1]) 278 | return self.parse_general_node(node, AttrCeil, [0], [0]) 279 | 280 | def parseCelu(self, node): 281 | self.assert_node_input_output(node, [1], [1]) 282 | return self.parse_general_node(node, AttrCelu, [0], [0]) 283 | 284 | def parseClip(self, node): 285 | self.assert_node_input_output(node, [1, 2, 3], [1]) 286 | return self.parse_general_node(node, AttrClip, range(len(node.input)), [0]) 287 | 288 | def parseCompress(self, node): 289 | self.assert_node_input_output(node, [2], [1]) 290 | return self.parse_general_node(node, AttrCompress, [0, 1], [0]) 291 | 292 | def parseConcat(self, node): 293 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 294 | return self.parse_general_node(node, AttrConcat, range(len(node.input)), [0]) 295 | 296 | def parseConcatFromSequence(self, node): 297 | self.assert_node_input_output(node, [1], [1]) 298 | return self.parse_general_node(node, AttrConcat, [0], [0]) 299 | 300 | def parseConstant(self, node): 301 | self.assert_node_input_output(node, [0], [1]) 302 | attrs = self.parse_attrs(node.attribute, dims_only=True) 303 | 304 | # Constant node could be weight param of conv node 305 | self.input_sizes[node.output[0]] = attrs['value'] 306 | 307 | G_attr = AttrConstant(name = node.output[0], **attrs) 308 | return self.get_networkx_node_edges(node, G_attr, [], [0]) 309 | 310 | def parseConstantOfShape(self, node): 311 | self.assert_node_input_output(node, [1], [1]) 312 | return self.parse_general_node(node, AttrConstantOfShape, [0], [0]) 313 | 314 | def parseConv(self, node): 315 | self.assert_node_input_output(node, [2, 3], [1]) 316 | attrs = self.parse_attrs(node.attribute) 317 | 318 | # Get input channel and output channel size 319 | conn_input_ids = [0] 320 | if node.input[1] in self.input_sizes: 321 | # param 322 | output_channel, input_channel = self.input_sizes[node.input[1]][:2] 323 | if 'kernel_shape' not in attrs: 324 | attrs['kernel_shape'] = self.input_sizes[node.input[1]][-2:] 325 | else: 326 | # weight as node 327 | output_channel, input_channel = -1, -1 328 | if 'kernel_shape' not in attrs: 329 | attrs['kernel_shape'] = -1 330 | conn_input_ids.append(1) 331 | bias = True if len(node.input) == 3 else False 332 | 333 | G_attr = AttrConv(input_channel, output_channel, bias, name = node.output[0], **attrs) 334 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 335 | 336 | def parseConvInteger(self, node): 337 | self.assert_node_input_output(node, [2, 3, 4], [1]) 338 | attrs = self.parse_attrs(node.attribute) 339 | 340 | # Get input channel and output channel size 341 | conn_input_ids = list(range(node.input)) 342 | if node.input[1] in self.input_sizes: 343 | # param 344 | output_channel, input_channel = self.input_sizes[node.input[1]][:2] 345 | if 'kernel_shape' not in attrs: 346 | attrs['kernel_shape'] = self.input_sizes[node.input[1]][-2:] 347 | conn_input_ids.remove(1) 348 | else: 349 | # weight as node 350 | output_channel, input_channel = -1, -1 351 | if 'kernel_shape' not in attrs: 352 | attrs['kernel_shape'] = -1 353 | 354 | G_attr = AttrConvInteger(input_channel, output_channel, name = node.output[0], **attrs) 355 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 356 | 357 | def parseConvTranspose(self, node): 358 | self.assert_node_input_output(node, [2, 3], [1]) 359 | attrs = self.parse_attrs(node.attribute) 360 | 361 | # Get input channel and output channel size 362 | conn_input_ids = [0] 363 | if node.input[1] in self.input_sizes: 364 | # param 365 | output_channel, input_channel = self.input_sizes[node.input[1]][:2] 366 | if 'kernel_shape' not in attrs: 367 | attrs['kernel_shape'] = self.input_sizes[node.input[1]][-2:] 368 | else: 369 | # weight as node 370 | output_channel, input_channel = -1, -1 371 | if 'kernel_shape' not in attrs: 372 | attrs['kernel_shape'] = -1 373 | conn_input_ids.append(1) 374 | bias = True if len(node.input) == 3 else False 375 | 376 | G_attr = AttrConvTranspose(input_channel, output_channel, bias, name = node.output[0], **attrs) 377 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 378 | 379 | def parseCos(self, node): 380 | self.assert_node_input_output(node, [1], [1]) 381 | return self.parse_general_node(node, AttrCos, [0], [0]) 382 | 383 | def parseCosh(self, node): 384 | self.assert_node_input_output(node, [1], [1]) 385 | return self.parse_general_node(node, AttrCosh, [0], [0]) 386 | 387 | def parseCumSum(self, node): 388 | self.assert_node_input_output(node, [2], [1]) 389 | return self.parse_general_node(node, AttrCumSum, [0, 1], [0]) 390 | 391 | def parseDepthToSpace(self, node): 392 | self.assert_node_input_output(node, [1], [1]) 393 | return self.parse_general_node(node, AttrDepthToSpace, [0], [0]) 394 | 395 | def parseDequantizeLinear(self, node): 396 | self.assert_node_input_output(node, [2, 3], [1]) 397 | return self.parse_general_node(node, AttrDequantizeLinear, range(len(node.input)), [0]) 398 | 399 | def parseDet(self, node): 400 | self.assert_node_input_output(node, [1], [1]) 401 | return self.parse_general_node(node, AttrDet, [0], [0]) 402 | 403 | def parseDiv(self, node): 404 | self.assert_node_input_output(node, [2], [1]) 405 | return self.parse_general_node(node, AttrDiv, [0, 1], [0]) 406 | 407 | def parseDropout(self, node): 408 | self.assert_node_input_output(node, [1, 2, 3], [1, 2]) 409 | return self.parse_general_node(node, AttrDropout, range(len(node.input)), range(len(node.output))) 410 | 411 | def parseDynamicQuantizeLinear(self, node): 412 | self.assert_node_input_output(node, [1], [3]) 413 | return self.parse_general_node(node, AttrDynamicQuantizeLinear, [0], [0, 1, 2]) 414 | 415 | def parseEinsum(self, node): 416 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 417 | return self.parse_general_node(node, AttrEinsum, range(len(node.input)), [0]) 418 | 419 | def parseElu(self, node): 420 | self.assert_node_input_output(node, [1], [1]) 421 | return self.parse_general_node(node, AttrElu, [0], [0]) 422 | 423 | def parseEqual(self, node): 424 | self.assert_node_input_output(node, [2], [1]) 425 | return self.parse_general_node(node, AttrEqual, [0, 1], [0]) 426 | 427 | def parseErf(self, node): 428 | self.assert_node_input_output(node, [1], [1]) 429 | return self.parse_general_node(node, AttrErf, [0], [0]) 430 | 431 | def parseExp(self, node): 432 | self.assert_node_input_output(node, [1], [1]) 433 | return self.parse_general_node(node, AttrExp, [0], [0]) 434 | 435 | def parseExpand(self, node): 436 | self.assert_node_input_output(node, [2], [1]) 437 | return self.parse_general_node(node, AttrExpand, [0, 1], [0]) 438 | 439 | def parseEyeLike(self, node): 440 | self.assert_node_input_output(node, [1], [1]) 441 | return self.parse_general_node(node, AttrEyeLike, [0], [0]) 442 | 443 | def parseFlatten(self, node): 444 | self.assert_node_input_output(node, [1], [1]) 445 | return self.parse_general_node(node, AttrFlatten, [0], [0]) 446 | 447 | def parseFloor(self, node): 448 | self.assert_node_input_output(node, [1], [1]) 449 | return self.parse_general_node(node, AttrFloor, [0], [0]) 450 | 451 | def parseGRU(self, node): 452 | self.assert_node_input_output(node, [3, 4, 5, 6], [0, 1, 2]) 453 | return self.parse_general_node(node, AttrGRU, range(len(node.input)), range(len(node.output))) 454 | 455 | def parseGather(self, node): 456 | self.assert_node_input_output(node, [2], [1]) 457 | return self.parse_general_node(node, AttrGather, [0, 1], [0]) 458 | 459 | def parseGatherElements(self, node): 460 | self.assert_node_input_output(node, [2], [1]) 461 | return self.parse_general_node(node, AttrGatherElements, [0, 1], [0]) 462 | 463 | def parseGatherND(self, node): 464 | self.assert_node_input_output(node, [2], [1]) 465 | return self.parse_general_node(node, AttrGatherND, [0, 1], [0]) 466 | 467 | def parseGemm(self, node): 468 | self.assert_node_input_output(node, [2, 3], [1]) 469 | attrs = self.parse_attrs(node.attribute) 470 | 471 | # Get the input and output size 472 | conn_input_ids = [0] 473 | if node.input[1] in self.input_sizes: 474 | # params 475 | if 'transB' in attrs and attrs['transB'] != 0: 476 | output_size, input_size = self.input_sizes[node.input[1]][:2] 477 | else: 478 | input_size, output_size = self.input_sizes[node.input[1]][:2] 479 | else: 480 | # weight as node 481 | input_size, output_size = -1, -1 482 | conn_input_ids.append(1) 483 | bias = True if len(node.input) == 3 else False 484 | 485 | G_attr = AttrGemm(input_size, output_size, bias, name=node.output[0], **attrs) 486 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 487 | 488 | def parseGlobalAveragePool(self, node): 489 | self.assert_node_input_output(node, [1], [1]) 490 | return self.parse_general_node(node, AttrGlobalAveragePool, [0], [0]) 491 | 492 | def parseGlobalLpPool(self, node): 493 | self.assert_node_input_output(node, [1], [1]) 494 | return self.parse_general_node(node, AttrGlobalLpPool, [0], [0]) 495 | 496 | def parseGlobalMaxPool(self, node): 497 | self.assert_node_input_output(node, [1], [1]) 498 | return self.parse_general_node(node, AttrGlobalMaxPool, [0], [0]) 499 | 500 | def parseGreater(self, node): 501 | self.assert_node_input_output(node, [2], [1]) 502 | return self.parse_general_node(node, AttrGreater, [0, 1], [0]) 503 | 504 | def parseGreaterOrEqual(self, node): 505 | self.assert_node_input_output(node, [2], [1]) 506 | return self.parse_general_node(node, AttrGreaterOrEqual, [0, 1], [0]) 507 | 508 | def parseHardSigmoid(self, node): 509 | self.assert_node_input_output(node, [1], [1]) 510 | return self.parse_general_node(node, AttrHardSigmoid, [0], [0]) 511 | 512 | def parseHardmax(self, node): 513 | self.assert_node_input_output(node, [1], [1]) 514 | return self.parse_general_node(node, AttrHardmax, [0], [0]) 515 | 516 | def parseIdentity(self, node): 517 | self.assert_node_input_output(node, [1], [1]) 518 | return self.parse_general_node(node, AttrIdentity, [0], [0]) 519 | 520 | def parseIf(self, node): 521 | self.assert_node_input_output(node, [1], None, output_low_bound=1) 522 | return self.parse_general_node(node, AttrIf, [0], range(len(node.output))) 523 | 524 | def parseInstanceNormalization(self, node): 525 | self.assert_node_input_output(node, [3], [1]) 526 | attrs = self.parse_attrs(node.attribute) 527 | 528 | # Get the size of scale, B 529 | conn_input_ids = [0, 1, 2] 530 | scale_size = B_size = -1 531 | if node.input[1] in self.input_sizes: 532 | scale_size = self.input_sizes[node.input[1]] 533 | conn_input_ids.remove(1) 534 | if node.input[2] in self.input_sizes: 535 | B_size = self.input_sizes[node.input[2]] 536 | conn_input_ids.remove(2) 537 | 538 | G_attr = AttrInstanceNormalization(scale_size, B_size, name = node.output[0], **attrs) 539 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 540 | 541 | def parseIsInf(self, node): 542 | self.assert_node_input_output(node, [1], [1]) 543 | return self.parse_general_node(node, AttrIsInf, [0], [0]) 544 | 545 | def parseIsNaN(self, node): 546 | self.assert_node_input_output(node, [1], [1]) 547 | return self.parse_general_node(node, AttrIsNaN, [0], [0]) 548 | 549 | def parseLRN(self, node): 550 | self.assert_node_input_output(node, [1], [1]) 551 | return self.parse_general_node(node, AttrLRN, [0], [0]) 552 | 553 | def parseLSTM(self, node): 554 | self.assert_node_input_output(node, [3, 4, 5, 6, 7, 8], [0, 1, 2, 3]) 555 | return self.parse_general_node(node, AttrGRU, range(len(node.input)), range(len(node.output))) 556 | 557 | def parseLeakyRelu(self, node): 558 | self.assert_node_input_output(node, [1], [1]) 559 | return self.parse_general_node(node, AttrLeakyRelu, [0], [0]) 560 | 561 | def parseLess(self, node): 562 | self.assert_node_input_output(node, [2], [1]) 563 | return self.parse_general_node(node, AttrLess, [0, 1], [0]) 564 | 565 | def parseLessOrEqual(self, node): 566 | self.assert_node_input_output(node, [2], [1]) 567 | return self.parse_general_node(node, AttrLessOrEqual, [0, 1], [0]) 568 | 569 | def parseLog(self, node): 570 | self.assert_node_input_output(node, [1], [1]) 571 | return self.parse_general_node(node, AttrLog, [0], [0]) 572 | 573 | def parseLogSoftmax(self, node): 574 | self.assert_node_input_output(node, [1], [1]) 575 | return self.parse_general_node(node, AttrLogSoftmax, [0], [0]) 576 | 577 | def parseLoop(self, node): 578 | self.assert_node_input_output(node, None, None, input_low_bound=2, output_low_bound=1) 579 | return self.parse_general_node(node, AttrGRU, range(len(node.input)), range(len(node.output))) 580 | 581 | def parseLpNormalization(self, node): 582 | self.assert_node_input_output(node, [1], [1]) 583 | return self.parse_general_node(node, AttrLpNormalization, [0], [0]) 584 | 585 | def parseLpPool(self, node): 586 | self.assert_node_input_output(node, [1], [1]) 587 | return self.parse_general_node(node, AttrLpPool, [0], [0]) 588 | 589 | def parseMatMul(self, node): 590 | self.assert_node_input_output(node, [2], [1]) 591 | return self.parse_general_node(node, AttrMatMul, [0, 1], [0]) 592 | 593 | def parseMatMulInteger(self, node): 594 | self.assert_node_input_output(node, [2, 3, 4], [1]) 595 | return self.parse_general_node(node, AttrMatMulInteger, range(len(node.input)), [0]) 596 | 597 | def parseMax(self, node): 598 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 599 | return self.parse_general_node(node, AttrMax, range(len(node.input)), [0]) 600 | 601 | def parseMaxPool(self, node): 602 | self.assert_node_input_output(node, [1], [1, 2]) 603 | return self.parse_general_node(node, AttrMaxPool, [0], range(len(node.output))) 604 | 605 | def parseMaxRoiPool(self, node): 606 | self.assert_node_input_output(node, [2], [1]) 607 | return self.parse_general_node(node, AttrMaxRoiPool, [0, 1], [0]) 608 | 609 | def parseMaxUnpool(self, node): 610 | self.assert_node_input_output(node, [2, 3], [1]) 611 | return self.parse_general_node(node, AttrMaxUnpool, range(len(node.input)), [0]) 612 | 613 | def parseMean(self, node): 614 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 615 | return self.parse_general_node(node, AttrMean, range(len(node.input)), [0]) 616 | 617 | def parseMeanVarianceNormalization(self, node): 618 | self.assert_node_input_output(node, [1], [1]) 619 | return self.parse_general_node(node, AttrMeanVarianceNormalization, [0], [0]) 620 | 621 | def parseMin(self, node): 622 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 623 | return self.parse_general_node(node, AttrMin, range(len(node.input)), [0]) 624 | 625 | def parseMod(self, node): 626 | self.assert_node_input_output(node, [2], [1]) 627 | return self.parse_general_node(node, AttrMod, [0, 1], [0]) 628 | 629 | def parseMul(self, node): 630 | self.assert_node_input_output(node, [2], [1]) 631 | return self.parse_general_node(node, AttrMul, [0, 1], [0]) 632 | 633 | def parseMultinomial(self, node): 634 | self.assert_node_input_output(node, [1], [1]) 635 | return self.parse_general_node(node, AttrMultinomial, [0], [0]) 636 | 637 | def parseNeg(self, node): 638 | self.assert_node_input_output(node, [1], [1]) 639 | return self.parse_general_node(node, AttrNeg, [0], [0]) 640 | 641 | def parseNegativeLogLikelihoodLoss(self, node): 642 | self.assert_node_input_output(node, [2, 3], [1]) 643 | return self.parse_general_node(node, AttrNegativeLogLikelihoodLoss, range(len(node.input)), [0]) 644 | 645 | def parseNonMaxSuppression(self, node): 646 | self.assert_node_input_output(node, [2, 3, 4, 5], [1]) 647 | return self.parse_general_node(node, AttrNonMaxSuppression, range(len(node.input)), [0]) 648 | 649 | def parseNonZero(self, node): 650 | self.assert_node_input_output(node, [1], [1]) 651 | return self.parse_general_node(node, AttrNonZero, [0], [0]) 652 | 653 | def parseNot(self, node): 654 | self.assert_node_input_output(node, [1], [1]) 655 | return self.parse_general_node(node, AttrNot, [0], [0]) 656 | 657 | def parseOneHot(self, node): 658 | self.assert_node_input_output(node, [3], [1]) 659 | return self.parse_general_node(node, AttrOneHot, [0, 1, 2], [0]) 660 | 661 | def parseOr(self, node): 662 | self.assert_node_input_output(node, [2], [1]) 663 | return self.parse_general_node(node, AttrOr, [0, 1], [0]) 664 | 665 | def parsePRelu(self, node): 666 | self.assert_node_input_output(node, [2], [1]) 667 | return self.parse_general_node(node, AttrPRelu, [0, 1], [0]) 668 | 669 | def parsePad(self, node): 670 | self.assert_node_input_output(node, [1, 2, 3], [1]) 671 | return self.parse_general_node(node, AttrPad, range(len(node.input)), [0]) 672 | 673 | def parsePow(self, node): 674 | self.assert_node_input_output(node, [2], [1]) 675 | return self.parse_general_node(node, AttrPow, [0, 1], [0]) 676 | 677 | def parseQLinearConv(self, node): 678 | self.assert_node_input_output(node, [8, 9], [1]) 679 | attrs = self.parse_attrs(node.attribute) 680 | 681 | # Get input channel and output channel size 682 | conn_input_ids = list(range(node.input)) 683 | if node.input[3] in self.input_sizes: 684 | # param 685 | output_channel, input_channel = self.input_sizes[node.input[3]][:2] 686 | if 'kernel_shape' not in attrs: 687 | attrs['kernel_shape'] = self.input_sizes[node.input[3]][-2:] 688 | conn_input_ids.remove(3) 689 | else: 690 | # weight as node 691 | output_channel, input_channel = -1, -1 692 | if 'kernel_shape' not in attrs: 693 | attrs['kernel_shape'] = -1 694 | 695 | G_attr = AttrQLinearConv(input_channel, output_channel, name = node.output[0], **attrs) 696 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 697 | 698 | def parseQLinearMatMul(self, node): 699 | self.assert_node_input_output(node, [8], [1]) 700 | return self.parse_general_node(node, AttrQLinearMatMul, range(len(node.input)), [0]) 701 | 702 | def parseQuantizeLinear(self, node): 703 | self.assert_node_input_output(node, [2, 3], [1]) 704 | return self.parse_general_node(node, AttrQuantizeLinear, range(len(node.input)), [0]) 705 | 706 | def parseRNN(self, node): 707 | self.assert_node_input_output(node, [3, 4, 5, 6], [0, 1, 2]) 708 | return self.parse_general_node(node, AttrRNN, range(len(node.input)), range(len(node.output))) 709 | 710 | def parseRandomNormal(self, node): 711 | self.assert_node_input_output(node, [0], [1]) 712 | return self.parse_general_node(node, AttrRandomNormal, [], [0]) 713 | 714 | def parseRandomNormalLike(self, node): 715 | self.assert_node_input_output(node, [1], [1]) 716 | return self.parse_general_node(node, AttrRandomNormalLike, [0], [0]) 717 | 718 | def parseRandomUniform(self, node): 719 | self.assert_node_input_output(node, [0], [1]) 720 | return self.parse_general_node(node, AttrRandomUniform, [], [0]) 721 | 722 | def parseRandomUniformLike(self, node): 723 | self.assert_node_input_output(node, [1], [1]) 724 | return self.parse_general_node(node, AttrRandomUniformLike, [0], [0]) 725 | 726 | def parseRange(self, node): 727 | self.assert_node_input_output(node, [3], [1]) 728 | return self.parse_general_node(node, AttrRange, [0, 1, 2], [0]) 729 | 730 | def parseReciprocal(self, node): 731 | self.assert_node_input_output(node, [1], [1]) 732 | return self.parse_general_node(node, AttrReciprocal, [0], [0]) 733 | 734 | def parseReduceL1(self, node): 735 | self.assert_node_input_output(node, [1], [1]) 736 | return self.parse_general_node(node, AttrReduceL1, [0], [0]) 737 | 738 | def parseReduceL2(self, node): 739 | self.assert_node_input_output(node, [1], [1]) 740 | return self.parse_general_node(node, AttrReduceL2, [0], [0]) 741 | 742 | def parseReduceLogSum(self, node): 743 | self.assert_node_input_output(node, [1], [1]) 744 | return self.parse_general_node(node, AttrReduceLogSum, [0], [0]) 745 | 746 | def parseReduceLogSumExp(self, node): 747 | self.assert_node_input_output(node, [1], [1]) 748 | return self.parse_general_node(node, AttrReduceLogSumExp, [0], [0]) 749 | 750 | def parseReduceMax(self, node): 751 | self.assert_node_input_output(node, [1], [1]) 752 | return self.parse_general_node(node, AttrReduceMax, [0], [0]) 753 | 754 | def parseReduceMean(self, node): 755 | self.assert_node_input_output(node, [1], [1]) 756 | return self.parse_general_node(node, AttrReduceMean, [0], [0]) 757 | 758 | def parseReduceMin(self, node): 759 | self.assert_node_input_output(node, [1], [1]) 760 | return self.parse_general_node(node, AttrReduceMin, [0], [0]) 761 | 762 | def parseReduceProd(self, node): 763 | self.assert_node_input_output(node, [1], [1]) 764 | return self.parse_general_node(node, AttrReduceProd, [0], [0]) 765 | 766 | def parseReduceSum(self, node): 767 | self.assert_node_input_output(node, [1, 2], [1]) 768 | return self.parse_general_node(node, AttrReduceSum, range(len(node.input)), [0]) 769 | 770 | def parseReduceSumSquare(self, node): 771 | self.assert_node_input_output(node, [1], [1]) 772 | return self.parse_general_node(node, AttrReduceSumSquare, [0], [0]) 773 | 774 | def parseRelu(self, node): 775 | self.assert_node_input_output(node, [1], [1]) 776 | return self.parse_general_node(node, AttrRelu, [0], [0]) 777 | 778 | def parseReshape(self, node): 779 | self.assert_node_input_output(node, [1, 2], [1]) 780 | attrs = self.parse_attrs(node.attribute) 781 | 782 | # opset = [1, 5), length of node input == 1 and attr has shape 783 | conn_input_ids = [0] 784 | if 'shape' in attrs: 785 | shapes = attrs['shape'] 786 | else: 787 | if node.input[1] in self.input_sizes: 788 | # params 789 | shapes = self.input_sizes[node.input[1]] 790 | else: 791 | # shape as node 792 | shapes = () 793 | conn_input_ids.append(1) 794 | 795 | G_attr = AttrReshape(shapes, name = node.output[0], **attrs) 796 | return self.get_networkx_node_edges(node, G_attr, conn_input_ids, [0]) 797 | 798 | def parseResize(self, node): 799 | self.assert_node_input_output(node, [1, 2, 3, 4], [1]) 800 | return self.parse_general_node(node, AttrResize, range(len(node.input)), [0]) 801 | 802 | def parseReverseSequence(self, node): 803 | self.assert_node_input_output(node, [2], [1]) 804 | return self.parse_general_node(node, AttrReverseSequence, [0, 1], [0]) 805 | 806 | def parseRoiAlign(self, node): 807 | # note: official onnx only support 3 input, nart support 2 input 808 | self.assert_node_input_output(node, [2, 3], [1]) 809 | return self.parse_general_node(node, AttrRoiAlign, range(len(node.input)), [0]) 810 | 811 | def parseRound(self, node): 812 | self.assert_node_input_output(node, [1], [1]) 813 | return self.parse_general_node(node, AttrRound, [0], [0]) 814 | 815 | def parseScan(self, node): 816 | self.assert_node_input_output(node, None, None, input_low_bound=1, output_low_bound=1) 817 | return self.parse_general_node(node, AttrScan, range(len(node.input)), range(len(node.output))) 818 | 819 | def parseScatter(self, node): 820 | self.assert_node_input_output(node, [3], [1]) 821 | return self.parse_general_node(node, AttrScatter, [0, 1, 2], [0]) 822 | 823 | def parseScatterElements(self, node): 824 | self.assert_node_input_output(node, [3], [1]) 825 | return self.parse_general_node(node, AttrScatterElements, [0, 1, 2], [0]) 826 | 827 | def parseScatterND(self, node): 828 | self.assert_node_input_output(node, [3], [1]) 829 | return self.parse_general_node(node, AttrScatterND, [0, 1, 2], [0]) 830 | 831 | def parseSelu(self, node): 832 | self.assert_node_input_output(node, [1], [1]) 833 | return self.parse_general_node(node, AttrSelu, [0], [0]) 834 | 835 | def parseSequenceAt(self, node): 836 | self.assert_node_input_output(node, [2], [1]) 837 | return self.parse_general_node(node, AttrSequenceAt, [0, 1], [0]) 838 | 839 | def parseSequenceConstruct(self, node): 840 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 841 | return self.parse_general_node(node, AttrSequenceConstruct, range(len(node.input)), [0]) 842 | 843 | def parseSequenceEmpty(self, node): 844 | self.assert_node_input_output(node, [0], [1]) 845 | return self.parse_general_node(node, AttrSequenceEmpty, [], [0]) 846 | 847 | def parseSequenceErase(self, node): 848 | self.assert_node_input_output(node, [1, 2], [1]) 849 | return self.parse_general_node(node, AttrSequenceErase, range(len(node.input)), [0]) 850 | 851 | def parseSequenceInsert(self, node): 852 | self.assert_node_input_output(node, [2, 3], [1]) 853 | return self.parse_general_node(node, AttrSequenceInsert, range(len(node.input)), [0]) 854 | 855 | def parseSequenceLength(self, node): 856 | self.assert_node_input_output(node, [1], [1]) 857 | return self.parse_general_node(node, AttrSequenceLength, [0], [0]) 858 | 859 | def parseShape(self, node): 860 | self.assert_node_input_output(node, [1], [1]) 861 | return self.parse_general_node(node, AttrShape, [0], [0]) 862 | 863 | def parseShrink(self, node): 864 | self.assert_node_input_output(node, [1], [1]) 865 | return self.parse_general_node(node, AttrShrink, [0], [0]) 866 | 867 | def parseSigmoid(self, node): 868 | self.assert_node_input_output(node, [1], [1]) 869 | return self.parse_general_node(node, AttrSigmoid, [0], [0]) 870 | 871 | def parseSign(self, node): 872 | self.assert_node_input_output(node, [1], [1]) 873 | return self.parse_general_node(node, AttrSign, [0], [0]) 874 | 875 | def parseSin(self, node): 876 | self.assert_node_input_output(node, [1], [1]) 877 | return self.parse_general_node(node, AttrSin, [0], [0]) 878 | 879 | def parseSinh(self, node): 880 | self.assert_node_input_output(node, [1], [1]) 881 | return self.parse_general_node(node, AttrSinh, [0], [0]) 882 | 883 | def parseSize(self, node): 884 | self.assert_node_input_output(node, [1], [1]) 885 | return self.parse_general_node(node, AttrSize, [0], [0]) 886 | 887 | def parseSlice(self, node): 888 | self.assert_node_input_output(node, [1, 3, 4, 5], [1]) 889 | return self.parse_general_node(node, AttrSlice, range(len(node.input)), [0]) 890 | 891 | def parseSoftmax(self, node): 892 | self.assert_node_input_output(node, [1], [1]) 893 | return self.parse_general_node(node, AttrSoftmax, [0], [0]) 894 | 895 | def parseSoftmaxCrossEntropyLoss(self, node): 896 | self.assert_node_input_output(node, [2, 3], [1, 2]) 897 | return self.parse_general_node(node, AttrSoftmaxCrossEntropyLoss, range(len(node.input)), range(len(node.output))) 898 | 899 | def parseSoftplus(self, node): 900 | self.assert_node_input_output(node, [1], [1]) 901 | return self.parse_general_node(node, AttrSoftplus, [0], [0]) 902 | 903 | def parseSoftsign(self, node): 904 | self.assert_node_input_output(node, [1], [1]) 905 | return self.parse_general_node(node, AttrSoftsign, [0], [0]) 906 | 907 | def parseSpaceToDepth(self, node): 908 | self.assert_node_input_output(node, [1], [1]) 909 | return self.parse_general_node(node, AttrSpaceToDepth, [0], [0]) 910 | 911 | def parseSplit(self, node): 912 | self.assert_node_input_output(node, [1, 2], None, output_low_bound=1) 913 | return self.parse_general_node(node, AttrSplit, range(len(node.input)), range(len(node.output))) 914 | 915 | def parseSplitToSequence(self, node): 916 | self.assert_node_input_output(node, [1, 2], [1]) 917 | return self.parse_general_node(node, AttrSplitToSequence, range(len(node.input)), [0]) 918 | 919 | def parseSqrt(self, node): 920 | self.assert_node_input_output(node, [1], [1]) 921 | return self.parse_general_node(node, AttrSqrt, [0], [0]) 922 | 923 | def parseSqueeze(self, node): 924 | self.assert_node_input_output(node, [1, 2], [1]) 925 | return self.parse_general_node(node, AttrSqueeze, range(len(node.input)), [0]) 926 | 927 | def parseStringNormalizer(self, node): 928 | self.assert_node_input_output(node, [1], [1]) 929 | return self.parse_general_node(node, AttrStringNormalizer, [0], [0]) 930 | 931 | def parseSub(self, node): 932 | self.assert_node_input_output(node, [2], [1]) 933 | return self.parse_general_node(node, AttrSub, [0, 1], [0]) 934 | 935 | def parseSum(self, node): 936 | self.assert_node_input_output(node, None, [1], input_low_bound=1) 937 | return self.parse_general_node(node, AttrSum, range(len(node.input)), [0]) 938 | 939 | def parseTan(self, node): 940 | self.assert_node_input_output(node, [1], [1]) 941 | return self.parse_general_node(node, AttrTan, [0], [0]) 942 | 943 | def parseTanh(self, node): 944 | self.assert_node_input_output(node, [1], [1]) 945 | return self.parse_general_node(node, AttrTanh, [0], [0]) 946 | 947 | def parseTfIdfVectorizer(self, node): 948 | self.assert_node_input_output(node, [1], [1]) 949 | return self.parse_general_node(node, AttrTfIdfVectorizer, [0], [0]) 950 | 951 | def parseThresholdedRelu(self, node): 952 | self.assert_node_input_output(node, [1], [1]) 953 | return self.parse_general_node(node, AttrThresholdedRelu, [0], [0]) 954 | 955 | def parseTile(self, node): 956 | self.assert_node_input_output(node, [2, 3], [1]) 957 | return self.parse_general_node(node, AttrTile, range(len(node.input)), [0]) 958 | 959 | def parseTopK(self, node): 960 | self.assert_node_input_output(node, [1, 2], [2]) 961 | return self.parse_general_node(node, AttrTopK, range(len(node.input)), [0, 1]) 962 | 963 | def parseTranspose(self, node): 964 | self.assert_node_input_output(node, [1], [1]) 965 | return self.parse_general_node(node, AttrTranspose, [0], [0]) 966 | 967 | def parseUnique(self, node): 968 | self.assert_node_input_output(node, [1], [1, 2, 3, 4]) 969 | return self.parse_general_node(node, AttrUnique, [0], range(len(node.output))) 970 | 971 | def parseUnsqueeze(self, node): 972 | self.assert_node_input_output(node, [1, 2], [1]) 973 | return self.parse_general_node(node, AttrUnsqueeze, range(len(node.input)), [0]) 974 | 975 | def parseUpsample(self, node): 976 | self.assert_node_input_output(node, [1, 2], [1]) 977 | return self.parse_general_node(node, AttrUpsample, range(len(node.input)), [0]) 978 | 979 | def parseWhere(self, node): 980 | self.assert_node_input_output(node, [3], [1]) 981 | return self.parse_general_node(node, AttrWhere, [0, 1, 2], [0]) 982 | 983 | def parseXor(self, node): 984 | self.assert_node_input_output(node, [2], [1]) 985 | return self.parse_general_node(node, AttrWhere, [0, 1], [0]) -------------------------------------------------------------------------------- /predictor/feature/op_attribute.py: -------------------------------------------------------------------------------- 1 | # decorator for defining attributes in different onnx opsets 2 | def ATTR(*out_args, **out_kwargs): 3 | def wrapper1(func): 4 | def wrapper2(*in_args, **in_kwargs): 5 | 6 | # check if current opset support current attribute 7 | if check_version(in_kwargs['opsets'], out_kwargs['opsets']): 8 | attr_name = out_kwargs['name'] 9 | 10 | if attr_name not in in_kwargs: 11 | if 'default' not in out_kwargs: 12 | # opsets = [9, 10] for Slice, starts and ends 13 | # will be required for opset 9 but deprecated for opset 10. 14 | # the check will cause error 15 | raise Exception("ATTR {} is required while constructing!" \ 16 | .format(attr_name, in_kwargs['opsets'])) 17 | attr_value = out_kwargs['default'] 18 | else: 19 | attr_value = in_kwargs[attr_name] 20 | del in_kwargs[attr_name] 21 | 22 | # format functions, such like dilations, kernel_shape, ... 23 | if 'fmt_func' in out_kwargs: 24 | attr_value = out_kwargs['fmt_func'](attr_value) 25 | in_args[0].attributes[attr_name] = attr_value 26 | 27 | f = func(*in_args, **in_kwargs) 28 | return f 29 | return wrapper2 30 | return wrapper1 31 | 32 | 33 | # check if attr opset versions contain target opset versions 34 | def check_version(target_opsets, attr_opset): 35 | for v in target_opsets: 36 | if len(attr_opset) == 1: 37 | if v >= attr_opset[0]: 38 | return True 39 | elif len(attr_opset) == 2: 40 | if v >= attr_opset[0] and v < attr_opset[1]: 41 | return True 42 | else: 43 | raise Exception("ATTR opset length should be 1 or 2!") 44 | return False 45 | 46 | 47 | WARNINGS = set() 48 | def check_redundant_attrs(attrs, class_name=""): 49 | global WARNINGS 50 | tv = attrs['opsets'] 51 | del attrs['opsets'] 52 | if (not attrs) is False: 53 | WARNINGS.add("Warnings: unused args for <{}> in opsets <{}>: {}!".format(class_name, tv, attrs)) 54 | 55 | 56 | class FSize: 57 | 58 | @staticmethod 59 | def kernel(x): 60 | return FSize.tuple2(x, "Kernel") 61 | 62 | @staticmethod 63 | def stride(x): 64 | return FSize.tuple2(x, "Stride") 65 | 66 | @staticmethod 67 | def padding(x): 68 | return FSize.tuple4(x, "Padding") 69 | 70 | @staticmethod 71 | def dilation(x): 72 | return FSize.tuple2(x, "Dilation") 73 | 74 | @staticmethod 75 | def tuple2(x, note): 76 | if isinstance(x, int): 77 | return (x, x) 78 | if isinstance(x, list): 79 | x = tuple(x) 80 | if isinstance(x, tuple) and len(x) == 2: 81 | return x 82 | raise Exception("Unexcepted {} size: {}!".format(note, x)) 83 | 84 | @staticmethod 85 | def tuple4(x, note): 86 | if isinstance(x, int): 87 | return (x, x, x, x) 88 | if isinstance(x, list): 89 | x = tuple(x) 90 | if isinstance(x, tuple) and len(x) == 2: 91 | return (x[0], x[0], x[1], x[1]) 92 | if isinstance(x, tuple) and len(x) == 4: 93 | return x 94 | raise Exception("Unexcepted {} size: {}!".format(note, x)) 95 | 96 | 97 | # base class for attributes of oerations 98 | class AttrBaseOp(object): 99 | def __getattr__(self, k): 100 | 101 | # attribtes that is not important 102 | # different values in self.attributes 103 | if k == "node_properties": 104 | setattr(self, "node_properties", {}) 105 | return getattr(self, "node_properties") 106 | 107 | # attributes to be hashed and compared 108 | # different values in self.attributes denotes different nodes 109 | elif k == "attributes": 110 | setattr(self, "attributes", {}) 111 | return getattr(self, "attributes") 112 | 113 | else: 114 | raise(AttributeError(k)) 115 | 116 | # ----------------------------------------------------------------------------- 117 | # Attribute Class For Specific Operator 118 | # ----------------------------------------------------------------------------- 119 | 120 | # This file defines attributes of available operators from onnx 121 | class AttrInput(AttrBaseOp): 122 | 123 | def __init__(self, name=''): 124 | self.node_properties['name'] = name 125 | self.attributes['type'] = 'Input' 126 | 127 | 128 | class AttrAbs(AttrBaseOp): 129 | 130 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 131 | def __init__(self, name='', **kwargs): 132 | 133 | self.node_properties['name'] = name 134 | self.attributes['type'] = 'Abs' 135 | check_redundant_attrs(kwargs, self.__class__.__name__) 136 | 137 | 138 | class AttrAcos(AttrBaseOp): 139 | 140 | def __init__(self, name='', **kwargs): 141 | self.node_properties['name'] = name 142 | self.attributes['type'] = 'Acos' 143 | 144 | 145 | class AttrAcosh(AttrBaseOp): 146 | 147 | def __init__(self, name='', **kwargs): 148 | self.node_properties['name'] = name 149 | self.attributes['type'] = 'Acosh' 150 | 151 | 152 | class AttrAdd(AttrBaseOp): 153 | 154 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 155 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 156 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 157 | def __init__(self, name='', **kwargs): 158 | 159 | self.node_properties['name'] = name 160 | self.attributes['type'] = 'Add' 161 | check_redundant_attrs(kwargs, self.__class__.__name__) 162 | 163 | 164 | class AttrAnd(AttrBaseOp): 165 | 166 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 167 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 168 | def __init__(self, name='', **kwargs): 169 | 170 | self.node_properties['name'] = name 171 | self.attributes['type'] = 'And' 172 | check_redundant_attrs(kwargs, self.__class__.__name__) 173 | 174 | 175 | class AttrArgMax(AttrBaseOp): 176 | 177 | @ATTR(name='axis', opsets=( 1, ), default=0) # v1 add 178 | @ATTR(name='keepdims', opsets=( 1, ), default=1) # v1 add 179 | @ATTR(name='select_last_index', opsets=(12, ), default=0) # v12 add 180 | def __init__(self, name='', **kwargs): 181 | 182 | self.node_properties['name'] = name 183 | self.attributes['type'] = 'ArgMax' 184 | check_redundant_attrs(kwargs, self.__class__.__name__) 185 | 186 | 187 | class AttrArgMin(AttrBaseOp): 188 | 189 | @ATTR(name='axis', opsets=( 1, ), default=0) # v1 add 190 | @ATTR(name='keepdims', opsets=( 1, ), default=1) # v1 add 191 | @ATTR(name='select_last_index', opsets=(12, ), default=0) # v12 add 192 | def __init__(self, name='', **kwargs): 193 | 194 | self.node_properties['name'] = name 195 | self.attributes['type'] = 'ArgMin' 196 | check_redundant_attrs(kwargs, self.__class__.__name__) 197 | 198 | 199 | class AttrAsin(AttrBaseOp): 200 | 201 | def __init__(self, name='', **kwargs): 202 | self.node_properties['name'] = name 203 | self.attributes['type'] = 'Asin' 204 | 205 | 206 | class AttrAsinh(AttrBaseOp): 207 | 208 | def __init__(self, name='', **kwargs): 209 | self.node_properties['name'] = name 210 | self.attributes['type'] = 'Asinh' 211 | 212 | 213 | class AttrAtan(AttrBaseOp): 214 | 215 | def __init__(self, name='', **kwargs): 216 | self.node_properties['name'] = name 217 | self.attributes['type'] = 'Atan' 218 | 219 | 220 | class AttrAtanh(AttrBaseOp): 221 | 222 | def __init__(self, name='', **kwargs): 223 | self.node_properties['name'] = name 224 | self.attributes['type'] = 'Atanh' 225 | 226 | 227 | class AttrAveragePool(AttrBaseOp): 228 | 229 | @ATTR(name='auto_pad', opsets=( 1, ), default='NOTSET') # v1 add 230 | @ATTR(name='ceil_mode', opsets=(10, ), default=0) # v10 add 231 | @ATTR(name='count_include_pad', opsets=( 7, ), default=0) # v7 add 232 | @ATTR(name='pads', opsets=( 1, ), default=0, fmt_func=FSize.padding) # v1 add 233 | @ATTR(name='strides', opsets=( 1, ), default=1, fmt_func=FSize.stride) # v1 add 234 | @ATTR(name='kernel_shape', opsets=( 1, ), fmt_func=FSize.kernel) # v1 add, required 235 | def __init__(self, name='', **kwargs): 236 | 237 | self.node_properties['name'] = name 238 | self.attributes['type'] = 'AveragePool' 239 | check_redundant_attrs(kwargs, self.__class__.__name__) 240 | 241 | 242 | class AttrBatchNormalization(AttrBaseOp): 243 | 244 | @ATTR(name='epsilon', opsets=(1, ), default=1e-5) # v1 add 245 | @ATTR(name='momentum', opsets=(1, ), default=0.9) # v1 add 246 | @ATTR(name='consumed_inputs', opsets=(1, 6)) # v1 add, v6 delete, required 247 | @ATTR(name='is_test', opsets=(1, 7), default=0) # v1 add, v7 delete 248 | @ATTR(name='spatial', opsets=(1, 9), default=1) # v1 add, v9 delete 249 | def __init__(self, scale_size, B_size, mean_size, var_size, name='', **kwargs): 250 | 251 | self.node_properties['name'] = name 252 | self.attributes['type'] = 'BatchNormalization' 253 | check_redundant_attrs(kwargs, self.__class__.__name__) 254 | 255 | # sizes of scale, B, mean, var from weights 256 | self.attributes['scale_size'] = scale_size 257 | self.attributes['B_size'] = B_size 258 | self.attributes['mean_size'] = mean_size 259 | self.attributes['var_size'] = var_size 260 | 261 | 262 | class AttrBitShift(AttrBaseOp): 263 | 264 | @ATTR(name='direction', opsets=(11, )) # v11 add, required 265 | def __init__(self, name='', **kwargs): 266 | 267 | self.node_properties['name'] = name 268 | self.attributes['type'] = 'BitShift' 269 | check_redundant_attrs(kwargs, self.__class__.__name__) 270 | 271 | 272 | class AttrCast(AttrBaseOp): 273 | 274 | @ATTR(name='to', opsets=(1, )) # v1 add, required 275 | def __init__(self, name='', **kwargs): 276 | 277 | self.node_properties['name'] = name 278 | self.attributes['type'] = 'Cast' 279 | check_redundant_attrs(kwargs, self.__class__.__name__) 280 | 281 | 282 | class AttrCeil(AttrBaseOp): 283 | 284 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 285 | def __init__(self, name='', **kwargs): 286 | 287 | self.node_properties['name'] = name 288 | self.attributes['type'] = 'Ceil' 289 | check_redundant_attrs(kwargs, self.__class__.__name__) 290 | 291 | 292 | class AttrCelu(AttrBaseOp): 293 | 294 | @ATTR(name='alpha', opsets=(12, ), default=1.0) # v12 add 295 | def __init__(self, name='', **kwargs): 296 | 297 | self.node_properties['name'] = name 298 | self.attributes['type'] = 'Celu' 299 | check_redundant_attrs(kwargs, self.__class__.__name__) 300 | 301 | 302 | class AttrClip(AttrBaseOp): 303 | 304 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add 305 | @ATTR(name='max', opsets=(1, 11), default=3.402823e+38) # v1 add, v11 delete 306 | @ATTR(name='min', opsets=(1, 11), default=-3.402823e+38) # v1 add, v11 delete 307 | def __init__(self, name='', **kwargs): 308 | 309 | self.node_properties['name'] = name 310 | self.attributes['type'] = 'Clip' 311 | check_redundant_attrs(kwargs, self.__class__.__name__) 312 | 313 | 314 | class AttrCompress(AttrBaseOp): 315 | 316 | @ATTR(name='axis', opsets=(9, ), default=0) # v9 add 317 | def __init__(self, name='', **kwargs): 318 | 319 | self.node_properties['name'] = name 320 | self.attributes['type'] = 'Compress' 321 | check_redundant_attrs(kwargs, self.__class__.__name__) 322 | 323 | 324 | class AttrConcat(AttrBaseOp): 325 | 326 | @ATTR(name='axis', opsets=(1, 4), default=1) # v1 add, v4 delete 327 | @ATTR(name='axis', opsets=(4, )) # v4 add, required 328 | def __init__(self, name='', **kwargs): 329 | 330 | self.node_properties['name'] = name 331 | self.attributes['type'] = 'Concat' 332 | check_redundant_attrs(kwargs, self.__class__.__name__) 333 | 334 | 335 | class AttrConcatFromSequence(AttrBaseOp): 336 | 337 | @ATTR(name='axis', opsets=(11, )) # v11 add, required 338 | @ATTR(name='new_axis', opsets=(11, ), default=0) # v11 add 339 | def __init__(self, name='', **kwargs): 340 | 341 | self.node_properties['name'] = name 342 | self.attributes['type'] = 'ConcatFromSequence' 343 | check_redundant_attrs(kwargs, self.__class__.__name__) 344 | 345 | 346 | class AttrConstant(AttrBaseOp): 347 | 348 | @ATTR(name='value', opsets=( 1, ), default=None) # v1 add, record dims not value 349 | @ATTR(name='sparse_value', opsets=(11, ), default=None) # v11 add, record dims not value 350 | @ATTR(name='value_float', opsets=(12, ), default=0.0) # v12 add 351 | @ATTR(name='value_floats', opsets=(12, ), default=()) # v12 add 352 | @ATTR(name='value_int', opsets=(12, ), default=0) # v12 add 353 | @ATTR(name='value_ints', opsets=(12, ), default=()) # v12 add 354 | @ATTR(name='value_string', opsets=(12, ), default='') # v12 add 355 | @ATTR(name='value_strings', opsets=(12, ), default=()) # v12 add 356 | def __init__(self, name='', **kwargs): 357 | 358 | self.node_properties['name'] = name 359 | self.attributes['type'] = 'Constant' 360 | check_redundant_attrs(kwargs, self.__class__.__name__) 361 | 362 | 363 | class AttrConstantOfShape(AttrBaseOp): 364 | 365 | @ATTR(name='value', opsets=(9, ), default=None) # v9 add, record values 366 | def __init__(self, name='', **kwargs): 367 | 368 | self.node_properties['name'] = name 369 | self.attributes['type'] = 'ConstantOfShape' 370 | check_redundant_attrs(kwargs, self.__class__.__name__) 371 | 372 | 373 | class AttrConv(AttrBaseOp): 374 | 375 | @ATTR(name='auto_pad', opsets=(1, ), default='NOTSET') # v1 add 376 | @ATTR(name='group', opsets=(1, ), default=1) # v1 add 377 | @ATTR(name='pads', opsets=(1, ), default=0, fmt_func=FSize.padding) # v1 add 378 | @ATTR(name='strides', opsets=(1, ), default=1, fmt_func=FSize.stride) # v1 add 379 | @ATTR(name='dilations', opsets=(1, ), default=1, fmt_func=FSize.dilation) # v1 add 380 | @ATTR(name='kernel_shape', opsets=(1, ), fmt_func=FSize.kernel) # v1 add, required 381 | def __init__(self, input_channel, output_channel, bias, name='', **kwargs): 382 | 383 | self.node_properties['name'] = name 384 | self.attributes['type'] = 'Conv' 385 | check_redundant_attrs(kwargs, self.__class__.__name__) 386 | 387 | # input_channel & output_channel from weights 388 | self.attributes['input_channel'] = input_channel 389 | self.attributes['output_channel'] = output_channel 390 | self.attributes['bias'] = bias 391 | 392 | 393 | class AttrConvInteger(AttrBaseOp): 394 | 395 | @ATTR(name='auto_pad', opsets=(10, ), default='NOTSET') # v10 add 396 | @ATTR(name='group', opsets=(10, ), default=1) # v10 add 397 | @ATTR(name='pads', opsets=(10, ), default=0, fmt_func=FSize.padding) # v10 add 398 | @ATTR(name='strides', opsets=(10, ), default=1, fmt_func=FSize.stride) # v10 add 399 | @ATTR(name='dilations', opsets=(10, ), default=1, fmt_func=FSize.dilation) # v10 add 400 | @ATTR(name='kernel_shape', opsets=(10, ), fmt_func=FSize.kernel) # v10 add, required 401 | def __init__(self, input_channel, output_channel, name='', **kwargs): 402 | 403 | self.node_properties['name'] = name 404 | self.attributes['type'] = 'ConvInteger' 405 | check_redundant_attrs(kwargs, self.__class__.__name__) 406 | 407 | # input_channel & output_channel from weights 408 | self.attributes['input_channel'] = input_channel 409 | self.attributes['output_channel'] = output_channel 410 | 411 | 412 | class AttrConvTranspose(AttrBaseOp): 413 | 414 | @ATTR(name='auto_pad', opsets=(1, ), default='NOTSET') # v1 add 415 | @ATTR(name='group', opsets=(1, ), default=1) # v1 add 416 | @ATTR(name='output_shape', opsets=(1, ), default=()) # v1 add 417 | @ATTR(name='pads', opsets=(1, ), default=0, fmt_func=FSize.padding) # v1 add 418 | @ATTR(name='strides', opsets=(1, ), default=1, fmt_func=FSize.stride) # v1 add 419 | @ATTR(name='dilations', opsets=(1, ), default=1, fmt_func=FSize.dilation) # v1 add 420 | @ATTR(name='output_padding', opsets=(1, ), default=0, fmt_func=FSize.padding) # v1 add 421 | @ATTR(name='kernel_shape', opsets=(1, ), fmt_func=FSize.kernel) # v1 add, required 422 | def __init__(self, input_channel, output_channel, bias, name='', **kwargs): 423 | 424 | self.node_properties['name'] = name 425 | self.attributes['type'] = 'ConvTranspose' 426 | check_redundant_attrs(kwargs, self.__class__.__name__) 427 | 428 | # input_channel & output_channel from weights 429 | self.attributes['input_channel'] = input_channel 430 | self.attributes['output_channel'] = output_channel 431 | self.attributes['bias'] = bias 432 | 433 | 434 | class AttrCos(AttrBaseOp): 435 | 436 | def __init__(self, name='', **kwargs): 437 | self.node_properties['name'] = name 438 | self.attributes['type'] = 'Cos' 439 | 440 | 441 | class AttrCosh(AttrBaseOp): 442 | 443 | def __init__(self, name='', **kwargs): 444 | self.node_properties['name'] = name 445 | self.attributes['type'] = 'Cosh' 446 | 447 | 448 | class AttrCumSum(AttrBaseOp): 449 | 450 | @ATTR(name='exclusive', opsets=(11, ), default=0) # v11 add 451 | @ATTR(name='reverse', opsets=(11, ), default=0) # v11 add 452 | def __init__(self, name='', **kwargs): 453 | 454 | self.node_properties['name'] = name 455 | self.attributes['type'] = 'CumSum' 456 | check_redundant_attrs(kwargs, self.__class__.__name__) 457 | 458 | 459 | class AttrDepthToSpace(AttrBaseOp): 460 | 461 | @ATTR(name='blocksize', opsets=( 1, )) # v1 add, required 462 | @ATTR(name='mode', opsets=(11, ), default='DCR') # v11 add 463 | def __init__(self, name='', **kwargs): 464 | 465 | self.node_properties['name'] = name 466 | self.attributes['type'] = 'DepthToSpace' 467 | check_redundant_attrs(kwargs, self.__class__.__name__) 468 | 469 | 470 | class AttrDequantizeLinear(AttrBaseOp): 471 | 472 | @ATTR(name='axis', opsets=(13, ), default=1) # v13 add 473 | def __init__(self, name='', **kwargs): 474 | 475 | self.node_properties['name'] = name 476 | self.attributes['type'] = 'DequantizeLinear' 477 | check_redundant_attrs(kwargs, self.__class__.__name__) 478 | 479 | 480 | class AttrDet(AttrBaseOp): 481 | 482 | def __init__(self, name='', **kwargs): 483 | self.node_properties['name'] = name 484 | self.attributes['type'] = 'Det' 485 | 486 | 487 | class AttrDiv(AttrBaseOp): 488 | 489 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 490 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 491 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 492 | def __init__(self, name='', **kwargs): 493 | 494 | self.node_properties['name'] = name 495 | self.attributes['type'] = 'Div' 496 | check_redundant_attrs(kwargs, self.__class__.__name__) 497 | 498 | 499 | class AttrDropout(AttrBaseOp): 500 | 501 | @ATTR(name='consumed_inputs', opsets=( 1, 6), default=()) # v1 add, v6 delete 502 | @ATTR(name='is_test', opsets=( 1, 7), default=0) # v1 add, v7 delete 503 | @ATTR(name='ratio', opsets=( 1, 12), default=0.5) # v1 add, v12 delete 504 | @ATTR(name='seed', opsets=(12, ), default=0) # v12 add 505 | def __init__(self, name='', **kwargs): 506 | 507 | self.node_properties['name'] = name 508 | self.attributes['type'] = 'Dropout' 509 | check_redundant_attrs(kwargs, self.__class__.__name__) 510 | 511 | 512 | class AttrDynamicQuantizeLinear(AttrBaseOp): 513 | 514 | def __init__(self, name='', **kwargs): 515 | self.node_properties['name'] = name 516 | self.attributes['type'] = 'DynamicQuantizeLinear' 517 | 518 | 519 | class AttrEinsum(AttrBaseOp): 520 | 521 | @ATTR(name='equation', opsets=(12, )) # v12 add, required 522 | def __init__(self, name='', **kwargs): 523 | 524 | self.node_properties['name'] = name 525 | self.attributes['type'] = 'Einsum' 526 | check_redundant_attrs(kwargs, self.__class__.__name__) 527 | 528 | 529 | class AttrElu(AttrBaseOp): 530 | 531 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 532 | @ATTR(name='alpha', opsets=(1, ), default=1.0) # v1 add 533 | def __init__(self, name='', **kwargs): 534 | 535 | self.node_properties['name'] = name 536 | self.attributes['type'] = 'Elu' 537 | check_redundant_attrs(kwargs, self.__class__.__name__) 538 | 539 | 540 | class AttrEqual(AttrBaseOp): 541 | 542 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 543 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 544 | def __init__(self, name='', **kwargs): 545 | 546 | self.node_properties['name'] = name 547 | self.attributes['type'] = 'Equal' 548 | check_redundant_attrs(kwargs, self.__class__.__name__) 549 | 550 | 551 | class AttrErf(AttrBaseOp): 552 | 553 | def __init__(self, name='', **kwargs): 554 | self.node_properties['name'] = name 555 | self.attributes['type'] = 'Erf' 556 | 557 | 558 | class AttrExp(AttrBaseOp): 559 | 560 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 561 | def __init__(self, name='', **kwargs): 562 | 563 | self.node_properties['name'] = name 564 | self.attributes['type'] = 'Exp' 565 | check_redundant_attrs(kwargs, self.__class__.__name__) 566 | 567 | 568 | class AttrExpand(AttrBaseOp): 569 | 570 | def __init__(self, name='', **kwargs): 571 | self.node_properties['name'] = name 572 | self.attributes['type'] = 'Expand' 573 | 574 | 575 | class AttrEyeLike(AttrBaseOp): 576 | 577 | @ATTR(name='dtype', opsets=(9, ), default=0) # v9 add 578 | @ATTR(name='k', opsets=(9, ), default=0) # v9 add 579 | def __init__(self, name='', **kwargs): 580 | 581 | self.node_properties['name'] = name 582 | self.attributes['type'] = 'EyeLike' 583 | check_redundant_attrs(kwargs, self.__class__.__name__) 584 | 585 | 586 | class AttrFlatten(AttrBaseOp): 587 | 588 | @ATTR(name='axis', opsets=(1, ), default=1) # v1 add 589 | def __init__(self, name='', **kwargs): 590 | 591 | self.node_properties['name'] = name 592 | self.attributes['type'] = 'Flatten' 593 | check_redundant_attrs(kwargs, self.__class__.__name__) 594 | 595 | 596 | class AttrFloor(AttrBaseOp): 597 | 598 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 599 | def __init__(self, name='', **kwargs): 600 | 601 | self.node_properties['name'] = name 602 | self.attributes['type'] = 'Floor' 603 | check_redundant_attrs(kwargs, self.__class__.__name__) 604 | 605 | 606 | class AttrGRU(AttrBaseOp): 607 | 608 | @ATTR(name='activation_alpha', opsets=(1, ), default=()) # v1 add 609 | @ATTR(name='activation_beta', opsets=(1, ), default=()) # v1 add 610 | @ATTR(name='activations', opsets=(1, ), default=()) # v1 add 611 | @ATTR(name='clip', opsets=(1, ), default=0) # v1 add 612 | @ATTR(name='direction', opsets=(1, ), default='forward') # v1 add 613 | @ATTR(name='hidden_size', opsets=(1, ), default=1) # v1 add 614 | @ATTR(name='output_sequence', opsets=(1, 7), default=0) # v1 add, v7 delete 615 | @ATTR(name='linear_before_reset', opsets=(3, ), default=0) # v3 add 616 | def __init__(self, name='', **kwargs): 617 | 618 | self.node_properties['name'] = name 619 | self.attributes['type'] = 'GRU' 620 | check_redundant_attrs(kwargs, self.__class__.__name__) 621 | 622 | 623 | class AttrGather(AttrBaseOp): 624 | 625 | @ATTR(name='axis', opsets=(1, ), default=0) # v1 add 626 | def __init__(self, name='', **kwargs): 627 | 628 | self.node_properties['name'] = name 629 | self.attributes['type'] = 'Gather' 630 | check_redundant_attrs(kwargs, self.__class__.__name__) 631 | 632 | 633 | class AttrGatherElements(AttrBaseOp): 634 | 635 | @ATTR(name='axis', opsets=(11, ), default=0) # v11 add 636 | def __init__(self, name='', **kwargs): 637 | 638 | self.node_properties['name'] = name 639 | self.attributes['type'] = 'GatherElements' 640 | check_redundant_attrs(kwargs, self.__class__.__name__) 641 | 642 | 643 | class AttrGatherND(AttrBaseOp): 644 | 645 | @ATTR(name='batch_dims', opsets=(12, ), default=0) # v12 add 646 | def __init__(self, name='', **kwargs): 647 | 648 | self.node_properties['name'] = name 649 | self.attributes['type'] = 'GatherND' 650 | check_redundant_attrs(kwargs, self.__class__.__name__) 651 | 652 | 653 | class AttrGemm(AttrBaseOp): 654 | 655 | @ATTR(name='alpha', opsets=(1, ), default=1.0) # v1 add 656 | @ATTR(name='beta', opsets=(1, ), default=1.0) # v1 add 657 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 658 | @ATTR(name='transA', opsets=(1, ), default=0) # v1 add 659 | @ATTR(name='transB', opsets=(1, ), default=0) # v1 add 660 | def __init__(self, input_size, output_size, bias, name='', **kwargs): 661 | 662 | self.node_properties['name'] = name 663 | self.attributes['type'] = 'Gemm' 664 | check_redundant_attrs(kwargs, self.__class__.__name__) 665 | 666 | # input_size & output_size from weights 667 | self.attributes['input_size'] = input_size 668 | self.attributes['output_size'] = output_size 669 | self.attributes['bias'] = bias 670 | 671 | 672 | class AttrGlobalAveragePool(AttrBaseOp): 673 | 674 | def __init__(self, name='', **kwargs): 675 | self.node_properties['name'] = name 676 | self.attributes['type'] = 'GlobalAveragePool' 677 | 678 | 679 | class AttrGlobalLpPool(AttrBaseOp): 680 | 681 | @ATTR(name='p', opsets=(1, ), default=2) # v1 add 682 | def __init__(self, name='', **kwargs): 683 | 684 | self.node_properties['name'] = name 685 | self.attributes['type'] = 'GlobalLpPool' 686 | check_redundant_attrs(kwargs, self.__class__.__name__) 687 | 688 | 689 | class AttrGlobalMaxPool(AttrBaseOp): 690 | 691 | def __init__(self, name='', **kwargs): 692 | self.node_properties['name'] = name 693 | self.attributes['type'] = 'GlobalMaxPool' 694 | 695 | 696 | class AttrGreater(AttrBaseOp): 697 | 698 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 699 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 700 | def __init__(self, name='', **kwargs): 701 | 702 | self.node_properties['name'] = name 703 | self.attributes['type'] = 'Greater' 704 | check_redundant_attrs(kwargs, self.__class__.__name__) 705 | 706 | 707 | class AttrGreaterOrEqual(AttrBaseOp): 708 | 709 | def __init__(self, name='', **kwargs): 710 | self.node_properties['name'] = name 711 | self.attributes['type'] = 'GreaterOrEqual' 712 | 713 | 714 | class AttrHardSigmoid(AttrBaseOp): 715 | 716 | @ATTR(name='alpha', opsets=(1, ), default=0.2) # v1 add 717 | @ATTR(name='beta', opsets=(1, ), default=0.5) # v1 add 718 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 719 | def __init__(self, name='', **kwargs): 720 | 721 | self.node_properties['name'] = name 722 | self.attributes['type'] = 'HardSigmoid' 723 | check_redundant_attrs(kwargs, self.__class__.__name__) 724 | 725 | 726 | class AttrHardmax(AttrBaseOp): 727 | 728 | @ATTR(name='axis', opsets=( 1, 13), default=1) # v1 add, v13 delete 729 | @ATTR(name='axis', opsets=(13, ), default=-1) # v13 add 730 | def __init__(self, name='', **kwargs): 731 | 732 | self.node_properties['name'] = name 733 | self.attributes['type'] = 'Hardmax' 734 | check_redundant_attrs(kwargs, self.__class__.__name__) 735 | 736 | 737 | class AttrIdentity(AttrBaseOp): 738 | 739 | def __init__(self, name='', **kwargs): 740 | self.node_properties['name'] = name 741 | self.attributes['type'] = 'Identity' 742 | 743 | 744 | class AttrIf(AttrBaseOp): 745 | 746 | @ATTR(name='else_branch', opsets=(1, )) # v1 add 747 | @ATTR(name='then_branch', opsets=(1, )) # v1 add 748 | def __init__(self, name='', **kwargs): 749 | 750 | self.node_properties['name'] = name 751 | self.attributes['type'] = 'If' 752 | check_redundant_attrs(kwargs, self.__class__.__name__) 753 | 754 | 755 | class AttrInstanceNormalization(AttrBaseOp): 756 | 757 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 758 | @ATTR(name='epsilon', opsets=(1, ), default=1e-05) # v1 add 759 | def __init__(self, scale_size, B_size, name='', **kwargs): 760 | 761 | self.node_properties['name'] = name 762 | self.attributes['type'] = 'InstanceNormalization' 763 | check_redundant_attrs(kwargs, self.__class__.__name__) 764 | 765 | # sizes of scale, B from weights 766 | self.attributes['scale_size'] = scale_size 767 | self.attributes['B_size'] = B_size 768 | 769 | 770 | class AttrIsInf(AttrBaseOp): 771 | 772 | @ATTR(name='detect_negative', opsets=(10, ), default=1) # v10 add 773 | @ATTR(name='detect_positive', opsets=(10, ), default=1) # v10 add 774 | def __init__(self, name='', **kwargs): 775 | 776 | self.node_properties['name'] = name 777 | self.attributes['type'] = 'IsInf' 778 | check_redundant_attrs(kwargs, self.__class__.__name__) 779 | 780 | 781 | class AttrIsNaN(AttrBaseOp): 782 | 783 | def __init__(self, name='', **kwargs): 784 | self.node_properties['name'] = name 785 | self.attributes['type'] = 'IsNaN' 786 | 787 | 788 | class AttrLRN(AttrBaseOp): 789 | 790 | @ATTR(name='alpha', opsets=(1, ), default=0.0001) # v1 add 791 | @ATTR(name='beta', opsets=(1, ), default=0.75) # v1 add 792 | @ATTR(name='bias', opsets=(1, ), default=1.0) # v1 add 793 | @ATTR(name='size', opsets=(1, )) # v1 add, required 794 | def __init__(self, name='', **kwargs): 795 | 796 | self.node_properties['name'] = name 797 | self.attributes['type'] = 'LRN' 798 | check_redundant_attrs(kwargs, self.__class__.__name__) 799 | 800 | 801 | class AttrLSTM(AttrBaseOp): 802 | 803 | @ATTR(name='activation_alpha', opsets=(1, ), default=()) # v1 add 804 | @ATTR(name='activation_beta', opsets=(1, ), default=()) # v1 add 805 | @ATTR(name='activations', opsets=(1, ), default=()) # v1 add 806 | @ATTR(name='clip', opsets=(1, ), default=0) # v1 add 807 | @ATTR(name='direction', opsets=(1, ), default='forward') # v1 add 808 | @ATTR(name='hidden_size', opsets=(1, ), default=1) # v1 add 809 | @ATTR(name='input_forget', opsets=(1, ), default=0) # v1 add 810 | @ATTR(name='output_sequence', opsets=(1, 7), default=0) # v1 add, v7 delete 811 | def __init__(self, name='', **kwargs): 812 | 813 | self.node_properties['name'] = name 814 | self.attributes['type'] = 'LSTM' 815 | check_redundant_attrs(kwargs, self.__class__.__name__) 816 | 817 | 818 | class AttrLeakyRelu(AttrBaseOp): 819 | 820 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 821 | @ATTR(name='alpha', opsets=(1, ), default=0.01) # v1 add 822 | def __init__(self, name='', **kwargs): 823 | 824 | self.node_properties['name'] = name 825 | self.attributes['type'] = 'LeakyRelu' 826 | check_redundant_attrs(kwargs, self.__class__.__name__) 827 | 828 | 829 | class AttrLess(AttrBaseOp): 830 | 831 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 832 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 833 | def __init__(self, name='', **kwargs): 834 | 835 | self.node_properties['name'] = name 836 | self.attributes['type'] = 'Less' 837 | check_redundant_attrs(kwargs, self.__class__.__name__) 838 | 839 | 840 | class AttrLessOrEqual(AttrBaseOp): 841 | 842 | def __init__(self, name='', **kwargs): 843 | self.node_properties['name'] = name 844 | self.attributes['type'] = 'LessOrEqual' 845 | 846 | 847 | class AttrLog(AttrBaseOp): 848 | 849 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 850 | def __init__(self, name='', **kwargs): 851 | 852 | self.node_properties['name'] = name 853 | self.attributes['type'] = 'Log' 854 | check_redundant_attrs(kwargs, self.__class__.__name__) 855 | 856 | 857 | class AttrLogSoftmax(AttrBaseOp): 858 | 859 | @ATTR(name='axis', opsets=( 1, 13), default=1) # v1 add, v13 delete 860 | @ATTR(name='axis', opsets=(13, ), default=-1) # v13 add 861 | def __init__(self, name='', **kwargs): 862 | 863 | self.node_properties['name'] = name 864 | self.attributes['type'] = 'LogSoftmax' 865 | check_redundant_attrs(kwargs, self.__class__.__name__) 866 | 867 | 868 | class AttrLoop(AttrBaseOp): 869 | 870 | @ATTR(name='body', opsets=(1, )) # v1 add, required 871 | def __init__(self, name='', **kwargs): 872 | 873 | self.node_properties['name'] = name 874 | self.attributes['type'] = 'Loop' 875 | check_redundant_attrs(kwargs, self.__class__.__name__) 876 | 877 | 878 | class AttrLpNormalization(AttrBaseOp): 879 | 880 | @ATTR(name='axis', opsets=(1, ), default=-1) # v1 add 881 | @ATTR(name='p', opsets=(1, ), default=2) # v1 add 882 | def __init__(self, name='', **kwargs): 883 | 884 | self.node_properties['name'] = name 885 | self.attributes['type'] = 'LpNormalization' 886 | check_redundant_attrs(kwargs, self.__class__.__name__) 887 | 888 | 889 | class AttrLpPool(AttrBaseOp): 890 | 891 | @ATTR(name='auto_pad', opsets=(1, ), default='NOTSET') # v1 add 892 | @ATTR(name='kernel_shape', opsets=(1, ), fmt_func=FSize.kernel) # v1 add, required 893 | @ATTR(name='p', opsets=(1, ), default=2.0) # v1 add 894 | @ATTR(name='pads', opsets=(1, ), default=0, fmt_func=FSize.padding) # v1 add 895 | @ATTR(name='strides', opsets=(1, ), default=1, fmt_func=FSize.stride) # v1 add 896 | def __init__(self, name='', **kwargs): 897 | 898 | self.node_properties['name'] = name 899 | self.attributes['type'] = 'LpPool' 900 | check_redundant_attrs(kwargs, self.__class__.__name__) 901 | 902 | 903 | class AttrMatMul(AttrBaseOp): 904 | 905 | def __init__(self, name='', **kwargs): 906 | self.node_properties['name'] = name 907 | self.attributes['type'] = 'MatMul' 908 | 909 | 910 | class AttrMatMulInteger(AttrBaseOp): 911 | 912 | def __init__(self, name='', **kwargs): 913 | self.node_properties['name'] = name 914 | self.attributes['type'] = 'MatMulInteger' 915 | 916 | 917 | class AttrMax(AttrBaseOp): 918 | 919 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 920 | def __init__(self, name='', **kwargs): 921 | 922 | self.node_properties['name'] = name 923 | self.attributes['type'] = 'Max' 924 | check_redundant_attrs(kwargs, self.__class__.__name__) 925 | 926 | 927 | class AttrMaxPool(AttrBaseOp): 928 | 929 | @ATTR(name='auto_pad', opsets=( 1, ), default='NOTSET') # v1 add 930 | @ATTR(name='kernel_shape', opsets=( 1, ), fmt_func=FSize.kernel) # v1 add, required 931 | @ATTR(name='pads', opsets=( 1, ), default=0, fmt_func=FSize.padding) # v1 add 932 | @ATTR(name='strides', opsets=( 1, ), default=1, fmt_func=FSize.stride) # v1 add 933 | @ATTR(name='storage_order', opsets=( 8, ), default=0) # v8 add 934 | @ATTR(name='ceil_mode', opsets=(10, ), default=0) # v10 add 935 | @ATTR(name='dilations', opsets=(10, ), default=1, fmt_func=FSize.dilation) # v10 add 936 | def __init__(self, name='', **kwargs): 937 | 938 | self.node_properties['name'] = name 939 | self.attributes['type'] = 'MaxPool' 940 | check_redundant_attrs(kwargs, self.__class__.__name__) 941 | 942 | 943 | class AttrMaxRoiPool(AttrBaseOp): 944 | 945 | @ATTR(name='pooled_shape', opsets=(1, )) # v1 add 946 | @ATTR(name='spatial_scale', opsets=(1, ), default=1.0) # v1 add, required 947 | def __init__(self, name='', **kwargs): 948 | 949 | self.node_properties['name'] = name 950 | self.attributes['type'] = 'MaxRoiPool' 951 | check_redundant_attrs(kwargs, self.__class__.__name__) 952 | 953 | 954 | class AttrMaxUnpool(AttrBaseOp): 955 | 956 | @ATTR(name='kernel_shape', opsets=(9, )) # v9 add 957 | @ATTR(name='pads', opsets=(9, ), default=0) # v9 add 958 | @ATTR(name='strides', opsets=(9, ), default=1) # v9 add 959 | def __init__(self, name='', **kwargs): 960 | 961 | self.node_properties['name'] = name 962 | self.attributes['type'] = 'MaxUnpool' 963 | check_redundant_attrs(kwargs, self.__class__.__name__) 964 | 965 | 966 | class AttrMean(AttrBaseOp): 967 | 968 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 969 | def __init__(self, name='', **kwargs): 970 | 971 | self.node_properties['name'] = name 972 | self.attributes['type'] = 'Mean' 973 | check_redundant_attrs(kwargs, self.__class__.__name__) 974 | 975 | 976 | class AttrMeanVarianceNormalization(AttrBaseOp): 977 | 978 | @ATTR(name='axes', opsets=(9, ), default=[0, 2, 3]) # v9 add 979 | def __init__(self, name='', **kwargs): 980 | 981 | self.node_properties['name'] = name 982 | self.attributes['type'] = 'MeanVarianceNormalization' 983 | check_redundant_attrs(kwargs, self.__class__.__name__) 984 | 985 | 986 | class AttrMin(AttrBaseOp): 987 | 988 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 989 | def __init__(self, name='', **kwargs): 990 | 991 | self.node_properties['name'] = name 992 | self.attributes['type'] = 'Min' 993 | check_redundant_attrs(kwargs, self.__class__.__name__) 994 | 995 | 996 | class AttrMod(AttrBaseOp): 997 | 998 | @ATTR(name='fmod', opsets=(10, ), default=0) # v10 add, v6 delete 999 | def __init__(self, name='', **kwargs): 1000 | 1001 | self.node_properties['name'] = name 1002 | self.attributes['type'] = 'Mod' 1003 | check_redundant_attrs(kwargs, self.__class__.__name__) 1004 | 1005 | 1006 | class AttrMul(AttrBaseOp): 1007 | 1008 | @ATTR(name='axis', opsets=(1, 7)) # v1 add, v7 delete, required 1009 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 1010 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1011 | def __init__(self, name='', **kwargs): 1012 | 1013 | self.node_properties['name'] = name 1014 | self.attributes['type'] = 'Mul' 1015 | check_redundant_attrs(kwargs, self.__class__.__name__) 1016 | 1017 | 1018 | class AttrMultinomial(AttrBaseOp): 1019 | 1020 | @ATTR(name='dtype', opsets=(7, ), default=6) # v7 add 1021 | @ATTR(name='sample_size', opsets=(7, ), default=1) # v7 add 1022 | @ATTR(name='seed', opsets=(7, ), default=0.0) # v7 add 1023 | def __init__(self, name='', **kwargs): 1024 | 1025 | self.node_properties['name'] = name 1026 | self.attributes['type'] = 'Multinomial' 1027 | check_redundant_attrs(kwargs, self.__class__.__name__) 1028 | 1029 | 1030 | class AttrNeg(AttrBaseOp): 1031 | 1032 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1033 | def __init__(self, name='', **kwargs): 1034 | 1035 | self.node_properties['name'] = name 1036 | self.attributes['type'] = 'Neg' 1037 | check_redundant_attrs(kwargs, self.__class__.__name__) 1038 | 1039 | 1040 | class AttrNegativeLogLikelihoodLoss(AttrBaseOp): 1041 | 1042 | @ATTR(name='ignore_index', opsets=(12, ), default=-1) # v12 add 1043 | @ATTR(name='reduction', opsets=(12, ), default='mean') # v12 add 1044 | def __init__(self, name='', **kwargs): 1045 | 1046 | self.node_properties['name'] = name 1047 | self.attributes['type'] = 'NegativeLogLikelihoodLoss' 1048 | check_redundant_attrs(kwargs, self.__class__.__name__) 1049 | 1050 | 1051 | class AttrNonMaxSuppression(AttrBaseOp): 1052 | 1053 | @ATTR(name='center_point_box', opsets=(10, ), default=0) # v10 add 1054 | def __init__(self, name='', **kwargs): 1055 | 1056 | self.node_properties['name'] = name 1057 | self.attributes['type'] = 'NonMaxSuppression' 1058 | check_redundant_attrs(kwargs, self.__class__.__name__) 1059 | 1060 | 1061 | class AttrNonZero(AttrBaseOp): 1062 | 1063 | def __init__(self, name='', **kwargs): 1064 | self.node_properties['name'] = name 1065 | self.attributes['type'] = 'NonZero' 1066 | 1067 | 1068 | class AttrNot(AttrBaseOp): 1069 | 1070 | def __init__(self, name='', **kwargs): 1071 | self.node_properties['name'] = name 1072 | self.attributes['type'] = 'Not' 1073 | 1074 | 1075 | class AttrOneHot(AttrBaseOp): 1076 | 1077 | @ATTR(name='axis', opsets=(9, ), default=-1) # v9 add 1078 | def __init__(self, name='', **kwargs): 1079 | 1080 | self.node_properties['name'] = name 1081 | self.attributes['type'] = 'OneHot' 1082 | check_redundant_attrs(kwargs, self.__class__.__name__) 1083 | 1084 | 1085 | class AttrOr(AttrBaseOp): 1086 | 1087 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 1088 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 1089 | def __init__(self, name='', **kwargs): 1090 | 1091 | self.node_properties['name'] = name 1092 | self.attributes['type'] = 'Or' 1093 | check_redundant_attrs(kwargs, self.__class__.__name__) 1094 | 1095 | 1096 | class AttrPRelu(AttrBaseOp): 1097 | 1098 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1099 | def __init__(self, name='', **kwargs): 1100 | 1101 | self.node_properties['name'] = name 1102 | self.attributes['type'] = 'PRelu' 1103 | check_redundant_attrs(kwargs, self.__class__.__name__) 1104 | 1105 | 1106 | class AttrPad(AttrBaseOp): 1107 | 1108 | @ATTR(name='mode', opsets=(1, ), default='constant') # v1 add 1109 | @ATTR(name='paddings', opsets=(1, 2), fmt_func=FSize.padding) # v1 add, required 1110 | @ATTR(name='pads', opsets=(2, 11), fmt_func=FSize.padding) # v1 add, required 1111 | @ATTR(name='value', opsets=(1, 11), default=0.0) # v1 add 1112 | def __init__(self, name='', **kwargs): 1113 | 1114 | self.node_properties['name'] = name 1115 | self.attributes['type'] = 'Pad' 1116 | check_redundant_attrs(kwargs, self.__class__.__name__) 1117 | 1118 | 1119 | class AttrPow(AttrBaseOp): 1120 | 1121 | @ATTR(name='axis', opsets=(1, 7), default=0) # v1 add, v7 delete 1122 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 1123 | def __init__(self, name='', **kwargs): 1124 | 1125 | self.node_properties['name'] = name 1126 | self.attributes['type'] = 'Pow' 1127 | check_redundant_attrs(kwargs, self.__class__.__name__) 1128 | 1129 | 1130 | class AttrQLinearConv(AttrBaseOp): 1131 | 1132 | @ATTR(name='auto_pad', opsets=(10, ), default='NOTSET') # v10 add 1133 | @ATTR(name='dilations', opsets=(10, ), default=1, fmt_func=FSize.dilation) # v10 add 1134 | @ATTR(name='group', opsets=(10, ), default=1) # v10 add 1135 | @ATTR(name='kernel_shape', opsets=(10, ), fmt_func=FSize.kernel) # v10 add, required 1136 | @ATTR(name='pads', opsets=(10, ), default=0, fmt_func=FSize.padding) # v10 add 1137 | @ATTR(name='strides', opsets=(10, ), default=1, fmt_func=FSize.stride) # v10 add 1138 | def __init__(self, input_channel, output_channel, name='', **kwargs): 1139 | 1140 | self.node_properties['name'] = name 1141 | self.attributes['type'] = 'QLinearConv' 1142 | check_redundant_attrs(kwargs, self.__class__.__name__) 1143 | 1144 | # input_channel & output_channel from weights 1145 | self.attributes['input_channel'] = input_channel 1146 | self.attributes['output_channel'] = output_channel 1147 | 1148 | 1149 | class AttrQLinearMatMul(AttrBaseOp): 1150 | 1151 | def __init__(self, name='', **kwargs): 1152 | self.node_properties['name'] = name 1153 | self.attributes['type'] = 'QLinearMatMul' 1154 | 1155 | 1156 | class AttrQuantizeLinear(AttrBaseOp): 1157 | 1158 | @ATTR(name='axis', opsets=(13, ), default=1) # v13 add 1159 | def __init__(self, name='', **kwargs): 1160 | 1161 | self.node_properties['name'] = name 1162 | self.attributes['type'] = 'QuantizeLinear' 1163 | check_redundant_attrs(kwargs, self.__class__.__name__) 1164 | 1165 | 1166 | class AttrRNN(AttrBaseOp): 1167 | 1168 | @ATTR(name='activation_alpha', opsets=(1, ), default=()) # v1 add 1169 | @ATTR(name='activation_beta', opsets=(1, ), default=()) # v1 add 1170 | @ATTR(name='activations', opsets=(1, ), default=()) # v1 add 1171 | @ATTR(name='clip', opsets=(1, ), default=0) # v1 add 1172 | @ATTR(name='direction', opsets=(1, ), default='forward') # v1 add 1173 | @ATTR(name='hidden_size', opsets=(1, ), default=1) # v1 add 1174 | @ATTR(name='output_sequence', opsets=(1, 7), default=0) # v1 add, v7 delete 1175 | def __init__(self, name='', **kwargs): 1176 | 1177 | self.node_properties['name'] = name 1178 | self.attributes['type'] = 'RNN' 1179 | check_redundant_attrs(kwargs, self.__class__.__name__) 1180 | 1181 | 1182 | class AttrRandomNormal(AttrBaseOp): 1183 | 1184 | @ATTR(name='dtype', opsets=(1, ), default=1) # v1 add 1185 | @ATTR(name='mean', opsets=(1, ), default=0.0) # v1 add 1186 | @ATTR(name='scale', opsets=(1, ), default=1.0) # v1 add 1187 | @ATTR(name='seed', opsets=(1, ), default=1.0) # v1 add 1188 | @ATTR(name='shape', opsets=(1, )) # v1 add, required 1189 | def __init__(self, name='', **kwargs): 1190 | 1191 | self.node_properties['name'] = name 1192 | self.attributes['type'] = 'RandomNormal' 1193 | check_redundant_attrs(kwargs, self.__class__.__name__) 1194 | 1195 | 1196 | class AttrRandomNormalLike(AttrBaseOp): 1197 | 1198 | @ATTR(name='dtype', opsets=(1, ), default=-1) # v1 add 1199 | @ATTR(name='mean', opsets=(1, ), default=0.0) # v1 add 1200 | @ATTR(name='scale', opsets=(1, ), default=1.0) # v1 add 1201 | @ATTR(name='seed', opsets=(1, ), default=1.0) # v1 add 1202 | def __init__(self, name='', **kwargs): 1203 | 1204 | self.node_properties['name'] = name 1205 | self.attributes['type'] = 'RandomNormalLike' 1206 | check_redundant_attrs(kwargs, self.__class__.__name__) 1207 | 1208 | 1209 | class AttrRandomUniform(AttrBaseOp): 1210 | 1211 | @ATTR(name='dtype', opsets=(1, ), default=1) # v1 add 1212 | @ATTR(name='high', opsets=(1, ), default=1.0) # v1 add 1213 | @ATTR(name='low', opsets=(1, ), default=0.0) # v1 add 1214 | @ATTR(name='seed', opsets=(1, ), default=1.0) # v1 add 1215 | @ATTR(name='shape', opsets=(1, )) # v1 add, required 1216 | def __init__(self, name='', **kwargs): 1217 | 1218 | self.node_properties['name'] = name 1219 | self.attributes['type'] = 'RandomUniform' 1220 | check_redundant_attrs(kwargs, self.__class__.__name__) 1221 | 1222 | 1223 | class AttrRandomUniformLike(AttrBaseOp): 1224 | 1225 | @ATTR(name='dtype', opsets=(1, ), default=-1) # v1 add 1226 | @ATTR(name='high', opsets=(1, ), default=1.0) # v1 add 1227 | @ATTR(name='low', opsets=(1, ), default=0.0) # v1 add 1228 | @ATTR(name='seed', opsets=(1, ), default=1.0) # v1 add 1229 | def __init__(self, name='', **kwargs): 1230 | 1231 | self.node_properties['name'] = name 1232 | self.attributes['type'] = 'RandomUniformLike' 1233 | check_redundant_attrs(kwargs, self.__class__.__name__) 1234 | 1235 | 1236 | class AttrRange(AttrBaseOp): 1237 | 1238 | def __init__(self, name='', **kwargs): 1239 | self.node_properties['name'] = name 1240 | self.attributes['type'] = 'Range' 1241 | 1242 | 1243 | class AttrReciprocal(AttrBaseOp): 1244 | 1245 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1246 | def __init__(self, name='', **kwargs): 1247 | 1248 | self.node_properties['name'] = name 1249 | self.attributes['type'] = 'Reciprocal' 1250 | check_redundant_attrs(kwargs, self.__class__.__name__) 1251 | 1252 | 1253 | class AttrReduceL1(AttrBaseOp): 1254 | 1255 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1256 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1257 | def __init__(self, name='', **kwargs): 1258 | 1259 | self.node_properties['name'] = name 1260 | self.attributes['type'] = 'ReduceL1' 1261 | check_redundant_attrs(kwargs, self.__class__.__name__) 1262 | 1263 | 1264 | class AttrReduceL2(AttrBaseOp): 1265 | 1266 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1267 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1268 | def __init__(self, name='', **kwargs): 1269 | 1270 | self.node_properties['name'] = name 1271 | self.attributes['type'] = 'ReduceL2' 1272 | check_redundant_attrs(kwargs, self.__class__.__name__) 1273 | 1274 | 1275 | class AttrReduceLogSum(AttrBaseOp): 1276 | 1277 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1278 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1279 | def __init__(self, name='', **kwargs): 1280 | 1281 | self.node_properties['name'] = name 1282 | self.attributes['type'] = 'ReduceLogSum' 1283 | check_redundant_attrs(kwargs, self.__class__.__name__) 1284 | 1285 | 1286 | class AttrReduceLogSumExp(AttrBaseOp): 1287 | 1288 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1289 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1290 | def __init__(self, name='', **kwargs): 1291 | 1292 | self.node_properties['name'] = name 1293 | self.attributes['type'] = 'ReduceLogSumExp' 1294 | check_redundant_attrs(kwargs, self.__class__.__name__) 1295 | 1296 | 1297 | class AttrReduceMax(AttrBaseOp): 1298 | 1299 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1300 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1301 | def __init__(self, name='', **kwargs): 1302 | 1303 | self.node_properties['name'] = name 1304 | self.attributes['type'] = 'ReduceMax' 1305 | check_redundant_attrs(kwargs, self.__class__.__name__) 1306 | 1307 | 1308 | class AttrReduceMean(AttrBaseOp): 1309 | 1310 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1311 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1312 | def __init__(self, name='', **kwargs): 1313 | 1314 | self.node_properties['name'] = name 1315 | self.attributes['type'] = 'ReduceMean' 1316 | check_redundant_attrs(kwargs, self.__class__.__name__) 1317 | 1318 | 1319 | class AttrReduceMin(AttrBaseOp): 1320 | 1321 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1322 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1323 | def __init__(self, name='', **kwargs): 1324 | 1325 | self.node_properties['name'] = name 1326 | self.attributes['type'] = 'ReduceMin' 1327 | check_redundant_attrs(kwargs, self.__class__.__name__) 1328 | 1329 | 1330 | class AttrReduceProd(AttrBaseOp): 1331 | 1332 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1333 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1334 | def __init__(self, name='', **kwargs): 1335 | 1336 | self.node_properties['name'] = name 1337 | self.attributes['type'] = 'ReduceProd' 1338 | check_redundant_attrs(kwargs, self.__class__.__name__) 1339 | 1340 | 1341 | class AttrReduceSum(AttrBaseOp): 1342 | 1343 | @ATTR(name='axes', opsets=(1, 13), default=()) # v1 add, v13 delete 1344 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1345 | @ATTR(name='noop_with_empty_axes', opsets=(13, ), default=0) # v13 add 1346 | def __init__(self, name='', **kwargs): 1347 | 1348 | self.node_properties['name'] = name 1349 | self.attributes['type'] = 'ReduceSum' 1350 | check_redundant_attrs(kwargs, self.__class__.__name__) 1351 | 1352 | 1353 | class AttrReduceSumSquare(AttrBaseOp): 1354 | 1355 | @ATTR(name='axes', opsets=(1, ), default=()) # v1 add 1356 | @ATTR(name='keepdims', opsets=(1, ), default=1) # v1 add 1357 | def __init__(self, name='', **kwargs): 1358 | 1359 | self.node_properties['name'] = name 1360 | self.attributes['type'] = 'ReduceSumSquare' 1361 | check_redundant_attrs(kwargs, self.__class__.__name__) 1362 | 1363 | 1364 | class AttrRelu(AttrBaseOp): 1365 | 1366 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1367 | def __init__(self, name='', **kwargs): 1368 | 1369 | self.node_properties['name'] = name 1370 | self.attributes['type'] = 'Relu' 1371 | check_redundant_attrs(kwargs, self.__class__.__name__) 1372 | 1373 | 1374 | class AttrReshape(AttrBaseOp): 1375 | 1376 | @ATTR(name='shape', opsets=(1, 5), default=()) # v1 add, v5 delete 1377 | @ATTR(name='consumed_inputs', opsets=(1, 5), default=()) # v1 add, v5 delete 1378 | def __init__(self, shape, name='', **kwargs): 1379 | 1380 | self.node_properties['name'] = name 1381 | self.attributes['type'] = 'Reshape' 1382 | check_redundant_attrs(kwargs, self.__class__.__name__) 1383 | 1384 | # shape from inputs 1385 | self.attributes['shape'] = shape 1386 | 1387 | 1388 | 1389 | class AttrResize(AttrBaseOp): 1390 | 1391 | @ATTR(name='mode', opsets=(10, ), default='nearest') # v10 add 1392 | @ATTR(name='coordinate_transformation_mode', opsets=(11, ), default='half_pixel') # v10 add 1393 | @ATTR(name='cubic_coeff_a', opsets=(11, ), default=-0.75) # v10 add 1394 | @ATTR(name='exclude_outside', opsets=(11, ), default=0) # v10 add 1395 | @ATTR(name='extrapolation_value', opsets=(11, ), default=0.0) # v10 add 1396 | @ATTR(name='nearest_mode', opsets=(11, ), default='round_prefer_floor') # v10 add 1397 | def __init__(self, name='', **kwargs): 1398 | 1399 | self.node_properties['name'] = name 1400 | self.attributes['type'] = 'Resize' 1401 | check_redundant_attrs(kwargs, self.__class__.__name__) 1402 | 1403 | 1404 | class AttrReverseSequence(AttrBaseOp): 1405 | 1406 | @ATTR(name='batch_axis', opsets=(10, ), default=1) # v10 add 1407 | @ATTR(name='time_axis', opsets=(10, ), default=0) # v10 add 1408 | def __init__(self, name='', **kwargs): 1409 | 1410 | self.node_properties['name'] = name 1411 | self.attributes['type'] = 'ReverseSequence' 1412 | check_redundant_attrs(kwargs, self.__class__.__name__) 1413 | 1414 | 1415 | class AttrRoiAlign(AttrBaseOp): 1416 | 1417 | @ATTR(name='mode', opsets=(10, ), default='avg') # v10 add 1418 | @ATTR(name='output_height', opsets=(10, ), default=1) # v10 add 1419 | @ATTR(name='output_width', opsets=(10, ), default=1) # v10 add 1420 | @ATTR(name='sampling_ratio', opsets=(10, ), default=0) # v10 add 1421 | @ATTR(name='spatial_scale', opsets=(10, ), default=1.0) # v10 add 1422 | def __init__(self, name='', **kwargs): 1423 | 1424 | self.node_properties['name'] = name 1425 | self.attributes['type'] = 'RoiAlign' 1426 | check_redundant_attrs(kwargs, self.__class__.__name__) 1427 | 1428 | 1429 | class AttrRound(AttrBaseOp): 1430 | 1431 | def __init__(self, name='', **kwargs): 1432 | self.node_properties['name'] = name 1433 | self.attributes['type'] = 'Round' 1434 | 1435 | 1436 | class AttrScan(AttrBaseOp): 1437 | 1438 | @ATTR(name='body', opsets=(8, )) # v8 add, required 1439 | @ATTR(name='directions', opsets=(8, 9), default=()) # v8 add, v9 delete 1440 | @ATTR(name='num_scan_inputs', opsets=(8, )) # v8 add, required 1441 | @ATTR(name='scan_input_axes', opsets=(9, ), default=()) # v9 add 1442 | @ATTR(name='scan_input_directions', opsets=(9, ), default=()) # v9 add 1443 | @ATTR(name='scan_output_axes', opsets=(9, ), default=()) # v9 add 1444 | @ATTR(name='scan_output_directions', opsets=(9, ), default=()) # v9 add 1445 | def __init__(self, name='', **kwargs): 1446 | 1447 | self.node_properties['name'] = name 1448 | self.attributes['type'] = 'Scan' 1449 | check_redundant_attrs(kwargs, self.__class__.__name__) 1450 | 1451 | 1452 | class AttrScatter(AttrBaseOp): 1453 | 1454 | @ATTR(name='axis', opsets=(9, 11), default=0) # v9 add, v11 delete 1455 | def __init__(self, name='', **kwargs): 1456 | 1457 | self.node_properties['name'] = name 1458 | self.attributes['type'] = 'Scatter' 1459 | check_redundant_attrs(kwargs, self.__class__.__name__) 1460 | 1461 | 1462 | class AttrScatterElements(AttrBaseOp): 1463 | 1464 | @ATTR(name='axis', opsets=(11, ), default=0) # v9 add 1465 | def __init__(self, name='', **kwargs): 1466 | 1467 | self.node_properties['name'] = name 1468 | self.attributes['type'] = 'ScatterElements' 1469 | check_redundant_attrs(kwargs, self.__class__.__name__) 1470 | 1471 | 1472 | class AttrScatterND(AttrBaseOp): 1473 | 1474 | def __init__(self, name='', **kwargs): 1475 | self.node_properties['name'] = name 1476 | self.attributes['type'] = 'ScatterND' 1477 | 1478 | 1479 | class AttrSelu(AttrBaseOp): 1480 | 1481 | @ATTR(name='alpha', opsets=(1, ), default=1.67326) # v1 add 1482 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1483 | @ATTR(name='gamma', opsets=(1, ), default=1.0507) # v1 add 1484 | def __init__(self, name='', **kwargs): 1485 | 1486 | self.node_properties['name'] = name 1487 | self.attributes['type'] = 'Selu' 1488 | check_redundant_attrs(kwargs, self.__class__.__name__) 1489 | 1490 | 1491 | class AttrSequenceAt(AttrBaseOp): 1492 | 1493 | def __init__(self, name='', **kwargs): 1494 | self.node_properties['name'] = name 1495 | self.attributes['type'] = 'SequenceAt' 1496 | 1497 | 1498 | class AttrSequenceConstruct(AttrBaseOp): 1499 | 1500 | def __init__(self, name='', **kwargs): 1501 | self.node_properties['name'] = name 1502 | self.attributes['type'] = 'SequenceConstruct' 1503 | 1504 | 1505 | class AttrSequenceEmpty(AttrBaseOp): 1506 | 1507 | @ATTR(name='dtype', opsets=(11, ), default=-1) # v11 add 1508 | def __init__(self, name='', **kwargs): 1509 | 1510 | self.node_properties['name'] = name 1511 | self.attributes['type'] = 'SequenceEmpty' 1512 | check_redundant_attrs(kwargs, self.__class__.__name__) 1513 | 1514 | 1515 | class AttrSequenceErase(AttrBaseOp): 1516 | 1517 | def __init__(self, name='', **kwargs): 1518 | self.node_properties['name'] = name 1519 | self.attributes['type'] = 'SequenceErase' 1520 | 1521 | 1522 | class AttrSequenceInsert(AttrBaseOp): 1523 | 1524 | def __init__(self, name='', **kwargs): 1525 | self.node_properties['name'] = name 1526 | self.attributes['type'] = 'SequenceInsert' 1527 | 1528 | 1529 | class AttrSequenceLength(AttrBaseOp): 1530 | 1531 | def __init__(self, name='', **kwargs): 1532 | self.node_properties['name'] = name 1533 | self.attributes['type'] = 'SequenceLength' 1534 | 1535 | 1536 | class AttrShape(AttrBaseOp): 1537 | 1538 | def __init__(self, name='', **kwargs): 1539 | self.node_properties['name'] = name 1540 | self.attributes['type'] = 'Shape' 1541 | 1542 | 1543 | class AttrShrink(AttrBaseOp): 1544 | 1545 | @ATTR(name='bias', opsets=(9, ), default=0.0) # v9 add 1546 | @ATTR(name='lambd', opsets=(9, ), default=0.5) # v9 add 1547 | def __init__(self, name='', **kwargs): 1548 | 1549 | self.node_properties['name'] = name 1550 | self.attributes['type'] = 'Shrink' 1551 | check_redundant_attrs(kwargs, self.__class__.__name__) 1552 | 1553 | 1554 | class AttrSigmoid(AttrBaseOp): 1555 | 1556 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1557 | def __init__(self, name='', **kwargs): 1558 | 1559 | self.node_properties['name'] = name 1560 | self.attributes['type'] = 'Sigmoid' 1561 | check_redundant_attrs(kwargs, self.__class__.__name__) 1562 | 1563 | 1564 | class AttrSign(AttrBaseOp): 1565 | 1566 | def __init__(self, name='', **kwargs): 1567 | self.node_properties['name'] = name 1568 | self.attributes['type'] = 'Sign' 1569 | 1570 | 1571 | class AttrSin(AttrBaseOp): 1572 | 1573 | def __init__(self, name='', **kwargs): 1574 | self.node_properties['name'] = name 1575 | self.attributes['type'] = 'Sin' 1576 | 1577 | 1578 | class AttrSinh(AttrBaseOp): 1579 | 1580 | def __init__(self, name='', **kwargs): 1581 | self.node_properties['name'] = name 1582 | self.attributes['type'] = 'Sinh' 1583 | 1584 | 1585 | class AttrSize(AttrBaseOp): 1586 | 1587 | def __init__(self, name='', **kwargs): 1588 | self.node_properties['name'] = name 1589 | self.attributes['type'] = 'Size' 1590 | 1591 | 1592 | class AttrSlice(AttrBaseOp): 1593 | 1594 | @ATTR(name='axes', opsets=(1, 10), default=()) # v1 add, v10 delete 1595 | @ATTR(name='ends', opsets=(1, 10)) # v1 add, required, v10 delete 1596 | @ATTR(name='starts', opsets=(1, 10)) # v1 add, required, v10 delete 1597 | def __init__(self, name='', **kwargs): 1598 | 1599 | self.node_properties['name'] = name 1600 | self.attributes['type'] = 'Slice' 1601 | check_redundant_attrs(kwargs, self.__class__.__name__) 1602 | 1603 | 1604 | class AttrSoftmax(AttrBaseOp): 1605 | 1606 | @ATTR(name='axis', opsets=( 1, 13), default=1) # v1 add, v13 delete 1607 | @ATTR(name='axis', opsets=(13, ), default=-1) # v13 add 1608 | def __init__(self, name='', **kwargs): 1609 | 1610 | self.node_properties['name'] = name 1611 | self.attributes['type'] = 'Softmax' 1612 | check_redundant_attrs(kwargs, self.__class__.__name__) 1613 | 1614 | 1615 | class AttrSoftmaxCrossEntropyLoss(AttrBaseOp): 1616 | 1617 | @ATTR(name='ignore_index', opsets=(12, ), default=-1) # v12 add 1618 | @ATTR(name='reduction', opsets=(12, ), default='mean') # v12 add 1619 | def __init__(self, name='', **kwargs): 1620 | 1621 | self.node_properties['name'] = name 1622 | self.attributes['type'] = 'SoftmaxCrossEntropyLoss' 1623 | check_redundant_attrs(kwargs, self.__class__.__name__) 1624 | 1625 | 1626 | class AttrSoftplus(AttrBaseOp): 1627 | 1628 | def __init__(self, name='', **kwargs): 1629 | self.node_properties['name'] = name 1630 | self.attributes['type'] = 'Softplus' 1631 | 1632 | 1633 | class AttrSoftsign(AttrBaseOp): 1634 | 1635 | def __init__(self, name='', **kwargs): 1636 | self.node_properties['name'] = name 1637 | self.attributes['type'] = 'Softsign' 1638 | 1639 | 1640 | class AttrSpaceToDepth(AttrBaseOp): 1641 | 1642 | @ATTR(name='blocksize', opsets=(1, )) # v1 add, required 1643 | def __init__(self, name='', **kwargs): 1644 | 1645 | self.node_properties['name'] = name 1646 | self.attributes['type'] = 'SpaceToDepth' 1647 | check_redundant_attrs(kwargs, self.__class__.__name__) 1648 | 1649 | 1650 | class AttrSplit(AttrBaseOp): 1651 | 1652 | @ATTR(name='axis', opsets=(1, ), default=0) # v1 add 1653 | @ATTR(name='split', opsets=(1, 13), default=()) # v1 add, v13 delete 1654 | def __init__(self, name='', **kwargs): 1655 | 1656 | self.node_properties['name'] = name 1657 | self.attributes['type'] = 'Split' 1658 | check_redundant_attrs(kwargs, self.__class__.__name__) 1659 | 1660 | 1661 | class AttrSplitToSequence(AttrBaseOp): 1662 | 1663 | @ATTR(name='axis', opsets=(11, ), default=0) # v11 add 1664 | @ATTR(name='keepdims', opsets=(11, ), default=1) # v11 add 1665 | def __init__(self, name='', **kwargs): 1666 | 1667 | self.node_properties['name'] = name 1668 | self.attributes['type'] = 'SplitToSequence' 1669 | check_redundant_attrs(kwargs, self.__class__.__name__) 1670 | 1671 | 1672 | class AttrSqrt(AttrBaseOp): 1673 | 1674 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1675 | def __init__(self, name='', **kwargs): 1676 | 1677 | self.node_properties['name'] = name 1678 | self.attributes['type'] = 'Sqrt' 1679 | check_redundant_attrs(kwargs, self.__class__.__name__) 1680 | 1681 | 1682 | class AttrSqueeze(AttrBaseOp): 1683 | 1684 | @ATTR(name='axes', opsets=(1, 13), default=()) # v1 add, v13 delete 1685 | def __init__(self, name='', **kwargs): 1686 | 1687 | self.node_properties['name'] = name 1688 | self.attributes['type'] = 'Squeeze' 1689 | check_redundant_attrs(kwargs, self.__class__.__name__) 1690 | 1691 | 1692 | class AttrStringNormalizer(AttrBaseOp): 1693 | 1694 | @ATTR(name='case_change_action', opsets=(10, ), default='NONE') # v10 add 1695 | @ATTR(name='is_case_sensitive', opsets=(10, ), default=0) # v10 add 1696 | @ATTR(name='locale', opsets=(10, ), default='en_US') # v10 add 1697 | @ATTR(name='stopwords', opsets=(10, ), default=()) # v10 add 1698 | def __init__(self, name='', **kwargs): 1699 | 1700 | self.node_properties['name'] = name 1701 | self.attributes['type'] = 'StringNormalizer' 1702 | check_redundant_attrs(kwargs, self.__class__.__name__) 1703 | 1704 | 1705 | class AttrSub(AttrBaseOp): 1706 | 1707 | @ATTR(name='axis', opsets=(1, 7)) # v1 add, v7 delete, required 1708 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 1709 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1710 | def __init__(self, name='', **kwargs): 1711 | 1712 | self.node_properties['name'] = name 1713 | self.attributes['type'] = 'Sub' 1714 | check_redundant_attrs(kwargs, self.__class__.__name__) 1715 | 1716 | 1717 | class AttrSum(AttrBaseOp): 1718 | 1719 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1720 | def __init__(self, name='', **kwargs): 1721 | 1722 | self.node_properties['name'] = name 1723 | self.attributes['type'] = 'Sum' 1724 | check_redundant_attrs(kwargs, self.__class__.__name__) 1725 | 1726 | 1727 | class AttrTan(AttrBaseOp): 1728 | 1729 | def __init__(self, name='', **kwargs): 1730 | self.node_properties['name'] = name 1731 | self.attributes['type'] = 'Tan' 1732 | 1733 | 1734 | class AttrTanh(AttrBaseOp): 1735 | 1736 | @ATTR(name='consumed_inputs', opsets=(1, 6), default=()) # v1 add, v6 delete 1737 | def __init__(self, name='', **kwargs): 1738 | 1739 | self.node_properties['name'] = name 1740 | self.attributes['type'] = 'Tanh' 1741 | check_redundant_attrs(kwargs, self.__class__.__name__) 1742 | 1743 | 1744 | class AttrTfIdfVectorizer(AttrBaseOp): 1745 | 1746 | @ATTR(name='max_gram_length', opsets=(9, )) # v9 add, required 1747 | @ATTR(name='max_skip_count', opsets=(9, )) # v9 add, required 1748 | @ATTR(name='min_gram_length', opsets=(9, )) # v9 add, required 1749 | @ATTR(name='mode', opsets=(9, )) # v9 add, required 1750 | @ATTR(name='ngram_counts', opsets=(9, )) # v9 add, required 1751 | @ATTR(name='ngram_indexes', opsets=(9, )) # v9 add, required 1752 | @ATTR(name='pool_int64s', opsets=(9, ), default=()) # v9 add 1753 | @ATTR(name='pool_strings', opsets=(9, ), default=()) # v9 add 1754 | @ATTR(name='weights', opsets=(9, ), default=()) # v9 add 1755 | def __init__(self, name='', **kwargs): 1756 | 1757 | self.node_properties['name'] = name 1758 | self.attributes['type'] = 'TfIdfVectorizer' 1759 | check_redundant_attrs(kwargs, self.__class__.__name__) 1760 | 1761 | 1762 | class AttrThresholdedRelu(AttrBaseOp): 1763 | 1764 | @ATTR(name='alpha', opsets=(10, ), default=1.0) # v10 add 1765 | def __init__(self, name='', **kwargs): 1766 | 1767 | self.node_properties['name'] = name 1768 | self.attributes['type'] = 'ThresholdedRelu' 1769 | check_redundant_attrs(kwargs, self.__class__.__name__) 1770 | 1771 | 1772 | class AttrTile(AttrBaseOp): 1773 | 1774 | def __init__(self, name='', **kwargs): 1775 | self.node_properties['name'] = name 1776 | self.attributes['type'] = 'Tile' 1777 | 1778 | 1779 | class AttrTopK(AttrBaseOp): 1780 | 1781 | @ATTR(name='axis', opsets=( 1, ), default=-1) # v1 add 1782 | @ATTR(name='k', opsets=( 1, 10)) # v1 add, required, v10 delete 1783 | @ATTR(name='largest', opsets=(11, ), default=1) # v11 add 1784 | @ATTR(name='sorted', opsets=(11, ), default=1) # v11 add 1785 | def __init__(self, name='', **kwargs): 1786 | 1787 | self.node_properties['name'] = name 1788 | self.attributes['type'] = 'TopK' 1789 | check_redundant_attrs(kwargs, self.__class__.__name__) 1790 | 1791 | 1792 | class AttrTranspose(AttrBaseOp): 1793 | 1794 | @ATTR(name='perm', opsets=(1, ), default=()) # v1 add 1795 | def __init__(self, name='', **kwargs): 1796 | 1797 | self.node_properties['name'] = name 1798 | self.attributes['type'] = 'Transpose' 1799 | check_redundant_attrs(kwargs, self.__class__.__name__) 1800 | 1801 | 1802 | class AttrUnique(AttrBaseOp): 1803 | 1804 | @ATTR(name='axis', opsets=(11, ), default=-1) # v11 add 1805 | @ATTR(name='sorted', opsets=(11, ), default=1) # v11 add 1806 | def __init__(self, name='', **kwargs): 1807 | 1808 | self.node_properties['name'] = name 1809 | self.attributes['type'] = 'Unique' 1810 | check_redundant_attrs(kwargs, self.__class__.__name__) 1811 | 1812 | 1813 | class AttrUnsqueeze(AttrBaseOp): 1814 | 1815 | @ATTR(name='axes', opsets=(1, 13)) # v11 add, required, v13 delete 1816 | def __init__(self, name='', **kwargs): 1817 | 1818 | self.node_properties['name'] = name 1819 | self.attributes['type'] = 'Unsqueeze' 1820 | check_redundant_attrs(kwargs, self.__class__.__name__) 1821 | 1822 | 1823 | class AttrUpsample(AttrBaseOp): 1824 | 1825 | @ATTR(name='height_scale', opsets=(1, 7)) # v1 add, required 1826 | @ATTR(name='width_scale', opsets=(1, 7)) # v1 add, required 1827 | @ATTR(name='mode', opsets=(1, 10), default='nearest') # v1 add, v10 delete 1828 | @ATTR(name='scales', opsets=(7, 9)) # v7 add, required, v9 delete 1829 | def __init__(self, name='', **kwargs): 1830 | 1831 | self.node_properties['name'] = name 1832 | self.attributes['type'] = 'Upsample' 1833 | check_redundant_attrs(kwargs, self.__class__.__name__) 1834 | 1835 | 1836 | class AttrWhere(AttrBaseOp): 1837 | 1838 | def __init__(self, name='', **kwargs): 1839 | self.node_properties['name'] = name 1840 | self.attributes['type'] = 'Where' 1841 | 1842 | 1843 | class AttrXor(AttrBaseOp): 1844 | 1845 | @ATTR(name='axis', opsets=(1, 7), default=-1) # v1 add, v7 delete 1846 | @ATTR(name='broadcast', opsets=(1, 7), default=0) # v1 add, v7 delete 1847 | def __init__(self, name='', **kwargs): 1848 | 1849 | self.node_properties['name'] = name 1850 | self.attributes['type'] = 'Xor' 1851 | check_redundant_attrs(kwargs, self.__class__.__name__) -------------------------------------------------------------------------------- /predictor/main.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import time 4 | import copy 5 | import random 6 | import torch 7 | import logging 8 | import argparse 9 | import numpy as np 10 | 11 | import torch.nn.functional as F 12 | from torch_geometric.data import DataLoader 13 | from dataset import GraphLatencyDataset 14 | 15 | 16 | class Metric(object): 17 | def __init__(self): 18 | self.all = self.init_pack() 19 | self.plts = {} 20 | 21 | def init_pack(self): 22 | return { 23 | 'cnt': 0, 24 | 'apes': [], # absolute percentage error 25 | 'errbnd_cnt': np.array([0.0, 0.0, 0.0]), # error bound count 26 | 'errbnd_val': np.array([0.1, 0.05, 0.01]), # error bound value: 0.1, 0.05, 0.01 27 | } 28 | 29 | def update_pack(self, ps, gs, pack): 30 | for i in range(len(ps)): 31 | ape = np.abs(ps[i] - gs[i]) / gs[i] 32 | pack['errbnd_cnt'][ape <= pack['errbnd_val']] += 1 33 | pack['apes'].append(ape) 34 | pack['cnt'] += len(ps) 35 | 36 | def measure_pack(self, pack): 37 | acc = np.mean(pack['apes']) 38 | err = (pack['errbnd_cnt'] / pack['cnt'])[0] 39 | return acc, err, pack['cnt'] 40 | 41 | def update(self, ps, gs, plts=None): 42 | self.update_pack(ps, gs, self.all) 43 | if plts: 44 | for idx, plt in enumerate(plts): 45 | if plt not in self.plts: 46 | self.plts[plt] = self.init_pack() 47 | self.update_pack([ps[idx]], [gs[idx]], self.plts[plt]) 48 | 49 | def get(self, plt=None): 50 | if plt is None: 51 | return self.measure_pack(self.all) 52 | else: 53 | return self.measure_pack(self.plts[plt]) 54 | 55 | 56 | class Trainer(object): 57 | 58 | def __init__(self): 59 | self.args = self.init_args() 60 | self.logger = self.init_logger() 61 | self.logger.info("Loading args: \n{}".format(self.args)) 62 | 63 | if self.args.gpu >= 0: 64 | os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(self.args.gpu) 65 | if not torch.cuda.is_available(): 66 | self.logger.error("No GPU={} found!".format(self.args.gpu)) 67 | 68 | self.logger.info("Loading dataset:") 69 | if self.args.train_test_stage: 70 | self.train_loader, self.test_loader = self.init_train_test_dataset() 71 | else: 72 | self.train_loader, self.test_loader = self.init_unseen_structure_dataset() 73 | 74 | self.logger.info("Loading model:") 75 | self.model, self.start_epoch, self.best_acc = self.init_model() 76 | self.best_err = 0 77 | # print("Model:", self.model) 78 | 79 | self.device = torch.device('cuda' if self.args.gpu >= 0 and torch.cuda.is_available() else 'cpu') 80 | self.model = self.model.to(self.device) 81 | self.optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, self.model.parameters()), lr=self.args.lr) 82 | 83 | def init_args(self): 84 | parser = argparse.ArgumentParser() 85 | parser.add_argument('--lr', type=float, default=0.001) 86 | parser.add_argument('--gpu', type=int, default=0, help="gpu id, < 0 means no gpu") 87 | parser.add_argument('--steps', type=str, default="", help='learning decay') 88 | parser.add_argument('--epochs', type=int, default=100) 89 | parser.add_argument('--batch_size', type=int, default=1) 90 | parser.add_argument('--momentum', type=float, default=0.9) 91 | parser.add_argument('--weight_decay', type=float, default=1e-4) 92 | 93 | parser.add_argument('--gnn_layer', type=str) 94 | parser.add_argument('--data_root', type=str) 95 | parser.add_argument('--all_latency_file', type=str) 96 | parser.add_argument('--test_model_type', type=str) 97 | parser.add_argument('--train_model_types', type=str) 98 | parser.add_argument('--train_test_stage', action='store_true') 99 | parser.add_argument('--norm_sf', action='store_true') 100 | parser.add_argument('--train_num', type=int, default=-1) 101 | 102 | parser.add_argument('--onnx_dir', type=str) 103 | parser.add_argument('--override_data', action='store_true') 104 | 105 | parser.add_argument('--log', type=str) 106 | parser.add_argument('--pretrain', type=str) 107 | parser.add_argument('--resume', type=str) 108 | parser.add_argument('--model_dir', type=str) 109 | 110 | parser.add_argument('--print_freq', type=int, default=10) 111 | parser.add_argument('--ckpt_save_freq', type=int, default=20) 112 | parser.add_argument('--test_freq', type=int, default=10) 113 | parser.add_argument('--only_test', action='store_true') 114 | parser.add_argument('--hidden_size', type=int, default=512) 115 | parser.add_argument('--num_node_features', type=int, default=44) 116 | parser.add_argument('--multi_plt', type=str, default="") 117 | 118 | args = parser.parse_args() 119 | args.steps = [int(x) for x in args.steps.split(',')] if args.steps else [] 120 | args.norm_sf = True if args.norm_sf else False 121 | # {plt_id: order_id} 122 | args.multi_plt = {int(x): k for k, x in enumerate(args.multi_plt.split(','))} if args.multi_plt else {} 123 | return args 124 | 125 | 126 | def init_logger(self): 127 | if not os.path.exists("log"): 128 | os.makedirs("log") 129 | 130 | logger = logging.getLogger("FEID") 131 | logger.setLevel(level = logging.INFO) 132 | formatter = logging.Formatter("%(asctime)s-%(filename)s:%(lineno)d" \ 133 | "-%(levelname)s-%(message)s") 134 | 135 | # log file stream 136 | handler = logging.FileHandler(self.args.log) 137 | handler.setLevel(logging.INFO) 138 | handler.setFormatter(formatter) 139 | 140 | # log console stream 141 | console = logging.StreamHandler() 142 | console.setLevel(logging.INFO) 143 | console.setFormatter(formatter) 144 | 145 | logger.addHandler(handler) 146 | logger.addHandler(console) 147 | 148 | return logger 149 | 150 | 151 | def init_unseen_structure_dataset(self): 152 | model_types = set() 153 | for line in open(self.args.all_latency_file).readlines(): 154 | model_types.add(line.split()[4]) 155 | assert self.args.test_model_type in model_types 156 | test_model_types = set([self.args.test_model_type]) 157 | if self.args.train_model_types: 158 | train_model_types = set(self.args.train_model_types.split(',')) 159 | train_model_types = train_model_types & model_types 160 | else: 161 | train_model_types = model_types - test_model_types 162 | assert len(train_model_types) > 0 163 | allow_platforms=self.args.multi_plt.keys() if self.args.multi_plt else None 164 | 165 | self.logger.info("Train model types: {}".format(train_model_types)) 166 | self.logger.info("Test model types: {}".format(test_model_types)) 167 | self.logger.info("Platforms: {}".format(allow_platforms)) 168 | 169 | train_set = GraphLatencyDataset( 170 | self.args.data_root, 171 | self.args.onnx_dir, 172 | self.args.all_latency_file, 173 | override_data=self.args.override_data, 174 | model_types=train_model_types, 175 | platforms=allow_platforms, 176 | ) 177 | test_set = GraphLatencyDataset( 178 | self.args.data_root, 179 | self.args.onnx_dir, 180 | self.args.all_latency_file, 181 | override_data=self.args.override_data, 182 | model_types=test_model_types, 183 | platforms=allow_platforms, 184 | ) 185 | 186 | self.logger.info("Train data = {}, Test data = {}".format(len(train_set), len(test_set))) 187 | train_loader = DataLoader(dataset=train_set, batch_size=self.args.batch_size, shuffle=True) 188 | test_loader = DataLoader(dataset=test_set, batch_size=self.args.batch_size, shuffle=False) 189 | return train_loader, test_loader 190 | 191 | 192 | def init_train_test_dataset(self): 193 | allow_platforms=self.args.multi_plt.keys() 194 | train_set = GraphLatencyDataset( 195 | self.args.data_root, 196 | self.args.onnx_dir, 197 | self.args.all_latency_file, 198 | override_data=self.args.override_data, 199 | train_test_stage="train", 200 | platforms=allow_platforms, 201 | sample_num=self.args.train_num, 202 | ) 203 | test_set = GraphLatencyDataset( 204 | self.args.data_root, 205 | self.args.onnx_dir, 206 | self.args.all_latency_file, 207 | override_data=self.args.override_data, 208 | train_test_stage="test", 209 | platforms=allow_platforms, 210 | ) 211 | 212 | self.logger.info("Train data = {}, Test data = {} for platforms {}".format( 213 | len(train_set), len(test_set), allow_platforms 214 | )) 215 | self.logger.info("Platforms: {}".format(allow_platforms)) 216 | train_loader = DataLoader(dataset=train_set, batch_size=self.args.batch_size, shuffle=True) 217 | test_loader = DataLoader(dataset=test_set, batch_size=self.args.batch_size, shuffle=False) 218 | return train_loader, test_loader 219 | 220 | 221 | def init_model(self): 222 | best_acc = 1e9 223 | start_epoch = 1 224 | 225 | from model import Net 226 | model = Net( 227 | num_node_features=self.args.num_node_features, 228 | gnn_layer=self.args.gnn_layer, 229 | gnn_hidden=self.args.hidden_size, 230 | fc_hidden=self.args.hidden_size, 231 | multi_plt=self.args.multi_plt, 232 | norm_sf=self.args.norm_sf, 233 | ) 234 | 235 | if self.args.pretrain: 236 | self.logger.info("Loading pretrain: {}".format(self.args.pretrain)) 237 | ckpt = torch.load(self.args.pretrain) 238 | model.load_state_dict(ckpt['state_dict'], strict = False) 239 | self.logger.info("Loaded pretrain: {}".format(self.args.pretrain)) 240 | 241 | if self.args.resume: 242 | self.logger.info("Loading checkpoint: {}".format(self.args.resume)) 243 | ckpt = torch.load(self.args.resume) 244 | start_epoch, best_acc = ckpt['epoch'], ckpt['best_acc'] 245 | model.load_state_dict(ckpt['state_dict'], strict = True) 246 | self.logger.info("loaded checkpoint: {} (epoch {} best {:.2f})". \ 247 | format(self.args.resume, start_epoch, best_acc)) 248 | 249 | return model, start_epoch, best_acc 250 | 251 | 252 | def adjust_learning_rate(self, epoch): 253 | ind = len(list(filter(lambda x: x <= epoch, self.args.steps))) 254 | lr = self.args.lr * (0.1 ** ind) 255 | for param_group in self.optimizer.param_groups: 256 | param_group['lr'] = lr 257 | return lr 258 | 259 | 260 | def format_second(self, secs): 261 | return "Exa(h:m:s):{:0>2}:{:0>2}:{:0>2}".format( \ 262 | int(secs / 3600), int((secs % 3600) / 60), int(secs % 60)) 263 | 264 | 265 | def save_checkpoint(self, epoch, best = False): 266 | epoch_str = "best" if best else "e{}".format(epoch) 267 | model_path = "{}/ckpt_{}.pth".format(self.args.model_dir, epoch_str) 268 | 269 | if not os.path.exists(self.args.model_dir): 270 | os.makedirs(self.args.model_dir) 271 | 272 | torch.save({ 273 | 'epoch': epoch + 1, 274 | 'state_dict': self.model.state_dict(), 275 | 'best_acc': self.best_acc, 276 | }, model_path) 277 | self.logger.info("Checkpoint saved to {}".format(model_path)) 278 | return 279 | 280 | 281 | def train_epoch(self, epoch): 282 | self.model.train() 283 | t0 = time.time() 284 | metric = Metric() 285 | 286 | lr = self.adjust_learning_rate(epoch) 287 | num_iter = len(self.train_loader) 288 | 289 | for iteration, batch in enumerate(self.train_loader): 290 | torch.cuda.empty_cache() 291 | 292 | data, static_feature, _, plt_id = batch 293 | data.y = data.y.view(-1, 1) 294 | data = data.to(self.device) 295 | static_feature = static_feature.to(self.device) 296 | 297 | self.optimizer.zero_grad() 298 | pred_cost = self.model(data, static_feature) 299 | 300 | # multi plt training, gather the specific out channel 301 | if len(self.args.multi_plt) > 1: 302 | gather_ids = torch.LongTensor([self.args.multi_plt[int(x)] for x in plt_id]).view(-1, 1) 303 | gather_ids = gather_ids.to(self.device) 304 | pred_cost = torch.gather(pred_cost, 1, gather_ids) 305 | 306 | loss = F.mse_loss(pred_cost / data.y, data.y / data.y) 307 | loss.backward() 308 | self.optimizer.step() 309 | 310 | ps = pred_cost.data.cpu().numpy()[:, 0].tolist() 311 | gs = data.y.data.cpu().numpy()[:, 0].tolist() 312 | metric.update(ps, gs) 313 | acc, err, cnt = metric.get() 314 | 315 | if iteration % self.args.print_freq == 0: 316 | t1 = time.time() 317 | speed = (t1 - t0) / (iteration + 1) 318 | exp_time = self.format_second(speed * (num_iter * (self.args.epochs - epoch + 1) - iteration)) 319 | 320 | self.logger.info("Epoch[{}/{}]({}/{}) Lr:{:.8f} Loss:{:.5f} MAPE:{:.5f} " \ 321 | "ErrBnd(0.1):{:.5f} Speed:{:.2f} ms/iter {}" .format( \ 322 | epoch, self.args.epochs, iteration, num_iter, lr, loss.data, acc, \ 323 | err, speed * 1000, exp_time)) 324 | return acc 325 | 326 | 327 | def test(self): 328 | torch.manual_seed(1234) 329 | torch.cuda.manual_seed_all(1234) 330 | self.model.eval() 331 | t0 = time.time() 332 | num_iter = len(self.test_loader) 333 | if num_iter <= 0: 334 | return 0, 0 335 | metric = Metric() 336 | 337 | with torch.no_grad(): 338 | for iteration, batch in enumerate(self.test_loader): 339 | torch.cuda.empty_cache() 340 | 341 | data, static_feature, graph_name, plt_id = batch 342 | data.y = data.y.view(-1, 1) 343 | data = data.to(self.device) 344 | static_feature = static_feature.to(self.device) 345 | pred_cost = self.model(data, static_feature) 346 | 347 | # multi plt training, gather the specific out channel 348 | if len(self.args.multi_plt) > 1: 349 | gather_ids = torch.LongTensor([self.args.multi_plt[int(x)] for x in plt_id]).view(-1, 1) 350 | gather_ids = gather_ids.to(self.device) 351 | pred_cost = torch.gather(pred_cost, 1, gather_ids) 352 | 353 | #pred_cost = torch.exp(pred_cost) 354 | ps = pred_cost.data.cpu().numpy()[:, 0].tolist() 355 | gs = data.y.data.cpu().numpy()[:, 0].tolist() 356 | plts = plt_id.data.cpu().numpy().tolist() 357 | metric.update(ps, gs, plts) 358 | acc, err, cnt = metric.get() 359 | 360 | if iteration > 0 and iteration % 50 == 0: 361 | self.logger.info("[{}/{}] MAPE: {:.5f} ErrBnd(0.1): {:.5f}".format( 362 | iteration, num_iter, acc, err)) 363 | 364 | t1 = time.time() 365 | speed = (t1 - t0) / num_iter * 1000 366 | acc, err, cnt = metric.get() 367 | 368 | self.logger.info(" ------------------------------------------------------------------") 369 | self.logger.info(" * Speed: {:.5f} ms/iter".format(speed)) 370 | self.logger.info(" * MAPE: {:.5f}".format(acc)) 371 | self.logger.info(" * ErrorBound (0.1): {}".format(err)) 372 | self.logger.info(" ------------------------------------------------------------------") 373 | 374 | if self.args.multi_plt: 375 | accs, errs = [], [] 376 | for plt in self.args.multi_plt.keys(): 377 | acc, err, cnt = metric.get(plt=plt) 378 | accs.append(acc) 379 | errs.append(err) 380 | self.logger.info("Platform {}, MAPE={:.5f}, ErrorBound(0.1)={:.5f}".format(plt, acc, err)) 381 | self.logger.info(" ------------------------------------------------------------------") 382 | self.logger.info(" * Platform Average") 383 | self.logger.info(" * MAPE: {:.5f}".format(np.mean(accs))) 384 | self.logger.info(" * ErrorBound (0.1): {}".format(np.mean(errs))) 385 | self.logger.info(" ------------------------------------------------------------------") 386 | 387 | torch.manual_seed(time.time()) 388 | torch.cuda.manual_seed_all(time.time()) 389 | return acc, err 390 | 391 | 392 | def train(self): 393 | for epoch in range(self.start_epoch, self.args.epochs + 1): 394 | self.args.only_test = False 395 | self.train_epoch(epoch) 396 | 397 | if epoch > 0 and epoch % self.args.ckpt_save_freq == 0: 398 | self.save_checkpoint(epoch) 399 | 400 | if epoch > 0 and epoch % self.args.test_freq == 0: 401 | self.args.only_test = True 402 | acc, err = self.test() 403 | 404 | if acc < self.best_acc: 405 | self.best_acc = acc 406 | self.best_err = err 407 | self.save_checkpoint(epoch, best = True) 408 | self.logger.info("Train over, best acc = {:.5f}, err = {}".format( 409 | self.best_acc, self.best_err 410 | )) 411 | return 412 | 413 | 414 | def run(self): 415 | if self.args.only_test: 416 | self.test() 417 | else: 418 | self.train() 419 | 420 | 421 | if __name__ == "__main__": 422 | trainer = Trainer() 423 | trainer.run() 424 | -------------------------------------------------------------------------------- /predictor/model.py: -------------------------------------------------------------------------------- 1 | import math 2 | import torch 3 | from torch import nn 4 | import torch.nn.functional as F 5 | import torch_geometric.nn as tgnn 6 | from torch_scatter import scatter 7 | 8 | 9 | def init_tensor(tensor, init_type, nonlinearity): 10 | if tensor is None or init_type is None: 11 | return 12 | if init_type =='thomas': 13 | size = tensor.size(-1) 14 | stdv = 1. / math.sqrt(size) 15 | nn.init.uniform_(tensor, -stdv, stdv) 16 | elif init_type == 'kaiming_normal_in': 17 | nn.init.kaiming_normal_(tensor, mode='fan_in', nonlinearity=nonlinearity) 18 | elif init_type == 'kaiming_normal_out': 19 | nn.init.kaiming_normal_(tensor, mode='fan_out', nonlinearity=nonlinearity) 20 | elif init_type == 'kaiming_uniform_in': 21 | nn.init.kaiming_uniform_(tensor, mode='fan_in', nonlinearity=nonlinearity) 22 | elif init_type == 'kaiming_uniform_out': 23 | nn.init.kaiming_uniform_(tensor, mode='fan_out', nonlinearity=nonlinearity) 24 | elif init_type == 'orthogonal': 25 | nn.init.orthogonal_(tensor, gain=nn.init.calculate_gain(nonlinearity)) 26 | else: 27 | raise ValueError(f'Unknown initialization type: {init_type}') 28 | 29 | 30 | class PredictFC(torch.nn.Module): 31 | def __init__(self, 32 | input_feature, 33 | fc_hidden): 34 | super(PredictFC, self).__init__() 35 | self.fc_1 = nn.Linear(input_feature, fc_hidden) 36 | self.fc_2 = nn.Linear(fc_hidden, fc_hidden) 37 | self.fc_relu_1 = nn.ReLU() 38 | self.fc_relu_2 = nn.ReLU() 39 | self.fc_drop_1 = nn.Dropout(p=0.05) 40 | self.fc_drop_2 = nn.Dropout(p=0.05) 41 | self.predictor = nn.Linear(fc_hidden, 1) 42 | 43 | def forward(self, x): 44 | x = self.fc_1(x) 45 | x = self.fc_relu_1(x) 46 | x = self.fc_drop_1(x) 47 | x = self.fc_2(x) 48 | x = self.fc_relu_2(x) 49 | x = self.fc_drop_2(x) 50 | x = self.predictor(x) 51 | return x 52 | 53 | 54 | # reduce_func: "sum", "mul", "mean", "min", "max" 55 | class Net(torch.nn.Module): 56 | def __init__( 57 | self, 58 | num_node_features=44, 59 | gnn_layer="SAGEConv", 60 | gnn_hidden=512, 61 | fc_hidden=512, 62 | reduce_func="sum", 63 | multi_plt={}, 64 | norm_sf=False, 65 | ): 66 | super(Net, self).__init__() 67 | 68 | self.reduce_func = reduce_func 69 | self.num_node_features = num_node_features 70 | self.multi_plt = multi_plt 71 | self.norm_sf = norm_sf 72 | self.gnn_layer_func = getattr(tgnn, gnn_layer) 73 | 74 | self.graph_conv_1 = self.gnn_layer_func(num_node_features, gnn_hidden, normalize=True) 75 | self.graph_conv_2 = self.gnn_layer_func(gnn_hidden, gnn_hidden, normalize=True) 76 | self.gnn_drop_1 = nn.Dropout(p=0.05) 77 | self.gnn_drop_2 = nn.Dropout(p=0.05) 78 | self.gnn_relu1 = nn.ReLU() 79 | self.gnn_relu2 = nn.ReLU() 80 | 81 | out_dim = 1 if len(multi_plt) <= 1 else len(multi_plt) 82 | if out_dim > 1: 83 | for i in range(out_dim): 84 | self.add_module(f'head_{i}', PredictFC(gnn_hidden + 4, fc_hidden)) 85 | else: 86 | if self.norm_sf: 87 | self.norm_sf_linear = nn.Linear(4, gnn_hidden) 88 | self.norm_sf_drop = nn.Dropout(p=0.05) 89 | self.norm_sf_relu = nn.ReLU() 90 | sf_hidden = gnn_hidden 91 | else: 92 | sf_hidden = 4 93 | self.fc_1 = nn.Linear(gnn_hidden + sf_hidden, fc_hidden) 94 | self.fc_2 = nn.Linear(fc_hidden, fc_hidden) 95 | self.fc_drop_1 = nn.Dropout(p=0.05) 96 | self.fc_drop_2 = nn.Dropout(p=0.05) 97 | self.fc_relu1 = nn.ReLU() 98 | self.fc_relu2 = nn.ReLU() 99 | self.predictor = nn.Linear(fc_hidden, out_dim) 100 | self._initialize_weights() 101 | 102 | def _initialize_weights(self): 103 | for m in self.modules(): 104 | if isinstance(m, nn.Linear): 105 | init_tensor(m.weight, "thomas", "relu") 106 | init_tensor(m.bias, "thomas", "relu") 107 | elif isinstance(m, self.gnn_layer_func): 108 | pass 109 | 110 | def forward(self, data, static_feature): 111 | x, A = data.x, data.edge_index 112 | x = self.graph_conv_1(x, A) 113 | x = self.gnn_relu1(x) 114 | x = self.gnn_drop_1(x) 115 | 116 | x = self.graph_conv_2(x, A) 117 | x = self.gnn_relu2(x) 118 | x = self.gnn_drop_2(x) 119 | 120 | gnn_feat = scatter(x, data.batch, dim=0, reduce=self.reduce_func) 121 | if self.norm_sf: 122 | static_feature = self.norm_sf_linear(static_feature) 123 | static_feature = self.norm_sf_drop(static_feature) 124 | static_feature = self.norm_sf_relu(static_feature) 125 | x = torch.cat([gnn_feat, static_feature], dim=1) 126 | 127 | if len(self.multi_plt) > 1: 128 | xs = [] 129 | for i in range(len(self.multi_plt)): 130 | predictor = getattr(self, f'head_{i}') 131 | z = predictor(x) 132 | xs.append(z) 133 | x = torch.cat(xs, dim=1) 134 | else: 135 | x = self.fc_1(x) 136 | x = self.fc_relu1(x) 137 | x = self.fc_drop_1(x) 138 | x = self.fc_2(x) 139 | x = self.fc_relu2(x) 140 | feat = self.fc_drop_2(x) 141 | x = self.predictor(feat) 142 | 143 | pred = -F.logsigmoid(x) # (0, +inf) 144 | return pred -------------------------------------------------------------------------------- /predictor/tocsv.py: -------------------------------------------------------------------------------- 1 | import re 2 | import sys 3 | import numpy as np 4 | 5 | fname = sys.argv[1] 6 | fold = int(sys.argv[2]) 7 | if len(sys.argv) > 3: 8 | out_path = "test_{}.csv".format(sys.argv[3]) 9 | else: 10 | out_path = "test.csv" 11 | mapes = [] 12 | erbds = [] 13 | 14 | with open(fname) as f: 15 | for line in f.readlines(): 16 | if "* MAPE" in line: 17 | mapes.append(float(line.split(' ')[-1])) 18 | if "* ErrorBound" in line: 19 | line = line.replace(']', '').rstrip() 20 | items = re.split(r'[ \[]', line) 21 | items = list(filter(lambda x: x!="", items)) 22 | errs = [float(x) for x in items[-1:]] 23 | erbds.append(errs[0]) 24 | 25 | mapes = np.array(mapes).reshape(-1, fold) 26 | erbds = np.array(erbds).reshape(-1, fold) 27 | assert mapes.shape[0] == erbds.shape[0] 28 | cnt = mapes.shape[0] 29 | 30 | with open(out_path, "w") as f: 31 | for i in range(cnt): 32 | 33 | s1 = ",".join(["{:.4f}".format(round(x, 4)) for x in mapes[i]]) 34 | s2 = ",".join(["{:.4f}".format(round(x, 4)) for x in erbds[i]]) 35 | m1 = "{:.4f}".format(round(mapes[i].mean(), 4)) 36 | m2 = "{:.4f}".format(round(erbds[i].mean(), 4)) 37 | 38 | f.write("{},{},{},{}\n".format(s1, m1, s2, m2)) 39 | print("[{}] mape: {} [{}] erbd: {} [{}]".format(i + 1, s1, m1, s2, m2)) 40 | 41 | avg_mape = round(mapes.reshape(-1).mean(), 4) 42 | avg_erbd = round(erbds.reshape(-1).mean(), 4) 43 | 44 | f.write("," * (fold - 1)) 45 | f.write("{:.4f}".format(avg_mape)) 46 | f.write("," * (fold + 1)) 47 | f.write("{:.4f}".format(avg_erbd)) 48 | 49 | print("AVG MAPE: [{:.4f}]".format(avg_mape)) 50 | print("AVG ERBD: [{:.4f}]".format(avg_erbd)) 51 | -------------------------------------------------------------------------------- /test_onnx/resnet18-v1-7-no-weight.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ModelTC/NNLQP/6da64f035baa2cc5ba356071bee28b1c5b71f988/test_onnx/resnet18-v1-7-no-weight.onnx --------------------------------------------------------------------------------