├── LICENSE ├── README.md ├── _step_by_step ├── README.md ├── convert_BatchNormalization_training_true.ipynb ├── model_converter_merge_batchnormalization_to_depthwiseconv.ipynb ├── model_converter_merge_conv2d_relu.ipynb ├── model_converter_merge_conv_batchnormalization.ipynb ├── model_converter_merge_depthwiseconv_batchnormalization.ipynb ├── model_converter_split_separableconv2d.ipynb ├── unpack_model.ipynb └── yolov4-tiny-keras-custom.ipynb ├── converter ├── converter.py └── model_adaptation.py ├── extra_layers ├── BinaryOp.py ├── Clip.py ├── CustomObjects.py ├── OutputSplit.py ├── Padding.py ├── Resize.py └── UnaryOp.py ├── keras2ncnn.py ├── model_zoo ├── memory_estimator │ └── get_mem.py └── segmentation │ └── hair │ └── model_000 │ ├── CelebA_PrismaNet_256_hair_seg_model_opt_001.bin │ ├── CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5 │ ├── CelebA_PrismaNet_256_hair_seg_model_opt_001.param │ ├── README.md │ ├── demo.png │ └── demo.py ├── optimization ├── graph │ ├── ActivationReLU_max_split.py │ ├── BatchNormalization_DepthwiseConv2D_transform.py │ ├── Conv2DActivation_merge.py │ ├── Conv2DBatchNormalization_merge.py │ ├── Conv2DReLU_merge.py │ ├── Conv2DSigmoid_merge.py │ ├── Conv2DSoftmax_split.py │ ├── DropLayer.py │ ├── ReLU_max_split.py │ ├── SeparableConv2D_split.py │ └── layer_template.py └── optimize_graph.py ├── requirements.txt ├── run_test.py └── unit_test ├── helper.py ├── simple_model ├── Adain.py ├── EncoderDecoder.py ├── MultipleInput.py ├── UNet.py └── Unstructured.py ├── single_layer ├── Activation.py ├── BinaryOp.py ├── Conv2D.py ├── Conv2DTranspose.py ├── Dense.py ├── DepthwiseConv2D.py ├── Merge.py ├── Normalization.py ├── Pooling2D.py ├── ReshapeFlatten.py ├── UnaryOp.py └── UpSampling2D.py └── unit_test_data ├── person_001_1024x1204.jpg ├── person_1024x1204.jpg ├── person_640x640.jpg └── persons_000.jpg /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 azeme1 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # keras2ncnn 2 | ## Export Keras model to Tencent/NCNN. 3 | ### Supported Keras layers and Features 4 | * Functional Model API (the Sequential and Model as Layer features should be transformed in to flat Functional API Model) 5 | * InputLayer 6 | * ReLU/ReLU6/LeakyReLU/Softmax/Sigmoid 7 | * Clip 8 | * Reshape/Flatten(converted with NCNN::Reshape) 9 | * MaxPooling2D/AveragePooling2D/MaxPool2D/AvgPool2D 10 | * GlobalMaxPooling2D/GlobalAveragePooling2D/GlobalMaxPool2D/GlobalAvgPool2D 11 | * ZeroPadding2D/ReflectPadding2D 12 | * BatchNormalization/InstanceNormalization 13 | * Conv2D/DepthwiseConv2D/SeparableConv2D(converted with split into NCNN::ConvolutionDepthWise->NCNN::Convolution) 14 | * Concatenate/Add/Multiply 15 | * UpSampling2D(nearest neighbour/bilinear) 16 | * Conv2DTranspose(only for the even strides) 17 | 18 | ### Supported Keras Layer Optimization 19 | * Conv2D/DepthwiseConv2D/Conv2DTranspose with BatchNormalization 20 | * BatchNormalization with Conv2D/DepthwiseConv2D/Conv2DTranspose(In Progress) 21 | * ZeroPadding2D(In Progress :: Fusion with Convolution/Pooling) 22 | 23 | 24 | ### Unit tests is written 25 | * Unit tests with [Python NCNN inference - pyncnn](https://github.com/caishanli/pyncnn) installed 26 | * Latest models tested tf_nightly-2.5.0.dev20201130-cp37-cp37m-win_amd64.whl / tf_nightly_gpu-2.5.0.dev20201130-cp37-cp37m-win_amd64.whl 27 | * Latest NCNN revision https://github.com/Tencent/ncnn/commit/25b224479cbe535ce35ca92556c8f17d9b9f1951 28 | * Latest PyNCNN revision https://github.com/caishanli/pyncnn/commit/63c77b8bd75dae8e2601b918e92a5050a3fee8df 29 | 30 | ### Preconverted models 31 | Some 'preconverted' models can be downloaded from 32 | [DropBox](https://www.dropbox.com/sh/8anok3k3jxjj81i/AADWMLad_V0MKs4ySN2mgPPda?dl=0) 33 | 34 | ## Requirements installation 35 | The code was tested with python3.7 with TensorFlow 1.x/2.x (CPU). The code should work with python3.x . 36 | The behaviour with TensorFlow GPU/TPU. 37 | ``` 38 | git clone https://github.com/azeme1/keras2ncnn.git 39 | cd keras2ncnn 40 | pip3 install -r requirements.txt 41 | ``` 42 | ## Usage 43 | The model zoo folder contains the sample model 44 | ([CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5](./model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5)) 45 | as well as the result of the conversion 46 | (graph: [CelebA_PrismaNet_256_hair_seg_model_opt_001.param](./model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.param) and 47 | weights: [CelebA_PrismaNet_256_hair_seg_model_opt_001.bin](./model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.bin)) 48 | Load the model from the '.hdf5' file 49 | ``` 50 | python3 keras2ncnn.py --model_path=model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5 51 | ``` 52 | Load the model from the '.json' file (the weights should be located at the same folder in '.hdf5') 53 | ``` 54 | python3 keras2ncnn.py --model_path=model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.json 55 | ``` 56 | ## Useful Links 57 | ### Tencent/NCNN documentation 58 | * [https://github.com/Tencent/ncnn/wiki/how-to-implement-custom-layer-step-by-step] 59 | * [https://github.com/Tencent/ncnn/wiki/param-and-model-file-structure] 60 | * [https://github.com/Tencent/ncnn/wiki/operation-param-weight-table] 61 | * [https://github.com/MarsTechHAN/keras2ncnn] 62 | ### Graph visualization 63 | * [https://github.com/lutzroeder/netron] 64 | 65 | ## TODO List 66 | ### Code 67 | * Add support for the Dense Layer 68 | * Fix layer name length issue 69 | * Export models from Keras applications applications 70 | * Model in Model support 71 | * Sequential API support 72 | * Mixed mode API support 73 | ### Upcoming Models 74 | * [https://github.com/thangtran480/hair-segmentation] 75 | * [https://github.com/ItchyHiker/Hair_Segmentation_Keras] 76 | * [https://github.com/gaelkt/HairNets] 77 | * [https://github.com/JungUnYun/Hair_segmentation] 78 | * [https://github.com/YawenSun9/deep-hair-segmentation] 79 | * [https://github.com/mostafa-shalaby84/hair_segmentation_matting] 80 | 81 | ### Thanx 82 | * [https://github.com/cvzakharchenko] 83 | * [https://github.com/nvoronetskiy] 84 | 85 | ## Important note 86 | Sometimes good result can be achieved with Tensorflow conversion approach 87 | 88 | ``` 89 | import tensorflow as tf 90 | model = tf.keras.models.load_model("model.h5") 91 | model.save("saved_model") 92 | ``` 93 | 94 | after that convert 'model.ckpt' or 'model.pb' with following scripts 95 | * [https://github.com/Tencent/ncnn/tree/master/tools/mlir] 96 | * [https://github.com/Tencent/ncnn/tree/master/tools/tensorflow] 97 | -------------------------------------------------------------------------------- /_step_by_step/README.md: -------------------------------------------------------------------------------- 1 | This folder contains sample for developing layer fusion merging operations over Keras config. -------------------------------------------------------------------------------- /_step_by_step/convert_BatchNormalization_training_true.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 4, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "from tensorflow.keras.models import Model\n", 10 | "from tensorflow.keras.layers import Input, BatchNormalization\n", 11 | "import numpy as np" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": 59, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "in_layer = Input((32, 32, 3))\n", 21 | "c_layer = BatchNormalization()(in_layer, training=True)\n", 22 | "model = Model(in_layer, c_layer)" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 60, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "g = np.random.uniform(-1,+1,size=3)\n", 32 | "b = np.random.uniform(-1,+1,size=3)\n", 33 | "model.layers[1].set_weights([g, \n", 34 | " b, \n", 35 | " np.random.uniform(-1,+1,size=3),\n", 36 | " np.random.uniform(-1,+1,size=3)])" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 61, 42 | "metadata": {}, 43 | "outputs": [ 44 | { 45 | "name": "stdout", 46 | "output_type": "stream", 47 | "text": [ 48 | "WARNING:tensorflow:AutoGraph could not transform .predict_function at 0x00000163694335E8> and will run it as-is.\n", 49 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 50 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 51 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 52 | "WARNING: AutoGraph could not transform .predict_function at 0x00000163694335E8> and will run it as-is.\n", 53 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 54 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 55 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n" 56 | ] 57 | } 58 | ], 59 | "source": [ 60 | "x_in = np.random.uniform(0.,1., size=(1, 32, 32, 3))\n", 61 | "y_keras = model.predict(x_in)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 62, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "y_np = +b + g*(x_in - x_in.mean((0,1,2)))/x_in.std((0,1,2))" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 63, 76 | "metadata": {}, 77 | "outputs": [ 78 | { 79 | "data": { 80 | "text/plain": [ 81 | "-8.643739387495412e-08" 82 | ] 83 | }, 84 | "execution_count": 63, 85 | "metadata": {}, 86 | "output_type": "execute_result" 87 | } 88 | ], 89 | "source": [ 90 | "(y_np - y_keras).mean()" 91 | ] 92 | } 93 | ], 94 | "metadata": { 95 | "kernelspec": { 96 | "display_name": "Python 3", 97 | "language": "python", 98 | "name": "python3" 99 | }, 100 | "language_info": { 101 | "codemirror_mode": { 102 | "name": "ipython", 103 | "version": 3 104 | }, 105 | "file_extension": ".py", 106 | "mimetype": "text/x-python", 107 | "name": "python", 108 | "nbconvert_exporter": "python", 109 | "pygments_lexer": "ipython3", 110 | "version": "3.7.6" 111 | } 112 | }, 113 | "nbformat": 4, 114 | "nbformat_minor": 4 115 | } 116 | -------------------------------------------------------------------------------- /_step_by_step/model_converter_merge_batchnormalization_to_depthwiseconv.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n", 11 | "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": 2, 17 | "metadata": {}, 18 | "outputs": [ 19 | { 20 | "name": "stderr", 21 | "output_type": "stream", 22 | "text": [ 23 | "Using TensorFlow backend.\n", 24 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 25 | " _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n", 26 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 27 | " _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n", 28 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 29 | " _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n", 30 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 31 | " _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n", 32 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 33 | " _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n", 34 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 35 | " np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n", 36 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 37 | " _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n", 38 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 39 | " _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n", 40 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 41 | " _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n", 42 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 43 | " _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n", 44 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 45 | " _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n", 46 | "C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", 47 | " np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n" 48 | ] 49 | } 50 | ], 51 | "source": [ 52 | "from copy import deepcopy\n", 53 | "import imageio\n", 54 | "import numpy as np\n", 55 | "import cv2\n", 56 | "import matplotlib.pyplot as plt\n", 57 | "import keras\n", 58 | "import numpy as np\n", 59 | "import tensorflow as tf\n", 60 | "import tensorflow.keras.backend as K\n", 61 | "from tqdm import tqdm\n", 62 | "from tensorflow.keras import Input\n", 63 | "from tensorflow.keras.models import Model\n", 64 | "from tensorflow.keras.layers import Layer, InputSpec, DepthwiseConv2D, BatchNormalization, Activation, Conv2D, Add, Conv2DTranspose\n", 65 | "from tensorflow.keras import backend as K\n", 66 | "from tensorflow.keras.backend import int_shape, permute_dimensions\n", 67 | "from collections import OrderedDict\n", 68 | "from tensorflow.keras.models import Model, load_model\n", 69 | "from tensorflow.keras.layers import Input, SeparableConv2D\n", 70 | "from tensorflow.keras.initializers import RandomNormal, RandomUniform" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 3, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "def rename_layer(model_config, src_name, dst_name):\n", 80 | " for layer_item in model_config['input_layers']:\n", 81 | " input_condition = layer_item[0] == src_name\n", 82 | " if input_condition:\n", 83 | " # print('Rename Input:: ', layer_item[0])\n", 84 | " layer_item[0] = dst_name\n", 85 | "\n", 86 | " for layer_item in model_config['output_layers']:\n", 87 | " output_condition = layer_item[0] == src_name\n", 88 | " if output_condition:\n", 89 | " # print('Rename Output:: ', layer_item[0])\n", 90 | " layer_item[0] = dst_name\n", 91 | " # print(rename_condition, layer_list[0])\n", 92 | "\n", 93 | " for layer_item in model_config['layers']:\n", 94 | " name_condition = layer_item['name'] == src_name\n", 95 | " if name_condition:\n", 96 | " # print('Rename Layers:: name', layer_item['name'])\n", 97 | " layer_item['name'] = dst_name\n", 98 | "\n", 99 | " name_condition = layer_item['config']['name'] == src_name\n", 100 | " if name_condition:\n", 101 | " # print('Rename Layers Config:: config', layer_item['config']['name'])\n", 102 | " layer_item['config']['name'] = dst_name\n", 103 | " # print(' Rename Layers Config:: config', layer_item['config']['name'])\n", 104 | "\n", 105 | " if len(layer_item['inbound_nodes']) > 0:\n", 106 | " for item in layer_item['inbound_nodes'][0]:\n", 107 | " inbound_condition = item[0] == src_name\n", 108 | " if inbound_condition:\n", 109 | " # print('Rename Layers Inbound::', item[0])\n", 110 | " item[0] = dst_name\n", 111 | " return model_config" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 4, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "DepthwiseConv2D_config_template = {'name': '_node_name',\n", 121 | " 'class_name': 'DepthwiseConv2D',\n", 122 | " 'config': {'name': '_node_name',\n", 123 | " 'trainable': True,\n", 124 | " 'dtype': 'float32',\n", 125 | " 'kernel_size': (1, 1),\n", 126 | " 'strides': (1, 1),\n", 127 | " 'padding': 'same',\n", 128 | " 'data_format': 'channels_last',\n", 129 | " 'dilation_rate': (1, 1),\n", 130 | " 'activation': 'linear',\n", 131 | " 'use_bias': False,\n", 132 | " 'bias_initializer': {'class_name': 'Zeros', 'config': {'dtype': 'float32'}},\n", 133 | " 'bias_regularizer': None,\n", 134 | " 'activity_regularizer': None,\n", 135 | " 'bias_constraint': None,\n", 136 | " 'depth_multiplier': 1,\n", 137 | " 'depthwise_initializer': {'class_name': 'GlorotUniform',\n", 138 | " 'config': {'seed': None, 'dtype': 'float32'}},\n", 139 | " 'depthwise_regularizer': None,\n", 140 | " 'depthwise_constraint': None},\n", 141 | " 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]}" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 5, 147 | "metadata": {}, 148 | "outputs": [ 149 | { 150 | "name": "stdout", 151 | "output_type": "stream", 152 | "text": [ 153 | "WARNING:tensorflow:From C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\keras\\initializers.py:143: calling RandomNormal.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 154 | "Instructions for updating:\n", 155 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n", 156 | "WARNING:tensorflow:From C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\keras\\initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 157 | "Instructions for updating:\n", 158 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n", 159 | "WARNING:tensorflow:From C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\ops\\init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 160 | "Instructions for updating:\n", 161 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n" 162 | ] 163 | } 164 | ], 165 | "source": [ 166 | "placeholder = a = Input((32, 32, 3), name='data')\n", 167 | "x = BatchNormalization(name='src', \n", 168 | " beta_initializer=RandomNormal(),\n", 169 | " gamma_initializer=RandomNormal(),\n", 170 | " moving_mean_initializer=RandomNormal(),\n", 171 | " moving_variance_initializer=RandomUniform(1,2))(placeholder)\n", 172 | "# x = Add()([a, x])\n", 173 | "src_model = Model(placeholder, x, name='src_model')\n", 174 | "\n", 175 | "placeholder = Input((32, 32, 3), name='data')\n", 176 | "x = DepthwiseConv2D((1, 1), padding='same', name='dst')(placeholder)\n", 177 | "dst_model = Model(placeholder, x, name='dst_model')" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": 6, 183 | "metadata": {}, 184 | "outputs": [], 185 | "source": [ 186 | "gamma, beta, mean, var = src_model.get_layer('src').get_weights()\n", 187 | "eps = src_model.get_layer('src').get_config()['epsilon']\n", 188 | "a = gamma / np.sqrt(var + eps)\n", 189 | "weight = a.reshape((1,1,-1,1))\n", 190 | "bias = -a*mean + beta\n", 191 | "dst_model.get_layer('dst').set_weights([weight, bias])" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": 7, 197 | "metadata": {}, 198 | "outputs": [ 199 | { 200 | "name": "stdout", 201 | "output_type": "stream", 202 | "text": [ 203 | "4.6063215e-06\n" 204 | ] 205 | } 206 | ], 207 | "source": [ 208 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 209 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 17, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "def transfer_BatchNormalization_DepthwiseConv2D(src_model, dst_model, transfer_rule):\n", 219 | " gamma, beta, mean, var = src_model.get_layer(transfer_rule['src']).get_weights()\n", 220 | " eps = src_model.get_layer(transfer_rule['src']).get_config()['epsilon']\n", 221 | " a = gamma / np.sqrt(var + eps)\n", 222 | " weight = a.reshape((1,1,-1,1))\n", 223 | " bias = -a*mean + beta\n", 224 | " \n", 225 | " dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias])\n", 226 | " \n", 227 | "def detect_transform_BatchNormalization_DepthwiseConv2D(keras_config):\n", 228 | " index_list = []\n", 229 | " for i, item in enumerate(keras_config['layers']):\n", 230 | " if item['class_name'] == 'BatchNormalization':\n", 231 | " index_list.append(i)\n", 232 | " return index_list\n", 233 | "\n", 234 | "def apply_transform_BatchNormalization_DepthwiseConv2D(keras_config):\n", 235 | " index_list = detect_transform_BatchNormalization_DepthwiseConv2D(keras_config)\n", 236 | " weight_transfer_rule_dict = {}\n", 237 | " while len(index_list) > 0:\n", 238 | " i = index_list[0]\n", 239 | " r_layer_config = keras_config['layers'].pop(i)\n", 240 | " i_layer_config = deepcopy(DepthwiseConv2D_config_template)\n", 241 | " \n", 242 | " i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes']\n", 243 | " i_layer_config['config']['name'] = i_layer_config['name'] = r_layer_config['name']\n", 244 | " i_layer_config['config']['use_bias'] = True\n", 245 | " i_layer_config['config']['kernel_size'] = (1, 1) \n", 246 | " keras_config['layers'].insert(i, i_layer_config)\n", 247 | " weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_BatchNormalization_DepthwiseConv2D,\n", 248 | " 'src': r_layer_config['name'],\n", 249 | " 'dst': i_layer_config['name']} \n", 250 | " index_list = detect_transform_BatchNormalization_DepthwiseConv2D(keras_config)\n", 251 | " return keras_config, weight_transfer_rule_dict" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": 18, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "def transfer_weights(src_model, dst_model, weight_transfer_rule_dict):\n", 261 | " for dst_layer in tqdm(dst_model.layers):\n", 262 | " if dst_layer.name in weight_transfer_rule_dict:\n", 263 | " transfer_rule = weight_transfer_rule_dict[dst_layer.name]\n", 264 | " func = transfer_rule['transfer_call']\n", 265 | " func(src_model, dst_model, transfer_rule)\n", 266 | " else:\n", 267 | " src_model.get_layer(dst_layer.name).set_weights(dst_layer.get_weights())" 268 | ] 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": 19, 273 | "metadata": {}, 274 | "outputs": [], 275 | "source": [ 276 | "dst_model_config, weight_transfer_rule_dict = apply_transform_BatchNormalization_DepthwiseConv2D(src_model.get_config())" 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 20, 282 | "metadata": {}, 283 | "outputs": [ 284 | { 285 | "name": "stdout", 286 | "output_type": "stream", 287 | "text": [ 288 | "WARNING:tensorflow:From C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\ops\\init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 289 | "Instructions for updating:\n", 290 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n", 291 | "WARNING:tensorflow:From C:\\Users\\olga\\Miniconda3\\lib\\site-packages\\tensorflow\\python\\ops\\init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 292 | "Instructions for updating:\n", 293 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n" 294 | ] 295 | } 296 | ], 297 | "source": [ 298 | "dst_model = Model.from_config(dst_model_config)" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": 21, 304 | "metadata": {}, 305 | "outputs": [ 306 | { 307 | "name": "stderr", 308 | "output_type": "stream", 309 | "text": [ 310 | "100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 42.52it/s]\n" 311 | ] 312 | } 313 | ], 314 | "source": [ 315 | "transfer_weights(src_model, dst_model, weight_transfer_rule_dict)" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": 22, 321 | "metadata": {}, 322 | "outputs": [ 323 | { 324 | "name": "stdout", 325 | "output_type": "stream", 326 | "text": [ 327 | "4.6752393e-06\n" 328 | ] 329 | } 330 | ], 331 | "source": [ 332 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 333 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": 25, 339 | "metadata": {}, 340 | "outputs": [ 341 | { 342 | "data": { 343 | "text/plain": [ 344 | "False" 345 | ] 346 | }, 347 | "execution_count": 25, 348 | "metadata": {}, 349 | "output_type": "execute_result" 350 | } 351 | ], 352 | "source": [ 353 | "0.000820159912 < 1e-4" 354 | ] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "execution_count": 24, 359 | "metadata": {}, 360 | "outputs": [ 361 | { 362 | "data": { 363 | "text/plain": [ 364 | "0.001" 365 | ] 366 | }, 367 | "execution_count": 24, 368 | "metadata": {}, 369 | "output_type": "execute_result" 370 | } 371 | ], 372 | "source": [ 373 | "10e-4" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": null, 379 | "metadata": {}, 380 | "outputs": [], 381 | "source": [] 382 | } 383 | ], 384 | "metadata": { 385 | "kernelspec": { 386 | "display_name": "Python 3", 387 | "language": "python", 388 | "name": "python3" 389 | }, 390 | "language_info": { 391 | "codemirror_mode": { 392 | "name": "ipython", 393 | "version": 3 394 | }, 395 | "file_extension": ".py", 396 | "mimetype": "text/x-python", 397 | "name": "python", 398 | "nbconvert_exporter": "python", 399 | "pygments_lexer": "ipython3", 400 | "version": "3.7.6" 401 | } 402 | }, 403 | "nbformat": 4, 404 | "nbformat_minor": 4 405 | } 406 | -------------------------------------------------------------------------------- /_step_by_step/model_converter_merge_conv2d_relu.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n", 11 | "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "from copy import deepcopy\n", 21 | "import imageio\n", 22 | "import numpy as np\n", 23 | "import cv2\n", 24 | "import matplotlib.pyplot as plt\n", 25 | "import keras\n", 26 | "import numpy as np\n", 27 | "import tensorflow as tf\n", 28 | "import tensorflow.keras.backend as K\n", 29 | "from tqdm import tqdm\n", 30 | "from tensorflow.keras import Input\n", 31 | "from tensorflow.keras.models import Model\n", 32 | "from tensorflow.keras.layers import Layer, InputSpec, DepthwiseConv2D, BatchNormalization, Activation, Conv2D, Add, Conv2DTranspose\n", 33 | "from tensorflow.keras import backend as K\n", 34 | "from tensorflow.keras.backend import int_shape, permute_dimensions\n", 35 | "from collections import OrderedDict\n", 36 | "from tensorflow.keras.models import Model, load_model\n", 37 | "from tensorflow.keras.layers import Input, SeparableConv2D, ReLU\n", 38 | "from tensorflow.keras.initializers import RandomNormal, RandomUniform" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "def rename_layer(model_config, src_name, dst_name):\n", 48 | " for layer_item in model_config['input_layers']:\n", 49 | " input_condition = layer_item[0] == src_name\n", 50 | " if input_condition:\n", 51 | " # print('Rename Input:: ', layer_item[0])\n", 52 | " layer_item[0] = dst_name\n", 53 | "\n", 54 | " for layer_item in model_config['output_layers']:\n", 55 | " output_condition = layer_item[0] == src_name\n", 56 | " if output_condition:\n", 57 | " # print('Rename Output:: ', layer_item[0])\n", 58 | " layer_item[0] = dst_name\n", 59 | " # print(rename_condition, layer_list[0])\n", 60 | "\n", 61 | " for layer_item in model_config['layers']:\n", 62 | " name_condition = layer_item['name'] == src_name\n", 63 | " if name_condition:\n", 64 | " # print('Rename Layers:: name', layer_item['name'])\n", 65 | " layer_item['name'] = dst_name\n", 66 | "\n", 67 | " name_condition = layer_item['config']['name'] == src_name\n", 68 | " if name_condition:\n", 69 | " # print('Rename Layers Config:: config', layer_item['config']['name'])\n", 70 | " layer_item['config']['name'] = dst_name\n", 71 | " # print(' Rename Layers Config:: config', layer_item['config']['name'])\n", 72 | "\n", 73 | " if len(layer_item['inbound_nodes']) > 0:\n", 74 | " for item in layer_item['inbound_nodes'][0]:\n", 75 | " inbound_condition = item[0] == src_name\n", 76 | " if inbound_condition:\n", 77 | " # print('Rename Layers Inbound::', item[0])\n", 78 | " item[0] = dst_name\n", 79 | " return model_config" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "placeholder = Input((32, 32, 3), name='data')\n", 89 | "x = a = DepthwiseConv2D(4, (7,7), padding='same', name='src', bias_initializer=RandomNormal())(placeholder)\n", 90 | "x = ReLU(name='no_name')(x)\n", 91 | "# x = Add()([a, x])\n", 92 | "src_model = Model(placeholder, x, name='src_model')\n", 93 | "\n", 94 | "placeholder = Input((32, 32, 3), name='data')\n", 95 | "x = DepthwiseConv2D(4, (7, 7), padding='same', name='dst', activation='relu')(placeholder)\n", 96 | "dst_model = Model(placeholder, x, name='dst_model')" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": null, 102 | "metadata": {}, 103 | "outputs": [], 104 | "source": [] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": null, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | "weight, bias = src_model.get_layer('src').get_weights()\n", 113 | "dst_model.get_layer('dst').set_weights([weight, bias])" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": null, 119 | "metadata": {}, 120 | "outputs": [], 121 | "source": [ 122 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 123 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "def transfer_Conv2DReLU_Conv2D(src_model, dst_model, transfer_rule):\n", 133 | " dst_model.get_layer(transfer_rule['dst']).set_weights(src_model.get_layer(transfer_rule['src']).get_weights())\n", 134 | "\n", 135 | "\n", 136 | "def get_outbound_nodes(keras_config):\n", 137 | " outbound_dict = {}\n", 138 | " index_dict = {}\n", 139 | " for _i, _layer in enumerate(keras_config['layers']):\n", 140 | " out_node_name = _layer['name']\n", 141 | " index_dict[out_node_name] = _i\n", 142 | " if len(_layer['inbound_nodes']) == 0:\n", 143 | " continue\n", 144 | " in_node_name = _layer['inbound_nodes'][0][0][0]\n", 145 | " if in_node_name in outbound_dict:\n", 146 | " outbound_dict[in_node_name] += [out_node_name]\n", 147 | " else:\n", 148 | " outbound_dict[in_node_name] = [out_node_name]\n", 149 | "\n", 150 | " return outbound_dict, index_dict\n", 151 | "\n", 152 | "\n", 153 | "def detect_transform_Conv2DReLU(keras_config):\n", 154 | " index_list = []\n", 155 | " outbound_dict, index_dict = get_outbound_nodes(keras_config)\n", 156 | " for i, item in enumerate(keras_config['layers']):\n", 157 | " if item['class_name'] == 'ReLU':\n", 158 | " in_node_name = item['inbound_nodes'][0][0][0]\n", 159 | " in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name']\n", 160 | " if in_node_class_name in ['Conv2D', 'DepthwiseConv2D']:\n", 161 | " _activation = keras_config['layers'][index_dict[in_node_name]]['config']['activation']\n", 162 | " if _activation in ['linear', 'relu']:\n", 163 | " if len(outbound_dict[in_node_name]) == 1:\n", 164 | " index_list.append(i)\n", 165 | " return index_list, index_dict\n", 166 | "\n", 167 | "\n", 168 | "def check_Conv2DReLU(keras_config):\n", 169 | " return len(detect_transform_Conv2DReLU(keras_config)) > 0\n", 170 | "\n", 171 | "\n", 172 | "def apply_transform_Conv2DReLU(keras_config):\n", 173 | " index_list, index_dict = detect_transform_Conv2DReLU(keras_config)\n", 174 | " weight_transfer_rule_dict = {}\n", 175 | " while len(index_list) > 0:\n", 176 | " i = index_list[0]\n", 177 | " r_layer_config = keras_config['layers'].pop(i)\n", 178 | " src_name = r_layer_config['name']\n", 179 | " dst_name = r_layer_config['inbound_nodes'][0][0][0]\n", 180 | " keras_config['layers'][index_dict[dst_name]]['config']['activation'] = 'relu'\n", 181 | " transfer_call = transfer_Conv2DReLU_Conv2D\n", 182 | "\n", 183 | " keras_config = rename_layer(keras_config, src_name, dst_name)\n", 184 | " merged_dst = dst_name + '_M'\n", 185 | " keras_config = rename_layer(keras_config, dst_name, merged_dst)\n", 186 | " weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_call,\n", 187 | " 'src': dst_name,\n", 188 | " 'dst': merged_dst}\n", 189 | " index_list, index_dict = detect_transform_Conv2DReLU(keras_config)\n", 190 | " return keras_config, weight_transfer_rule_dict\n" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": {}, 197 | "outputs": [], 198 | "source": [ 199 | "def transfer_weights(src_model, dst_model, weight_transfer_rule_dict):\n", 200 | " for dst_layer in tqdm(dst_model.layers):\n", 201 | " if dst_layer.name in weight_transfer_rule_dict:\n", 202 | " transfer_rule = weight_transfer_rule_dict[dst_layer.name]\n", 203 | " func = transfer_rule['transfer_call']\n", 204 | " func(src_model, dst_model, transfer_rule)\n", 205 | " else:\n", 206 | " src_model.get_layer(dst_layer.name).set_weights(dst_layer.get_weights())" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": null, 212 | "metadata": {}, 213 | "outputs": [], 214 | "source": [ 215 | "dst_model_config, weight_transfer_rule_dict = apply_transform_Conv2DReLU(src_model.get_config())" 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": null, 221 | "metadata": {}, 222 | "outputs": [], 223 | "source": [ 224 | "dst_model = Model.from_config(dst_model_config)" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": null, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "transfer_weights(src_model, dst_model, weight_transfer_rule_dict)" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": null, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 243 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "dst_model.summary()" 253 | ] 254 | } 255 | ], 256 | "metadata": { 257 | "kernelspec": { 258 | "display_name": "Python 3", 259 | "language": "python", 260 | "name": "python3" 261 | }, 262 | "language_info": { 263 | "codemirror_mode": { 264 | "name": "ipython", 265 | "version": 3 266 | }, 267 | "file_extension": ".py", 268 | "mimetype": "text/x-python", 269 | "name": "python", 270 | "nbconvert_exporter": "python", 271 | "pygments_lexer": "ipython3", 272 | "version": "3.7.6" 273 | } 274 | }, 275 | "nbformat": 4, 276 | "nbformat_minor": 4 277 | } 278 | -------------------------------------------------------------------------------- /_step_by_step/model_converter_merge_conv_batchnormalization.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n", 11 | "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "from copy import deepcopy\n", 21 | "import imageio\n", 22 | "import numpy as np\n", 23 | "import cv2\n", 24 | "import matplotlib.pyplot as plt\n", 25 | "import keras\n", 26 | "import numpy as np\n", 27 | "import tensorflow as tf\n", 28 | "import tensorflow.keras.backend as K\n", 29 | "from tqdm import tqdm\n", 30 | "from tensorflow.keras import Input\n", 31 | "from tensorflow.keras.models import Model\n", 32 | "from tensorflow.keras.layers import Layer, InputSpec, DepthwiseConv2D, BatchNormalization, Activation, Conv2D, Add, Conv2DTranspose\n", 33 | "from tensorflow.keras import backend as K\n", 34 | "from tensorflow.keras.backend import int_shape, permute_dimensions\n", 35 | "from collections import OrderedDict\n", 36 | "from tensorflow.keras.models import Model, load_model\n", 37 | "from tensorflow.keras.layers import Input, SeparableConv2D\n", 38 | "from tensorflow.keras.initializers import RandomNormal, RandomUniform" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "def rename_layer(model_config, src_name, dst_name):\n", 48 | " for layer_item in model_config['input_layers']:\n", 49 | " input_condition = layer_item[0] == src_name\n", 50 | " if input_condition:\n", 51 | " # print('Rename Input:: ', layer_item[0])\n", 52 | " layer_item[0] = dst_name\n", 53 | "\n", 54 | " for layer_item in model_config['output_layers']:\n", 55 | " output_condition = layer_item[0] == src_name\n", 56 | " if output_condition:\n", 57 | " # print('Rename Output:: ', layer_item[0])\n", 58 | " layer_item[0] = dst_name\n", 59 | " # print(rename_condition, layer_list[0])\n", 60 | "\n", 61 | " for layer_item in model_config['layers']:\n", 62 | " name_condition = layer_item['name'] == src_name\n", 63 | " if name_condition:\n", 64 | " # print('Rename Layers:: name', layer_item['name'])\n", 65 | " layer_item['name'] = dst_name\n", 66 | "\n", 67 | " name_condition = layer_item['config']['name'] == src_name\n", 68 | " if name_condition:\n", 69 | " # print('Rename Layers Config:: config', layer_item['config']['name'])\n", 70 | " layer_item['config']['name'] = dst_name\n", 71 | " # print(' Rename Layers Config:: config', layer_item['config']['name'])\n", 72 | "\n", 73 | " if len(layer_item['inbound_nodes']) > 0:\n", 74 | " for item in layer_item['inbound_nodes'][0]:\n", 75 | " inbound_condition = item[0] == src_name\n", 76 | " if inbound_condition:\n", 77 | " # print('Rename Layers Inbound::', item[0])\n", 78 | " item[0] = dst_name\n", 79 | " return model_config" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "Conv2D_config_template = {\n", 89 | " 'name': '_node_name',\n", 90 | " 'class_name': 'Conv2D',\n", 91 | " 'config': {'name': '_node_name',\n", 92 | " 'trainable': True,\n", 93 | " 'dtype': 'float32',\n", 94 | " 'filters': 1,\n", 95 | " 'kernel_size': (1, 1),\n", 96 | " 'strides': (1, 1),\n", 97 | " 'padding': 'same',\n", 98 | " 'data_format': 'channels_last',\n", 99 | " 'dilation_rate': (1, 1),\n", 100 | " 'activation': 'linear',\n", 101 | " 'use_bias': True,\n", 102 | " 'kernel_initializer': {'class_name': 'GlorotUniform',\n", 103 | " 'config': {'seed': None}},\n", 104 | " 'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n", 105 | " 'kernel_regularizer': None,\n", 106 | " 'bias_regularizer': None,\n", 107 | " 'activity_regularizer': None,\n", 108 | " 'kernel_constraint': None,\n", 109 | " 'bias_constraint': None},\n", 110 | " 'name': '_node_name',\n", 111 | " 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]}" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": null, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "placeholder = Input((32, 32, 3), name='data')\n", 121 | "x = a = Conv2D(4, (7,7), padding='same', name='src_1', bias_initializer=RandomNormal())(placeholder)\n", 122 | "x = BatchNormalization(name='src_2', \n", 123 | " beta_initializer=RandomNormal(),\n", 124 | " gamma_initializer=RandomNormal(),\n", 125 | " moving_mean_initializer=RandomNormal(),\n", 126 | " moving_variance_initializer=RandomUniform(1,2))(x)\n", 127 | "# x = Add()([a, x])\n", 128 | "src_model = Model(placeholder, x, name='src_model')\n", 129 | "\n", 130 | "placeholder = Input((32, 32, 3), name='data')\n", 131 | "x = Conv2D(4, (7, 7), padding='same', name='dst_1')(placeholder)\n", 132 | "dst_model = Model(placeholder, x, name='dst_model')" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "weight, bias = src_model.get_layer('src_1').get_weights()\n", 142 | "gamma, beta, mean, var = src_model.get_layer('src_2').get_weights()\n", 143 | "eps = src_model.get_layer('src_2').get_config()['epsilon']\n", 144 | "a = gamma / np.sqrt(var + eps)\n", 145 | "weight = weight*a.reshape((1,1,-1,1))\n", 146 | "bias = a*(bias - mean) + beta\n", 147 | "dst_model.get_layer('dst_1').set_weights([weight, bias])" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": null, 153 | "metadata": {}, 154 | "outputs": [], 155 | "source": [ 156 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 157 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "def transfer_Conv2DBatchNormalization_Conv2D(src_model, dst_model, transfer_rule):\n", 167 | " layer_c = src_model.get_layer(transfer_rule['src_c'])\n", 168 | " weigths_c = layer_c.get_weights()\n", 169 | " layer_b = src_model.get_layer(transfer_rule['src_b'])\n", 170 | " weigths_b = layer_b.get_weights()\n", 171 | " \n", 172 | " eps = layer_b.get_config()['epsilon']\n", 173 | " weight, bias = weigths_c\n", 174 | " gamma, beta, mean, var = weigths_b\n", 175 | " \n", 176 | " a = gamma / np.sqrt(var + eps)\n", 177 | " weight = weight*a\n", 178 | " bias = a*(bias - mean) + beta\n", 179 | " \n", 180 | " dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias])\n", 181 | "\n", 182 | "def get_outbound_nodes(keras_config):\n", 183 | " outbound_dict = {}\n", 184 | " index_dict = {}\n", 185 | " for _i, _layer in enumerate(keras_config['layers']):\n", 186 | " out_node_name = _layer['name']\n", 187 | " index_dict[out_node_name] = _i\n", 188 | " if len(_layer['inbound_nodes']) == 0:\n", 189 | " continue\n", 190 | " in_node_name = _layer['inbound_nodes'][0][0][0]\n", 191 | " if in_node_name in outbound_dict:\n", 192 | " outbound_dict[in_node_name] += [out_node_name]\n", 193 | " else:\n", 194 | " outbound_dict[in_node_name] = [out_node_name]\n", 195 | " \n", 196 | " return outbound_dict, index_dict\n", 197 | " \n", 198 | "def detect_transform_Conv2DBatchNormalization(keras_config):\n", 199 | " index_list = []\n", 200 | " outbound_dict, index_dict = get_outbound_nodes(keras_config)\n", 201 | " for i, item in enumerate(keras_config['layers']):\n", 202 | " if item['class_name'] == 'BatchNormalization':\n", 203 | " in_node_name = item['inbound_nodes'][0][0][0]\n", 204 | " in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name']\n", 205 | " if in_node_class_name == 'Conv2D':\n", 206 | " if len(outbound_dict[in_node_name]) == 1:\n", 207 | " index_list.append(i)\n", 208 | " return index_list\n", 209 | "\n", 210 | "def apply_transform_Conv2DBatchNormalization(keras_config):\n", 211 | " index_list = detect_transform_Conv2DBatchNormalization(keras_config)\n", 212 | " weight_transfer_rule_dict = {}\n", 213 | " while len(index_list) > 0:\n", 214 | " i = index_list[0]\n", 215 | " r_layer_config = keras_config['layers'].pop(i)\n", 216 | " src_name = r_layer_config['name']\n", 217 | " dst_name = r_layer_config['inbound_nodes'][0][0][0]\n", 218 | " keras_config = rename_layer(keras_config, src_name, dst_name)\n", 219 | " merged_dst = dst_name + '_M'\n", 220 | " keras_config = rename_layer(keras_config, dst_name, merged_dst)\n", 221 | " weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_Conv2DBatchNormalization_Conv2D,\n", 222 | " 'src_c': dst_name, \n", 223 | " 'src_b': src_name, \n", 224 | " 'dst':merged_dst} \n", 225 | " index_list = detect_transform_Conv2DBatchNormalization(keras_config)\n", 226 | " return keras_config, weight_transfer_rule_dict" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "def transfer_weights(src_model, dst_model, weight_transfer_rule_dict):\n", 236 | " for dst_layer in tqdm(dst_model.layers):\n", 237 | " if dst_layer.name in weight_transfer_rule_dict:\n", 238 | " transfer_rule = weight_transfer_rule_dict[dst_layer.name]\n", 239 | " func = transfer_rule['transfer_call']\n", 240 | " func(src_model, dst_model, transfer_rule)\n", 241 | " else:\n", 242 | " src_model.get_layer(dst_layer.name).set_weights(dst_layer.get_weights())" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "metadata": {}, 249 | "outputs": [], 250 | "source": [ 251 | "dst_model_config, weight_transfer_rule_dict = apply_transform_Conv2DBatchNormalization(src_model.get_config())" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": null, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "dst_model = Model.from_config(dst_model_config)" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": null, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [ 269 | "transfer_weights(src_model, dst_model, weight_transfer_rule_dict)" 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": null, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [ 278 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 279 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": {}, 293 | "outputs": [], 294 | "source": [] 295 | }, 296 | { 297 | "cell_type": "code", 298 | "execution_count": null, 299 | "metadata": {}, 300 | "outputs": [], 301 | "source": [] 302 | } 303 | ], 304 | "metadata": { 305 | "kernelspec": { 306 | "display_name": "Python 3", 307 | "language": "python", 308 | "name": "python3" 309 | }, 310 | "language_info": { 311 | "codemirror_mode": { 312 | "name": "ipython", 313 | "version": 3 314 | }, 315 | "file_extension": ".py", 316 | "mimetype": "text/x-python", 317 | "name": "python", 318 | "nbconvert_exporter": "python", 319 | "pygments_lexer": "ipython3", 320 | "version": "3.7.6" 321 | } 322 | }, 323 | "nbformat": 4, 324 | "nbformat_minor": 4 325 | } 326 | -------------------------------------------------------------------------------- /_step_by_step/model_converter_merge_depthwiseconv_batchnormalization.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n", 11 | "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "from copy import deepcopy\n", 21 | "import imageio\n", 22 | "import numpy as np\n", 23 | "import cv2\n", 24 | "import matplotlib.pyplot as plt\n", 25 | "import keras\n", 26 | "import numpy as np\n", 27 | "import tensorflow as tf\n", 28 | "import tensorflow.keras.backend as K\n", 29 | "from tqdm import tqdm\n", 30 | "from tensorflow.keras import Input\n", 31 | "from tensorflow.keras.models import Model\n", 32 | "from tensorflow.keras.layers import Layer, InputSpec, DepthwiseConv2D, BatchNormalization, Activation, Conv2D, Add, Conv2DTranspose\n", 33 | "from tensorflow.keras import backend as K\n", 34 | "from tensorflow.keras.backend import int_shape, permute_dimensions\n", 35 | "from collections import OrderedDict\n", 36 | "from tensorflow.keras.models import Model, load_model\n", 37 | "from tensorflow.keras.layers import Input, SeparableConv2D\n", 38 | "from tensorflow.keras.initializers import RandomNormal, RandomUniform" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "def rename_layer(model_config, src_name, dst_name):\n", 48 | " for layer_item in model_config['input_layers']:\n", 49 | " input_condition = layer_item[0] == src_name\n", 50 | " if input_condition:\n", 51 | " # print('Rename Input:: ', layer_item[0])\n", 52 | " layer_item[0] = dst_name\n", 53 | "\n", 54 | " for layer_item in model_config['output_layers']:\n", 55 | " output_condition = layer_item[0] == src_name\n", 56 | " if output_condition:\n", 57 | " # print('Rename Output:: ', layer_item[0])\n", 58 | " layer_item[0] = dst_name\n", 59 | " # print(rename_condition, layer_list[0])\n", 60 | "\n", 61 | " for layer_item in model_config['layers']:\n", 62 | " name_condition = layer_item['name'] == src_name\n", 63 | " if name_condition:\n", 64 | " # print('Rename Layers:: name', layer_item['name'])\n", 65 | " layer_item['name'] = dst_name\n", 66 | "\n", 67 | " name_condition = layer_item['config']['name'] == src_name\n", 68 | " if name_condition:\n", 69 | " # print('Rename Layers Config:: config', layer_item['config']['name'])\n", 70 | " layer_item['config']['name'] = dst_name\n", 71 | " # print(' Rename Layers Config:: config', layer_item['config']['name'])\n", 72 | "\n", 73 | " if len(layer_item['inbound_nodes']) > 0:\n", 74 | " for item in layer_item['inbound_nodes'][0]:\n", 75 | " inbound_condition = item[0] == src_name\n", 76 | " if inbound_condition:\n", 77 | " # print('Rename Layers Inbound::', item[0])\n", 78 | " item[0] = dst_name\n", 79 | " return model_config" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "placeholder = Input((32, 32, 3), name='data')\n", 89 | "x = a = DepthwiseConv2D(4, (7,7), padding='same', name='src_1', bias_initializer=RandomNormal())(placeholder)\n", 90 | "x = BatchNormalization(name='src_2', \n", 91 | " beta_initializer=RandomNormal(),\n", 92 | " gamma_initializer=RandomNormal(),\n", 93 | " moving_mean_initializer=RandomNormal(),\n", 94 | " moving_variance_initializer=RandomUniform(1,2))(x)\n", 95 | "# x = Add()([a, x])\n", 96 | "src_model = Model(placeholder, x, name='src_model')\n", 97 | "\n", 98 | "placeholder = Input((32, 32, 3), name='data')\n", 99 | "x = DepthwiseConv2D(4, (7, 7), padding='same', name='dst_1')(placeholder)\n", 100 | "dst_model = Model(placeholder, x, name='dst_model')" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "a.shape" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "weight, bias = src_model.get_layer('src_1').get_weights()\n", 119 | "gamma, beta, mean, var = src_model.get_layer('src_2').get_weights()\n", 120 | "eps = src_model.get_layer('src_2').get_config()['epsilon']\n", 121 | "a = (gamma / np.sqrt(var + eps))\n", 122 | "weight = weight*a.reshape((1,1,-1,1))\n", 123 | "bias = a*(bias - mean) + beta\n", 124 | "dst_model.get_layer('dst_1').set_weights([weight, bias])" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": null, 130 | "metadata": {}, 131 | "outputs": [], 132 | "source": [ 133 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 134 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "def transfer_Conv2DBatchNormalization_Conv2D(src_model, dst_model, transfer_rule):\n", 144 | " layer_c = src_model.get_layer(transfer_rule['src_c'])\n", 145 | " weigths_c = layer_c.get_weights()\n", 146 | " layer_b = src_model.get_layer(transfer_rule['src_b'])\n", 147 | " weigths_b = layer_b.get_weights()\n", 148 | " \n", 149 | " eps = layer_b.get_config()['epsilon']\n", 150 | " weight, bias = weigths_c\n", 151 | " gamma, beta, mean, var = weigths_b\n", 152 | " \n", 153 | " a = gamma / np.sqrt(var + eps)\n", 154 | " weight = weight*a\n", 155 | " bias = a*(bias - mean) + beta\n", 156 | " \n", 157 | " dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias])\n", 158 | "\n", 159 | "def get_outbound_nodes(keras_config):\n", 160 | " outbound_dict = {}\n", 161 | " index_dict = {}\n", 162 | " for _i, _layer in enumerate(keras_config['layers']):\n", 163 | " out_node_name = _layer['name']\n", 164 | " index_dict[out_node_name] = _i\n", 165 | " if len(_layer['inbound_nodes']) == 0:\n", 166 | " continue\n", 167 | " in_node_name = _layer['inbound_nodes'][0][0][0]\n", 168 | " if in_node_name in outbound_dict:\n", 169 | " outbound_dict[in_node_name] += [out_node_name]\n", 170 | " else:\n", 171 | " outbound_dict[in_node_name] = [out_node_name]\n", 172 | " \n", 173 | " return outbound_dict, index_dict\n", 174 | " \n", 175 | "def detect_transform_Conv2DBatchNormalization(keras_config):\n", 176 | " index_list = []\n", 177 | " outbound_dict, index_dict = get_outbound_nodes(keras_config)\n", 178 | " for i, item in enumerate(keras_config['layers']):\n", 179 | " if item['class_name'] == 'BatchNormalization':\n", 180 | " in_node_name = item['inbound_nodes'][0][0][0]\n", 181 | " in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name']\n", 182 | " if in_node_class_name == 'Conv2D':\n", 183 | " if len(outbound_dict[in_node_name]) == 1:\n", 184 | " index_list.append(i)\n", 185 | " return index_list\n", 186 | "\n", 187 | "def apply_transform_Conv2DBatchNormalization(keras_config):\n", 188 | " index_list = detect_transform_Conv2DBatchNormalization(keras_config)\n", 189 | " weight_transfer_rule_dict = {}\n", 190 | " while len(index_list) > 0:\n", 191 | " i = index_list[0]\n", 192 | " r_layer_config = keras_config['layers'].pop(i)\n", 193 | " src_name = r_layer_config['name']\n", 194 | " dst_name = r_layer_config['inbound_nodes'][0][0][0]\n", 195 | " keras_config = rename_layer(keras_config, src_name, dst_name)\n", 196 | " merged_dst = dst_name + '_M'\n", 197 | " keras_config = rename_layer(keras_config, dst_name, merged_dst)\n", 198 | " weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_Conv2DBatchNormalization_Conv2D,\n", 199 | " 'src_c': dst_name, \n", 200 | " 'src_b': src_name, \n", 201 | " 'dst':merged_dst} \n", 202 | " index_list = detect_transform_Conv2DBatchNormalization(keras_config)\n", 203 | " return keras_config, weight_transfer_rule_dict" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": null, 209 | "metadata": {}, 210 | "outputs": [], 211 | "source": [ 212 | "def transfer_weights(src_model, dst_model, weight_transfer_rule_dict):\n", 213 | " for dst_layer in tqdm(dst_model.layers):\n", 214 | " if dst_layer.name in weight_transfer_rule_dict:\n", 215 | " transfer_rule = weight_transfer_rule_dict[dst_layer.name]\n", 216 | " func = transfer_rule['transfer_call']\n", 217 | " func(src_model, dst_model, transfer_rule)\n", 218 | " else:\n", 219 | " src_model.get_layer(dst_layer.name).set_weights(dst_layer.get_weights())" 220 | ] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "execution_count": null, 225 | "metadata": {}, 226 | "outputs": [], 227 | "source": [ 228 | "dst_model_config, weight_transfer_rule_dict = apply_transform_Conv2DBatchNormalization(src_model.get_config())" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": null, 234 | "metadata": {}, 235 | "outputs": [], 236 | "source": [ 237 | "dst_model = Model.from_config(dst_model_config)" 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "execution_count": null, 243 | "metadata": {}, 244 | "outputs": [], 245 | "source": [ 246 | "transfer_weights(src_model, dst_model, weight_transfer_rule_dict)" 247 | ] 248 | }, 249 | { 250 | "cell_type": "code", 251 | "execution_count": null, 252 | "metadata": {}, 253 | "outputs": [], 254 | "source": [ 255 | "x_in = np.random.uniform(size=(1,) + dst_model.input_shape[1:])\n", 256 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": null, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "dst_model.summary()" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": null, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": {}, 279 | "outputs": [], 280 | "source": [] 281 | } 282 | ], 283 | "metadata": { 284 | "kernelspec": { 285 | "display_name": "Python 3", 286 | "language": "python", 287 | "name": "python3" 288 | }, 289 | "language_info": { 290 | "codemirror_mode": { 291 | "name": "ipython", 292 | "version": 3 293 | }, 294 | "file_extension": ".py", 295 | "mimetype": "text/x-python", 296 | "name": "python", 297 | "nbconvert_exporter": "python", 298 | "pygments_lexer": "ipython3", 299 | "version": "3.7.6" 300 | } 301 | }, 302 | "nbformat": 4, 303 | "nbformat_minor": 4 304 | } 305 | -------------------------------------------------------------------------------- /_step_by_step/model_converter_split_separableconv2d.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import os\n", 10 | "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n", 11 | "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "from copy import deepcopy\n", 21 | "import imageio\n", 22 | "import numpy as np\n", 23 | "import cv2\n", 24 | "import matplotlib.pyplot as plt\n", 25 | "import keras\n", 26 | "import numpy as np\n", 27 | "import tensorflow as tf\n", 28 | "import tensorflow.keras.backend as K\n", 29 | "from tqdm import tqdm\n", 30 | "from tensorflow.keras import Input\n", 31 | "from tensorflow.keras.models import Model\n", 32 | "from tensorflow.keras.layers import Layer, InputSpec, DepthwiseConv2D, BatchNormalization, Activation, Conv2D, concatenate, Conv2DTranspose\n", 33 | "from tensorflow.keras import backend as K\n", 34 | "from tensorflow.keras.backend import int_shape, permute_dimensions\n", 35 | "from collections import OrderedDict\n", 36 | "from tensorflow.keras.models import Model, load_model\n", 37 | "from tensorflow.keras.layers import Input, SeparableConv2D" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "Conv2D_config_template = {\n", 47 | " 'name': '_node_name',\n", 48 | " 'class_name': 'Conv2D',\n", 49 | " 'config': {'name': '_node_name',\n", 50 | " 'trainable': True,\n", 51 | " 'dtype': 'float32',\n", 52 | " 'filters': 1,\n", 53 | " 'kernel_size': (1, 1),\n", 54 | " 'strides': (1, 1),\n", 55 | " 'padding': 'same',\n", 56 | " 'data_format': 'channels_last',\n", 57 | " 'dilation_rate': (1, 1),\n", 58 | " 'activation': 'linear',\n", 59 | " 'use_bias': True,\n", 60 | " 'kernel_initializer': {'class_name': 'GlorotUniform',\n", 61 | " 'config': {'seed': None}},\n", 62 | " 'bias_initializer': {'class_name': 'Zeros', 'config': {}},\n", 63 | " 'kernel_regularizer': None,\n", 64 | " 'bias_regularizer': None,\n", 65 | " 'activity_regularizer': None,\n", 66 | " 'kernel_constraint': None,\n", 67 | " 'bias_constraint': None},\n", 68 | " 'name': '_node_name',\n", 69 | " 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]}\n", 70 | "\n", 71 | "DepthwiseConv2D_config_template = {'name': '_node_name',\n", 72 | " 'class_name': 'DepthwiseConv2D',\n", 73 | " 'config': {'name': '_node_name',\n", 74 | " 'trainable': True,\n", 75 | " 'dtype': 'float32',\n", 76 | " 'kernel_size': (3, 3),\n", 77 | " 'strides': (1, 1),\n", 78 | " 'padding': 'same',\n", 79 | " 'data_format': 'channels_last',\n", 80 | " 'dilation_rate': (1, 1),\n", 81 | " 'activation': 'linear',\n", 82 | " 'use_bias': False,\n", 83 | " 'bias_initializer': {'class_name': 'Zeros', 'config': {'dtype': 'float32'}},\n", 84 | " 'bias_regularizer': None,\n", 85 | " 'activity_regularizer': None,\n", 86 | " 'bias_constraint': None,\n", 87 | " 'depth_multiplier': 1,\n", 88 | " 'depthwise_initializer': {'class_name': 'GlorotUniform',\n", 89 | " 'config': {'seed': None, 'dtype': 'float32'}},\n", 90 | " 'depthwise_regularizer': None,\n", 91 | " 'depthwise_constraint': None},\n", 92 | " 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]}" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "placeholder = Input((32, 32, 3), name='data')\n", 102 | "x = SeparableConv2D(4, (5,5), padding='same', name='src_layer')(placeholder)\n", 103 | "src_model = Model(placeholder, x, name='src_model')\n", 104 | "\n", 105 | "placeholder = Input((32, 32, 3), name='data')\n", 106 | "x = DepthwiseConv2D((5,5), padding='same', name='dst_1', use_bias=False)(placeholder)\n", 107 | "x = Conv2D(4, (1, 1), padding='same', name='dst_2')(x)\n", 108 | "dst_model = Model(placeholder, x, name='dst_model')" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "metadata": {}, 115 | "outputs": [], 116 | "source": [ 117 | "w_1, w_2, b = src_model.get_layer('src_layer').get_weights()\n", 118 | "dst_model.get_layer('dst_1').set_weights([w_1])\n", 119 | "dst_model.get_layer('dst_2').set_weights([w_2, b])" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": null, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "x_in = np.random.uniform(size=(1, 32, 32, 3))\n", 129 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": null, 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "def transfer_SeparableConv2D_DepthwiseConv2D(src_model, dst_model, transfer_rule):\n", 146 | " _weigths = src_model.get_layer(transfer_rule['src']).get_weights()\n", 147 | " dst_model.get_layer(transfer_rule['dst']).set_weights([_weigths[0]])\n", 148 | " \n", 149 | "def transfer_SeparableConv2D_Conv2D(src_model, dst_model, transfer_rule):\n", 150 | " _weigths = src_model.get_layer(transfer_rule['src']).get_weights()\n", 151 | "# if len(_weigths) == 3:\n", 152 | "# w_1, w_2, b = _weigths\n", 153 | "# assert len(_weigths) == 3, 'Check use_bias==False rule'\n", 154 | " dst_model.get_layer(transfer_rule['dst']).set_weights(_weigths[1:])\n", 155 | "\n", 156 | "# detect correction\n", 157 | "def detect_transform_SeparableConv2D(keras_config):\n", 158 | " index_list = []\n", 159 | " for i, item in enumerate(keras_config['layers']):\n", 160 | " if item['class_name'] == 'SeparableConv2D':\n", 161 | " index_list.append(i)\n", 162 | " return index_list\n", 163 | "\n", 164 | "def apply_transform_SeparableConv2D(keras_config):\n", 165 | " index_list = detect_transform_SeparableConv2D(keras_config)\n", 166 | " weight_transfer_rule_dict = {}\n", 167 | " while len(index_list) > 0:\n", 168 | " i = index_list[0]\n", 169 | " r_layer_config = keras_config['layers'].pop(i)\n", 170 | " # Transfer DepthWise\n", 171 | " i_layer_config = deepcopy(DepthwiseConv2D_config_template)\n", 172 | " #TODO :: check unique name\n", 173 | " prev_name = i_layer_config['name'] = r_layer_config['name'] + f'_dwc_{i}'\n", 174 | " for key in DepthwiseConv2D_config_template['config'].keys():\n", 175 | " if key in r_layer_config['config']:\n", 176 | " i_layer_config['config'][key] = r_layer_config['config'][key]\n", 177 | " i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes']\n", 178 | " i_layer_config['config']['name'] = i_layer_config['name']\n", 179 | " i_layer_config['config']['use_bias'] = False\n", 180 | " keras_config['layers'].insert(i, i_layer_config)\n", 181 | " weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_SeparableConv2D_DepthwiseConv2D,\n", 182 | " 'src': r_layer_config['name'], 'dst':i_layer_config['name']}\n", 183 | " \n", 184 | " \n", 185 | " # Transfer Conv\n", 186 | " i_layer_config = deepcopy(Conv2D_config_template)\n", 187 | " for key in set(Conv2D_config_template['config'].keys()) - set(['kernel_size', \n", 188 | " 'strides', 'dilation_rate']):\n", 189 | " if key in r_layer_config['config']:\n", 190 | " i_layer_config['config'][key] = r_layer_config['config'][key]\n", 191 | " i_layer_config['name'] = r_layer_config['name']\n", 192 | " i_layer_config['inbound_nodes'] = [[[prev_name, 0, 0, {}]]]\n", 193 | " \n", 194 | " keras_config['layers'].insert(i+1, i_layer_config)\n", 195 | " weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_SeparableConv2D_Conv2D,\n", 196 | " 'src': r_layer_config['name'], 'dst':i_layer_config['name']}\n", 197 | " \n", 198 | " index_list = detect_transform_SeparableConv2D(keras_config)\n", 199 | " return keras_config, weight_transfer_rule_dict" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "def transfer_weights(src_model, dst_model, weight_transfer_rule_dict):\n", 209 | " for dst_layer in tqdm(dst_model.layers):\n", 210 | " if dst_layer.name in weight_transfer_rule_dict:\n", 211 | " transfer_rule = weight_transfer_rule_dict[dst_layer.name]\n", 212 | " func = transfer_rule['transfer_call']\n", 213 | " func(src_model, dst_model, transfer_rule)\n", 214 | " else:\n", 215 | " src_model.get_layer(dst_layer.name).set_weights(dst_layer.get_weights())" 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": null, 221 | "metadata": {}, 222 | "outputs": [], 223 | "source": [ 224 | "dst_model_config, weight_transfer_rule_dict = apply_transform_SeparableConv2D(src_model.get_config())" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": null, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "dst_model = Model.from_config(dst_model_config)" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": null, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "transfer_weights(src_model, dst_model, weight_transfer_rule_dict)" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "metadata": {}, 249 | "outputs": [], 250 | "source": [ 251 | "x_in = np.random.uniform(size=(1,) + _dst_model.input_shape[1:])\n", 252 | "print(np.abs(dst_model.predict(x_in) - src_model.predict(x_in)).sum())" 253 | ] 254 | } 255 | ], 256 | "metadata": { 257 | "kernelspec": { 258 | "display_name": "Python 3", 259 | "language": "python", 260 | "name": "python3" 261 | }, 262 | "language_info": { 263 | "codemirror_mode": { 264 | "name": "ipython", 265 | "version": 3 266 | }, 267 | "file_extension": ".py", 268 | "mimetype": "text/x-python", 269 | "name": "python", 270 | "nbconvert_exporter": "python", 271 | "pygments_lexer": "ipython3", 272 | "version": "3.7.6" 273 | } 274 | }, 275 | "nbformat": 4, 276 | "nbformat_minor": 4 277 | } 278 | -------------------------------------------------------------------------------- /_step_by_step/unpack_model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [ 8 | { 9 | "name": "stdout", 10 | "output_type": "stream", 11 | "text": [ 12 | "2.5.0-dev20201130\n" 13 | ] 14 | } 15 | ], 16 | "source": [ 17 | "import tensorflow as tf\n", 18 | "print(tf.__version__)" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 2, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "from tensorflow.keras.models import load_model, Model\n", 28 | "import matplotlib.pyplot as plt\n", 29 | "import cv2" 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": 3, 35 | "metadata": {}, 36 | "outputs": [ 37 | { 38 | "name": "stdout", 39 | "output_type": "stream", 40 | "text": [ 41 | "WARNING:tensorflow:Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.\n" 42 | ] 43 | } 44 | ], 45 | "source": [ 46 | "# Read original model\n", 47 | "model = load_model('../model_zoo/variouse/issue_00003/fiop_dumb_model.h5')" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 4, 53 | "metadata": {}, 54 | "outputs": [ 55 | { 56 | "name": "stdout", 57 | "output_type": "stream", 58 | "text": [ 59 | "Model: \"sequential_1\"\n", 60 | "_________________________________________________________________\n", 61 | "Layer (type) Output Shape Param # \n", 62 | "=================================================================\n", 63 | "model_2 (Functional) (None, 3, 3, 2048) 5347520 \n", 64 | "_________________________________________________________________\n", 65 | "flatten_1 (Flatten) (None, 18432) 0 \n", 66 | "_________________________________________________________________\n", 67 | "dense_2 (Dense) (None, 12) 221196 \n", 68 | "_________________________________________________________________\n", 69 | "dense_3 (Dense) (None, 8) 104 \n", 70 | "_________________________________________________________________\n", 71 | "dense_4 (Dense) (None, 1) 9 \n", 72 | "=================================================================\n", 73 | "Total params: 5,568,829\n", 74 | "Trainable params: 5,540,797\n", 75 | "Non-trainable params: 28,032\n", 76 | "_________________________________________________________________\n" 77 | ] 78 | } 79 | ], 80 | "source": [ 81 | "model.summary()" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": 5, 87 | "metadata": {}, 88 | "outputs": [], 89 | "source": [ 90 | "# Select inner model\n", 91 | "inner_model = model.get_layer('model_2')" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 6, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "# Adjust config of the new model\n", 101 | "model_config = model.get_config()\n", 102 | "fixed_config = inner_model.get_config()\n", 103 | "\n", 104 | "i_layers = fixed_config['input_layers']\n", 105 | "o_layers = [[model_config['layers'][-1]['config']['name'], 0, 0]]\n", 106 | "# o_layers = [[['flatten_1',0,0]]]\n", 107 | "\n", 108 | "add_layers = []\n", 109 | "in_layer = fixed_config['layers'][-1]['name']\n", 110 | "for item in model_config['layers'][2:]:\n", 111 | " item['name'] = item['config']['name']\n", 112 | " item['inbound_nodes'] = [[[in_layer, 0, 0, {}]]]\n", 113 | " in_layer = item['config']['name']\n", 114 | " add_layers.append(item)\n", 115 | " \n", 116 | "fixed_config['layers'] += add_layers\n", 117 | "fixed_config['input_layers'] = i_layers\n", 118 | "fixed_config['output_layers'] = o_layers\n", 119 | "\n", 120 | "fixed_model = Model.from_config(fixed_config)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 7, 126 | "metadata": {}, 127 | "outputs": [], 128 | "source": [ 129 | "# Transfer weights to the new model\n", 130 | "for layer in inner_model.layers:\n", 131 | " fixed_model.get_layer(layer.name).set_weights(layer.get_weights())\n", 132 | " \n", 133 | "for layer in model.layers[2:]:\n", 134 | " fixed_model.get_layer(layer.name).set_weights(layer.get_weights())" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 8, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "in_data = cv2.imread('../unit_test/unit_test_data/person_640x640.jpg')\n", 144 | "in_data = cv2.resize(in_data, tuple(reversed(model.input_shape[1:3])))[None,...]" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": 9, 150 | "metadata": {}, 151 | "outputs": [ 152 | { 153 | "name": "stdout", 154 | "output_type": "stream", 155 | "text": [ 156 | "WARNING:tensorflow:AutoGraph could not transform .predict_function at 0x000001E8563AFF78> and will run it as-is.\n", 157 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 158 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 159 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 160 | "WARNING: AutoGraph could not transform .predict_function at 0x000001E8563AFF78> and will run it as-is.\n", 161 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 162 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 163 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 164 | "[[0.20348462]]\n", 165 | "WARNING:tensorflow:AutoGraph could not transform .predict_function at 0x000001E84AF52828> and will run it as-is.\n", 166 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 167 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 168 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 169 | "WARNING: AutoGraph could not transform .predict_function at 0x000001E84AF52828> and will run it as-is.\n", 170 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 171 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 172 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 173 | "[[0.20348462]]\n" 174 | ] 175 | } 176 | ], 177 | "source": [ 178 | "# Check the correctnes of the predictions\n", 179 | "print(model.predict(in_data))\n", 180 | "print(fixed_model.predict(in_data))" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": 11, 186 | "metadata": {}, 187 | "outputs": [], 188 | "source": [ 189 | "fixed_model.save('../model_zoo/variouse/issue_00003/fiop_dumb_model_fixed.h5')" 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": 14, 195 | "metadata": {}, 196 | "outputs": [ 197 | { 198 | "name": "stdout", 199 | "output_type": "stream", 200 | "text": [ 201 | "WARNING:tensorflow:AutoGraph could not transform ._wrapped_model at 0x000001E85C6B1DC8> and will run it as-is.\n", 202 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 203 | "Cause: module 'gast' has no attribute 'Constant'\n", 204 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 205 | "WARNING: AutoGraph could not transform ._wrapped_model at 0x000001E85C6B1DC8> and will run it as-is.\n", 206 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 207 | "Cause: module 'gast' has no attribute 'Constant'\n", 208 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 209 | "WARNING:tensorflow:AutoGraph could not transform .signature_wrapper at 0x000001E85F7DADC8> and will run it as-is.\n", 210 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 211 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 212 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 213 | "WARNING: AutoGraph could not transform .signature_wrapper at 0x000001E85F7DADC8> and will run it as-is.\n", 214 | "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n", 215 | "Cause: 'arguments' object has no attribute 'posonlyargs'\n", 216 | "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n", 217 | "INFO:tensorflow:Assets written to: ../model_zoo/variouse/issue_00003/fiop_dumb_model_pb\\assets\n" 218 | ] 219 | } 220 | ], 221 | "source": [ 222 | "# Cheating approach :: Ussing TensorFlow Converter\n", 223 | "# https://github.com/Tencent/ncnn/tree/master/tools/mlir\n", 224 | "# https://github.com/Tencent/ncnn/tree/master/tools/tensorflow\n", 225 | "fixed_model.save(\"../model_zoo/variouse/issue_00003/fiop_dumb_model_pb\")" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": null, 231 | "outputs": [], 232 | "source": [], 233 | "metadata": { 234 | "collapsed": false, 235 | "pycharm": { 236 | "name": "#%%\n" 237 | } 238 | } 239 | } 240 | ], 241 | "metadata": { 242 | "kernelspec": { 243 | "display_name": "Python 3", 244 | "language": "python", 245 | "name": "python3" 246 | }, 247 | "language_info": { 248 | "codemirror_mode": { 249 | "name": "ipython", 250 | "version": 3 251 | }, 252 | "file_extension": ".py", 253 | "mimetype": "text/x-python", 254 | "name": "python", 255 | "nbconvert_exporter": "python", 256 | "pygments_lexer": "ipython3", 257 | "version": "3.7.6" 258 | } 259 | }, 260 | "nbformat": 4, 261 | "nbformat_minor": 4 262 | } -------------------------------------------------------------------------------- /_step_by_step/yolov4-tiny-keras-custom.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import sys\n", 10 | "sys.path.append('../../../research_git/yolov4-tiny-keras/')\n", 11 | "import os" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "from nets.yolo4_tiny import yolo_body, yolo_head\n", 21 | "from nets.loss import yolo_loss" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "from keras.utils.io_utils import h5dict\n", 31 | "from keras.models import load_model, Model\n", 32 | "from keras.layers import InputLayer, Input\n", 33 | "import json" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "filepath = '../model_zoo/detection/yolo4/model_000/tf_a85.h5'\n", 43 | "f = h5dict(filepath, 'r')" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [ 52 | "model_config = f['model_config']\n", 53 | "model_config = json.loads(model_config.decode('utf-8'))" 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": null, 59 | "metadata": {}, 60 | "outputs": [], 61 | "source": [ 62 | "#Cut loss layer\n", 63 | "model_config['config']['layers'][0]['config']['batch_input_shape'] = [None, 416, 416, 3]\n", 64 | "model_config['config']['input_layers'] = model_config['config']['input_layers'][:1]\n", 65 | "model_config['config']['output_layers'] = [_i[:3] for _i in model_config['config']['layers'][-1]['inbound_nodes'][0]][:2]\n", 66 | "model_config['config']['layers'].pop(-1)\n", 67 | "print(\"\")" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "fixed_model = Model.from_config(model_config['config'])" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "fixed_model.load_weights(filepath)" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": null, 91 | "metadata": {}, 92 | "outputs": [], 93 | "source": [ 94 | "out_file_path = '../model_zoo/detection/yolo4/model_000/tf_a85_fixed.h5'\n", 95 | "fixed_model.compile(loss='mae', optimizer='adam')\n", 96 | "fixed_model.save(out_file_path)" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": null, 102 | "metadata": {}, 103 | "outputs": [], 104 | "source": [] 105 | } 106 | ], 107 | "metadata": { 108 | "kernelspec": { 109 | "display_name": "Python 3", 110 | "language": "python", 111 | "name": "python3" 112 | }, 113 | "language_info": { 114 | "codemirror_mode": { 115 | "name": "ipython", 116 | "version": 3 117 | }, 118 | "file_extension": ".py", 119 | "mimetype": "text/x-python", 120 | "name": "python", 121 | "nbconvert_exporter": "python", 122 | "pygments_lexer": "ipython3", 123 | "version": "3.7.6" 124 | } 125 | }, 126 | "nbformat": 4, 127 | "nbformat_minor": 4 128 | } 129 | -------------------------------------------------------------------------------- /converter/model_adaptation.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | 3 | from extra_layers.Clip import Clip 4 | from extra_layers.CustomObjects import extra_custom_objects 5 | from extra_layers.OutputSplit import OutputSplit 6 | from tensorflow.keras.models import Model 7 | 8 | 9 | def convert_blob(in_item): 10 | if type(in_item) == list: 11 | out_item = in_item 12 | else: 13 | out_item = [in_item] 14 | return out_item 15 | 16 | 17 | def clean_node_name(in_name): 18 | out_name = in_name.replace(':', '_').replace('/', 'l').lower() 19 | return out_name 20 | 21 | def is_multi_output(keras_model_config): 22 | for layer_index, layer_item in enumerate(keras_model_config['layers']): 23 | inbound_nodes = layer_item['inbound_nodes'] 24 | if len(inbound_nodes) == 0: 25 | continue 26 | else: 27 | inbound_node_list = [node_item[0] for node_item in inbound_nodes[0]] 28 | if len(inbound_node_list) > 1: 29 | return True 30 | return False 31 | 32 | 33 | def rename_layer(model_config, src_name, dst_name): 34 | for layer_item in model_config['input_layers']: 35 | input_condition = layer_item[0] == src_name 36 | if input_condition: 37 | # print('Rename Input:: ', layer_item[0]) 38 | layer_item[0] = dst_name 39 | 40 | for layer_item in model_config['output_layers']: 41 | output_condition = layer_item[0] == src_name 42 | if output_condition: 43 | # print('Rename Output:: ', layer_item[0]) 44 | layer_item[0] = dst_name 45 | # print(rename_condition, layer_list[0]) 46 | 47 | for layer_item in model_config['layers']: 48 | name_condition = layer_item['name'] == src_name 49 | if name_condition: 50 | # print('Rename Layers:: name', layer_item['name']) 51 | layer_item['name'] = dst_name 52 | 53 | name_condition = layer_item['config']['name'] == src_name 54 | if name_condition: 55 | # print('Rename Layers Config:: config', layer_item['config']['name']) 56 | layer_item['config']['name'] = dst_name 57 | # print(' Rename Layers Config:: config', layer_item['config']['name']) 58 | 59 | if len(layer_item['inbound_nodes']) > 0: 60 | for item in layer_item['inbound_nodes'][0]: 61 | inbound_condition = item[0] == src_name 62 | if inbound_condition: 63 | # print('Rename Layers Inbound::', item[0]) 64 | item[0] = dst_name 65 | return model_config 66 | 67 | 68 | def get_outbound_nodes(keras_model_config): 69 | outbound_nodes_dict = {} 70 | layer_index_dict = {} 71 | 72 | for layer_index, layer_item in enumerate(keras_model_config['layers']): 73 | out_node_name = layer_item['name'] 74 | layer_index_dict[out_node_name] = layer_index 75 | 76 | inbound_nodes = layer_item['inbound_nodes'] 77 | if len(inbound_nodes) == 0: 78 | continue 79 | else: 80 | inbound_node_list = [node_item[0] for node_item in inbound_nodes[0]] 81 | 82 | for in_node_name in inbound_node_list: 83 | # print(in_node_name, out_node_name) 84 | if in_node_name in outbound_nodes_dict: 85 | outbound_nodes_dict[in_node_name].append(out_node_name) 86 | else: 87 | outbound_nodes_dict[in_node_name] = [out_node_name] 88 | 89 | for in_node_name in keras_model_config['output_layers']: 90 | _in_node_name = in_node_name[0] 91 | if _in_node_name in outbound_nodes_dict: 92 | continue 93 | outbound_nodes_dict[_in_node_name] = [] 94 | 95 | return outbound_nodes_dict, layer_index_dict 96 | 97 | 98 | def split_output_nodes(keras_model_config): 99 | protected_class_names = ['OutputSplit'] 100 | 101 | nccn2keras_layer_list = [] 102 | count_index = 0 103 | model_config_layers = deepcopy(keras_model_config['layers']) 104 | model_config = deepcopy(keras_model_config) 105 | 106 | outbound_nodes_dict, layer_index_dict = get_outbound_nodes(keras_model_config) 107 | 108 | for layer_item in model_config_layers: 109 | out_node_name = layer_item['name'] 110 | nccn2keras_layer_list.append(layer_item) 111 | split_condition = len(outbound_nodes_dict[out_node_name]) > 1 and ( 112 | not (layer_item['class_name'] in protected_class_names)) 113 | # print(split_condition) 114 | if split_condition: 115 | count_index += 1 116 | # print("-----", out_node_name) 117 | # print(layer_item) 118 | # print(outbound_nodes_dict[out_node_name]) 119 | split_node_name = f'ncnn_split_{str(count_index)}' 120 | split_layer_config = {'class_name': 'OutputSplit', 'config': {'name': split_node_name, 121 | 'trainable': True, 122 | 'dtype': 'float32', 123 | 'count': len( 124 | outbound_nodes_dict[out_node_name])}, 125 | 'name': split_node_name, 126 | 'inbound_nodes': [[[out_node_name, 0, 0, {}]]]} 127 | nccn2keras_layer_list.append(split_layer_config) 128 | use_index = 0 129 | for item in outbound_nodes_dict[out_node_name]: 130 | p_item = model_config_layers[layer_index_dict[item]]['inbound_nodes'] 131 | # print('----') 132 | for t_item in p_item[0]: 133 | if t_item[0] == out_node_name: 134 | # print(' ', t_item) 135 | t_item[0] = split_node_name 136 | t_item[2] = use_index 137 | use_index += 1 138 | # print(' ', t_item) 139 | else: 140 | continue 141 | 142 | model_config['layers'] = nccn2keras_layer_list 143 | return model_config 144 | 145 | 146 | def adapt_keras_model(keras_model, model_name): 147 | keras_model_config = keras_model.get_config() 148 | keras_model_config['name'] = model_name 149 | # assert len(keras_model_config['input_layers']) == len( 150 | # keras_model_config['output_layers']) == 1, 'Multi IN/OUT is not supported' 151 | if len(keras_model_config['output_layers']) != 1: 152 | print("WARNING :: Multi Output Detected!!!") 153 | if is_multi_output(keras_model_config): 154 | adapted_model_config = split_output_nodes(keras_model_config) 155 | # adapted_model_config = rename_layer(adapted_model_config, keras_model_config['input_layers'][0][0], 'data') 156 | # adapted_model_config = rename_layer(adapted_model_config, keras_model_config['output_layers'][0][0], 'output') 157 | adapted_keras_model = Model.from_config(adapted_model_config, custom_objects=extra_custom_objects) 158 | else: 159 | adapted_keras_model = Model.from_config(keras_model_config, custom_objects=extra_custom_objects) 160 | adapted_keras_model.set_weights(keras_model.get_weights()) 161 | return adapted_keras_model 162 | 163 | 164 | -------------------------------------------------------------------------------- /extra_layers/BinaryOp.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.layers import Subtract 2 | from tensorflow.python.util.tf_export import tf_export 3 | 4 | 5 | @tf_export('keras.layers.Div') 6 | class Div(Subtract): 7 | def _merge_function(self, inputs): 8 | if len(inputs) != 2: 9 | raise ValueError('A `Div` layer should be called ' 10 | 'on exactly 2 inputs') 11 | return inputs[0]/(inputs[1]) -------------------------------------------------------------------------------- /extra_layers/Clip.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.layers import Layer 2 | from tensorflow.keras import backend as K 3 | import tensorflow as tf 4 | from tensorflow.python.keras.utils import tf_utils 5 | 6 | class Clip(Layer): 7 | def __init__(self, min_value=-1, max_value=+1, **kwargs): 8 | super(Clip, self).__init__(**kwargs) 9 | self.min_value = K.cast_to_floatx(min_value) 10 | self.max_value = K.cast_to_floatx(max_value) 11 | 12 | @tf.function(experimental_relax_shapes=True) 13 | def call(self, inputs, **kwargs): 14 | return K.clip(inputs, min_value=self.min_value, max_value=self.max_value) 15 | 16 | def get_config(self): 17 | config = { 18 | 'min_value': self.min_value, 19 | 'max_value': self.max_value, 20 | } 21 | base_config = super(Clip, self).get_config() 22 | return dict(list(base_config.items()) + list(config.items())) 23 | 24 | @tf_utils.shape_type_conversion 25 | def compute_output_shape(self, input_shape): 26 | return input_shape 27 | -------------------------------------------------------------------------------- /extra_layers/CustomObjects.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from extra_layers.Clip import Clip 3 | from extra_layers.OutputSplit import OutputSplit 4 | from extra_layers.Padding import ReflectPadding2D 5 | from extra_layers.BinaryOp import Div 6 | from extra_layers.UnaryOp import Sqrt 7 | from extra_layers.UnaryOp import Swish 8 | from extra_layers.Resize import Interp 9 | from extra_layers.UnaryOp import DropConnect 10 | 11 | extra_custom_objects = {'relu6': tf.nn.relu6, 12 | 'OutputSplit': OutputSplit, 'Clip': Clip, 'ReflectPadding2D': ReflectPadding2D, 13 | 'Div': Div, 'Sqrt': Sqrt, 'Interp': Interp, 'Swish': Swish, 'DropConnect': DropConnect} 14 | 15 | 16 | -------------------------------------------------------------------------------- /extra_layers/OutputSplit.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.layers import Layer 2 | from tensorflow.python.keras.utils import tf_utils 3 | import tensorflow as tf 4 | 5 | class OutputSplit(Layer): 6 | def __init__(self, count, **kwargs): 7 | self.count = count 8 | super(OutputSplit, self).__init__(**kwargs) 9 | 10 | def build(self, inputShape): 11 | super(OutputSplit, self).build(inputShape) 12 | 13 | @tf.function(experimental_relax_shapes=True) 14 | def call(self, x, **kwargs): 15 | return [tf.identity(x)] * self.count 16 | 17 | @tf_utils.shape_type_conversion 18 | def compute_output_shape(self, input_shape): 19 | return [input_shape] * self.count 20 | 21 | def get_config(self): 22 | base_config = super(OutputSplit, self).get_config() 23 | base_config['count'] = self.count 24 | return base_config 25 | 26 | -------------------------------------------------------------------------------- /extra_layers/Padding.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from tensorflow.python.util.tf_export import tf_export 3 | from tensorflow.keras.layers import ZeroPadding2D 4 | 5 | 6 | @tf_export('keras.layers.ZeroPadding2D') 7 | class ReflectPadding2D(ZeroPadding2D): 8 | def __init__(self, padding=(1, 1), data_format=None, **kwargs): 9 | super(ReflectPadding2D, self).__init__(**kwargs) 10 | 11 | def call(self, inputs): 12 | padding = self.padding 13 | data_format = self.data_format 14 | assert len(padding) == 2 15 | assert len(padding[0]) == 2 16 | assert len(padding[1]) == 2 17 | if data_format is None: 18 | data_format = 'channels_last' 19 | if data_format not in {'channels_first', 'channels_last'}: 20 | raise ValueError('Unknown data_format: ' + str(data_format)) 21 | 22 | if data_format == 'channels_first': 23 | pattern = [[0, 0], [0, 0], list(padding[0]), list(padding[1])] 24 | else: 25 | pattern = [[0, 0], list(padding[0]), list(padding[1]), [0, 0]] 26 | return tf.pad(inputs, pattern, mode='REFLECT') 27 | 28 | -------------------------------------------------------------------------------- /extra_layers/Resize.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from tensorflow.keras import layers 3 | class Interp(layers.Layer): 4 | 5 | def __init__(self, new_size, **kwargs): 6 | self.new_size = new_size 7 | super(Interp, self).__init__(**kwargs) 8 | 9 | def build(self, input_shape): 10 | super(Interp, self).build(input_shape) 11 | 12 | def call(self, inputs, **kwargs): 13 | new_height, new_width = self.new_size 14 | try: 15 | resized = tf.image.resize(inputs, [new_height, new_width]) 16 | except AttributeError: 17 | resized = tf.image.resize_images(inputs, [new_height, new_width], 18 | align_corners=True) 19 | return resized 20 | 21 | def compute_output_shape(self, input_shape): 22 | return tuple([None, 23 | self.new_size[0], 24 | self.new_size[1], 25 | input_shape[3]]) 26 | 27 | def get_config(self): 28 | config = super(Interp, self).get_config() 29 | config['new_size'] = self.new_size 30 | return config -------------------------------------------------------------------------------- /extra_layers/UnaryOp.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from tensorflow.keras.layers import ReLU 3 | from tensorflow.keras import backend as K 4 | from tensorflow.keras.layers import Layer 5 | from tensorflow.python.util.tf_export import tf_export 6 | 7 | 8 | @tf_export('keras.layers.RSqrt') 9 | class RSqrt(ReLU): 10 | def call(self, inputs): 11 | return tf.rsqrt(inputs) 12 | 13 | 14 | @tf_export('keras.layers.Sqrt') 15 | class Sqrt(ReLU): 16 | def call(self, inputs): 17 | return K.sqrt(inputs) 18 | 19 | @tf_export('keras.layers.Swish') 20 | class Swish(ReLU): 21 | def call(self, inputs): 22 | return tf.nn.swish(inputs) 23 | 24 | class DropConnect(Layer): 25 | def __init__(self, drop_connect_rate=0.0, **kwargs): 26 | super().__init__(**kwargs) 27 | self.drop_connect_rate = drop_connect_rate 28 | 29 | def call(self, inputs, training=None): 30 | def drop_connect(): 31 | keep_prob = 1.0 - self.drop_connect_rate 32 | 33 | # Compute drop_connect tensor 34 | batch_size = tf.shape(inputs)[0] 35 | random_tensor = keep_prob 36 | random_tensor += tf.random_uniform( 37 | [batch_size, 1, 1, 1], dtype=inputs.dtype 38 | ) 39 | binary_tensor = tf.floor(random_tensor) 40 | output = tf.div(inputs, keep_prob) * binary_tensor 41 | return output 42 | 43 | return K.in_train_phase(drop_connect, inputs, training=training) 44 | 45 | def get_config(self): 46 | config = super().get_config() 47 | config["drop_connect_rate"] = self.drop_connect_rate 48 | return config -------------------------------------------------------------------------------- /keras2ncnn.py: -------------------------------------------------------------------------------- 1 | import os 2 | import tensorflow as tf 3 | from tensorflow.keras.models import load_model, Model 4 | from unit_test.helper import save_config, get_test_item_mean_std 5 | from converter.converter import conver_model 6 | from converter.model_adaptation import adapt_keras_model 7 | from optimization.optimize_graph import apply_transformations, check_transform 8 | 9 | import argparse 10 | os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" 11 | os.environ["CUDA_VISIBLE_DEVICES"] = "-1" 12 | 13 | if __name__ == '__main__': 14 | """ 15 | Quick command lines 16 | --model_path=model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5 17 | --model_path=model_zoo/segmentation/hair/model_001/checkpoint.hdf5 18 | --model_path=model_zoo/segmentation/hair/model_002/gaelkt_HairNets.hdf5 19 | """ 20 | parser = argparse.ArgumentParser(description='Process some integers.') 21 | parser.add_argument('--model_path', help='The path to the Keras model file (usually *.hdf5) ') 22 | parser.add_argument('--export_path', default=None, help='The path for the script result output') 23 | parser.add_argument('--export_model_name', default=None, help='The model file name') 24 | parser.add_argument('--save_optimized', default=True, help='The model file name') 25 | parser.add_argument('--test_config', default=None, help='The model file name') 26 | args = parser.parse_args() 27 | inference_mode = 'NCHW' 28 | 29 | model_path = args.model_path 30 | export_path = args.export_path 31 | export_model_name = args.export_model_name 32 | save_optimized = args.save_optimized 33 | if export_path is None: 34 | export_path = os.path.dirname(model_path) 35 | if export_model_name is None: 36 | export_model_name = '.'.join(os.path.basename(model_path).split('.')[:-1]) 37 | 38 | model_ext = model_path.split('.')[-1] 39 | if model_ext.lower() == 'json': 40 | from tensorflow.keras.models import model_from_json 41 | keras_model_in = model_from_json(open(model_path).read()) 42 | keras_model_in.load_weights(model_path.replace(model_ext, 'hdf5')) 43 | else: 44 | keras_model_in = load_model(model_path, custom_objects={'tf': tf, 'Functional': Model}) 45 | keras_model = apply_transformations(keras_model_in) 46 | if save_optimized: 47 | keras_model.save('.'.join(model_path.split('.')[:-1]) + '_optimized.hdf5') 48 | 49 | adapted_keras_model = adapt_keras_model(keras_model, export_model_name) 50 | target_shape = keras_model.input_shape[1:3] 51 | x_in_item = get_test_item_mean_std(target_shape) 52 | 53 | print('\n' + ('='*20) + 'Overall Transform' + ('='*20)) 54 | check_transform(keras_model_in, adapted_keras_model) 55 | string_list, weight_list, _ = conver_model(adapted_keras_model) 56 | save_config(string_list, weight_list, adapted_keras_model.name, export_path) -------------------------------------------------------------------------------- /model_zoo/memory_estimator/get_mem.py: -------------------------------------------------------------------------------- 1 | import ncnn 2 | import numpy as np 3 | 4 | out_config_path = 'faceDetector_from_va_320_240_simple_opt.param' 5 | out_weights_path = 'faceDetector_from_va_320_240_simple_opt.bin' 6 | 7 | net = ncnn.Net() 8 | net.load_param(out_config_path) 9 | net.load_model(out_weights_path) 10 | 11 | num_threads = 4 12 | error_th = 1.e-5 13 | ex = net.create_extractor() 14 | ex.set_num_threads(num_threads) 15 | 16 | y_size, x_size, c_size = 320, 320, 3 17 | in_layer_list = [layer.name for layer in net.layers if layer.type == 'Input'] 18 | for item in in_layer_list: 19 | frame = np.random.uniform(0, 255, size=(y_size, x_size, c_size)).astype(np.uint8) 20 | mat_in = ncnn.Mat.from_pixels(frame, ncnn.Mat.PixelType.PIXEL_BGR, x_size, y_size) 21 | ex.input(item, mat_in) 22 | 23 | s = 0 24 | mat_out = ncnn.Mat() 25 | for blob in net.blobs: 26 | ex.extract(blob.name, mat_out) 27 | np_out = np.array(mat_out) 28 | s += np.prod(np_out.shape) 29 | print(blob.name, np_out.shape, np.prod(np_out.shape)) 30 | 31 | print('>>', s, s*4/(2**20)) 32 | 33 | # frame = np.random.uniform(0, 255, size=fix_none_in_shape(target_shape)[1:]).astype(np.uint8) 34 | # mat_in = ncnn.Mat.from_pixels_resize(frame, ncnn.Mat.PixelType.PIXEL_BGR, src_x, src_y, target_x, target_y) 35 | # 36 | # ... 37 | # 38 | # for blob in net.blobs: 39 | # print(blob.name) -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.bin -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5 -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.param: -------------------------------------------------------------------------------- 1 | 7767517 2 | 193 220 3 | Input data 0 1 data 0=256 1=256 2=3 4 | Convolution conv2d_1 1 1 data conv2d_1lrelu_0 0=16 1=3 2=1 3=1 4=1 5=1 6=432 8=0 9=1 11=3 12=1 13=1 14=1 15=1 16=1 5 | Clip re_lu_1 1 1 conv2d_1lrelu_0 re_lu_1lpartitionedcall_0 0=0.0 1=6.0 6 | Split ncnn_split_1 1 2 re_lu_1lpartitionedcall_0 ncnn_split_1lpartitionedcall_0 ncnn_split_1lpartitionedcall_1 7 | ConvolutionDepthWise sep_dwconv2d_split_1 1 1 ncnn_split_1lpartitionedcall_0 sep_dwconv2d_split_1lbiasadd_0 0=16 1=3 2=1 3=1 4=1 5=1 6=144 7=16 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 8 | Convolution sep_conv2d_split_1 1 1 sep_dwconv2d_split_1lbiasadd_0 sep_conv2d_split_1lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=256 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 9 | Clip re_lu_2 1 1 sep_conv2d_split_1lrelu_0 re_lu_2lpartitionedcall_0 0=0.0 1=6.0 10 | ConvolutionDepthWise batch_normalization_3 1 1 re_lu_2lpartitionedcall_0 batch_normalization_3lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=16 7=16 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 11 | Clip re_lu_3 1 1 batch_normalization_3lrelu_0 re_lu_3lpartitionedcall_0 0=0.0 1=6.0 12 | Eltwise add_1 2 1 re_lu_3lpartitionedcall_0 ncnn_split_1lpartitionedcall_1 add_1ladd_0 0=1 13 | Split ncnn_split_2 1 2 add_1ladd_0 ncnn_split_2lpartitionedcall_0 ncnn_split_2lpartitionedcall_1 14 | Convolution conv2d_2 1 1 ncnn_split_2lpartitionedcall_0 conv2d_2lrelu_0 0=32 1=3 2=1 3=2 4=0 5=1 6=4608 8=0 9=1 11=3 12=1 13=2 14=0 15=1 16=1 15 | Clip re_lu_4 1 1 conv2d_2lrelu_0 re_lu_4lpartitionedcall_0 0=0.0 1=6.0 16 | Split ncnn_split_3 1 2 re_lu_4lpartitionedcall_0 ncnn_split_3lpartitionedcall_0 ncnn_split_3lpartitionedcall_1 17 | ConvolutionDepthWise sep_dwconv2d_split_2 1 1 ncnn_split_3lpartitionedcall_0 sep_dwconv2d_split_2lbiasadd_0 0=32 1=3 2=1 3=1 4=1 5=1 6=288 7=32 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 18 | Convolution sep_conv2d_split_2 1 1 sep_dwconv2d_split_2lbiasadd_0 sep_conv2d_split_2lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=1024 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 19 | Clip re_lu_5 1 1 sep_conv2d_split_2lrelu_0 re_lu_5lpartitionedcall_0 0=0.0 1=6.0 20 | ConvolutionDepthWise batch_normalization_6 1 1 re_lu_5lpartitionedcall_0 batch_normalization_6lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=32 7=32 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 21 | Clip re_lu_6 1 1 batch_normalization_6lrelu_0 re_lu_6lpartitionedcall_0 0=0.0 1=6.0 22 | Eltwise add_2 2 1 re_lu_6lpartitionedcall_0 ncnn_split_3lpartitionedcall_1 add_2ladd_0 0=1 23 | Split ncnn_split_4 1 2 add_2ladd_0 ncnn_split_4lpartitionedcall_0 ncnn_split_4lpartitionedcall_1 24 | Convolution conv2d_3 1 1 ncnn_split_4lpartitionedcall_0 conv2d_3lrelu_0 0=64 1=3 2=1 3=2 4=0 5=1 6=18432 8=0 9=1 11=3 12=1 13=2 14=0 15=1 16=1 25 | Clip re_lu_7 1 1 conv2d_3lrelu_0 re_lu_7lpartitionedcall_0 0=0.0 1=6.0 26 | Split ncnn_split_5 1 2 re_lu_7lpartitionedcall_0 ncnn_split_5lpartitionedcall_0 ncnn_split_5lpartitionedcall_1 27 | ConvolutionDepthWise sep_dwconv2d_split_3 1 1 ncnn_split_5lpartitionedcall_0 sep_dwconv2d_split_3lbiasadd_0 0=64 1=3 2=1 3=1 4=1 5=1 6=576 7=64 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 28 | Convolution sep_conv2d_split_3 1 1 sep_dwconv2d_split_3lbiasadd_0 sep_conv2d_split_3lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=4096 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 29 | Clip re_lu_8 1 1 sep_conv2d_split_3lrelu_0 re_lu_8lpartitionedcall_0 0=0.0 1=6.0 30 | ConvolutionDepthWise batch_normalization_9 1 1 re_lu_8lpartitionedcall_0 batch_normalization_9lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=64 7=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 31 | Clip re_lu_9 1 1 batch_normalization_9lrelu_0 re_lu_9lpartitionedcall_0 0=0.0 1=6.0 32 | Eltwise add_3 2 1 re_lu_9lpartitionedcall_0 ncnn_split_5lpartitionedcall_1 add_3ladd_0 0=1 33 | Split ncnn_split_6 1 2 add_3ladd_0 ncnn_split_6lpartitionedcall_0 ncnn_split_6lpartitionedcall_1 34 | Convolution conv2d_4 1 1 ncnn_split_6lpartitionedcall_0 conv2d_4lrelu_0 0=128 1=3 2=1 3=2 4=0 5=1 6=73728 8=0 9=1 11=3 12=1 13=2 14=0 15=1 16=1 35 | Clip re_lu_10 1 1 conv2d_4lrelu_0 re_lu_10lpartitionedcall_0 0=0.0 1=6.0 36 | Split ncnn_split_7 1 2 re_lu_10lpartitionedcall_0 ncnn_split_7lpartitionedcall_0 ncnn_split_7lpartitionedcall_1 37 | ConvolutionDepthWise sep_dwconv2d_split_4 1 1 ncnn_split_7lpartitionedcall_0 sep_dwconv2d_split_4lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 38 | Convolution sep_conv2d_split_4 1 1 sep_dwconv2d_split_4lbiasadd_0 sep_conv2d_split_4lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 39 | Clip re_lu_11 1 1 sep_conv2d_split_4lrelu_0 re_lu_11lpartitionedcall_0 0=0.0 1=6.0 40 | ConvolutionDepthWise batch_normalization_12 1 1 re_lu_11lpartitionedcall_0 batch_normalization_12lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 41 | Clip re_lu_12 1 1 batch_normalization_12lrelu_0 re_lu_12lpartitionedcall_0 0=0.0 1=6.0 42 | Eltwise add_4 2 1 re_lu_12lpartitionedcall_0 ncnn_split_7lpartitionedcall_1 add_4ladd_0 0=1 43 | Split ncnn_split_8 1 2 add_4ladd_0 ncnn_split_8lpartitionedcall_0 ncnn_split_8lpartitionedcall_1 44 | ConvolutionDepthWise sep_dwconv2d_split_5 1 1 ncnn_split_8lpartitionedcall_0 sep_dwconv2d_split_5lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 45 | Convolution sep_conv2d_split_5 1 1 sep_dwconv2d_split_5lbiasadd_0 sep_conv2d_split_5lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 46 | Clip re_lu_13 1 1 sep_conv2d_split_5lrelu_0 re_lu_13lpartitionedcall_0 0=0.0 1=6.0 47 | ConvolutionDepthWise batch_normalization_14 1 1 re_lu_13lpartitionedcall_0 batch_normalization_14lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 48 | Clip re_lu_14 1 1 batch_normalization_14lrelu_0 re_lu_14lpartitionedcall_0 0=0.0 1=6.0 49 | Eltwise add_5 2 1 re_lu_14lpartitionedcall_0 ncnn_split_8lpartitionedcall_1 add_5ladd_0 0=1 50 | Split ncnn_split_9 1 2 add_5ladd_0 ncnn_split_9lpartitionedcall_0 ncnn_split_9lpartitionedcall_1 51 | Convolution conv2d_5 1 1 ncnn_split_9lpartitionedcall_0 conv2d_5lrelu_0 0=128 1=3 2=1 3=2 4=0 5=1 6=147456 8=0 9=1 11=3 12=1 13=2 14=0 15=1 16=1 52 | Clip re_lu_15 1 1 conv2d_5lrelu_0 re_lu_15lpartitionedcall_0 0=0.0 1=6.0 53 | Split ncnn_split_10 1 2 re_lu_15lpartitionedcall_0 ncnn_split_10lpartitionedcall_0 ncnn_split_10lpartitionedcall_1 54 | ConvolutionDepthWise sep_dwconv2d_split_6 1 1 ncnn_split_10lpartitionedcall_0 sep_dwconv2d_split_6lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 55 | Convolution sep_conv2d_split_6 1 1 sep_dwconv2d_split_6lbiasadd_0 sep_conv2d_split_6lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 56 | Clip re_lu_16 1 1 sep_conv2d_split_6lrelu_0 re_lu_16lpartitionedcall_0 0=0.0 1=6.0 57 | ConvolutionDepthWise batch_normalization_17 1 1 re_lu_16lpartitionedcall_0 batch_normalization_17lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 58 | Clip re_lu_17 1 1 batch_normalization_17lrelu_0 re_lu_17lpartitionedcall_0 0=0.0 1=6.0 59 | Eltwise add_6 2 1 re_lu_17lpartitionedcall_0 ncnn_split_10lpartitionedcall_1 add_6ladd_0 0=1 60 | Split ncnn_split_11 1 2 add_6ladd_0 ncnn_split_11lpartitionedcall_0 ncnn_split_11lpartitionedcall_1 61 | ConvolutionDepthWise sep_dwconv2d_split_7 1 1 ncnn_split_11lpartitionedcall_0 sep_dwconv2d_split_7lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 62 | Convolution sep_conv2d_split_7 1 1 sep_dwconv2d_split_7lbiasadd_0 sep_conv2d_split_7lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 63 | Clip re_lu_18 1 1 sep_conv2d_split_7lrelu_0 re_lu_18lpartitionedcall_0 0=0.0 1=6.0 64 | ConvolutionDepthWise batch_normalization_19 1 1 re_lu_18lpartitionedcall_0 batch_normalization_19lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 65 | Clip re_lu_19 1 1 batch_normalization_19lrelu_0 re_lu_19lpartitionedcall_0 0=0.0 1=6.0 66 | Eltwise add_7 2 1 re_lu_19lpartitionedcall_0 ncnn_split_11lpartitionedcall_1 add_7ladd_0 0=1 67 | Split ncnn_split_12 1 2 add_7ladd_0 ncnn_split_12lpartitionedcall_0 ncnn_split_12lpartitionedcall_1 68 | ConvolutionDepthWise sep_dwconv2d_split_8 1 1 ncnn_split_12lpartitionedcall_0 sep_dwconv2d_split_8lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 69 | Convolution sep_conv2d_split_8 1 1 sep_dwconv2d_split_8lbiasadd_0 sep_conv2d_split_8lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 70 | Clip re_lu_20 1 1 sep_conv2d_split_8lrelu_0 re_lu_20lpartitionedcall_0 0=0.0 1=6.0 71 | ConvolutionDepthWise batch_normalization_21 1 1 re_lu_20lpartitionedcall_0 batch_normalization_21lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 72 | Clip re_lu_21 1 1 batch_normalization_21lrelu_0 re_lu_21lpartitionedcall_0 0=0.0 1=6.0 73 | Eltwise add_8 2 1 re_lu_21lpartitionedcall_0 ncnn_split_12lpartitionedcall_1 add_8ladd_0 0=1 74 | Split ncnn_split_13 1 2 add_8ladd_0 ncnn_split_13lpartitionedcall_0 ncnn_split_13lpartitionedcall_1 75 | ConvolutionDepthWise sep_dwconv2d_split_9 1 1 ncnn_split_13lpartitionedcall_0 sep_dwconv2d_split_9lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 76 | Convolution sep_conv2d_split_9 1 1 sep_dwconv2d_split_9lbiasadd_0 sep_conv2d_split_9lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 77 | Clip re_lu_22 1 1 sep_conv2d_split_9lrelu_0 re_lu_22lpartitionedcall_0 0=0.0 1=6.0 78 | ConvolutionDepthWise batch_normalization_23 1 1 re_lu_22lpartitionedcall_0 batch_normalization_23lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 79 | Clip re_lu_23 1 1 batch_normalization_23lrelu_0 re_lu_23lpartitionedcall_0 0=0.0 1=6.0 80 | Eltwise add_9 2 1 re_lu_23lpartitionedcall_0 ncnn_split_13lpartitionedcall_1 add_9ladd_0 0=1 81 | Split ncnn_split_14 1 2 add_9ladd_0 ncnn_split_14lpartitionedcall_0 ncnn_split_14lpartitionedcall_1 82 | ConvolutionDepthWise sep_dwconv2d_split_10 1 1 ncnn_split_14lpartitionedcall_0 sep_dwconv2d_split_10lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 83 | Convolution sep_conv2d_split_10 1 1 sep_dwconv2d_split_10lbiasadd_0 sep_conv2d_split_10lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 84 | Clip re_lu_24 1 1 sep_conv2d_split_10lrelu_0 re_lu_24lpartitionedcall_0 0=0.0 1=6.0 85 | ConvolutionDepthWise batch_normalization_25 1 1 re_lu_24lpartitionedcall_0 batch_normalization_25lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 86 | Clip re_lu_25 1 1 batch_normalization_25lrelu_0 re_lu_25lpartitionedcall_0 0=0.0 1=6.0 87 | Eltwise add_10 2 1 re_lu_25lpartitionedcall_0 ncnn_split_14lpartitionedcall_1 add_10ladd_0 0=1 88 | Split ncnn_split_15 1 2 add_10ladd_0 ncnn_split_15lpartitionedcall_0 ncnn_split_15lpartitionedcall_1 89 | ConvolutionDepthWise sep_dwconv2d_split_11 1 1 ncnn_split_15lpartitionedcall_0 sep_dwconv2d_split_11lbiasadd_0 0=128 1=3 2=1 3=1 4=1 5=1 6=1152 7=128 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 90 | Convolution sep_conv2d_split_11 1 1 sep_dwconv2d_split_11lbiasadd_0 sep_conv2d_split_11lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=16384 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 91 | Clip re_lu_26 1 1 sep_conv2d_split_11lrelu_0 re_lu_26lpartitionedcall_0 0=0.0 1=6.0 92 | ConvolutionDepthWise batch_normalization_27 1 1 re_lu_26lpartitionedcall_0 batch_normalization_27lrelu_0 0=128 1=1 2=1 3=1 4=0 5=1 6=128 7=128 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 93 | Clip re_lu_27 1 1 batch_normalization_27lrelu_0 re_lu_27lpartitionedcall_0 0=0.0 1=6.0 94 | Eltwise add_11 2 1 re_lu_27lpartitionedcall_0 ncnn_split_15lpartitionedcall_1 add_11ladd_0 0=1 95 | Interp up_sampling2d_1 1 1 add_11ladd_0 up_sampling2d_1lresizelresizebilinear_0 0=2 1=2.0 2=2.0 96 | Eltwise add_12 2 1 up_sampling2d_1lresizelresizebilinear_0 ncnn_split_9lpartitionedcall_1 add_12ladd_0 0=1 97 | Convolution conv2d_6 1 1 add_12ladd_0 conv2d_6lrelu_0 0=64 1=3 2=1 3=1 4=1 5=1 6=73728 8=0 9=1 11=3 12=1 13=1 14=1 15=1 16=1 98 | Clip re_lu_28 1 1 conv2d_6lrelu_0 re_lu_28lpartitionedcall_0 0=0.0 1=6.0 99 | Split ncnn_split_16 1 2 re_lu_28lpartitionedcall_0 ncnn_split_16lpartitionedcall_0 ncnn_split_16lpartitionedcall_1 100 | ConvolutionDepthWise sep_dwconv2d_split_12 1 1 ncnn_split_16lpartitionedcall_0 sep_dwconv2d_split_12lbiasadd_0 0=64 1=3 2=1 3=1 4=1 5=1 6=576 7=64 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 101 | Convolution sep_conv2d_split_12 1 1 sep_dwconv2d_split_12lbiasadd_0 sep_conv2d_split_12lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=4096 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 102 | Clip re_lu_29 1 1 sep_conv2d_split_12lrelu_0 re_lu_29lpartitionedcall_0 0=0.0 1=6.0 103 | ConvolutionDepthWise batch_normalization_30 1 1 re_lu_29lpartitionedcall_0 batch_normalization_30lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=64 7=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 104 | Clip re_lu_30 1 1 batch_normalization_30lrelu_0 re_lu_30lpartitionedcall_0 0=0.0 1=6.0 105 | Eltwise add_13 2 1 re_lu_30lpartitionedcall_0 ncnn_split_16lpartitionedcall_1 add_13ladd_0 0=1 106 | Split ncnn_split_17 1 2 add_13ladd_0 ncnn_split_17lpartitionedcall_0 ncnn_split_17lpartitionedcall_1 107 | ConvolutionDepthWise sep_dwconv2d_split_13 1 1 ncnn_split_17lpartitionedcall_0 sep_dwconv2d_split_13lbiasadd_0 0=64 1=3 2=1 3=1 4=1 5=1 6=576 7=64 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 108 | Convolution sep_conv2d_split_13 1 1 sep_dwconv2d_split_13lbiasadd_0 sep_conv2d_split_13lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=4096 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 109 | Clip re_lu_31 1 1 sep_conv2d_split_13lrelu_0 re_lu_31lpartitionedcall_0 0=0.0 1=6.0 110 | ConvolutionDepthWise batch_normalization_32 1 1 re_lu_31lpartitionedcall_0 batch_normalization_32lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=64 7=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 111 | Clip re_lu_32 1 1 batch_normalization_32lrelu_0 re_lu_32lpartitionedcall_0 0=0.0 1=6.0 112 | Eltwise add_14 2 1 re_lu_32lpartitionedcall_0 ncnn_split_17lpartitionedcall_1 add_14ladd_0 0=1 113 | Split ncnn_split_18 1 2 add_14ladd_0 ncnn_split_18lpartitionedcall_0 ncnn_split_18lpartitionedcall_1 114 | ConvolutionDepthWise sep_dwconv2d_split_14 1 1 ncnn_split_18lpartitionedcall_0 sep_dwconv2d_split_14lbiasadd_0 0=64 1=3 2=1 3=1 4=1 5=1 6=576 7=64 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 115 | Convolution sep_conv2d_split_14 1 1 sep_dwconv2d_split_14lbiasadd_0 sep_conv2d_split_14lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=4096 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 116 | Clip re_lu_33 1 1 sep_conv2d_split_14lrelu_0 re_lu_33lpartitionedcall_0 0=0.0 1=6.0 117 | ConvolutionDepthWise batch_normalization_34 1 1 re_lu_33lpartitionedcall_0 batch_normalization_34lrelu_0 0=64 1=1 2=1 3=1 4=0 5=1 6=64 7=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 118 | Clip re_lu_34 1 1 batch_normalization_34lrelu_0 re_lu_34lpartitionedcall_0 0=0.0 1=6.0 119 | Eltwise add_15 2 1 re_lu_34lpartitionedcall_0 ncnn_split_18lpartitionedcall_1 add_15ladd_0 0=1 120 | Interp up_sampling2d_2 1 1 add_15ladd_0 up_sampling2d_2lresizelresizebilinear_0 0=2 1=2.0 2=2.0 121 | Eltwise add_16 2 1 up_sampling2d_2lresizelresizebilinear_0 ncnn_split_6lpartitionedcall_1 add_16ladd_0 0=1 122 | Convolution conv2d_7 1 1 add_16ladd_0 conv2d_7lrelu_0 0=32 1=3 2=1 3=1 4=1 5=1 6=18432 8=0 9=1 11=3 12=1 13=1 14=1 15=1 16=1 123 | Clip re_lu_35 1 1 conv2d_7lrelu_0 re_lu_35lpartitionedcall_0 0=0.0 1=6.0 124 | Split ncnn_split_19 1 2 re_lu_35lpartitionedcall_0 ncnn_split_19lpartitionedcall_0 ncnn_split_19lpartitionedcall_1 125 | ConvolutionDepthWise sep_dwconv2d_split_15 1 1 ncnn_split_19lpartitionedcall_0 sep_dwconv2d_split_15lbiasadd_0 0=32 1=3 2=1 3=1 4=1 5=1 6=288 7=32 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 126 | Convolution sep_conv2d_split_15 1 1 sep_dwconv2d_split_15lbiasadd_0 sep_conv2d_split_15lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=1024 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 127 | Clip re_lu_36 1 1 sep_conv2d_split_15lrelu_0 re_lu_36lpartitionedcall_0 0=0.0 1=6.0 128 | ConvolutionDepthWise batch_normalization_37 1 1 re_lu_36lpartitionedcall_0 batch_normalization_37lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=32 7=32 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 129 | Clip re_lu_37 1 1 batch_normalization_37lrelu_0 re_lu_37lpartitionedcall_0 0=0.0 1=6.0 130 | Eltwise add_17 2 1 re_lu_37lpartitionedcall_0 ncnn_split_19lpartitionedcall_1 add_17ladd_0 0=1 131 | Split ncnn_split_20 1 2 add_17ladd_0 ncnn_split_20lpartitionedcall_0 ncnn_split_20lpartitionedcall_1 132 | ConvolutionDepthWise sep_dwconv2d_split_16 1 1 ncnn_split_20lpartitionedcall_0 sep_dwconv2d_split_16lbiasadd_0 0=32 1=3 2=1 3=1 4=1 5=1 6=288 7=32 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 133 | Convolution sep_conv2d_split_16 1 1 sep_dwconv2d_split_16lbiasadd_0 sep_conv2d_split_16lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=1024 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 134 | Clip re_lu_38 1 1 sep_conv2d_split_16lrelu_0 re_lu_38lpartitionedcall_0 0=0.0 1=6.0 135 | ConvolutionDepthWise batch_normalization_39 1 1 re_lu_38lpartitionedcall_0 batch_normalization_39lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=32 7=32 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 136 | Clip re_lu_39 1 1 batch_normalization_39lrelu_0 re_lu_39lpartitionedcall_0 0=0.0 1=6.0 137 | Eltwise add_18 2 1 re_lu_39lpartitionedcall_0 ncnn_split_20lpartitionedcall_1 add_18ladd_0 0=1 138 | Split ncnn_split_21 1 2 add_18ladd_0 ncnn_split_21lpartitionedcall_0 ncnn_split_21lpartitionedcall_1 139 | ConvolutionDepthWise sep_dwconv2d_split_17 1 1 ncnn_split_21lpartitionedcall_0 sep_dwconv2d_split_17lbiasadd_0 0=32 1=3 2=1 3=1 4=1 5=1 6=288 7=32 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 140 | Convolution sep_conv2d_split_17 1 1 sep_dwconv2d_split_17lbiasadd_0 sep_conv2d_split_17lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=1024 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 141 | Clip re_lu_40 1 1 sep_conv2d_split_17lrelu_0 re_lu_40lpartitionedcall_0 0=0.0 1=6.0 142 | ConvolutionDepthWise batch_normalization_41 1 1 re_lu_40lpartitionedcall_0 batch_normalization_41lrelu_0 0=32 1=1 2=1 3=1 4=0 5=1 6=32 7=32 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 143 | Clip re_lu_41 1 1 batch_normalization_41lrelu_0 re_lu_41lpartitionedcall_0 0=0.0 1=6.0 144 | Eltwise add_19 2 1 re_lu_41lpartitionedcall_0 ncnn_split_21lpartitionedcall_1 add_19ladd_0 0=1 145 | Interp up_sampling2d_3 1 1 add_19ladd_0 up_sampling2d_3lresizelresizebilinear_0 0=2 1=2.0 2=2.0 146 | Eltwise add_20 2 1 up_sampling2d_3lresizelresizebilinear_0 ncnn_split_4lpartitionedcall_1 add_20ladd_0 0=1 147 | Convolution conv2d_8 1 1 add_20ladd_0 conv2d_8lrelu_0 0=16 1=3 2=1 3=1 4=1 5=1 6=4608 8=0 9=1 11=3 12=1 13=1 14=1 15=1 16=1 148 | Clip re_lu_42 1 1 conv2d_8lrelu_0 re_lu_42lpartitionedcall_0 0=0.0 1=6.0 149 | Split ncnn_split_22 1 2 re_lu_42lpartitionedcall_0 ncnn_split_22lpartitionedcall_0 ncnn_split_22lpartitionedcall_1 150 | ConvolutionDepthWise sep_dwconv2d_split_18 1 1 ncnn_split_22lpartitionedcall_0 sep_dwconv2d_split_18lbiasadd_0 0=16 1=3 2=1 3=1 4=1 5=1 6=144 7=16 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 151 | Convolution sep_conv2d_split_18 1 1 sep_dwconv2d_split_18lbiasadd_0 sep_conv2d_split_18lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=256 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 152 | Clip re_lu_43 1 1 sep_conv2d_split_18lrelu_0 re_lu_43lpartitionedcall_0 0=0.0 1=6.0 153 | ConvolutionDepthWise batch_normalization_44 1 1 re_lu_43lpartitionedcall_0 batch_normalization_44lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=16 7=16 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 154 | Clip re_lu_44 1 1 batch_normalization_44lrelu_0 re_lu_44lpartitionedcall_0 0=0.0 1=6.0 155 | Eltwise add_21 2 1 re_lu_44lpartitionedcall_0 ncnn_split_22lpartitionedcall_1 add_21ladd_0 0=1 156 | Split ncnn_split_23 1 2 add_21ladd_0 ncnn_split_23lpartitionedcall_0 ncnn_split_23lpartitionedcall_1 157 | ConvolutionDepthWise sep_dwconv2d_split_19 1 1 ncnn_split_23lpartitionedcall_0 sep_dwconv2d_split_19lbiasadd_0 0=16 1=3 2=1 3=1 4=1 5=1 6=144 7=16 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 158 | Convolution sep_conv2d_split_19 1 1 sep_dwconv2d_split_19lbiasadd_0 sep_conv2d_split_19lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=256 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 159 | Clip re_lu_45 1 1 sep_conv2d_split_19lrelu_0 re_lu_45lpartitionedcall_0 0=0.0 1=6.0 160 | ConvolutionDepthWise batch_normalization_46 1 1 re_lu_45lpartitionedcall_0 batch_normalization_46lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=16 7=16 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 161 | Clip re_lu_46 1 1 batch_normalization_46lrelu_0 re_lu_46lpartitionedcall_0 0=0.0 1=6.0 162 | Eltwise add_22 2 1 re_lu_46lpartitionedcall_0 ncnn_split_23lpartitionedcall_1 add_22ladd_0 0=1 163 | Split ncnn_split_24 1 2 add_22ladd_0 ncnn_split_24lpartitionedcall_0 ncnn_split_24lpartitionedcall_1 164 | ConvolutionDepthWise sep_dwconv2d_split_20 1 1 ncnn_split_24lpartitionedcall_0 sep_dwconv2d_split_20lbiasadd_0 0=16 1=3 2=1 3=1 4=1 5=1 6=144 7=16 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 165 | Convolution sep_conv2d_split_20 1 1 sep_dwconv2d_split_20lbiasadd_0 sep_conv2d_split_20lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=256 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 166 | Clip re_lu_47 1 1 sep_conv2d_split_20lrelu_0 re_lu_47lpartitionedcall_0 0=0.0 1=6.0 167 | ConvolutionDepthWise batch_normalization_48 1 1 re_lu_47lpartitionedcall_0 batch_normalization_48lrelu_0 0=16 1=1 2=1 3=1 4=0 5=1 6=16 7=16 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 168 | Clip re_lu_48 1 1 batch_normalization_48lrelu_0 re_lu_48lpartitionedcall_0 0=0.0 1=6.0 169 | Eltwise add_23 2 1 re_lu_48lpartitionedcall_0 ncnn_split_24lpartitionedcall_1 add_23ladd_0 0=1 170 | Interp up_sampling2d_4 1 1 add_23ladd_0 up_sampling2d_4lresizelresizebilinear_0 0=2 1=2.0 2=2.0 171 | Eltwise add_24 2 1 up_sampling2d_4lresizelresizebilinear_0 ncnn_split_2lpartitionedcall_1 add_24ladd_0 0=1 172 | Convolution conv2d_9 1 1 add_24ladd_0 conv2d_9lrelu_0 0=8 1=3 2=1 3=1 4=1 5=1 6=1152 8=0 9=1 11=3 12=1 13=1 14=1 15=1 16=1 173 | Clip re_lu_49 1 1 conv2d_9lrelu_0 re_lu_49lpartitionedcall_0 0=0.0 1=6.0 174 | Split ncnn_split_25 1 2 re_lu_49lpartitionedcall_0 ncnn_split_25lpartitionedcall_0 ncnn_split_25lpartitionedcall_1 175 | ConvolutionDepthWise sep_dwconv2d_split_21 1 1 ncnn_split_25lpartitionedcall_0 sep_dwconv2d_split_21lbiasadd_0 0=8 1=3 2=1 3=1 4=1 5=1 6=72 7=8 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 176 | Convolution sep_conv2d_split_21 1 1 sep_dwconv2d_split_21lbiasadd_0 sep_conv2d_split_21lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 177 | Clip re_lu_50 1 1 sep_conv2d_split_21lrelu_0 re_lu_50lpartitionedcall_0 0=0.0 1=6.0 178 | ConvolutionDepthWise batch_normalization_51 1 1 re_lu_50lpartitionedcall_0 batch_normalization_51lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=8 7=8 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 179 | Clip re_lu_51 1 1 batch_normalization_51lrelu_0 re_lu_51lpartitionedcall_0 0=0.0 1=6.0 180 | Eltwise add_25 2 1 re_lu_51lpartitionedcall_0 ncnn_split_25lpartitionedcall_1 add_25ladd_0 0=1 181 | Split ncnn_split_26 1 2 add_25ladd_0 ncnn_split_26lpartitionedcall_0 ncnn_split_26lpartitionedcall_1 182 | ConvolutionDepthWise sep_dwconv2d_split_22 1 1 ncnn_split_26lpartitionedcall_0 sep_dwconv2d_split_22lbiasadd_0 0=8 1=3 2=1 3=1 4=1 5=1 6=72 7=8 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 183 | Convolution sep_conv2d_split_22 1 1 sep_dwconv2d_split_22lbiasadd_0 sep_conv2d_split_22lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 184 | Clip re_lu_52 1 1 sep_conv2d_split_22lrelu_0 re_lu_52lpartitionedcall_0 0=0.0 1=6.0 185 | ConvolutionDepthWise batch_normalization_53 1 1 re_lu_52lpartitionedcall_0 batch_normalization_53lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=8 7=8 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 186 | Clip re_lu_53 1 1 batch_normalization_53lrelu_0 re_lu_53lpartitionedcall_0 0=0.0 1=6.0 187 | Eltwise add_26 2 1 re_lu_53lpartitionedcall_0 ncnn_split_26lpartitionedcall_1 add_26ladd_0 0=1 188 | Split ncnn_split_27 1 2 add_26ladd_0 ncnn_split_27lpartitionedcall_0 ncnn_split_27lpartitionedcall_1 189 | ConvolutionDepthWise sep_dwconv2d_split_23 1 1 ncnn_split_27lpartitionedcall_0 sep_dwconv2d_split_23lbiasadd_0 0=8 1=3 2=1 3=1 4=1 5=1 6=72 7=8 8=0 9=0 11=3 12=1 13=1 14=1 15=1 16=1 190 | Convolution sep_conv2d_split_23 1 1 sep_dwconv2d_split_23lbiasadd_0 sep_conv2d_split_23lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=64 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 191 | Clip re_lu_54 1 1 sep_conv2d_split_23lrelu_0 re_lu_54lpartitionedcall_0 0=0.0 1=6.0 192 | ConvolutionDepthWise batch_normalization_55 1 1 re_lu_54lpartitionedcall_0 batch_normalization_55lrelu_0 0=8 1=1 2=1 3=1 4=0 5=1 6=8 7=8 8=0 9=1 11=1 12=1 13=1 14=0 15=0 16=0 193 | Clip re_lu_55 1 1 batch_normalization_55lrelu_0 re_lu_55lpartitionedcall_0 0=0.0 1=6.0 194 | Eltwise add_27 2 1 re_lu_55lpartitionedcall_0 ncnn_split_27lpartitionedcall_1 add_27ladd_0 0=1 195 | Convolution softmax_conv 1 1 add_27ladd_0 softmax_convlsigmoid_0 0=1 1=1 2=1 3=1 4=0 5=1 6=8 8=0 9=4 11=1 12=1 13=1 14=0 15=0 16=0 196 | -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/README.md: -------------------------------------------------------------------------------- 1 | # Model source 2 | The model taken from [https://github.com/ItchyHiker/Hair_Segmentation_Keras.git]. 3 | The model was adapted to current script keras2ncnn.py. If you have [pyncnn](https://github.com/caishanli/pyncnn) installed 4 | in you environment you should be able to reproduce the inference result 5 | (see [demo.py](demo.py)) - with MAE :: 9.897014-08: 6 | 7 | ![alt Keras vs NCNN inference result](demo.png?raw=true "Keras vs NCNN inference result") 8 | -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/demo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/model_zoo/segmentation/hair/model_000/demo.png -------------------------------------------------------------------------------- /model_zoo/segmentation/hair/model_000/demo.py: -------------------------------------------------------------------------------- 1 | import ncnn 2 | import cv2 3 | import numpy as np 4 | from tensorflow.keras.models import load_model 5 | 6 | keras_model = load_model('model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.hdf5') 7 | 8 | net = ncnn.Net() 9 | net.load_param('model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.param') 10 | net.load_model('model_zoo/segmentation/hair/model_000/CelebA_PrismaNet_256_hair_seg_model_opt_001.bin') 11 | num_threads = 4 12 | 13 | frame_bgr = cv2.imread('./unit_test/unit_test_data/person_001_1024x1204.jpg') 14 | target_y, target_x = (256, 256) 15 | frame_bgr_show = cv2.resize(frame_bgr, (target_x, target_y)) 16 | frame = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB) 17 | src_y, src_x = frame.shape[:2] 18 | 19 | 20 | input_data = cv2.resize(frame, (target_x, target_y)) 21 | mean_value = input_data.reshape((-1, 3)).mean(0)[::-1] 22 | inv_std_value = 1. / (1.e-5 + input_data.reshape((-1, 3)).std(0))[::-1] 23 | 24 | mat_in = ncnn.Mat.from_pixels_resize(frame, ncnn.Mat.PixelType.PIXEL_RGB, src_x, src_y, target_x, target_y) 25 | mat_in.substract_mean_normalize(mean_value, inv_std_value) 26 | 27 | mean, std = cv2.meanStdDev(frame) 28 | 29 | keras_in = (np.array(input_data)[None] - mean_value)*inv_std_value 30 | keras_out = keras_model.predict(keras_in)[0, ..., 0] 31 | 32 | ex = net.create_extractor() 33 | ex.set_num_threads(num_threads) 34 | ex.input("data", mat_in) 35 | mat_out = ncnn.Mat() 36 | ex.extract("softmax_convlsigmoid_0", mat_out) 37 | ncnn_out = np.array(mat_out) 38 | ncnn_out = np.transpose(ncnn_out, (1, 2, 0)) 39 | 40 | out_file_name = 'model_zoo/segmentation/hair/model_000/demo.png' 41 | print(f'MAE :: {np.abs(keras_out.flatten() - ncnn_out.flatten()).mean()}') 42 | print(f'Output saved {out_file_name}') 43 | 44 | frame_bgr_show = np.pad(frame_bgr_show, ((7, 7), (7, 7), (0, 0)), constant_values=128) 45 | 46 | keras_out_show = cv2.cvtColor((255*keras_out).astype(np.uint8), cv2.COLOR_GRAY2BGR) 47 | keras_out_show = cv2.putText(keras_out_show, 'Keras Result', (20, 20), cv2.FONT_HERSHEY_PLAIN, 1.5, (0, 0, 255), 2) 48 | keras_out_show = np.pad(keras_out_show, ((7, 7), (7, 7), (0, 0)), constant_values=128) 49 | ncnn_out_show = cv2.cvtColor((255*ncnn_out).astype(np.uint8), cv2.COLOR_GRAY2BGR) 50 | ncnn_out_show = cv2.putText(ncnn_out_show, 'NCNN Result', (20, 20), cv2.FONT_HERSHEY_PLAIN, 1.5, (0, 0, 255), 2) 51 | ncnn_out_show = np.pad(ncnn_out_show, ((7, 7), (7, 7), (0, 0)), constant_values=128) 52 | 53 | show_frame = np.hstack([frame_bgr_show, keras_out_show, ncnn_out_show]) 54 | cv2.imwrite(out_file_name, show_frame) 55 | 56 | 57 | 58 | -------------------------------------------------------------------------------- /optimization/graph/ActivationReLU_max_split.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | from optimization.graph.layer_template import ReLU_config_template, Clip_config_template 3 | 4 | info_ActivationReLU_max = 'Activation(relu6): Activation(relu6) is not supported withing NCNN, split by ordinal ReLU and Clip' 5 | 6 | def transfer_ActivationReLU_max(src_model, dst_model, transfer_rule): 7 | pass 8 | 9 | def detect_transform_ActivationReLU_max(keras_config): 10 | index_list = [] 11 | for i, item in enumerate(keras_config['layers']): 12 | if item['class_name'] == 'Activation': 13 | if item['config']['activation'] == 'relu6': 14 | index_list.append(i) 15 | return index_list 16 | 17 | def check_ActivationReLU_max_transfrom(keras_config): 18 | return len(detect_transform_ActivationReLU_max(keras_config)) > 0 19 | 20 | def apply_transform_ActivationReLU_max(keras_config): 21 | index_list = detect_transform_ActivationReLU_max(keras_config) 22 | weight_transfer_rule_dict = {} 23 | while len(index_list) > 0: 24 | i = index_list[0] 25 | r_layer_config = keras_config['layers'].pop(i) 26 | # Transfer DepthWise 27 | i_layer_config = deepcopy(ReLU_config_template) 28 | # TODO :: check unique name 29 | prev_name = i_layer_config['name'] = r_layer_config['name'] + f'_dwc_{i}' 30 | for key in i_layer_config['config'].keys(): 31 | if key in r_layer_config['config']: 32 | i_layer_config['config'][key] = r_layer_config['config'][key] 33 | # Copy parameters 34 | i_layer_config['config']['threshold'] = 0 35 | i_layer_config['config']['max_value'] = None 36 | i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes'] 37 | i_layer_config['config']['name'] = i_layer_config['name'] 38 | keras_config['layers'].insert(i, i_layer_config) 39 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_ActivationReLU_max, 40 | 'src': r_layer_config['name'], 41 | 'dst': i_layer_config['name']} 42 | 43 | # Transfer Clip parameters 44 | i_layer_config = deepcopy(Clip_config_template) 45 | for key in i_layer_config['config'].keys(): 46 | if key in r_layer_config['config']: 47 | i_layer_config['config'][key] = r_layer_config['config'][key] 48 | i_layer_config['config']['max_value'] = 6 49 | i_layer_config['config']['min_value'] = 0 50 | i_layer_config['name'] = r_layer_config['name'] 51 | i_layer_config['inbound_nodes'] = [[[prev_name, 0, 0, {}]]] 52 | keras_config['layers'].insert(i + 1, i_layer_config) 53 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_ActivationReLU_max, 54 | 'src': r_layer_config['name'], 55 | 'dst': i_layer_config['name']} 56 | 57 | index_list = detect_transform_ActivationReLU_max(keras_config) 58 | return keras_config, weight_transfer_rule_dict 59 | 60 | -------------------------------------------------------------------------------- /optimization/graph/BatchNormalization_DepthwiseConv2D_transform.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from copy import deepcopy 3 | 4 | from optimization.graph.layer_template import DepthwiseConv2D_config_template 5 | 6 | info_BatchNormalization_DepthwiseConv2D = 'BatchNormalization->DepthwiseConv2D: Single BatchNormalization is ' \ 7 | 'recommended to transform in to DepthwiseConv2D for subsequent Activation ' \ 8 | 'merge...' 9 | 10 | 11 | def transfer_BatchNormalization_DepthwiseConv2D(src_model, dst_model, transfer_rule): 12 | gamma, beta, mean, var = src_model.get_layer(transfer_rule['src']).get_weights() 13 | eps = src_model.get_layer(transfer_rule['src']).get_config()['epsilon'] 14 | a = gamma / np.sqrt(var + eps) 15 | weight = a.reshape((1, 1, -1, 1)) 16 | bias = -a * mean + beta 17 | 18 | dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias]) 19 | 20 | 21 | def detect_transform_BatchNormalization_DepthwiseConv2D(keras_config): 22 | index_list = [] 23 | for i, item in enumerate(keras_config['layers']): 24 | if item['class_name'] == 'BatchNormalization': 25 | index_list.append(i) 26 | return index_list 27 | 28 | 29 | def check_BatchNormalization_DepthwiseConv2D(keras_config): 30 | return len(detect_transform_BatchNormalization_DepthwiseConv2D(keras_config)) > 0 31 | 32 | 33 | def apply_transform_BatchNormalization_DepthwiseConv2D(keras_config): 34 | index_list = detect_transform_BatchNormalization_DepthwiseConv2D(keras_config) 35 | weight_transfer_rule_dict = {} 36 | while len(index_list) > 0: 37 | i = index_list[0] 38 | r_layer_config = keras_config['layers'].pop(i) 39 | i_layer_config = deepcopy(DepthwiseConv2D_config_template) 40 | 41 | i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes'] 42 | i_layer_config['config']['name'] = i_layer_config['name'] = r_layer_config['name'] 43 | i_layer_config['config']['use_bias'] = True 44 | i_layer_config['config']['kernel_size'] = (1, 1) 45 | keras_config['layers'].insert(i, i_layer_config) 46 | weight_transfer_rule_dict[i_layer_config['name']] = { 47 | 'transfer_call': transfer_BatchNormalization_DepthwiseConv2D, 48 | 'src': r_layer_config['name'], 49 | 'dst': i_layer_config['name']} 50 | index_list = detect_transform_BatchNormalization_DepthwiseConv2D(keras_config) 51 | return keras_config, weight_transfer_rule_dict 52 | -------------------------------------------------------------------------------- /optimization/graph/Conv2DActivation_merge.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from tensorflow.keras.layers import ReLU 3 | from converter.model_adaptation import rename_layer, get_outbound_nodes 4 | 5 | info_Conv2DActivation = 'Conv2D->Activation: Inline operations should be merged' 6 | 7 | 8 | def transfer_Conv2DActivation_Conv2D(src_model, dst_model, transfer_rule): 9 | dst_model.get_layer(transfer_rule['dst']).set_weights(src_model.get_layer(transfer_rule['src']).get_weights()) 10 | 11 | 12 | def detect_transform_Conv2DActivation(keras_config): 13 | index_list = [] 14 | outbound_dict, index_dict = get_outbound_nodes(keras_config) 15 | for i, item in enumerate(keras_config['layers']): 16 | if item['class_name'] == 'Activation': 17 | __activation = item['config']['activation'] 18 | if not (__activation in ['linear', 'relu', 'sigmoid']): 19 | continue 20 | in_node_name = item['inbound_nodes'][0][0][0] 21 | in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name'] 22 | if in_node_class_name in ['Conv2D', 'DepthwiseConv2D', 'Conv2DTranspose']: 23 | _activation = keras_config['layers'][index_dict[in_node_name]]['config']['activation'] 24 | if _activation in ['linear', __activation]: 25 | if len(outbound_dict[in_node_name]) == 1: 26 | index_list.append(i) 27 | return index_list, index_dict 28 | 29 | 30 | def check_Conv2DActivation(keras_config): 31 | return len(detect_transform_Conv2DActivation(keras_config)[0]) > 0 32 | 33 | 34 | def apply_transform_Conv2DActivation(keras_config): 35 | index_list, index_dict = detect_transform_Conv2DActivation(keras_config) 36 | weight_transfer_rule_dict = {} 37 | while len(index_list) > 0: 38 | i = index_list[0] 39 | r_layer_config = keras_config['layers'].pop(i) 40 | src_name = r_layer_config['name'] 41 | dst_name = r_layer_config['inbound_nodes'][0][0][0] 42 | keras_config['layers'][index_dict[dst_name]]['config']['activation'] = r_layer_config['config']['activation'] 43 | transfer_call = transfer_Conv2DActivation_Conv2D 44 | 45 | keras_config = rename_layer(keras_config, src_name, dst_name) 46 | merged_dst = dst_name # + '_M' TODO Decide to rename the layer 47 | keras_config = rename_layer(keras_config, dst_name, merged_dst) 48 | weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_call, 49 | 'src': dst_name, 50 | 'dst': merged_dst} 51 | index_list, index_dict = detect_transform_Conv2DActivation(keras_config) 52 | return keras_config, weight_transfer_rule_dict 53 | -------------------------------------------------------------------------------- /optimization/graph/Conv2DBatchNormalization_merge.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from converter.model_adaptation import rename_layer, get_outbound_nodes 4 | 5 | info_Conv2DBatchNormalization = 'Conv2D/DepthwiseConv2D->BatchNormalization: Subsequent linear operations can be easily merged for inference' 6 | 7 | 8 | def transfer_Conv2DBatchNormalization_Conv2D(src_model, dst_model, transfer_rule): 9 | layer_c = src_model.get_layer(transfer_rule['src_c']) 10 | weigths_c = layer_c.get_weights() 11 | layer_b = src_model.get_layer(transfer_rule['src_b']) 12 | weigths_b = layer_b.get_weights() 13 | 14 | eps = layer_b.get_config()['epsilon'] 15 | if len(weigths_c) == 2: 16 | weight, bias = weigths_c 17 | else: 18 | weight, = weigths_c 19 | bias = 0 20 | gamma, beta, mean, var = weigths_b 21 | 22 | a = gamma / np.sqrt(var + eps) 23 | weight = weight * a.reshape((1, 1, 1, -1)) 24 | bias = a * (bias - mean) + beta 25 | 26 | dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias]) 27 | 28 | def transfer_Conv2DTransposeBatchNormalization_Conv2DTranspose(src_model, dst_model, transfer_rule): 29 | layer_c = src_model.get_layer(transfer_rule['src_c']) 30 | weigths_c = layer_c.get_weights() 31 | layer_b = src_model.get_layer(transfer_rule['src_b']) 32 | weigths_b = layer_b.get_weights() 33 | 34 | eps = layer_b.get_config()['epsilon'] 35 | if len(weigths_c) == 2: 36 | weight, bias = weigths_c 37 | else: 38 | weight, = weigths_c 39 | bias = 0 40 | gamma, beta, mean, var = weigths_b 41 | 42 | a = gamma / np.sqrt(var + eps) 43 | weight = weight * a.reshape((1, 1, -1, 1)) 44 | bias = a * (bias - mean) + beta 45 | 46 | dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias]) 47 | 48 | def transfer_DepthwiseConv2DBatchNormalization_Conv2D(src_model, dst_model, transfer_rule): 49 | layer_c = src_model.get_layer(transfer_rule['src_c']) 50 | weigths_c = layer_c.get_weights() 51 | layer_b = src_model.get_layer(transfer_rule['src_b']) 52 | weigths_b = layer_b.get_weights() 53 | 54 | eps = layer_b.get_config()['epsilon'] 55 | if len(weigths_c) == 2: 56 | weight, bias = weigths_c 57 | else: 58 | weight, = weigths_c 59 | bias = 0 60 | gamma, beta, mean, var = weigths_b 61 | 62 | a = gamma / np.sqrt(var + eps) 63 | weight = weight * a.reshape((1, 1, -1, 1)) 64 | bias = a * (bias - mean) + beta 65 | 66 | dst_model.get_layer(transfer_rule['dst']).set_weights([weight, bias]) 67 | 68 | 69 | mapping_class_dict = {'Conv2D': transfer_Conv2DBatchNormalization_Conv2D, 70 | 'DepthwiseConv2D': transfer_DepthwiseConv2DBatchNormalization_Conv2D, 71 | 'Conv2DTranspose': transfer_Conv2DTransposeBatchNormalization_Conv2DTranspose} 72 | 73 | 74 | def detect_transform_Conv2DBatchNormalization(keras_config): 75 | index_list = [] 76 | outbound_dict, index_dict = get_outbound_nodes(keras_config) 77 | for i, item in enumerate(keras_config['layers']): 78 | if item['class_name'] == 'BatchNormalization': 79 | in_node_name = item['inbound_nodes'][0][0][0] 80 | in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name'] 81 | if in_node_class_name in mapping_class_dict: 82 | _activation = keras_config['layers'][index_dict[in_node_name]]['config']['activation'] 83 | if not (_activation in ['linear']): 84 | continue 85 | try: 86 | if item['inbound_nodes'][0][0][-1]['training'] in [1, True]: 87 | continue 88 | except Exception as e_item: 89 | print(str(e_item)) 90 | 91 | if len(outbound_dict[in_node_name]) == 1: 92 | index_list.append(i) 93 | return index_list, index_dict 94 | 95 | 96 | def check_Conv2DBatchNormalization(keras_config): 97 | return len(detect_transform_Conv2DBatchNormalization(keras_config)[0]) > 0 98 | 99 | 100 | def apply_transform_Conv2DBatchNormalization(keras_config): 101 | index_list, index_dict = detect_transform_Conv2DBatchNormalization(keras_config) 102 | weight_transfer_rule_dict = {} 103 | while len(index_list) > 0: 104 | i = index_list[0] 105 | r_layer_config = keras_config['layers'].pop(i) 106 | src_name = r_layer_config['name'] 107 | dst_name = r_layer_config['inbound_nodes'][0][0][0] 108 | keras_config['layers'][index_dict[dst_name]]['config']['use_bias'] = True 109 | transfer_call = mapping_class_dict[keras_config['layers'][index_dict[dst_name]]['class_name']] 110 | 111 | keras_config = rename_layer(keras_config, src_name, dst_name) 112 | merged_dst = dst_name # + '_M' TODO Decide to rename the layer 113 | keras_config = rename_layer(keras_config, dst_name, merged_dst) 114 | weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_call, 115 | 'src_c': dst_name, 116 | 'src_b': src_name, 117 | 'dst': merged_dst} 118 | index_list, index_dict = detect_transform_Conv2DBatchNormalization(keras_config) 119 | return keras_config, weight_transfer_rule_dict 120 | -------------------------------------------------------------------------------- /optimization/graph/Conv2DReLU_merge.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from tensorflow.keras.layers import ReLU 3 | from converter.model_adaptation import rename_layer, get_outbound_nodes 4 | 5 | info_Conv2DReLU = 'Conv2D->ReLU: Inline operations should be merged' 6 | 7 | 8 | def transfer_Conv2DReLU_Conv2D(src_model, dst_model, transfer_rule): 9 | dst_model.get_layer(transfer_rule['dst']).set_weights(src_model.get_layer(transfer_rule['src']).get_weights()) 10 | 11 | 12 | # def get_outbound_nodes(keras_config): 13 | # outbound_dict = {} 14 | # index_dict = {} 15 | # for _i, _layer in enumerate(keras_config['layers']): 16 | # out_node_name = _layer['name'] 17 | # index_dict[out_node_name] = _i 18 | # if len(_layer['inbound_nodes']) == 0: 19 | # continue 20 | # in_node_name = _layer['inbound_nodes'][0][0][0] 21 | # if in_node_name in outbound_dict: 22 | # outbound_dict[in_node_name] += [out_node_name] 23 | # else: 24 | # outbound_dict[in_node_name] = [out_node_name] 25 | # 26 | # return outbound_dict, index_dict 27 | 28 | 29 | def detect_transform_Conv2DReLU(keras_config): 30 | index_list = [] 31 | outbound_dict, index_dict = get_outbound_nodes(keras_config) 32 | for i, item in enumerate(keras_config['layers']): 33 | if item['class_name'] == 'ReLU': 34 | if 'max_value' in item['config']: 35 | if item['config']['max_value'] is not None: 36 | continue 37 | if 'negative_slope' in item['config']: 38 | if item['config']['negative_slope'] > 0.: 39 | continue 40 | in_node_name = item['inbound_nodes'][0][0][0] 41 | in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name'] 42 | if in_node_class_name in ['Conv2D', 'DepthwiseConv2D', 'Conv2DTranspose']: 43 | _activation = keras_config['layers'][index_dict[in_node_name]]['config']['activation'] 44 | if _activation in ['linear', 'relu']: 45 | if len(outbound_dict[in_node_name]) == 1: 46 | index_list.append(i) 47 | return index_list, index_dict 48 | 49 | 50 | def check_Conv2DReLU(keras_config): 51 | return len(detect_transform_Conv2DReLU(keras_config)[0]) > 0 52 | 53 | 54 | def apply_transform_Conv2DReLU(keras_config): 55 | index_list, index_dict = detect_transform_Conv2DReLU(keras_config) 56 | weight_transfer_rule_dict = {} 57 | while len(index_list) > 0: 58 | i = index_list[0] 59 | r_layer_config = keras_config['layers'].pop(i) 60 | src_name = r_layer_config['name'] 61 | dst_name = r_layer_config['inbound_nodes'][0][0][0] 62 | keras_config['layers'][index_dict[dst_name]]['config']['activation'] = 'relu' 63 | transfer_call = transfer_Conv2DReLU_Conv2D 64 | 65 | keras_config = rename_layer(keras_config, src_name, dst_name) 66 | merged_dst = dst_name # + '_M' TODO Decide to rename the layer 67 | keras_config = rename_layer(keras_config, dst_name, merged_dst) 68 | weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_call, 69 | 'src': dst_name, 70 | 'dst': merged_dst} 71 | index_list, index_dict = detect_transform_Conv2DReLU(keras_config) 72 | return keras_config, weight_transfer_rule_dict 73 | -------------------------------------------------------------------------------- /optimization/graph/Conv2DSigmoid_merge.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from tensorflow.keras.layers import ReLU 3 | from converter.model_adaptation import rename_layer, get_outbound_nodes 4 | 5 | info_Conv2DSigmoid = 'Conv2D->Sigmoid: Inline operations should be merged' 6 | 7 | 8 | def transfer_Conv2DSigmoid_Conv2D(src_model, dst_model, transfer_rule): 9 | dst_model.get_layer(transfer_rule['dst']).set_weights(src_model.get_layer(transfer_rule['src']).get_weights()) 10 | 11 | 12 | def detect_transform_Conv2DSigmoid(keras_config): 13 | index_list = [] 14 | outbound_dict, index_dict = get_outbound_nodes(keras_config) 15 | for i, item in enumerate(keras_config['layers']): 16 | if item['class_name'] == 'Sigmoid': 17 | in_node_name = item['inbound_nodes'][0][0][0] 18 | in_node_class_name = keras_config['layers'][index_dict[in_node_name]]['class_name'] 19 | if in_node_class_name in ['Conv2D', 'DepthwiseConv2D', 'Conv2DTranspose']: 20 | _activation = keras_config['layers'][index_dict[in_node_name]]['config']['activation'] 21 | if _activation in ['linear', 'sigmoid']: 22 | if len(outbound_dict[in_node_name]) == 1: 23 | index_list.append(i) 24 | return index_list, index_dict 25 | 26 | 27 | def check_Conv2DSigmoid(keras_config): 28 | return len(detect_transform_Conv2DSigmoid(keras_config)[0]) > 0 29 | 30 | 31 | def apply_transform_Conv2DSigmoid(keras_config): 32 | index_list, index_dict = detect_transform_Conv2DSigmoid(keras_config) 33 | weight_transfer_rule_dict = {} 34 | while len(index_list) > 0: 35 | i = index_list[0] 36 | r_layer_config = keras_config['layers'].pop(i) 37 | src_name = r_layer_config['name'] 38 | dst_name = r_layer_config['inbound_nodes'][0][0][0] 39 | keras_config['layers'][index_dict[dst_name]]['config']['activation'] = 'sigmoid' 40 | transfer_call = transfer_Conv2DSigmoid_Conv2D 41 | 42 | keras_config = rename_layer(keras_config, src_name, dst_name) 43 | merged_dst = dst_name # + '_M' TODO Decide to rename the layer 44 | keras_config = rename_layer(keras_config, dst_name, merged_dst) 45 | weight_transfer_rule_dict[merged_dst] = {'transfer_call': transfer_call, 46 | 'src': dst_name, 47 | 'dst': merged_dst} 48 | index_list, index_dict = detect_transform_Conv2DSigmoid(keras_config) 49 | return keras_config, weight_transfer_rule_dict 50 | -------------------------------------------------------------------------------- /optimization/graph/Conv2DSoftmax_split.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | from optimization.graph.layer_template import Conv2D_config_template, Softmax_config_template 3 | 4 | info_Conv2DSoftmax = 'Conv2DSoftmax: Conv2D with Softmax activation can not be transformed. Transforming this to ' \ 5 | 'pair Conv2D->Softmax' 6 | 7 | def transfer_Conv2DSoftmax_Conv2D(src_model, dst_model, transfer_rule): 8 | dst_model.get_layer(transfer_rule['dst']).set_weights(src_model.get_layer(transfer_rule['src']).get_weights()) 9 | 10 | 11 | def transfer_Conv2DSoftmax_Softmax(src_model, dst_model, transfer_rule): 12 | pass 13 | 14 | 15 | def detect_transform_Conv2DSoftmax(keras_config): 16 | index_list = [] 17 | for i, item in enumerate(keras_config['layers']): 18 | if item['class_name'] == 'Conv2D': 19 | if item['config']['activation'] == 'softmax': 20 | index_list.append(i) 21 | return index_list 22 | 23 | def check_Conv2DSoftmax_transfrom(keras_config): 24 | return len(detect_transform_Conv2DSoftmax(keras_config)) > 0 25 | 26 | def apply_transform_Conv2DSoftmax(keras_config): 27 | index_list = detect_transform_Conv2DSoftmax(keras_config) 28 | weight_transfer_rule_dict = {} 29 | while len(index_list) > 0: 30 | i = index_list[0] 31 | r_layer_config = keras_config['layers'].pop(i) 32 | # Transfer DepthWise 33 | i_layer_config = deepcopy(Conv2D_config_template) 34 | # TODO :: check unique name 35 | prev_name = i_layer_config['name'] = r_layer_config['name'] + f'_act_{i}' 36 | for key in Conv2D_config_template['config'].keys(): 37 | if key in r_layer_config['config']: 38 | i_layer_config['config'][key] = r_layer_config['config'][key] 39 | i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes'] 40 | i_layer_config['config']['name'] = i_layer_config['name'] 41 | i_layer_config['config']['activation'] = 'linear' 42 | keras_config['layers'].insert(i, i_layer_config) 43 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_Conv2DSoftmax_Conv2D, 44 | 'src': r_layer_config['name'], 45 | 'dst': i_layer_config['name']} 46 | 47 | # Transfer Conv 48 | i_layer_config = deepcopy(Softmax_config_template) 49 | for key in set(Softmax_config_template['config'].keys()): 50 | if key in r_layer_config['config']: 51 | i_layer_config['config'][key] = r_layer_config['config'][key] 52 | i_layer_config['name'] = r_layer_config['name'] 53 | i_layer_config['inbound_nodes'] = [[[prev_name, 0, 0, {}]]] 54 | 55 | keras_config['layers'].insert(i + 1, i_layer_config) 56 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_Conv2DSoftmax_Softmax, 57 | 'src': r_layer_config['name'], 58 | 'dst': i_layer_config['name']} 59 | 60 | index_list = detect_transform_Conv2DSoftmax(keras_config) 61 | return keras_config, weight_transfer_rule_dict 62 | 63 | -------------------------------------------------------------------------------- /optimization/graph/DropLayer.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from copy import deepcopy 3 | 4 | from converter.model_adaptation import get_outbound_nodes, rename_layer 5 | 6 | info_DropLayer = 'Dropout: Some nodes can de skipped in inference mode' 7 | 8 | 9 | def transfer_DropLayer(src_model, dst_model, transfer_rule): 10 | pass 11 | 12 | 13 | def detect_transform_DropLayer(keras_config): 14 | index_list = [] 15 | for i, item in enumerate(keras_config['layers']): 16 | if item['class_name'] in ['Dropout', 'DropConnect']: 17 | index_list.append(i) 18 | return index_list 19 | 20 | 21 | def check_DropLayer(keras_config): 22 | return len(detect_transform_DropLayer(keras_config)) > 0 23 | 24 | 25 | def apply_transform_DropLayer(keras_config): 26 | index_list = detect_transform_DropLayer(keras_config) 27 | weight_transfer_rule_dict = {} 28 | while len(index_list) > 0: 29 | i = index_list[0] 30 | r_layer_config = keras_config['layers'].pop(i) 31 | src_name = r_layer_config['name'] 32 | dst_name = r_layer_config['inbound_nodes'][0][0][0] 33 | keras_config = rename_layer(keras_config, src_name, dst_name) 34 | 35 | index_list = detect_transform_DropLayer(keras_config) 36 | return keras_config, weight_transfer_rule_dict 37 | -------------------------------------------------------------------------------- /optimization/graph/ReLU_max_split.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | from optimization.graph.layer_template import ReLU_config_template, Clip_config_template 3 | 4 | info_ReLU_max = 'ReLU: Max Clip is not supported withing NCNN, split by ordinal ReLU and Clip' 5 | 6 | def transfer_ReLU_max(src_model, dst_model, transfer_rule): 7 | pass 8 | 9 | def detect_transform_ReLU_max(keras_config): 10 | index_list = [] 11 | for i, item in enumerate(keras_config['layers']): 12 | if item['class_name'] == 'ReLU': 13 | if 'max_value' in item['config']: 14 | if item['config']['max_value'] is not None: 15 | index_list.append(i) 16 | return index_list 17 | 18 | def check_ReLU_max_transfrom(keras_config): 19 | return len(detect_transform_ReLU_max(keras_config)) > 0 20 | 21 | def apply_transform_ReLU_max(keras_config): 22 | index_list = detect_transform_ReLU_max(keras_config) 23 | weight_transfer_rule_dict = {} 24 | while len(index_list) > 0: 25 | i = index_list[0] 26 | r_layer_config = keras_config['layers'].pop(i) 27 | # Transfer DepthWise 28 | i_layer_config = deepcopy(ReLU_config_template) 29 | # TODO :: check unique name 30 | prev_name = i_layer_config['name'] = r_layer_config['name'] + f'_dwc_{i}' 31 | for key in i_layer_config['config'].keys(): 32 | if key in r_layer_config['config']: 33 | i_layer_config['config'][key] = r_layer_config['config'][key] 34 | # Copy parameters 35 | i_layer_config['config']['threshold'] = 0 36 | i_layer_config['config']['max_value'] = None 37 | i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes'] 38 | i_layer_config['config']['name'] = i_layer_config['name'] 39 | keras_config['layers'].insert(i, i_layer_config) 40 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_ReLU_max, 41 | 'src': r_layer_config['name'], 42 | 'dst': i_layer_config['name']} 43 | 44 | # Transfer Clip parameters 45 | i_layer_config = deepcopy(Clip_config_template) 46 | for key in i_layer_config['config'].keys(): 47 | if key in r_layer_config['config']: 48 | i_layer_config['config'][key] = r_layer_config['config'][key] 49 | i_layer_config['config']['max_value'] = r_layer_config['config']['max_value'] 50 | i_layer_config['config']['min_value'] = r_layer_config['config']['threshold'] 51 | i_layer_config['name'] = r_layer_config['name'] 52 | i_layer_config['inbound_nodes'] = [[[prev_name, 0, 0, {}]]] 53 | keras_config['layers'].insert(i + 1, i_layer_config) 54 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_ReLU_max, 55 | 'src': r_layer_config['name'], 56 | 'dst': i_layer_config['name']} 57 | 58 | index_list = detect_transform_ReLU_max(keras_config) 59 | return keras_config, weight_transfer_rule_dict 60 | 61 | -------------------------------------------------------------------------------- /optimization/graph/SeparableConv2D_split.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | from optimization.graph.layer_template import Conv2D_config_template, DepthwiseConv2D_config_template 3 | 4 | info_SeparableConv2D = 'SeparableConv2D: It looks like was not implemented in NCNN, splitting it to DepthwiseConv2D ' \ 5 | 'and Conv2D ' 6 | 7 | def transfer_SeparableConv2D_DepthwiseConv2D(src_model, dst_model, transfer_rule): 8 | _weigths = src_model.get_layer(transfer_rule['src']).get_weights() 9 | dst_model.get_layer(transfer_rule['dst']).set_weights([_weigths[0]]) 10 | 11 | def transfer_SeparableConv2D_Conv2D(src_model, dst_model, transfer_rule): 12 | _weigths = src_model.get_layer(transfer_rule['src']).get_weights() 13 | dst_model.get_layer(transfer_rule['dst']).set_weights(_weigths[1:]) 14 | 15 | def detect_transform_SeparableConv2D(keras_config): 16 | index_list = [] 17 | for i, item in enumerate(keras_config['layers']): 18 | if item['class_name'] == 'SeparableConv2D': 19 | index_list.append(i) 20 | return index_list 21 | 22 | def check_SeparableConv2D_transfrom(keras_config): 23 | return len(detect_transform_SeparableConv2D(keras_config)) > 0 24 | 25 | def apply_transform_SeparableConv2D(keras_config): 26 | index_list = detect_transform_SeparableConv2D(keras_config) 27 | weight_transfer_rule_dict = {} 28 | while len(index_list) > 0: 29 | i = index_list[0] 30 | r_layer_config = keras_config['layers'].pop(i) 31 | # Transfer DepthWise 32 | i_layer_config = deepcopy(DepthwiseConv2D_config_template) 33 | # TODO :: check unique name 34 | prev_name = i_layer_config['name'] = r_layer_config['name'] + f'_dwc_{i}' 35 | for key in DepthwiseConv2D_config_template['config'].keys(): 36 | if key in r_layer_config['config']: 37 | i_layer_config['config'][key] = r_layer_config['config'][key] 38 | i_layer_config['inbound_nodes'] = r_layer_config['inbound_nodes'] 39 | i_layer_config['config']['name'] = i_layer_config['name'] 40 | i_layer_config['config']['use_bias'] = False 41 | keras_config['layers'].insert(i, i_layer_config) 42 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_SeparableConv2D_DepthwiseConv2D, 43 | 'src': r_layer_config['name'], 44 | 'dst': i_layer_config['name']} 45 | 46 | # Transfer Conv 47 | i_layer_config = deepcopy(Conv2D_config_template) 48 | for key in set(Conv2D_config_template['config'].keys()) - {'kernel_size', 'strides', 'dilation_rate'}: 49 | if key in r_layer_config['config']: 50 | i_layer_config['config'][key] = r_layer_config['config'][key] 51 | i_layer_config['name'] = r_layer_config['name'] 52 | i_layer_config['inbound_nodes'] = [[[prev_name, 0, 0, {}]]] 53 | 54 | keras_config['layers'].insert(i + 1, i_layer_config) 55 | weight_transfer_rule_dict[i_layer_config['name']] = {'transfer_call': transfer_SeparableConv2D_Conv2D, 56 | 'src': r_layer_config['name'], 57 | 'dst': i_layer_config['name']} 58 | 59 | index_list = detect_transform_SeparableConv2D(keras_config) 60 | return keras_config, weight_transfer_rule_dict 61 | 62 | -------------------------------------------------------------------------------- /optimization/graph/layer_template.py: -------------------------------------------------------------------------------- 1 | Conv2D_config_template = { 2 | 'name': '_node_name', 3 | 'class_name': 'Conv2D', 4 | 'config': {'name': '_node_name', 5 | 'trainable': True, 6 | 'dtype': 'float32', 7 | 'filters': 1, 8 | 'kernel_size': (1, 1), 9 | 'strides': (1, 1), 10 | 'padding': 'same', 11 | 'data_format': 'channels_last', 12 | 'dilation_rate': (1, 1), 13 | 'activation': 'linear', 14 | 'use_bias': True, 15 | 'kernel_initializer': {'class_name': 'GlorotUniform', 16 | 'config': {'seed': None}}, 17 | 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 18 | 'kernel_regularizer': None, 19 | 'bias_regularizer': None, 20 | 'activity_regularizer': None, 21 | 'kernel_constraint': None, 22 | 'bias_constraint': None}, 23 | 'name': '_node_name', 24 | 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]} 25 | 26 | DepthwiseConv2D_config_template = {'name': '_node_name', 27 | 'class_name': 'DepthwiseConv2D', 28 | 'config': {'name': '_node_name', 29 | 'trainable': True, 30 | 'dtype': 'float32', 31 | 'kernel_size': (1, 1), 32 | 'strides': (1, 1), 33 | 'padding': 'same', 34 | 'data_format': 'channels_last', 35 | 'dilation_rate': (1, 1), 36 | 'activation': 'linear', 37 | 'use_bias': False, 38 | 'bias_initializer': {'class_name': 'Zeros', 39 | 'config': {'dtype': 'float32'}}, 40 | 'bias_regularizer': None, 41 | 'activity_regularizer': None, 42 | 'bias_constraint': None, 43 | 'depth_multiplier': 1, 44 | 'depthwise_initializer': {'class_name': 'GlorotUniform', 45 | 'config': {'seed': None, 'dtype': 'float32'}}, 46 | 'depthwise_regularizer': None, 47 | 'depthwise_constraint': None}, 48 | 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]} 49 | 50 | ReLU_config_template = {'class_name': 'ReLU', 51 | 'config': {'name': '_node_name', 52 | 'trainable': True, 53 | 'dtype': 'float32', 54 | 'max_value': None, 55 | 'negative_slope': 0., 56 | 'threshold': 0.}, 57 | 'name': '_node_name', 58 | 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]} 59 | 60 | Softmax_config_template = {'class_name': 'Softmax', 61 | 'config': {'axis': -1, 62 | 'dtype': 'float32', 63 | 'name': 'softmax_1', 64 | 'trainable': True}, 65 | 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]], 66 | 'name': '_in_node_name'} 67 | 68 | Clip_config_template = {'class_name': 'Clip', 69 | 'config': {'name': '_node_name', 70 | 'trainable': True, 71 | 'dtype': 'float32', 72 | 'min_value': -1., 73 | 'max_value': +1.}, 74 | 'name': '_node_name', 75 | 'inbound_nodes': [[['_in_node_name', 0, 0, {}]]]} 76 | -------------------------------------------------------------------------------- /optimization/optimize_graph.py: -------------------------------------------------------------------------------- 1 | from tqdm import tqdm 2 | import numpy as np 3 | from unit_test.helper import fix_none_in_shape 4 | from tensorflow.keras.models import Model 5 | 6 | from extra_layers.CustomObjects import extra_custom_objects 7 | from optimization.graph.SeparableConv2D_split import check_SeparableConv2D_transfrom, apply_transform_SeparableConv2D, \ 8 | info_SeparableConv2D 9 | from optimization.graph.Conv2DBatchNormalization_merge import check_Conv2DBatchNormalization, \ 10 | apply_transform_Conv2DBatchNormalization, info_Conv2DBatchNormalization 11 | from optimization.graph.BatchNormalization_DepthwiseConv2D_transform import check_BatchNormalization_DepthwiseConv2D, \ 12 | apply_transform_BatchNormalization_DepthwiseConv2D, info_BatchNormalization_DepthwiseConv2D 13 | from optimization.graph.Conv2DSoftmax_split import check_Conv2DSoftmax_transfrom, \ 14 | apply_transform_Conv2DSoftmax, info_Conv2DSoftmax 15 | 16 | from optimization.graph.Conv2DReLU_merge import check_Conv2DReLU, apply_transform_Conv2DReLU, info_Conv2DReLU 17 | from optimization.graph.Conv2DSigmoid_merge import check_Conv2DSigmoid, apply_transform_Conv2DSigmoid, info_Conv2DSigmoid 18 | from optimization.graph.Conv2DActivation_merge import check_Conv2DActivation, apply_transform_Conv2DActivation, info_Conv2DActivation 19 | from optimization.graph.ReLU_max_split import check_ReLU_max_transfrom, apply_transform_ReLU_max, info_ReLU_max 20 | from optimization.graph.ActivationReLU_max_split import check_ActivationReLU_max_transfrom, apply_transform_ActivationReLU_max, info_ActivationReLU_max 21 | 22 | from optimization.graph.DropLayer import check_DropLayer, \ 23 | apply_transform_DropLayer, info_DropLayer 24 | 25 | 26 | info_list = [info_DropLayer, 27 | info_ReLU_max, 28 | info_ActivationReLU_max, 29 | info_SeparableConv2D, 30 | info_Conv2DBatchNormalization, 31 | info_BatchNormalization_DepthwiseConv2D, 32 | info_Conv2DReLU, 33 | info_Conv2DSigmoid, 34 | info_Conv2DActivation, 35 | info_Conv2DSoftmax, 36 | ] 37 | check_transform_list = [check_DropLayer, 38 | check_ReLU_max_transfrom, 39 | check_ActivationReLU_max_transfrom, 40 | check_SeparableConv2D_transfrom, 41 | check_Conv2DBatchNormalization, 42 | check_BatchNormalization_DepthwiseConv2D, 43 | check_Conv2DReLU, 44 | check_Conv2DSigmoid, 45 | check_Conv2DActivation, 46 | check_Conv2DSoftmax_transfrom, 47 | ] 48 | apply_transform_list = [apply_transform_DropLayer, 49 | apply_transform_ReLU_max, 50 | apply_transform_ActivationReLU_max, 51 | apply_transform_SeparableConv2D, 52 | apply_transform_Conv2DBatchNormalization, 53 | apply_transform_BatchNormalization_DepthwiseConv2D, 54 | apply_transform_Conv2DReLU, 55 | apply_transform_Conv2DSigmoid, 56 | apply_transform_Conv2DActivation, 57 | apply_transform_Conv2DSoftmax, 58 | ] 59 | 60 | 61 | def transfer_weights(src_model, dst_model, weight_transfer_rule_dict): 62 | print('\nWeight transfer...') 63 | for dst_layer in tqdm(dst_model.layers): 64 | if dst_layer.name in weight_transfer_rule_dict: 65 | transfer_rule = weight_transfer_rule_dict[dst_layer.name] 66 | func = transfer_rule['transfer_call'] 67 | func(src_model, dst_model, transfer_rule) 68 | else: 69 | dst_layer.set_weights(src_model.get_layer(dst_layer.name).get_weights()) 70 | 71 | def check_transform(src_model, dst_model, debug=True): 72 | if debug: 73 | print("Checking Transfer :: Random value check") 74 | if type(src_model.input_shape) == list: 75 | x_in = [np.random.uniform(size=fix_none_in_shape(item)) for item in src_model.input_shape] 76 | else: 77 | _shape = tuple(32 if item is None else item for item in src_model.input_shape[1:]) 78 | x_in = np.random.uniform(size=(1,) + _shape) 79 | dst_output = dst_model.predict(x_in) 80 | src_output = src_model.predict(x_in) 81 | if isinstance(dst_output,list): 82 | for _i, (src_item, dst_item) in enumerate(zip(src_output, dst_output)): 83 | transform_error = np.abs(src_item - dst_item).mean() 84 | if debug: 85 | print(f" Output {_i} Transform Error (is less 1e-5) :: {transform_error} , {transform_error < 1e-5}") 86 | else: 87 | transform_error = np.abs(dst_output - src_output).mean() 88 | if debug: 89 | print(f" Transform Error (is less 1e-5) :: {transform_error} , {transform_error < 1e-5}") 90 | return transform_error 91 | 92 | 93 | def apply_transformations(in_model): 94 | src_model = None 95 | src_model_config = in_model.get_config() 96 | k = 1024 97 | for info_txt, check_func, apply_func in zip(info_list[:k], check_transform_list[:k], apply_transform_list[:k]): 98 | if check_func(src_model_config): 99 | if src_model is None: 100 | print('Preparation for the transformation...\n') 101 | src_model = Model.from_config(in_model.get_config(), custom_objects=extra_custom_objects) 102 | transfer_weights(in_model, src_model, {}) 103 | check_transform(in_model, src_model) 104 | 105 | print(info_txt) 106 | src_model_config = src_model.get_config() 107 | dst_model_config, weight_transfer_rule_dict = apply_func(src_model_config) 108 | dst_model = Model.from_config(dst_model_config, custom_objects=extra_custom_objects) 109 | transfer_weights(src_model, dst_model, weight_transfer_rule_dict) 110 | check_transform(in_model, dst_model) 111 | 112 | del src_model 113 | src_model = dst_model 114 | else: 115 | print(f'Nothing to do with {info_txt.split(":")[0]} transform\n') 116 | 117 | if src_model is None: 118 | return in_model 119 | 120 | return src_model 121 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | absl-py==0.10.0 2 | astor==0.8.1 3 | certifi==2020.6.20 4 | gast==0.2.2 5 | google-pasta==0.2.0 6 | grpcio==1.32.0 7 | h5py==2.10.0 8 | imageio==2.9.0 9 | importlib-metadata==2.0.0 10 | Keras-Applications==1.0.8 11 | Keras-Preprocessing==1.1.2 12 | Markdown==3.3 13 | numpy==1.19.2 14 | opencv-python==4.4.0.44 15 | opt-einsum==3.3.0 16 | Pillow==7.2.0 17 | protobuf==3.13.0 18 | six==1.15.0 19 | tensorboard==1.15.0 20 | tensorflow==1.15.0 21 | tensorflow-estimator==1.15.1 22 | termcolor==1.1.0 23 | tqdm 24 | Werkzeug==1.0.1 25 | wincertstore==0.2 26 | wrapt==1.12.1 27 | zipp==3.3.0 28 | -------------------------------------------------------------------------------- /run_test.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" 3 | os.environ["CUDA_VISIBLE_DEVICES"] = "-1" 4 | from tensorflow.keras import backend as K 5 | from tensorflow.keras.models import load_model 6 | 7 | import ncnn 8 | import numpy as np 9 | import cv2 10 | 11 | from converter.converter import conver_model 12 | from converter.model_adaptation import adapt_keras_model, convert_blob, clean_node_name 13 | from optimization.optimize_graph import apply_transformations, check_transform 14 | from unit_test.helper import save_config 15 | from extra_layers.CustomObjects import extra_custom_objects 16 | from unit_test.helper import fix_none_in_shape 17 | # from unit_test.single_layer.Activation import model_list 18 | # from unit_test.single_layer.Conv2D import model_list 19 | # from unit_test.single_layer.DepthwiseConv2D import model_list 20 | # from unit_test.single_layer.Normalization import model_list 21 | # from unit_test.single_layer.Pooling2D import model_list 22 | # from unit_test.single_layer.UpSampling2D import model_list 23 | # from unit_test.single_layer.ReshapeFlatten import model_list 24 | # from unit_test.single_layer.Dense import model_list 25 | # from unit_test.single_layer.Merge import model_list 26 | # from unit_test.single_layer.Conv2DTranspose import model_list 27 | # from unit_test.single_layer.UnaryOp import model_list 28 | # from unit_test.single_layer.BinaryOp import model_list 29 | 30 | # from unit_test.simple_model.EncoderDecoder import model_list 31 | # from unit_test.simple_model.UNet import model_list 32 | # from unit_test.simple_model.MultipleInput import model_list 33 | # from unit_test.simple_model.Adain import model_list 34 | 35 | # model_list = [load_model('unit_test_output/encoder.hdf5', custom_objects=extra_custom_objects), 36 | # load_model('unit_test_output/decoder.hdf5', custom_objects=extra_custom_objects), 37 | # load_model('unit_test_output/adain.hdf5', custom_objects=extra_custom_objects), 38 | # ] 39 | # model_list = [load_model('unit_test_output/keras_arbitrary_style_transfer.hdf5', custom_objects=extra_custom_objects)] 40 | # model_list = [load_model('divamgupta/image-segmentation-keras/_adaptation/pspnet_50_ADE_20K.hdf5', 41 | # custom_objects=extra_custom_objects)] 42 | # model_list = [load_model('divamgupta/image-segmentation-keras/_adaptation/pspnet_101_voc12.hdf5', 43 | # custom_objects=extra_custom_objects)] 44 | # model_list = [load_model('model_privat/style_transfer/pix2pix/cats_v1.hdf5')] # code demo 45 | # model_list = [load_model('model_zoo/detection/AIZOOTech_I_FaceMaskDetection/face_mask_detection_optimized.hdf5')] #issue 1 46 | # model_list = [load_model('./model_zoo/variouse/issue_00003/fiop_dumb_model_fixed.h5')] #issue 3 47 | # model_list = [load_model('model_zoo/variouse/issue_00006/deconv_fin_munet.h5')] #issue 6 48 | # from tensorflow.keras.losses import mae 49 | # model_list = [load_model('model_zoo/variouse/issue_00011/weights_v2.h5', custom_objects={'bce_dice_loss': mae, 50 | # 'IOU': mae})] #issue 11 51 | # from tensorflow.keras.initializers import RandomNormal 52 | # extra_custom_objects['EfficientConv2DKernelInitializer'] = RandomNormal 53 | # model_list = [load_model('model_zoo/variouse/issue_00010/WHENet_fixed.h5', custom_objects=extra_custom_objects)] #issue 10 54 | from tensorflow.keras.losses import mae 55 | import tensorflow 56 | model_list = [load_model('model_zoo/variouse/issue_00014/weights_40.h5', custom_objects={'relu6': tensorflow.nn.relu6, 57 | 'bce_dice_loss': mae, 58 | 'IOU': mae})] #issue 11 59 | 60 | def mat_to_numpy_4(mat_array): 61 | np_array = np.array(mat_array) 62 | assert len(np_array.shape) == 3, f"Wrong Array Shape {np_array.shape}" 63 | np_array = np_array.reshape((1,) + np_array.shape) 64 | return np_array 65 | 66 | 67 | def mat_to_numpy_3(mat_array): 68 | np_array = np.array(mat_array) 69 | assert len(np_array.shape) == 2, f"Wrong Array Shape {np_array.shape}" 70 | np_array = np_array.reshape((1,) + np_array.shape) 71 | return np_array 72 | 73 | 74 | def mat_to_numpy_2(mat_array): 75 | np_array = np.array(mat_array) 76 | assert len(np_array.shape) == 1, f"Wrong Array Shape {np_array.shape}" 77 | np_array = np_array.reshape((1,) + np_array.shape) 78 | return np_array 79 | 80 | 81 | def tensor_nchw2nhwc_4(in_data): 82 | return np.transpose(in_data, (0, 2, 3, 1)) 83 | 84 | 85 | def tensor_nchw2nhwc_3(in_data): 86 | return np.transpose(in_data, (0, 2, 1)) 87 | 88 | 89 | def tensor_nchw2nhwc_2(in_data): 90 | return in_data 91 | 92 | 93 | def tensor4_ncnn2keras(mat_array): 94 | if mat_array.dims == 3: 95 | return tensor_nchw2nhwc_4(mat_to_numpy_4(mat_array)) 96 | elif mat_array.dims == 2: 97 | return tensor_nchw2nhwc_3(mat_to_numpy_3(mat_array)) 98 | elif mat_array.dims == 1: 99 | return tensor_nchw2nhwc_2(mat_to_numpy_2(mat_array)) 100 | else: 101 | print(f'tensor4_ncnn2keras :: {mat_array.dims}') 102 | raise NotImplemented 103 | 104 | 105 | export_root = './unit_test_output/' 106 | 107 | for keras_model_in in model_list: 108 | # keras_model_in.summary() 109 | keras_model = apply_transformations(keras_model_in) 110 | adapted_keras_model = adapt_keras_model(keras_model, keras_model.name) 111 | check_transform(keras_model_in, adapted_keras_model, False) 112 | string_list, weight_list, layer_name_list = conver_model(adapted_keras_model, False, False) 113 | 114 | export_path = os.path.join(export_root, '', keras_model.name) 115 | os.makedirs(export_path, exist_ok=True) 116 | out_config_path, out_weights_path = save_config(string_list, weight_list, adapted_keras_model.name, export_path, 117 | debug=False) 118 | 119 | net = ncnn.Net() 120 | net.load_param(out_config_path) 121 | net.load_model(out_weights_path) 122 | 123 | num_threads = 4 124 | error_th = 1.e-5 125 | ex = net.create_extractor() 126 | ex.set_num_threads(num_threads) 127 | 128 | if type(keras_model.input_shape) != list: 129 | target_shape_list = [keras_model.input_shape] 130 | else: 131 | target_shape_list = keras_model.input_shape 132 | 133 | keras_in_list = [] 134 | for input_index, target_shape in enumerate(target_shape_list): 135 | target_shape = fix_none_in_shape(target_shape) 136 | src_x, src_y = target_x, target_y = target_shape[1:3] 137 | 138 | if target_shape[-1] == 3: 139 | frame = np.random.uniform(0, 255, size=fix_none_in_shape(target_shape)[1:]).astype(np.uint8) 140 | mat_in = ncnn.Mat.from_pixels_resize(frame, ncnn.Mat.PixelType.PIXEL_BGR, src_x, src_y, target_x, target_y) 141 | mean = np.array([0] * 3) 142 | std = 1. / np.array([255.] * 3) 143 | mat_in.substract_mean_normalize(mean, std) 144 | keras_tensor = (frame[None, ...] - mean) * std 145 | else: 146 | frame = np.random.uniform(0., +1., size=fix_none_in_shape(target_shape)) 147 | mat_in = ncnn.Mat(np.transpose(frame, (0, 3, 1, 2))[0].astype(np.float32)) 148 | keras_tensor = np.transpose(np.array(mat_in)[None, ...], (0, 2, 3, 1)) 149 | 150 | 151 | # Check input 152 | keras_tensor_cmp = tensor4_ncnn2keras(mat_in) 153 | keras_in_list.append(keras_tensor) 154 | assert np.abs(keras_tensor - keras_tensor_cmp).mean() < 1.e-5, 'Bad Input Tensor!' 155 | 156 | ncnn_input_name = clean_node_name(adapted_keras_model.inputs[input_index].name) 157 | ex.input(ncnn_input_name, mat_in) 158 | 159 | print('\n' + ('=' * 20) + 'Test mode from ./run_test.py' + ('=' * 20)) 160 | print('\n' + ('=' * 20) + 'By Layer Comparison ' + ('=' * 20)) 161 | mat_out = ncnn.Mat() 162 | inference_sum = 0 163 | for layer in adapted_keras_model.layers: 164 | layer_name = layer.name 165 | layer_output = convert_blob(layer.output) 166 | 167 | test_keras_model = K.function(adapted_keras_model.inputs, adapted_keras_model.get_layer(layer_name).output) 168 | keras_output = test_keras_model(keras_in_list) 169 | 170 | for item, tensor_true in zip(layer_output, convert_blob(keras_output)): 171 | 172 | tensor_name = clean_node_name(item.name) 173 | ex.extract(tensor_name, mat_out) 174 | 175 | tensor_exp = tensor4_ncnn2keras(mat_out) 176 | inference_sum += np.prod(tensor_true.shape) 177 | try: 178 | error_exp = np.abs(tensor_true - tensor_exp).mean() 179 | print(f'Layer - {layer_name} inference MAE :: {error_exp} < {str(error_th)} {error_exp < error_th} ' + 180 | f"Keras::{tensor_true.shape} / NCNN::{tensor_exp.shape}") 181 | except Exception as e_item: 182 | print('-' * 10, layer_name, '-' * 10) 183 | print(layer_name, str(e_item)) 184 | ... 185 | 186 | inference_sum_float32 = int(0.5 + (inference_sum*4.)/(2**20)) 187 | print(f'Estimated float32 inference memory :: {inference_sum_float32} MB') 188 | print(f'{export_path}') 189 | -------------------------------------------------------------------------------- /unit_test/helper.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | import os 3 | import numpy as np 4 | import imageio 5 | import cv2 6 | import tensorflow as tf 7 | 8 | 9 | def fix_none_in_shape(in_shape): 10 | default_shape = (1, 32, 32, 32) 11 | out_shape = [] 12 | for src, default in zip(in_shape, default_shape): 13 | if src is None: 14 | out_shape.append(default) 15 | else: 16 | out_shape.append(src) 17 | return tuple(out_shape) 18 | 19 | 20 | def clean_name(in_str): 21 | out_str = "".join(in_str.split()).replace('(', '').replace(')', '').replace(',', 'x').strip() 22 | return out_str 23 | 24 | 25 | def tf_random_seed(): 26 | if tf.__version__.split('.')[0] == '1': 27 | tf.random.set_random_seed(7) 28 | else: 29 | tf.random.set_seed(7) 30 | 31 | 32 | def np_random_seed(): 33 | np.random.seed(7) 34 | 35 | 36 | def get_test_item_zero_one(target_shape): 37 | file_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '', 'unit_test_data/person_001_1024x1204.jpg') 38 | frame_data = imageio.imread(file_path) 39 | frame_data = cv2.resize(frame_data, tuple(reversed(target_shape[:2]))) / 255. 40 | return frame_data[None, ...] 41 | 42 | 43 | def get_test_item_mean_std(target_shape): 44 | file_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '', 'unit_test_data/person_001_1024x1204.jpg') 45 | frame_data = imageio.imread(file_path) 46 | frame_data = cv2.resize(frame_data, tuple(reversed(target_shape[:2]))) / 255. 47 | frame_data = (frame_data - frame_data.mean()) / frame_data.std() 48 | return frame_data[None, ...] 49 | 50 | 51 | def save_layer_unit_test(model, model_name, root_folder, inference_mode, get_test_item, dtype=np.float32): 52 | target_shape = model.input_shape[1:3] 53 | x_in = get_test_item(target_shape) 54 | unit_test_path = os.path.join(root_folder, '', model_name, '', 'unit_test') 55 | os.makedirs(unit_test_path, exist_ok=True) 56 | dtype_string = str(dtype.__name__) 57 | for layer_index, layer in enumerate(model.layers): 58 | if layer_index == 0: 59 | # TODO:: FIX input name in NCNN 60 | y_out = x_in.copy().astype(dtype) 61 | layer_unit_test_path = os.path.join(unit_test_path, '', f'{model.input_names[0]}.{dtype_string}') 62 | elif layer.name in model.output_names: 63 | pass 64 | # TODO:: FIX output name in NCNN 65 | # layer_unit_test_path = os.path.join(unit_test_path, '', f"{'output'}.{dtype_string}") 66 | # test_model = Model(model.input, layer.output) 67 | # y_out = test_model.predict(x_in) 68 | else: 69 | layer_unit_test_path = os.path.join(unit_test_path, '', f'{layer.name}.{dtype_string}') 70 | test_model = Model(model.input, layer.output) 71 | y_out = test_model.predict(x_in) 72 | if isinstance(y_out, list): 73 | y_out = y_out[0] 74 | if inference_mode == 'NCHW': 75 | y_out = np.transpose(y_out, (0, 3, 1, 2)) 76 | y_out.astype(dtype).tofile(layer_unit_test_path) 77 | 78 | 79 | def save_config(string_list, weight_list, model_name, root_folder, dtype=np.float32, debug=True): 80 | out_config_path = os.path.join(root_folder, '', f'{model_name}.param') 81 | out_weights_path = os.path.join(root_folder, '', f'{model_name}.bin') 82 | 83 | if debug: 84 | for item in string_list: 85 | print(item) 86 | 87 | with open(out_config_path, 'w') as f: 88 | for item in string_list: 89 | f.write(item + '\n') 90 | 91 | with open(out_weights_path, 'wb') as f: 92 | for item in weight_list: 93 | layer_class, weight_list_item = item 94 | if layer_class in ['Conv2D', 'DepthwiseConv2D', 'Conv2DTranspose', 'Dense']: 95 | f.write(np.array([0], dtype=np.uint32).tobytes()) 96 | for weight_array in weight_list_item: 97 | f.write(weight_array.astype(dtype).tobytes()) 98 | 99 | return out_config_path, out_weights_path 100 | -------------------------------------------------------------------------------- /unit_test/simple_model/Adain.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.layers import Input, GlobalAveragePooling2D, Multiply, Subtract, DepthwiseConv2D, Add, Reshape 2 | from extra_layers.BinaryOp import Div 3 | from extra_layers.UnaryOp import Sqrt 4 | from tensorflow.keras.models import Model 5 | import numpy as np 6 | 7 | in_c = Input((None, None, 512)) 8 | in_s = Input((None, None, 512)) 9 | epsilon = 1.e-5 10 | 11 | epsilon_tensor = DepthwiseConv2D((1, 1), weights=[np.full((1, 1, int(in_c.shape[-1]), 1), 0.), 12 | np.full((int(in_c.shape[-1]),), -epsilon)])(in_c) 13 | zero_tensor = DepthwiseConv2D((1, 1), weights=[np.full((1, 1, int(in_c.shape[-1]), 1), 0.), 14 | np.full((int(in_c.shape[-1]),), 0)])(in_s) 15 | 16 | mean_s = GlobalAveragePooling2D()(in_s) 17 | diff_s = Subtract()([in_s, mean_s]) 18 | diff_square_s = Multiply()([diff_s, diff_s]) 19 | var_s = GlobalAveragePooling2D()(diff_square_s) 20 | 21 | mean_c = GlobalAveragePooling2D()(in_c) 22 | diff_c = Subtract()([in_c, mean_c]) 23 | diff_square_c = Multiply()([diff_c, diff_c]) 24 | var_c = GlobalAveragePooling2D()(diff_square_c) 25 | 26 | var_c = Reshape((1, 1, int(var_c.shape[-1])))(var_c) 27 | var_s = Reshape((1, 1, int(var_s.shape[-1])))(var_s) 28 | 29 | var_c = Subtract()([var_c, epsilon_tensor]) 30 | var_c = Sqrt()(var_c) 31 | var_s = Subtract()([var_s, epsilon_tensor]) 32 | var_s = Sqrt()(var_s) 33 | mean_s = Subtract()([mean_s, epsilon_tensor]) 34 | 35 | x = Div()([diff_c, var_c]) 36 | x = Multiply()([var_s, x]) 37 | x = Add()([x, mean_s]) 38 | 39 | keras_adain_model = Model([in_c, in_s], x) 40 | 41 | model_list = [keras_adain_model] 42 | 43 | keras_adain_model.save('adain.hdf5') -------------------------------------------------------------------------------- /unit_test/simple_model/EncoderDecoder.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Conv2D, UpSampling2D, BatchNormalization, MaxPooling2D, Activation, ReLU 3 | from unit_test.helper import tf_random_seed 4 | 5 | model_list = [] 6 | 7 | for model_depth in [1, 2]: 8 | tf_random_seed() 9 | placeholder = x = Input((32, 32, 3), name='data') 10 | for j in range(model_depth): 11 | x = Conv2D(8, (3,3), padding='same')(x) 12 | x = BatchNormalization()(x) 13 | x = ReLU()(x) 14 | x = MaxPooling2D((2,2))(x) 15 | 16 | for j in range(model_depth): 17 | x = Conv2D(8, (3,3), padding='same')(x) 18 | x = BatchNormalization()(x) 19 | x = ReLU()(x) 20 | x = UpSampling2D((2,2), interpolation='bilinear')(x) 21 | 22 | x = Conv2D(1, (3, 3), padding='same')(x) 23 | x = Activation('sigmoid')(x) 24 | 25 | model = Model(placeholder, x, name=f'simple_model_encoder_decoder_{str(model_depth).zfill(3)}') 26 | model_list.append(model) 27 | 28 | -------------------------------------------------------------------------------- /unit_test/simple_model/MultipleInput.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Conv2D, UpSampling2D, BatchNormalization, MaxPooling2D, Activation, ReLU, Add 3 | from unit_test.helper import tf_random_seed 4 | 5 | model_list = [] 6 | 7 | for model_depth in [1]: 8 | tf_random_seed() 9 | placeholder_z = Input((32, 32, 3), name='data_1') 10 | placeholder_y = Input((32, 32, 3), name='data_2') 11 | x = Add()([placeholder_y, placeholder_z]) 12 | 13 | model = Model([placeholder_y, placeholder_z], x, name=f'multi_input_model_{str(model_depth).zfill(3)}') 14 | model_list.append(model) 15 | -------------------------------------------------------------------------------- /unit_test/simple_model/UNet.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model, load_model 2 | from tensorflow.keras.layers import Input, Conv2D, UpSampling2D, BatchNormalization, MaxPooling2D, Activation, \ 3 | ReLU, Concatenate, LeakyReLU, Add 4 | from unit_test.helper import tf_random_seed 5 | 6 | model_list = [] 7 | 8 | for model_depth in [1, 2]: 9 | tf_random_seed() 10 | join_list = [] 11 | placeholder = x = Input((32, 32, 3), name='data') 12 | for j in range(model_depth): 13 | x = Conv2D(8, (3, 3), padding='same')(x) 14 | x = BatchNormalization()(x) 15 | x = ReLU()(x) 16 | join_list.append(x) 17 | x = MaxPooling2D((2,2))(x) 18 | 19 | join_list = list(reversed(join_list)) 20 | for j in range(model_depth): 21 | x = Conv2D(8, (3,3), padding='same')(x) 22 | x = BatchNormalization()(x) 23 | x = ReLU()(x) 24 | x = UpSampling2D((2,2), interpolation='bilinear')(x) 25 | x = Concatenate(axis=-1)([x, join_list[j]]) 26 | 27 | x = Conv2D(1, (3, 3), padding='same')(x) 28 | x = Activation('sigmoid')(x) 29 | 30 | model = Model(placeholder, x, name=f'simple_model_unet_{str(model_depth).zfill(3)}') 31 | model_list.append(model) 32 | 33 | filter_count_list = [12, 16, 32, 64, 128, 128 + 64, 256] 34 | kernel_size_list = [(3, 3)] * (len(filter_count_list) - 1) + [(4, 4)] 35 | iter_count_list = [2] * 7 36 | in_layer = c_layer = Input((256, 256, 3), name='data') 37 | upsamplig_type = 'nearest' # 'bilinear' 38 | 39 | join_layer_list = [] 40 | for i in range(len(filter_count_list) - 1): 41 | for j in range(iter_count_list[i]): 42 | c_layer = Conv2D(filter_count_list[i], kernel_size_list[i], padding='same')(c_layer) 43 | # c_layer = BatchNormalization()(c_layer) 44 | # c_layer = ReLU()(c_layer) 45 | c_layer = LeakyReLU(0.1)(c_layer) 46 | join_layer_list.append(c_layer) 47 | c_layer = MaxPooling2D((2, 2))(c_layer) 48 | # print(c_layer, kernel_size_list[i]) 49 | 50 | join_layer_list = list(reversed(join_layer_list)) 51 | filter_count_list = list(reversed(filter_count_list[:-1])) 52 | kernel_size_list = list(reversed(kernel_size_list[:-1])) 53 | for i in range(len(filter_count_list)): 54 | c_layer = UpSampling2D((2, 2), interpolation=upsamplig_type)(c_layer) 55 | c_layer = Concatenate(axis=-1)([c_layer, join_layer_list[i]]) 56 | for j in range(iter_count_list[i]): 57 | c_layer = Conv2D(filter_count_list[i], kernel_size_list[i], padding='same')(c_layer) 58 | # c_layer = BatchNormalization()(c_layer) 59 | # c_layer = ReLU()(c_layer) 60 | c_layer = LeakyReLU(0.1)(c_layer) 61 | # print(c_layer, kernel_size_list[i]) 62 | 63 | c_layer = Conv2D(1, (5, 5), padding='same')(c_layer) 64 | # c_layer = BatchNormalization()(c_layer) 65 | c_layer = Activation('sigmoid')(c_layer) 66 | 67 | model = Model(in_layer, c_layer, name='selfie_photo_segmentation') 68 | model_list.append(model) -------------------------------------------------------------------------------- /unit_test/simple_model/Unstructured.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.layers import Input, Conv2D, ReLU, Concatenate 2 | from tensorflow.keras.models import Model 3 | from unit_test.helper import tf_random_seed 4 | 5 | model_list = [] 6 | 7 | tf_random_seed() 8 | # x_in = Input((32, 32, 4)) 9 | # x_0 = Conv2D(1, (3, 3))(x_in) 10 | # x_1 = ReLU()(x_0) 11 | # x_3 = Conv2D(4, (1, 1))(x_0) 12 | # x_4 = Concatenate()([x_0, x_1]) 13 | # x_5 = Concatenate(axis=-1)([x_0, x_4]) 14 | # model = Model(x_in, x_5) 15 | x_in = Input((32, 32, 3)) 16 | x_0 = Conv2D(1, (3, 3))(x_in) 17 | s_plit_0 = (x_0) 18 | x_1 = ReLU()(s_plit_0) 19 | s_plit_1 = (x_1) 20 | 21 | x_3 = Conv2D(4, (1,1))(s_plit_0) 22 | x_4 = Concatenate(axis=-1)([s_plit_1, x_3]) 23 | x_5 = Concatenate(axis=-1)([s_plit_0, s_plit_1, x_4]) 24 | 25 | model = Model(x_in, x_5) 26 | 27 | model_list.append(model) -------------------------------------------------------------------------------- /unit_test/single_layer/Activation.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Activation, ReLU, LeakyReLU 3 | from unit_test.helper import tf_random_seed 4 | 5 | model_list = [] 6 | for activation_item in ['relu', 'softmax', 'sigmoid']: 7 | tf_random_seed() 8 | placeholder = Input((32, 32, 3), name='data') 9 | x = Activation(activation_item)(placeholder) 10 | model = Model(placeholder, x, name=f'model_single_layer_{activation_item}') 11 | model_list.append(model) 12 | 13 | for op_item, name_item in zip([ReLU(), LeakyReLU(0.1), LeakyReLU(0.25)], 14 | ['ReLU_0.0', 'ReLU_0.1', 'ReLU_0.25']): 15 | tf_random_seed() 16 | placeholder = Input((32, 32, 3), name='data') 17 | x = op_item(placeholder) 18 | model = Model(placeholder, x, name=f'model_single_layer_{op_item.__class__.__name__.lower()}_{name_item}') 19 | model_list.append(model) 20 | -------------------------------------------------------------------------------- /unit_test/single_layer/BinaryOp.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Subtract, DepthwiseConv2D 3 | from extra_layers.BinaryOp import Div 4 | from unit_test.helper import tf_random_seed, np_random_seed 5 | import numpy as np 6 | 7 | model_list = [] 8 | for op_item in [Subtract(), Div()]: 9 | epsilon = 1.e-5 10 | tf_random_seed() 11 | np_random_seed() 12 | placeholder = Input((32, 32, 3), name='data') 13 | x = placeholder 14 | y = DepthwiseConv2D((1, 1), weights=[np.random.uniform(+1, +2, size=(1, 1, int(x.shape[-1]), 1)), 15 | np.full((int(x.shape[-1]),), epsilon)])(x) 16 | x = op_item([x, y]) 17 | model = Model(placeholder, x, name=f'model_single_layer_{op_item.__class__.__name__.lower()}') 18 | model_list.append(model) 19 | -------------------------------------------------------------------------------- /unit_test/single_layer/Conv2D.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Conv2D 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | for layer_item, kernel_size in zip([Conv2D]*4, [(1, 1), (3, 3), (5, 5)]): 7 | tf_random_seed() 8 | placeholder = Input((32, 32, 3), name='data') 9 | x = layer_item(4, kernel_size)(placeholder) 10 | model = Model(placeholder, x, 11 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(kernel_size))}') 12 | model_list.append(model) -------------------------------------------------------------------------------- /unit_test/single_layer/Conv2DTranspose.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Conv2DTranspose 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | for layer_item, kernel_size in zip([Conv2DTranspose]*3, [(3, 3), (5, 5), (7, 7)]): 7 | tf_random_seed() 8 | placeholder = Input((32, 32, 3), name='data') 9 | x = layer_item(4, kernel_size, strides=(2, 2), padding='same')(placeholder) 10 | model = Model(placeholder, x, 11 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(kernel_size))}') 12 | model_list.append(model) 13 | 14 | for layer_item, kernel_size in zip([Conv2DTranspose] * 3, [(2, 2), (4, 4), (6, 6)]): 15 | tf_random_seed() 16 | placeholder = Input((32, 32, 3), name='data') 17 | x = layer_item(4, kernel_size, strides=(2, 2), padding='same')(placeholder) 18 | model = Model(placeholder, x, 19 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(kernel_size))}') 20 | model_list.append(model) -------------------------------------------------------------------------------- /unit_test/single_layer/Dense.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Reshape, Flatten, Dense 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | 7 | for layer_item, units in zip([Dense]*3, [128, 64, 32]): 8 | tf_random_seed() 9 | placeholder = Input((32, 32, 3), name='data') 10 | x = Flatten()(placeholder) 11 | x = Dense(units, activation='relu')(x) 12 | model = Model(placeholder, x, 13 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(units))}') 14 | model_list.append(model) 15 | -------------------------------------------------------------------------------- /unit_test/single_layer/DepthwiseConv2D.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, DepthwiseConv2D 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | for layer_item, kernel_size in zip([DepthwiseConv2D]*4, [(1, 1), (3, 3), (5, 5)]): 7 | tf_random_seed() 8 | placeholder = Input((32, 32, 3), name='data') 9 | x = layer_item(kernel_size)(placeholder) 10 | model = Model(placeholder, x, 11 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(kernel_size))}') 12 | model_list.append(model) -------------------------------------------------------------------------------- /unit_test/single_layer/Merge.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Activation, Concatenate, Add, Multiply 3 | from unit_test.helper import tf_random_seed 4 | 5 | model_list = [] 6 | for activation_item in ['relu', 'sigmoid', 'softmax']: 7 | for merge_item in [Concatenate(axis=-1), Add(), Multiply()]: 8 | tf_random_seed() 9 | placeholder = Input((32, 32, 3), name='data') 10 | x_0 = Activation(activation_item)(placeholder) 11 | x_1 = Activation(activation_item)(placeholder) 12 | x = merge_item([x_0, x_1]) 13 | model = Model(placeholder, x, 14 | name=f'model_single_layer_{activation_item}_{merge_item.__class__.__name__.lower()}') 15 | model_list.append(model) 16 | -------------------------------------------------------------------------------- /unit_test/single_layer/Normalization.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, BatchNormalization 3 | from unit_test.helper import tf_random_seed 4 | import numpy as np 5 | 6 | model_list = [] 7 | for training_flag in [False, True]: 8 | for layer_item, _ in zip([BatchNormalization], [None]): 9 | tf_random_seed() 10 | np.random.seed(7) 11 | placeholder = Input((32, 32, 3), name='data') 12 | x = layer_item()(placeholder, training_flag) 13 | model = Model(placeholder, x, 14 | name=f'model_single_layer_{str(training_flag)}_{layer_item.__name__.lower()}') 15 | 16 | model.layers[1].set_weights([np.random.uniform(0, 1, size=3), 17 | np.random.uniform(0, 1, size=3), 18 | np.random.uniform(0, 1, size=3), 19 | np.random.uniform(0, 1, size=3)]) 20 | 21 | model_list.append(model) 22 | -------------------------------------------------------------------------------- /unit_test/single_layer/Pooling2D.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, MaxPooling2D, AveragePooling2D, AvgPool2D, MaxPool2D 3 | from tensorflow.keras.layers import GlobalAveragePooling2D, GlobalAvgPool2D, GlobalMaxPooling2D, GlobalMaxPool2D 4 | from unit_test.helper import tf_random_seed, clean_name 5 | 6 | model_list = [] 7 | for layer_item, pool_item in zip([MaxPooling2D, AveragePooling2D, MaxPool2D, AvgPool2D], 8 | [2, (2, 2), 4, (4, 4)]): 9 | tf_random_seed() 10 | placeholder = Input((32,32,3), name='data') 11 | op_item = layer_item(pool_item) 12 | x = op_item(placeholder) 13 | model = Model(placeholder, x, 14 | name=f'model_single_layer_{op_item.__class__.__name__.lower()}_{clean_name(str(pool_item))}') 15 | model_list.append(model) 16 | 17 | for layer_item in [GlobalMaxPooling2D, GlobalAveragePooling2D, GlobalMaxPool2D, GlobalAvgPool2D]: 18 | tf_random_seed() 19 | placeholder = Input((32,32,3), name='data') 20 | op_item = layer_item() 21 | x = op_item(placeholder) 22 | model = Model(placeholder, x, 23 | name=f'model_single_layer_{op_item.__class__.__name__.lower()}') 24 | model_list.append(model) 25 | -------------------------------------------------------------------------------- /unit_test/single_layer/ReshapeFlatten.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Reshape, Flatten 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | for layer_item, target_shape in zip([Reshape] * 5, 7 | [(-1, 3), (-1, 64, 3), (16, 64, 3), (32 * 32, 3), (-1, )]): 8 | tf_random_seed() 9 | placeholder = Input((32, 32, 3), name='data') 10 | x = layer_item(target_shape)(placeholder) 11 | model = Model(placeholder, x, 12 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(target_shape))}') 13 | model_list.append(model) 14 | 15 | for layer_item in [Flatten]: 16 | tf_random_seed() 17 | placeholder = Input((32, 32, 3), name='data') 18 | x = layer_item()(placeholder) 19 | model = Model(placeholder, x, 20 | name=f'model_single_layer_{layer_item.__name__.lower()}') 21 | model_list.append(model) 22 | -------------------------------------------------------------------------------- /unit_test/single_layer/UnaryOp.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, Multiply 3 | from extra_layers.UnaryOp import Sqrt 4 | from unit_test.helper import tf_random_seed 5 | 6 | model_list = [] 7 | for op_item in [Sqrt()]: 8 | tf_random_seed() 9 | placeholder = Input((32, 32, 3), name='data') 10 | x = Multiply()([placeholder, placeholder]) 11 | x = op_item(x) 12 | model = Model(placeholder, x, name=f'model_single_layer_{op_item.__class__.__name__.lower()}') 13 | model_list.append(model) 14 | -------------------------------------------------------------------------------- /unit_test/single_layer/UpSampling2D.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.models import Model 2 | from tensorflow.keras.layers import Input, UpSampling2D 3 | from unit_test.helper import tf_random_seed, clean_name 4 | 5 | model_list = [] 6 | for layer_item, size_item, interpolation in zip([UpSampling2D]*4, 7 | [(2, 2), (4, 4), (2, 2), (4, 4)], 8 | ['nearest', 'nearest', 'bilinear', 'bilinear']): 9 | tf_random_seed() 10 | placeholder = Input((32, 32, 3), name='data') 11 | x = layer_item(size_item, interpolation=interpolation)(placeholder) 12 | model = Model(placeholder, x, 13 | name=f'model_single_layer_{layer_item.__name__.lower()}_{clean_name(str(size_item))}_{interpolation}') 14 | model_list.append(model) 15 | 16 | -------------------------------------------------------------------------------- /unit_test/unit_test_data/person_001_1024x1204.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/unit_test/unit_test_data/person_001_1024x1204.jpg -------------------------------------------------------------------------------- /unit_test/unit_test_data/person_1024x1204.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/unit_test/unit_test_data/person_1024x1204.jpg -------------------------------------------------------------------------------- /unit_test/unit_test_data/person_640x640.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/unit_test/unit_test_data/person_640x640.jpg -------------------------------------------------------------------------------- /unit_test/unit_test_data/persons_000.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/azeme1/keras2ncnn/c708f4347e8a7895e4cdf120900bacd361e36128/unit_test/unit_test_data/persons_000.jpg --------------------------------------------------------------------------------