├── .gitignore ├── CHANGES.txt ├── Dockerfile ├── LICENSE ├── README.md ├── pixel_decoder ├── __init__.py ├── inception_back.py ├── inception_unet.py ├── linknet_back.py ├── linknet_unet.py ├── loss.py ├── main.py ├── predict.py ├── resnet_back.py ├── resnet_unet.py ├── train.py ├── utils.py └── version.py ├── requirements-dev.txt ├── requirements.txt ├── result_showcase ├── TZ_road_out.png ├── TZ_road_out1.png ├── TZ_road_out3.png └── tz_road_prediction.geojson ├── setup.py └── test └── test_main.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *~ 3 | *.egg-info 4 | tiles/ 5 | labels/ 6 | *.eggs 7 | MANIFEST 8 | .DS_Store 9 | .coverage 10 | .cache 11 | data 12 | /integration* 13 | .idea/ 14 | .ipynb_checkpoints/ 15 | trained_models_out/ 16 | predictions/ 17 | -------------------------------------------------------------------------------- /CHANGES.txt: -------------------------------------------------------------------------------- 1 | 0.1.0: 2 | - initial release 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM nvidia/cuda:9.0-cudnn7-runtime-ubuntu16.04 2 | 3 | # Use a fixed apt-get repo to stop intermittent failures due to flaky httpredir connections, 4 | # as described by Lionel Chan at http://stackoverflow.com/a/37426929/5881346 5 | RUN sed -i "s/httpredir.debian.org/debian.uchicago.edu/" /etc/apt/sources.list && \ 6 | apt-get update && apt-get install -y --no-install-recommends apt-utils && \ 7 | apt-get update && apt-get install -y build-essential && \ 8 | apt-get update --fix-missing && apt-get install -y wget bzip2 ca-certificates \ 9 | libglib2.0-0 libxext6 libsm6 libxrender1 \ 10 | git mercurial subversion zip unzip 11 | 12 | # install Anaconda3 13 | ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 14 | 15 | RUN echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh && \ 16 | wget --quiet https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh -O ~/anaconda.sh && \ 17 | /bin/bash ~/anaconda.sh -b -p /opt/conda && \ 18 | rm ~/anaconda.sh && \ 19 | apt-get install -y curl grep sed dpkg && \ 20 | TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o "/v.*\"" | sed 's:^..\(.*\).$:\1:'` && \ 21 | curl -L "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb" > tini.deb && \ 22 | dpkg -i tini.deb && \ 23 | rm tini.deb && \ 24 | apt-get clean 25 | 26 | ENV PATH /opt/conda/bin:$PATH 27 | 28 | RUN apt-get update && apt-get install -y libglu1 vim && pip install Cython tensorflow==1.8.1 keras==2.2.1 \ 29 | h5py matplotlib pandas pylint scikit-image scikit-learn scipy seaborn Shapely tqdm opencv-python 30 | 31 | 32 | WORKDIR /work 33 | 34 | # copy entire directory where docker file is into docker container at /work 35 | COPY . /work/ 36 | 37 | RUN pip install -e . 38 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2018 Development Seed 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Pixel Decoder 2 | 3 | ![computervision](https://user-images.githubusercontent.com/14057932/37719364-3e953da0-2cfb-11e8-8140-f5f12bb806d9.png) 4 | In **computer vision**, there are three challenges: image classification, object detection and **semantic segmentation**. As you see above, semantic segmentation can segment an image into different parts and objects (e.g.grass, cat, tree, sky). 5 | 6 | Pixel Decoder is a tool that contains several current available semantic segmentation algorithms. **Pixel Decoder** includes Standard Unet and its modified versions, Tiramisu and SegNet. SegNet is the algorithm that Skynet was built on. All the algorithms that live inside Pixel Decoder are convolutional neural networks are all in a structure that called encoder-decoder. 7 | ![encoder-decoder](https://user-images.githubusercontent.com/14057932/37719742-14b23582-2cfc-11e8-8242-a3773df31bc2.png) 8 | The encoder reads in the image pixels and compresses the information in vector, downsample to save computing memory; and the decoder works on reconstructing the pixels spatial information and output the desired outcome. Some UNet-like algorithms were adopted from SpaceNet challenge solutions. 9 | 10 | All these algorithms are built with Tensorflow and Keras. These are some results for road segmentation from Pixel Decoder we got. 11 |

12 | 13 |

14 | 15 |

16 | 17 |

18 | 19 |

20 | 21 |

22 | ### Installation 23 | 24 | ```bash 25 | git clone https://github.com/Geoyi/pixel-decoder 26 | cd pixel-decoder 27 | pip install -e . 28 | ``` 29 | 30 | ### Train 31 | 32 | ```bash 33 | pixel_decoder train --batch_size=4 \ 34 | --imgs_folder=tiles \ 35 | --masks_folder=labels \ 36 | --models_folder=trained_models_out \ 37 | --model_id=resnet_unet \ 38 | --origin_shape_no=256 \ 39 | --border_no=32 40 | ``` 41 | It takes in the training dataset that created from [`Label Maker`](https://github.com/developmentseed/label-maker). 42 | 43 | 44 | - `batch_size`: batch size for the training; 45 | - `imgs_folder`: is the directory for RGB images to train; 46 | - `masks_folder`: is the directory for labeled mask to train; 47 | - `model_id`: is the neural net architecture to train with. We have - `resnet_unet`, `inception_unet`, `linknet_unet`, `SegNet`, `Tiramisu` as model_id live in **Pixel Decoder**. 48 | - `origin_shape_no`: 256 is the default image tile shape from [Label Maker](https://github.com/developmentseed/label-maker); 49 | - `border_no`: it's set to 32. It's a additional 32 pixel to add on 256 by 256 image tile to become 320 by 320 to get rid of U-Net's edge distortion. 50 | 51 | 52 | ### Predict 53 | After the model is trained and you see a trained model weight in your model directory, run: 54 | 55 | 56 | ```bash 57 | pixel_decoder predict --imgs_folder=tiles \ 58 | --test_folder=test_images \ 59 | --models_folder=trained_models_out \ 60 | --pred_folder=predictions \ 61 | --model_id=resnet_unet \ 62 | --origin_shape_no=256 \ 63 | --border_no=32 64 | ``` 65 | 66 | - `imgs_folder`: is the directory for RGB images to train; 67 | - `masks_folder`: is the directory for labeled mask to train. It uses to get the stats, e.g. mean and standard deviation, from training images. 68 | - `test_folder`: is the directory for test images. 69 | - `pred_folder`: a directory that saved all the predicted test image from test_folder; 70 | - `model_id`: is the neural net architecture to train with. We have - `resnet_unet`, `inception_unet`, `linknet_unet`, `SegNet`, `Tiramisu` as model_id live in **Pixel Decoder**. 71 | - `origin_shape_no`: 256 is the default image tile shape from [Label Maker](https://github.com/developmentseed/label-maker); 72 | - `border_no`: it's set to 32. It's a additional 32 pixel to add on 256 by 256 image tile to become 320 by 320 to get rid of U-Net's edge distortion. 73 | 74 | ## Run Pixel Decoder on AWS Deep Learning AMI instance with **GUPs** 75 | 76 | ### Install Nvidia-Docker on your instance 77 | - [Docker installation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html) on AWS EC2. Instruction for Nvidia Docker installation [here](https://towardsdatascience.com/using-docker-to-set-up-a-deep-learning-environment-on-aws-6af37a78c551). 78 | 79 | - Build provide docker image from the Dockerfile 80 | 81 | ```bash 82 | git clone https://github.com/Geoyi/pixel-decoder 83 | cd pixel-decoder 84 | nvidia-docker build -t pixel_decoder . 85 | ``` 86 | 87 | - Run nvidia-docker and Pixel Decoder 88 | 89 | ```bash 90 | nvidia-docker run -v $PWD:/work -it pixel_decoder bash 91 | ``` 92 | 93 | - Install Pixel Decoder and train the model 94 | 95 | **Train** 96 | ```bash 97 | pixel_decoder train --batch_size=4 \ 98 | --imgs_folder=tiles \ 99 | --masks_folder=labels \ 100 | --models_folder=trained_models_out \ 101 | --model_id=resnet_unet \ 102 | --origin_shape_no=256 \ 103 | --border_no=32 104 | ``` 105 | 106 | **Predict** 107 | 108 | ```bash 109 | pixel_decoder predict --imgs_folder=tiles \ 110 | --test_folder=test_images \ 111 | --models_folder=trained_models_out \ 112 | --pred_folder=predictions \ 113 | --model_id=resnet_unet \ 114 | --origin_shape_no=256 \ 115 | --border_no=32 116 | ``` 117 | 118 | ## About 119 | To run a neural net, e.g `resnet_unet`, you can create ready-to-train dataset from [Label Maker](https://github.com/developmentseed/label-maker). A detail walkthrough notebook will come soon. 120 | pixel_decoder was built on top of python-seed that created by [Development Seed](). 121 | -------------------------------------------------------------------------------- /pixel_decoder/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/pixel_decoder/__init__.py -------------------------------------------------------------------------------- /pixel_decoder/inception_back.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """Inception-ResNet V2 model for Keras. 3 | 4 | Model naming and structure follows TF-slim implementation (which has some additional 5 | layers and different number of filters from the original arXiv paper): 6 | https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_resnet_v2.py 7 | 8 | Pre-trained ImageNet weights are also converted from TF-slim, which can be found in: 9 | https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models 10 | 11 | # Reference 12 | - [Inception-v4, Inception-ResNet and the Impact of 13 | Residual Connections on Learning](https://arxiv.org/abs/1602.07261) 14 | 15 | """ 16 | from __future__ import absolute_import 17 | from __future__ import division 18 | from __future__ import print_function 19 | 20 | import os 21 | import warnings 22 | 23 | from keras.models import Model 24 | from keras.layers import Activation 25 | from keras.layers import AveragePooling2D 26 | from keras.layers import BatchNormalization 27 | from keras.layers import Concatenate 28 | from keras.layers import Conv2D 29 | from keras.layers import Dense 30 | from keras.layers import GlobalAveragePooling2D 31 | from keras.layers import GlobalMaxPooling2D 32 | from keras.layers import Input 33 | from keras.layers import Lambda 34 | from keras.layers import MaxPooling2D 35 | from keras.utils.data_utils import get_file 36 | from keras.engine.topology import get_source_inputs 37 | from keras.applications import imagenet_utils 38 | from keras.applications.imagenet_utils import _obtain_input_shape 39 | from keras.applications.imagenet_utils import decode_predictions 40 | from keras import backend as K 41 | 42 | 43 | BASE_WEIGHT_URL = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.7/' 44 | 45 | 46 | def preprocess_input(x): 47 | """Preprocesses a numpy array encoding a batch of images. 48 | 49 | # Arguments 50 | x: a 4D numpy array consists of RGB values within [0, 255]. 51 | 52 | # Returns 53 | Preprocessed array. 54 | """ 55 | return imagenet_utils.preprocess_input(x, mode='tf') 56 | 57 | 58 | def conv2d_bn(x, 59 | filters, 60 | kernel_size, 61 | strides=1, 62 | padding='same', 63 | activation='relu', 64 | use_bias=False, 65 | name=None): 66 | """Utility function to apply conv + BN. 67 | 68 | # Arguments 69 | x: input tensor. 70 | filters: filters in `Conv2D`. 71 | kernel_size: kernel size as in `Conv2D`. 72 | strides: strides in `Conv2D`. 73 | padding: padding mode in `Conv2D`. 74 | activation: activation in `Conv2D`. 75 | use_bias: whether to use a bias in `Conv2D`. 76 | name: name of the ops; will become `name + '_ac'` for the activation 77 | and `name + '_bn'` for the batch norm layer. 78 | 79 | # Returns 80 | Output tensor after applying `Conv2D` and `BatchNormalization`. 81 | """ 82 | x = Conv2D(filters, 83 | kernel_size, 84 | strides=strides, 85 | padding=padding, 86 | use_bias=use_bias, 87 | name=name)(x) 88 | if not use_bias: 89 | bn_axis = 1 if K.image_data_format() == 'channels_first' else 3 90 | bn_name = None if name is None else name + '_bn' 91 | x = BatchNormalization(axis=bn_axis, scale=False, name=bn_name)(x) 92 | if activation is not None: 93 | ac_name = None if name is None else name + '_ac' 94 | x = Activation(activation, name=ac_name)(x) 95 | return x 96 | 97 | 98 | def inception_resnet_block(x, scale, block_type, block_idx, activation='relu'): 99 | """Adds a Inception-ResNet block. 100 | 101 | This function builds 3 types of Inception-ResNet blocks mentioned 102 | in the paper, controlled by the `block_type` argument (which is the 103 | block name used in the official TF-slim implementation): 104 | - Inception-ResNet-A: `block_type='block35'` 105 | - Inception-ResNet-B: `block_type='block17'` 106 | - Inception-ResNet-C: `block_type='block8'` 107 | 108 | # Arguments 109 | x: input tensor. 110 | scale: scaling factor to scale the residuals (i.e., the output of 111 | passing `x` through an inception module) before adding them 112 | to the shortcut branch. Let `r` be the output from the residual branch, 113 | the output of this block will be `x + scale * r`. 114 | block_type: `'block35'`, `'block17'` or `'block8'`, determines 115 | the network structure in the residual branch. 116 | block_idx: an `int` used for generating layer names. The Inception-ResNet blocks 117 | are repeated many times in this network. We use `block_idx` to identify 118 | each of the repetitions. For example, the first Inception-ResNet-A block 119 | will have `block_type='block35', block_idx=0`, ane the layer names will have 120 | a common prefix `'block35_0'`. 121 | activation: activation function to use at the end of the block 122 | (see [activations](../activations.md)). 123 | When `activation=None`, no activation is applied 124 | (i.e., "linear" activation: `a(x) = x`). 125 | 126 | # Returns 127 | Output tensor for the block. 128 | 129 | # Raises 130 | ValueError: if `block_type` is not one of `'block35'`, 131 | `'block17'` or `'block8'`. 132 | """ 133 | if block_type == 'block35': 134 | branch_0 = conv2d_bn(x, 32, 1) 135 | branch_1 = conv2d_bn(x, 32, 1) 136 | branch_1 = conv2d_bn(branch_1, 32, 3) 137 | branch_2 = conv2d_bn(x, 32, 1) 138 | branch_2 = conv2d_bn(branch_2, 48, 3) 139 | branch_2 = conv2d_bn(branch_2, 64, 3) 140 | branches = [branch_0, branch_1, branch_2] 141 | elif block_type == 'block17': 142 | branch_0 = conv2d_bn(x, 192, 1) 143 | branch_1 = conv2d_bn(x, 128, 1) 144 | branch_1 = conv2d_bn(branch_1, 160, [1, 7]) 145 | branch_1 = conv2d_bn(branch_1, 192, [7, 1]) 146 | branches = [branch_0, branch_1] 147 | elif block_type == 'block8': 148 | branch_0 = conv2d_bn(x, 192, 1) 149 | branch_1 = conv2d_bn(x, 192, 1) 150 | branch_1 = conv2d_bn(branch_1, 224, [1, 3]) 151 | branch_1 = conv2d_bn(branch_1, 256, [3, 1]) 152 | branches = [branch_0, branch_1] 153 | else: 154 | raise ValueError('Unknown Inception-ResNet block type. ' 155 | 'Expects "block35", "block17" or "block8", ' 156 | 'but got: ' + str(block_type)) 157 | 158 | block_name = block_type + '_' + str(block_idx) 159 | channel_axis = 1 if K.image_data_format() == 'channels_first' else 3 160 | mixed = Concatenate(axis=channel_axis, name=block_name + '_mixed')(branches) 161 | up = conv2d_bn(mixed, 162 | K.int_shape(x)[channel_axis], 163 | 1, 164 | activation=None, 165 | use_bias=True, 166 | name=block_name + '_conv') 167 | 168 | x = Lambda(lambda inputs, scale: inputs[0] + inputs[1] * scale, 169 | output_shape=K.int_shape(x)[1:], 170 | arguments={'scale': scale}, 171 | name=block_name)([x, up]) 172 | if activation is not None: 173 | x = Activation(activation, name=block_name + '_ac')(x) 174 | return x 175 | 176 | 177 | def InceptionResNetV2(include_top=True, 178 | weights='imagenet', 179 | input_tensor=None, 180 | input_shape=None, 181 | pooling=None, 182 | classes=1000): 183 | """Instantiates the Inception-ResNet v2 architecture. 184 | 185 | Optionally loads weights pre-trained on ImageNet. 186 | Note that when using TensorFlow, for best performance you should 187 | set `"image_data_format": "channels_last"` in your Keras config 188 | at `~/.keras/keras.json`. 189 | 190 | The model and the weights are compatible with TensorFlow, Theano and 191 | CNTK backends. The data format convention used by the model is 192 | the one specified in your Keras config file. 193 | 194 | Note that the default input image size for this model is 299x299, instead 195 | of 224x224 as in the VGG16 and ResNet models. Also, the input preprocessing 196 | function is different (i.e., do not use `imagenet_utils.preprocess_input()` 197 | with this model. Use `preprocess_input()` defined in this module instead). 198 | 199 | # Arguments 200 | include_top: whether to include the fully-connected 201 | layer at the top of the network. 202 | weights: one of `None` (random initialization), 203 | 'imagenet' (pre-training on ImageNet), 204 | or the path to the weights file to be loaded. 205 | input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) 206 | to use as image input for the model. 207 | input_shape: optional shape tuple, only to be specified 208 | if `include_top` is `False` (otherwise the input shape 209 | has to be `(299, 299, 3)` (with `'channels_last'` data format) 210 | or `(3, 299, 299)` (with `'channels_first'` data format). 211 | It should have exactly 3 inputs channels, 212 | and width and height should be no smaller than 139. 213 | E.g. `(150, 150, 3)` would be one valid value. 214 | pooling: Optional pooling mode for feature extraction 215 | when `include_top` is `False`. 216 | - `None` means that the output of the model will be 217 | the 4D tensor output of the last convolutional layer. 218 | - `'avg'` means that global average pooling 219 | will be applied to the output of the 220 | last convolutional layer, and thus 221 | the output of the model will be a 2D tensor. 222 | - `'max'` means that global max pooling will be applied. 223 | classes: optional number of classes to classify images 224 | into, only to be specified if `include_top` is `True`, and 225 | if no `weights` argument is specified. 226 | 227 | # Returns 228 | A Keras `Model` instance. 229 | 230 | # Raises 231 | ValueError: in case of invalid argument for `weights`, 232 | or invalid input shape. 233 | """ 234 | if not (weights in {'imagenet', None} or os.path.exists(weights)): 235 | raise ValueError('The `weights` argument should be either ' 236 | '`None` (random initialization), `imagenet` ' 237 | '(pre-training on ImageNet), ' 238 | 'or the path to the weights file to be loaded.') 239 | 240 | if weights == 'imagenet' and include_top and classes != 1000: 241 | raise ValueError('If using `weights` as imagenet with `include_top`' 242 | ' as true, `classes` should be 1000') 243 | 244 | # Determine proper input shape 245 | input_shape = _obtain_input_shape( 246 | input_shape, 247 | default_size=299, 248 | min_size=139, 249 | data_format=K.image_data_format(), 250 | require_flatten=False, 251 | weights=weights) 252 | 253 | if input_tensor is None: 254 | img_input = Input(shape=input_shape) 255 | else: 256 | if not K.is_keras_tensor(input_tensor): 257 | img_input = Input(tensor=input_tensor, shape=input_shape) 258 | else: 259 | img_input = input_tensor 260 | 261 | # Stem block: 35 x 35 x 192 262 | x = conv2d_bn(img_input, 32, 3, strides=2, padding='same') 263 | x = conv2d_bn(x, 32, 3, padding='same') 264 | x = conv2d_bn(x, 64, 3) 265 | x = MaxPooling2D(3, strides=2, padding='same')(x) 266 | x = conv2d_bn(x, 80, 1, padding='same') 267 | x = conv2d_bn(x, 192, 3, padding='same') 268 | x = MaxPooling2D(3, strides=2, padding='same')(x) 269 | 270 | # Mixed 5b (Inception-A block): 35 x 35 x 320 271 | branch_0 = conv2d_bn(x, 96, 1) 272 | branch_1 = conv2d_bn(x, 48, 1) 273 | branch_1 = conv2d_bn(branch_1, 64, 5) 274 | branch_2 = conv2d_bn(x, 64, 1) 275 | branch_2 = conv2d_bn(branch_2, 96, 3) 276 | branch_2 = conv2d_bn(branch_2, 96, 3) 277 | branch_pool = AveragePooling2D(3, strides=1, padding='same')(x) 278 | branch_pool = conv2d_bn(branch_pool, 64, 1) 279 | branches = [branch_0, branch_1, branch_2, branch_pool] 280 | channel_axis = 1 if K.image_data_format() == 'channels_first' else 3 281 | x = Concatenate(axis=channel_axis, name='mixed_5b')(branches) 282 | 283 | # 10x block35 (Inception-ResNet-A block): 35 x 35 x 320 284 | for block_idx in range(1, 11): 285 | x = inception_resnet_block(x, 286 | scale=0.17, 287 | block_type='block35', 288 | block_idx=block_idx) 289 | 290 | # Mixed 6a (Reduction-A block): 17 x 17 x 1088 291 | branch_0 = conv2d_bn(x, 384, 3, strides=2, padding='same') 292 | branch_1 = conv2d_bn(x, 256, 1) 293 | branch_1 = conv2d_bn(branch_1, 256, 3) 294 | branch_1 = conv2d_bn(branch_1, 384, 3, strides=2, padding='same') 295 | branch_pool = MaxPooling2D(3, strides=2, padding='same')(x) 296 | branches = [branch_0, branch_1, branch_pool] 297 | x = Concatenate(axis=channel_axis, name='mixed_6a')(branches) 298 | 299 | # 20x block17 (Inception-ResNet-B block): 17 x 17 x 1088 300 | for block_idx in range(1, 21): 301 | x = inception_resnet_block(x, 302 | scale=0.1, 303 | block_type='block17', 304 | block_idx=block_idx) 305 | 306 | # Mixed 7a (Reduction-B block): 8 x 8 x 2080 307 | branch_0 = conv2d_bn(x, 256, 1) 308 | branch_0 = conv2d_bn(branch_0, 384, 3, strides=2, padding='same') 309 | branch_1 = conv2d_bn(x, 256, 1) 310 | branch_1 = conv2d_bn(branch_1, 288, 3, strides=2, padding='same') 311 | branch_2 = conv2d_bn(x, 256, 1) 312 | branch_2 = conv2d_bn(branch_2, 288, 3) 313 | branch_2 = conv2d_bn(branch_2, 320, 3, strides=2, padding='same') 314 | branch_pool = MaxPooling2D(3, strides=2, padding='same')(x) 315 | branches = [branch_0, branch_1, branch_2, branch_pool] 316 | x = Concatenate(axis=channel_axis, name='mixed_7a')(branches) 317 | 318 | # 10x block8 (Inception-ResNet-C block): 8 x 8 x 2080 319 | for block_idx in range(1, 10): 320 | x = inception_resnet_block(x, 321 | scale=0.2, 322 | block_type='block8', 323 | block_idx=block_idx) 324 | x = inception_resnet_block(x, 325 | scale=1., 326 | activation=None, 327 | block_type='block8', 328 | block_idx=10) 329 | 330 | # Final convolution block: 8 x 8 x 1536 331 | x = conv2d_bn(x, 1536, 1, name='conv_7b') 332 | 333 | if include_top: 334 | # Classification block 335 | x = GlobalAveragePooling2D(name='avg_pool')(x) 336 | x = Dense(classes, activation='softmax', name='predictions')(x) 337 | else: 338 | if pooling == 'avg': 339 | x = GlobalAveragePooling2D()(x) 340 | elif pooling == 'max': 341 | x = GlobalMaxPooling2D()(x) 342 | 343 | # Ensure that the model takes into account 344 | # any potential predecessors of `input_tensor` 345 | if input_tensor is not None: 346 | inputs = get_source_inputs(input_tensor) 347 | else: 348 | inputs = img_input 349 | 350 | # Create model 351 | model = Model(inputs, x, name='inception_resnet_v2') 352 | 353 | # Load weights 354 | if weights == 'imagenet': 355 | if K.image_data_format() == 'channels_first': 356 | if K.backend() == 'tensorflow': 357 | warnings.warn('You are using the TensorFlow backend, yet you ' 358 | 'are using the Theano ' 359 | 'image data format convention ' 360 | '(`image_data_format="channels_first"`). ' 361 | 'For best performance, set ' 362 | '`image_data_format="channels_last"` in ' 363 | 'your Keras config ' 364 | 'at ~/.keras/keras.json.') 365 | if include_top: 366 | fname = 'inception_resnet_v2_weights_tf_dim_ordering_tf_kernels.h5' 367 | weights_path = get_file(fname, 368 | BASE_WEIGHT_URL + fname, 369 | cache_subdir='models', 370 | file_hash='e693bd0210a403b3192acc6073ad2e96') 371 | else: 372 | fname = 'inception_resnet_v2_weights_tf_dim_ordering_tf_kernels_notop.h5' 373 | weights_path = get_file(fname, 374 | BASE_WEIGHT_URL + fname, 375 | cache_subdir='models', 376 | file_hash='d19885ff4a710c122648d3b5c3b684e4') 377 | model.load_weights(weights_path) 378 | elif weights is not None: 379 | model.load_weights(weights) 380 | 381 | return model 382 | -------------------------------------------------------------------------------- /pixel_decoder/inception_unet.py: -------------------------------------------------------------------------------- 1 | from keras import backend as K 2 | from keras.models import Model 3 | from keras.layers import Input, BatchNormalization, Conv2D, MaxPooling2D, AveragePooling2D, concatenate, Concatenate, UpSampling2D, Activation, SpatialDropout2D, RepeatVector, Reshape 4 | 5 | from keras.losses import binary_crossentropy 6 | from inception_back import InceptionResNetV2, inception_resnet_block, conv2d_bn 7 | 8 | 9 | # channel_axis = bn_axis 10 | 11 | def conv_block(prev, num_filters, kernel=(3, 3), strides=(1, 1), act='relu', prefix=None): 12 | name = None 13 | bn_axis = 3 14 | if prefix is not None: 15 | name = prefix + '_conv' 16 | conv = Conv2D(num_filters, kernel, padding='same', kernel_initializer='he_normal', strides=strides, name=name)(prev) 17 | if prefix is not None: 18 | name = prefix + '_norm' 19 | conv = BatchNormalization(name=name, axis=bn_axis)(conv) 20 | if prefix is not None: 21 | name = prefix + '_act' 22 | conv = Activation(act, name=name)(conv) 23 | return conv 24 | 25 | def get_inception_resnet_v2_unet(input_shape,channel_no, weights='imagenet'): 26 | inp = Input(input_shape + (channel_no,)) 27 | 28 | # Stem block: 35 x 35 x 192 29 | x = conv2d_bn(inp, 32, 3, strides=2, padding='same') 30 | x = conv2d_bn(x, 32, 3, padding='same') 31 | x = conv2d_bn(x, 64, 3) 32 | conv1 = x 33 | x = MaxPooling2D(3, strides=2, padding='same')(x) 34 | x = conv2d_bn(x, 80, 1, padding='same') 35 | x = conv2d_bn(x, 192, 3, padding='same') 36 | conv2 = x 37 | x = MaxPooling2D(3, strides=2, padding='same')(x) 38 | 39 | # Mixed 5b (Inception-A block): 35 x 35 x 320 40 | branch_0 = conv2d_bn(x, 96, 1) 41 | branch_1 = conv2d_bn(x, 48, 1) 42 | branch_1 = conv2d_bn(branch_1, 64, 5) 43 | branch_2 = conv2d_bn(x, 64, 1) 44 | branch_2 = conv2d_bn(branch_2, 96, 3) 45 | branch_2 = conv2d_bn(branch_2, 96, 3) 46 | branch_pool = AveragePooling2D(3, strides=1, padding='same')(x) 47 | branch_pool = conv2d_bn(branch_pool, 64, 1) 48 | branches = [branch_0, branch_1, branch_2, branch_pool] 49 | channel_no = 1 if K.image_data_format() == 'channels_first' else 3 50 | x = Concatenate(axis=channel_no, name='mixed_5b')(branches) 51 | 52 | # 10x block35 (Inception-ResNet-A block): 35 x 35 x 320 53 | for block_idx in range(1, 11): 54 | x = inception_resnet_block(x, 55 | scale=0.17, 56 | block_type='block35', 57 | block_idx=block_idx) 58 | conv3 = x 59 | # Mixed 6a (Reduction-A block): 17 x 17 x 1088 60 | branch_0 = conv2d_bn(x, 384, 3, strides=2, padding='same') 61 | branch_1 = conv2d_bn(x, 256, 1) 62 | branch_1 = conv2d_bn(branch_1, 256, 3) 63 | branch_1 = conv2d_bn(branch_1, 384, 3, strides=2, padding='same') 64 | branch_pool = MaxPooling2D(3, strides=2, padding='same')(x) 65 | branches = [branch_0, branch_1, branch_pool] 66 | x = Concatenate(axis=channel_axis, name='mixed_6a')(branches) 67 | 68 | # 20x block17 (Inception-ResNet-B block): 17 x 17 x 1088 69 | for block_idx in range(1, 21): 70 | x = inception_resnet_block(x, 71 | scale=0.1, 72 | block_type='block17', 73 | block_idx=block_idx) 74 | conv4 = x 75 | # Mixed 7a (Reduction-B block): 8 x 8 x 2080 76 | branch_0 = conv2d_bn(x, 256, 1) 77 | branch_0 = conv2d_bn(branch_0, 384, 3, strides=2, padding='same') 78 | branch_1 = conv2d_bn(x, 256, 1) 79 | branch_1 = conv2d_bn(branch_1, 288, 3, strides=2, padding='same') 80 | branch_2 = conv2d_bn(x, 256, 1) 81 | branch_2 = conv2d_bn(branch_2, 288, 3) 82 | branch_2 = conv2d_bn(branch_2, 320, 3, strides=2, padding='same') 83 | branch_pool = MaxPooling2D(3, strides=2, padding='same')(x) 84 | branches = [branch_0, branch_1, branch_2, branch_pool] 85 | x = Concatenate(axis=channel_axis, name='mixed_7a')(branches) 86 | 87 | # 10x block8 (Inception-ResNet-C block): 8 x 8 x 2080 88 | for block_idx in range(1, 10): 89 | x = inception_resnet_block(x, 90 | scale=0.2, 91 | block_type='block8', 92 | block_idx=block_idx) 93 | x = inception_resnet_block(x, 94 | scale=1., 95 | activation=None, 96 | block_type='block8', 97 | block_idx=10) 98 | 99 | # Final convolution block: 8 x 8 x 1536 100 | x = conv2d_bn(x, 1536, 1, name='conv_7b') 101 | conv5 = x 102 | 103 | up6 = concatenate([UpSampling2D()(conv5), conv4], axis=-1) 104 | conv6 = conv_block(up6, 128) 105 | conv6 = conv_block(conv6, 128) 106 | 107 | up7 = concatenate([UpSampling2D()(conv6), conv3], axis=-1) 108 | conv7 = conv_block(up7, 96) 109 | conv7 = conv_block(conv7, 96) 110 | 111 | up8 = concatenate([UpSampling2D()(conv7), conv2], axis=-1) 112 | conv8 = conv_block(up8, 64) 113 | conv8 = conv_block(conv8, 64) 114 | 115 | up9 = concatenate([UpSampling2D()(conv8), conv1], axis=-1) 116 | conv9 = conv_block(up9, 48) 117 | conv9 = conv_block(conv9, 48) 118 | 119 | up10 = concatenate([UpSampling2D()(conv9), inp], axis=-1) 120 | conv10 = conv_block(up10, 32) 121 | conv10 = conv_block(conv10, 32) 122 | # conv10 = SpatialDropout2D(0.33)(conv10) 123 | res = Conv2D(1, (1, 1), activation='sigmoid')(conv10) 124 | model = Model(inp, res) 125 | 126 | if weights == 'imagenet': 127 | inception_resnet_v2 = InceptionResNetV2(weights=weights, include_top=False, input_shape=input_shape + (3,)) 128 | for i in range(2, len(inception_resnet_v2.layers)-1): 129 | model.layers[i].set_weights(inception_resnet_v2.layers[i].get_weights()) 130 | model.layers[i].trainable = False 131 | 132 | return model 133 | -------------------------------------------------------------------------------- /pixel_decoder/linknet_back.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/pixel_decoder/linknet_back.py -------------------------------------------------------------------------------- /pixel_decoder/linknet_unet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/pixel_decoder/linknet_unet.py -------------------------------------------------------------------------------- /pixel_decoder/loss.py: -------------------------------------------------------------------------------- 1 | 2 | from keras import backend as K 3 | from keras.losses import binary_crossentropy 4 | 5 | 6 | def dice_coef(y_true, y_pred): 7 | y_true_f = K.flatten(y_true) 8 | y_pred_f = K.flatten(y_pred) 9 | intersection = K.sum(y_true_f * y_pred_f) 10 | return (2. * intersection + 1) / (K.sum(y_true_f) + K.sum(y_pred_f) + 1) 11 | 12 | def dice_coef_rounded(y_true, y_pred): 13 | y_true_f = K.flatten(K.round(y_true)) 14 | y_pred_f = K.flatten(K.round(y_pred)) 15 | intersection = K.sum(y_true_f * y_pred_f) 16 | return (2. * intersection + 1) / (K.sum(y_true_f) + K.sum(y_pred_f) + 1) 17 | 18 | def dice_coef_loss(y_true, y_pred): 19 | return 1 - (dice_coef(y_true, y_pred)) 20 | 21 | def dice_logloss(y_true, y_pred): 22 | return binary_crossentropy(y_true, y_pred) * 0.5 + dice_coef_loss(y_true, y_pred) * 0.5 23 | 24 | def dice_logloss2(y_true, y_pred): 25 | return binary_crossentropy(y_true, y_pred) * 0.75 + dice_coef_loss(y_true, y_pred) * 0.25 26 | 27 | def dice_logloss3(y_true, y_pred): 28 | return binary_crossentropy(y_true, y_pred) * 0.15 + dice_coef_loss(y_true, y_pred) * 0.85 29 | 30 | #from https://www.kaggle.com/lyakaap/weighing-boundary-pixels-loss-script-by-keras2 31 | # weight: weighted tensor(same shape with mask image) 32 | def weighted_bce_loss(y_true, y_pred, weight): 33 | # avoiding overflow 34 | epsilon = 1e-7 35 | y_pred = K.clip(y_pred, epsilon, 1. - epsilon) 36 | logit_y_pred = K.log(y_pred / (1. - y_pred)) 37 | 38 | # https://www.tensorflow.org/api_docs/python/tf/nn/weighted_cross_entropy_with_logits 39 | loss = (1. - y_true) * logit_y_pred + (1. + (weight - 1.) * y_true) * \ 40 | (K.log(1. + K.exp(-K.abs(logit_y_pred))) + K.maximum(-logit_y_pred, 0.)) 41 | return K.sum(loss) / K.sum(weight) 42 | 43 | def weighted_dice_loss(y_true, y_pred, weight): 44 | smooth = 1. 45 | w, m1, m2 = weight * weight, y_true, y_pred 46 | intersection = (m1 * m2) 47 | score = (2. * K.sum(w * intersection) + smooth) / (K.sum(w * m1) + K.sum(w * m2) + smooth) 48 | loss = 1. - K.sum(score) 49 | return loss 50 | 51 | def weighted_bce_dice_loss(y_true, y_pred): 52 | y_true = K.cast(y_true, 'float32') 53 | y_pred = K.cast(y_pred, 'float32') 54 | # if we want to get same size of output, kernel size must be odd number 55 | averaged_mask = K.pool2d( 56 | y_true, pool_size=(11, 11), strides=(1, 1), padding='same', pool_mode='avg') 57 | border = K.cast(K.greater(averaged_mask, 0.005), 'float32') * K.cast(K.less(averaged_mask, 0.995), 'float32') 58 | weight = K.ones_like(averaged_mask) 59 | w0 = K.sum(weight) 60 | weight += border * 2 61 | w1 = K.sum(weight) 62 | weight *= (w0 / w1) 63 | loss = weighted_bce_loss(y_true, y_pred, weight) + \ 64 | weighted_dice_loss(y_true, y_pred, weight) 65 | return loss 66 | -------------------------------------------------------------------------------- /pixel_decoder/main.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import argparse 4 | import logging 5 | from os import makedirs, path as op 6 | 7 | 8 | from pixel_decoder.version import __version__ 9 | from pixel_decoder.train import train 10 | from pixel_decoder.predict import predict 11 | 12 | logger = logging.getLogger(__name__) 13 | 14 | 15 | def parse_args(args): 16 | desc = 'pixel_decoder (v%s)' % __version__ 17 | dhf = argparse.ArgumentDefaultsHelpFormatter 18 | parser0 = argparse.ArgumentParser(description=desc) 19 | 20 | pparser = argparse.ArgumentParser(add_help=False) 21 | pparser.add_argument('--version', help='Print version and exit', action='version', version=__version__) 22 | pparser.add_argument('--log', default=2, type=int, 23 | help='0:all, 1:debug, 2:info, 3:warning, 4:error, 5:critical') 24 | 25 | subparsers = parser0.add_subparsers(dest='command') 26 | 27 | parser = subparsers.add_parser('train', parents=[pparser], help='train the model', formatter_class=dhf) 28 | parser.add_argument('-bz', '--batch_size', help='batch size for the training', default=16, type=int, required=True) 29 | parser.add_argument('-imgs', '--imgs_folder', help='directory for RGB images to train', type=str, required=True) 30 | parser.add_argument('-masks', '--masks_folder', help='directory for labeled mask to train', type=str, required=True) 31 | parser.add_argument('-model', '--models_folder', default='trained_models', help='directory for storing output files', type=str, required=True) 32 | parser.add_argument('-mid', '--model_id', help='model id from README', default='resnet_unet', type=str, required=False) 33 | parser.add_argument('-ors', '--origin_shape_no', help='the image shape of the input training images', default=256, type=int, required=False) 34 | parser.add_argument('-border', '--border_no', help='pixels number to add on training image and mask to get rid of edge effects from unet', default=32, type=int, required=False) 35 | 36 | 37 | # train(batch_size, imgs_folder, masks_folder, models_folder, model_id='resnet_unet', origin_shape=(256, 256), border=(32, 32), channel_no = 3) 38 | 39 | parser = subparsers.add_parser('predict', parents=[pparser], help='predict with test data', formatter_class=dhf) 40 | parser.add_argument('-test', '--test_folder', help='directory for RGB images to predict', required=True) 41 | parser.add_argument('-imgs', '--imgs_folder', help='directory for labeled mask to train', type=str, required=True) 42 | parser.add_argument('-model', '--models_folder', default='trained_models', help='directory for storing output files', type=str, required=True) 43 | parser.add_argument('-pred', '--pred_folder', help='directory to save the predicted images', required=True) 44 | parser.add_argument('-mid', '--model_id', help='model id from README', default='resnet_unet',type=str, required=False) 45 | parser.add_argument('-ors', '--origin_shape_no', help='the image shape of the input training images', default=256, type=int, required=False) 46 | parser.add_argument('-border', '--border_no', help='pixels number to add on training image and mask to get rid of edge effects from unet', default=32, type=int, required=False) 47 | parsed_args = vars(parser0.parse_args(args)) 48 | 49 | return parsed_args 50 | 51 | def main(cmd, **kwargs): 52 | if cmd == 'train': 53 | train(**kwargs) 54 | elif cmd == 'predict': 55 | predict(**kwargs) 56 | 57 | def cli(): 58 | args = parse_args(sys.argv[1:]) 59 | logger.setLevel(args.pop('log') * 10) 60 | main(args.pop('command'), **args) 61 | 62 | 63 | if __name__ == "__main__": 64 | cli() 65 | -------------------------------------------------------------------------------- /pixel_decoder/predict.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import os 3 | import sys 4 | from os import path, listdir, mkdir 5 | import numpy as np 6 | import random 7 | import tensorflow as tf 8 | 9 | import timeit 10 | import cv2 11 | # import skimage.io 12 | from tqdm import tqdm 13 | np.random.seed(1) 14 | tf.set_random_seed(1) 15 | np.seterr(divide='ignore', invalid='ignore') 16 | 17 | from pixel_decoder.utils import dataformat, stats_data, open_image, preprocess_inputs_std, cache_stats 18 | from pixel_decoder.resnet_unet import get_resnet_unet 19 | 20 | def predict(imgs_folder, test_folder, models_folder, pred_folder, origin_shape_no, border_no, model_id, channel_no=3): 21 | origin_shape = (origin_shape_no, origin_shape_no) 22 | rgb_index = [0, 1, 2] 23 | border = (border_no, border_no) 24 | input_shape = (origin_shape[0] + border[0] + border[1] , origin_shape[1] + border[0] + border[1]) 25 | means, stds = cache_stats(imgs_folder) 26 | if not path.isdir(pred_folder):mkdir(os.path.join(os.getcwd(),pred_folder)) 27 | if not path.isdir(path.join(pred_folder, model_id)):mkdir(path.join(pred_folder, model_id)) 28 | if model_id == 'resnet_unet': 29 | model = get_resnet_unet(input_shape, channel_no) 30 | else: 31 | from inception_unet import get_inception_resnet_v2_unet 32 | model = get_inception_resnet_v2_unet(input_shape, channel_no) 33 | model.load_weights(path.join(models_folder, '{}_weights4.h5'.format(model_id))) 34 | 35 | if not path.isdir(models_folder): 36 | mkdir(models_folder) 37 | 38 | for f in tqdm(sorted(listdir(path.join(test_folder)))): 39 | format = dataformat(path.join(test_folder, f)) 40 | if path.isfile(path.join(test_folder, f)) and '.tif' in f: 41 | img_id = f.split('.')[0] 42 | 43 | fpath = path.join(test_folder, f) 44 | img = open_image(fpath) 45 | # if img.shape[0] != origin_shape[0]: 46 | # img= cv2.resize(img, origin_shape) 47 | # else:img = img 48 | if channel_no == 8:img = img 49 | else: 50 | band_index = rgb_index 51 | img = img[:, :, band_index] 52 | img = cv2.copyMakeBorder(img, border[0], border[1], border[0], border[1], cv2.BORDER_REFLECT_101) 53 | inp = [] 54 | inp.append(img) 55 | inp.append(np.rot90(img, k=1)) 56 | inp = np.asarray(inp) 57 | inp = preprocess_inputs_std(inp, means, stds) 58 | pred = model.predict(inp) 59 | mask = pred[0] + np.rot90(pred[1], k=3) 60 | mask /= 2 61 | mask_index1 = border[0] 62 | mask_index2 = input_shape[1] - border[1] 63 | mask = mask[mask_index1:mask_index2, mask_index1:mask_index2, ...] 64 | mask = mask * 255 65 | mask = mask.astype('uint8') 66 | cv2.imwrite(path.join(pred_folder, model_id,'{}.png'.format(img_id)), mask, [cv2.IMWRITE_PNG_COMPRESSION, 9]) 67 | -------------------------------------------------------------------------------- /pixel_decoder/resnet_back.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ResNet50 model for Keras. 3 | 4 | # Reference: 5 | 6 | - [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) 7 | 8 | Adapted from code contributed by BigMoyan. 9 | """ 10 | from __future__ import print_function 11 | from __future__ import absolute_import 12 | 13 | import warnings 14 | 15 | from keras.layers import Input 16 | from keras import layers 17 | from keras.layers import Dense 18 | from keras.layers import Activation 19 | from keras.layers import Flatten 20 | from keras.layers import Conv2D 21 | from keras.layers import MaxPooling2D 22 | from keras.layers import AveragePooling2D 23 | from keras.layers import GlobalAveragePooling2D 24 | from keras.layers import GlobalMaxPooling2D 25 | from keras.layers import BatchNormalization 26 | from keras.models import Model 27 | from keras import backend as K 28 | from keras.engine.topology import get_source_inputs 29 | from keras.utils import layer_utils 30 | from keras.utils.data_utils import get_file 31 | from keras_applications.imagenet_utils import decode_predictions 32 | from keras_applications.imagenet_utils import preprocess_input 33 | from keras_applications.imagenet_utils import _obtain_input_shape 34 | 35 | 36 | WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5' 37 | WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5' 38 | 39 | 40 | def identity_block(input_tensor, kernel_size, filters, stage, block): 41 | """The identity block is the block that has no conv layer at shortcut. 42 | 43 | # Arguments 44 | input_tensor: input tensor 45 | kernel_size: default 3, the kernel size of middle conv layer at main path 46 | filters: list of integers, the filters of 3 conv layer at main path 47 | stage: integer, current stage label, used for generating layer names 48 | block: 'a','b'..., current block label, used for generating layer names 49 | 50 | # Returns 51 | Output tensor for the block. 52 | """ 53 | filters1, filters2, filters3 = filters 54 | if K.image_data_format() == 'channels_last': 55 | bn_axis = 3 56 | else: 57 | bn_axis = 1 58 | conv_name_base = 'res' + str(stage) + block + '_branch' 59 | bn_name_base = 'bn' + str(stage) + block + '_branch' 60 | 61 | x = Conv2D(filters1, (1, 1), name=conv_name_base + '2a')(input_tensor) 62 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2a')(x) 63 | x = Activation('relu')(x) 64 | 65 | x = Conv2D(filters2, kernel_size, 66 | padding='same', name=conv_name_base + '2b')(x) 67 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2b')(x) 68 | x = Activation('relu')(x) 69 | 70 | x = Conv2D(filters3, (1, 1), name=conv_name_base + '2c')(x) 71 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2c')(x) 72 | 73 | x = layers.add([x, input_tensor]) 74 | x = Activation('relu')(x) 75 | return x 76 | 77 | 78 | def conv_block(input_tensor, kernel_size, filters, stage, block, strides=(2, 2)): 79 | """A block that has a conv layer at shortcut. 80 | 81 | # Arguments 82 | input_tensor: input tensor 83 | kernel_size: default 3, the kernel size of middle conv layer at main path 84 | filters: list of integers, the filters of 3 conv layer at main path 85 | stage: integer, current stage label, used for generating layer names 86 | block: 'a','b'..., current block label, used for generating layer names 87 | 88 | # Returns 89 | Output tensor for the block. 90 | 91 | Note that from stage 3, the first conv layer at main path is with strides=(2,2) 92 | And the shortcut should have strides=(2,2) as well 93 | """ 94 | filters1, filters2, filters3 = filters 95 | if K.image_data_format() == 'channels_last': 96 | bn_axis = 3 97 | else: 98 | bn_axis = 1 99 | conv_name_base = 'res' + str(stage) + block + '_branch' 100 | bn_name_base = 'bn' + str(stage) + block + '_branch' 101 | 102 | x = Conv2D(filters1, (1, 1), strides=strides, 103 | name=conv_name_base + '2a')(input_tensor) 104 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2a')(x) 105 | x = Activation('relu')(x) 106 | 107 | x = Conv2D(filters2, kernel_size, padding='same', 108 | name=conv_name_base + '2b')(x) 109 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2b')(x) 110 | x = Activation('relu')(x) 111 | 112 | x = Conv2D(filters3, (1, 1), name=conv_name_base + '2c')(x) 113 | x = BatchNormalization(axis=bn_axis, name=bn_name_base + '2c')(x) 114 | 115 | shortcut = Conv2D(filters3, (1, 1), strides=strides, 116 | name=conv_name_base + '1')(input_tensor) 117 | shortcut = BatchNormalization(axis=bn_axis, name=bn_name_base + '1')(shortcut) 118 | 119 | x = layers.add([x, shortcut]) 120 | x = Activation('relu')(x) 121 | return x 122 | 123 | 124 | def ResNet50(include_top=True, weights='imagenet', 125 | input_tensor=None, input_shape=None, 126 | pooling=None, 127 | classes=1000): 128 | """Instantiates the ResNet50 architecture. 129 | 130 | Optionally loads weights pre-trained 131 | on ImageNet. Note that when using TensorFlow, 132 | for best performance you should set 133 | `image_data_format='channels_last'` in your Keras config 134 | at ~/.keras/keras.json. 135 | 136 | The model and the weights are compatible with both 137 | TensorFlow and Theano. The data format 138 | convention used by the model is the one 139 | specified in your Keras config file. 140 | 141 | # Arguments 142 | include_top: whether to include the fully-connected 143 | layer at the top of the network. 144 | weights: one of `None` (random initialization) 145 | or 'imagenet' (pre-training on ImageNet). 146 | input_tensor: optional Keras tensor (i.e. output of `layers.Input()`) 147 | to use as image input for the model. 148 | input_shape: optional shape tuple, only to be specified 149 | if `include_top` is False (otherwise the input shape 150 | has to be `(224, 224, 3)` (with `channels_last` data format) 151 | or `(3, 224, 224)` (with `channels_first` data format). 152 | It should have exactly 3 inputs channels, 153 | and width and height should be no smaller than 197. 154 | E.g. `(200, 200, 3)` would be one valid value. 155 | pooling: Optional pooling mode for feature extraction 156 | when `include_top` is `False`. 157 | - `None` means that the output of the model will be 158 | the 4D tensor output of the 159 | last convolutional layer. 160 | - `avg` means that global average pooling 161 | will be applied to the output of the 162 | last convolutional layer, and thus 163 | the output of the model will be a 2D tensor. 164 | - `max` means that global max pooling will 165 | be applied. 166 | classes: optional number of classes to classify images 167 | into, only to be specified if `include_top` is True, and 168 | if no `weights` argument is specified. 169 | 170 | # Returns 171 | A Keras model instance. 172 | 173 | # Raises 174 | ValueError: in case of invalid argument for `weights`, 175 | or invalid input shape. 176 | """ 177 | if weights not in {'imagenet', None}: 178 | raise ValueError('The `weights` argument should be either ' 179 | '`None` (random initialization) or `imagenet` ' 180 | '(pre-training on ImageNet).') 181 | 182 | if weights == 'imagenet' and include_top and classes != 1000: 183 | raise ValueError('If using `weights` as imagenet with `include_top`' 184 | ' as true, `classes` should be 1000') 185 | 186 | # Determine proper input shape 187 | input_shape = _obtain_input_shape(input_shape, 188 | default_size=224, 189 | min_size=197, 190 | data_format=K.image_data_format(), 191 | require_flatten=include_top, 192 | weights=weights) 193 | 194 | if input_tensor is None: 195 | img_input = Input(shape=input_shape) 196 | else: 197 | if not K.is_keras_tensor(input_tensor): 198 | img_input = Input(tensor=input_tensor, shape=input_shape) 199 | else: 200 | img_input = input_tensor 201 | if K.image_data_format() == 'channels_last': 202 | bn_axis = 3 203 | else: 204 | bn_axis = 1 205 | 206 | x = Conv2D( 207 | 64, (7, 7), strides=(2, 2), padding='same', name='conv1')(img_input) 208 | x = BatchNormalization(axis=bn_axis, name='bn_conv1')(x) 209 | x = Activation('relu')(x) 210 | x = MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x) 211 | 212 | x = conv_block(x, 3, [64, 64, 256], stage=2, block='a', strides=(1, 1)) 213 | x = identity_block(x, 3, [64, 64, 256], stage=2, block='b') 214 | x = identity_block(x, 3, [64, 64, 256], stage=2, block='c') 215 | 216 | x = conv_block(x, 3, [128, 128, 512], stage=3, block='a') 217 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='b') 218 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='c') 219 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='d') 220 | 221 | x = conv_block(x, 3, [256, 256, 1024], stage=4, block='a') 222 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='b') 223 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='c') 224 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='d') 225 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='e') 226 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='f') 227 | 228 | x = conv_block(x, 3, [512, 512, 2048], stage=5, block='a') 229 | x = identity_block(x, 3, [512, 512, 2048], stage=5, block='b') 230 | x = identity_block(x, 3, [512, 512, 2048], stage=5, block='c') 231 | 232 | x = AveragePooling2D((7, 7), name='avg_pool')(x) 233 | 234 | if include_top: 235 | x = Flatten()(x) 236 | x = Dense(classes, activation='softmax', name='fc1000')(x) 237 | else: 238 | if pooling == 'avg': 239 | x = GlobalAveragePooling2D()(x) 240 | elif pooling == 'max': 241 | x = GlobalMaxPooling2D()(x) 242 | 243 | # Ensure that the model takes into account 244 | # any potential predecessors of `input_tensor`. 245 | if input_tensor is not None: 246 | inputs = get_source_inputs(input_tensor) 247 | else: 248 | inputs = img_input 249 | # Create model. 250 | model = Model(inputs, x, name='resnet50') 251 | 252 | # load weights 253 | if weights == 'imagenet': 254 | if include_top: 255 | weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels.h5', 256 | WEIGHTS_PATH, 257 | cache_subdir='models', 258 | md5_hash='a7b3fe01876f51b976af0dea6bc144eb') 259 | else: 260 | weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5', 261 | WEIGHTS_PATH_NO_TOP, 262 | cache_subdir='models', 263 | md5_hash='a268eb855778b3df3c7506639542a6af') 264 | model.load_weights(weights_path) 265 | if K.backend() == 'theano': 266 | layer_utils.convert_all_kernels_in_model(model) 267 | if include_top: 268 | maxpool = model.get_layer(name='avg_pool') 269 | shape = maxpool.output_shape[1:] 270 | dense = model.get_layer(name='fc1000') 271 | layer_utils.convert_dense_weights_data_format(dense, shape, 'channels_first') 272 | 273 | if K.image_data_format() == 'channels_first' and K.backend() == 'tensorflow': 274 | warnings.warn('You are using the TensorFlow backend, yet you ' 275 | 'are using the Theano ' 276 | 'image data format convention ' 277 | '(`image_data_format="channels_first"`). ' 278 | 'For best performance, set ' 279 | '`image_data_format="channels_last"` in ' 280 | 'your Keras config ' 281 | 'at ~/.keras/keras.json.') 282 | return model 283 | -------------------------------------------------------------------------------- /pixel_decoder/resnet_unet.py: -------------------------------------------------------------------------------- 1 | # from keras.applications.vgg16 import VGG16 2 | # from keras import backend as K 3 | from keras.models import Model 4 | from keras.layers import Input, BatchNormalization, Conv2D, MaxPooling2D, concatenate, UpSampling2D, Activation 5 | from pixel_decoder.resnet_back import ResNet50, identity_block 6 | from pixel_decoder.resnet_back import conv_block as resnet_conv_block 7 | # from keras.losses import binary_crossentropy 8 | # from loss import dice_coef, dice_coef_rounded, dice_coef_loss, dice_logloss, dice_logloss2, dice_logloss3, weighted_bce_loss, weighted_bce_dice_loss 9 | # bn_axis = 3 10 | # channel_axis = bn_axis 11 | 12 | def conv_block(prev, num_filters, kernel=(3, 3), strides=(1, 1), act='relu', prefix=None): 13 | bn_axis = 3 14 | name = None 15 | if prefix is not None: 16 | name = prefix + '_conv' 17 | conv = Conv2D(num_filters, kernel, padding='same', kernel_initializer='he_normal', strides=strides, name=name)(prev) 18 | if prefix is not None: 19 | name = prefix + '_norm' 20 | conv = BatchNormalization(name=name, axis=bn_axis)(conv) 21 | if prefix is not None: 22 | name = prefix + '_act' 23 | conv = Activation(act, name=name)(conv) 24 | return conv 25 | 26 | def get_resnet_unet(input_shape,channel_no, classes = 1, weights='imagenet'): 27 | bn_axis = 3 28 | inp = Input(input_shape + (channel_no,)) 29 | 30 | x = Conv2D( 31 | 64, (7, 7), strides=(2, 2), padding='same', name='conv1')(inp) 32 | x = BatchNormalization(axis=bn_axis, name='bn_conv1')(x) 33 | x = Activation('relu')(x) 34 | conv1 = x 35 | x = MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x) 36 | 37 | x = resnet_conv_block(x, 3, [64, 64, 256], stage=2, block='a', strides=(1, 1)) 38 | x = identity_block(x, 3, [64, 64, 256], stage=2, block='b') 39 | x = identity_block(x, 3, [64, 64, 256], stage=2, block='c') 40 | enc1 = x 41 | 42 | x = resnet_conv_block(x, 3, [128, 128, 512], stage=3, block='a') 43 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='b') 44 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='c') 45 | x = identity_block(x, 3, [128, 128, 512], stage=3, block='d') 46 | enc2 = x 47 | 48 | x = resnet_conv_block(x, 3, [256, 256, 1024], stage=4, block='a') 49 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='b') 50 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='c') 51 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='d') 52 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='e') 53 | x = identity_block(x, 3, [256, 256, 1024], stage=4, block='f') 54 | enc3 = x 55 | 56 | x = resnet_conv_block(x, 3, [512, 512, 2048], stage=5, block='a') 57 | x = identity_block(x, 3, [512, 512, 2048], stage=5, block='b') 58 | x = identity_block(x, 3, [512, 512, 2048], stage=5, block='c') 59 | enc4 = x 60 | 61 | up6 = concatenate([UpSampling2D()(enc4), enc3], axis=-1) 62 | conv6 = conv_block(up6, 128) 63 | conv6 = conv_block(conv6, 128) 64 | 65 | up7 = concatenate([UpSampling2D()(conv6), enc2], axis=-1) 66 | conv7 = conv_block(up7, 96) 67 | conv7 = conv_block(conv7, 96) 68 | 69 | up8 = concatenate([UpSampling2D()(conv7), enc1], axis=-1) 70 | conv8 = conv_block(up8, 64) 71 | conv8 = conv_block(conv8, 64) 72 | 73 | up9 = concatenate([UpSampling2D()(conv8), conv1], axis=-1) 74 | conv9 = conv_block(up9, 48) 75 | conv9 = conv_block(conv9, 48) 76 | 77 | up10 = concatenate([UpSampling2D()(conv9), inp], axis=-1) 78 | conv10 = conv_block(up10, 32) 79 | conv10 = conv_block(conv10, 32) 80 | res = Conv2D(classes, (1, 1), activation='sigmoid')(conv10) 81 | model = Model(inp, res) 82 | 83 | if weights == 'imagenet': 84 | resnet = ResNet50(input_shape=input_shape + (3,), include_top=False, weights=weights) 85 | for i in range(2, len(resnet.layers)-1): 86 | model.layers[i].set_weights(resnet.layers[i].get_weights()) 87 | model.layers[i].trainable = False 88 | 89 | return model 90 | -------------------------------------------------------------------------------- /pixel_decoder/train.py: -------------------------------------------------------------------------------- 1 | # import os 2 | from os import path, mkdir 3 | from pixel_decoder.utils import datafiles, cache_stats, batch_data_generator, val_data_generator 4 | 5 | import numpy as np 6 | np.random.seed(1) 7 | import random 8 | random.seed(1) 9 | import tensorflow as tf 10 | tf.set_random_seed(1) 11 | from sklearn.model_selection import KFold 12 | # import cv2 13 | from keras.optimizers import SGD, Adam 14 | from keras import metrics 15 | from keras.callbacks import ModelCheckpoint 16 | from pixel_decoder.loss import dice_coef, dice_logloss2, dice_logloss3, dice_coef_rounded, dice_logloss 17 | from pixel_decoder.resnet_unet import get_resnet_unet 18 | import keras.backend as K 19 | 20 | def train(batch_size, imgs_folder, masks_folder, models_folder, model_id, origin_shape_no, border_no, classes =1, channel_no=3): 21 | origin_shape = (int(origin_shape_no), int(origin_shape_no)) 22 | border = (int(border_no), int(border_no)) 23 | input_shape = origin_shape 24 | all_files, all_masks = datafiles(imgs_folder, masks_folder) 25 | means, stds = cache_stats(imgs_folder) 26 | if model_id == 'resnet_unet': 27 | model = get_resnet_unet(input_shape, channel_no, classes) 28 | else: 29 | print('No model loaded!') 30 | 31 | if not path.isdir(models_folder): 32 | mkdir(models_folder) 33 | 34 | kf = KFold(n_splits=4, shuffle=True, random_state=1) 35 | for all_train_idx, all_val_idx in kf.split(all_files): 36 | train_idx = [] 37 | val_idx = [] 38 | 39 | for i in all_train_idx: 40 | train_idx.append(i) 41 | for i in all_val_idx: 42 | val_idx.append(i) 43 | 44 | validation_steps = int(len(val_idx) / batch_size) 45 | steps_per_epoch = int(len(train_idx) / batch_size) 46 | 47 | if validation_steps == 0 or steps_per_epoch == 0: 48 | continue 49 | 50 | print('steps_per_epoch', steps_per_epoch, 'validation_steps', validation_steps) 51 | 52 | np.random.seed(11) 53 | random.seed(11) 54 | tf.set_random_seed(11) 55 | print(model.summary()) 56 | batch_data_generat = batch_data_generator(train_idx, batch_size, means, stds, imgs_folder, masks_folder, models_folder, channel_no, border_no, origin_shape_no) 57 | val_data_generat = val_data_generator(val_idx, batch_size, validation_steps, means, stds, imgs_folder, masks_folder, models_folder, channel_no, border_no, origin_shape_no) 58 | 59 | 60 | model.compile(loss=dice_logloss3, 61 | optimizer=SGD(lr=5e-2, decay=1e-6, momentum=0.9, nesterov=True), 62 | metrics=[dice_coef, dice_coef_rounded, metrics.binary_crossentropy]) 63 | 64 | model_checkpoint = ModelCheckpoint(path.join(models_folder, '{}_weights.h5'.format(model_id)), monitor='val_dice_coef_rounded', 65 | save_best_only=True, save_weights_only=True, mode='max') 66 | model.fit_generator(generator=batch_data_generat, 67 | epochs=25, steps_per_epoch=steps_per_epoch, verbose=2, 68 | validation_data=val_data_generat, 69 | validation_steps=validation_steps, 70 | callbacks=[model_checkpoint]) 71 | for l in model.layers: 72 | l.trainable = True 73 | model.compile(loss=dice_logloss3, 74 | optimizer=Adam(lr=1e-3), 75 | metrics=[dice_coef, dice_coef_rounded, metrics.binary_crossentropy]) 76 | 77 | model.fit_generator(generator=batch_data_generat, 78 | epochs=40, steps_per_epoch=steps_per_epoch, verbose=2, 79 | validation_data=val_data_generat, 80 | validation_steps=validation_steps, 81 | callbacks=[model_checkpoint]) 82 | model.optimizer = Adam(lr=2e-4) 83 | model.fit_generator(generator=batch_data_generat, 84 | epochs=25, steps_per_epoch=steps_per_epoch, verbose=2, 85 | validation_data=val_data_generat, 86 | validation_steps=validation_steps, 87 | callbacks=[model_checkpoint]) 88 | 89 | np.random.seed(22) 90 | random.seed(22) 91 | tf.set_random_seed(22) 92 | model.load_weights(path.join(models_folder, '{}_weights.h5'.format(model_id))) 93 | model.compile(loss=dice_logloss, 94 | optimizer=Adam(lr=5e-4), 95 | metrics=[dice_coef, dice_coef_rounded, metrics.binary_crossentropy]) 96 | model_checkpoint2 = ModelCheckpoint(path.join(models_folder, '{}_weights2.h5'.format(model_id)), monitor='val_dice_coef_rounded', 97 | save_best_only=True, save_weights_only=True, mode='max') 98 | model.fit_generator(generator=batch_data_generat, 99 | epochs=30, steps_per_epoch=steps_per_epoch, verbose=2, 100 | validation_data=val_data_generat, 101 | validation_steps=validation_steps, 102 | callbacks=[model_checkpoint2]) 103 | optimizer=Adam(lr=1e-5) 104 | model.fit_generator(generator=batch_data_generat, 105 | epochs=20, steps_per_epoch=steps_per_epoch, verbose=2, 106 | validation_data=val_data_generat, 107 | validation_steps=validation_steps, 108 | callbacks=[model_checkpoint2]) 109 | 110 | np.random.seed(33) 111 | random.seed(33) 112 | tf.set_random_seed(33) 113 | model.load_weights(path.join(models_folder, '{}_weights2.h5'.format(model_id))) 114 | model.compile(loss=dice_logloss2, 115 | optimizer=Adam(lr=5e-5), 116 | metrics=[dice_coef, dice_coef_rounded, metrics.binary_crossentropy]) 117 | model_checkpoint3 = ModelCheckpoint(path.join(models_folder, '{}_weights3.h5'.format(model_id)), monitor='val_dice_coef_rounded', 118 | save_best_only=True, save_weights_only=True, mode='max') 119 | model.fit_generator(generator=batch_data_generat, 120 | epochs=50, steps_per_epoch=steps_per_epoch, verbose=2, 121 | validation_data=val_data_generat, 122 | validation_steps=validation_steps, 123 | callbacks=[model_checkpoint3]) 124 | 125 | np.random.seed(44) 126 | random.seed(44) 127 | tf.set_random_seed(44) 128 | model.load_weights(path.join(models_folder, '{}_weights3.h5'.format(model_id))) 129 | model.compile(loss=dice_logloss3, 130 | optimizer=Adam(lr=2e-5), 131 | metrics=[dice_coef, dice_coef_rounded, metrics.binary_crossentropy]) 132 | model_checkpoint4 = ModelCheckpoint(path.join(models_folder, '{}_weights4.h5'.format(model_id)), monitor='val_dice_coef_rounded', 133 | save_best_only=True, save_weights_only=True, mode='max') 134 | model.fit_generator(generator=batch_data_generat, 135 | epochs=50, steps_per_epoch=steps_per_epoch, verbose=2, 136 | validation_data=val_data_generat, 137 | validation_steps=validation_steps, 138 | callbacks=[model_checkpoint4]) 139 | K.clear_session() 140 | -------------------------------------------------------------------------------- /pixel_decoder/utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | from os import path, listdir, mkdir 4 | import numpy as np 5 | np.random.seed(1) 6 | import random 7 | random.seed(1) 8 | import tensorflow as tf 9 | tf.set_random_seed(1) 10 | import timeit 11 | import cv2 12 | import skimage.io 13 | 14 | def dataformat(fn): 15 | basename, ext = os.path.splitext(fn) 16 | return ext 17 | 18 | def stats_data(data): 19 | if len(data.shape) > 3: 20 | means = np.mean(data, axis = (0, 1, 2)) 21 | stds = np.std(data, axis = (0, 1, 2)) 22 | else: 23 | means = np.mean(data, axis = (0, 1)) 24 | stds = np.std(data, axis = (0, 1)) 25 | return means, stds 26 | 27 | def color_scale(arr): 28 | """correct the wv-3 bands to be a composed bands of value between 0 255""" 29 | axis = (0, 1) 30 | str_arr = (arr - np.min(arr, axis = axis))*255.0/(np.max(arr, axis = axis) - np.min(arr, axis = axis)) 31 | return str_arr 32 | 33 | def open_image(fn): 34 | format = dataformat(fn) 35 | if format == '.tif': 36 | arr = skimage.io.imread(fn, plugin='tifffile').astype('float32') 37 | else: 38 | arr = skimage.io.imread(fn).astype('float32') 39 | img = color_scale(arr) 40 | return img 41 | 42 | def cache_stats(imgs_folder): 43 | imgs = [] 44 | for f in listdir(path.join(imgs_folder)): 45 | format = dataformat(f) 46 | if path.isfile(path.join(imgs_folder, f)) and str(format) in f: 47 | fpath = path.join(imgs_folder, f) 48 | img = open_image(fpath) 49 | img_ = np.expand_dims(img, axis=0) 50 | imgs.append(img) 51 | imgs_arr = np.array(imgs) 52 | dt_means, dt_stds = stats_data(imgs_arr) 53 | # print("mean for the dataset is {}".format(dt_means)) 54 | # print("Std for the dataset is {}".format(dt_stds)) 55 | return dt_means,dt_stds 56 | 57 | def preprocess_inputs_std(x, mean, std): 58 | """The means and stds are train and validation base. 59 | It need to be train's stds and means. It might be ok since we are doing KFold split here""" 60 | zero_msk = (x == 0) 61 | x = np.asarray(x, dtype='float32') 62 | x -= mean 63 | x /= std 64 | x[zero_msk] = 0 65 | return x 66 | 67 | def datafiles(imgs_folder, masks_folder): 68 | all_files = [] 69 | all_masks = [] 70 | t0 = timeit.default_timer() 71 | # imgs_folder = sys.argv[2] 72 | # masks_folder = os.path.join(os.getcwd(),sys.argv[3]) 73 | # models_folder = os.path.join(os.getcwd(),sys.argv[4]) 74 | for f in sorted(listdir(path.join(os.getcwd(), imgs_folder))): 75 | if path.isfile(path.join(os.getcwd(),imgs_folder, f)) and dataformat(path.join(os.getcwd(),imgs_folder, f)) in f: 76 | img_id = f.split('.')[0] 77 | all_files.append(path.join(os.getcwd(), imgs_folder, f)) 78 | all_masks.append(path.join(masks_folder, '{0}.{1}'.format(img_id, 'png'))) 79 | all_files = np.asarray(all_files) 80 | all_masks = np.asarray(all_masks) 81 | return all_files, all_masks 82 | 83 | # all_files,all_masks = datafiles(imgs_folder, masks_folder, models_folder) 84 | def rotate_image(image, angle, scale, imgs_folder, masks_folder): 85 | all_files,all_masks = datafiles(imgs_folder, masks_folder) 86 | image_center = tuple(np.array(image.shape[:2])/2) 87 | rot_mat = cv2.getRotationMatrix2D(image_center, angle, scale) 88 | result = cv2.warpAffine(image, rot_mat, image.shape[:2],flags=cv2.INTER_LINEAR) 89 | return result 90 | 91 | # = cache_stats(imgs_folder) 92 | 93 | def batch_data_generator(train_idx, batch_size, means, stds, imgs_folder, masks_folder, models_folder, channel_no, border_no, origin_shape_no): 94 | origin_shape = (int(origin_shape_no), int(origin_shape_no)) 95 | border = (border_no, border_no) 96 | all_files, all_masks = datafiles(imgs_folder, masks_folder) 97 | input_shape = origin_shape 98 | rgb_index = [0, 1, 2] 99 | inputs = [] 100 | outputs = [] 101 | while True: 102 | np.random.shuffle(train_idx) 103 | for i in train_idx: 104 | img = open_image(all_files[i]) 105 | 106 | if img.shape[0] != origin_shape[0]: 107 | img= cv2.resize(img, origin_shape) 108 | else: 109 | img = img 110 | if channel_no == 8: 111 | img = img 112 | else: 113 | band_index = rgb_index 114 | img = img[:, :, band_index] 115 | #msk = cv2.imread(all_masks[i], cv2.IMREAD_UNCHANGED)[..., 0] 116 | msk = skimage.io.imread(all_masks[i]) 117 | #print(all_files[i], all_masks[i]) 118 | #print(msk.shape) 119 | if random.random() > 0.5: 120 | scale = 0.9 + random.random() * 0.2 121 | angle = random.randint(0, 41) - 24 122 | img = rotate_image(img, angle, scale, imgs_folder, masks_folder) 123 | msk = rotate_image(msk, angle, scale, imgs_folder, masks_folder) 124 | 125 | x0 = random.randint(0, img.shape[1] - input_shape[1]) 126 | y0 = random.randint(0, img.shape[0] - input_shape[0]) 127 | img = img[y0:y0+input_shape[0], x0:x0+input_shape[1], :] 128 | msk = (msk > 127) * 1 129 | msk = msk[..., np.newaxis] 130 | otp = msk[y0:y0+input_shape[0], x0:x0+input_shape[1], :] 131 | 132 | if random.random() > 0.5: 133 | img = img[:, ::-1, ...] 134 | otp = otp[:, ::-1, ...] 135 | 136 | rot = random.randrange(4) 137 | if rot > 0: 138 | img = np.rot90(img, k=rot) 139 | otp = np.rot90(otp, k=rot) 140 | 141 | inputs.append(img) 142 | outputs.append(otp) 143 | 144 | if len(inputs) == batch_size: 145 | inputs = np.asarray(inputs) 146 | outputs = np.asarray(outputs, dtype='float') 147 | inputs = preprocess_inputs_std(inputs, means, stds) 148 | yield inputs, outputs 149 | inputs = [] 150 | outputs = [] 151 | 152 | def val_data_generator(val_idx, batch_size, validation_steps, means, stds, imgs_folder, masks_folder, models_folder, channel_no, border_no, origin_shape_no): 153 | origin_shape = (int(origin_shape_no), int(origin_shape_no)) 154 | border = (border_no, border_no) 155 | all_files, all_masks = datafiles(imgs_folder, masks_folder) 156 | input_shape = origin_shape 157 | means, stds = cache_stats(imgs_folder) 158 | all_files,all_masks = datafiles(imgs_folder, masks_folder) 159 | rgb_index = [0, 1, 2] 160 | while True: 161 | inputs = [] 162 | outputs = [] 163 | step_id = 0 164 | for i in val_idx: 165 | img0 = open_image(all_files[i]) 166 | if img0.shape[0] != origin_shape[0]: 167 | img0= cv2.resize(img0, origin_shape) 168 | else: 169 | img0 = img0 170 | if channel_no == 8:img0 = img0 171 | else: 172 | band_index = rgb_index 173 | img0 = img0[:, :, band_index] 174 | msk = skimage.io.imread(all_masks[i]) 175 | if len(msk.shape)<=2: 176 | msk = np.expand_dims(msk, 2) 177 | msk = (msk > 127) * 1 178 | inputs.append(img0) 179 | outputs.append(msk) 180 | if len(inputs) == batch_size: 181 | step_id += 1 182 | inputs = np.asarray(inputs) 183 | outputs = np.asarray(outputs, dtype='float') 184 | inputs = preprocess_inputs_std(inputs, means, stds) 185 | yield inputs, outputs 186 | inputs = [] 187 | outputs = [] 188 | if step_id == validation_steps: 189 | break 190 | -------------------------------------------------------------------------------- /pixel_decoder/version.py: -------------------------------------------------------------------------------- 1 | __version__ = '0.1.0' 2 | -------------------------------------------------------------------------------- /requirements-dev.txt: -------------------------------------------------------------------------------- 1 | pytest>=3.6.1 2 | pytest-cov>=2.5.1 3 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Cython 2 | tensorflow==1.15.2 3 | keras==2.2.1 4 | h5py 5 | matplotlib 6 | pandas 7 | pylint 8 | scikit-image 9 | scikit-learn 10 | scipy 11 | seaborn 12 | Shapely 13 | tqdm 14 | opencv-python 15 | -------------------------------------------------------------------------------- /result_showcase/TZ_road_out.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/result_showcase/TZ_road_out.png -------------------------------------------------------------------------------- /result_showcase/TZ_road_out1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/result_showcase/TZ_road_out1.png -------------------------------------------------------------------------------- /result_showcase/TZ_road_out3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Geoyi/pixel-decoder/470c3373d183996f929e7a2a48e3df3634a4f589/result_showcase/TZ_road_out3.png -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from setuptools import setup, find_packages 3 | from imp import load_source 4 | from os import path 5 | import io 6 | 7 | __version__ = load_source('pixel_decoder.version', 'pixel_decoder/version.py').__version__ 8 | 9 | here = path.abspath(path.dirname(__file__)) 10 | 11 | # get the dependencies and installs 12 | with io.open(path.join(here, 'requirements.txt'), encoding='utf-8') as f: 13 | all_reqs = f.read().split('\n') 14 | 15 | install_requires = [x.strip() for x in all_reqs if 'git+' not in x] 16 | dependency_links = [x.strip().replace('git+', '') for x in all_reqs if 'git+' not in x] 17 | 18 | #readme 19 | with open('README.md') as f: 20 | readme = f.read() 21 | 22 | setup( 23 | name='pixel_decoder', 24 | author='Zhuangfang NaNa Yi', 25 | author_email='nana@developmentseed.org', 26 | version=__version__, 27 | description='A deep learning algorithm collection to train satellite segmentation', 28 | url='https://github.com/Geoyi/pixel-decoder', 29 | license='MIT', 30 | classifiers=[ 31 | 'Intended Audience :: Developers', 32 | 'Programming Language :: Python :: 3.6', 33 | ], 34 | keywords='', 35 | entry_points={ 36 | 'console_scripts': ['pixel_decoder=pixel_decoder.main:cli'], 37 | }, 38 | packages=find_packages(exclude=['docs', 'tests*']), 39 | include_package_data=True, 40 | install_requires=install_requires, 41 | dependency_links=dependency_links, 42 | long_description=readme, 43 | long_description_content_type="text/markdown" 44 | ) 45 | -------------------------------------------------------------------------------- /test/test_main.py: -------------------------------------------------------------------------------- 1 | import os 2 | import unittest 3 | 4 | testpath = os.path.dirname(__file__) 5 | 6 | 7 | class Test(unittest.TestCase): 8 | """ Test main module """ 9 | 10 | def test_main(self): 11 | """ Run main function """ 12 | pass 13 | --------------------------------------------------------------------------------