├── download_models.sh ├── download_images.sh ├── .gitignore ├── README.md ├── functions.py ├── deploy.py ├── HowToUse.ipynb └── WindowsOpt.py /download_models.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # download ich model 4 | # inceptionv3_ich_sigmoid_U255_nch2 5 | wget -P models/inceptionv3_ich_sigmoid_U255_nch2 https://www.dropbox.com/s/q0alvz3q2igl7ky/model.json 6 | wget -P models/inceptionv3_ich_sigmoid_U255_nch2 https://www.dropbox.com/s/6nwceeuzv3li3zc/weights.h5 7 | 8 | # download stone model 9 | # inceptionv3_stone_sigmoid_U255_nch2 10 | wget -P models/inceptionv3_stone_sigmoid_U255_nch2 https://www.dropbox.com/s/sh4u609trmc9upd/model.json 11 | wget -P models/inceptionv3_stone_sigmoid_U255_nch2 https://www.dropbox.com/s/aooc92dz2zof5t1/weights.h5 -------------------------------------------------------------------------------- /download_images.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # download example ich images 4 | wget -P images/ich https://www.dropbox.com/s/iv513ybg4fuuynf/CASE000826__S3__I0009.npy 5 | wget -P images/ich https://www.dropbox.com/s/iv513ybg4fuuynf/CASE000834__S2__I0016.npy 6 | wget -P images/ich https://www.dropbox.com/s/iv513ybg4fuuynf/CASE000839__S6__I0030.npy 7 | wget -P images/ich https://www.dropbox.com/s/iv513ybg4fuuynf/CASE000846__S2__I0018.npy 8 | wget -P images/ich https://www.dropbox.com/s/iv513ybg4fuuynf/CASE000856__S2__I0013.npy 9 | 10 | # download example stone images 11 | wget -P images/stone https://www.dropbox.com/s/fsjewzjsek5or90/CASE000402__S5__I0083.npy 12 | wget -P images/stone https://www.dropbox.com/s/fsjewzjsek5or90/CASE000422__S3__I0040.npy 13 | wget -P images/stone https://www.dropbox.com/s/fsjewzjsek5or90/CASE000465__S3__I0069.npy 14 | wget -P images/stone https://www.dropbox.com/s/fsjewzjsek5or90/CASE000483__S3__I0061.npy -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | 106 | # Pycharm project files 107 | .idea 108 | 109 | # data folder of this project 110 | images/ 111 | models/ 112 | results/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Window Setting Optimization 2 | This is the reference implementation of the window setting optimization (WSO) layer for medical image deep learning. 3 | 4 | Lee H, Kim M, Do S. Practical Window Setting Optimization for Medical Image Deep Learning. arXiv preprint arXiv:1812.00572. 2018 Dec 3..
5 | arXiv link: https://arxiv.org/abs/1812.00572 6 | 7 | ## Abstract 8 | The recent advancements in deep learning have allowed for numerous applications in computed tomography (CT), with potential to improve diagnostic accuracy, speed of interpretation, and clinical efficiency. However, the deep learning community has to date neglected window display settings - a key feature of clinical CT interpretation and opportunity for additional optimization. Here we propose a window setting optimization (WSO) module that is fully trainable with convolutional neural networks (CNNs) to find optimal window settings for clinical performance. Our approach was inspired by the method commonly used by practicing radiologists to interpret CT images by adjusting window settings to increase the visualization of certain pathologies. Our approach provides optimal window ranges to enhance the conspicuity of abnormalities, and was used to enable performance enhancement for intracranial hemorrhage and urinary stone detection. On each task, the WSO model outperformed models trained over the full range of Hounsfield unit values in CT images, as well as images windowed with pre-defined settings. The WSO module can be readily applied to any analysis of CT images, and can be further generalized to tasks on other medical imaging modalities. 9 | 10 | ## How to use 11 | - Define WSO layer with your own model (see HowToUse.ipynb or WindowsOpt.py) 12 | - Deploy trained model in the original paper (see Deploy_code_WindowsOptimizer.ipynb or deploy.py) 13 | - Including download sample images andmodels 14 | 15 | ## Requirement for test codes 16 | - Keras 2.2.4 17 | - tensorflow-gpu 1.12.0 18 | - openCV 3.4.2 19 | - matplotlib 2.2.2 20 | 21 | 22 | ## Citing windows optimization 23 | Please cite windows optimization in your publications if it helps your research: 24 | 25 | ``` 26 | @article{lee2018practical, 27 | title={Practical Window Setting Optimization for Medical Image Deep Learning}, 28 | author={Lee, Hyunkwang and Kim, Myeongchan and Do, Synho}, 29 | journal={arXiv preprint arXiv:1812.00572}, 30 | year={2018} 31 | } 32 | ``` 33 | -------------------------------------------------------------------------------- /functions.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import cv2 4 | import numpy as np 5 | import tensorflow as tf 6 | from keras import backend as K 7 | from keras.models import model_from_json 8 | 9 | # [WL, WW] 10 | dict_window_settings = { 11 | "brain": (50., 100.), 12 | "subdural": (50., 130.), 13 | "abdomen": (40., 400.), 14 | "bone": (300., 1500.), 15 | "ich_init": ["brain", "subdural"], 16 | "stone_init": ["bone", "abdomen"], 17 | } 18 | 19 | def set_gpu_config(gpu_ids): 20 | # set gpu_id 21 | str_gpu_id = ",".join([str(item) for item in gpu_ids]) 22 | os.environ["CUDA_VISIBLE_DEVICES"] = str_gpu_id 23 | 24 | # make TF not to allocate all of the memory 25 | config = tf.ConfigProto() 26 | config.gpu_options.allow_growth=True 27 | sess = tf.Session(config=config) 28 | K.set_session(sess) 29 | 30 | def load_model(model_json_file, model_weights_file, window_func, window_upbound): 31 | 32 | def upbound_relu(x, window_upbound=window_upbound): 33 | return K.minimum(K.maximum(x,0),window_upbound) 34 | 35 | def upbound_sigmoid(x, window_upbound=window_upbound): 36 | return window_upbound*K.sigmoid(x) 37 | 38 | # load model 39 | jf = open(model_json_file, "r") 40 | model_json = jf.read() 41 | jf.close() 42 | if window_func == "relu": 43 | model = model_from_json(model_json, custom_objects={"upbound_relu":upbound_relu}) 44 | elif window_func == "sigmoid": 45 | model = model_from_json(model_json, custom_objects={"upbound_sigmoid":upbound_sigmoid}) 46 | else: 47 | raise ValueError() 48 | 49 | # load weights 50 | model.load_weights(model_weights_file) 51 | 52 | return model 53 | 54 | def define_model_func(model): 55 | predict = model.get_layer("predict").output 56 | windowed_maps = model.get_layer("window_act").output 57 | model_func = K.function([model.input, K.learning_phase()], [predict, windowed_maps]) 58 | return model_func 59 | 60 | def histogram_equalization(img): 61 | """ 62 | Convert min ~ max range to 0 ~ 255 range 63 | """ 64 | 65 | img = img.astype(np.float64) 66 | 67 | min_val = np.min(img) 68 | max_val = np.max(img) 69 | img = img - min_val 70 | img = img/(max_val-min_val+1e-6) 71 | 72 | img = img*255.0 73 | img = img.astype(np.uint8) 74 | 75 | return img 76 | 77 | def save_image(result_file_path, image, hist_equal=False): 78 | if hist_equal: 79 | image = histogram_equalization(image) 80 | 81 | # save original image 82 | cv2.imwrite(result_file_path, image) 83 | 84 | 85 | def get_init_conv_params(wl, ww, act_window, upbound_value): 86 | if act_window == 'sigmoid': 87 | w_new, b_new = get_init_conv_params_sigmoid(wl, ww, upbound_value=upbound_value) 88 | elif act_window == 'relu': 89 | w_new, b_new = get_init_conv_params_relu(wl, ww, upbound_value=upbound_value) 90 | else: 91 | ## TODO : make a proper exception 92 | raise Exception() 93 | return w_new, b_new 94 | 95 | def get_init_conv_params_relu(wl, ww, upbound_value=255.): 96 | w = upbound_value / ww 97 | b = -1. * upbound_value * (wl - ww / 2.) / ww 98 | return (w, b) 99 | 100 | def get_init_conv_params_sigmoid(wl, ww, smooth=None, upbound_value=255.): 101 | if smooth is None: 102 | smooth = upbound_value / 255.0 103 | 104 | w = 2./ww * np.log(upbound_value/smooth - 1.) 105 | b = -2.*wl/ww * np.log(upbound_value/smooth - 1.) 106 | return (w, b) 107 | 108 | def get_window_settings_relu(w, b, upbound_value=255.): 109 | wl = upbound_value/(2.*w) - b/w 110 | ww = upbound_value / w 111 | return (wl, ww) 112 | 113 | def get_window_settings_sigmoid(w, b, smooth=None, upbound_value=255.): 114 | if smooth is None: 115 | smooth = upbound_value / 255.0 116 | 117 | wl = b/w 118 | ww = 2./w * np.log(upbound_value/smooth - 1.) 119 | return (wl, ww) 120 | 121 | def get_pretrained_model_from_internet(): 122 | ## TODO : A Pretrained Inception-V3 model with ICH, StoneAI 123 | pass 124 | -------------------------------------------------------------------------------- /deploy.py: -------------------------------------------------------------------------------- 1 | import os 2 | import subprocess 3 | import shlex 4 | 5 | from functions import * 6 | 7 | # configurations for dirname 8 | image_base_dir = "images" 9 | model_base_dir = "models" 10 | result_base_dir = "results" 11 | 12 | def check_model_exist(): 13 | if not os.path.isdir(model_base_dir): 14 | return False 15 | if os.listdir(model_base_dir) == ['inceptionv3_ich_sigmoid_U255_nch2', 'inceptionv3_stone_sigmoid_U255_nch2']: 16 | return True 17 | return False 18 | 19 | def check_image_exist(): 20 | if not os.path.isdir(image_base_dir): 21 | return False 22 | if os.listdir(image_base_dir) == ['inceptionv3_ich_sigmoid_U255_nch2', 'inceptionv3_stone_sigmoid_U255_nch2']: 23 | return True 24 | return False 25 | 26 | if __name__ == "__main__": 27 | # ----------------------------------------------------------- 28 | # Load sample images 29 | # ----------------------------------------------------------- 30 | if not check_image_exist(): 31 | print("Downloading sample images...") 32 | subprocess.call(shlex.split("bash download_images.sh")) 33 | 34 | # ----------------------------------------------------------- 35 | # Load trained model 36 | # ----------------------------------------------------------- 37 | if not check_model_exist(): 38 | print("Downloading trained models") 39 | subprocess.call(shlex.split("bash download_models.sh")) 40 | 41 | # ----------------------------------------------------------- 42 | # Set up configurations 43 | # ----------------------------------------------------------- 44 | task = "ich" 45 | labels = ["ich-negative", "ich-positive"] 46 | model_name = "inceptionv3_ich_sigmoid_U255_nch2" 47 | 48 | # task = "stone" 49 | # labels = ["stone-negative", "stone-positive"] 50 | # model_name = "inceptionv3_stone_sigmoid_U255_nch2" 51 | 52 | # window function configurations 53 | window_func = "sigmoid" 54 | window_upbound = 255 55 | window_nch = 2 56 | 57 | ## GPU usage 58 | list_gpus = [0] 59 | 60 | ## Dir setting 61 | image_dir = "{}/{}".format(image_base_dir, task) 62 | model_dir = "{}/{}".format(model_base_dir, model_name) 63 | 64 | # set gpu 65 | set_gpu_config(list_gpus) 66 | num_labels = len(labels) 67 | 68 | # make result dir 69 | result_dir = "{}/{}".format(result_base_dir, model_name) 70 | if not os.path.exists(result_dir): 71 | command = "sudo mkdir -p {}".format(result_dir) 72 | print(command) 73 | subprocess.call(shlex.split(command)) 74 | 75 | command = "sudo chmod 664 {}".format(result_dir) 76 | print(command) 77 | subprocess.call(shlex.split(command)) 78 | 79 | #----------------------------------------------------------- 80 | # Load model and weights 81 | #----------------------------------------------------------- 82 | model_json_file = "{}/model.json".format(model_dir) 83 | model_weights_file = "{}/weights.h5".format(model_dir) 84 | model = load_model(model_json_file, model_weights_file, window_func, window_upbound) 85 | 86 | #---------------------------------------------- 87 | # Test 88 | #---------------------------------------------- 89 | model_func = define_model_func(model) 90 | 91 | image_idx = 0 92 | batches = 0 93 | dict_results = {"gt":[], "pred":[], "prob":[]} 94 | # shape: batch_test_images = (1,512,512,1), batch_test_labels = (1,1) 95 | 96 | for image_file in os.listdir(image_dir): 97 | if image_file.startswith("."): 98 | continue 99 | image_name = image_file.split(".")[0] 100 | image_file_path = "{}/{}".format(image_dir, image_file) 101 | # load image 102 | ori_image = np.load(image_file_path) 103 | test_image = ori_image[np.newaxis,:,:,np.newaxis] 104 | 105 | # deploy model 106 | prob, windowed_maps = model_func([test_image, 0]) 107 | prob = prob[0,0] 108 | pred = 1 if prob>=0.5 else 0 109 | 110 | # save original image 111 | save_image("{}/{}.png".format(result_dir, image_name), ori_image, hist_equal=True) 112 | # save windowed images 113 | for ch_idx in range(window_nch): 114 | window_image = windowed_maps[0,:,:,ch_idx] 115 | result_file_path = "{}/{}_window_ch{}.png".format(result_dir, image_name, ch_idx+1) 116 | save_image(result_file_path, window_image, hist_equal=True) 117 | 118 | print("[{}] prediction : {}".format(image_name, labels[pred])) -------------------------------------------------------------------------------- /HowToUse.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 2, 6 | "metadata": {}, 7 | "outputs": [ 8 | { 9 | "name": "stderr", 10 | "output_type": "stream", 11 | "text": [ 12 | "Using TensorFlow backend.\n" 13 | ] 14 | } 15 | ], 16 | "source": [ 17 | "from WindowsOpt import *" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": {}, 23 | "source": [ 24 | "## configurations" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 3, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "## configurations\n", 34 | "\n", 35 | "# ## For a single-channel WSOlayer\n", 36 | "# nch_window = 1\n", 37 | "# act_window = \"sigmoid\"\n", 38 | "# upbound_window = 255.0\n", 39 | "# init_windows = \"brain\"\n", 40 | "\n", 41 | "## For multi-channel WSOlayer\n", 42 | "nch_window = 2\n", 43 | "act_window = \"sigmoid\"\n", 44 | "upbound_window = 255.0\n", 45 | "init_windows = \"ich_init\"\n", 46 | "\n", 47 | "optimizer = SGD(lr=0.0001, decay=0, momentum=0.9, nesterov=True)\n", 48 | "input_shape = (512, 512, 1)\n", 49 | "input_tensor = keras.layers.Input(shape=input_shape, name=\"input\")\n" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "## Building model" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 4, 62 | "metadata": {}, 63 | "outputs": [], 64 | "source": [ 65 | "#### NOTE\n", 66 | "## Define a window setting optimization layer\n", 67 | "x = WindowOptimizer(nch_window=nch_window, act_window=act_window, upbound_window=upbound_window,\n", 68 | " kernel_initializer=\"he_normal\",\n", 69 | " kernel_regularizer=regularizers.l2(0.5 * 1e-5)\n", 70 | " )(input_tensor)\n", 71 | "\n", 72 | "## ... add some your layer here\n", 73 | "x = Conv2D(32, (3, 3), activation=None, padding=\"same\", name=\"conv1\")(x)\n", 74 | "x = Activation(\"relu\", name=\"conv1_relu\")(x)\n", 75 | "x = MaxPooling2D((7, 7), strides=(3, 3), name=\"pool1\")(x)\n", 76 | "x = Conv2D(256, (3, 3), activation=None, padding=\"same\", name=\"conv2\")(x)\n", 77 | "x = Activation(\"relu\", name=\"conv2_relu\")(x)\n", 78 | "x = MaxPooling2D((7, 7), strides=(3, 3), name=\"pool2\")(x)\n", 79 | "x = GlobalAveragePooling2D(name=\"gap\")(x)\n", 80 | "outputs = Dense(1, activation='sigmoid', name=\"fc\")(x)\n", 81 | "\n", 82 | "model = Model(inputs=input_tensor, outputs=outputs, name=\"main_model\")\n" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "## Initialize value" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": 5, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "#### NOTE\n", 99 | "## Initialize parameters of window setting opt module\n", 100 | "model = initialize_window_setting(model, act_window=act_window, init_windows=init_windows, upbound_window=upbound_window)\n" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "## Complie and check parameters" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 6, 113 | "metadata": {}, 114 | "outputs": [ 115 | { 116 | "name": "stdout", 117 | "output_type": "stream", 118 | "text": [ 119 | "_________________________________________________________________\n", 120 | "Layer (type) Output Shape Param # \n", 121 | "=================================================================\n", 122 | "input (InputLayer) (None, 512, 512, 1) 0 \n", 123 | "_________________________________________________________________\n", 124 | "window_conv (Conv2D) (None, 512, 512, 2) 4 \n", 125 | "_________________________________________________________________\n", 126 | "window_act (Activation) (None, 512, 512, 2) 0 \n", 127 | "_________________________________________________________________\n", 128 | "conv1 (Conv2D) (None, 512, 512, 32) 608 \n", 129 | "_________________________________________________________________\n", 130 | "conv1_relu (Activation) (None, 512, 512, 32) 0 \n", 131 | "_________________________________________________________________\n", 132 | "pool1 (MaxPooling2D) (None, 169, 169, 32) 0 \n", 133 | "_________________________________________________________________\n", 134 | "conv2 (Conv2D) (None, 169, 169, 256) 73984 \n", 135 | "_________________________________________________________________\n", 136 | "conv2_relu (Activation) (None, 169, 169, 256) 0 \n", 137 | "_________________________________________________________________\n", 138 | "pool2 (MaxPooling2D) (None, 55, 55, 256) 0 \n", 139 | "_________________________________________________________________\n", 140 | "gap (GlobalAveragePooling2D) (None, 256) 0 \n", 141 | "_________________________________________________________________\n", 142 | "fc (Dense) (None, 1) 257 \n", 143 | "=================================================================\n", 144 | "Total params: 74,853\n", 145 | "Trainable params: 74,853\n", 146 | "Non-trainable params: 0\n", 147 | "_________________________________________________________________\n", 148 | "window optimization modeul set up (initialized with ich_init settings)\n", 149 | "(WL, WW)=['brain', 'subdural']\n", 150 | "Loaded parameter : w=[0.11074668 0.08518976] b=[-5.5373344 -4.259488 ]\n", 151 | "Expected paramter(brain) : w=[0.11074668] b=[-5.5373344]\n", 152 | "Expected paramter(subdural) : w=[0.08518976] b=[-4.259488]\n" 153 | ] 154 | } 155 | ], 156 | "source": [ 157 | "\n", 158 | "model.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=[\"accuracy\"])\n", 159 | "model.summary()\n", 160 | "\n", 161 | "## Double check initialized parameters for WSO\n", 162 | "names = [weight.name for layer in model.layers for weight in layer.weights]\n", 163 | "# print(names)\n", 164 | "weights = model.get_weights()\n", 165 | "\n", 166 | "for name, weight in zip(names, weights):\n", 167 | " if \"window_conv\" in name:\n", 168 | " if \"kernel:0\" in name:\n", 169 | " ws = weight\n", 170 | " if \"bias:0\" in name:\n", 171 | " bs = weight\n", 172 | "\n", 173 | "print(\"window optimization modeul set up (initialized with {} settings)\".format(init_windows))\n", 174 | "print(\"(WL, WW)={}\".format(dict_window_settings[init_windows]))\n", 175 | "print(\"Loaded parameter : w={} b={}\".format(ws[0, 0, 0, :], bs)) # check result\n", 176 | "print(\"Expected paramter(brain) : w=[0.11074668] b=[-5.5373344]\")\n", 177 | "print(\"Expected paramter(subdural) : w=[0.08518976] b=[-4.259488]\")\n" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": null, 183 | "metadata": {}, 184 | "outputs": [], 185 | "source": [] 186 | } 187 | ], 188 | "metadata": { 189 | "kernelspec": { 190 | "display_name": "Python 3", 191 | "language": "python", 192 | "name": "python3" 193 | }, 194 | "language_info": { 195 | "codemirror_mode": { 196 | "name": "ipython", 197 | "version": 3 198 | }, 199 | "file_extension": ".py", 200 | "mimetype": "text/x-python", 201 | "name": "python", 202 | "nbconvert_exporter": "python", 203 | "pygments_lexer": "ipython3", 204 | "version": "3.6.7" 205 | } 206 | }, 207 | "nbformat": 4, 208 | "nbformat_minor": 2 209 | } 210 | -------------------------------------------------------------------------------- /WindowsOpt.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import keras 3 | from keras import layers 4 | from keras import backend as K 5 | from keras import regularizers 6 | from keras.preprocessing import image 7 | from keras.initializers import Constant 8 | from keras.layers import Input, Dropout, Lambda, MaxPooling1D, MaxPooling2D, Dense, Activation, Concatenate, GlobalAveragePooling2D, GlobalAveragePooling1D, AveragePooling2D, UpSampling2D, ZeroPadding2D, Softmax, Layer 9 | from keras.layers.convolutional import Conv1D, Conv2D 10 | from keras.layers.normalization import BatchNormalization 11 | from keras.optimizers import SGD, Adam 12 | from keras.callbacks import ModelCheckpoint, LearningRateScheduler, TensorBoard, Callback 13 | from keras.models import Model, model_from_json 14 | from keras.applications.inception_v3 import InceptionV3 15 | 16 | from functions import * 17 | 18 | #------------------------------------- 19 | # Window Optimization layer : simple function 20 | #------------------------------------- 21 | 22 | def WindowOptimizer(act_window="sigmoid", upbound_window=255.0, nch_window=1, **kwargs): 23 | ''' 24 | :param act_window: str. sigmoid or relu 25 | :param upbound_window: float. a upbound value of window 26 | :param nch_window: int. number of channels 27 | :param init_windows: str or list. If list, len of list should be same with nch_window. 28 | :param kwargs: other parameters for convolution layer. 29 | 30 | :return: windows optimizer layer 31 | 32 | # Input shape 33 | Arbitrary. 34 | # Output shape 35 | Same shape as input. 36 | ''' 37 | 38 | ## TODO: customizable layer name 39 | wc_name = 'window_conv' 40 | wa_name = 'window_act' 41 | 42 | # print("WindowOptim kwargs : ", kwargs) 43 | conv_layer = WinOptConv(nch_window=nch_window, conv_layer_name=wc_name, **kwargs) 44 | act_layer = WinOptActivation(act_window=act_window, upbound_window=upbound_window, act_layer_name=wa_name) 45 | 46 | ## Return layer funcion 47 | def window_func(x): 48 | x = conv_layer(x) 49 | x = act_layer(x) 50 | return x 51 | 52 | return window_func 53 | 54 | def WinOptConv(nch_window, conv_layer_name, **kwargs): 55 | conv_layer = Conv2D(filters=nch_window, kernel_size=(1, 1), strides=(1, 1), padding="same", 56 | name=conv_layer_name, 57 | **kwargs) 58 | return conv_layer 59 | 60 | def WinOptActivation(act_window, upbound_window, act_layer_name): 61 | def upbound_relu(x): 62 | return K.minimum(K.maximum(x, 0), upbound_window) 63 | 64 | def upbound_sigmoid(x): 65 | return upbound_window * K.sigmoid(x) 66 | 67 | if act_window == "relu": 68 | act_layer = Activation(upbound_relu, name=act_layer_name) 69 | elif act_window == "sigmoid": 70 | act_layer = Activation(upbound_sigmoid, name=act_layer_name) 71 | else: 72 | ## Todo: make a proper exception for here 73 | raise Exception() 74 | 75 | return act_layer 76 | 77 | def get_w_b_with_name(window_name, act_window, upbound_value): 78 | ## Get window settings from dictionay 79 | wl, ww = dict_window_settings[window_name] 80 | ## Set convolution layer 81 | w_new, b_new = get_init_conv_params(wl, ww, act_window, upbound_value) 82 | return w_new, b_new 83 | 84 | def is_list(params): 85 | return type(params) == list 86 | 87 | def is_str(params): 88 | return type(params) == str 89 | 90 | def is_tuple(params): 91 | return type(params) == tuple 92 | 93 | 94 | def initialize_window_setting(model, act_window="sigmoid", init_windows="abdomen", conv_layer_name="window_conv", upbound_window=255.0): 95 | ''' 96 | :param model: 97 | :param act_window: str. 'sigmoid' or 'relu' 98 | :param init_windows: str or list of str. see config.py 99 | :param conv_layer_name: str. a name of window 100 | :return: mdoel. with loaded weight. 101 | ''' 102 | 103 | # get all layer names 104 | layer_names = [layer.name for layer in model.layers] 105 | windows_setting = dict_window_settings[init_windows] 106 | 107 | # multi-channel window settings 108 | if is_list(windows_setting): 109 | window_settings = [dict_window_settings[name] for name in windows_setting] 110 | n_window_settings = len(window_settings ) 111 | 112 | w_conv_ori, b_conv_ori = model.layers[layer_names.index(conv_layer_name)].get_weights() 113 | n_windows = w_conv_ori.shape[-1] 114 | w_conv_new = np.zeros((1,1,1,n_windows), dtype=np.float32) 115 | b_conv_new = np.zeros(n_windows, dtype=np.float32) 116 | 117 | for idx, window_setting in enumerate(window_settings): 118 | wl, ww = window_setting 119 | w_new, b_new = get_init_conv_params(wl, ww, act_window, upbound_window) 120 | w_conv_new[0,0,0,range(idx, n_windows, n_window_settings)] = w_new 121 | b_conv_new[range(idx, n_windows, n_window_settings)] = b_new 122 | 123 | model.layers[layer_names.index(conv_layer_name)].set_weights([w_conv_new, b_conv_new]) 124 | 125 | # single-channel window setting 126 | elif is_tuple(windows_setting): 127 | wl, ww = windows_setting 128 | w_new, b_new = get_init_conv_params(wl, ww, act_window, upbound_window) 129 | 130 | w_conv_ori, b_conv_ori = model.layers[layer_names.index(conv_layer_name)].get_weights() 131 | w_conv_new = np.zeros_like(w_conv_ori) 132 | w_conv_new[0,0,0,:] = w_new * np.ones(w_conv_ori.shape[-1], dtype=w_conv_ori.dtype) 133 | b_conv_new = b_new * np.ones(b_conv_ori.shape, dtype=b_conv_ori.dtype) 134 | 135 | model.layers[layer_names.index(conv_layer_name)].set_weights([w_conv_new, b_conv_new]) 136 | 137 | else: 138 | raise ValueError() 139 | 140 | return model 141 | 142 | 143 | ## Test codes of this module 144 | if __name__ == "__main__": 145 | 146 | ## WSO configurations 147 | # ## For a single-channel WSOlayer 148 | # nch_window = 1 149 | # act_window = "sigmoid" 150 | # upbound_window = 255.0 151 | # init_windows = "brain" 152 | 153 | # For multi-channel WSOlayer 154 | nch_window = 2 155 | act_window = "sigmoid" 156 | upbound_window = 255.0 157 | init_windows = "ich_init" 158 | 159 | 160 | # x = load_example_dicom() # They should be 2d-HU values matrix 161 | input_shape = (512, 512, 1) 162 | input_tensor = keras.layers.Input(shape=input_shape, name="input") 163 | 164 | #### NOTE 165 | ## Define a window setting optimization layer 166 | x = WindowOptimizer(nch_window=nch_window, act_window=act_window, upbound_window=upbound_window, 167 | kernel_initializer="he_normal", 168 | kernel_regularizer=regularizers.l2(0.5 * 1e-5) 169 | )(input_tensor) 170 | 171 | ## ... add some your layer here 172 | x = Conv2D(32, (3, 3), activation=None, padding="same", name="conv1")(x) 173 | x = Activation("relu", name="conv1_relu")(x) 174 | x = MaxPooling2D((7, 7), strides=(3, 3), name="pool1")(x) 175 | x = Conv2D(256, (3, 3), activation=None, padding="same", name="conv2")(x) 176 | x = Activation("relu", name="conv2_relu")(x) 177 | x = MaxPooling2D((7, 7), strides=(3, 3), name="pool2")(x) 178 | x = GlobalAveragePooling2D(name="gap")(x) 179 | outputs = Dense(1, activation='sigmoid')(x) 180 | 181 | model = Model(inputs=input_tensor, outputs=outputs, name="main_model") 182 | 183 | #### NOTE 184 | ## Initialize parameters of window setting opt module 185 | model = initialize_window_setting(model, act_window=act_window, init_windows=init_windows, upbound_window=upbound_window) 186 | 187 | optimizer = SGD(lr=0.0001, decay=0, momentum=0.9, nesterov=True) 188 | model.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=["accuracy"]) 189 | model.summary() 190 | 191 | ## Double check initialized parameters for WSO 192 | names = [weight.name for layer in model.layers for weight in layer.weights] 193 | # print(names) 194 | weights = model.get_weights() 195 | 196 | for name, weight in zip(names, weights): 197 | if "window_conv" in name: 198 | if "kernel:0" in name: 199 | ws = weight 200 | if "bias:0" in name: 201 | bs = weight 202 | 203 | print("window optimization modeul set up (initialized with {} settings)".format(init_windows)) 204 | print("(WL, WW)={}".format(dict_window_settings[init_windows])) 205 | print("Loaded parameter : w={} b={}".format(ws[0, 0, 0, :], bs)) # check result 206 | print("Expected paramter(brain) : w=[0.11074668] b=[-5.5373344]") 207 | print("Expected paramter(subdural) : w=[0.08518976] b=[-4.259488]") 208 | 209 | 210 | --------------------------------------------------------------------------------