├── .gitignore ├── 2D ├── 01_Train-Solution.ipynb ├── 01_Train.ipynb ├── 02_Inference-Solution.ipynb ├── 02_Inference.ipynb ├── 03_Inference-OpenVINO.ipynb ├── README.md ├── argparser.py ├── dataloader.py ├── images │ ├── docker_run.png │ ├── figure1.png │ ├── model.png │ ├── pred10591.png │ ├── pred1100.png │ ├── pred28.png │ ├── pred40.png │ ├── pred400.png │ ├── pred4385.png │ ├── pred4560.png │ ├── pred5566.png │ ├── pred5673.png │ ├── pred61.png │ ├── pred6433.png │ ├── pred7864.png │ ├── pred8722.png │ ├── pred8889.png │ ├── pred9003.png │ ├── run_brats_usage.png │ ├── train_usage.png │ └── unet.png ├── libs │ ├── __init__.py │ └── pconv_layer.py ├── model.py ├── model_pconv.py ├── plot_openvino_inference_examples.py ├── plot_tf_inference_examples.py ├── settings.py └── train.py ├── 3D ├── 3d_unet.ipynb ├── README.md ├── argparser.py ├── dataloader.py ├── horovod_command.sh ├── images │ ├── 3d_commandline.png │ ├── 3d_unet_plot_predictions.png │ ├── BRATS_105.png │ ├── BRATS_152.png │ ├── BRATS_152_img.avi │ ├── BRATS_152_img3D.avi │ ├── BRATS_152_img3D.gif │ ├── BRATS_195.png │ ├── BRATS_195_img.avi │ ├── BRATS_195_img.gif │ ├── BRATS_426.png │ ├── tensorboard.png │ └── training_memory_3d_unet.png ├── model.py ├── plot_predictions.ipynb ├── quantize_model.ipynb ├── run_unet_horovod.sh ├── script_slurm ├── settings.py ├── train.py └── train_horovod.py ├── LICENSE ├── README.md ├── single-node ├── 3D_BraTS2019 │ ├── argparser.py │ ├── dataloader.py │ ├── files.csv │ ├── model.py │ ├── settings.py │ └── train.py ├── README.md ├── histology.ipynb ├── histology.py ├── kmeans-daal4py-distributed.py └── mpi-tester-qsub.ipynb └── testing ├── README.md └── testing.py /.gitignore: -------------------------------------------------------------------------------- 1 | #Project specifics 2 | 2D/output/* 3 | 2D/inference_examples/* 4 | 2D/inference_examples_openvino/* 5 | 6 | 7 | # Byte-compiled / optimized / DLL files 8 | __pycache__/ 9 | *.py[cod] 10 | *$py.class 11 | 12 | # C extensions 13 | *.so 14 | 15 | # Distribution / packaging 16 | .Python 17 | env/ 18 | build/ 19 | develop-eggs/ 20 | dist/ 21 | downloads/ 22 | eggs/ 23 | .eggs/ 24 | lib/ 25 | lib64/ 26 | parts/ 27 | sdist/ 28 | var/ 29 | wheels/ 30 | *.egg-info/ 31 | .installed.cfg 32 | *.egg 33 | 34 | # PyInstaller 35 | # Usually these files are written by a python script from a template 36 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 37 | *.manifest 38 | *.spec 39 | 40 | # Installer logs 41 | pip-log.txt 42 | pip-delete-this-directory.txt 43 | 44 | # Unit test / coverage reports 45 | htmlcov/ 46 | .tox/ 47 | .coverage 48 | .coverage.* 49 | .cache 50 | nosetests.xml 51 | coverage.xml 52 | *.cover 53 | .hypothesis/ 54 | 55 | # Translations 56 | *.mo 57 | *.pot 58 | 59 | # Django stuff: 60 | *.log 61 | local_settings.py 62 | 63 | # Flask stuff: 64 | instance/ 65 | .webassets-cache 66 | 67 | # Scrapy stuff: 68 | .scrapy 69 | 70 | # Sphinx documentation 71 | docs/_build/ 72 | 73 | # PyBuilder 74 | target/ 75 | 76 | # Jupyter Notebook 77 | .ipynb_checkpoints 78 | 79 | # pyenv 80 | .python-version 81 | 82 | # celery beat schedule file 83 | celerybeat-schedule 84 | 85 | # SageMath parsed files 86 | *.sage.py 87 | 88 | # dotenv 89 | .env 90 | 91 | # virtualenv 92 | .venv 93 | venv/ 94 | ENV/ 95 | 96 | # Spyder project settings 97 | .spyderproject 98 | .spyproject 99 | 100 | # Rope project settings 101 | .ropeproject 102 | 103 | # mkdocs documentation 104 | /site 105 | 106 | # mypy 107 | .mypy_cache/ 108 | -------------------------------------------------------------------------------- /2D/01_Train.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Train a model\n", 8 | "\n", 9 | "In this tutorial we will use TensorFlow to train a model." 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import sys\n", 19 | "import platform\n", 20 | "import os\n", 21 | "\n", 22 | "print(\"Python version: {}\".format(sys.version))\n", 23 | "print(\"{}\".format(platform.platform()))" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "# Biomedical Image Segmentation with U-Net\n", 31 | "\n", 32 | "In this code example, we apply the U-Net architecture to segment brain tumors from raw MRI scans as shown below. With relatively little data we are able to train a U-Net model to accurately predict where tumors exist. \n", 33 | "\n", 34 | "The Dice coefficient (the standard metric for the BraTS dataset used in the study) for our model is about 0.82-0.88. Menze et al. [reported](http://ieeexplore.ieee.org/document/6975210/) that expert neuroradiologists manually segmented these tumors with a cross-rater Dice score of 0.75-0.85, meaning that the model’s predictions are on par with what expert physicians have made." 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "Since its introduction two years ago, the [U-Net](https://arxiv.org/pdf/1505.04597.pdf0) architecture has been used to create deep learning models for segmenting [nerves](https://github.com/jocicmarko/ultrasound-nerve-segmentation) in ultrasound images, [lungs](https://www.kaggle.com/c/data-science-bowl-2017#tutorial) in CT scans, and even [interference](https://github.com/jakeret/tf_unet) in radio telescopes.\n", 49 | "\n", 50 | "## What is U-Net?\n", 51 | "U-Net is designed like an [auto-encoder](https://en.wikipedia.org/wiki/Autoencoder). It has an encoding path (“contracting”) paired with a decoding path (“expanding”) which gives it the “U” shape. However, in contrast to the autoencoder, U-Net predicts a pixelwise segmentation map of the input image rather than classifying the input image as a whole. For each pixel in the original image, it asks the question: “To which class does this pixel belong?” This flexibility allows U-Net to predict different parts of the tumor simultaneously." 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "This module loads the data generator from `dataloader.py`, creates a TensorFlow/Keras model from `model.py`, trains the model on the data, and then saves the best model." 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "metadata": {}, 71 | "source": [ 72 | "### TensorFlow Version Check\n", 73 | "\n", 74 | "Check to see what version of TensorFlow is installed and if it has [Intel DNNL optimizations](https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": null, 80 | "metadata": { 81 | "scrolled": true 82 | }, 83 | "outputs": [], 84 | "source": [ 85 | "def test_intel_tensorflow():\n", 86 | " \"\"\"\n", 87 | " Check if Intel version of TensorFlow is installed\n", 88 | " \"\"\"\n", 89 | " import tensorflow as tf\n", 90 | " \n", 91 | " print(\"We are using Tensorflow version {}\".format(tf.__version__))\n", 92 | " \n", 93 | " major_version = int(tf.__version__.split(\".\")[0])\n", 94 | " if major_version >= 2:\n", 95 | " from tensorflow.python import _pywrap_util_port\n", 96 | " print(\"Intel-optimizations (DNNL) enabled:\", _pywrap_util_port.IsMklEnabled())\n", 97 | " else:\n", 98 | " print(\"Intel-optimizations (DNNL) enabled:\", tf.pywrap_tensorflow.IsMklEnabled()) \n", 99 | "\n", 100 | "test_intel_tensorflow()" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "## Training Time!" 108 | ] 109 | }, 110 | { 111 | "cell_type": "markdown", 112 | "metadata": {}, 113 | "source": [ 114 | "The bulk of the training section can be broken down in 4 simple steps:\n", 115 | "1. Load the training data\n", 116 | "1. Define the model\n", 117 | "3. Train the model on the data\n", 118 | "4. Evaluate the best model\n", 119 | " " 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "#### Step 1 : Loading the BraTS data set from the tf.data loader" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": null, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "data_path = \"/data/medical_decathlon/Task01_BrainTumour/\"\n", 136 | "\n", 137 | "crop_dim=128 # Original resolution (240)\n", 138 | "batch_size = 128\n", 139 | "seed=816\n", 140 | "train_test_split=0.85" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": {}, 147 | "outputs": [], 148 | "source": [ 149 | "from dataloader import DatasetGenerator, get_decathlon_filelist\n", 150 | "\n", 151 | "trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=data_path, seed=seed, split=train_test_split)\n", 152 | "\n", 153 | "# TODO: Fill in the missing parameters (...) for the data generator \n", 154 | "\n", 155 | "ds_train = DatasetGenerator(...)\n", 156 | "\n", 157 | "ds_validation = DatasetGenerator(...)\n", 158 | "\n", 159 | "ds_test = DatasetGenerator(...)\n" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "## Plot some samples of the dataset\n", 167 | "\n", 168 | "We can use the DatasetGenerator's plot_samples function to plot a few samples of the dataset. Note that with `augment` set to True, we have randomly cropped, flipped, and rotated the images." 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": null, 174 | "metadata": { 175 | "scrolled": true 176 | }, 177 | "outputs": [], 178 | "source": [ 179 | "ds_train.plot_samples()" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": {}, 186 | "outputs": [], 187 | "source": [ 188 | "ds_validation.plot_samples()" 189 | ] 190 | }, 191 | { 192 | "cell_type": "markdown", 193 | "metadata": {}, 194 | "source": [ 195 | "#### Step 2: Define the model" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": null, 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "from model import unet\n", 205 | "\n", 206 | "print(\"-\" * 30)\n", 207 | "print(\"Creating and compiling model ...\")\n", 208 | "print(\"-\" * 30)\n", 209 | "\n", 210 | "unet_model = unet(fms=16, learning_rate=1e-4, use_dropout=False, use_upsampling=False)\n", 211 | "\n", 212 | "model = unet_model.create_model(\n", 213 | " ds_train.get_input_shape(), \n", 214 | " ds_train.get_output_shape())\n", 215 | "\n", 216 | "model_filename, model_callbacks = unet_model.get_callbacks()\n", 217 | "\n", 218 | "# # If there is a current saved file, then load weights and start from there.\n", 219 | "# saved_model = os.path.join(args.output_path, args.inference_filename)\n", 220 | "# if os.path.isfile(saved_model):\n", 221 | "# model.load_weights(saved_model)" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "The code snippet below draws the model using Keras' built-in `plot_model`. Compare with the implementation of `model.py`" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": null, 234 | "metadata": {}, 235 | "outputs": [], 236 | "source": [ 237 | "from tensorflow.keras.utils import plot_model\n", 238 | "from IPython.display import Image\n", 239 | "\n", 240 | "plot_model(model,\n", 241 | " to_file='images/model.png',\n", 242 | " show_shapes=True,\n", 243 | " show_layer_names=True,\n", 244 | " rankdir='TB'\n", 245 | " )\n", 246 | "Image('images/model.png')" 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "#### Step 3: Train the model on the data" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "metadata": {}, 260 | "outputs": [], 261 | "source": [ 262 | "import datetime\n", 263 | "\n", 264 | "start_time = datetime.datetime.now()\n", 265 | "print(\"Training started at {}\".format(start_time))\n", 266 | "\n", 267 | "n_epoch = 2 # Train for this many epochs\n", 268 | "\n", 269 | "history = model.fit(ds_train,\n", 270 | " epochs=n_epoch,\n", 271 | " validation_data=ds_validation,\n", 272 | " verbose=1,\n", 273 | " callbacks=model_callbacks)\n", 274 | "\n", 275 | "print(\"Total time elapsed for training = {} seconds\".format(datetime.datetime.now() - start_time))\n", 276 | "print(\"Training finished at {}\".format(datetime.datetime.now()))\n", 277 | " \n", 278 | "# Append training log\n", 279 | "# with open(\"training.log\",\"a+\") as fp:\n", 280 | "# fp.write(\"{}: {}\\n\".format(datetime.datetime.now(),\n", 281 | "# history.history[\"val_dice_coef\"]))" 282 | ] 283 | }, 284 | { 285 | "cell_type": "markdown", 286 | "metadata": {}, 287 | "source": [ 288 | "#### Step 4: Evaluate the best model" 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": null, 294 | "metadata": {}, 295 | "outputs": [], 296 | "source": [ 297 | "print(\"-\" * 30)\n", 298 | "print(\"Loading the best trained model ...\")\n", 299 | "print(\"-\" * 30)\n", 300 | "unet_model.evaluate_model(model_filename, ds_testing)" 301 | ] 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "metadata": {}, 306 | "source": [ 307 | "## End: In this tutorial, you have learnt:\n", 308 | "* What is the U-Net model\n", 309 | "* Comparing training times - Tensorflow DNNL vs Tensorflow (stock)\n", 310 | "* How to tweak a series of environment variables to get better performance out of DNNL\n", 311 | "* How to tweak a series of Tensorflow-related and neural-network specific parameters for better performance" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "*Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. SPDX-License-Identifier: EPL-2.0*" 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "metadata": {}, 324 | "source": [ 325 | "*Copyright (c) 2019-2020 Intel Corporation*" 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "execution_count": null, 331 | "metadata": {}, 332 | "outputs": [], 333 | "source": [] 334 | } 335 | ], 336 | "metadata": { 337 | "kernelspec": { 338 | "display_name": "Python 3", 339 | "language": "python", 340 | "name": "python3" 341 | }, 342 | "language_info": { 343 | "codemirror_mode": { 344 | "name": "ipython", 345 | "version": 3 346 | }, 347 | "file_extension": ".py", 348 | "mimetype": "text/x-python", 349 | "name": "python", 350 | "nbconvert_exporter": "python", 351 | "pygments_lexer": "ipython3", 352 | "version": "3.7.10" 353 | } 354 | }, 355 | "nbformat": 4, 356 | "nbformat_minor": 2 357 | } 358 | -------------------------------------------------------------------------------- /2D/02_Inference.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Inference example for trained 2D U-Net model on BraTS.\n", 8 | "Takes a trained model and performs inference on a few validation examples." 9 | ] 10 | }, 11 | { 12 | "cell_type": "code", 13 | "execution_count": null, 14 | "metadata": {}, 15 | "outputs": [], 16 | "source": [ 17 | "import sys\n", 18 | "import platform\n", 19 | "import os\n", 20 | "\n", 21 | "print(\"Python version: {}\".format(sys.version))\n", 22 | "print(\"{}\".format(platform.platform()))" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "def test_intel_tensorflow():\n", 32 | " \"\"\"\n", 33 | " Check if Intel version of TensorFlow is installed\n", 34 | " \"\"\"\n", 35 | " import tensorflow as tf\n", 36 | " \n", 37 | " print(\"We are using Tensorflow version {}\".format(tf.__version__))\n", 38 | " \n", 39 | " major_version = int(tf.__version__.split(\".\")[0])\n", 40 | " if major_version >= 2:\n", 41 | " from tensorflow.python import _pywrap_util_port\n", 42 | " print(\"Intel-optimizations (DNNL) enabled:\", _pywrap_util_port.IsMklEnabled())\n", 43 | " else:\n", 44 | " print(\"Intel-optimizations (DNNL) enabled:\", tf.pywrap_tensorflow.IsMklEnabled()) \n", 45 | "\n", 46 | "test_intel_tensorflow()" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "saved_model_dir = \"./output/2d_unet_decathlon\"" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "# Create output directory for images\n", 65 | "png_directory = \"inference_examples\"\n", 66 | "if not os.path.exists(png_directory):\n", 67 | " os.makedirs(png_directory)\n", 68 | " \n", 69 | "model_filename = os.path.join(saved_model_dir)" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "#### Define the DICE coefficient and loss function" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "The Sørensen–Dice coefficient is a statistic used for comparing the similarity of two samples. Given two sets, X and Y, it is defined as\n", 84 | "\n", 85 | "\\begin{equation}\n", 86 | "dice = \\frac{2|X\\cap Y|}{|X|+|Y|}\n", 87 | "\\end{equation}" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "import numpy as np\n", 97 | "\n", 98 | "def calc_dice(target, prediction, smooth=0.0001):\n", 99 | " \"\"\"\n", 100 | " Sorensen Dice coefficient\n", 101 | " \"\"\"\n", 102 | " prediction = np.round(prediction)\n", 103 | "\n", 104 | " numerator = 2.0 * np.sum(target * prediction) + smooth\n", 105 | " denominator = np.sum(target) + np.sum(prediction) + smooth\n", 106 | " coef = numerator / denominator\n", 107 | "\n", 108 | " return coef\n", 109 | "\n", 110 | "def calc_soft_dice(target, prediction, smooth=0.0001):\n", 111 | " \"\"\"\n", 112 | " Sorensen (Soft) Dice coefficient - Don't round predictions\n", 113 | " \"\"\"\n", 114 | " numerator = 2.0 * np.sum(target * prediction) + smooth\n", 115 | " denominator = np.sum(target) + np.sum(prediction) + smooth\n", 116 | " coef = numerator / denominator\n", 117 | "\n", 118 | " return coef" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "## Inference Time!" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "Inferencing in this example can be done in 3 simple steps:\n", 133 | "1. Load the data\n", 134 | "1. Load the Keras model \n", 135 | "1. Perform a `model.predict` on an input image (or set of images)" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "#### Step 1 : Load data" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": null, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [ 151 | "data_path = \"/data/medical_decathlon/Task01_BrainTumour/\"\n", 152 | "\n", 153 | "crop_dim=128 # Original resolution (240)\n", 154 | "batch_size = 128\n", 155 | "seed=816\n", 156 | "train_test_split=0.85" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "from dataloader import DatasetGenerator, get_decathlon_filelist\n", 166 | "\n", 167 | "trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=data_path, seed=seed, split=train_test_split)\n", 168 | "\n", 169 | "# TODO: Fill in the parameters for the dataset generator to return the `testing` data\n", 170 | "ds_test = DatasetGenerator(...)\n" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "#### Step 2 : Load the model" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": null, 183 | "metadata": {}, 184 | "outputs": [], 185 | "source": [ 186 | "from model import unet\n", 187 | "\n", 188 | "from tensorflow import keras as K\n", 189 | "model = K.models.load_model(saved_model_dir, compile=False, custom_objects=unet().custom_objects)" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "#### Step 3: Perform prediction on some images. \n", 197 | "The prediction results will be saved in the output directory for images, which is defined by the `png_directory` variable." 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": null, 203 | "metadata": {}, 204 | "outputs": [], 205 | "source": [ 206 | "import matplotlib.pyplot as plt\n", 207 | "%matplotlib inline\n", 208 | "\n", 209 | "import time\n", 210 | "\n", 211 | "def plot_results(ds):\n", 212 | " \n", 213 | " plt.figure(figsize=(10,10))\n", 214 | "\n", 215 | " img, msk = next(ds.ds)\n", 216 | "\n", 217 | " idx = np.argmax(np.sum(np.sum(msk[:,:,:,0], axis=1), axis=1)) # find the slice with the largest tumor\n", 218 | "\n", 219 | " plt.subplot(1, 3, 1)\n", 220 | " plt.imshow(img[idx, :, :, 0], cmap=\"bone\", origin=\"lower\")\n", 221 | " plt.title(\"MRI {}\".format(idx), fontsize=20)\n", 222 | "\n", 223 | " plt.subplot(1, 3, 2)\n", 224 | " plt.imshow(msk[idx, :, :], cmap=\"bone\", origin=\"lower\")\n", 225 | " plt.title(\"Ground truth\", fontsize=20)\n", 226 | "\n", 227 | " plt.subplot(1, 3, 3)\n", 228 | "\n", 229 | " print(\"Index {}: \".format(idx), end=\"\")\n", 230 | " \n", 231 | " # Predict using the TensorFlow model\n", 232 | " start_time = time.time()\n", 233 | " prediction = model.predict(img[[idx]])\n", 234 | " print(\"Elapsed time = {:.4f} msecs, \".format(1000.0*(time.time()-start_time)), end=\"\")\n", 235 | " \n", 236 | " plt.imshow(prediction[0,:,:,0], cmap=\"bone\", origin=\"lower\")\n", 237 | " dice_coef = calc_dice(msk[idx], prediction)\n", 238 | " print(\"Dice coefficient = {:.4f}, \".format(dice_coef), end=\"\")\n", 239 | " plt.title(\"Prediction\\nDice = {:.4f}\".format(dice_coef), fontsize=20)\n" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": {}, 246 | "outputs": [], 247 | "source": [ 248 | "plot_results(ds_test)" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": null, 254 | "metadata": {}, 255 | "outputs": [], 256 | "source": [ 257 | "plot_results(ds_test)" 258 | ] 259 | }, 260 | { 261 | "cell_type": "code", 262 | "execution_count": null, 263 | "metadata": {}, 264 | "outputs": [], 265 | "source": [ 266 | "plot_results(ds_test)" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": null, 272 | "metadata": { 273 | "scrolled": true 274 | }, 275 | "outputs": [], 276 | "source": [ 277 | "plot_results(ds_test)" 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": null, 283 | "metadata": {}, 284 | "outputs": [], 285 | "source": [ 286 | "plot_results(ds_test)" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": { 293 | "scrolled": false 294 | }, 295 | "outputs": [], 296 | "source": [ 297 | "plot_results(ds_test)" 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": null, 303 | "metadata": {}, 304 | "outputs": [], 305 | "source": [ 306 | "plot_results(ds_test)" 307 | ] 308 | }, 309 | { 310 | "cell_type": "markdown", 311 | "metadata": {}, 312 | "source": [ 313 | "# Can we perform inference even faster? Hmm.." 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "Let's find out. Move on the the next tutorial section." 321 | ] 322 | }, 323 | { 324 | "cell_type": "markdown", 325 | "metadata": {}, 326 | "source": [ 327 | "*Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. SPDX-License-Identifier: EPL-2.0*" 328 | ] 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "metadata": {}, 333 | "source": [ 334 | "*Copyright (c) 2019-2020 Intel Corporation*" 335 | ] 336 | }, 337 | { 338 | "cell_type": "code", 339 | "execution_count": null, 340 | "metadata": {}, 341 | "outputs": [], 342 | "source": [] 343 | } 344 | ], 345 | "metadata": { 346 | "kernelspec": { 347 | "display_name": "Python 3", 348 | "language": "python", 349 | "name": "python3" 350 | }, 351 | "language_info": { 352 | "codemirror_mode": { 353 | "name": "ipython", 354 | "version": 3 355 | }, 356 | "file_extension": ".py", 357 | "mimetype": "text/x-python", 358 | "name": "python", 359 | "nbconvert_exporter": "python", 360 | "pygments_lexer": "ipython3", 361 | "version": "3.7.10" 362 | } 363 | }, 364 | "nbformat": 4, 365 | "nbformat_minor": 2 366 | } 367 | -------------------------------------------------------------------------------- /2D/03_Inference-OpenVINO.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Inference example for trained 2D U-Net model on BraTS.\n", 8 | "In this tutorial, we will use the Intel® Distribution of OpenVINO™ Toolkit to perform inference. We will load the OpenVINO version of the model (IR) and perform inference on a few validation samples from the Decathlon dataset.\n", 9 | "\n", 10 | "This tutorial assumes that you have already downloaded and installed [Intel® OpenVINO™](https://software.intel.com/en-us/openvino-toolkit/choose-download) on your computer. These instructions assume version R5 2018." 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "### Inferencing with the Intel® Distribution of OpenVINO™ Toolkit" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": {}, 23 | "source": [ 24 | "In order to use Intel® OpenVINO™, we need to do a few steps:\n", 25 | "\n", 26 | "1. Use the OpenVINO Model Optimizer to convert the above freezed-model to the OpenVINO Intermediate Representation (IR) format\n", 27 | "1. Load data\n", 28 | "1. Validation\n", 29 | "1. Inference time :)" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "Like we have seen in the training tutorial, we define the Sorensen Dice score coefficient, to measure of the overlap between the prediction and ground truth masks:" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import numpy as np\n", 46 | "\n", 47 | "def calc_dice(target, prediction, smooth=0.0001):\n", 48 | " \"\"\"\n", 49 | " Sorensen Dice coefficient\n", 50 | " \"\"\"\n", 51 | " prediction = np.round(prediction)\n", 52 | "\n", 53 | " numerator = 2.0 * np.sum(target * prediction) + smooth\n", 54 | " denominator = np.sum(target) + np.sum(prediction) + smooth\n", 55 | " coef = numerator / denominator\n", 56 | "\n", 57 | " return coef\n", 58 | "\n", 59 | "def calc_soft_dice(target, prediction, smooth=0.0001):\n", 60 | " \"\"\"\n", 61 | " Sorensen (Soft) Dice coefficient - Don't round predictions\n", 62 | " \"\"\"\n", 63 | " numerator = 2.0 * np.sum(target * prediction) + smooth\n", 64 | " denominator = np.sum(target) + np.sum(prediction) + smooth\n", 65 | " coef = numerator / denominator\n", 66 | "\n", 67 | " return coef" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "### Step 1: Convert the Tensorflow* saved model to the Intermediate Representation (IR) with Intel® OpenVINO™" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "In this step, we will use the Intel® OpenVINO™'s `model optimizer` to convert the frozen TensorFlow* model to Intel® OpenVINO™ IR format. Once you have a frozen model, you can use the Intel® OpenVINO™ `model optimizer` to create the Intel® OpenVINO™ version." 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": {}, 87 | "source": [ 88 | "Let's first set up a few environment variables for the bash code snippet to work. The code below will create a **FP32** precision model. " 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "In order to call `mo_ty.py` directly from this script, it is advisable to source the OpenVINO vars in your `.bashrc` script:\n", 96 | "`source /opt/intel/openvino/bin/setupvars.sh`" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": null, 102 | "metadata": { 103 | "scrolled": true 104 | }, 105 | "outputs": [], 106 | "source": [ 107 | "saved_model_dir = \"./output/2d_unet_decathlon\"\n", 108 | "\n", 109 | "batch_size = 1\n", 110 | "\n", 111 | "import os\n", 112 | "\n", 113 | "# Create output directory for images\n", 114 | "png_directory = \"inference_examples\"\n", 115 | "if not os.path.exists(png_directory):\n", 116 | " os.makedirs(png_directory)\n", 117 | "\n", 118 | "if not os.path.exists(saved_model_dir):\n", 119 | " print('File {} doesn\\'t exist. Please make sure you have a trained TF model'.format(saved_model_dir))\n", 120 | "\n", 121 | "precision=\"FP32\"\n", 122 | "openvino_model_dir = os.path.join(\"output\", precision)\n", 123 | "openvino_model_name = \"2d_unet_decathlon\"\n", 124 | "\n", 125 | "!mo_tf.py \\\n", 126 | " --saved_model_dir $saved_model_dir \\\n", 127 | " --batch $batch_size \\\n", 128 | " --data_type $precision \\\n", 129 | " --output_dir $openvino_model_dir \\\n", 130 | " --model_name $openvino_model_name" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "If you would like to infer on a Neural Compute Stick, you will need to output a **FP16** precision model. To do this, you simply need to change the `--data_type` from FP32 to FP16:\n", 138 | "\n", 139 | "```\n", 140 | "!mo_tf.py \\\n", 141 | " --saved_model_dir $saved_model_dir \\\n", 142 | " --batch $batch_size \\\n", 143 | " --data_type FP16 \\\n", 144 | " --output_dir openvino_model_dir/FP16 \\\n", 145 | " --model_name 2d_unet_decathlon\n", 146 | "```" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "### Step 2: Load Data" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "execution_count": null, 159 | "metadata": {}, 160 | "outputs": [], 161 | "source": [ 162 | "data_path = \"/data/medical_decathlon/Task01_BrainTumour/\"\n", 163 | "\n", 164 | "crop_dim=128 # Original resolution (240)\n", 165 | "batch_size = 128\n", 166 | "seed=816\n", 167 | "train_test_split=0.85" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": null, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "from dataloader import DatasetGenerator, get_decathlon_filelist\n", 177 | "\n", 178 | "trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=data_path, seed=seed, split=train_test_split)\n", 179 | "\n", 180 | "ds_test = DatasetGenerator(testFiles, \n", 181 | " batch_size=batch_size, \n", 182 | " crop_dim=[crop_dim, crop_dim], \n", 183 | " augment=False, \n", 184 | " seed=seed)" 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "metadata": {}, 190 | "source": [ 191 | "### Step 3: Validation" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "### Inferencing with Intel® OpenVINO™" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "metadata": {}, 205 | "outputs": [], 206 | "source": [ 207 | "from openvino.inference_engine import IECore\n", 208 | "\n", 209 | "import numpy as np" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "The `print_stats` prints layer by layer inference times. This is good for profiling which ops are most costly in your model." 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [ 225 | "path_to_xml_file = \"{}.xml\".format(os.path.join(openvino_model_dir, openvino_model_name))\n", 226 | "path_to_bin_file = \"{}.bin\".format(os.path.join(openvino_model_dir, openvino_model_name))\n", 227 | "print(\"OpenVINO IR: {}, {}\".format(path_to_xml_file, path_to_bin_file))\n", 228 | "\n", 229 | "ie = IECore()\n", 230 | "net = ie.read_network(model=path_to_xml_file, weights=path_to_bin_file)\n", 231 | "\n", 232 | "input_layer_name = next(iter(net.input_info))\n", 233 | "output_layer_name = next(iter(net.outputs))\n", 234 | "print(\"Input layer name = {}\\nOutput layer name = {}\".format(input_layer_name, output_layer_name))\n", 235 | "\n", 236 | "exec_net = ie.load_network(network=net, device_name=\"CPU\", num_requests=1)" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": null, 242 | "metadata": {}, 243 | "outputs": [], 244 | "source": [ 245 | "import matplotlib.pyplot as plt\n", 246 | "%matplotlib inline\n", 247 | "\n", 248 | "import time \n", 249 | "\n", 250 | "def plot_results(ds):\n", 251 | " \n", 252 | " img, msk = next(ds.ds)\n", 253 | "\n", 254 | " idx = np.argmax(np.sum(np.sum(msk[:,:,:,0], axis=1), axis=1)) # find the slice with the largest tumor\n", 255 | "\n", 256 | " plt.figure(figsize=(10,10))\n", 257 | "\n", 258 | " plt.subplot(1, 3, 1)\n", 259 | " plt.imshow(img[idx, :, :, 0], cmap=\"bone\", origin=\"lower\")\n", 260 | " plt.title(\"MRI {}\".format(idx), fontsize=20)\n", 261 | "\n", 262 | " plt.subplot(1, 3, 2)\n", 263 | " plt.imshow(msk[idx, :, :], cmap=\"bone\", origin=\"lower\")\n", 264 | " plt.title(\"Ground truth\", fontsize=20)\n", 265 | "\n", 266 | " plt.subplot(1, 3, 3)\n", 267 | "\n", 268 | " # Predict using the OpenVINO model\n", 269 | " # NOTE: OpenVINO expects channels first for input and output\n", 270 | " # So we transpose the input and output\n", 271 | " start_time = time.time()\n", 272 | " res = exec_net.infer({input_layer_name: np.transpose(img[[idx]], [0,3,1,2])})\n", 273 | " prediction = np.transpose(res[output_layer_name], [0,2,3,1]) \n", 274 | " print(\"Elapsed time = {:.4f} msecs\".format(1000.0*(time.time()-start_time)))\n", 275 | "\n", 276 | " plt.imshow(prediction[0,:,:,0], cmap=\"bone\", origin=\"lower\")\n", 277 | " plt.title(\"Prediction\\nDice = {:.4f}\".format(calc_dice(msk[idx, :, :], prediction)), fontsize=20)\n" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "Plot the predictions with Matplotlib and save to PNG files" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": null, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "plot_results(ds_test)" 294 | ] 295 | }, 296 | { 297 | "cell_type": "code", 298 | "execution_count": null, 299 | "metadata": { 300 | "scrolled": true 301 | }, 302 | "outputs": [], 303 | "source": [ 304 | "plot_results(ds_test)" 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": null, 310 | "metadata": {}, 311 | "outputs": [], 312 | "source": [ 313 | "plot_results(ds_test)" 314 | ] 315 | }, 316 | { 317 | "cell_type": "code", 318 | "execution_count": null, 319 | "metadata": {}, 320 | "outputs": [], 321 | "source": [ 322 | "plot_results(ds_test)" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": null, 328 | "metadata": {}, 329 | "outputs": [], 330 | "source": [ 331 | "plot_results(ds_test)" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "plot_results(ds_test)" 341 | ] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "execution_count": null, 346 | "metadata": {}, 347 | "outputs": [], 348 | "source": [ 349 | "plot_results(ds_test)" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "*Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. SPDX-License-Identifier: EPL-2.0*" 357 | ] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": {}, 362 | "source": [ 363 | "*Copyright (c) 2019-2020 Intel Corporation*" 364 | ] 365 | } 366 | ], 367 | "metadata": { 368 | "kernelspec": { 369 | "display_name": "Python 3", 370 | "language": "python", 371 | "name": "python3" 372 | }, 373 | "language_info": { 374 | "codemirror_mode": { 375 | "name": "ipython", 376 | "version": 3 377 | }, 378 | "file_extension": ".py", 379 | "mimetype": "text/x-python", 380 | "name": "python", 381 | "nbconvert_exporter": "python", 382 | "pygments_lexer": "ipython3", 383 | "version": "3.7.10" 384 | } 385 | }, 386 | "nbformat": 4, 387 | "nbformat_minor": 2 388 | } 389 | -------------------------------------------------------------------------------- /2D/README.md: -------------------------------------------------------------------------------- 1 | # 2D U-Net for Medical Decathlon Dataset 2 | 3 | Please see our blog on the [IntelAI website](https://www.intel.ai/intel-neural-compute-stick-2-for-medical-imaging/) 4 | 5 | ![prediction4385](images/pred4385.png) 6 | 7 | 8 | Trains a 2D U-Net on the brain tumor segmentation (BraTS) subset of the [Medical Segmentation Decathlon](http://medicaldecathlon.com/) dataset. 9 | 10 | Steps: 11 | 1. Go to the [Medical Segmentation Decathlon](http://medicaldecathlon.com) website and download the [BraTS subset](https://drive.google.com/file/d/1A2IU8Sgea1h3fYLpYtFb2v7NYdMjvEhU/view?usp=sharing). The dataset has the [Creative Commons Attribution-ShareAlike 4.0 International license](https://creativecommons.org/licenses/by-sa/4.0/). 12 | 13 | 2. Untar the "Task01_BrainTumour.tar" file (e.g. `tar -xvf Task01_BrainTumour.tar`) 14 | 15 | 3. We use [conda virtual environments](https://www.anaconda.com/distribution/#download-section) to run Python scripts. Once you download and install conda, create a new conda environment with [TensorFlow* with Intel® DNNL](https://software.intel.com/en-us/articles/intel-optimization-for-tensorflow-installation-guide?page=1). Run the command: 16 | ``` 17 | conda create -c anaconda -n decathlon pip python=3.7 tensorflow tqdm psutil jupyter matplotlib 18 | ``` 19 | 20 | This has been tested with [TensorFlow 2.2](https://anaconda.org/anaconda/tensorflow-mkl) on Ubuntu 18.04 Linux. 21 | 22 | 4. Enable the new environment. Command: 23 | ``` 24 | conda activate decathlon 25 | ``` 26 | 27 | 5. Install the package [nibabel](http://nipy.org/nibabel/). Command: 28 | ``` 29 | pip install nibabel 30 | ``` 31 | 32 | 6. Run the command 33 | ``` 34 | python train.py --data_path $DECATHLON_ROOT_DIRECTORY 35 | ``` 36 | where $DECATHLON_ROOT_DIRECTORY is the root directory where you un-tarred the Decathlon dataset. 37 | 38 | ![brats_train](images/run_brats_usage.png) 39 | 40 | 7. [OpenVINO™](https://www.youtube.com/watch?v=kY9nZbX1DWM) - At the end of `train.py` you should see instructions on how to [convert the model](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html) for use with the [Intel® Distribution of the OpenVINO™ toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html). Once you have OpenVINO™ installed, you can run a command like the one below to create an OpenVINO™ intermediate representation (IR) of the TensorFlow model. If you are using the [Intel® Neural Compute Stick™ (NCS2)](https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html), simply replace the `FP32` with `FP16` in the command below: 41 | 42 | ``` 43 | source /opt/intel/openvino_2021/bin/setupvars.sh 44 | python $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo_tf.py \ 45 | --saved_model_dir ./output/2d_unet_decathlon \ 46 | --input_shape [1,128,128,4] \ 47 | --model_name 2d_unet_decathlon \ 48 | --output_dir ./output/FP32 \ 49 | --data_type FP32 50 | ``` 51 | 52 | This has been tested with the [Intel® Distribution of the OpenVINO™ toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html) 2021.2. 53 | 54 | 8. Once you have the OpenVINO™ IR model, you can run the command: 55 | 56 | ``` 57 | python plot_openvino_inference_examples.py --data_path $DECATHLON_ROOT_DIRECTORY --device CPU 58 | ``` 59 | 60 | It should give you the same output as the `plot_tf_inference_examples.py` but execute faster on the same CPU. You can try the options `--device GPU` or `--device=MYRIAD` if you have the [Intel® integrated GPU](https://ark.intel.com/content/www/us/en/ark/products/graphics/197532/intel-iris-plus-graphics.html) or [Intel® Neural Compute Stick™ (NCS2)](https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html) installed on your computer. 61 | 62 | For a complete demo showing the [Intel® Neural Compute Stick™ (NCS2)](https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html) try out the [Intel® DevCloud for the Edge](https://devcloud.intel.com/edge/advanced/sample_applications/). You'll be able to try running inference on lots of Intel® hardware using the same OpenVINO™ pipeline. 63 | 64 | ![prediction28](images/pred28.png) 65 | 66 | Tips for improving model: 67 | * The feature maps have been reduced so that the model will train using under 12GB of memory. If you have more memory to use, consider increasing the feature maps using the commandline argument `--featuremaps`. The results I plot in the images subfolder are from a model with `--featuremaps=32`. This will increase the complexity of the model (which will also increase its memory footprint but decrease its execution speed). 68 | * If you choose a subset with larger tensors (e.g. liver or lung), it is recommended to add another maxpooling level (and corresponding upsampling) to the U-Net model. This will of course increase the memory requirements and decrease execution speed, but should give better results because it considers an additional recepetive field/spatial size. 69 | * Consider different loss functions. The default loss function here is a weighted sum of `-log(Dice)` and `binary_crossentropy`. Different loss functions yield different loss curves and may result in better accuracy. However, you may need to adjust the `learning_rate` and number of epochs to train as you experiment with different loss functions. The commandline argument `--weight_dice_loss` defines the weight to each loss function (`loss = weight_dice_loss * -log(dice) + (1-weight_loss_dice)*binary_cross_entropy_loss`). 70 | 71 | ![run_train_command](images/train_usage.png) 72 | 73 | ![prediction61](images/pred61.png) 74 | ![prediction7864](images/pred7864.png) 75 | 76 | REFERENCES: 77 | 78 | 1. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp Γ, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SM, Ryan M, Sarikaya D, Schwartz L, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Van Leemput K. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)", IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694 79 | 80 | 2. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, Freymann JB, Farahani K, Davatzikos C. "Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features", Nature Scientific Data, 4:170117 (2017) DOI: 10.1038/sdata.2017.117 81 | 82 | 3. Simpson AL, Antonelli M, Bakas S, Bilello M, Farahani K, van Ginneken B, Kopp-Schneider A, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, Bilic P, Christ PF, Do RKG, Gollub M, Golia-Pernicka J, Heckers SH, Jarnagin WR, McHugo MK, Napel S, Vorontsov E, Maier-Hein L, Cardoso MJ. "A large annotated medical image dataset for the development and evaluation of segmentation algorithms." https://arxiv.org/abs/1902.09063 83 | 84 | 85 | ### Optimization notice 86 | Please see our [optimization notice](https://software.intel.com/en-us/articles/optimization-notice#opt-en). 87 | 88 | -------------------------------------------------------------------------------- /2D/argparser.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | """ 22 | This module just reads parameters from the command line. 23 | """ 24 | 25 | import argparse 26 | import settings # Use the custom settings.py file for default parameters 27 | import os 28 | 29 | parser = argparse.ArgumentParser( 30 | description="2D U-Net model (Keras/TF) on BraTS Decathlon dataset.", 31 | add_help=True, formatter_class=argparse.ArgumentDefaultsHelpFormatter) 32 | 33 | parser.add_argument("--data_path", default=settings.DATA_PATH, 34 | help="The path to the Medical Decathlon directory") 35 | parser.add_argument("--output_path", default=settings.OUT_PATH, 36 | help="the folder to save the model and checkpoints") 37 | parser.add_argument("--inference_filename", default=settings.INFERENCE_FILENAME, 38 | help="the Keras inference model filename") 39 | parser.add_argument("--use_upsampling", 40 | help="use upsampling instead of transposed convolution", 41 | action="store_true", default=settings.USE_UPSAMPLING) 42 | parser.add_argument("--num_threads", type=int, 43 | default=settings.NUM_INTRA_THREADS, 44 | help="the number of threads") 45 | parser.add_argument("--num_inter_threads", type=int, 46 | default=settings.NUM_INTER_THREADS, 47 | help="the number of intraop threads") 48 | parser.add_argument("--batch_size", type=int, default=settings.BATCH_SIZE, 49 | help="the batch size for training") 50 | parser.add_argument("--split", type=float, default=settings.TRAIN_TEST_SPLIT, 51 | help="Train/testing split for the data") 52 | parser.add_argument("--seed", type=int, default=settings.SEED, 53 | help="Seed for random number generation") 54 | parser.add_argument("--crop_dim", type=int, default=settings.CROP_DIM, 55 | help="Size to crop images (square, in pixels). If -1, then no cropping.") 56 | parser.add_argument("--blocktime", type=int, 57 | default=settings.BLOCKTIME, 58 | help="blocktime") 59 | parser.add_argument("--epochs", type=int, 60 | default=settings.EPOCHS, 61 | help="number of epochs to train") 62 | parser.add_argument("--learningrate", type=float, 63 | default=settings.LEARNING_RATE, 64 | help="learningrate") 65 | parser.add_argument("--weight_dice_loss", type=float, 66 | default=settings.WEIGHT_DICE_LOSS, 67 | help="Weight for the Dice loss compared to crossentropy") 68 | parser.add_argument("--featuremaps", type=int, 69 | default=settings.FEATURE_MAPS, 70 | help="How many feature maps in the model.") 71 | parser.add_argument("--use_pconv", help="use partial convolution based padding", 72 | action="store_true", 73 | default=settings.USE_PCONV) 74 | parser.add_argument("--channels_first", help="use channels first data format", 75 | action="store_true", default=settings.CHANNELS_FIRST) 76 | parser.add_argument("--print_model", help="print the model", 77 | action="store_true", 78 | default=settings.PRINT_MODEL) 79 | parser.add_argument("--use_dropout", 80 | default=settings.USE_DROPOUT, 81 | help="add spatial dropout layers 3/4", 82 | action="store_true", 83 | ) 84 | parser.add_argument("--use_augmentation", 85 | default=settings.USE_AUGMENTATION, 86 | help="use data augmentation on training images", 87 | action="store_true") 88 | parser.add_argument("--output_pngs", 89 | default="inference_examples", 90 | help="the directory for the output prediction pngs") 91 | parser.add_argument("--input_filename", 92 | help="Name of saved TensorFlow model directory", 93 | default=os.path.join(settings.OUT_PATH,settings.INFERENCE_FILENAME)) 94 | 95 | args = parser.parse_args() 96 | 97 | 98 | -------------------------------------------------------------------------------- /2D/dataloader.py: -------------------------------------------------------------------------------- 1 | from tensorflow.keras.utils import Sequence 2 | import numpy as np 3 | import os 4 | import json 5 | import settings 6 | 7 | def get_decathlon_filelist(data_path, seed=816, split=0.85): 8 | """ 9 | Get the paths for the original decathlon files 10 | """ 11 | json_filename = os.path.join(data_path, "dataset.json") 12 | 13 | try: 14 | with open(json_filename, "r") as fp: 15 | experiment_data = json.load(fp) 16 | except IOError as e: 17 | raise Exception("File {} doesn't exist. It should be part of the " 18 | "Decathlon directory".format(json_filename)) 19 | 20 | # Print information about the Decathlon experiment data 21 | print("*" * 30) 22 | print("=" * 30) 23 | print("Dataset name: ", experiment_data["name"]) 24 | print("Dataset description: ", experiment_data["description"]) 25 | print("Tensor image size: ", experiment_data["tensorImageSize"]) 26 | print("Dataset release: ", experiment_data["release"]) 27 | print("Dataset reference: ", experiment_data["reference"]) 28 | print("Dataset license: ", experiment_data["licence"]) # sic 29 | print("=" * 30) 30 | print("*" * 30) 31 | 32 | """ 33 | Randomize the file list. Then separate into training and 34 | validation lists. We won't use the testing set since we 35 | don't have ground truth masks for this; instead we'll 36 | split the validation set into separate test and validation 37 | sets. 38 | """ 39 | # Set the random seed so that always get same random mix 40 | np.random.seed(seed) 41 | numFiles = experiment_data["numTraining"] 42 | idxList = np.arange(numFiles) # List of file indices 43 | np.random.shuffle(idxList) # Shuffle the indices to randomize train/test/split 44 | 45 | trainIdx = int(np.floor(numFiles*split)) # index for the end of the training files 46 | trainList = idxList[:trainIdx] 47 | 48 | otherList = idxList[trainIdx:] 49 | numOther = len(otherList) 50 | otherIdx = numOther//2 # index for the end of the testing files 51 | validateList = otherList[:otherIdx] 52 | testList = otherList[otherIdx:] 53 | 54 | trainFiles = [] 55 | for idx in trainList: 56 | trainFiles.append(os.path.join(data_path, experiment_data["training"][idx]["label"])) 57 | 58 | validateFiles = [] 59 | for idx in validateList: 60 | validateFiles.append(os.path.join(data_path, experiment_data["training"][idx]["label"])) 61 | 62 | testFiles = [] 63 | for idx in testList: 64 | testFiles.append(os.path.join(data_path, experiment_data["training"][idx]["label"])) 65 | 66 | print("Number of training files = {}".format(len(trainList))) 67 | print("Number of validation files = {}".format(len(validateList))) 68 | print("Number of testing files = {}".format(len(testList))) 69 | 70 | return trainFiles, validateFiles, testFiles 71 | 72 | 73 | class DatasetGenerator(Sequence): 74 | """ 75 | TensorFlow Dataset from Python/NumPy Iterator 76 | """ 77 | 78 | def __init__(self, filenames, batch_size=8, crop_dim=[240,240], augment=False, seed=816): 79 | 80 | import nibabel as nib 81 | 82 | img = np.array(nib.load(filenames[0]).dataobj) # Load the first image 83 | self.slice_dim = 2 # We'll assume z-dimension (slice) is last 84 | # Determine the number of slices (we'll assume this is consistent for the other images) 85 | self.num_slices_per_scan = img.shape[self.slice_dim] 86 | 87 | # If crop_dim == -1, then don't crop 88 | if crop_dim[0] == -1: 89 | crop_dim[0] = img.shape[0] 90 | if crop_dim[1] == -1: 91 | crop_dim[1] = img.shape[1] 92 | self.crop_dim = crop_dim 93 | 94 | self.filenames = filenames 95 | self.batch_size = batch_size 96 | 97 | self.augment = augment 98 | self.seed = seed 99 | 100 | self.num_files = len(self.filenames) 101 | 102 | self.ds = self.get_dataset() 103 | 104 | def preprocess_img(self, img): 105 | """ 106 | Preprocessing for the image 107 | z-score normalize 108 | """ 109 | return (img - img.mean()) / img.std() 110 | 111 | def preprocess_label(self, label): 112 | """ 113 | Predict whole tumor. If you want to predict tumor sections, then 114 | just comment this out. 115 | """ 116 | label[label > 0] = 1.0 117 | 118 | return label 119 | 120 | def augment_data(self, img, msk): 121 | """ 122 | Data augmentation 123 | Flip image and mask. Rotate image and mask. 124 | """ 125 | 126 | if np.random.rand() > 0.5: 127 | ax = np.random.choice([0,1]) 128 | img = np.flip(img, ax) 129 | msk = np.flip(msk, ax) 130 | 131 | if np.random.rand() > 0.5: 132 | rot = np.random.choice([1, 2, 3]) # 90, 180, or 270 degrees 133 | 134 | img = np.rot90(img, rot, axes=[0,1]) # Rotate axes 0 and 1 135 | msk = np.rot90(msk, rot, axes=[0,1]) # Rotate axes 0 and 1 136 | 137 | return img, msk 138 | 139 | def crop_input(self, img, msk): 140 | """ 141 | Randomly crop the image and mask 142 | """ 143 | 144 | slices = [] 145 | 146 | # Do we randomize? 147 | is_random = self.augment and np.random.rand() > 0.5 148 | 149 | for idx, idy in enumerate(range(2)): # Go through each dimension 150 | 151 | cropLen = self.crop_dim[idx] 152 | imgLen = img.shape[idy] 153 | 154 | start = (imgLen-cropLen)//2 155 | 156 | ratio_crop = 0.20 # Crop up this this % of pixels for offset 157 | # Number of pixels to offset crop in this dimension 158 | offset = int(np.floor(start*ratio_crop)) 159 | 160 | if offset > 0: 161 | if is_random: 162 | start += np.random.choice(range(-offset, offset)) 163 | if ((start + cropLen) > imgLen): # Don't fall off the image 164 | start = (imgLen-cropLen)//2 165 | else: 166 | start = 0 167 | 168 | slices.append(slice(start, start+cropLen)) 169 | 170 | return img[tuple(slices)], msk[tuple(slices)] 171 | 172 | def generate_batch_from_files(self): 173 | """ 174 | Python generator which goes through a list of filenames to load. 175 | The files are 3D image (slice is dimension index 2 by default). However, 176 | we need to yield them as a batch of 2D slices. This generator 177 | keeps yielding a batch of 2D slices at a time until the 3D image is 178 | complete and then moves to the next 3D image in the filenames. 179 | An optional `randomize_slices` allows the user to randomize the 3D image 180 | slices after loading if desired. 181 | """ 182 | import nibabel as nib 183 | 184 | np.random.seed(self.seed) # Set a random seed 185 | 186 | idx = 0 187 | idy = 0 188 | 189 | while True: 190 | 191 | """ 192 | Pack N_IMAGES files at a time to queue 193 | """ 194 | NUM_QUEUED_IMAGES = 1 + self.batch_size // self.num_slices_per_scan # Get enough for full batch + 1 195 | 196 | for idz in range(NUM_QUEUED_IMAGES): 197 | 198 | label_filename = self.filenames[idx] 199 | 200 | #img_filename = label_filename.replace("_seg.nii.gz", "_flair.nii.gz") # BraTS 2018 201 | img_filename = label_filename.replace("labelsTr", "imagesTr") # Medical Decathlon 202 | 203 | img = np.array(nib.load(img_filename).dataobj) 204 | img = img[:,:,:,0] # Just take FLAIR channel (channel 0) 205 | img = self.preprocess_img(img) 206 | 207 | label = np.array(nib.load(label_filename).dataobj) 208 | label = self.preprocess_label(label) 209 | 210 | # Crop input and label 211 | img, label = self.crop_input(img, label) 212 | 213 | if idz == 0: 214 | img_stack = img 215 | label_stack = label 216 | 217 | else: 218 | 219 | img_stack = np.concatenate((img_stack,img), axis=self.slice_dim) 220 | label_stack = np.concatenate((label_stack,label), axis=self.slice_dim) 221 | 222 | idx += 1 223 | if idx >= len(self.filenames): 224 | idx = 0 225 | np.random.shuffle(self.filenames) # Shuffle the filenames for the next iteration 226 | 227 | img = img_stack 228 | label = label_stack 229 | 230 | num_slices = img.shape[self.slice_dim] 231 | 232 | if self.batch_size > num_slices: 233 | raise Exception("Batch size {} is greater than" 234 | " the number of slices in the image {}." 235 | " Data loader cannot be used.".format(self.batch_size, num_slices)) 236 | 237 | """ 238 | We can also randomize the slices so that no 2 runs will return the same slice order 239 | for a given file. This also helps get slices at the end that would be skipped 240 | if the number of slices is not the same as the batch order. 241 | """ 242 | if self.augment: 243 | slice_idx = np.random.choice(range(num_slices), num_slices) 244 | img = img[:,:,slice_idx] # Randomize the slices 245 | label = label[:,:,slice_idx] 246 | 247 | name = self.filenames[idx] 248 | 249 | if (idy + self.batch_size) < num_slices: # We have enough slices for batch 250 | img_batch, label_batch = img[:,:,idy:idy+self.batch_size], label[:,:,idy:idy+self.batch_size] 251 | 252 | else: # We need to pad the batch with slices 253 | 254 | img_batch, label_batch = img[:,:,-self.batch_size:], label[:,:,-self.batch_size:] # Get remaining slices 255 | 256 | if self.augment: 257 | img_batch, label_batch = self.augment_data(img_batch, label_batch) 258 | 259 | if len(np.shape(img_batch)) == 3: 260 | img_batch = np.expand_dims(img_batch, axis=-1) 261 | if len(np.shape(label_batch)) == 3: 262 | label_batch = np.expand_dims(label_batch, axis=-1) 263 | 264 | yield np.transpose(img_batch, [2,0,1,3]).astype(np.float32), np.transpose(label_batch, [2,0,1,3]).astype(np.float32) 265 | 266 | 267 | idy += self.batch_size 268 | if idy >= num_slices: # We finished this file, move to the next 269 | idy = 0 270 | idx += 1 271 | 272 | if idx >= len(self.filenames): 273 | idx = 0 274 | np.random.shuffle(self.filenames) # Shuffle the filenames for the next iteration 275 | 276 | 277 | def get_input_shape(self): 278 | """ 279 | Get image shape 280 | """ 281 | return [self.crop_dim[0], self.crop_dim[1], 1] 282 | 283 | def get_output_shape(self): 284 | """ 285 | Get label shape 286 | """ 287 | return [self.crop_dim[0], self.crop_dim[1], 1] 288 | 289 | def get_dataset(self): 290 | """ 291 | Return a dataset 292 | """ 293 | ds = self.generate_batch_from_files() 294 | 295 | return ds 296 | 297 | def __len__(self): 298 | return (self.num_slices_per_scan * self.num_files)//self.batch_size 299 | 300 | def __getitem__(self, idx): 301 | return next(self.ds) 302 | 303 | def plot_samples(self): 304 | """ 305 | Plot some random samples 306 | """ 307 | import matplotlib.pyplot as plt 308 | 309 | img, label = next(self.ds) 310 | 311 | print(img.shape) 312 | 313 | plt.figure(figsize=(10,10)) 314 | 315 | slice_num = 3 316 | plt.subplot(2,2,1) 317 | plt.imshow(img[slice_num,:,:,0]); 318 | plt.title("MRI, Slice #{}".format(slice_num)); 319 | 320 | plt.subplot(2,2,2) 321 | plt.imshow(label[slice_num,:,:,0]); 322 | plt.title("Tumor, Slice #{}".format(slice_num)); 323 | 324 | slice_num = self.batch_size - 1 325 | plt.subplot(2,2,3) 326 | plt.imshow(img[slice_num,:,:,0]); 327 | plt.title("MRI, Slice #{}".format(slice_num)); 328 | 329 | plt.subplot(2,2,4) 330 | plt.imshow(label[slice_num,:,:,0]); 331 | plt.title("Tumor, Slice #{}".format(slice_num)); 332 | -------------------------------------------------------------------------------- /2D/images/docker_run.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/docker_run.png -------------------------------------------------------------------------------- /2D/images/figure1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/figure1.png -------------------------------------------------------------------------------- /2D/images/model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/model.png -------------------------------------------------------------------------------- /2D/images/pred10591.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred10591.png -------------------------------------------------------------------------------- /2D/images/pred1100.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred1100.png -------------------------------------------------------------------------------- /2D/images/pred28.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred28.png -------------------------------------------------------------------------------- /2D/images/pred40.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred40.png -------------------------------------------------------------------------------- /2D/images/pred400.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred400.png -------------------------------------------------------------------------------- /2D/images/pred4385.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred4385.png -------------------------------------------------------------------------------- /2D/images/pred4560.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred4560.png -------------------------------------------------------------------------------- /2D/images/pred5566.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred5566.png -------------------------------------------------------------------------------- /2D/images/pred5673.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred5673.png -------------------------------------------------------------------------------- /2D/images/pred61.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred61.png -------------------------------------------------------------------------------- /2D/images/pred6433.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred6433.png -------------------------------------------------------------------------------- /2D/images/pred7864.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred7864.png -------------------------------------------------------------------------------- /2D/images/pred8722.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred8722.png -------------------------------------------------------------------------------- /2D/images/pred8889.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred8889.png -------------------------------------------------------------------------------- /2D/images/pred9003.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/pred9003.png -------------------------------------------------------------------------------- /2D/images/run_brats_usage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/run_brats_usage.png -------------------------------------------------------------------------------- /2D/images/train_usage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/train_usage.png -------------------------------------------------------------------------------- /2D/images/unet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/images/unet.png -------------------------------------------------------------------------------- /2D/libs/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/2D/libs/__init__.py -------------------------------------------------------------------------------- /2D/libs/pconv_layer.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | """ 22 | This is an implementation adapted from https://github.com/MathiasGruber/PConv-Keras 23 | This layer is Keras implementation of Partial Convolution based padding. 24 | See original contribution here: https://github.com/NVIDIA/partialconv 25 | """ 26 | 27 | import tensorflow as tf 28 | from keras.utils import conv_utils 29 | from keras import backend as K 30 | from keras.engine import InputSpec 31 | from keras.layers import Conv2D 32 | import keras 33 | 34 | class PConv2D(Conv2D): 35 | def __init__(self, *args, n_channels=3, mono=False, **kwargs): 36 | super().__init__(*args, **kwargs) 37 | self.input_spec = [InputSpec(ndim=4)] 38 | 39 | def build(self, input_shape): 40 | """Adapted from original _Conv() layer of Keras 41 | param input_shape: list of dimensions for [img] 42 | """ 43 | 44 | if self.data_format == 'channels_first': 45 | channel_axis = 1 46 | else: 47 | channel_axis = -1 48 | 49 | if input_shape[channel_axis] is None: 50 | raise ValueError('The channel dimension of the inputs should be defined. Found `None`.') 51 | 52 | self.input_dim = input_shape[channel_axis] 53 | 54 | # Image kernel 55 | kernel_shape = self.kernel_size + (self.input_dim, self.filters) 56 | self.kernel = self.add_weight(shape=kernel_shape, 57 | initializer=self.kernel_initializer, 58 | name='img_kernel', 59 | regularizer=self.kernel_regularizer, 60 | constraint=self.kernel_constraint) 61 | # # Mask kernel 62 | self.kernel_mask = K.ones(shape=self.kernel_size + (1, 1)) 63 | 64 | # Window size - used for normalization 65 | self.window_size = self.kernel_size[0] * self.kernel_size[1] 66 | 67 | if self.use_bias: 68 | self.bias = self.add_weight(shape=(self.filters,), 69 | initializer=self.bias_initializer, 70 | name='bias', 71 | regularizer=self.bias_regularizer, 72 | constraint=self.bias_constraint) 73 | else: 74 | self.bias = None 75 | self.built = True 76 | 77 | def call(self, inputs, mask=None): 78 | ''' 79 | We will be using the Keras conv2d method, and essentially we have 80 | to do here is multiply the mask with the input X, before we apply the 81 | convolutions. For the mask itself, we apply convolutions with all weights 82 | set to 1. 83 | Subsequently, we clip mask values to between 0 and 1 84 | ''' 85 | 86 | # inputs[1] = torch.ones(1, 1, input.data.shape[2], input.data.shape[3]).to(input) 87 | mask = K.ones(shape=(1, inputs.shape[1], inputs.shape[2], 1)) 88 | 89 | # Apply convolutions to mask 90 | update_mask = K.conv2d( 91 | mask, self.kernel_mask, 92 | strides=self.strides, 93 | padding=self.padding, 94 | data_format=self.data_format, 95 | dilation_rate=self.dilation_rate 96 | ) 97 | 98 | # Calculate the mask ratio on each pixel in the output mask 99 | mask_ratio = self.window_size / (update_mask + 1e-8) 100 | 101 | # Clip output to be between 0 and 1 102 | update_mask = K.clip(update_mask, 0, 1) 103 | 104 | # Remove ratio values where there are holes 105 | mask_ratio = mask_ratio * update_mask 106 | 107 | # Apply convolutions to image 108 | img_output = K.conv2d( 109 | inputs, self.kernel, 110 | strides=self.strides, 111 | padding=self.padding, 112 | data_format=self.data_format, 113 | dilation_rate=self.dilation_rate 114 | ) 115 | 116 | # Normalize image output 117 | img_output = img_output * mask_ratio 118 | 119 | # Apply bias only to the image (if chosen to do so) 120 | if self.use_bias: 121 | img_output = K.bias_add( 122 | img_output, 123 | self.bias, 124 | data_format=self.data_format) 125 | 126 | # Apply activations on the image 127 | if self.activation is not None: 128 | img_output = self.activation(img_output) 129 | 130 | return img_output 131 | 132 | def compute_output_shape(self, input_shape): 133 | if self.data_format == 'channels_last': 134 | space = input_shape[1:-1] 135 | new_space = [] 136 | for i in range(len(space)): 137 | new_dim = conv_utils.conv_output_length( 138 | space[i], 139 | self.kernel_size[i], 140 | padding='same', 141 | stride=self.strides[i], 142 | dilation=self.dilation_rate[i]) 143 | new_space.append(new_dim) 144 | new_shape = (input_shape[0],) + tuple(new_space) + (self.filters,) 145 | return new_shape 146 | if self.data_format == 'channels_first': 147 | space = input_shape[2:] 148 | new_space = [] 149 | for i in range(len(space)): 150 | new_dim = conv_utils.conv_output_length( 151 | space[i], 152 | self.kernel_size[i], 153 | padding='same', 154 | stride=self.strides[i], 155 | dilation=self.dilation_rate[i]) 156 | new_space.append(new_dim) 157 | new_shape = (input_shape[0], self.filters) + tuple(new_space) 158 | return new_shape 159 | -------------------------------------------------------------------------------- /2D/plot_openvino_inference_examples.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | """ 22 | Takes a trained model and performs inference on a few validation examples. 23 | """ 24 | import os 25 | 26 | import numpy as np 27 | import time 28 | import settings 29 | import argparse 30 | from dataloader import DatasetGenerator, get_decathlon_filelist 31 | 32 | from openvino.inference_engine import IECore 33 | 34 | import matplotlib 35 | import matplotlib.pyplot as plt 36 | matplotlib.use("Agg") 37 | 38 | 39 | parser = argparse.ArgumentParser( 40 | description="OpenVINO Inference example for trained 2D U-Net model on BraTS.", 41 | add_help=True, formatter_class=argparse.ArgumentDefaultsHelpFormatter) 42 | 43 | parser.add_argument("--data_path", default=settings.DATA_PATH, 44 | help="the path to the data") 45 | parser.add_argument("--output_path", default=settings.OUT_PATH, 46 | help="the folder to save the model and checkpoints") 47 | parser.add_argument("--inference_filename", default=settings.INFERENCE_FILENAME, 48 | help="the TensorFlow inference model filename") 49 | parser.add_argument("--device", default="CPU", 50 | help="the inference device") 51 | parser.add_argument("--output_pngs", default="inference_examples", 52 | help="the directory for the output prediction pngs") 53 | 54 | parser.add_argument("--intraop_threads", default=settings.NUM_INTRA_THREADS, 55 | type=int, help="Number of intra-op-parallelism threads") 56 | parser.add_argument("--interop_threads", default=settings.NUM_INTER_THREADS, 57 | type=int, help="Number of inter-op-parallelism threads") 58 | parser.add_argument("--crop_dim", default=settings.CROP_DIM, 59 | type=int, help="Crop dimension for images") 60 | parser.add_argument("--seed", default=settings.SEED, 61 | type=int, help="Random seed") 62 | parser.add_argument("--split", type=float, default=settings.TRAIN_TEST_SPLIT, 63 | help="Train/testing split for the data") 64 | 65 | args = parser.parse_args() 66 | 67 | def calc_dice(target, prediction, smooth=0.0001): 68 | """ 69 | Sorenson Dice 70 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 71 | where T is ground truth (target) mask and P is the prediction mask 72 | """ 73 | prediction = np.round(prediction) 74 | 75 | numerator = 2.0 * np.sum(target * prediction) + smooth 76 | denominator = np.sum(target) + np.sum(prediction) + smooth 77 | coef = numerator / denominator 78 | 79 | return coef 80 | 81 | 82 | def calc_soft_dice(target, prediction, smooth=0.0001): 83 | """ 84 | Sorensen (Soft) Dice coefficient - Don't round predictions 85 | """ 86 | numerator = 2.0 * np.sum(target * prediction) + smooth 87 | denominator = np.sum(target) + np.sum(prediction) + smooth 88 | coef = numerator / denominator 89 | 90 | return coef 91 | 92 | 93 | def plot_results(ds, batch_num, png_directory, exec_net, input_layer_name, output_layer_name): 94 | 95 | plt.figure(figsize=(10,10)) 96 | 97 | img, msk = next(ds.ds) 98 | 99 | idx = np.argmax(np.sum(np.sum(msk[:,:,:,0], axis=1), axis=1)) # find the slice with the largest tumor 100 | 101 | plt.subplot(1, 3, 1) 102 | plt.imshow(img[idx, :, :, 0], cmap="bone", origin="lower") 103 | plt.title("MRI {}".format(idx), fontsize=20) 104 | 105 | plt.subplot(1, 3, 2) 106 | plt.imshow(msk[idx, :, :], cmap="bone", origin="lower") 107 | plt.title("Ground truth", fontsize=20) 108 | 109 | plt.subplot(1, 3, 3) 110 | 111 | print("Index {}: ".format(idx), end="") 112 | 113 | # Predict using the OpenVINO model 114 | # NOTE: OpenVINO expects channels first for input and output 115 | # So we transpose the input and output 116 | start_time = time.time() 117 | res = exec_net.infer({input_layer_name: np.transpose(img[[idx]], [0,3,1,2])}) 118 | prediction = np.transpose(res[output_layer_name], [0,2,3,1]) 119 | print("Elapsed time = {:.4f} msecs, ".format(1000.0*(time.time()-start_time)), end="") 120 | 121 | plt.imshow(prediction[0,:,:,0], cmap="bone", origin="lower") 122 | dice_coef = calc_dice(msk[idx], prediction) 123 | plt.title("Prediction\nDice = {:.4f}".format(dice_coef), fontsize=20) 124 | 125 | print("Dice coefficient = {:.4f}, ".format(dice_coef), end="") 126 | 127 | save_name = os.path.join(png_directory, "prediction_openvino_{}_{}.png".format(batch_num, idx)) 128 | print("Saved as: {}".format(save_name)) 129 | plt.savefig(save_name) 130 | 131 | if __name__ == "__main__": 132 | 133 | model_filename = os.path.join(args.output_path, args.inference_filename) 134 | 135 | trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=args.data_path, seed=args.seed, split=args.split) 136 | 137 | ds_test = DatasetGenerator(testFiles, batch_size=128, crop_dim=[args.crop_dim,args.crop_dim], augment=False, seed=args.seed) 138 | 139 | if args.device != "CPU": 140 | precision="FP16" 141 | else: 142 | precision = "FP32" 143 | path_to_xml_file = "{}.xml".format(os.path.join(args.output_path, precision, args.inference_filename)) 144 | path_to_bin_file = "{}.bin".format(os.path.join(args.output_path, precision, args.inference_filename)) 145 | 146 | ie = IECore() 147 | net = ie.read_network(model=path_to_xml_file, weights=path_to_bin_file) 148 | 149 | input_layer_name = next(iter(net.input_info)) 150 | output_layer_name = next(iter(net.outputs)) 151 | print("Input layer name = {}\nOutput layer name = {}".format(input_layer_name, output_layer_name)) 152 | 153 | exec_net = ie.load_network(network=net, device_name=args.device, num_requests=1) 154 | 155 | # Create output directory for images 156 | png_directory = args.output_pngs 157 | if not os.path.exists(png_directory): 158 | os.makedirs(png_directory) 159 | 160 | for batch_num in range(10): 161 | plot_results(ds_test, batch_num, png_directory, exec_net, input_layer_name, output_layer_name) 162 | -------------------------------------------------------------------------------- /2D/plot_tf_inference_examples.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | """ 22 | Takes a trained model and performs inference on a few validation examples. 23 | """ 24 | import os 25 | os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2" # Get rid of the AVX, SSE warnings 26 | 27 | import numpy as np 28 | import tensorflow as tf 29 | import time 30 | from tensorflow import keras as K 31 | import settings 32 | import argparse 33 | from dataloader import DatasetGenerator, get_decathlon_filelist 34 | 35 | import matplotlib 36 | import matplotlib.pyplot as plt 37 | matplotlib.use("Agg") 38 | 39 | 40 | parser = argparse.ArgumentParser( 41 | description="TensorFlow Inference example for trained 2D U-Net model on BraTS.", 42 | add_help=True, formatter_class=argparse.ArgumentDefaultsHelpFormatter) 43 | 44 | parser.add_argument("--data_path", default=settings.DATA_PATH, 45 | help="the path to the data") 46 | parser.add_argument("--output_path", default=settings.OUT_PATH, 47 | help="the folder to save the model and checkpoints") 48 | parser.add_argument("--inference_filename", default=settings.INFERENCE_FILENAME, 49 | help="the TensorFlow inference model filename") 50 | parser.add_argument("--use_pconv",help="use partial convolution based padding", 51 | action="store_true", 52 | default=settings.USE_PCONV) 53 | parser.add_argument("--output_pngs", default="inference_examples", 54 | help="the directory for the output prediction pngs") 55 | 56 | parser.add_argument("--intraop_threads", default=settings.NUM_INTRA_THREADS, 57 | type=int, help="Number of intra-op-parallelism threads") 58 | parser.add_argument("--interop_threads", default=settings.NUM_INTER_THREADS, 59 | type=int, help="Number of inter-op-parallelism threads") 60 | parser.add_argument("--crop_dim", default=settings.CROP_DIM, 61 | type=int, help="Crop dimension for images") 62 | parser.add_argument("--seed", default=settings.SEED, 63 | type=int, help="Random seed") 64 | parser.add_argument("--split", type=float, default=settings.TRAIN_TEST_SPLIT, 65 | help="Train/testing split for the data") 66 | 67 | args = parser.parse_args() 68 | 69 | def test_intel_tensorflow(): 70 | """ 71 | Check if Intel version of TensorFlow is installed 72 | """ 73 | import tensorflow as tf 74 | 75 | print("We are using Tensorflow version {}".format(tf.__version__)) 76 | 77 | major_version = int(tf.__version__.split(".")[0]) 78 | if major_version >= 2: 79 | from tensorflow.python import _pywrap_util_port 80 | print("Intel-optimizations (DNNL) enabled:", _pywrap_util_port.IsMklEnabled()) 81 | else: 82 | print("Intel-optimizations (DNNL) enabled:", tf.pywrap_tensorflow.IsMklEnabled()) 83 | 84 | test_intel_tensorflow() 85 | 86 | 87 | def calc_dice(target, prediction, smooth=0.0001): 88 | """ 89 | Sorenson Dice 90 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 91 | where T is ground truth (target) mask and P is the prediction mask 92 | """ 93 | prediction = np.round(prediction) 94 | 95 | numerator = 2.0 * np.sum(target * prediction) + smooth 96 | denominator = np.sum(target) + np.sum(prediction) + smooth 97 | coef = numerator / denominator 98 | 99 | return coef 100 | 101 | 102 | def calc_soft_dice(target, prediction, smooth=0.0001): 103 | """ 104 | Sorensen (Soft) Dice coefficient - Don't round predictions 105 | """ 106 | numerator = 2.0 * np.sum(target * prediction) + smooth 107 | denominator = np.sum(target) + np.sum(prediction) + smooth 108 | coef = numerator / denominator 109 | 110 | return coef 111 | 112 | 113 | def plot_results(ds, batch_num, png_directory): 114 | 115 | plt.figure(figsize=(10,10)) 116 | 117 | img, msk = next(ds.ds) 118 | 119 | idx = np.argmax(np.sum(np.sum(msk[:,:,:,0], axis=1), axis=1)) # find the slice with the largest tumor 120 | 121 | plt.subplot(1, 3, 1) 122 | plt.imshow(img[idx, :, :, 0], cmap="bone", origin="lower") 123 | plt.title("MRI {}".format(idx), fontsize=20) 124 | 125 | plt.subplot(1, 3, 2) 126 | plt.imshow(msk[idx, :, :], cmap="bone", origin="lower") 127 | plt.title("Ground truth", fontsize=20) 128 | 129 | plt.subplot(1, 3, 3) 130 | 131 | print("Index {}: ".format(idx), end="") 132 | 133 | # Predict using the TensorFlow model 134 | start_time = time.time() 135 | prediction = model.predict(img[[idx]]) 136 | print("Elapsed time = {:.4f} msecs, ".format(1000.0*(time.time()-start_time)), end="") 137 | 138 | plt.imshow(prediction[0,:,:,0], cmap="bone", origin="lower") 139 | dice_coef = calc_dice(msk[idx], prediction) 140 | print("Dice coefficient = {:.4f}, ".format(dice_coef), end="") 141 | plt.title("Prediction\nDice = {:.4f}".format(dice_coef), fontsize=20) 142 | 143 | save_name = os.path.join(png_directory, "prediction_tf_{}_{}.png".format(batch_num, idx)) 144 | print("Saved as: {}".format(save_name)) 145 | plt.savefig(save_name) 146 | 147 | if __name__ == "__main__": 148 | 149 | model_filename = os.path.join(args.output_path, args.inference_filename) 150 | 151 | trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=args.data_path, seed=args.seed, split=args.split) 152 | 153 | ds_test = DatasetGenerator(testFiles, batch_size=128, crop_dim=[args.crop_dim,args.crop_dim], augment=False, seed=args.seed) 154 | 155 | # Load model 156 | if args.use_pconv: 157 | from model_pconv import unet 158 | unet_model = unet(use_pconv=True) 159 | else: 160 | from model import unet 161 | unet_model = unet() 162 | 163 | 164 | model = unet_model.load_model(model_filename) 165 | 166 | # Create output directory for images 167 | png_directory = args.output_pngs 168 | if not os.path.exists(png_directory): 169 | os.makedirs(png_directory) 170 | 171 | for batchnum in range(10): 172 | plot_results(ds_test, batchnum, png_directory) 173 | -------------------------------------------------------------------------------- /2D/settings.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | import psutil 21 | import os 22 | 23 | DATA_PATH=os.path.join("/data/medical_decathlon/Task01_BrainTumour") 24 | OUT_PATH = os.path.join("./output/") 25 | INFERENCE_FILENAME = "2d_unet_decathlon" 26 | 27 | EPOCHS = 30 # Number of epochs to train 28 | 29 | """ 30 | If the batch size is too small, then training is unstable. 31 | I believe this is because we are using 2D slicewise model. 32 | There are more slices without tumor than with tumor in the 33 | dataset so the data may become imbalanced if we choose too 34 | small of a batch size. There are, of course, many ways 35 | to handle imbalance training samples, but if we have 36 | enough memory, it is easiest just to select a sufficiently 37 | large batch size to make sure we have a few slices with 38 | tumors in each batch. 39 | """ 40 | BATCH_SIZE = 128 41 | 42 | # Using Adam optimizer 43 | LEARNING_RATE = 0.0001 # 0.00005 44 | WEIGHT_DICE_LOSS = 0.85 # Combined loss weight for dice versus BCE 45 | 46 | FEATURE_MAPS = 16 47 | PRINT_MODEL = True # Print the model 48 | 49 | # CPU specific parameters for multi-threading. 50 | # These can help take advantage of multi-core CPU systems 51 | # and significantly boosts training speed with MKL-DNN TensorFlow. 52 | BLOCKTIME = 0 53 | NUM_INTER_THREADS = 1 54 | # Default is to use the number of physical cores available 55 | 56 | # Figure out how many physical cores we have available 57 | # Minimum of either the CPU affinity or the number of physical cores 58 | import multiprocessing 59 | NUM_INTRA_THREADS = min(len(psutil.Process().cpu_affinity()), psutil.cpu_count(logical=False)) 60 | 61 | CROP_DIM=128 # Crop height and width to this size 62 | SEED=816 # Random seed 63 | TRAIN_TEST_SPLIT=0.80 # The train/test split 64 | 65 | CHANNELS_FIRST = False 66 | USE_UPSAMPLING = False 67 | USE_AUGMENTATION = True # Use data augmentation during training 68 | USE_DROPOUT = True # Use spatial dropout in model 69 | USE_PCONV = False # If True, Partial Convolution based padding will be used. See https://arxiv.org/pdf/1811.11718.pdf 70 | -------------------------------------------------------------------------------- /2D/train.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | """ 22 | This module loads the data from data.py, creates a TensorFlow/Keras model 23 | from model.py, trains the model on the data, and then saves the 24 | best model. 25 | """ 26 | 27 | import datetime 28 | import os 29 | 30 | import tensorflow as tf # conda install -c anaconda tensorflow 31 | import settings # Use the custom settings.py file for default parameters 32 | 33 | from dataloader import DatasetGenerator, get_decathlon_filelist 34 | 35 | import numpy as np 36 | 37 | from argparser import args 38 | 39 | """ 40 | For best CPU speed set the number of intra and inter threads 41 | to take advantage of multi-core systems. 42 | See https://github.com/intel/mkl-dnn 43 | """ 44 | 45 | os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2" # Get rid of the AVX, SSE warnings 46 | 47 | # If hyperthreading is enabled, then use 48 | os.environ["KMP_AFFINITY"] = "granularity=thread,compact,1,0" 49 | 50 | # If hyperthreading is NOT enabled, then use 51 | #os.environ["KMP_AFFINITY"] = "granularity=thread,compact" 52 | 53 | os.environ["KMP_BLOCKTIME"] = str(args.blocktime) 54 | 55 | os.environ["OMP_NUM_THREADS"] = str(args.num_threads) 56 | os.environ["INTRA_THREADS"] = str(args.num_threads) 57 | os.environ["INTER_THREADS"] = str(args.num_inter_threads) 58 | os.environ["KMP_SETTINGS"] = "0" # Show the settings at runtime 59 | 60 | def test_intel_tensorflow(): 61 | """ 62 | Check if Intel version of TensorFlow is installed 63 | """ 64 | import tensorflow as tf 65 | 66 | print("We are using Tensorflow version {}".format(tf.__version__)) 67 | 68 | major_version = int(tf.__version__.split(".")[0]) 69 | if major_version >= 2: 70 | from tensorflow.python import _pywrap_util_port 71 | print("Intel-optimizations (DNNL) enabled:", 72 | _pywrap_util_port.IsMklEnabled()) 73 | else: 74 | print("Intel-optimizations (DNNL) enabled:", 75 | tf.pywrap_tensorflow.IsMklEnabled()) 76 | 77 | if __name__ == "__main__": 78 | 79 | START_TIME = datetime.datetime.now() 80 | print("Started script on {}".format(START_TIME)) 81 | 82 | print("Runtime arguments = {}".format(args)) 83 | test_intel_tensorflow() # Print if we are using Intel-optimized TensorFlow 84 | 85 | """ 86 | Create a model, load the data, and train it. 87 | """ 88 | 89 | """ 90 | Step 1: Define a data loader 91 | """ 92 | print("-" * 30) 93 | print("Loading the data from the Medical Decathlon directory to a TensorFlow data loader ...") 94 | print("-" * 30) 95 | 96 | trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=args.data_path, seed=args.seed, split=args.split) 97 | 98 | ds_train = DatasetGenerator(trainFiles, batch_size=args.batch_size, crop_dim=[args.crop_dim,args.crop_dim], augment=True, seed=args.seed) 99 | ds_validation = DatasetGenerator(validateFiles, batch_size=args.batch_size, crop_dim=[args.crop_dim,args.crop_dim], augment=False, seed=args.seed) 100 | ds_test = DatasetGenerator(testFiles, batch_size=args.batch_size, crop_dim=[args.crop_dim,args.crop_dim], augment=False, seed=args.seed) 101 | 102 | print("-" * 30) 103 | print("Creating and compiling model ...") 104 | print("-" * 30) 105 | 106 | """ 107 | Step 2: Define the model 108 | """ 109 | if args.use_pconv: 110 | from model_pconv import unet 111 | else: 112 | from model import unet 113 | 114 | unet_model = unet(channels_first=args.channels_first, 115 | fms=args.featuremaps, 116 | output_path=args.output_path, 117 | inference_filename=args.inference_filename, 118 | learning_rate=args.learningrate, 119 | weight_dice_loss=args.weight_dice_loss, 120 | use_upsampling=args.use_upsampling, 121 | use_dropout=args.use_dropout, 122 | print_model=args.print_model) 123 | 124 | model = unet_model.create_model( 125 | ds_train.get_input_shape(), ds_train.get_output_shape()) 126 | 127 | model_filename, model_callbacks = unet_model.get_callbacks() 128 | 129 | """ 130 | Step 3: Train the model on the data 131 | """ 132 | print("-" * 30) 133 | print("Fitting model with training data ...") 134 | print("-" * 30) 135 | 136 | model.fit(ds_train, 137 | epochs=args.epochs, 138 | validation_data=ds_validation, 139 | verbose=1, 140 | callbacks=model_callbacks) 141 | 142 | """ 143 | Step 4: Evaluate the best model 144 | """ 145 | print("-" * 30) 146 | print("Loading the best trained model ...") 147 | print("-" * 30) 148 | 149 | unet_model.evaluate_model(model_filename, ds_test) 150 | 151 | """ 152 | Step 5: Print the command to convert TensorFlow model into OpenVINO format with model optimizer. 153 | """ 154 | print("-" * 30) 155 | print("-" * 30) 156 | unet_model.print_openvino_mo_command( 157 | model_filename, ds_test.get_input_shape()) 158 | 159 | print( 160 | "Total time elapsed for program = {} seconds".format( 161 | datetime.datetime.now() - 162 | START_TIME)) 163 | print("Stopped script on {}".format(datetime.datetime.now())) 164 | -------------------------------------------------------------------------------- /3D/README.md: -------------------------------------------------------------------------------- 1 | # 3D U-Net for Medical Decathlon Dataset 2 | 3 | ![pred152_3D](images/BRATS_152_img3D.gif 4 | "BRATS image #152: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives"). 5 | 6 | ## Trains a 3D U-Net on the brain tumor segmentation ([BraTS](https://www.med.upenn.edu/sbia/brats2017.html)) subset of the [Medical Segmentation Decathlon dataset](http://medicaldecathlon.com/) dataset. 7 | 8 | This model can achieve a [Dice coefficient](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1415224/) of > 0.80 on the whole tumor using just the [FLAIR](https://en.wikipedia.org/wiki/Fluid-attenuated_inversion_recovery) channel. 9 | 10 | ### Steps to train a new model: 11 | 12 | 1. Go to the [Medical Segmentation Decathlon](http://medicaldecathlon.com) website and download the [BraTS subset](https://drive.google.com/file/d/1A2IU8Sgea1h3fYLpYtFb2v7NYdMjvEhU/view?usp=sharing). The dataset has the [Creative Commons Attribution-ShareAlike 4.0 International license](https://creativecommons.org/licenses/by-sa/4.0/). 13 | 14 | 2. Untar the "Task01_BrainTumour.tar" file: 15 | ``` 16 | tar -xvf Task01_BrainTumour.tar 17 | ``` 18 | 3. We use [conda virtual environments](https://www.anaconda.com/distribution/#download-section) to run Python scripts. Once you download and install conda, create a new conda environment with [TensorFlow* with Intel® DNNL](https://software.intel.com/en-us/articles/intel-optimization-for-tensorflow-installation-guide?page=1). Run the command: 19 | ``` 20 | conda create -c anaconda -n decathlon pip python=3.7 tensorflow tqdm psutil jupyter matplotlib 21 | ``` 22 | 23 | This will create a new conda virtual environment called "decathlon" and install [TensorFlow* with Intel® DNNL](https://software.intel.com/en-us/articles/intel-optimization-for-tensorflow-installation-guide) for CPU training and inference. 24 | 25 | 4. Enable the new environment. Run the command: 26 | ``` 27 | conda activate decathlon 28 | ``` 29 | 5. Install the package [nibabel](http://nipy.org/nibabel/). Run the command: 30 | ``` 31 | pip install nibabel 32 | ``` 33 | 6. Run the command 34 | ``` 35 | python train.py --data_path $DECATHLON_ROOT_DIRECTORY 36 | ``` 37 | where `$DECATHLON_ROOT_DIRECTORY` is the root directory where you un-tarred the Decathlon dataset. 38 | 39 | ![commandline](images/3d_commandline.png) 40 | 41 | NOTE: The default settings take a [Height, Width, Depth] = [144, 144, 144] crop of the original image and mask using 8 images/masks per training batch. This requires over [40 gigabytes](images/training_memory_3d_unet.png) of memory to train the model. We trained our model on an Intel® Xeon® 8180 server with 384 GB of RAM. If you don't have enough memory or are getting out of memory (OOM) errors, you can pass `--tile_height=64 --tile_width=64 --tile_depth=64` to the `train.py` which will use a smaller ([64,64,64]) crop. You can also consider smaller batch sizes (e.g. `--batch_size=4` for a batch size of 4). 42 | 43 | ![tensorboard](images/tensorboard.png) 44 | 45 | 7. OpenVINO™ - At the end of `train.py` you should see instructions on how to [convert the model](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html) for use with the [Intel® Distribution of the OpenVINO™ toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html). Once you have OpenVINO™ installed, you can run a command like the one below to create an OpenVINO™ intermediate representation (IR) of the TensorFlow model. If you are using the [Intel® Neural Compute Stick™ (NCS2)](https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html), simply replace the `FP32` with `FP16` in the command below: 46 | 47 | ``` 48 | source /opt/intel/openvino_2021/bin/setupvars.sh 49 | python $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo_tf.py \ 50 | --saved_model_dir ./3d_unet_decathlon_final \ 51 | --batch 1 \ 52 | --model_name 3d_unet_model_for_decathlon \ 53 | --output_dir ./openvino_models/FP32 \ 54 | --data_type FP32 55 | ``` 56 | 57 | 8. Plot the predictions - A Jupyter Notebook (`plot_predictions.ipynb`) is provided which will load the testing dataset, the final TensorFlow model, and the OpenVINO™ model. It will perform inference using both models and compare the inference speed and outputs. 58 | 59 | ![plot_predictions](images/3d_unet_plot_predictions.png) 60 | 61 | ### Displaying the Results 62 | 63 | There are many programs that will display [Nifti](https://nifti.nimh.nih.gov/) 3D files. For the images above and below, the red overlay is the prediction from the model and the blue overlay is the ground truth mask. Any purple voxels are true positives. 64 | 65 | ![pred195](images/BRATS_195_img.gif "BRATS image #195: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives") 66 | 67 | ![pred152](images/BRATS_152.png "BRATS image #152: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives") 68 | 69 | ![pred426](images/BRATS_426.png "BRATS image #426: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives") 70 | 71 | 72 | REFERENCES: 73 | 1. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp Γ, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SM, Ryan M, Sarikaya D, Schwartz L, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Van Leemput K. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)", IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694 74 | 75 | 2. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, Freymann JB, Farahani K, Davatzikos C. "Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features", Nature Scientific Data, 4:170117 (2017) DOI: 10.1038/sdata.2017.117 76 | 77 | 3. Simpson AL, Antonelli M, Bakas S, Bilello M, Farahani K, van Ginneken B, Kopp-Schneider A, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, Bilic P, Christ PF, Do RKG, Gollub M, Golia-Pernicka J, Heckers SH, Jarnagin WR, McHugo MK, Napel S, Vorontsov E, Maier-Hein L, Cardoso MJ. "A large annotated medical image dataset for the development and evaluation of segmentation algorithms." https://arxiv.org/abs/1902.09063 78 | 79 | ### Optimization notice 80 | Please see our [optimization notice](https://software.intel.com/en-us/articles/optimization-notice#opt-en). 81 | 82 | -------------------------------------------------------------------------------- /3D/argparser.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | import settings 22 | import argparse 23 | 24 | parser = argparse.ArgumentParser( 25 | description="Train 3D U-Net model", add_help=True, 26 | formatter_class=argparse.ArgumentDefaultsHelpFormatter) 27 | 28 | parser.add_argument("--data_path", 29 | default=settings.DATA_PATH, 30 | help="Root directory for Medical Decathlon dataset") 31 | parser.add_argument("--epochs", 32 | type=int, 33 | default=settings.EPOCHS, 34 | help="Number of epochs") 35 | parser.add_argument("--saved_model_name", 36 | default=settings.SAVED_MODEL_NAME, 37 | help="Save model to this path") 38 | parser.add_argument("--batch_size", 39 | type=int, 40 | default=settings.BATCH_SIZE, 41 | help="Training batch size") 42 | parser.add_argument("--tile_height", 43 | type=int, 44 | default=settings.TILE_HEIGHT, 45 | help="Size of the 3D patch height") 46 | parser.add_argument("--tile_width", 47 | type=int, 48 | default=settings.TILE_WIDTH, 49 | help="Size of the 3D patch width") 50 | parser.add_argument("--tile_depth", 51 | type=int, 52 | default=settings.TILE_DEPTH, 53 | help="Size of the 3D patch depth") 54 | parser.add_argument("--number_input_channels", 55 | type=int, 56 | default=settings.NUMBER_INPUT_CHANNELS, 57 | help="Number of input channels") 58 | parser.add_argument("--number_output_classes", 59 | type=int, 60 | default=settings.NUMBER_OUTPUT_CLASSES, 61 | help="Number of output classes/channels") 62 | parser.add_argument("--train_test_split", 63 | type=float, 64 | default=settings.TRAIN_TEST_SPLIT, 65 | help="Train/test split (0-1)") 66 | parser.add_argument("--validate_test_split", 67 | type=float, 68 | default=settings.VALIDATE_TEST_SPLIT, 69 | help="Validation/test split (0-1)") 70 | parser.add_argument("--print_model", 71 | action="store_true", 72 | default=settings.PRINT_MODEL, 73 | help="Print the summary of the model layers") 74 | parser.add_argument("--filters", 75 | type=int, 76 | default=settings.FILTERS, 77 | help="Number of filters in the first convolutional layer") 78 | parser.add_argument("--use_upsampling", 79 | action="store_true", 80 | default=settings.USE_UPSAMPLING, 81 | help="Use upsampling instead of transposed convolution") 82 | parser.add_argument("--random_seed", 83 | default=settings.RANDOM_SEED, 84 | help="Random seed for determinism") 85 | 86 | args = parser.parse_args() 87 | -------------------------------------------------------------------------------- /3D/dataloader.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | import tensorflow as tf 21 | import numpy as np 22 | import settings 23 | 24 | import nibabel as nib 25 | 26 | 27 | class DatasetGenerator: 28 | 29 | def __init__(self, crop_dim, 30 | data_path=settings.DATA_PATH, 31 | batch_size=settings.BATCH_SIZE, 32 | train_test_split=settings.TRAIN_TEST_SPLIT, 33 | validate_test_split=settings.VALIDATE_TEST_SPLIT, 34 | number_output_classes=settings.NUMBER_OUTPUT_CLASSES, 35 | random_seed=settings.RANDOM_SEED, 36 | shard=0): 37 | 38 | self.data_path = data_path 39 | self.batch_size = batch_size 40 | self.crop_dim = crop_dim 41 | self.train_test_split = train_test_split 42 | self.validate_test_split = validate_test_split 43 | self.number_output_classes = number_output_classes 44 | self.random_seed = random_seed 45 | self.shard = shard # For Horovod, gives different shard per worker 46 | 47 | self.create_file_list() 48 | 49 | self.ds_train, self.ds_val, self.ds_test = self.get_dataset() 50 | 51 | def create_file_list(self): 52 | """ 53 | Get list of the files from the BraTS raw data 54 | Split into training and testing sets. 55 | """ 56 | import os 57 | import json 58 | 59 | json_filename = os.path.join(self.data_path, "dataset.json") 60 | 61 | try: 62 | with open(json_filename, "r") as fp: 63 | experiment_data = json.load(fp) 64 | except IOError as e: 65 | print("File {} doesn't exist. It should be part of the " 66 | "Decathlon directory".format(json_filename)) 67 | 68 | self.output_channels = experiment_data["labels"] 69 | self.input_channels = experiment_data["modality"] 70 | self.description = experiment_data["description"] 71 | self.name = experiment_data["name"] 72 | self.release = experiment_data["release"] 73 | self.license = experiment_data["licence"] 74 | self.reference = experiment_data["reference"] 75 | self.tensorImageSize = experiment_data["tensorImageSize"] 76 | self.numFiles = experiment_data["numTraining"] 77 | 78 | """ 79 | Create a dictionary of tuples with image filename and label filename 80 | """ 81 | self.filenames = {} 82 | for idx in range(self.numFiles): 83 | self.filenames[idx] = [os.path.join(self.data_path, 84 | experiment_data["training"][idx]["image"]), 85 | os.path.join(self.data_path, 86 | experiment_data["training"][idx]["label"])] 87 | 88 | def print_info(self): 89 | """ 90 | Print the dataset information 91 | """ 92 | 93 | print("="*30) 94 | print("Dataset name: ", self.name) 95 | print("Dataset description: ", self.description) 96 | print("Tensor image size: ", self.tensorImageSize) 97 | print("Dataset release: ", self.release) 98 | print("Dataset reference: ", self.reference) 99 | print("Input channels: ", self.input_channels) 100 | print("Output labels: ", self.output_channels) 101 | print("Dataset license: ", self.license) 102 | print("="*30) 103 | 104 | def z_normalize_img(self, img): 105 | """ 106 | Normalize the image so that the mean value for each image 107 | is 0 and the standard deviation is 1. 108 | """ 109 | for channel in range(img.shape[-1]): 110 | 111 | img_temp = img[..., channel] 112 | img_temp = (img_temp - np.mean(img_temp)) / np.std(img_temp) 113 | 114 | img[..., channel] = img_temp 115 | 116 | return img 117 | 118 | def crop(self, img, msk, randomize): 119 | """ 120 | Randomly crop the image and mask 121 | """ 122 | 123 | slices = [] 124 | 125 | # Do we randomize? 126 | is_random = randomize and np.random.rand() > 0.5 127 | 128 | for idx in range(len(img.shape)-1): # Go through each dimension 129 | 130 | cropLen = self.crop_dim[idx] 131 | imgLen = img.shape[idx] 132 | 133 | start = (imgLen-cropLen)//2 134 | 135 | ratio_crop = 0.20 # Crop up this this % of pixels for offset 136 | # Number of pixels to offset crop in this dimension 137 | offset = int(np.floor(start*ratio_crop)) 138 | 139 | if offset > 0: 140 | if is_random: 141 | start += np.random.choice(range(-offset, offset)) 142 | if ((start + cropLen) > imgLen): # Don't fall off the image 143 | start = (imgLen-cropLen)//2 144 | else: 145 | start = 0 146 | 147 | slices.append(slice(start, start+cropLen)) 148 | 149 | return img[tuple(slices)], msk[tuple(slices)] 150 | 151 | def augment_data(self, img, msk): 152 | """ 153 | Data augmentation 154 | Flip image and mask. Rotate image and mask. 155 | """ 156 | 157 | # Determine if axes are equal and can be rotated 158 | # If the axes aren't equal then we can't rotate them. 159 | equal_dim_axis = [] 160 | for idx in range(0, len(self.crop_dim)): 161 | for jdx in range(idx+1, len(self.crop_dim)): 162 | if self.crop_dim[idx] == self.crop_dim[jdx]: 163 | equal_dim_axis.append([idx, jdx]) # Valid rotation axes 164 | dim_to_rotate = equal_dim_axis 165 | 166 | if np.random.rand() > 0.5: 167 | # Random 0,1 (axes to flip) 168 | ax = np.random.choice(np.arange(len(self.crop_dim)-1)) 169 | img = np.flip(img, ax) 170 | msk = np.flip(msk, ax) 171 | 172 | elif (len(dim_to_rotate) > 0) and (np.random.rand() > 0.5): 173 | rot = np.random.choice([1, 2, 3]) # 90, 180, or 270 degrees 174 | 175 | # This will choose the axes to rotate 176 | # Axes must be equal in size 177 | random_axis = dim_to_rotate[np.random.choice(len(dim_to_rotate))] 178 | 179 | img = np.rot90(img, rot, axes=random_axis) # Rotate axes 0 and 1 180 | msk = np.rot90(msk, rot, axes=random_axis) # Rotate axes 0 and 1 181 | 182 | return img, msk 183 | 184 | def read_nifti_file(self, idx, randomize=False): 185 | """ 186 | Read Nifti file 187 | """ 188 | 189 | idx = idx.numpy() 190 | imgFile = self.filenames[idx][0] 191 | mskFile = self.filenames[idx][1] 192 | 193 | img = np.array(nib.load(imgFile).dataobj) 194 | 195 | img = np.rot90(img[..., [0]]) # Just take the FLAIR channel (0) 196 | 197 | msk = np.rot90(np.array(nib.load(mskFile).dataobj)) 198 | 199 | """ 200 | "labels": { 201 | "0": "background", 202 | "1": "edema", 203 | "2": "non-enhancing tumor", 204 | "3": "enhancing tumour"} 205 | """ 206 | # Combine all masks but background 207 | if self.number_output_classes == 1: 208 | msk[msk > 0] = 1.0 209 | msk = np.expand_dims(msk, -1) 210 | else: 211 | msk_temp = np.zeros(list(msk.shape) + [self.number_output_classes]) 212 | for channel in range(self.number_output_classes): 213 | msk_temp[msk == channel, channel] = 1.0 214 | msk = msk_temp 215 | 216 | # Crop 217 | img, msk = self.crop(img, msk, randomize) 218 | 219 | # Normalize 220 | img = self.z_normalize_img(img) 221 | 222 | # Randomly rotate 223 | if randomize: 224 | img, msk = self.augment_data(img, msk) 225 | 226 | return img, msk 227 | 228 | def plot_images(self, ds, slice_num=90): 229 | """ 230 | Plot images from dataset 231 | """ 232 | import matplotlib.pyplot as plt 233 | 234 | plt.figure(figsize=(20, 20)) 235 | 236 | num_cols = 2 237 | 238 | msk_channel = 1 239 | img_channel = 0 240 | 241 | for img, msk in ds.take(1): 242 | bs = img.shape[0] 243 | 244 | for idx in range(bs): 245 | plt.subplot(bs, num_cols, idx*num_cols + 1) 246 | plt.imshow(img[idx, :, :, slice_num, img_channel], cmap="bone") 247 | plt.title("MRI", fontsize=18) 248 | plt.subplot(bs, num_cols, idx*num_cols + 2) 249 | plt.imshow(msk[idx, :, :, slice_num, msk_channel], cmap="bone") 250 | plt.title("Tumor", fontsize=18) 251 | 252 | plt.show() 253 | 254 | print("Mean pixel value of image = {}".format( 255 | np.mean(img[0, :, :, :, 0]))) 256 | 257 | def display_train_images(self, slice_num=90): 258 | """ 259 | Plots some training images 260 | """ 261 | self.plot_images(self.ds_train, slice_num) 262 | 263 | def display_validation_images(self, slice_num=90): 264 | """ 265 | Plots some validation images 266 | """ 267 | self.plot_images(self.ds_val, slice_num) 268 | 269 | def display_test_images(self, slice_num=90): 270 | """ 271 | Plots some test images 272 | """ 273 | self.plot_images(self.ds_test, slice_num) 274 | 275 | def get_train(self): 276 | """ 277 | Return train dataset 278 | """ 279 | return self.ds_train 280 | 281 | def get_test(self): 282 | """ 283 | Return test dataset 284 | """ 285 | return self.ds_test 286 | 287 | def get_validate(self): 288 | """ 289 | Return validation dataset 290 | """ 291 | return self.ds_val 292 | 293 | def get_dataset(self): 294 | """ 295 | Create a TensorFlow data loader 296 | """ 297 | self.num_train = int(self.numFiles * self.train_test_split) 298 | numValTest = self.numFiles - self.num_train 299 | 300 | ds = tf.data.Dataset.range(self.numFiles).shuffle( 301 | self.numFiles, self.random_seed) # Shuffle the dataset 302 | 303 | """ 304 | Horovod Sharding 305 | Here we are not actually dividing the dataset into shards 306 | but instead just reshuffling the training dataset for every 307 | shard. Then in the training loop we just go through the training 308 | dataset but the number of steps is divided by the number of shards. 309 | """ 310 | ds_train = ds.take(self.num_train).shuffle( 311 | self.num_train, self.shard) # Reshuffle based on shard 312 | ds_val_test = ds.skip(self.num_train) 313 | self.num_val = int(numValTest * self.validate_test_split) 314 | self.num_test = self.num_train - self.num_val 315 | ds_val = ds_val_test.take(self.num_val) 316 | ds_test = ds_val_test.skip(self.num_val) 317 | 318 | ds_train = ds_train.map(lambda x: tf.py_function(self.read_nifti_file, 319 | [x, True], [tf.float32, tf.float32]), 320 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 321 | ds_val = ds_val.map(lambda x: tf.py_function(self.read_nifti_file, 322 | [x, False], [tf.float32, tf.float32]), 323 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 324 | ds_test = ds_test.map(lambda x: tf.py_function(self.read_nifti_file, 325 | [x, False], [tf.float32, tf.float32]), 326 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 327 | 328 | ds_train = ds_train.repeat() 329 | ds_train = ds_train.batch(self.batch_size) 330 | ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE) 331 | 332 | batch_size_val = 4 333 | ds_val = ds_val.batch(batch_size_val) 334 | ds_val = ds_val.prefetch(tf.data.experimental.AUTOTUNE) 335 | 336 | batch_size_test = 1 337 | ds_test = ds_test.batch(batch_size_test) 338 | ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE) 339 | 340 | return ds_train, ds_val, ds_test 341 | 342 | 343 | if __name__ == "__main__": 344 | 345 | print("Load the data and plot a few examples") 346 | 347 | from argparser import args 348 | 349 | crop_dim = (args.tile_height, args.tile_width, 350 | args.tile_depth, args.number_input_channels) 351 | 352 | """ 353 | Load the dataset 354 | """ 355 | brats_data = DatasetGenerator(crop_dim, 356 | data_path=args.data_path, 357 | batch_size=args.batch_size, 358 | train_test_split=args.train_test_split, 359 | validate_test_split=args.validate_test_split, 360 | number_output_classes=args.number_output_classes, 361 | random_seed=args.random_seed) 362 | 363 | brats_data.print_info() 364 | -------------------------------------------------------------------------------- /3D/horovod_command.sh: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2020 Intel Corporation 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # 15 | # SPDX-License-Identifier: EPL-2.0 16 | # 17 | 18 | horovodrun -np 4 -H localhost:4 --binding-args="--map-by ppr:2:socket:pe=10" --mpi-args="--report-bindings" python train_horovod.py 19 | 20 | -------------------------------------------------------------------------------- /3D/images/3d_commandline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/3d_commandline.png -------------------------------------------------------------------------------- /3D/images/3d_unet_plot_predictions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/3d_unet_plot_predictions.png -------------------------------------------------------------------------------- /3D/images/BRATS_105.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_105.png -------------------------------------------------------------------------------- /3D/images/BRATS_152.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_152.png -------------------------------------------------------------------------------- /3D/images/BRATS_152_img.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_152_img.avi -------------------------------------------------------------------------------- /3D/images/BRATS_152_img3D.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_152_img3D.avi -------------------------------------------------------------------------------- /3D/images/BRATS_152_img3D.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_152_img3D.gif -------------------------------------------------------------------------------- /3D/images/BRATS_195.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_195.png -------------------------------------------------------------------------------- /3D/images/BRATS_195_img.avi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_195_img.avi -------------------------------------------------------------------------------- /3D/images/BRATS_195_img.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_195_img.gif -------------------------------------------------------------------------------- /3D/images/BRATS_426.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/BRATS_426.png -------------------------------------------------------------------------------- /3D/images/tensorboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/tensorboard.png -------------------------------------------------------------------------------- /3D/images/training_memory_3d_unet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/unet/3c31ae1fa991be0d354db3f57bdd36dd54868eb0/3D/images/training_memory_3d_unet.png -------------------------------------------------------------------------------- /3D/model.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | from argparser import args 21 | import tensorflow as tf 22 | from tensorflow import keras as K 23 | 24 | 25 | def dice_coef(target, prediction, axis=(1, 2, 3), smooth=0.0001): 26 | """ 27 | Sorenson Dice 28 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 29 | where T is ground truth mask and P is the prediction mask 30 | """ 31 | prediction = tf.round(prediction) # Round to 0 or 1 32 | 33 | intersection = tf.reduce_sum(target * prediction, axis=axis) 34 | union = tf.reduce_sum(target + prediction, axis=axis) 35 | numerator = tf.constant(2.) * intersection + smooth 36 | denominator = union + smooth 37 | coef = numerator / denominator 38 | 39 | return tf.reduce_mean(coef) 40 | 41 | 42 | def soft_dice_coef(target, prediction, axis=(1, 2, 3), smooth=0.0001): 43 | """ 44 | Sorenson (Soft) Dice - Don't round predictions 45 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 46 | where T is ground truth mask and P is the prediction mask 47 | """ 48 | intersection = tf.reduce_sum(target * prediction, axis=axis) 49 | union = tf.reduce_sum(target + prediction, axis=axis) 50 | numerator = tf.constant(2.) * intersection + smooth 51 | denominator = union + smooth 52 | coef = numerator / denominator 53 | 54 | return tf.reduce_mean(coef) 55 | 56 | 57 | def dice_loss(target, prediction, axis=(1, 2, 3), smooth=0.0001): 58 | """ 59 | Sorenson (Soft) Dice loss 60 | Using -log(Dice) as the loss since it is better behaved. 61 | Also, the log allows avoidance of the division which 62 | can help prevent underflow when the numbers are very small. 63 | """ 64 | intersection = tf.reduce_sum(prediction * target, axis=axis) 65 | p = tf.reduce_sum(prediction, axis=axis) 66 | t = tf.reduce_sum(target, axis=axis) 67 | numerator = tf.reduce_mean(intersection + smooth) 68 | denominator = tf.reduce_mean(t + p + smooth) 69 | dice_loss = -tf.math.log(2.*numerator) + tf.math.log(denominator) 70 | 71 | return dice_loss 72 | 73 | 74 | def unet_3d(input_dim, filters=args.filters, 75 | number_output_classes=args.number_output_classes, 76 | use_upsampling=args.use_upsampling, 77 | concat_axis=-1, model_name=args.saved_model_name): 78 | """ 79 | 3D U-Net 80 | """ 81 | 82 | def ConvolutionBlock(x, name, filters, params): 83 | """ 84 | Convolutional block of layers 85 | Per the original paper this is back to back 3D convs 86 | with batch norm and then ReLU. 87 | """ 88 | 89 | x = K.layers.Conv3D(filters=filters, **params, name=name+"_conv0")(x) 90 | x = K.layers.BatchNormalization(name=name+"_bn0")(x) 91 | x = K.layers.Activation("relu", name=name+"_relu0")(x) 92 | 93 | x = K.layers.Conv3D(filters=filters, **params, name=name+"_conv1")(x) 94 | x = K.layers.BatchNormalization(name=name+"_bn1")(x) 95 | x = K.layers.Activation("relu", name=name)(x) 96 | 97 | return x 98 | 99 | inputs = K.layers.Input(shape=input_dim, name="MRImages") 100 | 101 | params = dict(kernel_size=(3, 3, 3), activation=None, 102 | padding="same", 103 | kernel_initializer="he_uniform") 104 | 105 | # Transposed convolution parameters 106 | params_trans = dict(kernel_size=(2, 2, 2), strides=(2, 2, 2), 107 | padding="same", 108 | kernel_initializer="he_uniform") 109 | 110 | # BEGIN - Encoding path 111 | encodeA = ConvolutionBlock(inputs, "encodeA", filters, params) 112 | poolA = K.layers.MaxPooling3D(name="poolA", pool_size=(2, 2, 2))(encodeA) 113 | 114 | encodeB = ConvolutionBlock(poolA, "encodeB", filters*2, params) 115 | poolB = K.layers.MaxPooling3D(name="poolB", pool_size=(2, 2, 2))(encodeB) 116 | 117 | encodeC = ConvolutionBlock(poolB, "encodeC", filters*4, params) 118 | poolC = K.layers.MaxPooling3D(name="poolC", pool_size=(2, 2, 2))(encodeC) 119 | 120 | encodeD = ConvolutionBlock(poolC, "encodeD", filters*8, params) 121 | poolD = K.layers.MaxPooling3D(name="poolD", pool_size=(2, 2, 2))(encodeD) 122 | 123 | encodeE = ConvolutionBlock(poolD, "encodeE", filters*16, params) 124 | # END - Encoding path 125 | 126 | # BEGIN - Decoding path 127 | if use_upsampling: 128 | up = K.layers.UpSampling3D(name="upE", size=(2, 2, 2))(encodeE) 129 | else: 130 | up = K.layers.Conv3DTranspose(name="transconvE", filters=filters*8, 131 | **params_trans)(encodeE) 132 | concatD = K.layers.concatenate( 133 | [up, encodeD], axis=concat_axis, name="concatD") 134 | 135 | decodeC = ConvolutionBlock(concatD, "decodeC", filters*8, params) 136 | 137 | if use_upsampling: 138 | up = K.layers.UpSampling3D(name="upC", size=(2, 2, 2))(decodeC) 139 | else: 140 | up = K.layers.Conv3DTranspose(name="transconvC", filters=filters*4, 141 | **params_trans)(decodeC) 142 | concatC = K.layers.concatenate( 143 | [up, encodeC], axis=concat_axis, name="concatC") 144 | 145 | decodeB = ConvolutionBlock(concatC, "decodeB", filters*4, params) 146 | 147 | if use_upsampling: 148 | up = K.layers.UpSampling3D(name="upB", size=(2, 2, 2))(decodeB) 149 | else: 150 | up = K.layers.Conv3DTranspose(name="transconvB", filters=filters*2, 151 | **params_trans)(decodeB) 152 | concatB = K.layers.concatenate( 153 | [up, encodeB], axis=concat_axis, name="concatB") 154 | 155 | decodeA = ConvolutionBlock(concatB, "decodeA", filters*2, params) 156 | 157 | if use_upsampling: 158 | up = K.layers.UpSampling3D(name="upA", size=(2, 2, 2))(decodeA) 159 | else: 160 | up = K.layers.Conv3DTranspose(name="transconvA", filters=filters, 161 | **params_trans)(decodeA) 162 | concatA = K.layers.concatenate( 163 | [up, encodeA], axis=concat_axis, name="concatA") 164 | 165 | # END - Decoding path 166 | 167 | convOut = ConvolutionBlock(concatA, "convOut", filters, params) 168 | 169 | prediction = K.layers.Conv3D(name="PredictionMask", 170 | filters=number_output_classes, 171 | kernel_size=(1, 1, 1), 172 | activation="sigmoid")(convOut) 173 | 174 | model = K.models.Model(inputs=[inputs], outputs=[prediction], 175 | name=model_name) 176 | 177 | if args.print_model: 178 | model.summary() 179 | 180 | return model 181 | -------------------------------------------------------------------------------- /3D/plot_predictions.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Comparing OpenVINO™ to TensorFlow Predictions\n", 8 | "\n", 9 | "In this notebook, we'll go through how to perform predictions on our 3D U-Net using both TensorFlow and OpenVINO™. You should be able to see that OpenVINO™ inference gives a significant speedup to these predictions.\n", 10 | "\n", 11 | "We'll assume that you already ran `train.py` and have trained a TensorFlow 3D U-Net model on the BraTS Medical Decathlon dataset. We'll further assume that you have converted the final TensorFlow 3D U-Net model to OpenVINO™ by running something like:\n", 12 | "\n", 13 | "```\n", 14 | "source /opt/intel/openvino/bin/setupvars.sh\n", 15 | "python $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo_tf.py \\\n", 16 | " --saved_model_dir 3d_unet_decathlon_final \\\n", 17 | " --model_name 3d_unet_decathlon \\\n", 18 | " --batch 1 \\\n", 19 | " --output_dir openvino_models/FP32 \\\n", 20 | " --data_type FP32\n", 21 | "```" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "## Import the OpenVINO™ Python API" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "from openvino.inference_engine import IECore" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "## Import some other Python libraries" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "import numpy as np\n", 54 | "import time\n", 55 | "import matplotlib.pyplot as plt\n", 56 | "%matplotlib inline" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "## Load the dataset\n", 64 | "\n", 65 | "Note: We'll reuse the same data loader we used in training. Nevertheless, all we need to do is to provide the 3D MRI scan with the same preprocessing (normalization, cropping, etc) as a NumPy array." 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": null, 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "from dataloader import DatasetGenerator\n", 75 | "import settings\n", 76 | "\n", 77 | "crop_dim = (settings.TILE_HEIGHT, settings.TILE_WIDTH,\n", 78 | " settings.TILE_DEPTH, settings.NUMBER_INPUT_CHANNELS)\n", 79 | "\n", 80 | "settings.BATCH_SIZE = 1 \n", 81 | "\n", 82 | "brats_data = DatasetGenerator(crop_dim=crop_dim,\n", 83 | " data_path=settings.DATA_PATH,\n", 84 | " batch_size=settings.BATCH_SIZE,\n", 85 | " train_test_split=settings.TRAIN_TEST_SPLIT,\n", 86 | " validate_test_split=settings.VALIDATE_TEST_SPLIT,\n", 87 | " number_output_classes=settings.NUMBER_OUTPUT_CLASSES,\n", 88 | " random_seed=settings.RANDOM_SEED)\n", 89 | "\n", 90 | "brats_data.print_info() # Print dataset information" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "## Load the OpenVINO™ model" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": null, 103 | "metadata": {}, 104 | "outputs": [], 105 | "source": [ 106 | "openvino_filename = \"openvino_models/FP32/3d_unet_decathlon\"\n", 107 | "path_to_xml_file = \"{}.xml\".format(openvino_filename)\n", 108 | "path_to_bin_file = \"{}.bin\".format(openvino_filename)\n", 109 | "\n", 110 | "ie = IECore()\n", 111 | "net = ie.read_network(model=path_to_xml_file, weights=path_to_bin_file)\n" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": {}, 117 | "source": [ 118 | "## Load the OpenVINO™ model to the hardware device\n", 119 | "\n", 120 | "In this case our device is `CPU`. We could also use `MYRIAD` for the Intel® NCS2™ VPU or `GPU` for the Intel® GPU." 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": null, 126 | "metadata": {}, 127 | "outputs": [], 128 | "source": [ 129 | "input_layer_name = next(iter(net.input_info))\n", 130 | "output_layer_name = next(iter(net.outputs))\n", 131 | "print(\"Input layer name = {}\\nOutput layer name = {}\".format(input_layer_name, output_layer_name))\n", 132 | "\n", 133 | "exec_net = ie.load_network(network=net, device_name=\"CPU\", num_requests=1)\n" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "## Load the final TensorFlow model" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": { 147 | "scrolled": true 148 | }, 149 | "outputs": [], 150 | "source": [ 151 | "import tensorflow as tf\n", 152 | "\n", 153 | "tf_model = tf.keras.models.load_model(\"3d_unet_decathlon_final\", compile=False)\n", 154 | "tf_model.compile(optimizer=\"adam\", loss=\"binary_crossentropy\")" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "def test_intel_tensorflow():\n", 164 | " \"\"\"\n", 165 | " Check if Intel version of TensorFlow is installed\n", 166 | " \"\"\"\n", 167 | " import tensorflow as tf\n", 168 | "\n", 169 | " print(\"We are using Tensorflow version {}\".format(tf.__version__))\n", 170 | "\n", 171 | " major_version = int(tf.__version__.split(\".\")[0])\n", 172 | " if major_version >= 2:\n", 173 | " from tensorflow.python import _pywrap_util_port\n", 174 | " print(\"Intel-optimizations (DNNL) enabled:\",\n", 175 | " _pywrap_util_port.IsMklEnabled())\n", 176 | " else:\n", 177 | " print(\"Intel-optimizations (DNNL) enabled:\",\n", 178 | " tf.pywrap_tensorflow.IsMklEnabled())\n", 179 | "\n", 180 | "\n", 181 | "test_intel_tensorflow() # Prints if Intel-optimized TensorFlow is used." 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "## Calculate the Dice coefficient\n", 189 | "\n", 190 | "This measures the performance of the model from 0 to 1 where 1 means the model gives a perfect prediction." 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": {}, 197 | "outputs": [], 198 | "source": [ 199 | "def calc_dice(target, prediction, smooth=0.0001):\n", 200 | " \"\"\"\n", 201 | " Sorenson Dice\n", 202 | " \"\"\"\n", 203 | " prediction = np.round(prediction)\n", 204 | "\n", 205 | " numerator = 2.0 * np.sum(target * prediction) + smooth\n", 206 | " denominator = np.sum(target) + np.sum(prediction) + smooth\n", 207 | " coef = numerator / denominator\n", 208 | "\n", 209 | " return coef" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "## Plot the predictions for both OpenVINO™ and TensorFlow\n", 217 | "\n", 218 | "We'll also time the inference to compare." 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "def plot_predictions(img, msk):\n", 228 | " \n", 229 | " slicenum=np.argmax(np.sum(msk, axis=(1,2))) # Find the slice with the largest tumor section\n", 230 | "\n", 231 | " plt.figure(figsize=(20,20))\n", 232 | "\n", 233 | " plt.subplot(1,4,1)\n", 234 | " plt.title(\"MRI\", fontsize=20)\n", 235 | " plt.imshow(img[0,:,:,slicenum,0], cmap=\"bone\")\n", 236 | " plt.subplot(1,4,2)\n", 237 | " plt.imshow(msk[0,:,:,slicenum,0], cmap=\"bone\")\n", 238 | " plt.title(\"Ground truth\", fontsize=20)\n", 239 | "\n", 240 | " \"\"\"\n", 241 | " OpenVINO Model Prediction\n", 242 | " Note: OpenVINO assumes the input (and output) are organized as channels first (NCHWD)\n", 243 | " whereas TensorFlow assumes channels last (NHWDC). We'll use the NumPy transpose\n", 244 | " to change the order.\n", 245 | " \"\"\"\n", 246 | " start_time = time.time()\n", 247 | " res = exec_net.infer({input_layer_name: np.transpose(img, [0,4,1,2,3])})\n", 248 | " prediction_ov = np.transpose(res[output_layer_name], [0,2,3,4,1]) \n", 249 | " print(\"OpenVINO inference time = {:.4f} msecs\".format(1000.0*(time.time()-start_time)))\n", 250 | "\n", 251 | " plt.subplot(1,4,3)\n", 252 | " dice_coef_ov = calc_dice(msk,prediction_ov)\n", 253 | " plt.imshow(prediction_ov[0,:,:,slicenum,0], cmap=\"bone\")\n", 254 | " plt.title(\"OpenVINO Prediction\\nDice = {:.4f}\".format(dice_coef_ov), fontsize=20)\n", 255 | " \n", 256 | " \n", 257 | " \"\"\"\n", 258 | " TensorFlow Model Prediction\n", 259 | " \"\"\"\n", 260 | " start_time = time.time()\n", 261 | " prediction_tf = tf_model.predict(img)\n", 262 | " print(\"TensorFlow inference time = {:.4f} msecs\".format(1000.0*(time.time()-start_time)))\n", 263 | " \n", 264 | " plt.subplot(1,4,4)\n", 265 | " dice_coef_tf = calc_dice(msk,prediction_tf)\n", 266 | " plt.imshow(prediction_tf[0,:,:,slicenum,0], cmap=\"bone\")\n", 267 | " plt.title(\"TensorFlow Prediction\\nDice = {:.4f}\".format(dice_coef_tf), fontsize=20)" 268 | ] 269 | }, 270 | { 271 | "cell_type": "markdown", 272 | "metadata": {}, 273 | "source": [ 274 | "## Inference time\n", 275 | "\n", 276 | "Let's grab some data, perform inference, and plot the results." 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": null, 282 | "metadata": { 283 | "scrolled": true 284 | }, 285 | "outputs": [], 286 | "source": [ 287 | "ds = brats_data.get_test().take(1).as_numpy_iterator()\n", 288 | "for img, msk in ds:\n", 289 | " plot_predictions(img,msk)" 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": null, 295 | "metadata": {}, 296 | "outputs": [], 297 | "source": [ 298 | "ds = brats_data.get_test().take(1).as_numpy_iterator()\n", 299 | "for img, msk in ds:\n", 300 | " plot_predictions(img,msk)" 301 | ] 302 | }, 303 | { 304 | "cell_type": "code", 305 | "execution_count": null, 306 | "metadata": {}, 307 | "outputs": [], 308 | "source": [ 309 | "ds = brats_data.get_test().take(1).as_numpy_iterator()\n", 310 | "for img, msk in ds:\n", 311 | " plot_predictions(img,msk)" 312 | ] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "execution_count": null, 317 | "metadata": {}, 318 | "outputs": [], 319 | "source": [ 320 | "ds = brats_data.get_test().take(1).as_numpy_iterator()\n", 321 | "for img, msk in ds:\n", 322 | " plot_predictions(img,msk)" 323 | ] 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": {}, 328 | "source": [ 329 | "*Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. SPDX-License-Identifier: EPL-2.0*\n", 330 | "\n", 331 | "*Copyright (c) 2019-2020 Intel Corporation*" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [] 340 | } 341 | ], 342 | "metadata": { 343 | "kernelspec": { 344 | "display_name": "Python 3", 345 | "language": "python", 346 | "name": "python3" 347 | }, 348 | "language_info": { 349 | "codemirror_mode": { 350 | "name": "ipython", 351 | "version": 3 352 | }, 353 | "file_extension": ".py", 354 | "mimetype": "text/x-python", 355 | "name": "python", 356 | "nbconvert_exporter": "python", 357 | "pygments_lexer": "ipython3", 358 | "version": "3.7.9" 359 | } 360 | }, 361 | "nbformat": 4, 362 | "nbformat_minor": 4 363 | } 364 | -------------------------------------------------------------------------------- /3D/run_unet_horovod.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2019 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | source ~/.bashrc 22 | conda activate decathlon 23 | cd /home/unet/unet/3D 24 | python train_horovod.py --batch_size 4 --data_path /home/unet/data/decathlon/Task01_BrainTumour/ 25 | 26 | conda deactivate 27 | -------------------------------------------------------------------------------- /3D/script_slurm: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | #SBATCH --time=48:00:00 3 | #SBATCH --job-name=3d-unet-horovod 4 | #SBATCH -o /home/unet/logs/%x_%j.out # %x job name, %N for node name, %j for jobID 5 | #SBATCH --partition=6252 6 | #SBATCH --nodes=10 7 | 8 | data_path=/mnt/lustrefs/unet/decathlon/Task01_BrainTumour/ 9 | script_path=/home/unet/scripts/unet/3D/train_horovod.py 10 | 11 | mpirun -n 20 -ppn 2 -print-rank-map -genv I_MPI_PIN_DOMAIN=socket -genv OMP_NUM_THREADS=24 -genv OMP_PROC_BIND=true -genv KMP_BLOCKTIME=1 python3 $script_path --intraop_threads 24 --data_path $data_path 12 | -------------------------------------------------------------------------------- /3D/settings.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Copyright (c) 2020 Intel Corporation 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | # 17 | # SPDX-License-Identifier: EPL-2.0 18 | # 19 | 20 | DATA_PATH="../data/decathlon/Task01_BrainTumour/" 21 | SAVED_MODEL_NAME="3d_unet_decathlon" 22 | 23 | EPOCHS=40 24 | BATCH_SIZE=8 25 | TILE_HEIGHT=144 26 | TILE_WIDTH=144 27 | TILE_DEPTH=144 28 | NUMBER_INPUT_CHANNELS=1 29 | NUMBER_OUTPUT_CLASSES=1 30 | 31 | TRAIN_TEST_SPLIT=0.80 32 | VALIDATE_TEST_SPLIT=0.50 33 | 34 | PRINT_MODEL=False 35 | FILTERS=16 36 | USE_UPSAMPLING=False 37 | 38 | RANDOM_SEED=816 -------------------------------------------------------------------------------- /3D/train.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | import tensorflow as tf # TensorFlow 2 21 | from tensorflow import keras as K 22 | 23 | import os 24 | import datetime 25 | 26 | from argparser import args 27 | from dataloader import DatasetGenerator 28 | from model import dice_coef, soft_dice_coef, dice_loss, unet_3d 29 | 30 | 31 | def test_intel_tensorflow(): 32 | """ 33 | Check if Intel version of TensorFlow is installed 34 | """ 35 | import tensorflow as tf 36 | 37 | print("We are using Tensorflow version {}".format(tf.__version__)) 38 | 39 | major_version = int(tf.__version__.split(".")[0]) 40 | if major_version >= 2: 41 | from tensorflow.python import _pywrap_util_port 42 | print("Intel-optimizations (DNNL) enabled:", 43 | _pywrap_util_port.IsMklEnabled()) 44 | else: 45 | print("Intel-optimizations (DNNL) enabled:", 46 | tf.pywrap_tensorflow.IsMklEnabled()) 47 | 48 | print(args) 49 | test_intel_tensorflow() # Prints if Intel-optimized TensorFlow is used. 50 | 51 | """ 52 | crop_dim = Dimensions to crop the input tensor 53 | """ 54 | crop_dim = (args.tile_height, args.tile_width, 55 | args.tile_depth, args.number_input_channels) 56 | 57 | """ 58 | 1. Load the dataset 59 | """ 60 | brats_data = DatasetGenerator(crop_dim, 61 | data_path=args.data_path, 62 | batch_size=args.batch_size, 63 | train_test_split=args.train_test_split, 64 | validate_test_split=args.validate_test_split, 65 | number_output_classes=args.number_output_classes, 66 | random_seed=args.random_seed) 67 | 68 | brats_data.print_info() # Print dataset information 69 | 70 | """ 71 | 2. Create the TensorFlow model 72 | """ 73 | model = unet_3d(input_dim=crop_dim, filters=args.filters, 74 | number_output_classes=args.number_output_classes, 75 | use_upsampling=args.use_upsampling, 76 | concat_axis=-1, model_name=args.saved_model_name) 77 | 78 | local_opt = K.optimizers.Adam() 79 | model.compile(loss=dice_loss, 80 | metrics=[dice_coef, soft_dice_coef], 81 | optimizer=local_opt) 82 | 83 | checkpoint = K.callbacks.ModelCheckpoint(args.saved_model_name, 84 | verbose=1, 85 | save_best_only=True) 86 | 87 | # TensorBoard 88 | logs_dir = os.path.join( 89 | "logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S")) 90 | tb_logs = K.callbacks.TensorBoard(log_dir=logs_dir) 91 | 92 | callbacks = [checkpoint, tb_logs] 93 | 94 | """ 95 | 3. Train the model 96 | """ 97 | steps_per_epoch = brats_data.num_train // args.batch_size 98 | model.fit(brats_data.get_train(), epochs=args.epochs, 99 | steps_per_epoch=steps_per_epoch, 100 | validation_data=brats_data.get_validate(), 101 | callbacks=callbacks, 102 | verbose=1) 103 | 104 | """ 105 | 4. Load best model on validation dataset and run on the test 106 | dataset to show generalizability 107 | """ 108 | best_model = K.models.load_model(args.saved_model_name, 109 | custom_objects={"dice_loss": dice_loss, 110 | "dice_coef": dice_coef, 111 | "soft_dice_coef": soft_dice_coef}) 112 | 113 | print("\n\nEvaluating best model on the testing dataset.") 114 | print("=============================================") 115 | loss, dice_coef, soft_dice_coef = best_model.evaluate(brats_data.get_test()) 116 | 117 | print("Average Dice Coefficient on testing dataset = {:.4f}".format(dice_coef)) 118 | 119 | """ 120 | 5. Save the best model without the custom objects (dice, etc.) 121 | NOTE: You should be able to do .load_model(compile=False), but this 122 | appears to currently be broken in TF2. To compensate, we're 123 | just going to re-compile the model without the custom objects and 124 | save as a new model (with suffix "_final") 125 | """ 126 | final_model_name = args.saved_model_name + "_final" 127 | best_model.compile(loss="binary_crossentropy", metrics=["accuracy"], 128 | optimizer="adam") 129 | K.models.save_model(best_model, final_model_name, 130 | include_optimizer=False) 131 | 132 | """ 133 | Converting the model to OpenVINO 134 | """ 135 | print("\n\nConvert the TensorFlow model to OpenVINO by running:\n") 136 | print("source /opt/intel/openvino/bin/setupvars.sh") 137 | print("python $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo_tf.py \\") 138 | print(" --saved_model_dir {} \\".format(final_model_name)) 139 | print(" --model_name {} \\".format(args.saved_model_name)) 140 | print(" --batch 1 \\") 141 | print(" --output_dir {} \\".format(os.path.join("openvino_models", "FP32"))) 142 | print(" --data_type FP32\n\n") 143 | 144 | -------------------------------------------------------------------------------- /3D/train_horovod.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | from model import dice_coef, soft_dice_coef, dice_loss, unet_3d 21 | from dataloader import DatasetGenerator 22 | from argparser import args 23 | import datetime 24 | import os 25 | import tensorflow as tf # TensorFlow 2 26 | from tensorflow import keras as K 27 | import horovod.tensorflow.keras as hvd 28 | 29 | hvd.init() 30 | 31 | 32 | def test_intel_tensorflow(): 33 | """ 34 | Check if Intel version of TensorFlow is installed 35 | """ 36 | import tensorflow as tf 37 | 38 | print("We are using Tensorflow version {}".format(tf.__version__)) 39 | 40 | major_version = int(tf.__version__.split(".")[0]) 41 | if major_version >= 2: 42 | from tensorflow.python import _pywrap_util_port 43 | print("Intel-optimizations (DNNL) enabled:", 44 | _pywrap_util_port.IsMklEnabled()) 45 | else: 46 | print("Intel-optimizations (DNNL) enabled:", 47 | tf.pywrap_tensorflow.IsMklEnabled()) 48 | 49 | 50 | test_intel_tensorflow() # Prints if Intel-optimized TensorFlow is used. 51 | 52 | """ 53 | crop_dim = Dimensions to crop the input tensor 54 | """ 55 | crop_dim = (args.tile_height, args.tile_width, 56 | args.tile_depth, args.number_input_channels) 57 | 58 | """ 59 | 1. Load the dataset 60 | """ 61 | brats_data = DatasetGenerator(crop_dim, 62 | data_path=args.data_path, 63 | batch_size=args.batch_size, 64 | train_test_split=args.train_test_split, 65 | validate_test_split=args.validate_test_split, 66 | number_output_classes=args.number_output_classes, 67 | random_seed=args.random_seed, 68 | shard=hvd.rank()) 69 | 70 | if (hvd.rank() == 0): 71 | print("{} workers".format(hvd.size())) 72 | brats_data.print_info() # Print dataset information 73 | 74 | """ 75 | 2. Create the TensorFlow model 76 | """ 77 | model = unet_3d(input_dim=crop_dim, filters=args.filters, 78 | number_output_classes=args.number_output_classes, 79 | use_upsampling=args.use_upsampling, 80 | concat_axis=-1, model_name=args.saved_model_name) 81 | 82 | local_opt = K.optimizers.Adam() 83 | hvd_opt = hvd.DistributedOptimizer(local_opt) 84 | 85 | model.compile(loss=dice_loss, 86 | metrics=[dice_coef, soft_dice_coef], 87 | optimizer=hvd_opt) 88 | 89 | checkpoint = K.callbacks.ModelCheckpoint(args.saved_model_name, 90 | verbose=1, 91 | save_best_only=True) 92 | 93 | # TensorBoard 94 | logs_dir = os.path.join( 95 | "logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S")) 96 | tb_logs = K.callbacks.TensorBoard(log_dir=logs_dir) 97 | 98 | hvd_callbacks = hvd.callbacks.BroadcastGlobalVariablesCallback(0) 99 | 100 | if (hvd.rank() == 0): 101 | callbacks = [checkpoint, tb_logs, hvd_callbacks] 102 | else: 103 | callbacks = [hvd_callbacks] 104 | 105 | """ 106 | 3. Train the model 107 | """ 108 | steps_per_epoch = brats_data.num_train // (hvd.size() * args.batch_size) 109 | if steps_per_epoch < 2: # Make sure we have at least 2 steps per epoch 110 | steps_per_epoch = 2 111 | 112 | model.fit(brats_data.get_train(), epochs=args.epochs, 113 | steps_per_epoch=steps_per_epoch, 114 | validation_data=brats_data.get_validate(), 115 | callbacks=callbacks, 116 | verbose=1 if hvd.rank() == 0 else 0) 117 | 118 | 119 | if (hvd.rank() == 0): 120 | 121 | """ 122 | 4. Load best model on validation dataset and run on the test 123 | dataset to show generalizability 124 | """ 125 | 126 | best_model = K.models.load_model(args.saved_model_name, 127 | custom_objects={"dice_loss": dice_loss, 128 | "dice_coef": dice_coef, 129 | "soft_dice_coef": soft_dice_coef}) 130 | 131 | loss, dice_coef, soft_dice_coef = best_model.evaluate( 132 | brats_data.get_test()) 133 | 134 | print( 135 | "Average Dice Coefficient on test dataset = {:.4f}".format(dice_coef)) 136 | 137 | """ 138 | 5. Save the best model without the custom objects (dice, etc.) 139 | NOTE: You should be able to do .load_model(compile=False), but this 140 | appears to currently be broken in TF2. To compensate, we're 141 | just going to re-compile the model without the custom objects and 142 | save as a new model (with suffix "_final") 143 | """ 144 | best_model.compile(loss="binary_crossentropy", metrics=["accuracy"], 145 | optimizer="adam") 146 | K.models.save_model(best_model, args.saved_model_name + "_final", 147 | include_optimizer=False) 148 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DISCONTINUATION OF PROJECT # 2 | This project will no longer be maintained by Intel. 3 | Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project. 4 | Intel no longer accepts patches to this project. 5 | If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project. 6 | 7 | # Deep Learning Medical Decathlon Demos for Python* 8 | ### U-Net Biomedical Image Segmentation with Medical Decathlon Dataset. 9 | 10 | This repository contains [2D](https://github.com/IntelAI/unet/tree/master/2D) and [3D](https://github.com/IntelAI/unet/tree/master/3D) U-Net TensorFlow scripts for training models using the [Medical Decathlon](http://medicaldecathlon.com/) dataset (http://medicaldecathlon.com/). 11 | 12 | ![pred152_3D](3D/images/BRATS_152_img3D.gif 13 | "BRATS image #152: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives"). ![pred195](3D/images/BRATS_195_img.gif "BRATS image #195: Purple voxels indicate a perfect prediction by the model. Red are false positives. Blue are false negatives") 14 | 15 | ### Citation 16 | David Ojika, Bhavesh Patel, G. Athony Reina, Trent Boyer, Chad Martin and Prashant Shah. “Addressing the Memory Bottleneck in AI Model Training”, Workshop on MLOps Systems, Austin TX (2020) held in conjunction with Third Conference on Machine Learning and Systems (MLSys). https://arxiv.org/abs/2003.08732 17 | -------------------------------------------------------------------------------- /single-node/3D_BraTS2019/argparser.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | 21 | import settings 22 | import argparse 23 | 24 | parser = argparse.ArgumentParser( 25 | description="Train 3D U-Net model", add_help=True, 26 | formatter_class=argparse.ArgumentDefaultsHelpFormatter) 27 | 28 | parser.add_argument("--data_path", 29 | default=settings.DATA_PATH, 30 | help="Root directory for BraTS dataset") 31 | parser.add_argument("--csv_list", 32 | default=settings.CSV_LIST, 33 | help="CSV file with image and label filenames") 34 | parser.add_argument("--epochs", 35 | type=int, 36 | default=settings.EPOCHS, 37 | help="Number of epochs") 38 | parser.add_argument("--saved_model_name", 39 | default=settings.SAVED_MODEL_NAME, 40 | help="Save model to this path") 41 | parser.add_argument("--batch_size", 42 | type=int, 43 | default=settings.BATCH_SIZE, 44 | help="Training batch size") 45 | parser.add_argument("--tile_height", 46 | type=int, 47 | default=settings.TILE_HEIGHT, 48 | help="Size of the 3D patch height") 49 | parser.add_argument("--tile_width", 50 | type=int, 51 | default=settings.TILE_WIDTH, 52 | help="Size of the 3D patch width") 53 | parser.add_argument("--tile_depth", 54 | type=int, 55 | default=settings.TILE_DEPTH, 56 | help="Size of the 3D patch depth") 57 | parser.add_argument("--number_input_channels", 58 | type=int, 59 | default=settings.NUMBER_INPUT_CHANNELS, 60 | help="Number of input channels") 61 | parser.add_argument("--number_output_classes", 62 | type=int, 63 | default=settings.NUMBER_OUTPUT_CLASSES, 64 | help="Number of output classes/channels") 65 | parser.add_argument("--train_test_split", 66 | type=float, 67 | default=settings.TRAIN_TEST_SPLIT, 68 | help="Train/test split (0-1)") 69 | parser.add_argument("--validate_test_split", 70 | type=float, 71 | default=settings.VALIDATE_TEST_SPLIT, 72 | help="Validation/test split (0-1)") 73 | parser.add_argument("--print_model", 74 | action="store_true", 75 | default=settings.PRINT_MODEL, 76 | help="Print the summary of the model layers") 77 | parser.add_argument("--filters", 78 | type=int, 79 | default=settings.FILTERS, 80 | help="Number of filters in the first convolutional layer") 81 | parser.add_argument("--use_upsampling", 82 | action="store_true", 83 | default=settings.USE_UPSAMPLING, 84 | help="Use upsampling instead of transposed convolution") 85 | parser.add_argument("--random_seed", 86 | default=settings.RANDOM_SEED, 87 | help="Random seed for determinism") 88 | 89 | args = parser.parse_args() 90 | -------------------------------------------------------------------------------- /single-node/3D_BraTS2019/dataloader.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | import tensorflow as tf 21 | import numpy as np 22 | import settings 23 | 24 | import nibabel as nib 25 | 26 | 27 | class DatasetGenerator: 28 | 29 | def __init__(self, crop_dim, 30 | data_path=settings.DATA_PATH, 31 | csv_list=settings.CSV_LIST, 32 | batch_size=settings.BATCH_SIZE, 33 | train_test_split=settings.TRAIN_TEST_SPLIT, 34 | validate_test_split=settings.VALIDATE_TEST_SPLIT, 35 | number_output_classes=settings.NUMBER_OUTPUT_CLASSES, 36 | random_seed=settings.RANDOM_SEED, 37 | shard=0): 38 | 39 | self.data_path = data_path 40 | self.csv_list = csv_list 41 | self.batch_size = batch_size 42 | self.crop_dim = crop_dim 43 | self.train_test_split = train_test_split 44 | self.validate_test_split = validate_test_split 45 | self.number_output_classes = number_output_classes 46 | self.random_seed = random_seed 47 | self.shard = shard # For Horovod, gives different shard per worker 48 | 49 | self.create_file_list() 50 | 51 | self.ds_train, self.ds_val, self.ds_test = self.get_dataset() 52 | 53 | def create_file_list(self): 54 | """ 55 | Get list of the files from the BraTS raw data 56 | Split into training and testing sets. 57 | Reads the BraTS files to use from a CSV file list 58 | """ 59 | import os, csv 60 | 61 | self.filenames = {} 62 | 63 | with open(self.csv_list, 'r') as csv_file: 64 | csv_reader = csv.reader(csv_file, delimiter=',') 65 | idx = 0 66 | next(csv_reader) # skip header 67 | for row in csv_reader: 68 | if len(row[0]) > 0: # Skip blank lines 69 | self.filenames[idx] = [os.path.join(self.data_path, row[0]), os.path.join(self.data_path, row[1])] 70 | idx += 1 71 | 72 | self.numFiles = idx 73 | 74 | def z_normalize_img(self, img): 75 | """ 76 | Normalize the image so that the mean value for each image 77 | is 0 and the standard deviation is 1. 78 | """ 79 | for channel in range(img.shape[-1]): 80 | 81 | img_temp = img[..., channel] 82 | img_temp = (img_temp - np.mean(img_temp)) / np.std(img_temp) 83 | 84 | img[..., channel] = img_temp 85 | 86 | return img 87 | 88 | def crop(self, img, msk, randomize): 89 | """ 90 | Randomly crop the image and mask 91 | """ 92 | 93 | slices = [] 94 | 95 | # Do we randomize? 96 | is_random = randomize and np.random.rand() > 0.5 97 | 98 | for idx in range(len(img.shape)-1): # Go through each dimension 99 | 100 | cropLen = self.crop_dim[idx] 101 | imgLen = img.shape[idx] 102 | 103 | start = (imgLen-cropLen)//2 104 | 105 | ratio_crop = 0.20 # Crop up this this % of pixels for offset 106 | # Number of pixels to offset crop in this dimension 107 | offset = int(np.floor(start*ratio_crop)) 108 | 109 | if offset > 0: 110 | if is_random: 111 | start += np.random.choice(range(-offset, offset)) 112 | if ((start + cropLen) > imgLen): # Don't fall off the image 113 | start = (imgLen-cropLen)//2 114 | else: 115 | start = 0 116 | 117 | slices.append(slice(start, start+cropLen)) 118 | 119 | return img[tuple(slices)], msk[tuple(slices)] 120 | 121 | def augment_data(self, img, msk): 122 | """ 123 | Data augmentation 124 | Flip image and mask. Rotate image and mask. 125 | """ 126 | 127 | # Determine if axes are equal and can be rotated 128 | # If the axes aren't equal then we can't rotate them. 129 | equal_dim_axis = [] 130 | for idx in range(0, len(self.crop_dim)): 131 | for jdx in range(idx+1, len(self.crop_dim)): 132 | if self.crop_dim[idx] == self.crop_dim[jdx]: 133 | equal_dim_axis.append([idx, jdx]) # Valid rotation axes 134 | dim_to_rotate = equal_dim_axis 135 | 136 | if np.random.rand() > 0.5: 137 | # Random 0,1 (axes to flip) 138 | ax = np.random.choice(np.arange(len(self.crop_dim)-1)) 139 | img = np.flip(img, ax) 140 | msk = np.flip(msk, ax) 141 | 142 | elif (len(dim_to_rotate) > 0) and (np.random.rand() > 0.5): 143 | rot = np.random.choice([1, 2, 3]) # 90, 180, or 270 degrees 144 | 145 | # This will choose the axes to rotate 146 | # Axes must be equal in size 147 | random_axis = dim_to_rotate[np.random.choice(len(dim_to_rotate))] 148 | 149 | img = np.rot90(img, rot, axes=random_axis) # Rotate axes 0 and 1 150 | msk = np.rot90(msk, rot, axes=random_axis) # Rotate axes 0 and 1 151 | 152 | return img, msk 153 | 154 | def read_nifti_file(self, idx, randomize=False): 155 | """ 156 | Read Nifti file 157 | """ 158 | 159 | idx = idx.numpy() 160 | imgFile = self.filenames[idx][0] 161 | mskFile = self.filenames[idx][1] 162 | 163 | img = np.array(nib.load(imgFile).dataobj) 164 | 165 | img = np.expand_dims(np.rot90(img), -1) 166 | 167 | msk = np.rot90(np.array(nib.load(mskFile).dataobj)) 168 | 169 | """ 170 | "labels": { 171 | "0": "background", 172 | "1": "edema", 173 | "2": "non-enhancing tumor", 174 | "3": "enhancing tumour"} 175 | """ 176 | # Combine all masks but background 177 | if self.number_output_classes == 1: 178 | msk[msk > 0] = 1.0 179 | msk = np.expand_dims(msk, -1) 180 | else: 181 | msk_temp = np.zeros(list(msk.shape) + [self.number_output_classes]) 182 | for channel in range(self.number_output_classes): 183 | msk_temp[msk == channel, channel] = 1.0 184 | msk = msk_temp 185 | 186 | # Crop 187 | img, msk = self.crop(img, msk, randomize) 188 | 189 | # Normalize 190 | img = self.z_normalize_img(img) 191 | 192 | # Randomly rotate 193 | if randomize: 194 | img, msk = self.augment_data(img, msk) 195 | 196 | return img, msk 197 | 198 | def plot_images(self, ds, slice_num=90): 199 | """ 200 | Plot images from dataset 201 | """ 202 | import matplotlib.pyplot as plt 203 | 204 | plt.figure(figsize=(20, 20)) 205 | 206 | num_cols = 2 207 | 208 | msk_channel = 1 209 | img_channel = 0 210 | 211 | for img, msk in ds.take(1): 212 | bs = img.shape[0] 213 | 214 | for idx in range(bs): 215 | plt.subplot(bs, num_cols, idx*num_cols + 1) 216 | plt.imshow(img[idx, :, :, slice_num, img_channel], cmap="bone") 217 | plt.title("MRI", fontsize=18) 218 | plt.subplot(bs, num_cols, idx*num_cols + 2) 219 | plt.imshow(msk[idx, :, :, slice_num, msk_channel], cmap="bone") 220 | plt.title("Tumor", fontsize=18) 221 | 222 | plt.show() 223 | 224 | print("Mean pixel value of image = {}".format( 225 | np.mean(img[0, :, :, :, 0]))) 226 | 227 | def display_train_images(self, slice_num=90): 228 | """ 229 | Plots some training images 230 | """ 231 | self.plot_images(self.ds_train, slice_num) 232 | 233 | def display_validation_images(self, slice_num=90): 234 | """ 235 | Plots some validation images 236 | """ 237 | self.plot_images(self.ds_val, slice_num) 238 | 239 | def display_test_images(self, slice_num=90): 240 | """ 241 | Plots some test images 242 | """ 243 | self.plot_images(self.ds_test, slice_num) 244 | 245 | def get_train(self): 246 | """ 247 | Return train dataset 248 | """ 249 | return self.ds_train 250 | 251 | def get_test(self): 252 | """ 253 | Return test dataset 254 | """ 255 | return self.ds_test 256 | 257 | def get_validate(self): 258 | """ 259 | Return validation dataset 260 | """ 261 | return self.ds_val 262 | 263 | def get_dataset(self): 264 | """ 265 | Create a TensorFlow data loader 266 | """ 267 | self.num_train = int(self.numFiles * self.train_test_split) 268 | numValTest = self.numFiles - self.num_train 269 | 270 | ds = tf.data.Dataset.range(self.numFiles).shuffle( 271 | self.numFiles, self.random_seed) # Shuffle the dataset 272 | 273 | """ 274 | Horovod Sharding 275 | Here we are not actually dividing the dataset into shards 276 | but instead just reshuffling the training dataset for every 277 | shard. Then in the training loop we just go through the training 278 | dataset but the number of steps is divided by the number of shards. 279 | """ 280 | ds_train = ds.take(self.num_train).shuffle( 281 | self.num_train, self.shard) # Reshuffle based on shard 282 | ds_val_test = ds.skip(self.num_train) 283 | self.num_val = int(numValTest * self.validate_test_split) 284 | self.num_test = self.num_train - self.num_val 285 | ds_val = ds_val_test.take(self.num_val) 286 | ds_test = ds_val_test.skip(self.num_val) 287 | 288 | ds_train = ds_train.map(lambda x: tf.py_function(self.read_nifti_file, 289 | [x, True], [tf.float32, tf.float32]), 290 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 291 | ds_val = ds_val.map(lambda x: tf.py_function(self.read_nifti_file, 292 | [x, False], [tf.float32, tf.float32]), 293 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 294 | ds_test = ds_test.map(lambda x: tf.py_function(self.read_nifti_file, 295 | [x, False], [tf.float32, tf.float32]), 296 | num_parallel_calls=tf.data.experimental.AUTOTUNE) 297 | 298 | ds_train = ds_train.repeat() 299 | ds_train = ds_train.batch(self.batch_size) 300 | ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE) 301 | 302 | batch_size_val = 4 303 | ds_val = ds_val.batch(batch_size_val) 304 | ds_val = ds_val.prefetch(tf.data.experimental.AUTOTUNE) 305 | 306 | batch_size_test = 1 307 | ds_test = ds_test.batch(batch_size_test) 308 | ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE) 309 | 310 | return ds_train, ds_val, ds_test 311 | 312 | 313 | if __name__ == "__main__": 314 | 315 | print("Load the data and plot a few examples") 316 | 317 | from argparser import args 318 | 319 | crop_dim = (args.tile_height, args.tile_width, 320 | args.tile_depth, args.number_input_channels) 321 | 322 | """ 323 | Load the dataset 324 | """ 325 | brats_data = DatasetGenerator(crop_dim, 326 | data_path=args.data_path, 327 | batch_size=args.batch_size, 328 | train_test_split=args.train_test_split, 329 | validate_test_split=args.validate_test_split, 330 | number_output_classes=args.number_output_classes, 331 | random_seed=args.random_seed) 332 | 333 | brats_data.print_info() 334 | -------------------------------------------------------------------------------- /single-node/3D_BraTS2019/model.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | from argparser import args 21 | import tensorflow as tf 22 | from tensorflow import keras as K 23 | 24 | 25 | def dice_coef(target, prediction, axis=(1, 2, 3), smooth=0.0001): 26 | """ 27 | Sorenson Dice 28 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 29 | where T is ground truth mask and P is the prediction mask 30 | """ 31 | prediction = tf.round(prediction) # Round to 0 or 1 32 | 33 | intersection = tf.reduce_sum(target * prediction, axis=axis) 34 | union = tf.reduce_sum(target + prediction, axis=axis) 35 | numerator = tf.constant(2.) * intersection + smooth 36 | denominator = union + smooth 37 | coef = numerator / denominator 38 | 39 | return tf.reduce_mean(coef) 40 | 41 | 42 | def soft_dice_coef(target, prediction, axis=(1, 2, 3), smooth=0.0001): 43 | """ 44 | Sorenson (Soft) Dice - Don't round predictions 45 | \frac{ 2 \times \left | T \right | \cap \left | P \right |}{ \left | T \right | + \left | P \right | } 46 | where T is ground truth mask and P is the prediction mask 47 | """ 48 | intersection = tf.reduce_sum(target * prediction, axis=axis) 49 | union = tf.reduce_sum(target + prediction, axis=axis) 50 | numerator = tf.constant(2.) * intersection + smooth 51 | denominator = union + smooth 52 | coef = numerator / denominator 53 | 54 | return tf.reduce_mean(coef) 55 | 56 | 57 | def dice_loss(target, prediction, axis=(1, 2, 3), smooth=0.0001): 58 | """ 59 | Sorenson (Soft) Dice loss 60 | Using -log(Dice) as the loss since it is better behaved. 61 | Also, the log allows avoidance of the division which 62 | can help prevent underflow when the numbers are very small. 63 | """ 64 | intersection = tf.reduce_sum(prediction * target, axis=axis) 65 | p = tf.reduce_sum(prediction, axis=axis) 66 | t = tf.reduce_sum(target, axis=axis) 67 | numerator = tf.reduce_mean(intersection + smooth) 68 | denominator = tf.reduce_mean(t + p + smooth) 69 | dice_loss = -tf.math.log(2.*numerator) + tf.math.log(denominator) 70 | 71 | return dice_loss 72 | 73 | 74 | def unet_3d(input_dim, filters=args.filters, 75 | number_output_classes=args.number_output_classes, 76 | use_upsampling=args.use_upsampling, 77 | concat_axis=-1, model_name=args.saved_model_name): 78 | """ 79 | 3D U-Net 80 | """ 81 | 82 | def ConvolutionBlock(x, name, filters, params): 83 | """ 84 | Convolutional block of layers 85 | Per the original paper this is back to back 3D convs 86 | with batch norm and then ReLU. 87 | """ 88 | 89 | x = K.layers.Conv3D(filters=filters, **params, name=name+"_conv0")(x) 90 | x = K.layers.BatchNormalization(name=name+"_bn0")(x) 91 | x = K.layers.Activation("relu", name=name+"_relu0")(x) 92 | 93 | x = K.layers.Conv3D(filters=filters, **params, name=name+"_conv1")(x) 94 | x = K.layers.BatchNormalization(name=name+"_bn1")(x) 95 | x = K.layers.Activation("relu", name=name)(x) 96 | 97 | return x 98 | 99 | inputs = K.layers.Input(shape=input_dim, name="MRImages") 100 | 101 | params = dict(kernel_size=(3, 3, 3), activation=None, 102 | padding="same", 103 | kernel_initializer="he_uniform") 104 | 105 | # Transposed convolution parameters 106 | params_trans = dict(kernel_size=(2, 2, 2), strides=(2, 2, 2), 107 | padding="same", 108 | kernel_initializer="he_uniform") 109 | 110 | # BEGIN - Encoding path 111 | encodeA = ConvolutionBlock(inputs, "encodeA", filters, params) 112 | poolA = K.layers.MaxPooling3D(name="poolA", pool_size=(2, 2, 2))(encodeA) 113 | 114 | encodeB = ConvolutionBlock(poolA, "encodeB", filters*2, params) 115 | poolB = K.layers.MaxPooling3D(name="poolB", pool_size=(2, 2, 2))(encodeB) 116 | 117 | encodeC = ConvolutionBlock(poolB, "encodeC", filters*4, params) 118 | poolC = K.layers.MaxPooling3D(name="poolC", pool_size=(2, 2, 2))(encodeC) 119 | 120 | encodeD = ConvolutionBlock(poolC, "encodeD", filters*8, params) 121 | poolD = K.layers.MaxPooling3D(name="poolD", pool_size=(2, 2, 2))(encodeD) 122 | 123 | encodeE = ConvolutionBlock(poolD, "encodeE", filters*16, params) 124 | # END - Encoding path 125 | 126 | # BEGIN - Decoding path 127 | if use_upsampling: 128 | up = K.layers.UpSampling3D(name="upE", size=(2, 2, 2))(encodeE) 129 | else: 130 | up = K.layers.Conv3DTranspose(name="transconvE", filters=filters*8, 131 | **params_trans)(encodeE) 132 | concatD = K.layers.concatenate( 133 | [up, encodeD], axis=concat_axis, name="concatD") 134 | 135 | decodeC = ConvolutionBlock(concatD, "decodeC", filters*8, params) 136 | 137 | if use_upsampling: 138 | up = K.layers.UpSampling3D(name="upC", size=(2, 2, 2))(decodeC) 139 | else: 140 | up = K.layers.Conv3DTranspose(name="transconvC", filters=filters*4, 141 | **params_trans)(decodeC) 142 | concatC = K.layers.concatenate( 143 | [up, encodeC], axis=concat_axis, name="concatC") 144 | 145 | decodeB = ConvolutionBlock(concatC, "decodeB", filters*4, params) 146 | 147 | if use_upsampling: 148 | up = K.layers.UpSampling3D(name="upB", size=(2, 2, 2))(decodeB) 149 | else: 150 | up = K.layers.Conv3DTranspose(name="transconvB", filters=filters*2, 151 | **params_trans)(decodeB) 152 | concatB = K.layers.concatenate( 153 | [up, encodeB], axis=concat_axis, name="concatB") 154 | 155 | decodeA = ConvolutionBlock(concatB, "decodeA", filters*2, params) 156 | 157 | if use_upsampling: 158 | up = K.layers.UpSampling3D(name="upA", size=(2, 2, 2))(decodeA) 159 | else: 160 | up = K.layers.Conv3DTranspose(name="transconvA", filters=filters, 161 | **params_trans)(decodeA) 162 | concatA = K.layers.concatenate( 163 | [up, encodeA], axis=concat_axis, name="concatA") 164 | 165 | # END - Decoding path 166 | 167 | convOut = ConvolutionBlock(concatA, "convOut", filters, params) 168 | 169 | prediction = K.layers.Conv3D(name="PredictionMask", 170 | filters=number_output_classes, 171 | kernel_size=(1, 1, 1), 172 | activation="sigmoid")(convOut) 173 | 174 | model = K.models.Model(inputs=[inputs], outputs=[prediction], 175 | name=model_name) 176 | 177 | if args.print_model: 178 | model.summary() 179 | 180 | return model 181 | -------------------------------------------------------------------------------- /single-node/3D_BraTS2019/settings.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Copyright (c) 2020 Intel Corporation 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | # 17 | # SPDX-License-Identifier: EPL-2.0 18 | # 19 | 20 | DATA_PATH="/data/BRATS/MICCAI_BraTS_2019_Data_Training" 21 | CSV_LIST="files.csv" 22 | SAVED_MODEL_NAME="3d_unet_BRATS" 23 | 24 | EPOCHS=40 25 | BATCH_SIZE=8 26 | TILE_HEIGHT=144 27 | TILE_WIDTH=144 28 | TILE_DEPTH=144 29 | NUMBER_INPUT_CHANNELS=1 30 | NUMBER_OUTPUT_CLASSES=1 31 | 32 | TRAIN_TEST_SPLIT=0.80 33 | VALIDATE_TEST_SPLIT=0.50 34 | 35 | PRINT_MODEL=False 36 | FILTERS=16 37 | USE_UPSAMPLING=False 38 | 39 | RANDOM_SEED=816 -------------------------------------------------------------------------------- /single-node/3D_BraTS2019/train.py: -------------------------------------------------------------------------------- 1 | # 2 | # -*- coding: utf-8 -*- 3 | # 4 | # Copyright (c) 2020 Intel Corporation 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: EPL-2.0 19 | # 20 | import tensorflow as tf # TensorFlow 2 21 | from tensorflow import keras as K 22 | 23 | import os 24 | import datetime 25 | 26 | from argparser import args 27 | from dataloader import DatasetGenerator 28 | from model import dice_coef, soft_dice_coef, dice_loss, unet_3d 29 | 30 | 31 | def test_intel_tensorflow(): 32 | """ 33 | Check if Intel version of TensorFlow is installed 34 | """ 35 | import tensorflow as tf 36 | 37 | print("We are using Tensorflow version {}".format(tf.__version__)) 38 | 39 | major_version = int(tf.__version__.split(".")[0]) 40 | if major_version >= 2: 41 | from tensorflow.python import _pywrap_util_port 42 | print("Intel-optimizations (DNNL) enabled:", 43 | _pywrap_util_port.IsMklEnabled()) 44 | else: 45 | print("Intel-optimizations (DNNL) enabled:", 46 | tf.pywrap_tensorflow.IsMklEnabled()) 47 | 48 | print(args) 49 | test_intel_tensorflow() # Prints if Intel-optimized TensorFlow is used. 50 | 51 | """ 52 | crop_dim = Dimensions to crop the input tensor 53 | """ 54 | crop_dim = (args.tile_height, args.tile_width, 55 | args.tile_depth, args.number_input_channels) 56 | 57 | """ 58 | 1. Load the dataset 59 | """ 60 | brats_data = DatasetGenerator(crop_dim, 61 | data_path=args.data_path, 62 | csv_list=args.csv_list, 63 | batch_size=args.batch_size, 64 | train_test_split=args.train_test_split, 65 | validate_test_split=args.validate_test_split, 66 | number_output_classes=args.number_output_classes, 67 | random_seed=args.random_seed) 68 | 69 | """ 70 | 2. Create the TensorFlow model 71 | """ 72 | model = unet_3d(input_dim=crop_dim, filters=args.filters, 73 | number_output_classes=args.number_output_classes, 74 | use_upsampling=args.use_upsampling, 75 | concat_axis=-1, model_name=args.saved_model_name) 76 | 77 | local_opt = K.optimizers.Adam() 78 | model.compile(loss=dice_loss, 79 | metrics=[dice_coef, soft_dice_coef], 80 | optimizer=local_opt) 81 | 82 | checkpoint = K.callbacks.ModelCheckpoint(args.saved_model_name, 83 | verbose=1, 84 | save_best_only=True) 85 | 86 | # TensorBoard 87 | logs_dir = os.path.join( 88 | "logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S")) 89 | tb_logs = K.callbacks.TensorBoard(log_dir=logs_dir) 90 | 91 | callbacks = [checkpoint, tb_logs] 92 | 93 | """ 94 | 3. Train the model 95 | """ 96 | steps_per_epoch = brats_data.num_train // args.batch_size 97 | model.fit(brats_data.get_train(), epochs=args.epochs, 98 | steps_per_epoch=steps_per_epoch, 99 | validation_data=brats_data.get_validate(), 100 | callbacks=callbacks, 101 | verbose=1) 102 | 103 | """ 104 | 4. Load best model on validation dataset and run on the test 105 | dataset to show generalizability 106 | """ 107 | best_model = K.models.load_model(args.saved_model_name, 108 | custom_objects={"dice_loss": dice_loss, 109 | "dice_coef": dice_coef, 110 | "soft_dice_coef": soft_dice_coef}) 111 | 112 | print("\n\nEvaluating best model on the testing dataset.") 113 | print("=============================================") 114 | loss, dice_coef, soft_dice_coef = best_model.evaluate(brats_data.get_test()) 115 | 116 | print("Average Dice Coefficient on testing dataset = {:.4f}".format(dice_coef)) 117 | 118 | """ 119 | 5. Save the best model without the custom objects (dice, etc.) 120 | NOTE: You should be able to do .load_model(compile=False), but this 121 | appears to currently be broken in TF2. To compensate, we're 122 | just going to re-compile the model without the custom objects and 123 | save as a new model (with suffix "_final") 124 | """ 125 | final_model_name = args.saved_model_name + "_final" 126 | best_model.compile(loss="binary_crossentropy", metrics=["accuracy"], 127 | optimizer="adam") 128 | K.models.save_model(best_model, final_model_name, 129 | include_optimizer=False) 130 | 131 | """ 132 | Converting the model to OpenVINO 133 | """ 134 | print("\n\nConvert the TensorFlow model to OpenVINO by running:\n") 135 | print("source /opt/intel/openvino/bin/setupvars.sh") 136 | print("python $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo_tf.py \\") 137 | print(" --saved_model_dir {} \\".format(final_model_name)) 138 | print(" --model_name {} \\".format(args.saved_model_name)) 139 | print(" --batch 1 \\") 140 | print(" --output_dir {} \\".format(os.path.join("openvino_models", "FP32"))) 141 | print(" --data_type FP32\n\n") 142 | 143 | -------------------------------------------------------------------------------- /single-node/README.md: -------------------------------------------------------------------------------- 1 | This directory has been moved to: https://github.com/IntelAI/unet/tree/master/2D 2 | 3 | -------------------------------------------------------------------------------- /single-node/histology.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # # Intel-Optimized Histology Demo 5 | # 6 | # This demo uses the [colorectal histology images dataset](https://www.tensorflow.org/datasets/catalog/colorectal_histology) to train a simple convolutional neural network in TensorFlow. 7 | # 8 | # All images are RGB, 0.495 µm per pixel, digitized with an Aperio ScanScope (Aperio/Leica biosystems), magnification 20x. Histological samples are fully anonymized images of formalin-fixed paraffin-embedded human colorectal adenocarcinomas (primary tumors) from our pathology archive (Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany). 9 | 10 | # https://zenodo.org/record/53169#.X1bMe3lKguX 11 | # Kather, J. N., Zöllner, F. G., Bianconi, F., Melchers, S. M., Schad, L. R., Gaiser, T., … Weis, C.-A. (2016). Collection of textures in colorectal cancer histology [Data set]. Zenodo. http://doi.org/10.5281/zenodo.53169 12 | # 13 | # Kather JN, Weis CA, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Zollner F: Multi-class texture analysis in colorectal cancer histology (2016), Scientific Reports (in press) 14 | # 15 | # @article{kather2016multi, 16 | # title={Multi-class texture analysis in colorectal cancer histology}, 17 | # author={Kather, Jakob Nikolas and Weis, Cleo-Aron and Bianconi, Francesco and Melchers, Susanne M and Schad, Lothar R and Gaiser, Timo and Marx, Alexander and Z{"o}llner, Frank Gerrit}, 18 | # journal={Scientific reports}, 19 | # volume={6}, 20 | # pages={27988}, 21 | # year={2016}, 22 | # publisher={Nature Publishing Group} 23 | # } 24 | 25 | import os 26 | 27 | # export TF_DISABLE_MKL=1 28 | os.environ["TF_DISABLE_MKL"] = "0" # Disable Intel optimizations 29 | 30 | # export MKLDNN_VERBOSE=1 31 | #os.environ["MKLDNN_VERBOSE"] = "1" # 1 = Print log statements; 0 = silent 32 | 33 | os.environ["OMP_NUM_THREADS"] = "12" # Number of physical cores 34 | os.environ["KMP_BLOCKTIME"] = "1" 35 | 36 | # If hyperthreading is enabled, then use 37 | os.environ["KMP_AFFINITY"] = "granularity=thread,compact,1,0" 38 | 39 | # If hyperthreading is NOT enabled, then use 40 | #os.environ["KMP_AFFINITY"] = "granularity=thread,compact" 41 | 42 | import tensorflow as tf 43 | 44 | print("TensorFlow version = {}".format(tf.__version__)) 45 | 46 | print("Does TensorFlow have the Intel optimizations: {}".format(tf.python._pywrap_util_port.IsMklEnabled())) 47 | 48 | import numpy as np 49 | import matplotlib.pyplot as plt 50 | 51 | import tensorflow_datasets as tfds 52 | 53 | 54 | (ds), ds_info = tfds.load('colorectal_histology', data_dir=".", 55 | shuffle_files=True, split='train', 56 | with_info=True, as_supervised=True) 57 | 58 | assert isinstance(ds, tf.data.Dataset) 59 | print(ds_info) 60 | 61 | 62 | # ## Display a few examples from the dataset 63 | 64 | 65 | x_key, y_key = ds_info.supervised_keys 66 | ds_temp = ds.map(lambda x, y: {x_key: x, y_key: y}) 67 | tfds.show_examples(ds_info, ds_temp); 68 | 69 | ds_info.features['label'].names 70 | 71 | 72 | # ## Define the data loaders 73 | # 74 | 75 | n = ds_info.splits['train'].num_examples 76 | train_split_percentage = 0.80 77 | train_batch_size = 128 78 | test_batch_size = 16 79 | 80 | def normalize_img(image, label): 81 | """Normalizes images: `uint8` -> `float32`.""" 82 | return tf.cast(image, tf.float32) / 255., label 83 | 84 | def augment_img(image, label): 85 | """Augment images: `uint8` -> `float32`.""" 86 | 87 | image = tf.image.random_flip_left_right(image) # Random flip Left/Right 88 | image = tf.image.random_flip_up_down(image) # Random flip Up/Down 89 | 90 | return tf.cast(image, tf.float32) / 255., label # Normalize 0 to 1 for pixel values 91 | 92 | # Get train dataset 93 | ds_train = ds.take(int(n * train_split_percentage)) 94 | ds_train = ds_train.map( 95 | augment_img, num_parallel_calls=tf.data.experimental.AUTOTUNE) 96 | ds_train = ds_train.cache() 97 | ds_train = ds_train.shuffle(int(n * train_split_percentage)) 98 | ds_train = ds_train.batch(train_batch_size) 99 | ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE) 100 | 101 | # Get test dataset 102 | ds_test = ds.skip(int(n * train_split_percentage)).map( 103 | normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE) 104 | ds_test = ds_test.cache() 105 | ds_test = ds_test.batch(test_batch_size) 106 | ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE) 107 | 108 | 109 | # ## Define Model 110 | # 111 | # Here's a Convolutional neural network model. 112 | 113 | 114 | inputs = tf.keras.layers.Input(shape=ds_info.features['image'].shape) 115 | conv = tf.keras.layers.Conv2D(filters=16, kernel_size=(3,3), padding="same", activation="relu")(inputs) 116 | conv = tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3), padding="same", activation="relu")(conv) 117 | maxpool = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(conv) 118 | 119 | conv = tf.keras.layers.Conv2D(filters=64, kernel_size=(3,3), padding="same", activation="relu")(maxpool) 120 | conv = tf.keras.layers.Conv2D(filters=128, kernel_size=(3,3), padding="same", activation="relu")(conv) 121 | concat = tf.keras.layers.concatenate([maxpool, conv]) 122 | maxpool = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(concat) 123 | 124 | conv = tf.keras.layers.Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu")(maxpool) 125 | conv = tf.keras.layers.Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu")(conv) 126 | concat = tf.keras.layers.concatenate([maxpool, conv]) 127 | maxpool = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(concat) 128 | 129 | conv = tf.keras.layers.Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu")(maxpool) 130 | conv = tf.keras.layers.Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu")(conv) 131 | concat = tf.keras.layers.concatenate([maxpool, conv]) 132 | maxpool = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(concat) 133 | 134 | flat = tf.keras.layers.Flatten()(maxpool) 135 | dense = tf.keras.layers.Dense(128)(flat) 136 | drop = tf.keras.layers.Dropout(0.5)(dense) 137 | 138 | predict = tf.keras.layers.Dense(ds_info.features['label'].num_classes)(drop) 139 | 140 | model = tf.keras.models.Model(inputs=[inputs], outputs=[predict]) 141 | 142 | model.compile( 143 | loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True), 144 | optimizer='adam', 145 | metrics=[tf.metrics.SparseCategoricalAccuracy()], 146 | ) 147 | 148 | model.summary() 149 | 150 | 151 | # ## Train the model on the dataset 152 | 153 | 154 | # Create a callback that saves the model 155 | model_dir = "checkpoints" 156 | checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=model_dir, 157 | save_best_only=True, 158 | verbose=1) 159 | 160 | 161 | 162 | 163 | # Create callback for Early Stopping of training 164 | early_stopping_callback = tf.keras.callbacks.EarlyStopping(patience=8) # Stop once validation loss plateaus for patience epochs 165 | 166 | 167 | # In[12]: 168 | 169 | 170 | # TensorBoard logs 171 | tb_logs_dir = "logs" 172 | os.makedirs(tb_logs_dir, exist_ok=True) 173 | tb_callback = tf.keras.callbacks.TensorBoard(log_dir=tb_logs_dir) 174 | 175 | 176 | # ## Call *fit* to train the model 177 | 178 | # In[ ]: 179 | 180 | 181 | def train_model(epochs): 182 | history = model.fit( 183 | ds_train, 184 | epochs=epochs, 185 | validation_data=ds_test, 186 | callbacks=[checkpoint_callback, early_stopping_callback, tb_callback] 187 | ) 188 | return history 189 | 190 | epochs=5 # Run for this many epochs 191 | history = train_model(epochs) 192 | 193 | 194 | # ## Load the best model 195 | 196 | # In[ ]: 197 | 198 | 199 | print("Loading the best model") 200 | model = tf.keras.models.load_model(model_dir) 201 | 202 | 203 | # ## Evaluate the best model on the test dataset 204 | 205 | # In[ ]: 206 | 207 | 208 | print("Evaluating the best model on the test dataset") 209 | _, accuracy = model.evaluate(ds_test) 210 | print("\nModel accuracy on test dataset = {:.1f}%".format(100.0*accuracy)) 211 | 212 | 213 | # ## Display some predictions on the test data 214 | # 215 | # We grab a random subset of the test dataset and plot the image along with the ground truth label, the TensorFlow model prediction, and the OpenVINO model prediction. 216 | 217 | # In[ ]: 218 | 219 | 220 | test_data = tfds.as_numpy(ds_test.shuffle(100).take(1)) # Take 1 random batch 221 | 222 | for image, label in test_data: 223 | num = 8 # len(label) 224 | cols = 2 225 | plt.figure(figsize=(25,25)) 226 | 227 | for idx in range(num): 228 | 229 | plt.subplot(int(np.ceil(num/cols)), cols, idx+1) 230 | plt.imshow(image[idx]) 231 | plt.axis("off") 232 | 233 | # TensorFlow model prediction 234 | tf_predict = ds_info.features['label'].names[model.predict(image[[idx]]).argmax()] 235 | 236 | plt.title("Truth = {}\nTensorFlow Predict = {}".format(ds_info.features['label'].names[label[idx]], tf_predict)) 237 | 238 | 239 | -------------------------------------------------------------------------------- /single-node/kmeans-daal4py-distributed.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from time import time 3 | import daal4py as d4p 4 | from PIL import Image 5 | 6 | d4p.daalinit() 7 | n_colors = 8 8 | 9 | init_algo = d4p.kmeans_init(n_colors, method="plusPlusDense", distributed=True) 10 | 11 | max_iter = 300 12 | acc_tres = 1e-4 13 | 14 | img = Image.open('./Yushan.jpg') #https://commons.wikimedia.org/wiki/File:%E7%8E%89%E5%B1%B1%E4%B8%BB%E5%B3%B0_02.jpg 15 | img.load() 16 | 17 | china = np.array(img, dtype=np.float64) / 255 18 | 19 | # Load Image and transform to a 2D numpy array. 20 | w, h, d = original_shape = tuple(china.shape) 21 | assert d == 3 22 | image_array = np.reshape(china, (w * h, d)) 23 | o_colors = 344038 #Yushan 24 | n_slices = int(image_array.shape[0]/d4p.num_procs()) 25 | 26 | print("Number of MPI tasks: ", d4p.num_procs()) 27 | 28 | image_array = image_array[n_slices*d4p.my_procid():n_slices*d4p.my_procid()+n_slices,:] 29 | 30 | print("Fitting model on the data") 31 | t0 = time() 32 | 33 | # compute initial centroids 34 | init_result = init_algo.compute(image_array) 35 | assert init_result.centroids.shape[0] == n_colors 36 | # configure kmeans main object 37 | algo = d4p.kmeans(n_colors, max_iter, distributed=True ) 38 | # compute the clusters/centroids 39 | result = algo.compute(image_array, init_result.centroids) 40 | # Kmeans result objects provide centroids, goalFunction, nIterations and objectiveFunction 41 | assert result.centroids.shape[0] == n_colors 42 | assert result.nIterations <= max_iter 43 | 44 | print("Computation finished in in %0.3fs." % (time() - t0)) 45 | # Get labels for all points 46 | print("Predicting color indices on the full image (k-means)") 47 | 48 | t0 = time() 49 | algo = d4p.kmeans(n_colors, 0, assignFlag=True) 50 | prediction = algo.compute(image_array, result.centroids) 51 | labels = prediction.assignments 52 | 53 | print(" Completed in %0.3fs." % (time() - t0)) 54 | d4p.daalfini() 55 | -------------------------------------------------------------------------------- /testing/README.md: -------------------------------------------------------------------------------- 1 | # Testing script 2 | 3 | This provides a sanity check to show various 2D and 3D topologies with random data. 4 | 5 | `python testing.py` will test training on a 3D U-Net using randomly-generated, 6 | synthetic data. 7 | 8 | `python testing.py --D2` will test training on a 2D U-Net using randomly-generated, 9 | synthetic data. 10 | 11 | `python testing.py --D2 --inference` will test inference on a 2D U-Net using randomly-generated, 12 | synthetic data. 13 | 14 | ``` 15 | usage: testing.py [-h] [--dim_length DIM_LENGTH] [--num_channels NUM_CHANNELS] 16 | [--num_outputs NUM_OUTPUTS] [--bz BZ] [--lr LR] 17 | [--num_datapoints NUM_DATAPOINTS] [--epochs EPOCHS] 18 | [--intraop_threads INTRAOP_THREADS] 19 | [--interop_threads INTEROP_THREADS] [--blocktime BLOCKTIME] 20 | [--print_model] [--use_upsampling] [--D2] 21 | [--single_class_output] [--mkl_verbose] [--inference] 22 | [--ngraph] [--keras_api] [--channels_first] 23 | 24 | Sanity testing for 3D and 2D Convolution Models 25 | 26 | optional arguments: 27 | -h, --help show this help message and exit 28 | --dim_length DIM_LENGTH 29 | Tensor cube length of side 30 | --num_channels NUM_CHANNELS 31 | Number of channels 32 | --num_outputs NUM_OUTPUTS 33 | Number of outputs 34 | --bz BZ Batch size 35 | --lr LR Learning rate 36 | --num_datapoints NUM_DATAPOINTS 37 | Number of datapoints 38 | --epochs EPOCHS Number of epochs 39 | --intraop_threads INTRAOP_THREADS 40 | Number of intraop threads 41 | --interop_threads INTEROP_THREADS 42 | Number of interop threads 43 | --blocktime BLOCKTIME 44 | Block time for CPU threads 45 | --print_model Print the summary of the model layers 46 | --use_upsampling Use upsampling instead of transposed convolution 47 | --D2 Use 2D model and images instead of 3D. 48 | --single_class_output 49 | Use binary classifier instead of U-Net 50 | --mkl_verbose Print MKL debug statements. 51 | --inference Test inference speed. Default=Test training speed 52 | --ngraph Use ngraph 53 | --keras_api Use Keras API. False=Use tf.keras 54 | --channels_first Channels first. NCHW 55 | ``` 56 | --------------------------------------------------------------------------------