├── .DS_Store ├── Code ├── .DS_Store ├── .ipynb_checkpoints │ └── Celltracker-checkpoint.ipynb ├── Celltracker.ipynb ├── adaptivethresh.py ├── graph_construction.py ├── gvf.py ├── main.py ├── matching.py └── watershed.py ├── LICENSE ├── README.md ├── images ├── .DS_Store ├── enhance_images.gif ├── mitosis_final.gif ├── nomolizedimg.gif └── plotimg.gif └── report └── cell_tracking_final.pdf /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/.DS_Store -------------------------------------------------------------------------------- /Code/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/Code/.DS_Store -------------------------------------------------------------------------------- /Code/.ipynb_checkpoints/Celltracker-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Cell Tracking Program\n", 8 | "
\n", 9 | " The propose of this project is to develop an algorithm to realize HeLa cell cycle analysis by cell segmentation and cell tracking. Our segmentation algorithm includes binarization, nuclei center detection and nuclei boundary delineating; and our tracking algorithm includes neighboring graph construction, optimal matching, cell division, death, segmentation errors detection and processing, and refined segmentation and matching results. Our chosen testing and training datasets are Histone 2B (H2B)-GFP expressing HeLa cells provided by Mitocheck Consortium. This project used Jaccard index to measure the segmentation accuracy and TRA method for tracking. Our results, respectably 69.51% and 74.61%, demonstrated the validity of the developed algorithm in investigation of cancer cell cycle, the problems and further improvements of our algorithm are also mentioned. \n", 10 | "

" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "## 0. Prepare Import File and Import Image Set" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": { 24 | "collapsed": true 25 | }, 26 | "outputs": [], 27 | "source": [ 28 | "%matplotlib inline\n", 29 | "\n", 30 | "import os\n", 31 | "import cv2\n", 32 | "import PIL.Image\n", 33 | "import sys\n", 34 | "import numpy as np\n", 35 | "from IPython.display import Image, display, clear_output\n", 36 | "import matplotlib.pyplot as plt\n", 37 | "import scipy\n", 38 | "\n", 39 | "def normalize(image):\n", 40 | " '''\n", 41 | " This function is to normalize the input grayscale image by\n", 42 | " substracting globle mean and dividing standard diviation for\n", 43 | " visualization. \n", 44 | "\n", 45 | " Input: a grayscale image\n", 46 | "\n", 47 | " Output: normolized grascale image\n", 48 | "\n", 49 | " '''\n", 50 | " cv2.normalize(image, image, 0, 255, cv2.NORM_MINMAX)\n", 51 | " return image\n", 52 | "\n", 53 | "# read image sequence\n", 54 | "path = \"PATH_TO_IMAGES\" # The dataset could be download through: http://www.codesolorzano.com/Challenges/CTC/Datasets.html\n", 55 | "for r,d,f in os.walk(path):\n", 56 | " images = []\n", 57 | " enhance_images = []\n", 58 | " f = sorted(f)\n", 59 | " for files in f:\n", 60 | " if files[-3:].lower()=='tif':\n", 61 | " temp = cv2.imread(os.path.join(r,files))\n", 62 | " gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 63 | " images.append(gray.copy())\n", 64 | " enhance_images.append(normalize(gray.copy()))\n", 65 | "\n", 66 | "print \"Total number of image is \", len(images)\n", 67 | "print \"The shape of image is \", images[0].shape, type(images[0][0,0])" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": { 74 | "collapsed": true 75 | }, 76 | "outputs": [], 77 | "source": [ 78 | "# Helper functions\n", 79 | "def display_image(img):\n", 80 | " assert img.ndim == 2 or img.ndim == 3\n", 81 | " h, w = img.shape[:2]\n", 82 | " if len(img.shape) == 3:\n", 83 | " img = cv2.resize(img, (w/3, h/3, 3))\n", 84 | " else:\n", 85 | " img = cv2.resize(img, (w/3, h/3))\n", 86 | " cv2.imwrite(\"temp_img.png\", img)\n", 87 | " img = Image(\"temp_img.png\")\n", 88 | " display(img)\n", 89 | "\n", 90 | "def vis_square(data, title=None):\n", 91 | " \"\"\"\n", 92 | " Take an array of shape (n, height, width) or (n, height, width, 3)\n", 93 | " and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)\n", 94 | " \"\"\"\n", 95 | " # resize image into small size\n", 96 | " _, h, w = data.shape[:3] \n", 97 | " width = int(np.ceil(1200. / np.sqrt(data.shape[0]))) # the width of showing image \n", 98 | " height = int(np.ceil(h*float(width)/float(w))) # the height of showing image \n", 99 | " if len(data.shape) == 4:\n", 100 | " temp = np.zeros((data.shape[0], height, width, 3))\n", 101 | " else:\n", 102 | " temp = np.zeros((data.shape[0], height, width))\n", 103 | " \n", 104 | " for i in range(data.shape[0]):\n", 105 | " if len(data.shape) == 4:\n", 106 | " temp[i] = cv2.resize(data[i], (width, height, 3))\n", 107 | " else:\n", 108 | " temp[i] = cv2.resize(data[i], (width, height))\n", 109 | " \n", 110 | " data = temp\n", 111 | " \n", 112 | " # force the number of filters to be square\n", 113 | " n = int(np.ceil(np.sqrt(data.shape[0])))\n", 114 | " padding = (((0, n ** 2 - data.shape[0]),\n", 115 | " (0, 2), (0, 2)) # add some space between filters\n", 116 | " + ((0, 0),) * (data.ndim - 3)) # don't pad the last dimension (if there is one)\n", 117 | " data = np.pad(data, padding, mode='constant', constant_values=255) # pad with ones (white)\n", 118 | " \n", 119 | " # tile the filters into an image\n", 120 | " data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))\n", 121 | " data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])\n", 122 | " \n", 123 | " # show image\n", 124 | " cv2.imwrite(\"temp_img.png\", data)\n", 125 | " img = Image(\"temp_img.png\")\n", 126 | " display(img)\n", 127 | "\n", 128 | "def cvt_npimg(images):\n", 129 | " \"\"\"\n", 130 | " Convert image sequence to numpy array\n", 131 | " \"\"\"\n", 132 | " h, w = images[0].shape[:2]\n", 133 | " if len(images[0].shape) == 3:\n", 134 | " out = np.zeros((len(images), h, w, 3))\n", 135 | " else:\n", 136 | " out = np.zeros((len(images), h, w))\n", 137 | " for i, img in enumerate(images):\n", 138 | " out[i] = img\n", 139 | " return out" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": { 146 | "collapsed": true 147 | }, 148 | "outputs": [], 149 | "source": [ 150 | "# Write image from different input\n", 151 | "def write_mask16(images, name, index=-1):\n", 152 | " \"\"\"\n", 153 | " Write image as 16 bits image\n", 154 | " \"\"\"\n", 155 | " if index == -1:\n", 156 | " for i, img in enumerate(images):\n", 157 | " if i < 10:\n", 158 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img.astype(np.uint16))\n", 159 | " elif i >= 10 and i < 100:\n", 160 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img.astype(np.uint16))\n", 161 | " else:\n", 162 | " cv2.imwrite(name+str(i)+\".tif\", img.astype(np.uint16))\n", 163 | " else:\n", 164 | " if index < 10:\n", 165 | " cv2.imwrite(name+\"00\"+str(index)+\".tif\", images.astype(np.uint16))\n", 166 | " elif index >= 10 and index < 100:\n", 167 | " cv2.imwrite(name+\"0\"+str(index)+\".tif\", images.astype(np.uint16))\n", 168 | " else:\n", 169 | " cv2.imwrite(name+str(index)+\".tif\", images.astype(np.uint16)) \n", 170 | "\n", 171 | "def write_mask8(images, name, index=-1):\n", 172 | " \"\"\"\n", 173 | " Write image as 8 bits image\n", 174 | " \"\"\"\n", 175 | " if index == -1:\n", 176 | " for i, img in enumerate(images):\n", 177 | " if i < 10:\n", 178 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img.astype(np.uint8))\n", 179 | " elif i >= 10 and i < 100:\n", 180 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img.astype(np.uint8))\n", 181 | " else:\n", 182 | " cv2.imwrite(name+str(i)+\".tif\", img.astype(np.uint8))\n", 183 | " else:\n", 184 | " if index < 10:\n", 185 | " cv2.imwrite(name+\"000\"+str(index)+\".tif\", images.astype(np.uint8))\n", 186 | " elif index >= 10 and index < 100:\n", 187 | " cv2.imwrite(name+\"00\"+str(index)+\".tif\", images.astype(np.uint8))\n", 188 | " elif index >= 100 and index < 1000:\n", 189 | " cv2.imwrite(name+\"0\"+str(index)+\".tif\", images.astype(np.uint8)) \n", 190 | " elif index >= 1000 and index < 10000:\n", 191 | " cv2.imwrite(name+str(index)+\".tif\", images.astype(np.uint8)) \n", 192 | " else:\n", 193 | " raise\n", 194 | "\n", 195 | "def write_pair8(images, name, index=-1):\n", 196 | " \"\"\"\n", 197 | " Write image as 8 bits image with dilation\n", 198 | " \"\"\"\n", 199 | " for i, img in enumerate(images): \n", 200 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))\n", 201 | " img = cv2.dilate((img*255).astype(np.uint8),kernel,iterations = 3)\n", 202 | " if i < 10:\n", 203 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img)\n", 204 | " elif i >= 10 and i < 100:\n", 205 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img)\n", 206 | " else:\n", 207 | " cv2.imwrite(name+str(i)+\".tif\", img)" 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "## 1. Cell Segmentatioin Part\n", 215 | "
\n", 216 | "\n", 217 | " 1. Adaptive Thresholding\n", 218 | "
\n", 219 | " This file is to compute adaptive thresholding of image sequence in \n", 220 | " order to generate binary image for Nuclei segmentation.\n", 221 | "\n", 222 | " Problem:\n", 223 | " Due to the low contrast of original image, the adaptive thresholding is not working. \n", 224 | " Therefore, we change to regular threshold with threshold value as 129.\n", 225 | "\n" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": null, 231 | "metadata": { 232 | "collapsed": true 233 | }, 234 | "outputs": [], 235 | "source": [ 236 | "th = None\n", 237 | "img = None\n", 238 | "\n", 239 | "class ADPTIVETHRESH():\n", 240 | " '''\n", 241 | " This class is to provide all function for adaptive thresholding.\n", 242 | "\n", 243 | " '''\n", 244 | " def __init__(self, images):\n", 245 | " self.images = []\n", 246 | " for img in images:\n", 247 | " if len(img.shape) == 3:\n", 248 | " img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 249 | " self.images.append(img.copy())\n", 250 | "\n", 251 | " def applythresh(self, threshold = 50):\n", 252 | " '''\n", 253 | " applythresh function is to convert original image to binary image by thresholding.\n", 254 | "\n", 255 | " Input: image sequence. E.g. [image0, image1, ...]\n", 256 | "\n", 257 | " Output: image sequence after thresholding. E.g. [image0, image1, ...]\n", 258 | " '''\n", 259 | " out = []\n", 260 | " markers = []\n", 261 | " binarymark = []\n", 262 | "\n", 263 | " for img in self.images:\n", 264 | " img = cv2.GaussianBlur(img,(5,5),0).astype(np.uint8)\n", 265 | " _, thresh = cv2.threshold(img,threshold,1,cv2.THRESH_BINARY)\n", 266 | "\n", 267 | " # Using morphlogical operations to imporve the quality of result\n", 268 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(9,9))\n", 269 | " thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)\n", 270 | "\n", 271 | " out.append(thresh)\n", 272 | "\n", 273 | " return out" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": null, 279 | "metadata": { 280 | "collapsed": true 281 | }, 282 | "outputs": [], 283 | "source": [ 284 | "# This part is for testing adaptivethresh.py with single image.\n", 285 | "# Input: an original image\n", 286 | "# Output: Thresholding image\n", 287 | "\n", 288 | "global th\n", 289 | "global img\n", 290 | "\n", 291 | "adaptive = ADPTIVETHRESH(enhance_images)\n", 292 | "th = adaptive.applythresh(50)" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "metadata": { 299 | "collapsed": true 300 | }, 301 | "outputs": [], 302 | "source": [ 303 | "# display images\n", 304 | "for i,img in enumerate(th):\n", 305 | " th[i] = img*255\n", 306 | "os.chdir(\".\")\n", 307 | "write_mask8(th, \"thresh\")\n", 308 | "out = cvt_npimg(th)\n", 309 | "vis_square(out)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "
\n", 317 | "\n", 318 | " 2. Gradient Filed Vector*\n", 319 | "
\n", 320 | " This file is to compute gradient vector field (GVF) and then find the Nuclei center \n", 321 | " with the GVF result. (This part is optinal and I recommend using the distance map directly)\n", 322 | "" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": null, 328 | "metadata": { 329 | "collapsed": true 330 | }, 331 | "outputs": [], 332 | "source": [ 333 | "from scipy import spatial as sp\n", 334 | "from scipy import ndimage\n", 335 | "from scipy.spatial import distance\n", 336 | "\n", 337 | "looplimit = 500\n", 338 | "\n", 339 | "newimg = None\n", 340 | "pair = None\n", 341 | "\n", 342 | "def inbounds(shape, indices):\n", 343 | " assert len(shape) == len(indices)\n", 344 | " for i, ind in enumerate(indices):\n", 345 | " if ind < 0 or ind >= shape[i]:\n", 346 | " return False\n", 347 | " return True\n", 348 | "\n", 349 | "class GVF():\n", 350 | " '''\n", 351 | " This class contains all function for calculating GVF and its following steps.\n", 352 | " \n", 353 | " '''\n", 354 | " def __init__(self, images, thresh):\n", 355 | " \n", 356 | " self.images = images\n", 357 | " self.thresh = thresh\n", 358 | "\n", 359 | " def distancemap(self):\n", 360 | " '''\n", 361 | " This function is to generate distance map of the thresh image. We use the opencv\n", 362 | " function distanceTransform to generate it. Moreover, in this case, we use Euclidiean\n", 363 | " Distance (DIST_L2) as a metric of distance. \n", 364 | "\n", 365 | " Input: None\n", 366 | "\n", 367 | " Output: Image distance map\n", 368 | "\n", 369 | " '''\n", 370 | " return [cv2.distanceTransform(self.thresh[i], distanceType=2, maskSize=0)\\\n", 371 | " for i in range(len(self.thresh))]\n", 372 | "\n", 373 | " def new_image(self, alpha, dismap):\n", 374 | " '''\n", 375 | " This function is to generate a new image combining the oringal image I0 with\n", 376 | " the distance map image Idis by following expression:\n", 377 | " Inew = I0 + alpha*Idis\n", 378 | " In this program, we choose alpha as 0.4.\n", 379 | "\n", 380 | " Input: the weight of distance map: alpha\n", 381 | " the distance map image\n", 382 | "\n", 383 | " Output: new grayscale image\n", 384 | "\n", 385 | " '''\n", 386 | "\n", 387 | " return [self.images[i] + alpha * dismap[i] for i in range(len(self.thresh))]\n", 388 | "\n", 389 | " def compute_gvf(self, newimage):\n", 390 | " '''\n", 391 | " This function is to compute the gradient vector of the imput image.\n", 392 | "\n", 393 | " Input: a grayscale image with size, say m * n * # of images\n", 394 | "\n", 395 | " Output: a 3 dimentional image with size, m * n * 2, where the last dimention is\n", 396 | " the gradient vector (gx, gy)\n", 397 | "\n", 398 | " '''\n", 399 | " kernel_size = 5 # kernel size for blur image before compute gradient\n", 400 | " newimage = [cv2.GaussianBlur((np.clip(newimage[i], 0, 255)).astype(np.uint8),(kernel_size,kernel_size),0)\\\n", 401 | " for i in range(len(self.thresh))]\n", 402 | " # use sobel operator to compute gradient \n", 403 | " temp = np.zeros((newimage[0].shape[0], newimage[0].shape[1], 2), np.float32) # store temp gradient image \n", 404 | " gradimg = [] # output gradient images (height * weight * # of images)\n", 405 | "\n", 406 | " for i in range(len(newimage)):\n", 407 | " # compute sobel operation in x, y directions\n", 408 | " gradx = cv2.Sobel(newimage[i],cv2.CV_64F,1,0,ksize=3)\n", 409 | " grady = cv2.Sobel(newimage[i],cv2.CV_64F,0,1,ksize=3)\n", 410 | " # add the gradient vector\n", 411 | " temp[:,:,0], temp[:,:,1] = gradx, grady\n", 412 | " gradimg.append(temp)\n", 413 | "\n", 414 | " return gradimg\n", 415 | "\n", 416 | " def find_certer(self, gvfimage, index):\n", 417 | " '''\n", 418 | " This function is to find the center of Nuclei.\n", 419 | "\n", 420 | " Input: the gradient vector image (height * weight * 2).\n", 421 | "\n", 422 | " Output: the record image height * weight).\n", 423 | "\n", 424 | " '''\n", 425 | " # Initialize a image to record seed candidates.\n", 426 | " imgpair = np.zeros(gvfimage.shape[:2])\n", 427 | "\n", 428 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))\n", 429 | " dilate = cv2.dilate(self.thresh[index].copy(), kernel, iterations = 1)\n", 430 | " erthresh = cv2.erode(dilate, kernel, iterations = 3)\n", 431 | " while erthresh.sum() > 0:\n", 432 | "\n", 433 | " print \"Image \", index, \"left: \", erthresh.sum(), \"points\"\n", 434 | " # Initialize partical coordinates [y, x]\n", 435 | " y0, x0 = np.where(erthresh>0)\n", 436 | " p0 = np.array([y0[0], x0[0], 1])\n", 437 | "\n", 438 | " # Initialize record coordicates [y, x]\n", 439 | " p1 = np.array([5000, 5000, 1])\n", 440 | "\n", 441 | " # mark the first non-zero point of thresh image to 0\n", 442 | " erthresh[p0[0], p0[1]] = 0\n", 443 | "\n", 444 | " # a variable to record if the point out of bound of image or \n", 445 | " # out of maximum loop times\n", 446 | " outbound = False\n", 447 | "\n", 448 | " # count loop times to limit max loop times\n", 449 | " count = 0\n", 450 | "\n", 451 | " while sp.distance.cdist([p0],[p1]) > 1:\n", 452 | "\n", 453 | " count += 1\n", 454 | " p1 = p0\n", 455 | " u = gvfimage[p0[0], p0[1], 1]\n", 456 | " v = gvfimage[p0[0], p0[1], 0]\n", 457 | " M = np.array([[1, 0, u],\\\n", 458 | " [0, 1, v],\\\n", 459 | " [0, 0, 1]], np.float32)\n", 460 | " p0 = M.dot(p0)\n", 461 | " if not inbounds(self.thresh[index].shape, (p0[0], p0[1])) or count > looplimit:\n", 462 | " outbound = True\n", 463 | " break\n", 464 | "\n", 465 | " if not outbound:\n", 466 | " imgpair[p0[0], p0[1]] += 1\n", 467 | " clear_output(wait=True)\n", 468 | "\n", 469 | " return imgpair.copy()" 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": null, 475 | "metadata": { 476 | "collapsed": true 477 | }, 478 | "outputs": [], 479 | "source": [ 480 | "# This part is for testing gvf.py with single image. (Optional)\n", 481 | "# Input: an original image\n", 482 | "# Output: Thresholding image and seed image\n", 483 | "\n", 484 | "global th\n", 485 | "global newimg\n", 486 | "global pair\n", 487 | "# Nuclei center detection\n", 488 | "gvf = GVF(images, th)\n", 489 | "dismap = gvf.distancemap()\n", 490 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 491 | "gradimg = gvf.compute_gvf(newimg)\n", 492 | "out = []\n", 493 | "pair = []\n", 494 | "pair_raw = []\n", 495 | "kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))\n", 496 | "i = 0\n", 497 | "for i,img in enumerate(gradimg):\n", 498 | " imgpair_raw = gvf.find_certer(img, i)\n", 499 | " pair_raw.append(imgpair_raw)\n", 500 | " neighborhood_size = 20\n", 501 | " data_max = ndimage.filters.maximum_filter(pair_raw[i], neighborhood_size)\n", 502 | " data_max[data_max==0] = 255\n", 503 | " pair.append((pair_raw[i] == data_max).astype(np.uint8))\n", 504 | " write_mask8([pair[i]], \"pair_raw\", i)\n", 505 | " os.chdir(\"PATH_TO_RESULTS\")\n", 506 | " y, x = np.where(pair[i]>0)\n", 507 | " points = zip(y[:], x[:])\n", 508 | " dmap = distance.cdist(points, points, 'euclidean')\n", 509 | " y, x = np.where(dmap<10)\n", 510 | " ps = zip(y[:], x[:])\n", 511 | " for p in ps:\n", 512 | " if p[0] != p[1]:\n", 513 | " pair[i][points[min(p[0], p[1])]] = 0\n", 514 | " dilation = cv2.dilate((pair[i]*255).astype(np.uint8),kernel,iterations = 3)\n", 515 | " out.append(dilation)\n", 516 | "\n", 517 | "out = cvt_npimg(out)\n", 518 | "vis_square(out)" 519 | ] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": {}, 524 | "source": [ 525 | "
\n", 526 | "\n", 527 | " GVF enhance*\n", 528 | "
\n", 529 | " This file is to amend the seed points for watershed. (This part is optinal and I recommend using the distance map directly)\n", 530 | "" 531 | ] 532 | }, 533 | { 534 | "cell_type": "code", 535 | "execution_count": null, 536 | "metadata": { 537 | "collapsed": true 538 | }, 539 | "outputs": [], 540 | "source": [ 541 | "from scipy import spatial as sp\n", 542 | "from scipy import ndimage\n", 543 | "from scipy.spatial import distance\n", 544 | "\n", 545 | "gvf = GVF(images, th)\n", 546 | "dismap = gvf.distancemap()\n", 547 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 548 | "# TODO this part is designed to amend the result of gvf. \n", 549 | "pair = []\n", 550 | "path=os.path.join(\"PATH_TO_RESULTS\")\n", 551 | "for r,d,f in os.walk(path):\n", 552 | " for files in f:\n", 553 | " if files[:5].lower()=='seed':\n", 554 | " print files\n", 555 | " temp = cv2.imread(os.path.join(r,files))\n", 556 | " temp = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 557 | " y, x = np.where(temp>0)\n", 558 | " points = zip(y[:], x[:])\n", 559 | " dmap = distance.cdist(points, points, 'euclidean')\n", 560 | " y, x = np.where(dmap<10)\n", 561 | " ps = zip(y[:], x[:])\n", 562 | " for p in ps:\n", 563 | " if p[0] != p[1]:\n", 564 | " temp[points[min(p[0], p[1])]] = 0\n", 565 | " pair.append(temp)\n", 566 | " clear_output(wait=True)\n", 567 | "print \"finish!\"" 568 | ] 569 | }, 570 | { 571 | "cell_type": "markdown", 572 | "metadata": {}, 573 | "source": [ 574 | "
\n", 575 | "\n", 576 | " 2. Distance Map (Recommend)\n", 577 | "
\n", 578 | " This file uses distance map to generate the seed points for watershed. Although it has nothing to do with GVF, you still need to load the GVF class, since it needs some helper functions in the class.\n", 579 | "" 580 | ] 581 | }, 582 | { 583 | "cell_type": "code", 584 | "execution_count": null, 585 | "metadata": { 586 | "collapsed": true 587 | }, 588 | "outputs": [], 589 | "source": [ 590 | "gvf = GVF(images, th)\n", 591 | "dismap = gvf.distancemap()\n", 592 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 593 | "out = []\n", 594 | "pair = []\n", 595 | "kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))\n", 596 | "for i,img in enumerate(dismap):\n", 597 | " neighborhood_size = 20\n", 598 | " data_max = ndimage.filters.maximum_filter(img, neighborhood_size)\n", 599 | " data_max[data_max==0] = 255\n", 600 | " pair.append((img == data_max).astype(np.uint8))\n", 601 | " y, x = np.where(pair[i]>0)\n", 602 | " points = zip(y[:], x[:])\n", 603 | " dmap = distance.cdist(points, points, 'euclidean')\n", 604 | " y, x = np.where(dmap<20)\n", 605 | " ps = zip(y[:], x[:])\n", 606 | " for p in ps:\n", 607 | " if p[0] != p[1]:\n", 608 | " pair[i][points[min(p[0], p[1])]] = 0\n", 609 | " dilation = cv2.dilate((pair[i]*255).astype(np.uint8),kernel,iterations = 1)\n", 610 | " out.append(dilation)\n", 611 | " os.chdir(\".\")\n", 612 | " write_mask8(dilation, \"seed_point\", i)\n", 613 | "\n", 614 | "out = cvt_npimg(out)\n", 615 | "vis_square(out)" 616 | ] 617 | }, 618 | { 619 | "cell_type": "markdown", 620 | "metadata": { 621 | "collapsed": true 622 | }, 623 | "source": [ 624 | "
\n", 625 | "\n", 626 | " 3. Watershed\n", 627 | "
\n", 628 | " This file is to compute watershed given the seed image in the gvf.py.\n", 629 | "" 630 | ] 631 | }, 632 | { 633 | "cell_type": "code", 634 | "execution_count": null, 635 | "metadata": { 636 | "collapsed": true 637 | }, 638 | "outputs": [], 639 | "source": [ 640 | "import cv2\n", 641 | "import numpy as np\n", 642 | "from numpy import unique\n", 643 | "import copy as cp\n", 644 | "\n", 645 | "bmarks = None\n", 646 | "marks = None\n", 647 | "\n", 648 | "class WATERSHED():\n", 649 | " '''\n", 650 | " This class contains all the function to compute watershed.\n", 651 | "\n", 652 | " '''\n", 653 | " def __init__(self, images, markers):\n", 654 | " self.images = images\n", 655 | " self.markers = markers\n", 656 | "\n", 657 | " def is_over_long(self, img, max_lenth=50):\n", 658 | " rows = np.any(img, axis=1)\n", 659 | " cols = np.any(img, axis=0)\n", 660 | " if not len(img[img>0]):\n", 661 | " return True\n", 662 | " rmin, rmax = np.where(rows)[0][[0, -1]]\n", 663 | " cmin, cmax = np.where(cols)[0][[0, -1]]\n", 664 | " if (rmax-rmin)>max_lenth or (cmax-cmin)>max_lenth:\n", 665 | " return True\n", 666 | " else:\n", 667 | " return False\n", 668 | " \n", 669 | " def watershed_compute(self):\n", 670 | " '''\n", 671 | " This function is to compute watershed given the newimage and the seed image\n", 672 | " (center candidates). In this function, we use cv2.watershed to implement watershed.\n", 673 | "\n", 674 | " Input: newimage (height * weight * # of images)\n", 675 | "\n", 676 | " Output: watershed images (height * weight * # of images)\n", 677 | "\n", 678 | " '''\n", 679 | " result = []\n", 680 | " outmark = []\n", 681 | " outbinary = []\n", 682 | "\n", 683 | " for i in range(len(self.images)):\n", 684 | " print \"image: \", i\n", 685 | " # generate a 3-channel image in order to use cv2.watershed\n", 686 | " imgcolor = np.zeros((self.images[i].shape[0], self.images[i].shape[1], 3), np.uint8)\n", 687 | " for c in range(3): \n", 688 | " imgcolor[:,:,c] = self.images[i]\n", 689 | "\n", 690 | " # compute marker image (labelling)\n", 691 | " if len(self.markers[i].shape) == 3:\n", 692 | " self.markers[i] = cv2.cvtColor(self.markers[i],cv2.COLOR_BGR2GRAY)\n", 693 | " _, mark = cv2.connectedComponents(self.markers[i])\n", 694 | " \n", 695 | " # watershed!\n", 696 | " mark = cv2.watershed(imgcolor,mark)\n", 697 | " \n", 698 | " u, counts = unique(mark, return_counts=True)\n", 699 | " counter = dict(zip(u, counts))\n", 700 | " for index in counter:\n", 701 | " temp_img = np.zeros_like(mark)\n", 702 | " temp_img[mark==index] = 255\n", 703 | " if self.is_over_long(temp_img):\n", 704 | " mark[mark==index] = 0\n", 705 | " continue\n", 706 | " if counter[index] > 3000:\n", 707 | " mark[mark==index] = 0\n", 708 | " continue\n", 709 | " \n", 710 | " labels = list(set(mark[mark>0]))\n", 711 | " length = len(labels)\n", 712 | " temp_img = mark.copy()\n", 713 | " for original, new in zip(labels, range(1,length+1)):\n", 714 | " temp_img[mark==original] = new\n", 715 | " mark = temp_img\n", 716 | " \n", 717 | " # mark image and add to the result \n", 718 | " temp = cv2.cvtColor(imgcolor,cv2.COLOR_BGR2GRAY)\n", 719 | " result.append(temp)\n", 720 | " outmark.append(mark.astype(np.uint8))\n", 721 | "\n", 722 | " binary = mark.copy()\n", 723 | " binary[mark>0] = 255\n", 724 | " outbinary.append(binary.astype(np.uint8))\n", 725 | " clear_output(wait=True)\n", 726 | "\n", 727 | " return result, outbinary, outmark" 728 | ] 729 | }, 730 | { 731 | "cell_type": "code", 732 | "execution_count": null, 733 | "metadata": { 734 | "collapsed": true 735 | }, 736 | "outputs": [], 737 | "source": [ 738 | "# This part is for testing watershed.py with single image.\n", 739 | "# Output: Binary image after watershed\n", 740 | "\n", 741 | "global bmarks\n", 742 | "global marks\n", 743 | "\n", 744 | "# watershed\n", 745 | "ws = WATERSHED(newimg, pair) \n", 746 | "wsimage, bmarks, marks = ws.watershed_compute()\n", 747 | "\n", 748 | "out = cvt_npimg(np.clip(bmarks, 0, 255)).astype(np.uint8)\n", 749 | "vis_square(out)\n", 750 | "os.chdir(\"PATH_TO_RESULT_MASK\")\n", 751 | "write_mask16(marks, \"mask\")\n", 752 | "os.chdir(\"PATH_TO_RESULT_BINARY\")\n", 753 | "write_mask8(out, \"binary\")\n", 754 | "clear_output(wait=True)" 755 | ] 756 | }, 757 | { 758 | "cell_type": "markdown", 759 | "metadata": {}, 760 | "source": [ 761 | "\n", 762 | " 4. Segmentation Evaluation\n", 763 | "
\n", 764 | " This file is to evaluate our algorithm about segmentation in jaccard coefficient.\n", 765 | "" 766 | ] 767 | }, 768 | { 769 | "cell_type": "code", 770 | "execution_count": null, 771 | "metadata": { 772 | "collapsed": true 773 | }, 774 | "outputs": [], 775 | "source": [ 776 | "def list2pts(ptslist):\n", 777 | " list_y = np.array([ptslist[0]])\n", 778 | " list_x = np.array([ptslist[1]])\n", 779 | " return np.append(list_y, list_x).reshape(2, len(list_y[0])).T\n", 780 | "\n", 781 | "def unique_rows(a):\n", 782 | " a = np.ascontiguousarray(a)\n", 783 | " unique_a = np.unique(a.view([('', a.dtype)]*a.shape[1]))\n", 784 | " return unique_a.view(a.dtype).reshape((unique_a.shape[0], a.shape[1]))\n", 785 | "\n", 786 | "# read image sequence\n", 787 | "# The training set locates at \"resource/training/01\" and \"resource/training/02\"\n", 788 | "# The ground truth of training set locates at \"resource/training/GT_01\" and \n", 789 | "# \"resource/training/GT_02\"\n", 790 | "# The testing set locates at \"resource/testing/01\" and \"resource/testing/02\"\n", 791 | "\n", 792 | "path = \"PATH_TO_GT_SEGMENTATION\"\n", 793 | "gts = []\n", 794 | "for r,d,f in os.walk(path):\n", 795 | " for files in f:\n", 796 | " if files[-3:].lower()=='tif':\n", 797 | " temp = cv2.imread(os.path.join(r,files), cv2.IMREAD_UNCHANGED)\n", 798 | " gts.append([temp, files[-6:-4]])\n", 799 | "print \"number of gts: \", len(gts)\n", 800 | "\n", 801 | "path= \"PATH_TO_SEGMENTATION_RESULTS\"\n", 802 | "binarymarks = []\n", 803 | "for r,d,f in os.walk(path):\n", 804 | " for files in f:\n", 805 | " if files[:4]=='mark':\n", 806 | " temp = cv2.imread(os.path.join(r,files))\n", 807 | " gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 808 | " binarymarks.append([gray, files[-6:-4]])\n", 809 | "print \"number of segmentation image: \", len(binarymarks)\n", 810 | "\n", 811 | "jaccards = []\n", 812 | "\n", 813 | "for gt in gts:\n", 814 | " for binarymark in binarymarks:\n", 815 | " if gt[1] == binarymark[1]:\n", 816 | " print \"enter...\", gt[1]\n", 817 | " list_pts = set(gt[0][gt[0]>0])\n", 818 | " list_seg = set(binarymark[0][binarymark[0]>0])\n", 819 | " for pt in list_pts:\n", 820 | " for seg in list_seg:\n", 821 | " pts_gt = np.where(gt[0]==pt)\n", 822 | " pts_seg = np.where(binarymark[0]==seg)\n", 823 | " pts_gt = list2pts(pts_gt)\n", 824 | "\n", 825 | " pts_seg = list2pts(pts_seg)\n", 826 | " pts = np.append(pts_gt, pts_seg).reshape(len(pts_gt)+len(pts_seg),2)\n", 827 | " union_pts = unique_rows(pts)\n", 828 | "\n", 829 | " union = float(len(union_pts))\n", 830 | " intersection = float(len(pts_seg) + len(pts_gt) - len(union_pts))\n", 831 | "\n", 832 | " if intersection/union > 0.5:\n", 833 | " jaccards.append(intersection/union)\n", 834 | "clear_output(wait=True)\n", 835 | "jaccard = float(sum(jaccards))/float(len(jaccards))\n", 836 | "print \"jaccard: \", jaccard, \"number of Nuclei: \", len(jaccards)" 837 | ] 838 | }, 839 | { 840 | "cell_type": "markdown", 841 | "metadata": {}, 842 | "source": [ 843 | "## 2. Cell Tracking Part\n", 844 | "
\n", 845 | "\n", 846 | " 1. Graph Construction\n", 847 | "
\n", 848 | " This file is to generate a neighboring graph contraction using \n", 849 | " Delaunary Triangulation.\n", 850 | "" 851 | ] 852 | }, 853 | { 854 | "cell_type": "code", 855 | "execution_count": null, 856 | "metadata": { 857 | "collapsed": true 858 | }, 859 | "outputs": [], 860 | "source": [ 861 | "centroid = None\n", 862 | "slope_length = None\n", 863 | "class GRAPH():\n", 864 | " '''\n", 865 | " This class contains all the functions needed to compute \n", 866 | " Delaunary Triangulation.\n", 867 | "\n", 868 | " '''\n", 869 | " def __init__(self, mark, binary, index):\n", 870 | " '''\n", 871 | " Input: the grayscale mark image with different label on each segments\n", 872 | " the binary image of the mark image\n", 873 | " the index of the image\n", 874 | "\n", 875 | " '''\n", 876 | " self.mark = mark[index]\n", 877 | " self.binary = binary[index]\n", 878 | " \n", 879 | " def rect_contains(self, rect, point):\n", 880 | " '''\n", 881 | " Check if a point is inside the image\n", 882 | "\n", 883 | " Input: the size of the image \n", 884 | " the point that want to test\n", 885 | "\n", 886 | " Output: if the point is inside the image\n", 887 | "\n", 888 | " '''\n", 889 | " if point[0] < rect[0] :\n", 890 | " return False\n", 891 | " elif point[1] < rect[1] :\n", 892 | " return False\n", 893 | " elif point[0] > rect[2] :\n", 894 | " return False\n", 895 | " elif point[1] > rect[3] :\n", 896 | " return False\n", 897 | " return True\n", 898 | " \n", 899 | " def draw_point(self, img, p, color ):\n", 900 | " '''\n", 901 | " Draw a point\n", 902 | "\n", 903 | " '''\n", 904 | " cv2.circle( img, (p[1], p[0]), 2, color, cv2.FILLED, 16, 0 )\n", 905 | " \n", 906 | " def draw_delaunay(self, img, subdiv, delaunay_color):\n", 907 | " '''\n", 908 | " Draw delaunay triangles and store these lines\n", 909 | "\n", 910 | " Input: the image want to draw\n", 911 | " the set of points: format as cv2.Subdiv2D\n", 912 | " the color want to use\n", 913 | "\n", 914 | " Output: the slope and length of each line ()\n", 915 | "\n", 916 | " '''\n", 917 | " triangleList = subdiv.getTriangleList();\n", 918 | " size = img.shape\n", 919 | " r = (0, 0, size[0], size[1])\n", 920 | "\n", 921 | " slope_length = [[]]\n", 922 | " for i in range(self.mark.max()-1):\n", 923 | " slope_length.append([])\n", 924 | "\n", 925 | " for t_i, t in enumerate(triangleList):\n", 926 | " \n", 927 | " pt1 = (int(t[0]), int(t[1]))\n", 928 | " pt2 = (int(t[2]), int(t[3]))\n", 929 | " pt3 = (int(t[4]), int(t[5]))\n", 930 | " \n", 931 | " if self.rect_contains(r, pt1) and self.rect_contains(r, pt2) and self.rect_contains(r, pt3):\n", 932 | " \n", 933 | " # draw lines\n", 934 | " cv2.line(img, (pt1[1], pt1[0]), (pt2[1], pt2[0]), delaunay_color, 1, 16, 0)\n", 935 | " cv2.line(img, (pt2[1], pt2[0]), (pt3[1], pt3[0]), delaunay_color, 1, 16, 0)\n", 936 | " cv2.line(img, (pt3[1], pt3[0]), (pt1[1], pt1[0]), delaunay_color, 1, 16, 0)\n", 937 | " \n", 938 | " # store the length of line segments and their slopes\n", 939 | " for p0 in [pt1, pt2, pt3]:\n", 940 | " for p1 in [pt1, pt2, pt3]:\n", 941 | " if p0 != p1:\n", 942 | " temp = self.length_slope(p0, p1)\n", 943 | " if temp not in slope_length[self.mark[p0]-1]:\n", 944 | " slope_length[self.mark[p0]-1].append(temp) \n", 945 | "\n", 946 | " return slope_length\n", 947 | "\n", 948 | " def length_slope(self, p0, p1):\n", 949 | " '''\n", 950 | " This function is to compute the length and theta for the given two points.\n", 951 | "\n", 952 | " Input: two points with the format (y, x)\n", 953 | "\n", 954 | " '''\n", 955 | " if p1[1]-p0[1]:\n", 956 | " slope = (p1[0]-p0[0]) / (p1[1]-p0[1])\n", 957 | " else:\n", 958 | " slope = 1e10\n", 959 | "\n", 960 | " length = np.sqrt((p1[0]-p0[0])**2 + (p1[1]-p0[1])**2)\n", 961 | "\n", 962 | " return length, slope\n", 963 | "\n", 964 | " def generate_points(self):\n", 965 | " '''\n", 966 | " Find the centroid of each segmentation\n", 967 | "\n", 968 | " '''\n", 969 | " centroids = []\n", 970 | " label = []\n", 971 | " max_label = self.mark.max()\n", 972 | "\n", 973 | " for i in range(1, max_label+1):\n", 974 | " img = self.mark.copy()\n", 975 | " img[img!=i] = 0\n", 976 | " if img.sum():\n", 977 | " _, contours,hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_TC89_KCOS)\n", 978 | " m = cv2.moments(contours[0])\n", 979 | "\n", 980 | " if m['m00']:\n", 981 | " label.append(i)\n", 982 | " centroids.append(( int(round(m['m01']/m['m00'])),\\\n", 983 | " int(round(m['m10']/m['m00'])) ))\n", 984 | " else:\n", 985 | " label.append(i)\n", 986 | " centroids.append(( 0,0 ))\n", 987 | "\n", 988 | " return centroids, label\n", 989 | "\n", 990 | " def run(self, animate = False):\n", 991 | " '''\n", 992 | " The pipline of graph construction.\n", 993 | "\n", 994 | " Input: if showing a animation (False for default)\n", 995 | "\n", 996 | " Output: centroids: # of segments * 2 (y, x)\n", 997 | " slopes and length: # of segments * # of slope_length\n", 998 | "\n", 999 | " '''\n", 1000 | " # Read in the image.\n", 1001 | " img_orig = self.binary.copy()\n", 1002 | " \n", 1003 | " # Rectangle to be used with Subdiv2D\n", 1004 | " size = img_orig.shape\n", 1005 | " rect = (0, 0, size[0], size[1])\n", 1006 | " \n", 1007 | " # Create an instance of Subdiv2D\n", 1008 | " subdiv = cv2.Subdiv2D(rect);\n", 1009 | " \n", 1010 | " # find the centroid of each segments\n", 1011 | " points, label = self.generate_points()\n", 1012 | "\n", 1013 | "\n", 1014 | " # add and sort the centroid to a numpy array for post processing\n", 1015 | " centroid = np.zeros((self.mark.max(), 2))\n", 1016 | " for p, l in zip(points, label):\n", 1017 | " centroid[l-1] = p\n", 1018 | "\n", 1019 | " outimg = []\n", 1020 | " # Insert points into subdiv\n", 1021 | " for idx_p, p in enumerate(points):\n", 1022 | " subdiv.insert(p)\n", 1023 | " \n", 1024 | " # Show animation\n", 1025 | " if animate:\n", 1026 | " img_copy = img_orig.copy()\n", 1027 | " # Draw delaunay triangles\n", 1028 | " self.draw_delaunay( img_copy, subdiv, (255, 255, 255));\n", 1029 | " outimg.append(img_copy)\n", 1030 | " display_image(img_copy)\n", 1031 | " img_copy = cv2.resize(img_copy, (314, 200))\n", 1032 | " cv2.imwrite(\"delaunay_\" + str(idx_p).zfill(3) + \".png\", img_copy)\n", 1033 | " clear_output(wait=True)\n", 1034 | " \n", 1035 | " # Draw delaunay triangles\n", 1036 | " slope_length = self.draw_delaunay( img_orig, subdiv, (255, 255, 255));\n", 1037 | " \n", 1038 | " # Draw points\n", 1039 | " for p in points :\n", 1040 | " self.draw_point(img_orig, p, (0,0,255))\n", 1041 | " \n", 1042 | " # show images\n", 1043 | " if animate:\n", 1044 | " display_image(img_orig)\n", 1045 | "\n", 1046 | " print \"length of centroid: \", len(centroid)\n", 1047 | " return centroid, slope_length" 1048 | ] 1049 | }, 1050 | { 1051 | "cell_type": "code", 1052 | "execution_count": null, 1053 | "metadata": { 1054 | "collapsed": true 1055 | }, 1056 | "outputs": [], 1057 | "source": [ 1058 | "# This part is the small test for graph_contruction.py.\n", 1059 | "# Input: grayscale marker image\n", 1060 | "# binary marker image\n", 1061 | "# Output: a text file includes the centroid and the length and slope for each neighbor. \n", 1062 | "\n", 1063 | "# Build Delaunay Triangulation\n", 1064 | "global centroid\n", 1065 | "global slope_length\n", 1066 | "centroid = []\n", 1067 | "slope_length = []\n", 1068 | "for i in range(len(images)):\n", 1069 | " print \" graph_construction: image \", i\n", 1070 | " print \"max pixel: \", marks[i].max()\n", 1071 | " graph = GRAPH(marks, bmarks, i)\n", 1072 | " if i == 0:\n", 1073 | " tempcentroid, tempslope_length = graph.run(True)\n", 1074 | " else:\n", 1075 | " tempcentroid, tempslope_length = graph.run()\n", 1076 | " centroid.append(tempcentroid)\n", 1077 | " slope_length.append(tempslope_length)\n", 1078 | " clear_output(wait=True)\n", 1079 | "print \"finish!\"" 1080 | ] 1081 | }, 1082 | { 1083 | "cell_type": "markdown", 1084 | "metadata": {}, 1085 | "source": [ 1086 | "\n", 1087 | " 2. Matching\n", 1088 | "
\n", 1089 | " This file is to match nuclei in two consecutive frames by Phase Controlled Optimal Matching. \n", 1090 | " It includes two part: \n", 1091 | "\t\t1) Dissimilarity measure\n", 1092 | "\t\t2) Matching\n", 1093 | "" 1094 | ] 1095 | }, 1096 | { 1097 | "cell_type": "code", 1098 | "execution_count": null, 1099 | "metadata": { 1100 | "collapsed": true 1101 | }, 1102 | "outputs": [], 1103 | "source": [ 1104 | "import imageio\n", 1105 | "from pyefd import elliptic_fourier_descriptors\n", 1106 | "\n", 1107 | "Max_dis = 100000\n", 1108 | "\n", 1109 | "def write_image(image, title, index, imgformat='.tif'):\n", 1110 | " if index < 10:\n", 1111 | " name = '00'+str(index)\n", 1112 | " else:\n", 1113 | " name = '0'+str(index)\n", 1114 | " cv2.imwrite(title+name+imgformat, image.astype(np.uint16))\n", 1115 | "\n", 1116 | "class FEAVECTOR():\n", 1117 | " '''\n", 1118 | " This class builds a feature vector for each segments.\n", 1119 | " The format of each vector is: \n", 1120 | " v(k,i) = [c(k,i), s(k, i), h(k, i), e(k, i)], where k is the \n", 1121 | " index of the image (frame) and i is the label of each segment. \n", 1122 | "\n", 1123 | " c(k,i): the centroid of each segment (y, x);\n", 1124 | " s(k,i): the binary shape of each segment;\n", 1125 | " h(k,i): the intensity distribution (hsitogram) of the segment;\n", 1126 | " e(k,i): the spatial distribution of the segment. Its format is \n", 1127 | " like (l(k, i, p), theta(k, i, p)), where p represent different\n", 1128 | " line connected with different segment. \n", 1129 | "\n", 1130 | " '''\n", 1131 | " def __init__(self, centroid=None, shape=None, histogram=None, spatial=None, \\\n", 1132 | " ID=None, start = None, end=None, label=None, ratio=None, area=None, cooc=None):\n", 1133 | " self.c = centroid\n", 1134 | " self.s = shape\n", 1135 | " self.h = histogram\n", 1136 | " self.e = spatial\n", 1137 | " self.id = ID\n", 1138 | " self.start = start\n", 1139 | " self.end = end\n", 1140 | " self.l = label\n", 1141 | " self.a = area\n", 1142 | " self.r = ratio\n", 1143 | " self.cm = cooc\n", 1144 | "\n", 1145 | " def add_id(self, num, index):\n", 1146 | " '''\n", 1147 | " This function adds cell id for each cell.\n", 1148 | "\n", 1149 | " '''\n", 1150 | " if index == 0:\n", 1151 | " self.id = np.linspace(1, num, num)\n", 1152 | " else:\n", 1153 | " self.id= np.linspace(-1, -1, num)\n", 1154 | "\n", 1155 | " def add_label(self):\n", 1156 | " '''\n", 1157 | " This function is to add labels for each neclei for post process.\n", 1158 | "\n", 1159 | " '''\n", 1160 | " self.l = np.linspace(0, 0, len(self.c))\n", 1161 | "\n", 1162 | " def set_centroid(self, centroid):\n", 1163 | " '''\n", 1164 | " This function sets the centroid for all neclei.\n", 1165 | "\n", 1166 | " Input: the set of centroid: # of images * # of neclei * 2 (y, x)\n", 1167 | "\n", 1168 | " Output: None\n", 1169 | "\n", 1170 | " '''\n", 1171 | " self.c = centroid\n", 1172 | "\n", 1173 | " def set_spatial(self, spatial):\n", 1174 | " '''\n", 1175 | " This function sets the spatial distrbution for all neclei.\n", 1176 | "\n", 1177 | " Input: the set of centroid: # of images * # of neclei * # of line segments (length, slope)\n", 1178 | "\n", 1179 | " Output: None\n", 1180 | "\n", 1181 | " '''\n", 1182 | " self.e = spatial\n", 1183 | "\n", 1184 | " def set_shape(self, image, marker):\n", 1185 | " '''\n", 1186 | " This function sets the binary shape for all necluei.\n", 1187 | "\n", 1188 | " Input: the original images: # of images * height * weight\n", 1189 | " the labeled images: # of images * nucei's height * nucei's weight ()\n", 1190 | "\n", 1191 | " Output: None\n", 1192 | "\n", 1193 | " '''\n", 1194 | " def boundingbox(image):\n", 1195 | " y, x = np.where(image)\n", 1196 | " return min(x), min(y), max(x), max(y)\n", 1197 | "\n", 1198 | " shape = []\n", 1199 | "\n", 1200 | " for label in range(1, marker.max()+1):\n", 1201 | " tempimg = marker.copy()\n", 1202 | " tempimg[tempimg!=label] = 0\n", 1203 | " tempimg[tempimg==label] = 1\n", 1204 | " if tempimg.sum():\n", 1205 | " minx, miny, maxx, maxy = boundingbox(tempimg)\n", 1206 | " shape.append((tempimg[miny:maxy+1, minx:maxx+1], image[miny:maxy+1, minx:maxx+1]))\n", 1207 | " else:\n", 1208 | " shape.append(([], []))\n", 1209 | "\n", 1210 | " self.s = shape\n", 1211 | "\n", 1212 | " def set_histogram(self):\n", 1213 | " '''\n", 1214 | " Note: this function must be implemneted after set_shape().\n", 1215 | "\n", 1216 | " '''\n", 1217 | " def computehistogram(image):\n", 1218 | " h, w = image.shape[:2]\n", 1219 | " his = np.zeros((256,1))\n", 1220 | " for y in range(h):\n", 1221 | " for x in range(w):\n", 1222 | " his[image[y, x], 0] += 1\n", 1223 | " return his\n", 1224 | "\n", 1225 | " assert self.s != None, \"this function must be implemneted after set_shape().\"\n", 1226 | "\n", 1227 | " his = []\n", 1228 | "\n", 1229 | " for j in range(len(self.s)):\n", 1230 | " img = self.s[j][1]\n", 1231 | " if len(img):\n", 1232 | " temphis = computehistogram(img)\n", 1233 | " his.append(temphis)\n", 1234 | " else:\n", 1235 | " his.append(np.zeros((256,1)))\n", 1236 | "\n", 1237 | " self.h = his\n", 1238 | "\n", 1239 | " def add_efd(self):\n", 1240 | " coeffs = []\n", 1241 | " for i in range(len(self.s)):\n", 1242 | " try:\n", 1243 | " _, contours, hierarchy = cv2.findContours(self.s[i][0].astype(np.uint8), 1, 2)\n", 1244 | " if not len(contours):\n", 1245 | " coeffs.append(0)\n", 1246 | " continue\n", 1247 | " cnt = contours[0]\n", 1248 | " if len(cnt) >= 5:\n", 1249 | " contour = []\n", 1250 | " for i in range(len(contours[0])):\n", 1251 | " contour.append(contours[0][i][0])\n", 1252 | " coeffs.append(elliptic_fourier_descriptors(contour, order=10, normalize=False)) \n", 1253 | " else:\n", 1254 | " coeffs.append(0)\n", 1255 | " except AttributeError:\n", 1256 | " coeffs.append(0)\n", 1257 | " self.r = coeffs\n", 1258 | "\n", 1259 | " def add_co_occurrence(self, level=10):\n", 1260 | " '''\n", 1261 | " This funciton is to generate co-occurrence matrix for each cell. The structure of\n", 1262 | " output coefficients is:\n", 1263 | " [Entropy, Energy, Contrast, Homogeneity]\n", 1264 | " '''\n", 1265 | " # generate P metrix.\n", 1266 | " self.cm = []\n", 1267 | " for j in range(len(self.s)):\n", 1268 | " if not len(self.s[j][1]):\n", 1269 | " p_0 = np.zeros((level,level))\n", 1270 | " p_45 = np.zeros((level,level))\n", 1271 | " p_90 = np.zeros((level,level))\n", 1272 | " p_135 = np.zeros((level,level))\n", 1273 | " self.cm.append([np.array([0, 0, 0, 0]),[p_0, p_45, p_90, p_135]])\n", 1274 | " continue\n", 1275 | " max_p, min_p = np.max(self.s[j][1]), np.min(self.s[j][1])\n", 1276 | " range_p = max_p - min_p\n", 1277 | " img = np.round((np.asarray(self.s[j][1]).astype(np.float32)-min_p)/range_p*level)\n", 1278 | " h, w = img.shape[:2]\n", 1279 | " p_0 = np.zeros((level,level))\n", 1280 | " p_45 = np.zeros((level,level))\n", 1281 | " p_90 = np.zeros((level,level))\n", 1282 | " p_135 = np.zeros((level,level))\n", 1283 | " for y in range(h):\n", 1284 | " for x in range(w):\n", 1285 | " try:\n", 1286 | " p_0[img[y,x],img[y,x+1]] += 1\n", 1287 | " except IndexError:\n", 1288 | " pass\n", 1289 | " try:\n", 1290 | " p_0[img[y,x],img[y,x-1]] += 1\n", 1291 | " except IndexError:\n", 1292 | " pass\n", 1293 | " try:\n", 1294 | " p_90[img[y,x],img[y+1,x]] += 1\n", 1295 | " except IndexError:\n", 1296 | " pass\n", 1297 | " try:\n", 1298 | " p_90[img[y,x],img[y-1,x]] += 1\n", 1299 | " except IndexError:\n", 1300 | " pass\n", 1301 | " try:\n", 1302 | " p_45[img[y,x],img[y+1,x+1]] += 1\n", 1303 | " except IndexError:\n", 1304 | " pass\n", 1305 | " try:\n", 1306 | " p_45[img[y,x],img[y-1,x-1]] += 1\n", 1307 | " except IndexError:\n", 1308 | " pass\n", 1309 | " try:\n", 1310 | " p_135[img[y,x],img[y+1,x-1]] += 1\n", 1311 | " except IndexError:\n", 1312 | " pass\n", 1313 | " try:\n", 1314 | " p_135[img[y,x],img[y-1,x+1]] += 1\n", 1315 | " except IndexError:\n", 1316 | " pass\n", 1317 | " Entropy, Energy, Contrast, Homogeneity = 0, 0, 0, 0\n", 1318 | " for y in range(10):\n", 1319 | " for x in range(10):\n", 1320 | " if 0 not in [p_0[y,x], p_45[y,x], p_90[y,x], p_135[y,x]]:\n", 1321 | " Entropy -= (p_0[y,x]*np.log2(p_0[y,x])+\\\n", 1322 | " p_45[y,x]*np.log2(p_45[y,x])+\\\n", 1323 | " p_90[y,x]*np.log2(p_90[y,x])+\\\n", 1324 | " p_135[y,x]*np.log2(p_135[y,x]))/4\n", 1325 | " else:\n", 1326 | " temp = 0\n", 1327 | " for p in [p_0[y,x], p_45[y,x], p_90[y,x], p_135[y,x]]:\n", 1328 | " if p != 0:\n", 1329 | " temp += p*np.log2(p)\n", 1330 | " Entropy -= temp/4\n", 1331 | " Energy += (p_0[y,x]**2+\\\n", 1332 | " p_45[y,x]**2+\\\n", 1333 | " p_90[y,x]**2+\\\n", 1334 | " p_135[y,x]**2)/4\n", 1335 | " Contrast += (x-y)**2*(p_0[y,x]+\\\n", 1336 | " p_45[y,x]+\\\n", 1337 | " p_90[y,x]+\\\n", 1338 | " p_135[y,x])/4\n", 1339 | " Homogeneity += (p_0[y,x]+\\\n", 1340 | " p_45[y,x]+\\\n", 1341 | " p_90[y,x]+\\\n", 1342 | " p_135[y,x])/(4*(1+abs(x-y)))\n", 1343 | " self.cm.append([np.array([Entropy, Energy, Contrast, Homogeneity]),[p_0, p_45, p_90, p_135]])\n", 1344 | "\n", 1345 | " def add_area(self):\n", 1346 | " area = []\n", 1347 | " for i in range(len(self.s)):\n", 1348 | " area.append(np.count_nonzero(self.s[i][0]))\n", 1349 | " self.a = area\n", 1350 | "\n", 1351 | " def generate_vector(self):\n", 1352 | " '''\n", 1353 | " This function is to convert the vector maxtrics into a list.\n", 1354 | "\n", 1355 | " Output: a list of vector: [v0, v1, ....]\n", 1356 | "\n", 1357 | " '''\n", 1358 | " vector = []\n", 1359 | " for i in range(len(self.c)):\n", 1360 | " vector.append(FEAVECTOR(centroid=self.c[i],shape=self.s[i],\\\n", 1361 | " histogram=self.h[i],spatial=self.e[i],\\\n", 1362 | " ID=self.id[i],label=self.l[i],\\\n", 1363 | " ratio=self.r[i],area=self.a[i], cooc=self.cm[i]))\n", 1364 | " return vector\n", 1365 | "\n", 1366 | "\n", 1367 | "\n", 1368 | "def set_date(vectors):\n", 1369 | " '''\n", 1370 | " This function is to add the start and end frame of each vector and\n", 1371 | " combine the vector with same id.\n", 1372 | "\n", 1373 | " Input: the list of vectors in different frames. \n", 1374 | "\n", 1375 | " Output: the list of vectors of all cell with different id. \n", 1376 | "\n", 1377 | " '''\n", 1378 | " max_id = 0\n", 1379 | " for vector in vectors:\n", 1380 | " for pv in vector:\n", 1381 | " if pv.id > max_id:\n", 1382 | " max_id = pv.id\n", 1383 | "\n", 1384 | " output = np.zeros((max_id, 4))\n", 1385 | " output[:,0] = np.linspace(1, max_id, max_id) # set the cell ID\n", 1386 | " output[:,1] = len(vectors)\n", 1387 | " for frame, vector in enumerate(vectors):\n", 1388 | " for pv in vector:\n", 1389 | " if output[pv.id-1][1] > frame: # set the start frame\n", 1390 | " output[pv.id-1][1] = frame\n", 1391 | " if output[pv.id-1][2] < frame: # set the end frame\n", 1392 | " output[pv.id-1][2] = frame\n", 1393 | " output[pv.id-1][3] = pv.l # set tht cell parent ID\n", 1394 | "\n", 1395 | " return output\n", 1396 | "\n", 1397 | "def write_info(vector, name):\n", 1398 | " '''\n", 1399 | " This function is to write info. of each vector.\n", 1400 | "\n", 1401 | " Input: the list of vector generated by set_date() and \n", 1402 | " the name of output file.\n", 1403 | "\n", 1404 | " '''\n", 1405 | " with open(name+\".txt\", \"w+\") as file:\n", 1406 | " for p in vector:\n", 1407 | " file.write(str(int(p[0]))+\" \"+\\\n", 1408 | " str(int(p[1]))+\" \"+\\\n", 1409 | " str(int(p[2]))+\" \"+\\\n", 1410 | " str(int(p[3]))+\"\\n\")" 1411 | ] 1412 | }, 1413 | { 1414 | "cell_type": "code", 1415 | "execution_count": null, 1416 | "metadata": { 1417 | "collapsed": true 1418 | }, 1419 | "outputs": [], 1420 | "source": [ 1421 | "# This part is to test the matching scheme with single image\n", 1422 | "# Input: the original image;\n", 1423 | "# the labeled image;\n", 1424 | "# the binary labeled image.\n", 1425 | "\n", 1426 | "vector = None\n", 1427 | "# Feature vector construction\n", 1428 | "global centroid\n", 1429 | "global slope_length\n", 1430 | "global vector\n", 1431 | "vector = []\n", 1432 | "max_id = 0\n", 1433 | "for i in range(len(images)):\n", 1434 | " print \" feature vector: image \", i\n", 1435 | " v = FEAVECTOR()\n", 1436 | " v.set_centroid(centroid[i])\n", 1437 | " v.set_spatial(slope_length[i])\n", 1438 | " v.set_shape(enhance_images[i], marks[i])\n", 1439 | " v.set_histogram()\n", 1440 | " v.add_label()\n", 1441 | " v.add_id(marks[i].max(), i)\n", 1442 | " v.add_efd()\n", 1443 | " v.add_area()\n", 1444 | " v.add_co_occurrence()\n", 1445 | " vector.append(v.generate_vector())\n", 1446 | " print \"num of nuclei: \", len(vector[i])\n", 1447 | " clear_output(wait=True)\n", 1448 | "\n", 1449 | "print \"finish\"" 1450 | ] 1451 | }, 1452 | { 1453 | "cell_type": "markdown", 1454 | "metadata": {}, 1455 | "source": [ 1456 | "This part is to get the sub-image for each cell and save as file. " 1457 | ] 1458 | }, 1459 | { 1460 | "cell_type": "code", 1461 | "execution_count": null, 1462 | "metadata": { 1463 | "collapsed": true 1464 | }, 1465 | "outputs": [], 1466 | "source": [ 1467 | "image_size = 70\n", 1468 | "counter = 0\n", 1469 | "for i, vt in enumerate(vector):\n", 1470 | " print \"Image: \", i\n", 1471 | " for v in vt:\n", 1472 | " h, w = v.s[1].shape[:2]\n", 1473 | " extend_x = (image_size - w) / 2\n", 1474 | " extend_y = (image_size - h) / 2\n", 1475 | " temp = cv2.copyMakeBorder(v.s[1], \\\n", 1476 | " extend_y, (image_size-extend_y-h), \\\n", 1477 | " extend_x, (image_size-extend_x-w), \\\n", 1478 | " cv2.BORDER_CONSTANT, value=0)\n", 1479 | " write_mask8(temp, \"cell_image\"+str(i)+\"_\", counter)\n", 1480 | " counter += 1\n", 1481 | " clear_output(wait=True)\n", 1482 | "print \"finish!\"" 1483 | ] 1484 | }, 1485 | { 1486 | "cell_type": "markdown", 1487 | "metadata": { 1488 | "collapsed": true 1489 | }, 1490 | "source": [ 1491 | "This part is to using ratio of the two axises of inerial as mitosis refinement to mactch cells." 1492 | ] 1493 | }, 1494 | { 1495 | "cell_type": "code", 1496 | "execution_count": null, 1497 | "metadata": { 1498 | "collapsed": true 1499 | }, 1500 | "outputs": [], 1501 | "source": [ 1502 | "class SIMPLE_MATCH():\n", 1503 | " '''\n", 1504 | " This class is simple matching a nucleus into a nucleus in the previous frame by \n", 1505 | " find the nearest neighborhood. \n", 1506 | "\n", 1507 | " '''\n", 1508 | " def __init__(self, index0, index1, images, vectors):\n", 1509 | " self.v0 = cp.copy(vectors[index0])\n", 1510 | " self.v1 = cp.copy(vectors[index1])\n", 1511 | " self.i0 = index0\n", 1512 | " self.i1 = index1\n", 1513 | " self.images = images\n", 1514 | " self.vs = cp.copy(vectors)\n", 1515 | "\n", 1516 | " def distance_measure(self, pv0, pv1, alpha1=0.5, alpha2=0.25, alpha3=0.25, phase = 1):\n", 1517 | " '''\n", 1518 | " This function measures the distence of the two given feature vectors. \n", 1519 | "\n", 1520 | " This distance metrics we use is:\n", 1521 | " d(v(k, i), v(k+1, j)) = alpha1 * d(c(k, i), c(k+1, j)) + \n", 1522 | " alpha2 * q1 * d(s(k, i), s(k+1, j)) +\n", 1523 | " alpha3 * q2 * d(h(k, i), h(k+1, j)) +\n", 1524 | " alpha4 * d(e(k, i), e(k+1, j))\n", 1525 | " Input: The two given feature vectors, \n", 1526 | " and the set of parameters.\n", 1527 | "\n", 1528 | " Output: the distance of the two given vectors. \n", 1529 | "\n", 1530 | " '''\n", 1531 | " def centriod_distance(c0, c1, D=30.):\n", 1532 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1533 | " return dist/D if dist < D else 1\n", 1534 | "\n", 1535 | " def efd_distance(r0, r1, order=8):\n", 1536 | " def find_max(max_value, test):\n", 1537 | " if max_value < test:\n", 1538 | " return test\n", 1539 | " return max_value\n", 1540 | " dis = 0\n", 1541 | " if type(r0) is not int and type(r1) is not int:\n", 1542 | " max_a, max_b, max_c, max_d = 0, 0, 0, 0\n", 1543 | " for o in range(order):\n", 1544 | " dis += ((r0[o][0]-r1[o][0])**2+\\\n", 1545 | " (r0[o][1]-r1[o][1])**2+\\\n", 1546 | " (r0[o][2]-r1[o][2])**2+\\\n", 1547 | " (r0[o][3]-r1[o][3])**2)\n", 1548 | " max_a = find_max(max_a, (r0[o][0]-r1[o][0])**2)\n", 1549 | " max_b = find_max(max_b, (r0[o][1]-r1[o][1])**2)\n", 1550 | " max_c = find_max(max_c, (r0[o][2]-r1[o][2])**2)\n", 1551 | " max_d = find_max(max_d, (r0[o][3]-r1[o][3])**2)\n", 1552 | " dis /= (order*(max_a+max_b+max_c+max_d))\n", 1553 | " if dis > 1.1:\n", 1554 | " print dis, max_a, max_b, max_c, max_d\n", 1555 | " raise\n", 1556 | " else:\n", 1557 | " dis = 1\n", 1558 | " return dis\n", 1559 | "\n", 1560 | " def cm_distance(cm0, cm1): \n", 1561 | " return ((cm0[0]-cm1[0])**2+\\\n", 1562 | " (cm0[1]-cm1[1])**2+\\\n", 1563 | " (cm0[2]-cm1[2])**2+\\\n", 1564 | " (cm0[3]-cm1[3])**2)/\\\n", 1565 | " (max(cm0[0],cm1[0])**2+\\\n", 1566 | " max(cm0[1],cm1[1])**2+\\\n", 1567 | " max(cm0[2],cm1[2])**2+\\\n", 1568 | " max(cm0[3],cm1[3])**2)\n", 1569 | "\n", 1570 | " if len(pv0.s[0]) and len(pv1.s[0]):\n", 1571 | " dist = \talpha1 * centriod_distance(pv0.c, pv1.c)+ \\\n", 1572 | " alpha2 * efd_distance(pv0.r, pv1.r, order=8) * phase + \\\n", 1573 | " alpha3 * cm_distance(pv0.cm[0], pv1.cm[0]) * phase\n", 1574 | " else:\n", 1575 | " dist = Max_dis\n", 1576 | "\n", 1577 | " return dist\n", 1578 | "\n", 1579 | " def phase_identify(self, pv1, min_times_MA2ma = 2, RNN=False):\n", 1580 | " '''\n", 1581 | " Phase identification returns 0 when mitosis appears, vice versa.\n", 1582 | "\n", 1583 | " '''\n", 1584 | " if not RNN:\n", 1585 | " _, contours, hierarchy = cv2.findContours(pv1.s[0].astype(np.uint8), 1, 2)\n", 1586 | " if not len(contours):\n", 1587 | " return 1\n", 1588 | " cnt = contours[0]\n", 1589 | " if len(cnt) >= 5:\n", 1590 | " (x,y),(ma,MA),angle = cv2.fitEllipse(cnt)\n", 1591 | " if ma and MA/ma > min_times_MA2ma:\n", 1592 | " return 0\n", 1593 | " elif not ma and MA:\n", 1594 | " return 0\n", 1595 | " else:\n", 1596 | " return 1\n", 1597 | " else:\n", 1598 | " return 1\n", 1599 | "\n", 1600 | " else:\n", 1601 | " try: \n", 1602 | " if model.predict([pv1.r.reshape(40)])[-1]:\n", 1603 | " return 0\n", 1604 | " else: \n", 1605 | " return 1\n", 1606 | " except AttributeError:\n", 1607 | " return 1\n", 1608 | "\n", 1609 | " def find_match(self, max_distance=1,a_1=0.5,a_2=0.25,a_3=0.25, rnn=False):\n", 1610 | " '''\n", 1611 | " This function is to find the nearest neighborhood between two\n", 1612 | " successive frame.\n", 1613 | "\n", 1614 | " '''\n", 1615 | " def centriod_distance(c0, c1, D=30.):\n", 1616 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1617 | " return dist/D if dist < D else 1\n", 1618 | "\n", 1619 | " for i, pv1 in enumerate(self.v1):\n", 1620 | " dist = np.ones((len(self.v0), 3), np.float32)*max_distance\n", 1621 | " count = 0\n", 1622 | " q = self.phase_identify(pv1, 3, RNN=rnn)\n", 1623 | " for j, pv0 in enumerate(self.v0):\n", 1624 | " if centriod_distance(pv0.c, pv1.c) < 1 and pv0.a:\n", 1625 | " dist[count][0] = self.distance_measure(pv0, pv1, alpha1=a_1, alpha2=a_2, alpha3=a_3, phase=q)\n", 1626 | " dist[count][1] = pv0.l\n", 1627 | " dist[count][2] = pv0.id\n", 1628 | " count += 1\n", 1629 | "\n", 1630 | " sort_dist = sorted(dist, key=lambda a_entry: a_entry[0])\n", 1631 | " print \"dis: \", sort_dist[0][0]\n", 1632 | " if sort_dist[0][0] < max_distance:\n", 1633 | " self.v1[i].l = sort_dist[0][1]\n", 1634 | " self.v1[i].id = sort_dist[0][2]\n", 1635 | "\n", 1636 | " def mitosis_refine(self, rnn=False):\n", 1637 | " '''\n", 1638 | " This function is to find died cell due to the by mitosis. \n", 1639 | "\n", 1640 | " '''\n", 1641 | " def find_sibling(pv0):\n", 1642 | " '''\n", 1643 | " This function is to find sibling cells according to the centroid of\n", 1644 | " pv0. The criteria of sibling is:\n", 1645 | " 1. the jaccard cooeficient of the two cells is above 0.5\n", 1646 | " 2. the sum of the two areas should in the range [A, 2.5A], where\n", 1647 | " A is the area of the pv0\n", 1648 | " 3. the position of the two cells should be not larger than 20 pixels.\n", 1649 | "\n", 1650 | " Input: pv0: the parent cell that you want to find siblings;\n", 1651 | "\n", 1652 | " Output: the index of the siblings.\n", 1653 | "\n", 1654 | " '''\n", 1655 | " def maxsize_image(image1, image2):\n", 1656 | " y1, x1 = np.where(image1)\n", 1657 | " y2, x2 = np.where(image2)\n", 1658 | " return min(min(x1), min(x2)), min(min(y1), min(y2)), \\\n", 1659 | " max(max(x1), max(x2)), max(max(y1), max(y2)),\n", 1660 | "\n", 1661 | " def symmetry(image, shape):\n", 1662 | " h, w = image.shape[:2] \n", 1663 | " newimg = np.zeros(shape)\n", 1664 | " newimg[:h, :w] = image\n", 1665 | " v = float(shape[0] - h)/2.\n", 1666 | " u = float(shape[1] - w)/2.\n", 1667 | " M = np.float32([[1,0,u],[0,1,v]])\n", 1668 | " return cv2.warpAffine(newimg,M,(shape[1],shape[0]))\n", 1669 | "\n", 1670 | " def jaccard(s0, s1):\n", 1671 | " minx, miny, maxx, maxy = maxsize_image(s0, s1)\n", 1672 | " height = maxy - miny + 1\n", 1673 | " width = maxx - minx + 1\n", 1674 | "\n", 1675 | " img0 = symmetry(s0, (height, width))\n", 1676 | " img1 = symmetry(s1, (height, width))\n", 1677 | "\n", 1678 | " num = 0.\n", 1679 | " deno = 0.\n", 1680 | " for y in range(height):\n", 1681 | " for x in range(width):\n", 1682 | " if img0[y, x] and img1[y, x]:\n", 1683 | " num += 1\n", 1684 | " if img0[y, x] or img1[y, x]:\n", 1685 | " deno += 1\n", 1686 | "\n", 1687 | " return num/deno\n", 1688 | "\n", 1689 | " sibling_cand = []\n", 1690 | " for i, pv1 in enumerate(self.v1): \n", 1691 | " if np.linalg.norm(pv1.c-pv0.c) < 50:\n", 1692 | " sibling_cand.append([pv1, i])\n", 1693 | "\n", 1694 | " sibling_pair = []\n", 1695 | " area = pv0.s[0].sum()\n", 1696 | " jaccard_value = []\n", 1697 | " for sibling0 in sibling_cand:\n", 1698 | " for sibling1 in sibling_cand:\n", 1699 | " if (sibling1[0].c != sibling0[0].c).all():\n", 1700 | " sum_area = sibling1[0].s[0].sum()+sibling0[0].s[0].sum()\n", 1701 | " similarity = jaccard(sibling0[0].s[0], sibling1[0].s[0])\n", 1702 | " if similarity > 0.4 and (sum_area > 2*area):\n", 1703 | " sibling_pair.append([sibling0, sibling1])\n", 1704 | " jaccard_value.append(similarity)\n", 1705 | " if len(jaccard_value):\n", 1706 | " return sibling_pair[np.argmax(jaccard_value)]\n", 1707 | " else:\n", 1708 | " return 0\n", 1709 | "\n", 1710 | " v1_ids = []\n", 1711 | " for pv1 in self.v1:\n", 1712 | " v1_ids.append(pv1.id)\n", 1713 | "\n", 1714 | " for i, pv0 in enumerate(self.v0):\n", 1715 | " if pv0.id not in v1_ids and len(pv0.s[0]) and self.phase_identify(pv0, 3, RNN=rnn):\n", 1716 | " sibling = find_sibling(pv0)\n", 1717 | " if sibling:\n", 1718 | " [s0, s1] = sibling\n", 1719 | " if s0[0].l==0 and s1[0].l==0 and \\\n", 1720 | " s0[0].id==-1 and s1[0].id==-1:\n", 1721 | " self.v1[s0[1]].l = pv0.id\n", 1722 | " self.v1[s1[1]].l = pv0.id\n", 1723 | "\n", 1724 | " return self.v1\n", 1725 | "\n", 1726 | " def match_missing(self, mask, max_frame = 1, max_distance = 10, min_shape_similarity = 0.6):\n", 1727 | " '''\n", 1728 | " This function is to match the cells that didn't show in the last frame caused by \n", 1729 | " program fault. In order to match them, we need to seach the cell in the previous \n", 1730 | " frame with in the certain range and with similar shape. \n", 1731 | "\n", 1732 | " '''\n", 1733 | " def centriod_distance(c0, c1):\n", 1734 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1735 | " return dist\n", 1736 | "\n", 1737 | " def maxsize_image(image1, image2):\n", 1738 | " y1, x1 = np.where(image1)\n", 1739 | " y2, x2 = np.where(image2)\n", 1740 | " return min(min(x1), min(x2)), min(min(y1), min(y2)), \\\n", 1741 | " max(max(x1), max(x2)), max(max(y1), max(y2)),\n", 1742 | "\n", 1743 | " def symmetry(image, shape):\n", 1744 | " h, w = image.shape[:2] \n", 1745 | " newimg = np.zeros(shape)\n", 1746 | " newimg[:h, :w] = image\n", 1747 | " v = float(shape[0] - h)/2.\n", 1748 | " u = float(shape[1] - w)/2.\n", 1749 | " M = np.float32([[1,0,u],[0,1,v]])\n", 1750 | " return cv2.warpAffine(newimg,M,(shape[1],shape[0]))\n", 1751 | "\n", 1752 | " def shape_similarity(s0, s1):\n", 1753 | " if len(s0) and len(s1):\n", 1754 | " minx, miny, maxx, maxy = maxsize_image(s0, s1)\n", 1755 | " height = maxy - miny + 1\n", 1756 | " width = maxx - minx + 1\n", 1757 | "\n", 1758 | " img0 = symmetry(s0, (height, width))\n", 1759 | " img1 = symmetry(s1, (height, width))\n", 1760 | "\n", 1761 | " num = 0.\n", 1762 | " deno = 0.\n", 1763 | " for y in range(height):\n", 1764 | " for x in range(width):\n", 1765 | " if img0[y, x] and img1[y, x]:\n", 1766 | " num += 1\n", 1767 | " if img0[y, x] or img1[y, x]:\n", 1768 | " deno += 1\n", 1769 | " return num/deno\n", 1770 | "\n", 1771 | " else:\n", 1772 | " return 0.\n", 1773 | "\n", 1774 | " def add_marker(index_find, index_new, pv0_id):\n", 1775 | " temp = mask[index_new]\n", 1776 | " find = mask[index_find]\n", 1777 | " temp[find==pv0_id] = pv0_id\n", 1778 | " return temp\n", 1779 | "\n", 1780 | " for i, pv1 in enumerate(self.v1):\n", 1781 | " if pv1.id == -1:\n", 1782 | " for index in range(1, max_frame+1):\n", 1783 | " if self.i0-index >= 0:\n", 1784 | " vt = self.vs[self.i0-index]\n", 1785 | " for pv0 in vt:\n", 1786 | " if centriod_distance(pv0.c, pv1.c) < max_distance and \\\n", 1787 | " shape_similarity(pv0.s[0], pv1.s[0]) > min_shape_similarity:\n", 1788 | " self.v1[i].id = pv0.id\n", 1789 | " self.v1[i].l = pv0.l\n", 1790 | " print \"missing in frame: \", self.i1, \"find in frame: \", \\\n", 1791 | " self.i0-index, \"ID: \", pv0.id, \" at: \", pv0.c\n", 1792 | " for i in range(self.i0-index+1, self.i1):\n", 1793 | " mask[i] = add_marker(self.i0-index, i, pv0.id)\n", 1794 | " return mask\n", 1795 | "\n", 1796 | " def new_id(self, vectors):\n", 1797 | " '''\n", 1798 | " This function is to add new labels for the necles that are marked as -1.\n", 1799 | "\n", 1800 | "\n", 1801 | " '''\n", 1802 | " def find_max_id(vectors):\n", 1803 | " max_id = 0\n", 1804 | " for vt in vectors:\n", 1805 | " for pt in vt:\n", 1806 | " if pt.id > max_id:\n", 1807 | " max_id = pt.id \n", 1808 | " return max_id\n", 1809 | " max_id = find_max_id(self.vs)\n", 1810 | " max_id += 1\n", 1811 | " for i, pv1 in enumerate(self.v1):\n", 1812 | " if pv1.id == -1:\n", 1813 | " self.v1[i].id = max_id\n", 1814 | " max_id += 1\n", 1815 | "\n", 1816 | " def generate_mask(self, marker, index, isfinal=False):\n", 1817 | " '''\n", 1818 | " This function is to generate a 16-bit image as mask image. \n", 1819 | "\n", 1820 | " '''\n", 1821 | " h, w = marker.shape[:2]\n", 1822 | " mask = np.zeros((h, w), np.uint16)\n", 1823 | " pts = list(set(marker[marker>0]))\n", 1824 | " if not isfinal:\n", 1825 | " assert len(pts)==len(self.v0), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v0))\n", 1826 | " for pt, pv in zip(pts, self.v0):\n", 1827 | " mask[marker==pt] = pv.id\n", 1828 | "\n", 1829 | " else:\n", 1830 | " assert len(pts)==len(self.v1), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v1))\n", 1831 | " for pt, pv in zip(pts, self.v1):\n", 1832 | " mask[marker==pt] = pv.id\n", 1833 | "\n", 1834 | " os.chdir(\".\")\n", 1835 | " write_mask16(mask, \"mask\", index)\n", 1836 | " os.chdir(os.pardir)\n", 1837 | " return mask\t\n", 1838 | "\n", 1839 | " def return_vectors(self):\n", 1840 | " '''\n", 1841 | " This function is to return the vectors that we have already \n", 1842 | " changed.\n", 1843 | "\n", 1844 | " Output: the vectors from the k+1 frame.\n", 1845 | "\n", 1846 | " '''\t\n", 1847 | " return self.v1" 1848 | ] 1849 | }, 1850 | { 1851 | "cell_type": "code", 1852 | "execution_count": null, 1853 | "metadata": { 1854 | "collapsed": true 1855 | }, 1856 | "outputs": [], 1857 | "source": [ 1858 | "import copy as cp\n", 1859 | "\n", 1860 | "mask = []\n", 1861 | "temp_vector = cp.deepcopy(vector)\n", 1862 | "\n", 1863 | "# Feature matching\n", 1864 | "for i in range(len(images)-1):\n", 1865 | " print \" Feature matching: image \", i\n", 1866 | " m = SIMPLE_MATCH(i,i+1,[images[i], images[i+1]], temp_vector)\n", 1867 | " mask.append(m.generate_mask(marks[i], i))\n", 1868 | " m.find_match(0.7,0.7,0.15,0.15)\n", 1869 | " temp_vector[i+1] = m.mitosis_refine()\n", 1870 | " m.new_id(temp_vector)\n", 1871 | " temp_vector[i+1] = m.return_vectors()\n", 1872 | " clear_output(wait=True)\n", 1873 | "\n", 1874 | "print \" Feature matching: image \", i+1\n", 1875 | "mask.append(m.generate_mask(marks[i+1], i+1, True))\n", 1876 | "os.chdir(\".\")\n", 1877 | "cells = set_date(temp_vector)\n", 1878 | "write_info(cells, \"res_track\")\n", 1879 | "print \"finish!\"" 1880 | ] 1881 | }, 1882 | { 1883 | "cell_type": "markdown", 1884 | "metadata": {}, 1885 | "source": [ 1886 | "This part generates the final marked result in \"gif\"." 1887 | ] 1888 | }, 1889 | { 1890 | "cell_type": "code", 1891 | "execution_count": null, 1892 | "metadata": { 1893 | "collapsed": true 1894 | }, 1895 | "outputs": [], 1896 | "source": [ 1897 | "# write gif image showing the final result\n", 1898 | "def find_max_id(temp_vector):\n", 1899 | " max_id = 0\n", 1900 | " for pv in temp_vector:\n", 1901 | " for p in pv:\n", 1902 | " if p.id > max_id:\n", 1903 | " max_id = p.id\n", 1904 | " return max_id\n", 1905 | "\n", 1906 | "# This part is to mark the result in the normolized image and \n", 1907 | "# write the gif image.\n", 1908 | "max_id = find_max_id(temp_vector)\n", 1909 | "colors = [np.random.randint(0, 255, size=max_id),\\\n", 1910 | " np.random.randint(0, 255, size=max_id),\\\n", 1911 | " np.random.randint(0, 255, size=max_id)]\n", 1912 | "font = cv2.FONT_HERSHEY_SIMPLEX \n", 1913 | "selecy_id = 9\n", 1914 | "enhance_imgs = []\n", 1915 | "for i, m in enumerate(mask):\n", 1916 | " print \" write the gif image: image \", i\n", 1917 | " enhance_imgs.append(cv2.cvtColor(enhance_images[i],cv2.COLOR_GRAY2RGB))\n", 1918 | " for pv in temp_vector[i]:\n", 1919 | " center = pv.c\n", 1920 | " if not pv.l:\n", 1921 | " color = (colors[0][int(pv.id)-1],\\\n", 1922 | " colors[1][int(pv.id)-1],\\\n", 1923 | " colors[2][int(pv.id)-1],)\n", 1924 | " else:\n", 1925 | " color = (colors[0][int(pv.l)-1],\\\n", 1926 | " colors[1][int(pv.l)-1],\\\n", 1927 | " colors[2][int(pv.l)-1],)\n", 1928 | "\n", 1929 | " if m[center[0], center[1]]:\n", 1930 | " enhance_imgs[i][m==pv.id] = color\n", 1931 | " cv2.putText(enhance_imgs[i],\\\n", 1932 | " str(int(pv.id)),(int(pv.c[1]), \\\n", 1933 | " int(pv.c[0])), \n", 1934 | " font, 0.5,\\\n", 1935 | " (255,255,255),1)\n", 1936 | " clear_output(wait=True)\n", 1937 | "os.chdir(\"PATH_TO_RESULT\")\n", 1938 | "imageio.mimsave('mitosis_final.gif', enhance_imgs, duration=0.6)\n", 1939 | "print \"finish!\"" 1940 | ] 1941 | } 1942 | ], 1943 | "metadata": { 1944 | "kernelspec": { 1945 | "display_name": "Python 2", 1946 | "language": "python", 1947 | "name": "python2" 1948 | }, 1949 | "language_info": { 1950 | "codemirror_mode": { 1951 | "name": "ipython", 1952 | "version": 2 1953 | }, 1954 | "file_extension": ".py", 1955 | "mimetype": "text/x-python", 1956 | "name": "python", 1957 | "nbconvert_exporter": "python", 1958 | "pygments_lexer": "ipython2", 1959 | "version": "2.7.13" 1960 | } 1961 | }, 1962 | "nbformat": 4, 1963 | "nbformat_minor": 1 1964 | } 1965 | -------------------------------------------------------------------------------- /Code/Celltracker.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Cell Tracking Program\n", 8 | "
\n", 9 | " The propose of this project is to develop an algorithm to realize HeLa cell cycle analysis by cell segmentation and cell tracking. Our segmentation algorithm includes binarization, nuclei center detection and nuclei boundary delineating; and our tracking algorithm includes neighboring graph construction, optimal matching, cell division, death, segmentation errors detection and processing, and refined segmentation and matching results. Our chosen testing and training datasets are Histone 2B (H2B)-GFP expressing HeLa cells provided by Mitocheck Consortium. This project used Jaccard index to measure the segmentation accuracy and TRA method for tracking. Our results, respectably 69.51% and 74.61%, demonstrated the validity of the developed algorithm in investigation of cancer cell cycle, the problems and further improvements of our algorithm are also mentioned. \n", 10 | "

" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "## 0. Prepare Import File and Import Image Set" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": { 24 | "collapsed": true 25 | }, 26 | "outputs": [], 27 | "source": [ 28 | "%matplotlib inline\n", 29 | "\n", 30 | "import os\n", 31 | "import cv2\n", 32 | "import PIL.Image\n", 33 | "import sys\n", 34 | "import numpy as np\n", 35 | "from IPython.display import Image, display, clear_output\n", 36 | "import matplotlib.pyplot as plt\n", 37 | "import scipy\n", 38 | "\n", 39 | "def normalize(image):\n", 40 | " '''\n", 41 | " This function is to normalize the input grayscale image by\n", 42 | " substracting globle mean and dividing standard diviation for\n", 43 | " visualization. \n", 44 | "\n", 45 | " Input: a grayscale image\n", 46 | "\n", 47 | " Output: normolized grascale image\n", 48 | "\n", 49 | " '''\n", 50 | " cv2.normalize(image, image, 0, 255, cv2.NORM_MINMAX)\n", 51 | " return image\n", 52 | "\n", 53 | "# read image sequence\n", 54 | "path = \"PATH_TO_IMAGES\" # The dataset could be download through: http://www.codesolorzano.com/Challenges/CTC/Datasets.html\n", 55 | "for r,d,f in os.walk(path):\n", 56 | " images = []\n", 57 | " enhance_images = []\n", 58 | " f = sorted(f)\n", 59 | " for files in f:\n", 60 | " if files[-3:].lower()=='tif':\n", 61 | " temp = cv2.imread(os.path.join(r,files))\n", 62 | " gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 63 | " images.append(gray.copy())\n", 64 | " enhance_images.append(normalize(gray.copy()))\n", 65 | "\n", 66 | "print \"Total number of image is \", len(images)\n", 67 | "print \"The shape of image is \", images[0].shape, type(images[0][0,0])" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": { 74 | "collapsed": true 75 | }, 76 | "outputs": [], 77 | "source": [ 78 | "# Helper functions\n", 79 | "def display_image(img):\n", 80 | " assert img.ndim == 2 or img.ndim == 3\n", 81 | " h, w = img.shape[:2]\n", 82 | " if len(img.shape) == 3:\n", 83 | " img = cv2.resize(img, (w/3, h/3, 3))\n", 84 | " else:\n", 85 | " img = cv2.resize(img, (w/3, h/3))\n", 86 | " cv2.imwrite(\"temp_img.png\", img)\n", 87 | " img = Image(\"temp_img.png\")\n", 88 | " display(img)\n", 89 | "\n", 90 | "def vis_square(data, title=None):\n", 91 | " \"\"\"\n", 92 | " Take an array of shape (n, height, width) or (n, height, width, 3)\n", 93 | " and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)\n", 94 | " \"\"\"\n", 95 | " # resize image into small size\n", 96 | " _, h, w = data.shape[:3] \n", 97 | " width = int(np.ceil(1200. / np.sqrt(data.shape[0]))) # the width of showing image \n", 98 | " height = int(np.ceil(h*float(width)/float(w))) # the height of showing image \n", 99 | " if len(data.shape) == 4:\n", 100 | " temp = np.zeros((data.shape[0], height, width, 3))\n", 101 | " else:\n", 102 | " temp = np.zeros((data.shape[0], height, width))\n", 103 | " \n", 104 | " for i in range(data.shape[0]):\n", 105 | " if len(data.shape) == 4:\n", 106 | " temp[i] = cv2.resize(data[i], (width, height, 3))\n", 107 | " else:\n", 108 | " temp[i] = cv2.resize(data[i], (width, height))\n", 109 | " \n", 110 | " data = temp\n", 111 | " \n", 112 | " # force the number of filters to be square\n", 113 | " n = int(np.ceil(np.sqrt(data.shape[0])))\n", 114 | " padding = (((0, n ** 2 - data.shape[0]),\n", 115 | " (0, 2), (0, 2)) # add some space between filters\n", 116 | " + ((0, 0),) * (data.ndim - 3)) # don't pad the last dimension (if there is one)\n", 117 | " data = np.pad(data, padding, mode='constant', constant_values=255) # pad with ones (white)\n", 118 | " \n", 119 | " # tile the filters into an image\n", 120 | " data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))\n", 121 | " data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])\n", 122 | " \n", 123 | " # show image\n", 124 | " cv2.imwrite(\"temp_img.png\", data)\n", 125 | " img = Image(\"temp_img.png\")\n", 126 | " display(img)\n", 127 | "\n", 128 | "def cvt_npimg(images):\n", 129 | " \"\"\"\n", 130 | " Convert image sequence to numpy array\n", 131 | " \"\"\"\n", 132 | " h, w = images[0].shape[:2]\n", 133 | " if len(images[0].shape) == 3:\n", 134 | " out = np.zeros((len(images), h, w, 3))\n", 135 | " else:\n", 136 | " out = np.zeros((len(images), h, w))\n", 137 | " for i, img in enumerate(images):\n", 138 | " out[i] = img\n", 139 | " return out" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": { 146 | "collapsed": true 147 | }, 148 | "outputs": [], 149 | "source": [ 150 | "# Write image from different input\n", 151 | "def write_mask16(images, name, index=-1):\n", 152 | " \"\"\"\n", 153 | " Write image as 16 bits image\n", 154 | " \"\"\"\n", 155 | " if index == -1:\n", 156 | " for i, img in enumerate(images):\n", 157 | " if i < 10:\n", 158 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img.astype(np.uint16))\n", 159 | " elif i >= 10 and i < 100:\n", 160 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img.astype(np.uint16))\n", 161 | " else:\n", 162 | " cv2.imwrite(name+str(i)+\".tif\", img.astype(np.uint16))\n", 163 | " else:\n", 164 | " if index < 10:\n", 165 | " cv2.imwrite(name+\"00\"+str(index)+\".tif\", images.astype(np.uint16))\n", 166 | " elif index >= 10 and index < 100:\n", 167 | " cv2.imwrite(name+\"0\"+str(index)+\".tif\", images.astype(np.uint16))\n", 168 | " else:\n", 169 | " cv2.imwrite(name+str(index)+\".tif\", images.astype(np.uint16)) \n", 170 | "\n", 171 | "def write_mask8(images, name, index=-1):\n", 172 | " \"\"\"\n", 173 | " Write image as 8 bits image\n", 174 | " \"\"\"\n", 175 | " if index == -1:\n", 176 | " for i, img in enumerate(images):\n", 177 | " if i < 10:\n", 178 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img.astype(np.uint8))\n", 179 | " elif i >= 10 and i < 100:\n", 180 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img.astype(np.uint8))\n", 181 | " else:\n", 182 | " cv2.imwrite(name+str(i)+\".tif\", img.astype(np.uint8))\n", 183 | " else:\n", 184 | " if index < 10:\n", 185 | " cv2.imwrite(name+\"000\"+str(index)+\".tif\", images.astype(np.uint8))\n", 186 | " elif index >= 10 and index < 100:\n", 187 | " cv2.imwrite(name+\"00\"+str(index)+\".tif\", images.astype(np.uint8))\n", 188 | " elif index >= 100 and index < 1000:\n", 189 | " cv2.imwrite(name+\"0\"+str(index)+\".tif\", images.astype(np.uint8)) \n", 190 | " elif index >= 1000 and index < 10000:\n", 191 | " cv2.imwrite(name+str(index)+\".tif\", images.astype(np.uint8)) \n", 192 | " else:\n", 193 | " raise\n", 194 | "\n", 195 | "def write_pair8(images, name, index=-1):\n", 196 | " \"\"\"\n", 197 | " Write image as 8 bits image with dilation\n", 198 | " \"\"\"\n", 199 | " for i, img in enumerate(images): \n", 200 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))\n", 201 | " img = cv2.dilate((img*255).astype(np.uint8),kernel,iterations = 3)\n", 202 | " if i < 10:\n", 203 | " cv2.imwrite(name+\"00\"+str(i)+\".tif\", img)\n", 204 | " elif i >= 10 and i < 100:\n", 205 | " cv2.imwrite(name+\"0\"+str(i)+\".tif\", img)\n", 206 | " else:\n", 207 | " cv2.imwrite(name+str(i)+\".tif\", img)" 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "## 1. Cell Segmentatioin Part\n", 215 | "
\n", 216 | "\n", 217 | " 1. Adaptive Thresholding\n", 218 | "
\n", 219 | " This file is to compute adaptive thresholding of image sequence in \n", 220 | " order to generate binary image for Nuclei segmentation.\n", 221 | "\n", 222 | " Problem:\n", 223 | " Due to the low contrast of original image, the adaptive thresholding is not working. \n", 224 | " Therefore, we change to regular threshold with threshold value as 129.\n", 225 | "\n" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": null, 231 | "metadata": { 232 | "collapsed": true 233 | }, 234 | "outputs": [], 235 | "source": [ 236 | "th = None\n", 237 | "img = None\n", 238 | "\n", 239 | "class ADPTIVETHRESH():\n", 240 | " '''\n", 241 | " This class is to provide all function for adaptive thresholding.\n", 242 | "\n", 243 | " '''\n", 244 | " def __init__(self, images):\n", 245 | " self.images = []\n", 246 | " for img in images:\n", 247 | " if len(img.shape) == 3:\n", 248 | " img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 249 | " self.images.append(img.copy())\n", 250 | "\n", 251 | " def applythresh(self, threshold = 50):\n", 252 | " '''\n", 253 | " applythresh function is to convert original image to binary image by thresholding.\n", 254 | "\n", 255 | " Input: image sequence. E.g. [image0, image1, ...]\n", 256 | "\n", 257 | " Output: image sequence after thresholding. E.g. [image0, image1, ...]\n", 258 | " '''\n", 259 | " out = []\n", 260 | " markers = []\n", 261 | " binarymark = []\n", 262 | "\n", 263 | " for img in self.images:\n", 264 | " img = cv2.GaussianBlur(img,(5,5),0).astype(np.uint8)\n", 265 | " _, thresh = cv2.threshold(img,threshold,1,cv2.THRESH_BINARY)\n", 266 | "\n", 267 | " # Using morphlogical operations to imporve the quality of result\n", 268 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(9,9))\n", 269 | " thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)\n", 270 | "\n", 271 | " out.append(thresh)\n", 272 | "\n", 273 | " return out" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": null, 279 | "metadata": { 280 | "collapsed": true 281 | }, 282 | "outputs": [], 283 | "source": [ 284 | "# This part is for testing adaptivethresh.py with single image.\n", 285 | "# Input: an original image\n", 286 | "# Output: Thresholding image\n", 287 | "\n", 288 | "global th\n", 289 | "global img\n", 290 | "\n", 291 | "adaptive = ADPTIVETHRESH(enhance_images)\n", 292 | "th = adaptive.applythresh(50)" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "metadata": { 299 | "collapsed": true 300 | }, 301 | "outputs": [], 302 | "source": [ 303 | "# display images\n", 304 | "for i,img in enumerate(th):\n", 305 | " th[i] = img*255\n", 306 | "os.chdir(\".\")\n", 307 | "write_mask8(th, \"thresh\")\n", 308 | "out = cvt_npimg(th)\n", 309 | "vis_square(out)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "
\n", 317 | "\n", 318 | " 2. Gradient Filed Vector*\n", 319 | "
\n", 320 | " This file is to compute gradient vector field (GVF) and then find the Nuclei center \n", 321 | " with the GVF result. (This part is optinal and I recommend using the distance map directly)\n", 322 | "" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": null, 328 | "metadata": { 329 | "collapsed": true 330 | }, 331 | "outputs": [], 332 | "source": [ 333 | "from scipy import spatial as sp\n", 334 | "from scipy import ndimage\n", 335 | "from scipy.spatial import distance\n", 336 | "\n", 337 | "looplimit = 500\n", 338 | "\n", 339 | "newimg = None\n", 340 | "pair = None\n", 341 | "\n", 342 | "def inbounds(shape, indices):\n", 343 | " assert len(shape) == len(indices)\n", 344 | " for i, ind in enumerate(indices):\n", 345 | " if ind < 0 or ind >= shape[i]:\n", 346 | " return False\n", 347 | " return True\n", 348 | "\n", 349 | "class GVF():\n", 350 | " '''\n", 351 | " This class contains all function for calculating GVF and its following steps.\n", 352 | " \n", 353 | " '''\n", 354 | " def __init__(self, images, thresh):\n", 355 | " \n", 356 | " self.images = images\n", 357 | " self.thresh = thresh\n", 358 | "\n", 359 | " def distancemap(self):\n", 360 | " '''\n", 361 | " This function is to generate distance map of the thresh image. We use the opencv\n", 362 | " function distanceTransform to generate it. Moreover, in this case, we use Euclidiean\n", 363 | " Distance (DIST_L2) as a metric of distance. \n", 364 | "\n", 365 | " Input: None\n", 366 | "\n", 367 | " Output: Image distance map\n", 368 | "\n", 369 | " '''\n", 370 | " return [cv2.distanceTransform(self.thresh[i], distanceType=2, maskSize=0)\\\n", 371 | " for i in range(len(self.thresh))]\n", 372 | "\n", 373 | " def new_image(self, alpha, dismap):\n", 374 | " '''\n", 375 | " This function is to generate a new image combining the oringal image I0 with\n", 376 | " the distance map image Idis by following expression:\n", 377 | " Inew = I0 + alpha*Idis\n", 378 | " In this program, we choose alpha as 0.4.\n", 379 | "\n", 380 | " Input: the weight of distance map: alpha\n", 381 | " the distance map image\n", 382 | "\n", 383 | " Output: new grayscale image\n", 384 | "\n", 385 | " '''\n", 386 | "\n", 387 | " return [self.images[i] + alpha * dismap[i] for i in range(len(self.thresh))]\n", 388 | "\n", 389 | " def compute_gvf(self, newimage):\n", 390 | " '''\n", 391 | " This function is to compute the gradient vector of the imput image.\n", 392 | "\n", 393 | " Input: a grayscale image with size, say m * n * # of images\n", 394 | "\n", 395 | " Output: a 3 dimentional image with size, m * n * 2, where the last dimention is\n", 396 | " the gradient vector (gx, gy)\n", 397 | "\n", 398 | " '''\n", 399 | " kernel_size = 5 # kernel size for blur image before compute gradient\n", 400 | " newimage = [cv2.GaussianBlur((np.clip(newimage[i], 0, 255)).astype(np.uint8),(kernel_size,kernel_size),0)\\\n", 401 | " for i in range(len(self.thresh))]\n", 402 | " # use sobel operator to compute gradient \n", 403 | " temp = np.zeros((newimage[0].shape[0], newimage[0].shape[1], 2), np.float32) # store temp gradient image \n", 404 | " gradimg = [] # output gradient images (height * weight * # of images)\n", 405 | "\n", 406 | " for i in range(len(newimage)):\n", 407 | " # compute sobel operation in x, y directions\n", 408 | " gradx = cv2.Sobel(newimage[i],cv2.CV_64F,1,0,ksize=3)\n", 409 | " grady = cv2.Sobel(newimage[i],cv2.CV_64F,0,1,ksize=3)\n", 410 | " # add the gradient vector\n", 411 | " temp[:,:,0], temp[:,:,1] = gradx, grady\n", 412 | " gradimg.append(temp)\n", 413 | "\n", 414 | " return gradimg\n", 415 | "\n", 416 | " def find_certer(self, gvfimage, index):\n", 417 | " '''\n", 418 | " This function is to find the center of Nuclei.\n", 419 | "\n", 420 | " Input: the gradient vector image (height * weight * 2).\n", 421 | "\n", 422 | " Output: the record image height * weight).\n", 423 | "\n", 424 | " '''\n", 425 | " # Initialize a image to record seed candidates.\n", 426 | " imgpair = np.zeros(gvfimage.shape[:2])\n", 427 | "\n", 428 | " kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))\n", 429 | " dilate = cv2.dilate(self.thresh[index].copy(), kernel, iterations = 1)\n", 430 | " erthresh = cv2.erode(dilate, kernel, iterations = 3)\n", 431 | " while erthresh.sum() > 0:\n", 432 | "\n", 433 | " print \"Image \", index, \"left: \", erthresh.sum(), \"points\"\n", 434 | " # Initialize partical coordinates [y, x]\n", 435 | " y0, x0 = np.where(erthresh>0)\n", 436 | " p0 = np.array([y0[0], x0[0], 1])\n", 437 | "\n", 438 | " # Initialize record coordicates [y, x]\n", 439 | " p1 = np.array([5000, 5000, 1])\n", 440 | "\n", 441 | " # mark the first non-zero point of thresh image to 0\n", 442 | " erthresh[p0[0], p0[1]] = 0\n", 443 | "\n", 444 | " # a variable to record if the point out of bound of image or \n", 445 | " # out of maximum loop times\n", 446 | " outbound = False\n", 447 | "\n", 448 | " # count loop times to limit max loop times\n", 449 | " count = 0\n", 450 | "\n", 451 | " while sp.distance.cdist([p0],[p1]) > 1:\n", 452 | "\n", 453 | " count += 1\n", 454 | " p1 = p0\n", 455 | " u = gvfimage[p0[0], p0[1], 1]\n", 456 | " v = gvfimage[p0[0], p0[1], 0]\n", 457 | " M = np.array([[1, 0, u],\\\n", 458 | " [0, 1, v],\\\n", 459 | " [0, 0, 1]], np.float32)\n", 460 | " p0 = M.dot(p0)\n", 461 | " if not inbounds(self.thresh[index].shape, (p0[0], p0[1])) or count > looplimit:\n", 462 | " outbound = True\n", 463 | " break\n", 464 | "\n", 465 | " if not outbound:\n", 466 | " imgpair[p0[0], p0[1]] += 1\n", 467 | " clear_output(wait=True)\n", 468 | "\n", 469 | " return imgpair.copy()" 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": null, 475 | "metadata": { 476 | "collapsed": true 477 | }, 478 | "outputs": [], 479 | "source": [ 480 | "# This part is for testing gvf.py with single image. (Optional)\n", 481 | "# Input: an original image\n", 482 | "# Output: Thresholding image and seed image\n", 483 | "\n", 484 | "global th\n", 485 | "global newimg\n", 486 | "global pair\n", 487 | "# Nuclei center detection\n", 488 | "gvf = GVF(images, th)\n", 489 | "dismap = gvf.distancemap()\n", 490 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 491 | "gradimg = gvf.compute_gvf(newimg)\n", 492 | "out = []\n", 493 | "pair = []\n", 494 | "pair_raw = []\n", 495 | "kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))\n", 496 | "i = 0\n", 497 | "for i,img in enumerate(gradimg):\n", 498 | " imgpair_raw = gvf.find_certer(img, i)\n", 499 | " pair_raw.append(imgpair_raw)\n", 500 | " neighborhood_size = 20\n", 501 | " data_max = ndimage.filters.maximum_filter(pair_raw[i], neighborhood_size)\n", 502 | " data_max[data_max==0] = 255\n", 503 | " pair.append((pair_raw[i] == data_max).astype(np.uint8))\n", 504 | " write_mask8([pair[i]], \"pair_raw\", i)\n", 505 | " os.chdir(\"PATH_TO_RESULTS\")\n", 506 | " y, x = np.where(pair[i]>0)\n", 507 | " points = zip(y[:], x[:])\n", 508 | " dmap = distance.cdist(points, points, 'euclidean')\n", 509 | " y, x = np.where(dmap<10)\n", 510 | " ps = zip(y[:], x[:])\n", 511 | " for p in ps:\n", 512 | " if p[0] != p[1]:\n", 513 | " pair[i][points[min(p[0], p[1])]] = 0\n", 514 | " dilation = cv2.dilate((pair[i]*255).astype(np.uint8),kernel,iterations = 3)\n", 515 | " out.append(dilation)\n", 516 | "\n", 517 | "out = cvt_npimg(out)\n", 518 | "vis_square(out)" 519 | ] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": {}, 524 | "source": [ 525 | "
\n", 526 | "\n", 527 | " GVF enhance*\n", 528 | "
\n", 529 | " This file is to amend the seed points for watershed. (This part is optinal and I recommend using the distance map directly)\n", 530 | "" 531 | ] 532 | }, 533 | { 534 | "cell_type": "code", 535 | "execution_count": null, 536 | "metadata": { 537 | "collapsed": true 538 | }, 539 | "outputs": [], 540 | "source": [ 541 | "from scipy import spatial as sp\n", 542 | "from scipy import ndimage\n", 543 | "from scipy.spatial import distance\n", 544 | "\n", 545 | "gvf = GVF(images, th)\n", 546 | "dismap = gvf.distancemap()\n", 547 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 548 | "# TODO this part is designed to amend the result of gvf. \n", 549 | "pair = []\n", 550 | "path=os.path.join(\"PATH_TO_RESULTS\")\n", 551 | "for r,d,f in os.walk(path):\n", 552 | " for files in f:\n", 553 | " if files[:5].lower()=='seed':\n", 554 | " print files\n", 555 | " temp = cv2.imread(os.path.join(r,files))\n", 556 | " temp = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 557 | " y, x = np.where(temp>0)\n", 558 | " points = zip(y[:], x[:])\n", 559 | " dmap = distance.cdist(points, points, 'euclidean')\n", 560 | " y, x = np.where(dmap<10)\n", 561 | " ps = zip(y[:], x[:])\n", 562 | " for p in ps:\n", 563 | " if p[0] != p[1]:\n", 564 | " temp[points[min(p[0], p[1])]] = 0\n", 565 | " pair.append(temp)\n", 566 | " clear_output(wait=True)\n", 567 | "print \"finish!\"" 568 | ] 569 | }, 570 | { 571 | "cell_type": "markdown", 572 | "metadata": {}, 573 | "source": [ 574 | "
\n", 575 | "\n", 576 | " 2. Distance Map (Recommend)\n", 577 | "
\n", 578 | " This file uses distance map to generate the seed points for watershed. Although it has nothing to do with GVF, you still need to load the GVF class, since it needs some helper functions in the class.\n", 579 | "" 580 | ] 581 | }, 582 | { 583 | "cell_type": "code", 584 | "execution_count": null, 585 | "metadata": { 586 | "collapsed": true 587 | }, 588 | "outputs": [], 589 | "source": [ 590 | "gvf = GVF(images, th)\n", 591 | "dismap = gvf.distancemap()\n", 592 | "newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4.\n", 593 | "out = []\n", 594 | "pair = []\n", 595 | "kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))\n", 596 | "for i,img in enumerate(dismap):\n", 597 | " neighborhood_size = 20\n", 598 | " data_max = ndimage.filters.maximum_filter(img, neighborhood_size)\n", 599 | " data_max[data_max==0] = 255\n", 600 | " pair.append((img == data_max).astype(np.uint8))\n", 601 | " y, x = np.where(pair[i]>0)\n", 602 | " points = zip(y[:], x[:])\n", 603 | " dmap = distance.cdist(points, points, 'euclidean')\n", 604 | " y, x = np.where(dmap<20)\n", 605 | " ps = zip(y[:], x[:])\n", 606 | " for p in ps:\n", 607 | " if p[0] != p[1]:\n", 608 | " pair[i][points[min(p[0], p[1])]] = 0\n", 609 | " dilation = cv2.dilate((pair[i]*255).astype(np.uint8),kernel,iterations = 1)\n", 610 | " out.append(dilation)\n", 611 | " os.chdir(\".\")\n", 612 | " write_mask8(dilation, \"seed_point\", i)\n", 613 | "\n", 614 | "out = cvt_npimg(out)\n", 615 | "vis_square(out)" 616 | ] 617 | }, 618 | { 619 | "cell_type": "markdown", 620 | "metadata": { 621 | "collapsed": true 622 | }, 623 | "source": [ 624 | "
\n", 625 | "\n", 626 | " 3. Watershed\n", 627 | "
\n", 628 | " This file is to compute watershed given the seed image in the gvf.py.\n", 629 | "" 630 | ] 631 | }, 632 | { 633 | "cell_type": "code", 634 | "execution_count": null, 635 | "metadata": { 636 | "collapsed": true 637 | }, 638 | "outputs": [], 639 | "source": [ 640 | "import cv2\n", 641 | "import numpy as np\n", 642 | "from numpy import unique\n", 643 | "import copy as cp\n", 644 | "\n", 645 | "bmarks = None\n", 646 | "marks = None\n", 647 | "\n", 648 | "class WATERSHED():\n", 649 | " '''\n", 650 | " This class contains all the function to compute watershed.\n", 651 | "\n", 652 | " '''\n", 653 | " def __init__(self, images, markers):\n", 654 | " self.images = images\n", 655 | " self.markers = markers\n", 656 | "\n", 657 | " def is_over_long(self, img, max_lenth=50):\n", 658 | " rows = np.any(img, axis=1)\n", 659 | " cols = np.any(img, axis=0)\n", 660 | " if not len(img[img>0]):\n", 661 | " return True\n", 662 | " rmin, rmax = np.where(rows)[0][[0, -1]]\n", 663 | " cmin, cmax = np.where(cols)[0][[0, -1]]\n", 664 | " if (rmax-rmin)>max_lenth or (cmax-cmin)>max_lenth:\n", 665 | " return True\n", 666 | " else:\n", 667 | " return False\n", 668 | " \n", 669 | " def watershed_compute(self):\n", 670 | " '''\n", 671 | " This function is to compute watershed given the newimage and the seed image\n", 672 | " (center candidates). In this function, we use cv2.watershed to implement watershed.\n", 673 | "\n", 674 | " Input: newimage (height * weight * # of images)\n", 675 | "\n", 676 | " Output: watershed images (height * weight * # of images)\n", 677 | "\n", 678 | " '''\n", 679 | " result = []\n", 680 | " outmark = []\n", 681 | " outbinary = []\n", 682 | "\n", 683 | " for i in range(len(self.images)):\n", 684 | " print \"image: \", i\n", 685 | " # generate a 3-channel image in order to use cv2.watershed\n", 686 | " imgcolor = np.zeros((self.images[i].shape[0], self.images[i].shape[1], 3), np.uint8)\n", 687 | " for c in range(3): \n", 688 | " imgcolor[:,:,c] = self.images[i]\n", 689 | "\n", 690 | " # compute marker image (labelling)\n", 691 | " if len(self.markers[i].shape) == 3:\n", 692 | " self.markers[i] = cv2.cvtColor(self.markers[i],cv2.COLOR_BGR2GRAY)\n", 693 | " _, mark = cv2.connectedComponents(self.markers[i])\n", 694 | " \n", 695 | " # watershed!\n", 696 | " mark = cv2.watershed(imgcolor,mark)\n", 697 | " \n", 698 | " u, counts = unique(mark, return_counts=True)\n", 699 | " counter = dict(zip(u, counts))\n", 700 | " for index in counter:\n", 701 | " temp_img = np.zeros_like(mark)\n", 702 | " temp_img[mark==index] = 255\n", 703 | " if self.is_over_long(temp_img):\n", 704 | " mark[mark==index] = 0\n", 705 | " continue\n", 706 | " if counter[index] > 3000:\n", 707 | " mark[mark==index] = 0\n", 708 | " continue\n", 709 | " \n", 710 | " labels = list(set(mark[mark>0]))\n", 711 | " length = len(labels)\n", 712 | " temp_img = mark.copy()\n", 713 | " for original, new in zip(labels, range(1,length+1)):\n", 714 | " temp_img[mark==original] = new\n", 715 | " mark = temp_img\n", 716 | " \n", 717 | " # mark image and add to the result \n", 718 | " temp = cv2.cvtColor(imgcolor,cv2.COLOR_BGR2GRAY)\n", 719 | " result.append(temp)\n", 720 | " outmark.append(mark.astype(np.uint8))\n", 721 | "\n", 722 | " binary = mark.copy()\n", 723 | " binary[mark>0] = 255\n", 724 | " outbinary.append(binary.astype(np.uint8))\n", 725 | " clear_output(wait=True)\n", 726 | "\n", 727 | " return result, outbinary, outmark" 728 | ] 729 | }, 730 | { 731 | "cell_type": "code", 732 | "execution_count": null, 733 | "metadata": { 734 | "collapsed": true 735 | }, 736 | "outputs": [], 737 | "source": [ 738 | "# This part is for testing watershed.py with single image.\n", 739 | "# Output: Binary image after watershed\n", 740 | "\n", 741 | "global bmarks\n", 742 | "global marks\n", 743 | "\n", 744 | "# watershed\n", 745 | "ws = WATERSHED(newimg, pair) \n", 746 | "wsimage, bmarks, marks = ws.watershed_compute()\n", 747 | "\n", 748 | "out = cvt_npimg(np.clip(bmarks, 0, 255)).astype(np.uint8)\n", 749 | "vis_square(out)\n", 750 | "os.chdir(\"PATH_TO_RESULT_MASK\")\n", 751 | "write_mask16(marks, \"mask\")\n", 752 | "os.chdir(\"PATH_TO_RESULT_BINARY\")\n", 753 | "write_mask8(out, \"binary\")\n", 754 | "clear_output(wait=True)" 755 | ] 756 | }, 757 | { 758 | "cell_type": "markdown", 759 | "metadata": {}, 760 | "source": [ 761 | "\n", 762 | " 4. Segmentation Evaluation\n", 763 | "
\n", 764 | " This file is to evaluate our algorithm about segmentation in jaccard coefficient.\n", 765 | "" 766 | ] 767 | }, 768 | { 769 | "cell_type": "code", 770 | "execution_count": null, 771 | "metadata": { 772 | "collapsed": true 773 | }, 774 | "outputs": [], 775 | "source": [ 776 | "def list2pts(ptslist):\n", 777 | " list_y = np.array([ptslist[0]])\n", 778 | " list_x = np.array([ptslist[1]])\n", 779 | " return np.append(list_y, list_x).reshape(2, len(list_y[0])).T\n", 780 | "\n", 781 | "def unique_rows(a):\n", 782 | " a = np.ascontiguousarray(a)\n", 783 | " unique_a = np.unique(a.view([('', a.dtype)]*a.shape[1]))\n", 784 | " return unique_a.view(a.dtype).reshape((unique_a.shape[0], a.shape[1]))\n", 785 | "\n", 786 | "# read image sequence\n", 787 | "# The training set locates at \"resource/training/01\" and \"resource/training/02\"\n", 788 | "# The ground truth of training set locates at \"resource/training/GT_01\" and \n", 789 | "# \"resource/training/GT_02\"\n", 790 | "# The testing set locates at \"resource/testing/01\" and \"resource/testing/02\"\n", 791 | "\n", 792 | "path = \"PATH_TO_GT_SEGMENTATION\"\n", 793 | "gts = []\n", 794 | "for r,d,f in os.walk(path):\n", 795 | " for files in f:\n", 796 | " if files[-3:].lower()=='tif':\n", 797 | " temp = cv2.imread(os.path.join(r,files), cv2.IMREAD_UNCHANGED)\n", 798 | " gts.append([temp, files[-6:-4]])\n", 799 | "print \"number of gts: \", len(gts)\n", 800 | "\n", 801 | "path= \"PATH_TO_SEGMENTATION_RESULTS\"\n", 802 | "binarymarks = []\n", 803 | "for r,d,f in os.walk(path):\n", 804 | " for files in f:\n", 805 | " if files[:4]=='mark':\n", 806 | " temp = cv2.imread(os.path.join(r,files))\n", 807 | " gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) \n", 808 | " binarymarks.append([gray, files[-6:-4]])\n", 809 | "print \"number of segmentation image: \", len(binarymarks)\n", 810 | "\n", 811 | "jaccards = []\n", 812 | "\n", 813 | "for gt in gts:\n", 814 | " for binarymark in binarymarks:\n", 815 | " if gt[1] == binarymark[1]:\n", 816 | " print \"enter...\", gt[1]\n", 817 | " list_pts = set(gt[0][gt[0]>0])\n", 818 | " list_seg = set(binarymark[0][binarymark[0]>0])\n", 819 | " for pt in list_pts:\n", 820 | " for seg in list_seg:\n", 821 | " pts_gt = np.where(gt[0]==pt)\n", 822 | " pts_seg = np.where(binarymark[0]==seg)\n", 823 | " pts_gt = list2pts(pts_gt)\n", 824 | "\n", 825 | " pts_seg = list2pts(pts_seg)\n", 826 | " pts = np.append(pts_gt, pts_seg).reshape(len(pts_gt)+len(pts_seg),2)\n", 827 | " union_pts = unique_rows(pts)\n", 828 | "\n", 829 | " union = float(len(union_pts))\n", 830 | " intersection = float(len(pts_seg) + len(pts_gt) - len(union_pts))\n", 831 | "\n", 832 | " if intersection/union > 0.5:\n", 833 | " jaccards.append(intersection/union)\n", 834 | "clear_output(wait=True)\n", 835 | "jaccard = float(sum(jaccards))/float(len(jaccards))\n", 836 | "print \"jaccard: \", jaccard, \"number of Nuclei: \", len(jaccards)" 837 | ] 838 | }, 839 | { 840 | "cell_type": "markdown", 841 | "metadata": {}, 842 | "source": [ 843 | "## 2. Cell Tracking Part\n", 844 | "
\n", 845 | "\n", 846 | " 1. Graph Construction\n", 847 | "
\n", 848 | " This file is to generate a neighboring graph contraction using \n", 849 | " Delaunary Triangulation.\n", 850 | "" 851 | ] 852 | }, 853 | { 854 | "cell_type": "code", 855 | "execution_count": null, 856 | "metadata": { 857 | "collapsed": true 858 | }, 859 | "outputs": [], 860 | "source": [ 861 | "centroid = None\n", 862 | "slope_length = None\n", 863 | "class GRAPH():\n", 864 | " '''\n", 865 | " This class contains all the functions needed to compute \n", 866 | " Delaunary Triangulation.\n", 867 | "\n", 868 | " '''\n", 869 | " def __init__(self, mark, binary, index):\n", 870 | " '''\n", 871 | " Input: the grayscale mark image with different label on each segments\n", 872 | " the binary image of the mark image\n", 873 | " the index of the image\n", 874 | "\n", 875 | " '''\n", 876 | " self.mark = mark[index]\n", 877 | " self.binary = binary[index]\n", 878 | " \n", 879 | " def rect_contains(self, rect, point):\n", 880 | " '''\n", 881 | " Check if a point is inside the image\n", 882 | "\n", 883 | " Input: the size of the image \n", 884 | " the point that want to test\n", 885 | "\n", 886 | " Output: if the point is inside the image\n", 887 | "\n", 888 | " '''\n", 889 | " if point[0] < rect[0] :\n", 890 | " return False\n", 891 | " elif point[1] < rect[1] :\n", 892 | " return False\n", 893 | " elif point[0] > rect[2] :\n", 894 | " return False\n", 895 | " elif point[1] > rect[3] :\n", 896 | " return False\n", 897 | " return True\n", 898 | " \n", 899 | " def draw_point(self, img, p, color ):\n", 900 | " '''\n", 901 | " Draw a point\n", 902 | "\n", 903 | " '''\n", 904 | " cv2.circle( img, (p[1], p[0]), 2, color, cv2.FILLED, 16, 0 )\n", 905 | " \n", 906 | " def draw_delaunay(self, img, subdiv, delaunay_color):\n", 907 | " '''\n", 908 | " Draw delaunay triangles and store these lines\n", 909 | "\n", 910 | " Input: the image want to draw\n", 911 | " the set of points: format as cv2.Subdiv2D\n", 912 | " the color want to use\n", 913 | "\n", 914 | " Output: the slope and length of each line ()\n", 915 | "\n", 916 | " '''\n", 917 | " triangleList = subdiv.getTriangleList();\n", 918 | " size = img.shape\n", 919 | " r = (0, 0, size[0], size[1])\n", 920 | "\n", 921 | " slope_length = [[]]\n", 922 | " for i in range(self.mark.max()-1):\n", 923 | " slope_length.append([])\n", 924 | "\n", 925 | " for t_i, t in enumerate(triangleList):\n", 926 | " \n", 927 | " pt1 = (int(t[0]), int(t[1]))\n", 928 | " pt2 = (int(t[2]), int(t[3]))\n", 929 | " pt3 = (int(t[4]), int(t[5]))\n", 930 | " \n", 931 | " if self.rect_contains(r, pt1) and self.rect_contains(r, pt2) and self.rect_contains(r, pt3):\n", 932 | " \n", 933 | " # draw lines\n", 934 | " cv2.line(img, (pt1[1], pt1[0]), (pt2[1], pt2[0]), delaunay_color, 1, 16, 0)\n", 935 | " cv2.line(img, (pt2[1], pt2[0]), (pt3[1], pt3[0]), delaunay_color, 1, 16, 0)\n", 936 | " cv2.line(img, (pt3[1], pt3[0]), (pt1[1], pt1[0]), delaunay_color, 1, 16, 0)\n", 937 | " \n", 938 | " # store the length of line segments and their slopes\n", 939 | " for p0 in [pt1, pt2, pt3]:\n", 940 | " for p1 in [pt1, pt2, pt3]:\n", 941 | " if p0 != p1:\n", 942 | " temp = self.length_slope(p0, p1)\n", 943 | " if temp not in slope_length[self.mark[p0]-1]:\n", 944 | " slope_length[self.mark[p0]-1].append(temp) \n", 945 | "\n", 946 | " return slope_length\n", 947 | "\n", 948 | " def length_slope(self, p0, p1):\n", 949 | " '''\n", 950 | " This function is to compute the length and theta for the given two points.\n", 951 | "\n", 952 | " Input: two points with the format (y, x)\n", 953 | "\n", 954 | " '''\n", 955 | " if p1[1]-p0[1]:\n", 956 | " slope = (p1[0]-p0[0]) / (p1[1]-p0[1])\n", 957 | " else:\n", 958 | " slope = 1e10\n", 959 | "\n", 960 | " length = np.sqrt((p1[0]-p0[0])**2 + (p1[1]-p0[1])**2)\n", 961 | "\n", 962 | " return length, slope\n", 963 | "\n", 964 | " def generate_points(self):\n", 965 | " '''\n", 966 | " Find the centroid of each segmentation\n", 967 | "\n", 968 | " '''\n", 969 | " centroids = []\n", 970 | " label = []\n", 971 | " max_label = self.mark.max()\n", 972 | "\n", 973 | " for i in range(1, max_label+1):\n", 974 | " img = self.mark.copy()\n", 975 | " img[img!=i] = 0\n", 976 | " if img.sum():\n", 977 | " _, contours,hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_TC89_KCOS)\n", 978 | " m = cv2.moments(contours[0])\n", 979 | "\n", 980 | " if m['m00']:\n", 981 | " label.append(i)\n", 982 | " centroids.append(( int(round(m['m01']/m['m00'])),\\\n", 983 | " int(round(m['m10']/m['m00'])) ))\n", 984 | " else:\n", 985 | " label.append(i)\n", 986 | " centroids.append(( 0,0 ))\n", 987 | "\n", 988 | " return centroids, label\n", 989 | "\n", 990 | " def run(self, animate = False):\n", 991 | " '''\n", 992 | " The pipline of graph construction.\n", 993 | "\n", 994 | " Input: if showing a animation (False for default)\n", 995 | "\n", 996 | " Output: centroids: # of segments * 2 (y, x)\n", 997 | " slopes and length: # of segments * # of slope_length\n", 998 | "\n", 999 | " '''\n", 1000 | " # Read in the image.\n", 1001 | " img_orig = self.binary.copy()\n", 1002 | " \n", 1003 | " # Rectangle to be used with Subdiv2D\n", 1004 | " size = img_orig.shape\n", 1005 | " rect = (0, 0, size[0], size[1])\n", 1006 | " \n", 1007 | " # Create an instance of Subdiv2D\n", 1008 | " subdiv = cv2.Subdiv2D(rect);\n", 1009 | " \n", 1010 | " # find the centroid of each segments\n", 1011 | " points, label = self.generate_points()\n", 1012 | "\n", 1013 | "\n", 1014 | " # add and sort the centroid to a numpy array for post processing\n", 1015 | " centroid = np.zeros((self.mark.max(), 2))\n", 1016 | " for p, l in zip(points, label):\n", 1017 | " centroid[l-1] = p\n", 1018 | "\n", 1019 | " outimg = []\n", 1020 | " # Insert points into subdiv\n", 1021 | " for idx_p, p in enumerate(points):\n", 1022 | " subdiv.insert(p)\n", 1023 | " \n", 1024 | " # Show animation\n", 1025 | " if animate:\n", 1026 | " img_copy = img_orig.copy()\n", 1027 | " # Draw delaunay triangles\n", 1028 | " self.draw_delaunay( img_copy, subdiv, (255, 255, 255));\n", 1029 | " outimg.append(img_copy)\n", 1030 | " display_image(img_copy)\n", 1031 | " img_copy = cv2.resize(img_copy, (314, 200))\n", 1032 | " cv2.imwrite(\"delaunay_\" + str(idx_p).zfill(3) + \".png\", img_copy)\n", 1033 | " clear_output(wait=True)\n", 1034 | " \n", 1035 | " # Draw delaunay triangles\n", 1036 | " slope_length = self.draw_delaunay( img_orig, subdiv, (255, 255, 255));\n", 1037 | " \n", 1038 | " # Draw points\n", 1039 | " for p in points :\n", 1040 | " self.draw_point(img_orig, p, (0,0,255))\n", 1041 | " \n", 1042 | " # show images\n", 1043 | " if animate:\n", 1044 | " display_image(img_orig)\n", 1045 | "\n", 1046 | " print \"length of centroid: \", len(centroid)\n", 1047 | " return centroid, slope_length" 1048 | ] 1049 | }, 1050 | { 1051 | "cell_type": "code", 1052 | "execution_count": null, 1053 | "metadata": { 1054 | "collapsed": true 1055 | }, 1056 | "outputs": [], 1057 | "source": [ 1058 | "# This part is the small test for graph_contruction.py.\n", 1059 | "# Input: grayscale marker image\n", 1060 | "# binary marker image\n", 1061 | "# Output: a text file includes the centroid and the length and slope for each neighbor. \n", 1062 | "\n", 1063 | "# Build Delaunay Triangulation\n", 1064 | "global centroid\n", 1065 | "global slope_length\n", 1066 | "centroid = []\n", 1067 | "slope_length = []\n", 1068 | "for i in range(len(images)):\n", 1069 | " print \" graph_construction: image \", i\n", 1070 | " print \"max pixel: \", marks[i].max()\n", 1071 | " graph = GRAPH(marks, bmarks, i)\n", 1072 | " if i == 0:\n", 1073 | " tempcentroid, tempslope_length = graph.run(True)\n", 1074 | " else:\n", 1075 | " tempcentroid, tempslope_length = graph.run()\n", 1076 | " centroid.append(tempcentroid)\n", 1077 | " slope_length.append(tempslope_length)\n", 1078 | " clear_output(wait=True)\n", 1079 | "print \"finish!\"" 1080 | ] 1081 | }, 1082 | { 1083 | "cell_type": "markdown", 1084 | "metadata": {}, 1085 | "source": [ 1086 | "\n", 1087 | " 2. Matching\n", 1088 | "
\n", 1089 | " This file is to match nuclei in two consecutive frames by Phase Controlled Optimal Matching. \n", 1090 | " It includes two part: \n", 1091 | "\t\t1) Dissimilarity measure\n", 1092 | "\t\t2) Matching\n", 1093 | "" 1094 | ] 1095 | }, 1096 | { 1097 | "cell_type": "code", 1098 | "execution_count": null, 1099 | "metadata": { 1100 | "collapsed": true 1101 | }, 1102 | "outputs": [], 1103 | "source": [ 1104 | "import imageio\n", 1105 | "from pyefd import elliptic_fourier_descriptors\n", 1106 | "\n", 1107 | "Max_dis = 100000\n", 1108 | "\n", 1109 | "def write_image(image, title, index, imgformat='.tif'):\n", 1110 | " if index < 10:\n", 1111 | " name = '00'+str(index)\n", 1112 | " else:\n", 1113 | " name = '0'+str(index)\n", 1114 | " cv2.imwrite(title+name+imgformat, image.astype(np.uint16))\n", 1115 | "\n", 1116 | "class FEAVECTOR():\n", 1117 | " '''\n", 1118 | " This class builds a feature vector for each segments.\n", 1119 | " The format of each vector is: \n", 1120 | " v(k,i) = [c(k,i), s(k, i), h(k, i), e(k, i)], where k is the \n", 1121 | " index of the image (frame) and i is the label of each segment. \n", 1122 | "\n", 1123 | " c(k,i): the centroid of each segment (y, x);\n", 1124 | " s(k,i): the binary shape of each segment;\n", 1125 | " h(k,i): the intensity distribution (hsitogram) of the segment;\n", 1126 | " e(k,i): the spatial distribution of the segment. Its format is \n", 1127 | " like (l(k, i, p), theta(k, i, p)), where p represent different\n", 1128 | " line connected with different segment. \n", 1129 | "\n", 1130 | " '''\n", 1131 | " def __init__(self, centroid=None, shape=None, histogram=None, spatial=None, \\\n", 1132 | " ID=None, start = None, end=None, label=None, ratio=None, area=None, cooc=None):\n", 1133 | " self.c = centroid\n", 1134 | " self.s = shape\n", 1135 | " self.h = histogram\n", 1136 | " self.e = spatial\n", 1137 | " self.id = ID\n", 1138 | " self.start = start\n", 1139 | " self.end = end\n", 1140 | " self.l = label\n", 1141 | " self.a = area\n", 1142 | " self.r = ratio\n", 1143 | " self.cm = cooc\n", 1144 | "\n", 1145 | " def add_id(self, num, index):\n", 1146 | " '''\n", 1147 | " This function adds cell id for each cell.\n", 1148 | "\n", 1149 | " '''\n", 1150 | " if index == 0:\n", 1151 | " self.id = np.linspace(1, num, num)\n", 1152 | " else:\n", 1153 | " self.id= np.linspace(-1, -1, num)\n", 1154 | "\n", 1155 | " def add_label(self):\n", 1156 | " '''\n", 1157 | " This function is to add labels for each neclei for post process.\n", 1158 | "\n", 1159 | " '''\n", 1160 | " self.l = np.linspace(0, 0, len(self.c))\n", 1161 | "\n", 1162 | " def set_centroid(self, centroid):\n", 1163 | " '''\n", 1164 | " This function sets the centroid for all neclei.\n", 1165 | "\n", 1166 | " Input: the set of centroid: # of images * # of neclei * 2 (y, x)\n", 1167 | "\n", 1168 | " Output: None\n", 1169 | "\n", 1170 | " '''\n", 1171 | " self.c = centroid\n", 1172 | "\n", 1173 | " def set_spatial(self, spatial):\n", 1174 | " '''\n", 1175 | " This function sets the spatial distrbution for all neclei.\n", 1176 | "\n", 1177 | " Input: the set of centroid: # of images * # of neclei * # of line segments (length, slope)\n", 1178 | "\n", 1179 | " Output: None\n", 1180 | "\n", 1181 | " '''\n", 1182 | " self.e = spatial\n", 1183 | "\n", 1184 | " def set_shape(self, image, marker):\n", 1185 | " '''\n", 1186 | " This function sets the binary shape for all necluei.\n", 1187 | "\n", 1188 | " Input: the original images: # of images * height * weight\n", 1189 | " the labeled images: # of images * nucei's height * nucei's weight ()\n", 1190 | "\n", 1191 | " Output: None\n", 1192 | "\n", 1193 | " '''\n", 1194 | " def boundingbox(image):\n", 1195 | " y, x = np.where(image)\n", 1196 | " return min(x), min(y), max(x), max(y)\n", 1197 | "\n", 1198 | " shape = []\n", 1199 | "\n", 1200 | " for label in range(1, marker.max()+1):\n", 1201 | " tempimg = marker.copy()\n", 1202 | " tempimg[tempimg!=label] = 0\n", 1203 | " tempimg[tempimg==label] = 1\n", 1204 | " if tempimg.sum():\n", 1205 | " minx, miny, maxx, maxy = boundingbox(tempimg)\n", 1206 | " shape.append((tempimg[miny:maxy+1, minx:maxx+1], image[miny:maxy+1, minx:maxx+1]))\n", 1207 | " else:\n", 1208 | " shape.append(([], []))\n", 1209 | "\n", 1210 | " self.s = shape\n", 1211 | "\n", 1212 | " def set_histogram(self):\n", 1213 | " '''\n", 1214 | " Note: this function must be implemneted after set_shape().\n", 1215 | "\n", 1216 | " '''\n", 1217 | " def computehistogram(image):\n", 1218 | " h, w = image.shape[:2]\n", 1219 | " his = np.zeros((256,1))\n", 1220 | " for y in range(h):\n", 1221 | " for x in range(w):\n", 1222 | " his[image[y, x], 0] += 1\n", 1223 | " return his\n", 1224 | "\n", 1225 | " assert self.s != None, \"this function must be implemneted after set_shape().\"\n", 1226 | "\n", 1227 | " his = []\n", 1228 | "\n", 1229 | " for j in range(len(self.s)):\n", 1230 | " img = self.s[j][1]\n", 1231 | " if len(img):\n", 1232 | " temphis = computehistogram(img)\n", 1233 | " his.append(temphis)\n", 1234 | " else:\n", 1235 | " his.append(np.zeros((256,1)))\n", 1236 | "\n", 1237 | " self.h = his\n", 1238 | "\n", 1239 | " def add_efd(self):\n", 1240 | " coeffs = []\n", 1241 | " for i in range(len(self.s)):\n", 1242 | " try:\n", 1243 | " _, contours, hierarchy = cv2.findContours(self.s[i][0].astype(np.uint8), 1, 2)\n", 1244 | " if not len(contours):\n", 1245 | " coeffs.append(0)\n", 1246 | " continue\n", 1247 | " cnt = contours[0]\n", 1248 | " if len(cnt) >= 5:\n", 1249 | " contour = []\n", 1250 | " for i in range(len(contours[0])):\n", 1251 | " contour.append(contours[0][i][0])\n", 1252 | " coeffs.append(elliptic_fourier_descriptors(contour, order=10, normalize=False)) \n", 1253 | " else:\n", 1254 | " coeffs.append(0)\n", 1255 | " except AttributeError:\n", 1256 | " coeffs.append(0)\n", 1257 | " self.r = coeffs\n", 1258 | "\n", 1259 | " def add_co_occurrence(self, level=10):\n", 1260 | " '''\n", 1261 | " This funciton is to generate co-occurrence matrix for each cell. The structure of\n", 1262 | " output coefficients is:\n", 1263 | " [Entropy, Energy, Contrast, Homogeneity]\n", 1264 | " '''\n", 1265 | " # generate P metrix.\n", 1266 | " self.cm = []\n", 1267 | " for j in range(len(self.s)):\n", 1268 | " if not len(self.s[j][1]):\n", 1269 | " p_0 = np.zeros((level,level))\n", 1270 | " p_45 = np.zeros((level,level))\n", 1271 | " p_90 = np.zeros((level,level))\n", 1272 | " p_135 = np.zeros((level,level))\n", 1273 | " self.cm.append([np.array([0, 0, 0, 0]),[p_0, p_45, p_90, p_135]])\n", 1274 | " continue\n", 1275 | " max_p, min_p = np.max(self.s[j][1]), np.min(self.s[j][1])\n", 1276 | " range_p = max_p - min_p\n", 1277 | " img = np.round((np.asarray(self.s[j][1]).astype(np.float32)-min_p)/range_p*level)\n", 1278 | " h, w = img.shape[:2]\n", 1279 | " p_0 = np.zeros((level,level))\n", 1280 | " p_45 = np.zeros((level,level))\n", 1281 | " p_90 = np.zeros((level,level))\n", 1282 | " p_135 = np.zeros((level,level))\n", 1283 | " for y in range(h):\n", 1284 | " for x in range(w):\n", 1285 | " try:\n", 1286 | " p_0[img[y,x],img[y,x+1]] += 1\n", 1287 | " except IndexError:\n", 1288 | " pass\n", 1289 | " try:\n", 1290 | " p_0[img[y,x],img[y,x-1]] += 1\n", 1291 | " except IndexError:\n", 1292 | " pass\n", 1293 | " try:\n", 1294 | " p_90[img[y,x],img[y+1,x]] += 1\n", 1295 | " except IndexError:\n", 1296 | " pass\n", 1297 | " try:\n", 1298 | " p_90[img[y,x],img[y-1,x]] += 1\n", 1299 | " except IndexError:\n", 1300 | " pass\n", 1301 | " try:\n", 1302 | " p_45[img[y,x],img[y+1,x+1]] += 1\n", 1303 | " except IndexError:\n", 1304 | " pass\n", 1305 | " try:\n", 1306 | " p_45[img[y,x],img[y-1,x-1]] += 1\n", 1307 | " except IndexError:\n", 1308 | " pass\n", 1309 | " try:\n", 1310 | " p_135[img[y,x],img[y+1,x-1]] += 1\n", 1311 | " except IndexError:\n", 1312 | " pass\n", 1313 | " try:\n", 1314 | " p_135[img[y,x],img[y-1,x+1]] += 1\n", 1315 | " except IndexError:\n", 1316 | " pass\n", 1317 | " Entropy, Energy, Contrast, Homogeneity = 0, 0, 0, 0\n", 1318 | " for y in range(10):\n", 1319 | " for x in range(10):\n", 1320 | " if 0 not in [p_0[y,x], p_45[y,x], p_90[y,x], p_135[y,x]]:\n", 1321 | " Entropy -= (p_0[y,x]*np.log2(p_0[y,x])+\\\n", 1322 | " p_45[y,x]*np.log2(p_45[y,x])+\\\n", 1323 | " p_90[y,x]*np.log2(p_90[y,x])+\\\n", 1324 | " p_135[y,x]*np.log2(p_135[y,x]))/4\n", 1325 | " else:\n", 1326 | " temp = 0\n", 1327 | " for p in [p_0[y,x], p_45[y,x], p_90[y,x], p_135[y,x]]:\n", 1328 | " if p != 0:\n", 1329 | " temp += p*np.log2(p)\n", 1330 | " Entropy -= temp/4\n", 1331 | " Energy += (p_0[y,x]**2+\\\n", 1332 | " p_45[y,x]**2+\\\n", 1333 | " p_90[y,x]**2+\\\n", 1334 | " p_135[y,x]**2)/4\n", 1335 | " Contrast += (x-y)**2*(p_0[y,x]+\\\n", 1336 | " p_45[y,x]+\\\n", 1337 | " p_90[y,x]+\\\n", 1338 | " p_135[y,x])/4\n", 1339 | " Homogeneity += (p_0[y,x]+\\\n", 1340 | " p_45[y,x]+\\\n", 1341 | " p_90[y,x]+\\\n", 1342 | " p_135[y,x])/(4*(1+abs(x-y)))\n", 1343 | " self.cm.append([np.array([Entropy, Energy, Contrast, Homogeneity]),[p_0, p_45, p_90, p_135]])\n", 1344 | "\n", 1345 | " def add_area(self):\n", 1346 | " area = []\n", 1347 | " for i in range(len(self.s)):\n", 1348 | " area.append(np.count_nonzero(self.s[i][0]))\n", 1349 | " self.a = area\n", 1350 | "\n", 1351 | " def generate_vector(self):\n", 1352 | " '''\n", 1353 | " This function is to convert the vector maxtrics into a list.\n", 1354 | "\n", 1355 | " Output: a list of vector: [v0, v1, ....]\n", 1356 | "\n", 1357 | " '''\n", 1358 | " vector = []\n", 1359 | " for i in range(len(self.c)):\n", 1360 | " vector.append(FEAVECTOR(centroid=self.c[i],shape=self.s[i],\\\n", 1361 | " histogram=self.h[i],spatial=self.e[i],\\\n", 1362 | " ID=self.id[i],label=self.l[i],\\\n", 1363 | " ratio=self.r[i],area=self.a[i], cooc=self.cm[i]))\n", 1364 | " return vector\n", 1365 | "\n", 1366 | "\n", 1367 | "\n", 1368 | "def set_date(vectors):\n", 1369 | " '''\n", 1370 | " This function is to add the start and end frame of each vector and\n", 1371 | " combine the vector with same id.\n", 1372 | "\n", 1373 | " Input: the list of vectors in different frames. \n", 1374 | "\n", 1375 | " Output: the list of vectors of all cell with different id. \n", 1376 | "\n", 1377 | " '''\n", 1378 | " max_id = 0\n", 1379 | " for vector in vectors:\n", 1380 | " for pv in vector:\n", 1381 | " if pv.id > max_id:\n", 1382 | " max_id = pv.id\n", 1383 | "\n", 1384 | " output = np.zeros((max_id, 4))\n", 1385 | " output[:,0] = np.linspace(1, max_id, max_id) # set the cell ID\n", 1386 | " output[:,1] = len(vectors)\n", 1387 | " for frame, vector in enumerate(vectors):\n", 1388 | " for pv in vector:\n", 1389 | " if output[pv.id-1][1] > frame: # set the start frame\n", 1390 | " output[pv.id-1][1] = frame\n", 1391 | " if output[pv.id-1][2] < frame: # set the end frame\n", 1392 | " output[pv.id-1][2] = frame\n", 1393 | " output[pv.id-1][3] = pv.l # set tht cell parent ID\n", 1394 | "\n", 1395 | " return output\n", 1396 | "\n", 1397 | "def write_info(vector, name):\n", 1398 | " '''\n", 1399 | " This function is to write info. of each vector.\n", 1400 | "\n", 1401 | " Input: the list of vector generated by set_date() and \n", 1402 | " the name of output file.\n", 1403 | "\n", 1404 | " '''\n", 1405 | " with open(name+\".txt\", \"w+\") as file:\n", 1406 | " for p in vector:\n", 1407 | " file.write(str(int(p[0]))+\" \"+\\\n", 1408 | " str(int(p[1]))+\" \"+\\\n", 1409 | " str(int(p[2]))+\" \"+\\\n", 1410 | " str(int(p[3]))+\"\\n\")" 1411 | ] 1412 | }, 1413 | { 1414 | "cell_type": "code", 1415 | "execution_count": null, 1416 | "metadata": { 1417 | "collapsed": true 1418 | }, 1419 | "outputs": [], 1420 | "source": [ 1421 | "# This part is to test the matching scheme with single image\n", 1422 | "# Input: the original image;\n", 1423 | "# the labeled image;\n", 1424 | "# the binary labeled image.\n", 1425 | "\n", 1426 | "vector = None\n", 1427 | "# Feature vector construction\n", 1428 | "global centroid\n", 1429 | "global slope_length\n", 1430 | "global vector\n", 1431 | "vector = []\n", 1432 | "max_id = 0\n", 1433 | "for i in range(len(images)):\n", 1434 | " print \" feature vector: image \", i\n", 1435 | " v = FEAVECTOR()\n", 1436 | " v.set_centroid(centroid[i])\n", 1437 | " v.set_spatial(slope_length[i])\n", 1438 | " v.set_shape(enhance_images[i], marks[i])\n", 1439 | " v.set_histogram()\n", 1440 | " v.add_label()\n", 1441 | " v.add_id(marks[i].max(), i)\n", 1442 | " v.add_efd()\n", 1443 | " v.add_area()\n", 1444 | " v.add_co_occurrence()\n", 1445 | " vector.append(v.generate_vector())\n", 1446 | " print \"num of nuclei: \", len(vector[i])\n", 1447 | " clear_output(wait=True)\n", 1448 | "\n", 1449 | "print \"finish\"" 1450 | ] 1451 | }, 1452 | { 1453 | "cell_type": "markdown", 1454 | "metadata": {}, 1455 | "source": [ 1456 | "This part is to get the sub-image for each cell and save as file. " 1457 | ] 1458 | }, 1459 | { 1460 | "cell_type": "code", 1461 | "execution_count": null, 1462 | "metadata": { 1463 | "collapsed": true 1464 | }, 1465 | "outputs": [], 1466 | "source": [ 1467 | "image_size = 70\n", 1468 | "counter = 0\n", 1469 | "for i, vt in enumerate(vector):\n", 1470 | " print \"Image: \", i\n", 1471 | " for v in vt:\n", 1472 | " h, w = v.s[1].shape[:2]\n", 1473 | " extend_x = (image_size - w) / 2\n", 1474 | " extend_y = (image_size - h) / 2\n", 1475 | " temp = cv2.copyMakeBorder(v.s[1], \\\n", 1476 | " extend_y, (image_size-extend_y-h), \\\n", 1477 | " extend_x, (image_size-extend_x-w), \\\n", 1478 | " cv2.BORDER_CONSTANT, value=0)\n", 1479 | " write_mask8(temp, \"cell_image\"+str(i)+\"_\", counter)\n", 1480 | " counter += 1\n", 1481 | " clear_output(wait=True)\n", 1482 | "print \"finish!\"" 1483 | ] 1484 | }, 1485 | { 1486 | "cell_type": "markdown", 1487 | "metadata": { 1488 | "collapsed": true 1489 | }, 1490 | "source": [ 1491 | "This part is to using ratio of the two axises of inerial as mitosis refinement to mactch cells." 1492 | ] 1493 | }, 1494 | { 1495 | "cell_type": "code", 1496 | "execution_count": null, 1497 | "metadata": { 1498 | "collapsed": true 1499 | }, 1500 | "outputs": [], 1501 | "source": [ 1502 | "class SIMPLE_MATCH():\n", 1503 | " '''\n", 1504 | " This class is simple matching a nucleus into a nucleus in the previous frame by \n", 1505 | " find the nearest neighborhood. \n", 1506 | "\n", 1507 | " '''\n", 1508 | " def __init__(self, index0, index1, images, vectors):\n", 1509 | " self.v0 = cp.copy(vectors[index0])\n", 1510 | " self.v1 = cp.copy(vectors[index1])\n", 1511 | " self.i0 = index0\n", 1512 | " self.i1 = index1\n", 1513 | " self.images = images\n", 1514 | " self.vs = cp.copy(vectors)\n", 1515 | "\n", 1516 | " def distance_measure(self, pv0, pv1, alpha1=0.5, alpha2=0.25, alpha3=0.25, phase = 1):\n", 1517 | " '''\n", 1518 | " This function measures the distence of the two given feature vectors. \n", 1519 | "\n", 1520 | " This distance metrics we use is:\n", 1521 | " d(v(k, i), v(k+1, j)) = alpha1 * d(c(k, i), c(k+1, j)) + \n", 1522 | " alpha2 * q1 * d(s(k, i), s(k+1, j)) +\n", 1523 | " alpha3 * q2 * d(h(k, i), h(k+1, j)) +\n", 1524 | " alpha4 * d(e(k, i), e(k+1, j))\n", 1525 | " Input: The two given feature vectors, \n", 1526 | " and the set of parameters.\n", 1527 | "\n", 1528 | " Output: the distance of the two given vectors. \n", 1529 | "\n", 1530 | " '''\n", 1531 | " def centriod_distance(c0, c1, D=30.):\n", 1532 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1533 | " return dist/D if dist < D else 1\n", 1534 | "\n", 1535 | " def efd_distance(r0, r1, order=8):\n", 1536 | " def find_max(max_value, test):\n", 1537 | " if max_value < test:\n", 1538 | " return test\n", 1539 | " return max_value\n", 1540 | " dis = 0\n", 1541 | " if type(r0) is not int and type(r1) is not int:\n", 1542 | " max_a, max_b, max_c, max_d = 0, 0, 0, 0\n", 1543 | " for o in range(order):\n", 1544 | " dis += ((r0[o][0]-r1[o][0])**2+\\\n", 1545 | " (r0[o][1]-r1[o][1])**2+\\\n", 1546 | " (r0[o][2]-r1[o][2])**2+\\\n", 1547 | " (r0[o][3]-r1[o][3])**2)\n", 1548 | " max_a = find_max(max_a, (r0[o][0]-r1[o][0])**2)\n", 1549 | " max_b = find_max(max_b, (r0[o][1]-r1[o][1])**2)\n", 1550 | " max_c = find_max(max_c, (r0[o][2]-r1[o][2])**2)\n", 1551 | " max_d = find_max(max_d, (r0[o][3]-r1[o][3])**2)\n", 1552 | " dis /= (order*(max_a+max_b+max_c+max_d))\n", 1553 | " if dis > 1.1:\n", 1554 | " print dis, max_a, max_b, max_c, max_d\n", 1555 | " raise\n", 1556 | " else:\n", 1557 | " dis = 1\n", 1558 | " return dis\n", 1559 | "\n", 1560 | " def cm_distance(cm0, cm1): \n", 1561 | " return ((cm0[0]-cm1[0])**2+\\\n", 1562 | " (cm0[1]-cm1[1])**2+\\\n", 1563 | " (cm0[2]-cm1[2])**2+\\\n", 1564 | " (cm0[3]-cm1[3])**2)/\\\n", 1565 | " (max(cm0[0],cm1[0])**2+\\\n", 1566 | " max(cm0[1],cm1[1])**2+\\\n", 1567 | " max(cm0[2],cm1[2])**2+\\\n", 1568 | " max(cm0[3],cm1[3])**2)\n", 1569 | "\n", 1570 | " if len(pv0.s[0]) and len(pv1.s[0]):\n", 1571 | " dist = \talpha1 * centriod_distance(pv0.c, pv1.c)+ \\\n", 1572 | " alpha2 * efd_distance(pv0.r, pv1.r, order=8) * phase + \\\n", 1573 | " alpha3 * cm_distance(pv0.cm[0], pv1.cm[0]) * phase\n", 1574 | " else:\n", 1575 | " dist = Max_dis\n", 1576 | "\n", 1577 | " return dist\n", 1578 | "\n", 1579 | " def phase_identify(self, pv1, min_times_MA2ma = 2, RNN=False):\n", 1580 | " '''\n", 1581 | " Phase identification returns 0 when mitosis appears, vice versa.\n", 1582 | "\n", 1583 | " '''\n", 1584 | " if not RNN:\n", 1585 | " _, contours, hierarchy = cv2.findContours(pv1.s[0].astype(np.uint8), 1, 2)\n", 1586 | " if not len(contours):\n", 1587 | " return 1\n", 1588 | " cnt = contours[0]\n", 1589 | " if len(cnt) >= 5:\n", 1590 | " (x,y),(ma,MA),angle = cv2.fitEllipse(cnt)\n", 1591 | " if ma and MA/ma > min_times_MA2ma:\n", 1592 | " return 0\n", 1593 | " elif not ma and MA:\n", 1594 | " return 0\n", 1595 | " else:\n", 1596 | " return 1\n", 1597 | " else:\n", 1598 | " return 1\n", 1599 | "\n", 1600 | " else:\n", 1601 | " try: \n", 1602 | " if model.predict([pv1.r.reshape(40)])[-1]:\n", 1603 | " return 0\n", 1604 | " else: \n", 1605 | " return 1\n", 1606 | " except AttributeError:\n", 1607 | " return 1\n", 1608 | "\n", 1609 | " def find_match(self, max_distance=1,a_1=0.5,a_2=0.25,a_3=0.25, rnn=False):\n", 1610 | " '''\n", 1611 | " This function is to find the nearest neighborhood between two\n", 1612 | " successive frame.\n", 1613 | "\n", 1614 | " '''\n", 1615 | " def centriod_distance(c0, c1, D=30.):\n", 1616 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1617 | " return dist/D if dist < D else 1\n", 1618 | "\n", 1619 | " for i, pv1 in enumerate(self.v1):\n", 1620 | " dist = np.ones((len(self.v0), 3), np.float32)*max_distance\n", 1621 | " count = 0\n", 1622 | " q = self.phase_identify(pv1, 3, RNN=rnn)\n", 1623 | " for j, pv0 in enumerate(self.v0):\n", 1624 | " if centriod_distance(pv0.c, pv1.c) < 1 and pv0.a:\n", 1625 | " dist[count][0] = self.distance_measure(pv0, pv1, alpha1=a_1, alpha2=a_2, alpha3=a_3, phase=q)\n", 1626 | " dist[count][1] = pv0.l\n", 1627 | " dist[count][2] = pv0.id\n", 1628 | " count += 1\n", 1629 | "\n", 1630 | " sort_dist = sorted(dist, key=lambda a_entry: a_entry[0])\n", 1631 | " print \"dis: \", sort_dist[0][0]\n", 1632 | " if sort_dist[0][0] < max_distance:\n", 1633 | " self.v1[i].l = sort_dist[0][1]\n", 1634 | " self.v1[i].id = sort_dist[0][2]\n", 1635 | "\n", 1636 | " def mitosis_refine(self, rnn=False):\n", 1637 | " '''\n", 1638 | " This function is to find died cell due to the by mitosis. \n", 1639 | "\n", 1640 | " '''\n", 1641 | " def find_sibling(pv0):\n", 1642 | " '''\n", 1643 | " This function is to find sibling cells according to the centroid of\n", 1644 | " pv0. The criteria of sibling is:\n", 1645 | " 1. the jaccard cooeficient of the two cells is above 0.5\n", 1646 | " 2. the sum of the two areas should in the range [A, 2.5A], where\n", 1647 | " A is the area of the pv0\n", 1648 | " 3. the position of the two cells should be not larger than 20 pixels.\n", 1649 | "\n", 1650 | " Input: pv0: the parent cell that you want to find siblings;\n", 1651 | "\n", 1652 | " Output: the index of the siblings.\n", 1653 | "\n", 1654 | " '''\n", 1655 | " def maxsize_image(image1, image2):\n", 1656 | " y1, x1 = np.where(image1)\n", 1657 | " y2, x2 = np.where(image2)\n", 1658 | " return min(min(x1), min(x2)), min(min(y1), min(y2)), \\\n", 1659 | " max(max(x1), max(x2)), max(max(y1), max(y2)),\n", 1660 | "\n", 1661 | " def symmetry(image, shape):\n", 1662 | " h, w = image.shape[:2] \n", 1663 | " newimg = np.zeros(shape)\n", 1664 | " newimg[:h, :w] = image\n", 1665 | " v = float(shape[0] - h)/2.\n", 1666 | " u = float(shape[1] - w)/2.\n", 1667 | " M = np.float32([[1,0,u],[0,1,v]])\n", 1668 | " return cv2.warpAffine(newimg,M,(shape[1],shape[0]))\n", 1669 | "\n", 1670 | " def jaccard(s0, s1):\n", 1671 | " minx, miny, maxx, maxy = maxsize_image(s0, s1)\n", 1672 | " height = maxy - miny + 1\n", 1673 | " width = maxx - minx + 1\n", 1674 | "\n", 1675 | " img0 = symmetry(s0, (height, width))\n", 1676 | " img1 = symmetry(s1, (height, width))\n", 1677 | "\n", 1678 | " num = 0.\n", 1679 | " deno = 0.\n", 1680 | " for y in range(height):\n", 1681 | " for x in range(width):\n", 1682 | " if img0[y, x] and img1[y, x]:\n", 1683 | " num += 1\n", 1684 | " if img0[y, x] or img1[y, x]:\n", 1685 | " deno += 1\n", 1686 | "\n", 1687 | " return num/deno\n", 1688 | "\n", 1689 | " sibling_cand = []\n", 1690 | " for i, pv1 in enumerate(self.v1): \n", 1691 | " if np.linalg.norm(pv1.c-pv0.c) < 50:\n", 1692 | " sibling_cand.append([pv1, i])\n", 1693 | "\n", 1694 | " sibling_pair = []\n", 1695 | " area = pv0.s[0].sum()\n", 1696 | " jaccard_value = []\n", 1697 | " for sibling0 in sibling_cand:\n", 1698 | " for sibling1 in sibling_cand:\n", 1699 | " if (sibling1[0].c != sibling0[0].c).all():\n", 1700 | " sum_area = sibling1[0].s[0].sum()+sibling0[0].s[0].sum()\n", 1701 | " similarity = jaccard(sibling0[0].s[0], sibling1[0].s[0])\n", 1702 | " if similarity > 0.4 and (sum_area > 2*area):\n", 1703 | " sibling_pair.append([sibling0, sibling1])\n", 1704 | " jaccard_value.append(similarity)\n", 1705 | " if len(jaccard_value):\n", 1706 | " return sibling_pair[np.argmax(jaccard_value)]\n", 1707 | " else:\n", 1708 | " return 0\n", 1709 | "\n", 1710 | " v1_ids = []\n", 1711 | " for pv1 in self.v1:\n", 1712 | " v1_ids.append(pv1.id)\n", 1713 | "\n", 1714 | " for i, pv0 in enumerate(self.v0):\n", 1715 | " if pv0.id not in v1_ids and len(pv0.s[0]) and self.phase_identify(pv0, 3, RNN=rnn):\n", 1716 | " sibling = find_sibling(pv0)\n", 1717 | " if sibling:\n", 1718 | " [s0, s1] = sibling\n", 1719 | " if s0[0].l==0 and s1[0].l==0 and \\\n", 1720 | " s0[0].id==-1 and s1[0].id==-1:\n", 1721 | " self.v1[s0[1]].l = pv0.id\n", 1722 | " self.v1[s1[1]].l = pv0.id\n", 1723 | "\n", 1724 | " return self.v1\n", 1725 | "\n", 1726 | " def match_missing(self, mask, max_frame = 1, max_distance = 10, min_shape_similarity = 0.6):\n", 1727 | " '''\n", 1728 | " This function is to match the cells that didn't show in the last frame caused by \n", 1729 | " program fault. In order to match them, we need to seach the cell in the previous \n", 1730 | " frame with in the certain range and with similar shape. \n", 1731 | "\n", 1732 | " '''\n", 1733 | " def centriod_distance(c0, c1):\n", 1734 | " dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2)\n", 1735 | " return dist\n", 1736 | "\n", 1737 | " def maxsize_image(image1, image2):\n", 1738 | " y1, x1 = np.where(image1)\n", 1739 | " y2, x2 = np.where(image2)\n", 1740 | " return min(min(x1), min(x2)), min(min(y1), min(y2)), \\\n", 1741 | " max(max(x1), max(x2)), max(max(y1), max(y2)),\n", 1742 | "\n", 1743 | " def symmetry(image, shape):\n", 1744 | " h, w = image.shape[:2] \n", 1745 | " newimg = np.zeros(shape)\n", 1746 | " newimg[:h, :w] = image\n", 1747 | " v = float(shape[0] - h)/2.\n", 1748 | " u = float(shape[1] - w)/2.\n", 1749 | " M = np.float32([[1,0,u],[0,1,v]])\n", 1750 | " return cv2.warpAffine(newimg,M,(shape[1],shape[0]))\n", 1751 | "\n", 1752 | " def shape_similarity(s0, s1):\n", 1753 | " if len(s0) and len(s1):\n", 1754 | " minx, miny, maxx, maxy = maxsize_image(s0, s1)\n", 1755 | " height = maxy - miny + 1\n", 1756 | " width = maxx - minx + 1\n", 1757 | "\n", 1758 | " img0 = symmetry(s0, (height, width))\n", 1759 | " img1 = symmetry(s1, (height, width))\n", 1760 | "\n", 1761 | " num = 0.\n", 1762 | " deno = 0.\n", 1763 | " for y in range(height):\n", 1764 | " for x in range(width):\n", 1765 | " if img0[y, x] and img1[y, x]:\n", 1766 | " num += 1\n", 1767 | " if img0[y, x] or img1[y, x]:\n", 1768 | " deno += 1\n", 1769 | " return num/deno\n", 1770 | "\n", 1771 | " else:\n", 1772 | " return 0.\n", 1773 | "\n", 1774 | " def add_marker(index_find, index_new, pv0_id):\n", 1775 | " temp = mask[index_new]\n", 1776 | " find = mask[index_find]\n", 1777 | " temp[find==pv0_id] = pv0_id\n", 1778 | " return temp\n", 1779 | "\n", 1780 | " for i, pv1 in enumerate(self.v1):\n", 1781 | " if pv1.id == -1:\n", 1782 | " for index in range(1, max_frame+1):\n", 1783 | " if self.i0-index >= 0:\n", 1784 | " vt = self.vs[self.i0-index]\n", 1785 | " for pv0 in vt:\n", 1786 | " if centriod_distance(pv0.c, pv1.c) < max_distance and \\\n", 1787 | " shape_similarity(pv0.s[0], pv1.s[0]) > min_shape_similarity:\n", 1788 | " self.v1[i].id = pv0.id\n", 1789 | " self.v1[i].l = pv0.l\n", 1790 | " print \"missing in frame: \", self.i1, \"find in frame: \", \\\n", 1791 | " self.i0-index, \"ID: \", pv0.id, \" at: \", pv0.c\n", 1792 | " for i in range(self.i0-index+1, self.i1):\n", 1793 | " mask[i] = add_marker(self.i0-index, i, pv0.id)\n", 1794 | " return mask\n", 1795 | "\n", 1796 | " def new_id(self, vectors):\n", 1797 | " '''\n", 1798 | " This function is to add new labels for the necles that are marked as -1.\n", 1799 | "\n", 1800 | "\n", 1801 | " '''\n", 1802 | " def find_max_id(vectors):\n", 1803 | " max_id = 0\n", 1804 | " for vt in vectors:\n", 1805 | " for pt in vt:\n", 1806 | " if pt.id > max_id:\n", 1807 | " max_id = pt.id \n", 1808 | " return max_id\n", 1809 | " max_id = find_max_id(self.vs)\n", 1810 | " max_id += 1\n", 1811 | " for i, pv1 in enumerate(self.v1):\n", 1812 | " if pv1.id == -1:\n", 1813 | " self.v1[i].id = max_id\n", 1814 | " max_id += 1\n", 1815 | "\n", 1816 | " def generate_mask(self, marker, index, isfinal=False):\n", 1817 | " '''\n", 1818 | " This function is to generate a 16-bit image as mask image. \n", 1819 | "\n", 1820 | " '''\n", 1821 | " h, w = marker.shape[:2]\n", 1822 | " mask = np.zeros((h, w), np.uint16)\n", 1823 | " pts = list(set(marker[marker>0]))\n", 1824 | " if not isfinal:\n", 1825 | " assert len(pts)==len(self.v0), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v0))\n", 1826 | " for pt, pv in zip(pts, self.v0):\n", 1827 | " mask[marker==pt] = pv.id\n", 1828 | "\n", 1829 | " else:\n", 1830 | " assert len(pts)==len(self.v1), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v1))\n", 1831 | " for pt, pv in zip(pts, self.v1):\n", 1832 | " mask[marker==pt] = pv.id\n", 1833 | "\n", 1834 | " os.chdir(\".\")\n", 1835 | " write_mask16(mask, \"mask\", index)\n", 1836 | " os.chdir(os.pardir)\n", 1837 | " return mask\t\n", 1838 | "\n", 1839 | " def return_vectors(self):\n", 1840 | " '''\n", 1841 | " This function is to return the vectors that we have already \n", 1842 | " changed.\n", 1843 | "\n", 1844 | " Output: the vectors from the k+1 frame.\n", 1845 | "\n", 1846 | " '''\t\n", 1847 | " return self.v1" 1848 | ] 1849 | }, 1850 | { 1851 | "cell_type": "code", 1852 | "execution_count": null, 1853 | "metadata": { 1854 | "collapsed": true 1855 | }, 1856 | "outputs": [], 1857 | "source": [ 1858 | "import copy as cp\n", 1859 | "\n", 1860 | "mask = []\n", 1861 | "temp_vector = cp.deepcopy(vector)\n", 1862 | "\n", 1863 | "# Feature matching\n", 1864 | "for i in range(len(images)-1):\n", 1865 | " print \" Feature matching: image \", i\n", 1866 | " m = SIMPLE_MATCH(i,i+1,[images[i], images[i+1]], temp_vector)\n", 1867 | " mask.append(m.generate_mask(marks[i], i))\n", 1868 | " m.find_match(0.7,0.7,0.15,0.15)\n", 1869 | " temp_vector[i+1] = m.mitosis_refine()\n", 1870 | " m.new_id(temp_vector)\n", 1871 | " temp_vector[i+1] = m.return_vectors()\n", 1872 | " clear_output(wait=True)\n", 1873 | "\n", 1874 | "print \" Feature matching: image \", i+1\n", 1875 | "mask.append(m.generate_mask(marks[i+1], i+1, True))\n", 1876 | "os.chdir(\".\")\n", 1877 | "cells = set_date(temp_vector)\n", 1878 | "write_info(cells, \"res_track\")\n", 1879 | "print \"finish!\"" 1880 | ] 1881 | }, 1882 | { 1883 | "cell_type": "markdown", 1884 | "metadata": {}, 1885 | "source": [ 1886 | "This part generates the final marked result in \"gif\"." 1887 | ] 1888 | }, 1889 | { 1890 | "cell_type": "code", 1891 | "execution_count": null, 1892 | "metadata": { 1893 | "collapsed": true 1894 | }, 1895 | "outputs": [], 1896 | "source": [ 1897 | "# write gif image showing the final result\n", 1898 | "def find_max_id(temp_vector):\n", 1899 | " max_id = 0\n", 1900 | " for pv in temp_vector:\n", 1901 | " for p in pv:\n", 1902 | " if p.id > max_id:\n", 1903 | " max_id = p.id\n", 1904 | " return max_id\n", 1905 | "\n", 1906 | "# This part is to mark the result in the normolized image and \n", 1907 | "# write the gif image.\n", 1908 | "max_id = find_max_id(temp_vector)\n", 1909 | "colors = [np.random.randint(0, 255, size=max_id),\\\n", 1910 | " np.random.randint(0, 255, size=max_id),\\\n", 1911 | " np.random.randint(0, 255, size=max_id)]\n", 1912 | "font = cv2.FONT_HERSHEY_SIMPLEX \n", 1913 | "selecy_id = 9\n", 1914 | "enhance_imgs = []\n", 1915 | "for i, m in enumerate(mask):\n", 1916 | " print \" write the gif image: image \", i\n", 1917 | " enhance_imgs.append(cv2.cvtColor(enhance_images[i],cv2.COLOR_GRAY2RGB))\n", 1918 | " for pv in temp_vector[i]:\n", 1919 | " center = pv.c\n", 1920 | " if not pv.l:\n", 1921 | " color = (colors[0][int(pv.id)-1],\\\n", 1922 | " colors[1][int(pv.id)-1],\\\n", 1923 | " colors[2][int(pv.id)-1],)\n", 1924 | " else:\n", 1925 | " color = (colors[0][int(pv.l)-1],\\\n", 1926 | " colors[1][int(pv.l)-1],\\\n", 1927 | " colors[2][int(pv.l)-1],)\n", 1928 | "\n", 1929 | " if m[center[0], center[1]]:\n", 1930 | " enhance_imgs[i][m==pv.id] = color\n", 1931 | " cv2.putText(enhance_imgs[i],\\\n", 1932 | " str(int(pv.id)),(int(pv.c[1]), \\\n", 1933 | " int(pv.c[0])), \n", 1934 | " font, 0.5,\\\n", 1935 | " (255,255,255),1)\n", 1936 | " clear_output(wait=True)\n", 1937 | "os.chdir(\"PATH_TO_RESULT\")\n", 1938 | "imageio.mimsave('mitosis_final.gif', enhance_imgs, duration=0.6)\n", 1939 | "print \"finish!\"" 1940 | ] 1941 | } 1942 | ], 1943 | "metadata": { 1944 | "kernelspec": { 1945 | "display_name": "Python 2", 1946 | "language": "python", 1947 | "name": "python2" 1948 | }, 1949 | "language_info": { 1950 | "codemirror_mode": { 1951 | "name": "ipython", 1952 | "version": 2 1953 | }, 1954 | "file_extension": ".py", 1955 | "mimetype": "text/x-python", 1956 | "name": "python", 1957 | "nbconvert_exporter": "python", 1958 | "pygments_lexer": "ipython2", 1959 | "version": "2.7.13" 1960 | } 1961 | }, 1962 | "nbformat": 4, 1963 | "nbformat_minor": 1 1964 | } 1965 | -------------------------------------------------------------------------------- /Code/adaptivethresh.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is to compute adaptive thresholding of image sequence in 3 | order to generate binary image for Nuclei segmentation. 4 | 5 | Problem: 6 | Due to the low contrast of original image, the adaptive thresholding is not working. 7 | Therefore, we change to regular threshold with threshold value as 129. 8 | ''' 9 | import cv2 10 | import sys 11 | import numpy as np 12 | 13 | class ADPTIVETHRESH(): 14 | ''' 15 | This class is to provide all function for adaptive thresholding. 16 | 17 | ''' 18 | def __init__(self, images): 19 | self.images = [] 20 | for img in images: 21 | if len(img.shape) == 3: 22 | img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 23 | self.images.append(img.copy()) 24 | 25 | def applythresh(self, threshold = 129): 26 | ''' 27 | applythresh function is to convert original image to binary image by thresholding. 28 | 29 | Input: image sequence. E.g. [image0, image1, ...] 30 | 31 | Output: image sequence after thresholding. E.g. [image0, image1, ...] 32 | ''' 33 | out = [] 34 | markers = [] 35 | binarymark = [] 36 | 37 | for img in self.images: 38 | img = cv2.GaussianBlur(img,(5,5),0).astype(np.uint8) 39 | _, thresh = cv2.threshold(img,threshold,1,cv2.THRESH_BINARY) 40 | 41 | # Using morphlogical operations to imporve the quality of result 42 | kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(9,9)) 43 | thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel) 44 | 45 | out.append(thresh) 46 | 47 | return out 48 | 49 | def main(): 50 | ''' 51 | This part is for testing adaptivethresh.py with single image. 52 | 53 | Input: an original image 54 | 55 | Output: Thresholding image 56 | 57 | ''' 58 | img = [cv2.imread(sys.argv[1])] 59 | adaptive = ADPTIVETHRESH(img) 60 | thresh, markers = adaptive.applythresh(10) 61 | cv2.imwrite("adaptive.tiff", thresh[0]*255) 62 | 63 | # if python says run, then we should run 64 | if __name__ == '__main__': 65 | main() -------------------------------------------------------------------------------- /Code/graph_construction.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is to generate a neighboring graph contraction using 3 | Delaunary Triangulation. 4 | 5 | ''' 6 | 7 | import cv2 8 | import numpy as np 9 | import random 10 | import sys 11 | import imageio 12 | 13 | class GRAPH(): 14 | ''' 15 | This class contains all the functions needed to compute 16 | Delaunary Triangulation. 17 | 18 | ''' 19 | def __init__(self, mark, binary, index): 20 | ''' 21 | Input: the grayscale mark image with different label on each segments 22 | the binary image of the mark image 23 | the index of the image 24 | 25 | ''' 26 | self.mark = mark[index] 27 | self.binary = binary[index] 28 | 29 | def rect_contains(self, rect, point): 30 | ''' 31 | Check if a point is inside the image 32 | 33 | Input: the size of the image 34 | the point that want to test 35 | 36 | Output: if the point is inside the image 37 | 38 | ''' 39 | if point[0] < rect[0] : 40 | return False 41 | elif point[1] < rect[1] : 42 | return False 43 | elif point[0] > rect[2] : 44 | return False 45 | elif point[1] > rect[3] : 46 | return False 47 | return True 48 | 49 | def draw_point(self, img, p, color ): 50 | ''' 51 | Draw a point 52 | 53 | ''' 54 | cv2.circle( img, (p[1], p[0]), 2, color, cv2.FILLED, 16, 0 ) 55 | 56 | def draw_delaunay(self, img, subdiv, delaunay_color ): 57 | ''' 58 | Draw delaunay triangles and store these lines 59 | 60 | Input: the image want to draw 61 | the set of points: format as cv2.Subdiv2D 62 | the color want to use 63 | 64 | Output: the slope and length of each line () 65 | 66 | ''' 67 | triangleList = subdiv.getTriangleList(); 68 | size = img.shape 69 | r = (0, 0, size[0], size[1]) 70 | 71 | slope_length = [[]] 72 | for i in range(self.mark.max()-1): 73 | slope_length.append([]) 74 | 75 | for t in triangleList: 76 | 77 | pt1 = (t[0], t[1]) 78 | pt2 = (t[2], t[3]) 79 | pt3 = (t[4], t[5]) 80 | 81 | if self.rect_contains(r, pt1) and self.rect_contains(r, pt2) and self.rect_contains(r, pt3): 82 | 83 | # draw lines 84 | cv2.line(img, (pt1[1], pt1[0]), (pt2[1], pt2[0]), delaunay_color, 1, 16, 0) 85 | cv2.line(img, (pt2[1], pt2[0]), (pt3[1], pt3[0]), delaunay_color, 1, 16, 0) 86 | cv2.line(img, (pt3[1], pt3[0]), (pt1[1], pt1[0]), delaunay_color, 1, 16, 0) 87 | 88 | # store the length of line segments and their slopes 89 | for p0 in [pt1, pt2, pt3]: 90 | for p1 in [pt1, pt2, pt3]: 91 | if p0 != p1: 92 | temp = self.length_slope(p0, p1) 93 | if temp not in slope_length[self.mark[p0]-1]: 94 | slope_length[self.mark[p0]-1].append(temp) 95 | 96 | return slope_length 97 | 98 | def length_slope(self, p0, p1): 99 | ''' 100 | This function is to compute the length and theta for the given two points. 101 | 102 | Input: two points with the format (y, x) 103 | 104 | ''' 105 | if p1[1]-p0[1]: 106 | slope = (p1[0]-p0[0]) / (p1[1]-p0[1]) 107 | else: 108 | slope = 1e10 109 | 110 | length = np.sqrt((p1[0]-p0[0])**2 + (p1[1]-p0[1])**2) 111 | 112 | return length, slope 113 | 114 | def generate_points(self): 115 | ''' 116 | Find the centroid of each segmentation 117 | 118 | ''' 119 | centroids = [] 120 | label = [] 121 | max_label = self.mark.max() 122 | 123 | for i in range(1, max_label+1): 124 | img = self.mark.copy() 125 | img[img!=i] = 0 126 | if img.sum(): 127 | _, contours,hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_TC89_KCOS) 128 | m = cv2.moments(contours[0]) 129 | 130 | if m['m00']: 131 | label.append(i) 132 | centroids.append(( int(round(m['m01']/m['m00'])),\ 133 | int(round(m['m10']/m['m00'])) )) 134 | else: 135 | label.append(i) 136 | centroids.append(( 0,0 )) 137 | 138 | return centroids, label 139 | 140 | def run(self, animate = False): 141 | ''' 142 | The pipline of graph construction. 143 | 144 | Input: if showing a animation (False for default) 145 | 146 | Output: centroids: # of segments * 2 (y, x) 147 | slopes and length: # of segments * # of slope_length 148 | 149 | ''' 150 | # Read in the image. 151 | img_orig = self.binary.copy() 152 | 153 | # Rectangle to be used with Subdiv2D 154 | size = img_orig.shape 155 | rect = (0, 0, size[0], size[1]) 156 | 157 | # Create an instance of Subdiv2D 158 | subdiv = cv2.Subdiv2D(rect); 159 | 160 | # find the centroid of each segments 161 | points, label = self.generate_points() 162 | 163 | 164 | # add and sort the centroid to a numpy array for post processing 165 | centroid = np.zeros((self.mark.max(), 2)) 166 | for p, l in zip(points, label): 167 | centroid[l-1] = p 168 | 169 | outimg = [] 170 | # Insert points into subdiv 171 | for p in points: 172 | subdiv.insert(p) 173 | 174 | # Show animation 175 | if animate: 176 | img_copy = img_orig.copy() 177 | # Draw delaunay triangles 178 | self.draw_delaunay( img_copy, subdiv, (255, 255, 255) ); 179 | outimg.append(img_copy) 180 | cv2.imshow("win_delaunay", img_copy) 181 | cv2.waitKey(50) 182 | 183 | imageio.mimsave('graph_contruction.gif', outimg, duration=0.3) 184 | # Draw delaunay triangles 185 | slope_length = self.draw_delaunay( img_orig, subdiv, (255, 255, 255) ); 186 | 187 | # Draw points 188 | for p in points : 189 | self.draw_point(img_orig, p, (0,0,255)) 190 | 191 | # show images 192 | if animate: 193 | cv2.imshow('img_orig',img_orig) 194 | k = cv2.waitKey(0) 195 | if k == 27: # wait for ESC key to exit 196 | cv2.destroyAllWindows() 197 | elif k == ord('s'): # wait for 's' key to save and exit 198 | cv2.imwrite('messigray.png',img) 199 | cv2.destroyAllWindows() 200 | 201 | return centroid, slope_length 202 | 203 | ''' 204 | This part is the small test for graph_contruction.py. 205 | 206 | Input: grayscale marker image 207 | binary marker image 208 | 209 | Output: a text file includes the centroid and the length and slope for each neighbor. 210 | 211 | ''' 212 | def main(): 213 | mark = [cv2.imread(sys.argv[1])] 214 | binary = [cv2.imread(sys.argv[2])] 215 | mark[0] = cv2.cvtColor(mark[0], cv2.COLOR_BGR2GRAY) 216 | binary[0] = cv2.cvtColor(binary[0], cv2.COLOR_BGR2GRAY) 217 | graph = GRAPH(mark, binary, 0) 218 | centroid, slope_length = graph.run(True) 219 | with open("centroid_slope_length.txt", "w+") as file: 220 | for i, p in enumerate(centroid): 221 | file.write(str(p[0])+" "+str(p[1])+" "+str(slope_length[i])+"\n") 222 | 223 | # if python says run, then we should run 224 | if __name__ == '__main__': 225 | main() -------------------------------------------------------------------------------- /Code/gvf.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is to compute gradient vector field (GVF) and then find the Nuclei center 3 | with the GVF result. 4 | 5 | ''' 6 | import cv2 7 | import numpy as np 8 | import sys 9 | from adaptivethresh import ADPTIVETHRESH as athresh 10 | from scipy import spatial as sp 11 | from scipy import ndimage 12 | from scipy.spatial import distance 13 | 14 | looplimit = 500 15 | 16 | def inbounds(shape, indices): 17 | assert len(shape) == len(indices) 18 | for i, ind in enumerate(indices): 19 | if ind < 0 or ind >= shape[i]: 20 | return False 21 | return True 22 | 23 | class GVF(): 24 | ''' 25 | This class contains all function for calculating GVF and its following steps. 26 | 27 | ''' 28 | def __init__(self, images, thresh): 29 | 30 | self.images = images 31 | self.thresh = thresh 32 | 33 | def distancemap(self): 34 | ''' 35 | This function is to generate distance map of the thresh image. We use the opencv 36 | function distanceTransform to generate it. Moreover, in this case, we use Euclidiean 37 | Distance (DIST_L2) as a metric of distance. 38 | 39 | Input: None 40 | 41 | Output: Image distance map 42 | 43 | ''' 44 | return [cv2.distanceTransform(self.thresh[i], distanceType=cv2.DIST_L2, maskSize=cv2.DIST_MASK_PRECISE)\ 45 | for i in range(len(self.thresh))] 46 | 47 | def new_image(self, alpha, dismap): 48 | ''' 49 | This function is to generate a new image combining the oringal image I0 with 50 | the distance map image Idis by following expression: 51 | Inew = I0 + alpha*Idis 52 | In this program, we choose alpha as 0.4. 53 | 54 | Input: the weight of distance map: alpha 55 | the distance map image 56 | 57 | Output: new grayscale image 58 | 59 | ''' 60 | 61 | return [self.images[i] + alpha * dismap[i] for i in range(len(self.thresh))] 62 | 63 | def compute_gvf(self, newimage): 64 | ''' 65 | This function is to compute the gradient vector of the imput image. 66 | 67 | Input: a grayscale image with size, say m * n * # of images 68 | 69 | Output: a 3 dimentional image with size, m * n * 2, where the last dimention is 70 | the gradient vector (gx, gy) 71 | 72 | ''' 73 | kernel_size = 5 # kernel size for blur image before compute gradient 74 | newimage = [cv2.GaussianBlur((np.clip(newimage[i], 0, 255)).astype(np.uint8),(kernel_size,kernel_size),0)\ 75 | for i in range(len(self.thresh))] 76 | # use sobel operator to compute gradient 77 | temp = np.zeros((newimage[0].shape[0], newimage[0].shape[1], 2), np.float32) # store temp gradient image 78 | gradimg = [] # output gradient images (height * weight * # of images) 79 | 80 | for i in range(len(newimage)): 81 | # compute sobel operation in x, y directions 82 | gradx = cv2.Sobel(newimage[i],cv2.CV_64F,1,0,ksize=3) 83 | grady = cv2.Sobel(newimage[i],cv2.CV_64F,0,1,ksize=3) 84 | # add the gradient vector 85 | temp[:,:,0], temp[:,:,1] = gradx, grady 86 | gradimg.append(temp) 87 | 88 | return gradimg 89 | 90 | 91 | def find_certer(self, gvfimage, index): 92 | ''' 93 | This function is to find the center of Nuclei. 94 | 95 | Input: the gradient vector image (height * weight * 2). 96 | 97 | Output: the record image height * weight). 98 | 99 | ''' 100 | # Initialize a image to record seed candidates. 101 | imgpair = np.zeros(gvfimage.shape[:2]) 102 | 103 | kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5)) 104 | erthresh = cv2.erode(self.thresh[index].copy(), kernel, iterations = 1) 105 | while erthresh.sum() > 0: 106 | 107 | print "how many left? ", erthresh.sum() 108 | # Initialize partical coordinates [y, x] 109 | y0, x0 = np.where(erthresh>0) 110 | p0 = np.array([y0[0], x0[0], 1]) 111 | 112 | # Initialize record coordicates [y, x] 113 | p1 = np.array([5000, 5000, 1]) 114 | 115 | # mark the first non-zero point of thresh image to 0 116 | erthresh[p0[0], p0[1]] = 0 117 | 118 | # a variable to record if the point out of bound of image or 119 | # out of maximum loop times 120 | outbound = False 121 | 122 | # count loop times to limit max loop times 123 | count = 0 124 | 125 | while sp.distance.cdist([p0],[p1]) > 1: 126 | 127 | count += 1 128 | p1 = p0 129 | u = gvfimage[p0[0], p0[1], 1] 130 | v = gvfimage[p0[0], p0[1], 0] 131 | M = np.array([[1, 0, u],\ 132 | [0, 1, v],\ 133 | [0, 0, 1]], np.float32) 134 | p0 = M.dot(p0) 135 | if not inbounds(self.thresh[index].shape, (p0[0], p0[1])) or count > looplimit: 136 | outbound = True 137 | if count > looplimit: 138 | print " count > looplimit..." 139 | break 140 | 141 | if not outbound: 142 | imgpair[p0[0], p0[1]] += 1 143 | 144 | imgpair_raw = imgpair.copy() 145 | 146 | # find local maximum in a certain range in order to remove multiple 147 | # point in one necleus. 148 | neighborhood_size = 20 149 | data_max = ndimage.filters.maximum_filter(imgpair, neighborhood_size) 150 | data_max[data_max==0] = 255 151 | imgpair = (imgpair == data_max) 152 | 153 | # in order to remove the points that has same value in one necleus that 154 | # cannot be remove by previous step, we measure the distance between 155 | # each point, and get rid of the points that to close to the other. 156 | binary_imgpair_raw = imgpair.copy() 157 | binary_imgpair_raw = binary_imgpair_raw.astype(np.uint8) 158 | binary_imgpair_raw[binary_imgpair_raw>0] = 255 159 | y, x = np.where(imgpair>0) 160 | points = zip(y[:], x[:]) 161 | dmap = distance.cdist(points, points, 'euclidean') 162 | y, x = np.where(dmap<20) 163 | ps = zip(y[:], x[:]) 164 | for p in ps: 165 | if p[0] != p[1]: 166 | imgpair[points[min(p[0], p[1])]] = 0 167 | 168 | return imgpair.astype(np.uint8)*255, binary_imgpair_raw, imgpair_raw 169 | 170 | def main(): 171 | ''' 172 | This part is for testing gvf.py with single image. 173 | 174 | Input: an original image 175 | 176 | Output: Thresholding image and seed image 177 | 178 | ''' 179 | images = [] 180 | temp = cv2.imread(sys.argv[1]) 181 | images.append(cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY)) 182 | # Binarization 183 | th = athresh(images) 184 | threh = th.applythresh() 185 | # Nuclei center detection 186 | gvf = GVF(images, threh) 187 | dismap = gvf.distancemap() 188 | newimg = gvf.new_image(0.4, dismap) # choose alpha as 0.4. 189 | gradimg = gvf.compute_gvf(newimg) 190 | 191 | imgpair = gvf.find_certer(gradimg[0], 0) 192 | cv2.imwrite('imgpair_test.tif', (np.clip(imgpair, 0, 255)).astype(np.uint8)) 193 | 194 | # if python says run, then we should run 195 | if __name__ == '__main__': 196 | main() 197 | -------------------------------------------------------------------------------- /Code/main.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is the main file that follows the paper "Multiple Nuclei 3 | Tracking Using Integer Programming for Quantitative Cancer Cell C- 4 | ycle Analysis". The proposed algorithm mainly contains two parts: 5 | Nuclei Segmentation and Nuclei Tracking. 6 | 7 | Nuclei Segmentation: 8 | It includes three steps: 9 | 1. Binarization - using adaptive thresholding - adaptivethresh.py 10 | 2. Nuclei center detection - using gradient vector feild (GVF) - gvf.py 11 | 3. Nuclei boundary delinating - using watershed algorithm - watershed.py 12 | 13 | Nuclei Tracking: 14 | It includes three steps: 15 | 1. Neighboring Graph Constrction 16 | 2. Optimal MatchingRAY 17 | 3. Cell Division, Death, Segmentation Errors Detection& Processing 18 | 19 | ''' 20 | import cv2 21 | import sys 22 | import os 23 | import numpy as np 24 | import imageio 25 | from adaptivethresh import ADPTIVETHRESH as athresh 26 | from gvf import GVF 27 | from matplotlib import pyplot as plt 28 | from watershed import WATERSHED as WS 29 | from graph_construction import GRAPH 30 | from matching import FEAVECTOR as FEA 31 | from matching import SIMPLE_MATCH as MAT 32 | 33 | def normalize(image): 34 | ''' 35 | This function is to normalize the input grayscale image by 36 | substracting globle mean and dividing standard diviation for 37 | visualization. 38 | 39 | Input: a grayscale image 40 | 41 | Output: normolized grascale image 42 | 43 | ''' 44 | img = image.copy().astype(np.float32) 45 | img -= np.mean(img) 46 | img /= np.linalg.norm(img) 47 | # img = (img - img.min() ) 48 | img = np.clip(img, 0, 255) 49 | img *= (1./float(img.max())) 50 | return (img*255).astype(np.uint8) 51 | 52 | # read image sequence 53 | # The training set locates at "resource/training/01" and "resource/training/02" 54 | # The ground truth of training set locates at "resource/training/GT_01" and 55 | # "resource/training/GT_02" 56 | # The testing set locates at "resource/testing/01" and "resource/testing/02" 57 | 58 | if sys.argv[1]: 59 | path=os.path.join("resource/training/01") 60 | else: 61 | path=os.path.join("resource/training/01") 62 | for r,d,f in os.walk(path): 63 | images = [] 64 | enhance_images = [] 65 | for files in f: 66 | if files[-3:].lower()=='tif': 67 | temp = cv2.imread(os.path.join(r,files)) 68 | gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) 69 | images.append(gray.copy()) 70 | enhance_images.append(normalize(gray.copy())) 71 | 72 | print "Total number of image is ", len(images) 73 | print "The shape of image is ", images[0].shape, type(images[0][0,0]) 74 | 75 | def write_image(image, title, index, imgformat='.tiff'): 76 | if index < 10: 77 | name = '0'+str(index) 78 | else: 79 | name = str(index) 80 | cv2.imwrite(title+name+imgformat, image) 81 | 82 | def main(): 83 | 84 | # # Binarization 85 | th = athresh(enhance_images) 86 | threh = th.applythresh() 87 | write_image(threh[0]*255, "threh", 0) 88 | # Nuclei center detection 89 | gvf = GVF(images, threh) 90 | dismap = gvf.distancemap() 91 | newimg = gvf.new_image(4, dismap) # choose alpha as 0.4. 92 | write_image((dismap[0]*10).astype(np.uint8), "dismap", 0) 93 | write_image(newimg[0], "newimg", 0) 94 | gradimg = gvf.compute_gvf(newimg) 95 | 96 | os.chdir("temporary_result") 97 | imgpairs = [] 98 | bin_imgpairs = [] 99 | imgpair_raws = [] 100 | for i, grad in enumerate(gradimg): 101 | imgpair, bin_imgpair, imgpair_raw = gvf.find_certer(grad, i) 102 | imgpairs.append(imgpair) 103 | bin_imgpairs.append(bin_imgpair) 104 | imgpair_raws.append(imgpair_raw) 105 | write_image(imgpair_raw, 'imgpair_raw', i) 106 | write_image(bin_imgpair, 'bin_imgpair', i) 107 | write_image(imgpair, 'imgpair', i) 108 | os.chdir(os.pardir) 109 | 110 | # watershed 111 | ws = WS(newimg, imgpair) 112 | wsimage, binarymark, mark = ws.watershed_compute() 113 | 114 | centroid = [] 115 | slope_length = [] 116 | # Build Delaunay Triangulation 117 | for i in range(len(images)): 118 | graph = GRAPH(mark, binarymark, i) 119 | tempcentroid, tempslope_length = graph.run(True) 120 | centroid.append(tempcentroid) 121 | slope_length.append(tempslope_length) 122 | 123 | # Build the Dissimilarity measure vector 124 | vector = [] 125 | for i in range(len(images)): 126 | print " feature vector: image ", i 127 | v = FEAVECTOR() 128 | v.set_centroid(centroid[i]) 129 | v.set_spatial(slope_length[i]) 130 | v.set_shape(images[i], markers[i]) 131 | v.set_histogram() 132 | v.add_label() 133 | v.add_id(markers[i].max(), i) 134 | vector.append(v.generate_vector()) 135 | 136 | print "num of nuclei: ", len(vector[i]) 137 | 138 | # Feature matching 139 | for i in range(len(images)-1): 140 | print " Feature matching: image ", i 141 | m = SIMPLE_MATCH(i,i+1,[images[i], images[i+1]], vector) 142 | mask.append(m.generate_mask(markers[i], i)) 143 | m.find_match(0.3) 144 | mask = m.match_missing(mask, max_frame=2, max_distance=20) 145 | vector[i+1] = m.mitosis_refine() 146 | m.new_id() 147 | vector[i+1] = m.return_vectors() 148 | 149 | # write images if necessary 150 | os.chdir("temporary_result") 151 | for i in range(len(images)): 152 | write_image(imgpair_raw, 'imgpair_raw', i) 153 | write_image(threh[i].astype(np.uint8)*255, 'threh', i) 154 | write_image(enhance_images[i], 'normalize', i) 155 | write_image(mark[i].astype(np.uint8), 'mark', i) 156 | write_image(binarymark[i].astype(np.uint8), 'binarymark', i) 157 | write_image(tempimg[i].astype(np.uint8), 'tempimg', i) 158 | write_image(dismap[i].astype(np.uint8), 'dismap', i) 159 | os.chdir(os.pardir) 160 | 161 | # if python says run, then we should run 162 | if __name__ == '__main__': 163 | main() -------------------------------------------------------------------------------- /Code/matching.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is to match nuclei in two consecutive frames by Phase Controlled Optimal Matching. 3 | It includes two part: 4 | 1) Dissimilarity measure 5 | 2) Matching 6 | 7 | ''' 8 | import cv2 9 | import numpy as np 10 | import sys 11 | from graph_construction import GRAPH 12 | import os 13 | import imageio 14 | 15 | import matplotlib.pyplot as plt 16 | from mpl_toolkits.mplot3d import Axes3D 17 | import matplotlib.cm as cm 18 | 19 | Max_dis = 100000 20 | 21 | def write_image(image, title, index, imgformat='.tif'): 22 | if index < 10: 23 | name = '00'+str(index) 24 | else: 25 | name = '0'+str(index) 26 | cv2.imwrite(title+name+imgformat, image.astype(np.uint16)) 27 | 28 | class FEAVECTOR(): 29 | ''' 30 | This class builds a feature vector for each segments. 31 | The format of each vector is: 32 | v(k,i) = [c(k,i), s(k, i), h(k, i), e(k, i)], where k is the 33 | index of the image (frame) and i is the label of each segment. 34 | 35 | c(k,i): the centroid of each segment (y, x); 36 | s(k,i): the binary shape of each segment; 37 | h(k,i): the intensity distribution (hsitogram) of the segment; 38 | e(k,i): the spatial distribution of the segment. Its format is 39 | like (l(k, i, p), theta(k, i, p)), where p represent different 40 | line connected with different segment. 41 | 42 | ''' 43 | def __init__(self, centroid=None, shape=None, histogram=None, spatial=None, \ 44 | ID=None, start = None, end=None, label=None): 45 | self.c = centroid 46 | self.s = shape 47 | self.h = histogram 48 | self.e = spatial 49 | self.id = ID 50 | self.start = start 51 | self.end = end 52 | self.l = label 53 | 54 | def add_id(self, num, index): 55 | ''' 56 | This function adds cell id for each cell. 57 | 58 | ''' 59 | if index == 0: 60 | self.id = np.linspace(1, num, num) 61 | else: 62 | self.id= np.linspace(-1, -1, num) 63 | 64 | def add_label(self): 65 | ''' 66 | This function is to add labels for each neclei for post process. 67 | 68 | ''' 69 | self.l = np.linspace(0, 0, len(self.c)) 70 | 71 | def set_centroid(self, centroid): 72 | ''' 73 | This function sets the centroid for all neclei. 74 | 75 | Input: the set of centroid: # of images * # of neclei * 2 (y, x) 76 | 77 | Output: None 78 | 79 | ''' 80 | self.c = centroid 81 | 82 | def set_spatial(self, spatial): 83 | ''' 84 | This function sets the spatial distrbution for all neclei. 85 | 86 | Input: the set of centroid: # of images * # of neclei * # of line segments (length, slope) 87 | 88 | Output: None 89 | 90 | ''' 91 | self.e = spatial 92 | 93 | def set_shape(self, image, marker): 94 | ''' 95 | This function sets the binary shape for all necluei. 96 | 97 | Input: the original images: # of images * height * weight 98 | the labeled images: # of images * nucei's height * nucei's weight () 99 | 100 | Output: None 101 | 102 | ''' 103 | def boundingbox(image): 104 | y, x = np.where(image) 105 | return min(x), min(y), max(x), max(y) 106 | 107 | shape = [] 108 | 109 | for label in range(1, marker.max()+1): 110 | tempimg = marker.copy() 111 | tempimg[tempimg!=label] = 0 112 | tempimg[tempimg==label] = 1 113 | if tempimg.sum(): 114 | minx, miny, maxx, maxy = boundingbox(tempimg) 115 | shape.append((tempimg[miny:maxy+1, minx:maxx+1], image[miny:maxy+1, minx:maxx+1])) 116 | else: 117 | shape.append(([], [])) 118 | 119 | self.s = shape 120 | 121 | def set_histogram(self): 122 | ''' 123 | Note: this function must be implemneted after set_shape(). 124 | 125 | ''' 126 | def computehistogram(image): 127 | h, w = image.shape[:2] 128 | his = np.zeros((256,1)) 129 | for y in range(h): 130 | for x in range(w): 131 | his[image[y, x], 0] += 1 132 | return his 133 | 134 | assert self.s != None, "this function must be implemneted after set_shape()." 135 | 136 | his = [] 137 | 138 | for j in range(len(self.s)): 139 | img = self.s[j][1] 140 | if len(img): 141 | temphis = computehistogram(img) 142 | his.append(temphis) 143 | else: 144 | his.append(np.zeros((256,1))) 145 | 146 | self.h = his 147 | 148 | def generate_vector(self): 149 | ''' 150 | This function is to convert the vector maxtrics into a list. 151 | 152 | Output: a list of vector: [v0, v1, ....] 153 | 154 | ''' 155 | vector = [] 156 | for i in range(len(self.c)): 157 | vector.append(FEAVECTOR(centroid=self.c[i], \ 158 | shape=self.s[i], \ 159 | histogram=self.h[i], \ 160 | spatial=self.e[i], \ 161 | ID=self.id[i], \ 162 | label=self.l[i])) 163 | return vector 164 | 165 | class SIMPLE_MATCH(): 166 | ''' 167 | This class is simple matching a nucleus into a nucleus in the previous frame by 168 | find the nearest neighborhood. 169 | 170 | ''' 171 | def __init__(self, index0, index1, images, vectors): 172 | self.v0 = vectors[index0] 173 | self.v1 = vectors[index1] 174 | self.i0 = index0 175 | self.i1 = index1 176 | self.images = images 177 | self.vs = vectors 178 | 179 | def distance_measure(self, pv0, pv1, alpha1=0.31, alpha2=0.15, alpha3=0.23, alpha4=0.31, phase = 1): 180 | ''' 181 | This function measures the distence of the two given feature vectors. 182 | 183 | This distance metrics we use is: 184 | d(v(k, i), v(k+1, j)) = alpha1 * d(c(k, i), c(k+1, j)) + 185 | alpha2 * q1 * d(s(k, i), s(k+1, j)) + 186 | alpha3 * q2 * d(h(k, i), h(k+1, j)) + 187 | alpha4 * d(e(k, i), e(k+1, j)) 188 | Input: The two given feature vectors, 189 | and the set of parameters. 190 | 191 | Output: the distance of the two given vectors. 192 | 193 | ''' 194 | def centriod_distance(c0, c1, D=30.): 195 | dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2) 196 | return dist/D if dist < D else 1 197 | 198 | def maxsize_image(image1, image2): 199 | y1, x1 = np.where(image1) 200 | y2, x2 = np.where(image2) 201 | return min(min(x1), min(x2)), min(min(y1), min(y2)), \ 202 | max(max(x1), max(x2)), max(max(y1), max(y2)), 203 | 204 | def symmetry(image, shape): 205 | h, w = image.shape[:2] 206 | newimg = np.zeros(shape) 207 | newimg[:h, :w] = image 208 | v = float(shape[0] - h)/2. 209 | u = float(shape[1] - w)/2. 210 | M = np.float32([[1,0,u],[0,1,v]]) 211 | return cv2.warpAffine(newimg,M,(shape[1],shape[0])) 212 | 213 | def shape_distance(s0, s1): 214 | minx, miny, maxx, maxy = maxsize_image(s0, s1) 215 | height = maxy - miny + 1 216 | width = maxx - minx + 1 217 | 218 | img0 = symmetry(s0, (height, width)) 219 | img1 = symmetry(s1, (height, width)) 220 | 221 | num = 0. 222 | deno = 0. 223 | for y in range(height): 224 | for x in range(width): 225 | if img0[y, x] and img1[y, x]: 226 | num += 1 227 | if img0[y, x] or img1[y, x]: 228 | deno += 1 229 | 230 | return 1 - num/deno 231 | 232 | def histogram_distance(h0, h1): 233 | max_p = max([img.max() for img in self.images]) 234 | min_p = min([img.min() for img in self.images]) 235 | 236 | num = 0. 237 | deno = 0. 238 | for t in range(min_p, max_p+1): 239 | num += abs(h0[t] - h1[t]) 240 | deno += max(h0[t], h1[t]) 241 | 242 | return num/deno 243 | 244 | def spatial_distance(e0, e1): 245 | m = float(len(e0)) 246 | n = float(len(e1)) 247 | if not n or not m: 248 | return Max_dis 249 | sumdis = 0. 250 | for pe1 in e1: 251 | dis = None 252 | for pe0 in e0: 253 | tempdis = abs(pe0[0]-pe1[0])/max(pe0[0], pe1[0]) * \ 254 | abs(pe0[1]-pe1[1]) 255 | if dis == None: 256 | dis = tempdis 257 | elif dis > tempdis: 258 | dis = tempdis 259 | sumdis += dis 260 | return sumdis*(1./n) 261 | 262 | if len(pv0.s[0]) and len(pv1.s[0]): 263 | dist = alpha1 * centriod_distance(pv0.c, pv1.c)+ \ 264 | alpha2 * shape_distance(pv0.s[1], pv1.s[1]) * phase + \ 265 | alpha3 * histogram_distance(pv0.h, pv1.h) * phase + \ 266 | alpha4 * spatial_distance(pv0.e, pv1.e) 267 | else: 268 | dist = Max_dis 269 | 270 | return dist 271 | 272 | def phase_identify(self, pv1, min_times_MA2ma = 2): 273 | _, contours, hierarchy = cv2.findContours(pv1.s[0].astype(np.uint8), 1, 2) 274 | if not len(contours): 275 | return 1 276 | cnt = contours[0] 277 | if len(cnt) >= 5: 278 | (x,y),(ma,MA),angle = cv2.fitEllipse(cnt) 279 | if ma and MA/ma > min_times_MA2ma: 280 | return 0 281 | elif not ma and MA: 282 | return 0 283 | else: 284 | return 1 285 | else: 286 | return 1 287 | 288 | def find_match(self, max_distance=1): 289 | ''' 290 | This function is to find the nearest neighborhood between two 291 | successive frame. 292 | 293 | ''' 294 | def centriod_distance(c0, c1, D=30.): 295 | dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2) 296 | return dist/D if dist < D else 1 297 | 298 | for i, pv1 in enumerate(self.v1): 299 | dist = np.ones((len(self.v0), 3), np.float32) 300 | count = 0 301 | q = self.phase_identify(pv1, 3) 302 | for j, pv0 in enumerate(self.v0): 303 | if centriod_distance(pv0.c, pv1.c) < 1: 304 | dist[count][0] = self.distance_measure(pv0, pv1, phase=q) 305 | dist[count][1] = pv0.l 306 | dist[count][2] = pv0.id 307 | count += 1 308 | sort_dist = sorted(dist, key=lambda a_entry: a_entry[0]) 309 | if sort_dist[0][0] < max_distance: 310 | self.v1[i].l = sort_dist[0][1] 311 | self.v1[i].id = sort_dist[0][2] 312 | 313 | def mitosis_refine(self): 314 | ''' 315 | This function is to find died cell due to the by mitosis. 316 | 317 | ''' 318 | def find_sibling(pv0): 319 | ''' 320 | This function is to find sibling cells according to the centroid of 321 | pv0. The criteria of sibling is: 322 | 1. the jaccard cooeficient of the two cells is above 0.5 323 | 2. the sum of the two areas should in the range [A, 2.5A], where 324 | A is the area of the pv0 325 | 3. the position of the two cells should be not larger than 20 pixels. 326 | 327 | Input: pv0: the parent cell that you want to find siblings; 328 | 329 | Output: the index of the siblings. 330 | 331 | ''' 332 | def maxsize_image(image1, image2): 333 | y1, x1 = np.where(image1) 334 | y2, x2 = np.where(image2) 335 | return min(min(x1), min(x2)), min(min(y1), min(y2)), \ 336 | max(max(x1), max(x2)), max(max(y1), max(y2)), 337 | 338 | def symmetry(image, shape): 339 | h, w = image.shape[:2] 340 | newimg = np.zeros(shape) 341 | newimg[:h, :w] = image 342 | v = float(shape[0] - h)/2. 343 | u = float(shape[1] - w)/2. 344 | M = np.float32([[1,0,u],[0,1,v]]) 345 | return cv2.warpAffine(newimg,M,(shape[1],shape[0])) 346 | 347 | def jaccard(s0, s1): 348 | minx, miny, maxx, maxy = maxsize_image(s0, s1) 349 | height = maxy - miny + 1 350 | width = maxx - minx + 1 351 | 352 | img0 = symmetry(s0, (height, width)) 353 | img1 = symmetry(s1, (height, width)) 354 | 355 | num = 0. 356 | deno = 0. 357 | for y in range(height): 358 | for x in range(width): 359 | if img0[y, x] and img1[y, x]: 360 | num += 1 361 | if img0[y, x] or img1[y, x]: 362 | deno += 1 363 | 364 | return num/deno 365 | 366 | sibling_cand = [] 367 | for i, pv1 in enumerate(self.v1): 368 | if np.linalg.norm(pv1.c-pv0.c) < 50: 369 | sibling_cand.append([pv1, i]) 370 | 371 | sibling_pair = [] 372 | area = pv0.s[0].sum() 373 | jaccard_value = [] 374 | for sibling0 in sibling_cand: 375 | for sibling1 in sibling_cand: 376 | if (sibling1[0].c != sibling0[0].c).all(): 377 | sum_area = sibling1[0].s[0].sum()+sibling0[0].s[0].sum() 378 | similarity = jaccard(sibling0[0].s[0], sibling1[0].s[0]) 379 | if similarity > 0.4 and (sum_area > 2*area): 380 | sibling_pair.append([sibling0, sibling1]) 381 | jaccard_value.append(similarity) 382 | if len(jaccard_value): 383 | return sibling_pair[np.argmax(jaccard_value)] 384 | else: 385 | return 0 386 | 387 | v1_ids = [] 388 | for pv1 in self.v1: 389 | v1_ids.append(pv1.id) 390 | 391 | for pv0 in self.v0: 392 | if pv0.id not in v1_ids and len(pv0.s[0]) and not self.phase_identify(pv0): 393 | sibling = find_sibling(pv0) 394 | if sibling: 395 | [s0, s1] = sibling 396 | if s0[0].l==0 and s1[0].l==0 and \ 397 | s0[0].id==-1 and s1[0].id==-1: 398 | self.v1[s0[1]].l = pv0.id 399 | self.v1[s1[1]].l = pv0.id 400 | 401 | return self.v1 402 | 403 | def match_missing(self, mask, max_frame = 1, max_distance = 10, min_shape_similarity = 0.6): 404 | ''' 405 | This function is to match the cells that didn't show in the last frame caused by 406 | program fault. In order to match them, we need to seach the cell in the previous 407 | frame with in the certain range and with similar shape. 408 | 409 | ''' 410 | def centriod_distance(c0, c1): 411 | dist = np.sqrt((c0[0]-c1[0])**2 + (c0[1]-c1[1])**2) 412 | return dist 413 | 414 | def maxsize_image(image1, image2): 415 | y1, x1 = np.where(image1) 416 | y2, x2 = np.where(image2) 417 | return min(min(x1), min(x2)), min(min(y1), min(y2)), \ 418 | max(max(x1), max(x2)), max(max(y1), max(y2)), 419 | 420 | def symmetry(image, shape): 421 | h, w = image.shape[:2] 422 | newimg = np.zeros(shape) 423 | newimg[:h, :w] = image 424 | v = float(shape[0] - h)/2. 425 | u = float(shape[1] - w)/2. 426 | M = np.float32([[1,0,u],[0,1,v]]) 427 | return cv2.warpAffine(newimg,M,(shape[1],shape[0])) 428 | 429 | def shape_similarity(s0, s1): 430 | if len(s0) and len(s1): 431 | minx, miny, maxx, maxy = maxsize_image(s0, s1) 432 | height = maxy - miny + 1 433 | width = maxx - minx + 1 434 | 435 | img0 = symmetry(s0, (height, width)) 436 | img1 = symmetry(s1, (height, width)) 437 | 438 | num = 0. 439 | deno = 0. 440 | for y in range(height): 441 | for x in range(width): 442 | if img0[y, x] and img1[y, x]: 443 | num += 1 444 | if img0[y, x] or img1[y, x]: 445 | deno += 1 446 | return num/deno 447 | 448 | else: 449 | return 0. 450 | 451 | def add_marker(index_find, index_new, pv0_id): 452 | temp = mask[index_new] 453 | find = mask[index_find] 454 | temp[find==pv0_id] = pv0_id 455 | print "index_new: ", index_new 456 | return temp 457 | 458 | for i, pv1 in enumerate(self.v1): 459 | if pv1.id == -1: 460 | for index in range(1, max_frame+1): 461 | if self.i0-index >= 0: 462 | vector = self.vs[self.i0-index] 463 | for pv0 in vector: 464 | if centriod_distance(pv0.c, pv1.c) < max_distance and \ 465 | shape_similarity(pv0.s[0], pv1.s[0]) > min_shape_similarity: 466 | self.v1[i].id = pv0.id 467 | self.v1[i].l = pv0.l 468 | print "missing in frame: ", self.i1, "find in frame: ", \ 469 | self.i0-index, "ID: ", pv0.id, " at: ", pv0.c 470 | for i in range(self.i0-index+1, self.i1): 471 | mask[i] = add_marker(self.i0-index, i, pv0.id) 472 | return mask 473 | 474 | def new_id(self): 475 | ''' 476 | This function is to add new labels for the necles that are marked as -1. 477 | 478 | 479 | ''' 480 | def find_max_id(vectors): 481 | max_id = 0 482 | for vector in vectors: 483 | for pt in vector: 484 | if pt.id > max_id: 485 | max_id = pt.id 486 | return max_id 487 | max_id = find_max_id(self.vs) 488 | max_id += 1 489 | for i, pv1 in enumerate(self.v1): 490 | if pv1.id == -1: 491 | self.v1[i].id = max_id 492 | max_id += 1 493 | 494 | def generate_mask(self, marker, index, isfinal=False): 495 | ''' 496 | This function is to generate a 16-bit image as mask image. 497 | 498 | ''' 499 | h, w = marker.shape[:2] 500 | mask = np.zeros((h, w), np.uint16) 501 | pts = list(set(marker[marker>0])) 502 | if not isfinal: 503 | assert len(pts)==len(self.v0), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v0)) 504 | for pt, pv in zip(pts, self.v0): 505 | mask[marker==pt] = pv.id 506 | 507 | else: 508 | assert len(pts)==len(self.v1), 'len(pts): %s != len(self.v0): %s' % (len(pts), len(self.v1)) 509 | for pt, pv in zip(pts, self.v1): 510 | mask[marker==pt] = pv.id 511 | 512 | os.chdir("test") 513 | write_image(mask, "mask", index) 514 | os.chdir(os.pardir) 515 | return mask 516 | 517 | def return_vectors(self): 518 | ''' 519 | This function is to return the vectors that we have already 520 | changed. 521 | 522 | Output: the vectors from the k+1 frame. 523 | 524 | ''' 525 | return self.v1 526 | 527 | def set_date(vectors): 528 | ''' 529 | This function is to add the start and end frame of each vector and 530 | combine the vector with same id. 531 | 532 | Input: the list of vectors in different frames. 533 | 534 | Output: the list of vectors of all cell with different id. 535 | 536 | ''' 537 | max_id = 0 538 | for vector in vectors: 539 | for pv in vector: 540 | if pv.id > max_id: 541 | max_id = pv.id 542 | 543 | print "max_id: ", max_id 544 | output = np.zeros((max_id, 4)) 545 | output[:,0] = np.linspace(1, max_id, max_id) # set the cell ID 546 | output[:,1] = len(vectors) 547 | for frame, vector in enumerate(vectors): 548 | for pv in vector: 549 | if output[pv.id-1][1] > frame: # set the start frame 550 | output[pv.id-1][1] = frame 551 | if output[pv.id-1][2] < frame: # set the end frame 552 | output[pv.id-1][2] = frame 553 | output[pv.id-1][3] = pv.l # set tht cell parent ID 554 | 555 | return output 556 | 557 | def write_info(vector, name): 558 | ''' 559 | This function is to write info. of each vector. 560 | 561 | Input: the list of vector generated by set_date() and 562 | the name of output file. 563 | 564 | ''' 565 | with open(name+".txt", "w+") as file: 566 | for p in vector: 567 | file.write(str(int(p[0]))+" "+\ 568 | str(int(p[1]))+" "+\ 569 | str(int(p[2]))+" "+\ 570 | str(int(p[3]))+"\n") 571 | 572 | ''' 573 | This part is to test the matching scheme with single image 574 | 575 | Input: the original image; 576 | the labeled image; 577 | the binary labeled image. 578 | 579 | ''' 580 | def main(): 581 | def normalize(image): 582 | ''' 583 | This function is to normalize the input grayscale image by 584 | substracting globle mean and dividing standard diviation for 585 | visualization. 586 | 587 | Input: a grayscale image 588 | 589 | Output: normolized grascale image 590 | 591 | ''' 592 | img = image.copy().astype(np.float32) 593 | img -= np.mean(img) 594 | img /= np.linalg.norm(img) 595 | img = (img - img.min() ) 596 | img *= (1./float(img.max())) 597 | return (img*255).astype(np.uint8) 598 | 599 | path=os.path.join("resource/training/01") 600 | for r,d,f in os.walk(path): 601 | images = [] 602 | enhance_images = [] 603 | for files in f: 604 | if files[-3:].lower()=='tif': 605 | temp = cv2.imread(os.path.join(r,files)) 606 | gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) 607 | images.append(gray.copy()) 608 | enhance_images.append(normalize(gray.copy())) 609 | 610 | print "Total number of image is ", len(images) 611 | print "The shape of image is ", images[0].shape, type(images[0][0,0]) 612 | 613 | path=os.path.join("test/mark_train_01") 614 | markers = [] 615 | for r,d,f in os.walk(path): 616 | for files in f: 617 | if files[:1].lower()=='m': 618 | temp = cv2.imread(os.path.join(r,files)) 619 | gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) 620 | markers.append(gray.copy()) 621 | 622 | path=os.path.join("test/binarymark_train_01") 623 | binarymark = [] 624 | for r,d,f in os.walk(path): 625 | for files in f: 626 | if files[:1].lower()=='b': 627 | temp = cv2.imread(os.path.join(r,files)) 628 | gray = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY) 629 | binarymark.append(gray.copy()) 630 | 631 | print "Total number of markers is ", len(markers) 632 | print "The shape of markers is ", markers[0].shape, type(markers[0][0,0]) 633 | 634 | print "Total number of binarymark is ", len(binarymark) 635 | print "The shape of binarymark is ", binarymark[0].shape, type(binarymark[0][0,0]) 636 | 637 | centroid = [] 638 | slope_length = [] 639 | 640 | # Build Delaunay Triangulation 641 | for i in range(len(images)): 642 | print " graph_construction: image ", i 643 | graph = GRAPH(markers, binarymark, i) 644 | tempcentroid, tempslope_length = graph.run() 645 | centroid.append(tempcentroid) 646 | slope_length.append(tempslope_length) 647 | 648 | # Feature vector construction 649 | vector = [] 650 | max_id = 0 651 | mask = [] 652 | for i in range(len(images)): 653 | print " feature vector: image ", i 654 | v = FEAVECTOR() 655 | v.set_centroid(centroid[i]) 656 | v.set_spatial(slope_length[i]) 657 | v.set_shape(images[i], markers[i]) 658 | v.set_histogram() 659 | v.add_label() 660 | v.add_id(markers[i].max(), i) 661 | vector.append(v.generate_vector()) 662 | 663 | print "num of nuclei: ", len(vector[i]) 664 | 665 | # Feature matching 666 | for i in range(len(images)-1): 667 | print " Feature matching: image ", i 668 | m = SIMPLE_MATCH(i,i+1,[images[i], images[i+1]], vector) 669 | mask.append(m.generate_mask(markers[i], i)) 670 | m.find_match(0.3) 671 | mask = m.match_missing(mask, max_frame=2, max_distance=20) 672 | vector[i+1] = m.mitosis_refine() 673 | m.new_id() 674 | vector[i+1] = m.return_vectors() 675 | 676 | print " Feature matching: image ", i+1 677 | mask.append(m.generate_mask(markers[i+1], i+1, True)) 678 | 679 | cells = set_date(vector) 680 | write_info(cells, "res_track") 681 | 682 | def find_max_id(vector): 683 | max_id = 0 684 | for pv in vector: 685 | for p in pv: 686 | if p.id > max_id: 687 | max_id = p.id 688 | return max_id 689 | 690 | # This part is to mark the result in the normolized image and 691 | # write the gif image. 692 | max_id = find_max_id(vector) 693 | colors = [np.random.randint(0, 255, size=max_id),\ 694 | np.random.randint(0, 255, size=max_id),\ 695 | np.random.randint(0, 255, size=max_id)] 696 | font = cv2.FONT_HERSHEY_SIMPLEX 697 | selecy_id = 9 698 | for i, m in enumerate(mask): 699 | print " write the gif image: image ", i 700 | enhance_images[i] = cv2.cvtColor(enhance_images[i],cv2.COLOR_GRAY2RGB) 701 | for pv in vector[i]: 702 | center = pv.c 703 | if pv.l == selecy_id or pv.id == selecy_id: 704 | if not pv.l: 705 | color = (colors[0][int(pv.id)-1],\ 706 | colors[1][int(pv.id)-1],\ 707 | colors[2][int(pv.id)-1],) 708 | else: 709 | color = (colors[0][int(pv.l)-1],\ 710 | colors[1][int(pv.l)-1],\ 711 | colors[2][int(pv.l)-1],) 712 | 713 | if m[center[0], center[1]]: 714 | enhance_images[i][m==pv.id] = color 715 | cv2.putText(enhance_images[i],\ 716 | str(int(pv.id)),(int(pv.c[1]), \ 717 | int(pv.c[0])), 718 | font, 0.5,\ 719 | (255,255,255),1) 720 | imageio.mimsave('mitosis_final.gif', enhance_images, duration=0.6) 721 | 722 | # This part is for showing the result in 3D plot. 723 | fig = plt.figure() 724 | ax = fig.add_subplot(111, projection='3d') 725 | colors = cm.rainbow(np.linspace(0, 1, max_id)) 726 | colors = np.random.permutation(colors) 727 | ax.set_xlim(0, images[0].shape[0]) 728 | ax.set_ylim(0, images[0].shape[1]) 729 | ax.set_zlim(0, len(images)) 730 | ax.set_xlabel('x axis') 731 | ax.set_ylabel('y axis') 732 | ax.set_zlabel('frame') 733 | plotimg = [] 734 | 735 | def scatter_pts(vector, ax, i): 736 | for pt in vector: 737 | if not len(pt.s[0]) or sum(pt.c) == 0: 738 | continue 739 | 740 | x_p = pt.c[0] 741 | y_p = pt.c[1] 742 | z_p = i 743 | if pt.l == selecy_id or (pt.id == selecy_id and pt.l==0): 744 | if not pt.l: 745 | ax.scatter(x_p, y_p, z_p, s=2, color=colors[int(pt.id)-1]) 746 | else: 747 | ax.scatter(x_p, y_p, z_p, s=2, color=colors[int(pt.l)-1]) 748 | 749 | scatter_pts(vector[1], ax, 0) 750 | plt.savefig('testplot.jpg', dpi=fig.dpi) 751 | plotimg.append(cv2.imread('testplot.jpg')) 752 | 753 | for i in range(len(images)-1): 754 | print " Feature plot: image ", i 755 | 756 | for pt1 in vector[i+1]: 757 | for pt0 in vector[i]: 758 | if (pt0.id == pt1.id or pt0.id == pt1.l) and \ 759 | (pt1.l == selecy_id or (pt1.id == selecy_id and pt1.l==0)): 760 | x1 = pt1.c[0] 761 | x0 = pt0.c[0] 762 | y1 = pt1.c[1] 763 | y0 = pt0.c[1] 764 | z1 = (i+1) 765 | z0 = i 766 | if not pt0.l: 767 | ax.plot([x1, x0], \ 768 | [y1, y0], \ 769 | [z1, z0], \ 770 | color=colors[int(pt0.id)-1]) 771 | else: 772 | ax.plot([x1, x0], \ 773 | [y1, y0], \ 774 | [z1, z0], \ 775 | color=colors[int(pt0.l)-1]) 776 | plt.savefig('testplot.jpg', dpi=fig.dpi) 777 | plotimg.append(cv2.imread('testplot.jpg')) 778 | 779 | plt.show() 780 | imageio.mimsave('temporary_result/plotimg.gif', plotimg, duration=0.6) 781 | 782 | 783 | # if python says run, then we should run 784 | if __name__ == '__main__': 785 | main() 786 | -------------------------------------------------------------------------------- /Code/watershed.py: -------------------------------------------------------------------------------- 1 | ''' 2 | This file is to compute watershed given the seed image in the gvf.py. 3 | 4 | ''' 5 | import cv2 6 | import numpy as np 7 | from numpy import unique 8 | 9 | class WATERSHED(): 10 | ''' 11 | This class contains all the function to compute watershed. 12 | 13 | ''' 14 | def __init__(self, images, markers): 15 | self.images = images 16 | self.markers = markers 17 | 18 | def watershed_compute(self): 19 | ''' 20 | This function is to compute watershed given the newimage and the seed image 21 | (center candidates). In this function, we use cv2.watershed to implement watershed. 22 | 23 | Input: newimage (height * weight * # of images) 24 | 25 | Output: watershed images (height * weight * # of images) 26 | 27 | ''' 28 | result = [] 29 | outmark = [] 30 | outbinary = [] 31 | 32 | for i in range(len(self.images)): 33 | # generate a 3-channel image in order to use cv2.watershed 34 | imgcolor = np.zeros((self.images[i].shape[0], self.images[i].shape[1], 3), np.uint8) 35 | for c in range(3): 36 | imgcolor[:,:,c] = self.images[i] 37 | 38 | # compute marker image (labelling) 39 | if len(self.markers[i].shape) == 3: 40 | self.markers[i] = cv2.cvtColor(self.markers[i],cv2.COLOR_BGR2GRAY) 41 | _, mark = cv2.connectedComponents(self.markers[i]) 42 | # mark = self.markers[i].astype(np.int32) 43 | # watershed! 44 | mark = cv2.watershed(imgcolor,mark) 45 | 46 | u, counts = unique(mark, return_counts=True) 47 | counter = dict(zip(u, counts)) 48 | for i in counter: 49 | if counter[i] > 1200: 50 | mark[mark==i] = 0 51 | 52 | kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(2,2)) 53 | mark = cv2.morphologyEx(mark.astype(np.uint8), cv2.MORPH_CLOSE, kernel) 54 | _, mark = cv2.connectedComponents(mark.astype(np.uint8)) 55 | 56 | # mark image and add to the result 57 | temp = cv2.cvtColor(imgcolor,cv2.COLOR_BGR2GRAY) 58 | result.append(temp) 59 | outmark.append(mark.astype(np.uint8)) 60 | 61 | binary = mark.copy() 62 | binary[mark>0] = 255 63 | outbinary.append(binary.astype(np.uint8)) 64 | 65 | return result, outbinary, outmark 66 | 67 | def main(): 68 | ''' 69 | This part is for testing watershed.py with single image. 70 | 71 | Input: an original image 72 | a seeds image 73 | 74 | Output: Binary image after watershed 75 | 76 | ''' 77 | images = [] 78 | image = cv2.imread(sys.argv[1]) 79 | images.append(cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)) 80 | markers = [] 81 | marker = cv2.imread(sys.argv[2]) 82 | markers.append(cv2.cvtColor(marker, cv2.COLOR_BGR2GRAY)) 83 | 84 | # watershed 85 | ws = WS(newimg, imgpair) 86 | wsimage, binarymark, mark = ws.watershed_compute() 87 | 88 | cv2.imwrite('binarymark.tif', (np.clip(binarymark, 0, 255)).astype(np.uint8)) 89 | 90 | # if python says run, then we should run 91 | if __name__ == '__main__': 92 | main() 93 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Hanxiang Hao 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Cancer-Cell-Tracking 2 | Python implementation on tracking cancer cell based on [Li et al.](https://www.ncbi.nlm.nih.gov/pubmed/19643704), which used watershed algorithm to segment cells and built a feature vector for cell tracking including the information of position, shape, spatial distribution and texture. 3 | 4 | ## Usage 5 | 1. The data can be found at [Cell Tracking Challenge Website](http://celltrackingchallenge.net). 6 | 7 | 2. ipython notebook: to better show the algorithm step by step, besides the python scripts, I also create a ipython notebook to visualize the interim results. 8 | 9 | 3. Some explanation of the scripts: 10 | ```Python 11 | main.py # the main procedure including all steps. 12 | ``` 13 | ```Python 14 | adaptivethresh.py # compute adaptive thresholding of image sequence in order to generate binary image for Nuclei segmentation. 15 | ``` 16 | ```Python 17 | gvf.py # compute gradient vector field (GVF) to find the seeds for following watershed. 18 | ``` 19 | ```Python 20 | watershed.py # segment cells 21 | ``` 22 | ```Python 23 | graph_construction.py # generate a neighboring graph contraction using Delaunary Triangulation. 24 | ``` 25 | ```Python 26 | matching.py # calculate feature vector for each cell and match cells. 27 | ``` 28 | 29 | ## Results 30 | 1. Result of original image sequence. 31 | 32 | ![ ](images/nomolizedimg.gif "Result of original image sequence") 33 | ---- 34 | 35 | 2. Result of tracking all cells. 36 | 37 | ![ ](images/enhance_images.gif "Result of tracking all cells") 38 | ---- 39 | 40 | 3. Result of tracking specific cell in mitosis. 41 | 42 | ![ ](images/mitosis_final.gif "Result of tracking specific cell in mitosis") 43 | ---- 44 | 45 | 4. Plot of the previous tracking. 46 | 47 | ![ ](images/plotimg.gif "Plot of tracking the mitosis cell") 48 | -------------------------------------------------------------------------------- /images/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/images/.DS_Store -------------------------------------------------------------------------------- /images/enhance_images.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/images/enhance_images.gif -------------------------------------------------------------------------------- /images/mitosis_final.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/images/mitosis_final.gif -------------------------------------------------------------------------------- /images/nomolizedimg.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/images/nomolizedimg.gif -------------------------------------------------------------------------------- /images/plotimg.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/images/plotimg.gif -------------------------------------------------------------------------------- /report/cell_tracking_final.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Connor323/Cancer-Cell-Tracking/abca1a4f6015d22311dca046db3ce41ebfe5ab30/report/cell_tracking_final.pdf --------------------------------------------------------------------------------