├── .gitignore ├── .dockerignore ├── test_image.jpg ├── requirements.txt ├── CONTRIBUTORS.md ├── src ├── constants.py ├── image_processing.py ├── image_io.py ├── image_masking.py └── labelling.py ├── .travis.yml ├── Dockerfile ├── .travis ├── cd.sh └── ci.py ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | .gitignore 3 | README.md 4 | test_page.html 5 | -------------------------------------------------------------------------------- /test_image.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CatalystCode/image-segmentation-auto-labels/HEAD/test_image.jpg -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | hug-middleware-cors==1.0.0 2 | hug==2.4.0 3 | numpy==1.11.0 4 | opencv-python==3.4.0.12 5 | scipy==1.0.0 6 | -------------------------------------------------------------------------------- /CONTRIBUTORS.md: -------------------------------------------------------------------------------- 1 | - Arccos Golf 2 | - Josh Shell 3 | - Kyle Steger 4 | 5 | - Microsoft 6 | - Clemens Wolff 7 | - Maragaret Meehan 8 | - Michael Perel 9 | -------------------------------------------------------------------------------- /src/constants.py: -------------------------------------------------------------------------------- 1 | from ast import literal_eval 2 | from os import getenv 3 | 4 | 5 | MASK_COLOR = literal_eval(getenv('MASK_COLOR', '(0, 0, 255)')) 6 | OUTPUT_IMAGE_FORMAT = getenv('OUTPUT_IMAGE_FORMAT', 'jpg') 7 | -------------------------------------------------------------------------------- /src/image_processing.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | 4 | 5 | def apply_morphology(mask, kernel=(10, 10), iterations=2): 6 | kernel = np.ones(kernel, np.uint8) 7 | 8 | mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, 9 | iterations=iterations) 10 | 11 | return mask 12 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | 3 | python: 4 | - 3.6 5 | 6 | cache: pip 7 | 8 | sudo: required 9 | 10 | services: 11 | - docker 12 | 13 | script: .travis/ci.py 14 | 15 | deploy: 16 | - provider: script 17 | script: .travis/cd.sh 18 | on: 19 | repo: CatalystCode/image-segmentation-auto-labels 20 | tags: true 21 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6 2 | 3 | RUN pip3 install --no-cache-dir numpy==1.11.0 scipy==1.0.0 opencv-python==3.4.0.12 4 | 5 | ADD requirements.txt /app/requirements.txt 6 | 7 | RUN pip3 install --no-cache-dir -r /app/requirements.txt 8 | 9 | ADD src /app 10 | ADD test_image.jpg /data/test_image.jpg 11 | 12 | RUN pip3 install --no-cache-dir flake8 \ 13 | && flake8 /app \ 14 | && pip3 uninstall --yes flake8 15 | 16 | RUN printf '#!/usr/bin/env bash\nhug -f /app/labelling.py -c $*' > /do \ 17 | && chmod +x /do 18 | 19 | WORKDIR /app 20 | 21 | CMD ["hug", "-f", "labelling.py", "-p", "80"] 22 | 23 | ENV MASK_COLOR="(0, 0, 255)" 24 | ENV OUTPUT_IMAGE_FORMAT="jpg" 25 | 26 | EXPOSE 80 27 | -------------------------------------------------------------------------------- /src/image_io.py: -------------------------------------------------------------------------------- 1 | from base64 import b64encode 2 | from urllib.request import urlopen 3 | 4 | import cv2 5 | import numpy as np 6 | 7 | 8 | def load_image(image_path): 9 | if image_path.startswith('http://') or image_path.startswith('https://'): 10 | image = urlopen(image_path).read() 11 | image = np.asarray(bytearray(image), dtype=np.uint8) 12 | image = cv2.imdecode(image, -1) 13 | else: 14 | image = cv2.imread(image_path) 15 | 16 | return image 17 | 18 | 19 | def serialize_image(image_rgb, image_type): 20 | _, image_jpg = cv2.imencode('.{}'.format(image_type), image_rgb) 21 | image_encoded = b64encode(image_jpg) 22 | return image_encoded 23 | -------------------------------------------------------------------------------- /.travis/cd.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | set -eo pipefail 4 | 5 | if [ -z "$TRAVIS_TAG" ]; then 6 | echo "Build is not a release, skipping CD" >&2; exit 0 7 | fi 8 | 9 | if [ -z "$DOCKER_USERNAME" ] || [ -z "$DOCKER_PASSWORD" ]; then 10 | echo "No docker credentials configured, unable to publish builds" >&2; exit 1 11 | fi 12 | 13 | docker_image_name="image_segmentation_auto_labels" 14 | 15 | current_tag="$DOCKER_USERNAME/$docker_image_name:$TRAVIS_TAG" 16 | latest_tag="$DOCKER_USERNAME/$docker_image_name:latest" 17 | context="$(dirname $0)/.." 18 | 19 | docker build --tag "$current_tag" "$context" 20 | docker login --username="$DOCKER_USERNAME" --password="$DOCKER_PASSWORD" 21 | docker push "$current_tag" 22 | 23 | docker tag "$current_tag" "$latest_tag" 24 | docker push "$latest_tag" 25 | -------------------------------------------------------------------------------- /src/image_masking.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | 4 | 5 | def otsu_grayscale(image_rgb): 6 | image_gray = cv2.cvtColor(image_rgb, cv2.COLOR_BGR2GRAY) 7 | _, mask = cv2.threshold(image_gray, 0, 255, 8 | cv2.THRESH_BINARY + cv2.THRESH_OTSU) 9 | return mask 10 | 11 | 12 | def _otsu_hsl(image_rgb, channel_name, flip): 13 | image_gray = cv2.cvtColor(image_rgb, cv2.COLOR_BGR2HLS) 14 | hue, lightness, saturation = np.split(image_gray, 3, axis=2) 15 | 16 | hsl = locals()[channel_name] 17 | hsl = hsl.reshape((hsl.shape[0], hsl.shape[1])) 18 | 19 | _, mask = cv2.threshold(hsl, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU) 20 | 21 | if flip: 22 | mask = ~mask 23 | 24 | return mask 25 | 26 | 27 | def otsu_hue(image_rgb): 28 | return _otsu_hsl(image_rgb, channel_name='hue', flip=True) 29 | 30 | 31 | def otsu_saturation(image_rgb): 32 | return _otsu_hsl(image_rgb, channel_name='saturation', flip=True) 33 | 34 | 35 | def otsu_lightness(image_rgb): 36 | return _otsu_hsl(image_rgb, channel_name='lightness', flip=False) 37 | -------------------------------------------------------------------------------- /.travis/ci.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from os.path import abspath 4 | from os.path import dirname 5 | from os.path import join 6 | from os.path import realpath 7 | from sys import path 8 | from unittest import TestCase 9 | 10 | root_dir = abspath(join(dirname(realpath(__file__)), '..')) 11 | path.append(join(root_dir, 'src')) 12 | 13 | import labelling # noqa 14 | 15 | 16 | class EndToEndTest(TestCase): 17 | algorithm = 'otsu_hue' 18 | 19 | def test_list_algorithms(self): 20 | algorithms = labelling.list_algorithms() 21 | 22 | self.assertGreater(len(algorithms), 0) 23 | self.assertIn(self.algorithm, algorithms) 24 | 25 | def test_create_mask(self): 26 | image = join(root_dir, 'test_image.jpg') 27 | algorithm = labelling.image_mask_algorithm_type(self.algorithm) 28 | morph = 0 29 | 30 | mask = labelling.create_mask(image, algorithm, morph) 31 | 32 | self.assertIn('content', mask) 33 | self.assertIn('type', mask) 34 | self.assertIn('encoding', mask) 35 | 36 | 37 | if __name__ == '__main__': 38 | from unittest import main 39 | main(verbosity=2) 40 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Microsoft Commercial Software Engineering team 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/labelling.py: -------------------------------------------------------------------------------- 1 | from inspect import getmembers 2 | from inspect import isfunction 3 | 4 | import hug 5 | from hug_middleware_cors import CORSMiddleware 6 | 7 | from constants import MASK_COLOR 8 | from constants import OUTPUT_IMAGE_FORMAT 9 | from image_io import load_image 10 | from image_io import serialize_image 11 | from image_processing import apply_morphology 12 | import image_masking 13 | 14 | api = hug.API(__name__) 15 | api.http.add_middleware(CORSMiddleware(api)) 16 | 17 | 18 | @hug.type(extend=hug.types.text) 19 | def image_mask_algorithm_type(value): 20 | """The masking algorithm to use for image pre-labelling.""" 21 | 22 | try: 23 | value = getattr(image_masking, value) 24 | except AttributeError: 25 | raise ValueError('Unknown masking algorithm: {}'.format(value)) 26 | else: 27 | return value 28 | 29 | 30 | @hug.get('/algorithms') 31 | @hug.cli() 32 | def list_algorithms(): 33 | """Returns a list of supported image masking algorithms.""" 34 | return [name for (name, _) in getmembers(image_masking, isfunction) 35 | if not name.startswith('_')] 36 | 37 | 38 | @hug.post('/mask') 39 | @hug.cli() 40 | def create_mask(image_path: hug.types.text, 41 | algorithm: image_mask_algorithm_type, 42 | morph: hug.types.number=1): 43 | """Runs the masking algorithm and returns a serialized masked image.""" 44 | 45 | image_rgb = load_image(image_path) 46 | 47 | mask = algorithm(image_rgb) 48 | 49 | if morph > 0: 50 | mask = apply_morphology(mask, iterations=morph) 51 | 52 | image_rgb[mask == 0] = MASK_COLOR 53 | 54 | serialized = serialize_image(image_rgb, OUTPUT_IMAGE_FORMAT) 55 | 56 | return { 57 | 'type': 'image/{}'.format(OUTPUT_IMAGE_FORMAT), 58 | 'encoding': 'base64', 59 | 'content': serialized 60 | } 61 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # image-segmentation-auto-labels 2 | 3 | [![CI status](https://travis-ci.org/CatalystCode/image-segmentation-auto-labels.svg?branch=master)](https://travis-ci.org/CatalystCode/image-segmentation-auto-labels) 4 | [![Docker status](https://img.shields.io/docker/pulls/cwolff/image_segmentation_auto_labels.svg)](https://hub.docker.com/r/cwolff/image_segmentation_auto_labels/) 5 | 6 | ## What's this? 7 | 8 | This repository contains a Python application that can be used to quickly 9 | generate labelled data for image segmentation tasks. The application can be 10 | run as a web service or command line tool and supports a number of algorithms 11 | to generate candidate image masks. 12 | 13 | More detail on the approaches implemented in this repository is available in 14 | the companion Azure Notebook: [Using Otsu's method to pre-label training data for image segmentation](https://notebooks.azure.com/clewolff/libraries/otsu/html/otsu.ipynb). 15 | 16 | ## Usage 17 | 18 | ### As a web service 19 | 20 | Pull and run the auto-labelling service via docker: 21 | 22 | ```sh 23 | docker run -p 8080:80 cwolff/image_segmentation_auto_labels 24 | ``` 25 | 26 | This will start the auto-labelling service on port 8080. There are two main 27 | routes in the service: 28 | 29 | ```sh 30 | # fetch a list of supported image masking algorithms 31 | curl 'http://localhost:8080/algorithms' 32 | 33 | # generate a mask for an image using the provided masking algorithm 34 | curl 'http://localhost:8080/mask' -H 'Content-Type: application/json' -d ' 35 | { 36 | "image_path": "/data/test_image.jpg", 37 | "algorithm": "otsu_hue", 38 | "morph": 0 39 | }' 40 | ``` 41 | 42 | You can use the [test page](https://catalystcode.github.io/image-segmentation-auto-labels/) 43 | to interactively experiment with the service. 44 | 45 | ![Screenshot of auto-labelling service test page](https://user-images.githubusercontent.com/1086421/38619525-520268ac-3d6a-11e8-8eb8-80e752dcb2af.png) 46 | 47 | ### As a command line tool 48 | 49 | Pull and run the auto-labelling tool via docker: 50 | 51 | ```sh 52 | # fetch a list of supported image masking algorithms 53 | docker run cwolff/image_segmentation_auto_labels /do list_algorithms 54 | 55 | # generate a mask for an image using the provided masking algorithm 56 | docker run cwolff/image_segmentation_auto_labels /do create_mask "/data/test_image.jpg" "otsu_hue" "0" 57 | ``` 58 | --------------------------------------------------------------------------------