├── .gitattributes ├── .gitignore ├── README.md ├── UNet.png ├── _config.yml ├── colorize.py ├── demo.png ├── demo ├── cat.jpeg ├── cat.png ├── cat2.jpeg ├── dog-gen.jpg └── dog.jpg ├── main.py ├── notebooks └── u-net.ipynb ├── static ├── color.jpg └── gray.jpg ├── templates └── index.html └── utils └── unet.hdf5 /.gitattributes: -------------------------------------------------------------------------------- 1 | *.hdf5 filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/* 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

Image-Colorization

2 |

Automatic Image Colorization using a Convolutional Network (U-Net)

3 | 9 |
10 |

Model Achitecture

11 |

The model uses U-Net architecture which uses skip connections to preserve the lower level details and structute of an image, that are lost due to contracting bottle-neck.

12 |

13 |
The U-Net Architecture 14 |

15 |
16 |

Demo

17 |

A web interface has been implemented, where a user uploads a grayscale image as input and gets a colored image displayed as output

18 |

19 |
Sample Run 20 |

21 |
22 |

Requirements

23 | 30 | -------------------------------------------------------------------------------- /UNet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/UNet.png -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-architect -------------------------------------------------------------------------------- /colorize.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from scipy.misc import imresize, imread, imsave 3 | #import tensorflow.compat.v1 as tf 4 | #tf.disable_v2_behavior() 5 | from tensorflow.keras.models import load_model 6 | 7 | model = load_model('./utils/unet.hdf5') 8 | #graph = tf.get_default_graph() 9 | 10 | def get_colored(gray_path='./static/gray.jpg', color_path='./static/color.jpg'): 11 | gray = imresize(imread(gray_path), (32,32)) 12 | if gray.shape[2]>=3: gray = gray.mean(axis=-1) 13 | gray = gray/255 14 | #with graph.as_default(): 15 | color = model.predict(gray.reshape((1,32,32,1)))[0] 16 | imsave(color_path, imresize(color, (200,200))) 17 | imsave(gray_path, imresize(gray, (200,200))) 18 | -------------------------------------------------------------------------------- /demo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo.png -------------------------------------------------------------------------------- /demo/cat.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo/cat.jpeg -------------------------------------------------------------------------------- /demo/cat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo/cat.png -------------------------------------------------------------------------------- /demo/cat2.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo/cat2.jpeg -------------------------------------------------------------------------------- /demo/dog-gen.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo/dog-gen.jpg -------------------------------------------------------------------------------- /demo/dog.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/demo/dog.jpg -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | #---FLASK APP--- 2 | from flask import Flask, render_template, request 3 | from colorize import get_colored 4 | import os 5 | app = Flask(__name__) 6 | 7 | @app.route('/', methods = ['GET', 'POST']) 8 | def upload_file(): 9 | gray_path = './static/gray.jpg' 10 | color_path = './static/color.jpg' 11 | if os.path.isfile(gray_path): os.remove(gray_path) 12 | if os.path.isfile(color_path): os.remove(color_path) 13 | 14 | if request.method == 'POST': 15 | f = request.files.get('file') 16 | if f is None: 17 | return "No Image Uploaded!!!" 18 | f.save(gray_path) 19 | get_colored() 20 | return render_template("index.html", after=True) 21 | return render_template("index.html", after=False) 22 | 23 | @app.after_request 24 | def add_header(response): 25 | response.headers['X-UA-Compatible'] = 'IE=Edge,chrome=1,firefox=1' 26 | response.headers['Cache-Control'] = "no-cache, no-store, must-revalidate" 27 | response.headers["Pragma"] = "no-cache" 28 | response.headers["Expires"] = "0" 29 | response.headers['Cache-Control'] = 'public, max-age=0' 30 | return response 31 | -------------------------------------------------------------------------------- /notebooks/u-net.ipynb: -------------------------------------------------------------------------------- 1 | {"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"12e62ee84362b5fb3261fbec5e70602d41e9e9b0"},"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport gc\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"43c069755fa02e5448c8a047d860162511ea6174"},"cell_type":"code","source":"import tensorflow as tf\nkeras = tf.keras\nimport keras.layers as L\nfrom keras.models import Model\ns = tf.InteractiveSession()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"41b2017df0b44415438b18e007b0da27e4622ed2"},"cell_type":"code","source":"def UNet():\n Xinpt = L.Input([None, None, 1])\n X0 = L.Conv2D(64, (3, 3), padding='same')(Xinpt)\n X0 = L.BatchNormalization()(X0)\n X0 = L.LeakyReLU(alpha=0.2)(X0) #l,b,64\n X0 = L.Conv2D(64, (3, 3), strides=1, padding='same')(X0)\n X0 = L.BatchNormalization()(X0)\n X0 = L.LeakyReLU(alpha=0.2)(X0) #l,b,64\n \n X1 = L.MaxPool2D((2, 2), strides=2)(X0) #l/2,b/2,64\n X1 = L.Conv2D(128, (3, 3), padding='same')(X1)\n X1 = L.BatchNormalization()(X1)\n X1 = L.LeakyReLU(alpha=0.2)(X1)\n X1 = L.Conv2D(128, (3, 3), padding='same')(X1)\n X1 = L.BatchNormalization()(X1)\n X1 = L.LeakyReLU(alpha=0.2)(X1) #l/2,b/2,128\n \n X2 = L.MaxPool2D((2, 2), strides=2)(X1) #l/4,b/4,128\n X2 = L.Conv2D(256, (3, 3), padding='same')(X2)\n X2 = L.BatchNormalization()(X2)\n X2 = L.LeakyReLU(alpha=0.2)(X2)\n X2 = L.Conv2D(256, (3, 3), padding='same')(X2)\n X2 = L.BatchNormalization()(X2)\n X2 = L.LeakyReLU(alpha=0.2)(X2) #l/4,b/4,256\n \n X3 = L.MaxPool2D((2, 2), strides=2)(X2) #l/8,b/8,256\n X3 = L.Conv2D(512, (3, 3), padding='same')(X3)\n X3 = L.BatchNormalization()(X3)\n X3 = L.LeakyReLU(alpha=0.2)(X3)\n X3 = L.Conv2D(512, (3, 3), padding='same')(X3)\n X3 = L.BatchNormalization()(X3)\n X3 = L.LeakyReLU(alpha=0.2)(X3) #l/8,b/8,512\n \n X4 = L.MaxPool2D((2, 2), strides=2)(X3) #l/16,b/16,512\n X4 = L.Conv2D(1024, (3, 3), padding='same')(X4)\n X4 = L.BatchNormalization()(X4)\n X4 = L.LeakyReLU(alpha=0.2)(X4)\n X4 = L.Conv2D(1024, (3, 3), padding='same')(X4)\n X4 = L.BatchNormalization()(X4)\n X4 = L.LeakyReLU(alpha=0.2)(X4) #l/16,b/16,1024\n \n X4 = L.Deconv2D(512, (2, 2), strides=2)(X4) #l/8,b/8,512\n X4 = L.Concatenate()([X3, X4]) #l/8,b/8,1024\n X4 = L.Conv2D(512, (3, 3), padding='same')(X4)\n X4 = L.BatchNormalization()(X4)\n X4 = L.Activation('relu')(X4)\n X4 = L.Conv2D(512, (3, 3), padding='same')(X4)\n X4 = L.BatchNormalization()(X4)\n X4 = L.Activation('relu')(X4) #l/8,b/8,512\n \n X3 = L.Deconv2D(256, (2, 2), strides=2)(X4) #l/4,b.4,256\n X3 = L.Concatenate()([X2, X3]) #l/4,b/4,512\n X3 = L.Conv2D(256, (3, 3), padding='same')(X3)\n X3 = L.BatchNormalization()(X3)\n X3 = L.Activation('relu')(X3)\n X3 = L.Conv2D(256, (3, 3), padding='same')(X3)\n X3 = L.BatchNormalization()(X3)\n X3 = L.Activation('relu')(X3) #l/4,b/4,256\n \n X2 = L.Deconv2D(128, (2, 2), strides=2)(X3) #l/2,b/2,128\n X2 = L.Concatenate()([X1, X2]) #l/2,b/2,256\n X2 = L.Conv2D(128, (3, 3), padding='same')(X2)\n X2 = L.BatchNormalization()(X2)\n X2 = L.Activation('relu')(X2)\n X2 = L.Conv2D(128, (3, 3), padding='same')(X2)\n X2 = L.BatchNormalization()(X2)\n X2 = L.Activation('relu')(X2) #l/2,b/2,128\n \n X1 = L.Deconv2D(64, (2, 2), strides=2)(X2) #l,b,64\n X1 = L.Concatenate()([X0, X1]) #l,b,128\n X1 = L.Conv2D(64, (3, 3), padding='same')(X1)\n X1 = L.BatchNormalization()(X1)\n X1 = L.Activation('relu')(X1)\n X1 = L.Conv2D(64, (3, 3), padding='same')(X1)\n X1 = L.BatchNormalization()(X1)\n X1 = L.Activation('relu')(X1) #l,b,64\n \n X0 = L.Conv2D(3, (1, 1), strides=1)(X1) #l,b,3 \n model = Model(inputs=Xinpt, outputs=X0)\n return model\n \n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"87da9c9cb545e4f7669b78f0fc643e1f9a7ac1fa"},"cell_type":"code","source":"model = UNet()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f35c328307baf2362945da0b166e6eaa3bc9656b"},"cell_type":"code","source":"model.compile('adam', loss='mean_squared_error', metrics=['mae', 'acc'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4d8a7616db0f6c4dac8d8a712264882f57129d17"},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"62f624a857741b8c5b1e4d32e6acd3a77b4f5817"},"cell_type":"code","source":"from keras.datasets import cifar10","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7dd01f3123fd3b5abd7a7f0a58ef8ae052e9642c"},"cell_type":"code","source":"train, test = cifar10.load_data()\ntrain = train[0]\ntest = test[0]\ntrain.shape, test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6e8c3ae7e045809c3b46dfef764aa82f058c9863"},"cell_type":"code","source":"train = train/255.\ntest = test/255.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"48a554b617824f698776c22bfc3429fba9edafc4"},"cell_type":"code","source":"gray_train = np.mean(train, axis=-1)\ngray_train = gray_train.reshape((*gray_train.shape, 1))\ngray_train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b5ecdc12b0faebac91fd37f9005f21641312baa0"},"cell_type":"code","source":"gray_test = np.mean(test, axis=-1)\ngray_test = gray_test.reshape((*gray_test.shape, 1))\ngray_test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6539d0d2b6f016edcaef55a8f94a642c23d95961"},"cell_type":"code","source":"train.max(), train.min(), gray_train.max(), gray_train.min()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"304281f9e4335bc012c9691fb6ce98374b08ff25"},"cell_type":"code","source":"def sample_batch(data, bsize):\n idxs = np.random.choice(np.arange(data.shape[0]), size=bsize)\n return idxs\n\ndef sample_images(data, gray, nrow):\n idx = sample_batch(gray, bsize=nrow)\n gray_batch = gray[idx]\n color_batch = data[idx]\n images = model.predict(gray_batch)\n if np.var(images)!=0:\n images = (images-images.min())/(images.max()-images.min())\n cnt=1\n for i in range(nrow):\n plt.subplot(nrow, 3, cnt)\n plt.imshow(color_batch[i].reshape((32, 32, 3)), cmap=\"gray\", interpolation='none')\n plt.subplot(nrow, 3, cnt+1)\n plt.imshow(gray_batch[i].reshape((32, 32)), cmap=\"gray\", interpolation='none')\n plt.subplot(nrow, 3, cnt+2)\n plt.imshow(images[i].reshape((32, 32, 3)), cmap=\"gray\", interpolation='none')\n cnt += 3\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"86958e24f56d9ff2c949008b7ed0e48253c51a77","scrolled":true},"cell_type":"code","source":"from IPython import display\nfor epoch in range(1, 101):\n display.clear_output(wait=True)\n print('Epoch:', epoch)\n model.fit(gray_train, train, batch_size=256, validation_data=(gray_test, test))\n sample_images(test, gray_test, 5)\n model.save('unet.hdf5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b2c734215b9247cefb4c08dcb0f3ddbbb93be80a"},"cell_type":"code","source":"model.save('unet.hdf5')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1} -------------------------------------------------------------------------------- /static/color.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/static/color.jpg -------------------------------------------------------------------------------- /static/gray.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/thevarunsharma/Image-Colorization/c64da1d9018aaf7dce15ae525e1b50302cf82fe6/static/gray.jpg -------------------------------------------------------------------------------- /templates/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | Image Colorizer 4 | 5 | 6 |

Image Colorizer

7 |
8 | 9 | 10 |
11 | {% if after %} 12 |
13 | 14 | 15 |
16 | {% endif %} 17 | 18 | 19 | -------------------------------------------------------------------------------- /utils/unet.hdf5: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:a7a95610e0887ab01a47c8b72e517205394b37a86ef4118d2e91ae30328db4d6 3 | size 372859088 4 | --------------------------------------------------------------------------------