├── README ├── README.md ├── images ├── color_source │ ├── 1.jpg │ ├── 2.jpg │ ├── 3.jpg │ ├── 4.jpg │ ├── 5.jpg │ └── 6.jpg ├── gray_source │ ├── 1.jpg │ ├── 2.jpg │ ├── 3.jpg │ ├── 4.jpg │ ├── 5.jpg │ └── 6.jpg └── target │ ├── 1.jpg │ ├── 2.jpg │ ├── 3.jpg │ ├── 4.jpg │ ├── 5.jpg │ └── 6.jpg ├── output ├── 1.jpg ├── 2.jpg ├── bf_1.jpg ├── bf_2.jpg ├── bf_4.jpg ├── jitter_1.jpg ├── jitter_2.jpg ├── jitter_3.jpg ├── jitter_4.jpg ├── jitter_5.jpg ├── jitter_6.jpg ├── overlay_1.jpg └── overlay_2.jpg └── src ├── compute_best_match.m ├── compute_fv.m ├── image_colorization_brute_force.m ├── image_colorization_jitter_sampling.m ├── image_colorization_pyramid.m ├── initialize.m ├── load_images.m ├── luminance_remap.m ├── main.m ├── sampling_jittered.m ├── save_image.m ├── sd_neighborhood.m ├── transfer_fv.m └── transfer_sample_fv.m /README: -------------------------------------------------------------------------------- 1 | ================================= 2 | = README = 3 | ================================= 4 | 5 | Link to my presentation: https://www.youtube.com/embed/2OS8EFP9fng 6 | 7 | The file structure currently contains three main folders that looks like: 8 | index.html #webpage 9 | README 10 | |--src/ #where all the matlab code is in. main.m is in here. 11 | |--images/ #input images 12 | |--output/ # output images 13 | 14 | Instructions: 15 | In order to run all my code, I have a main.m which is currently configured with boolean variables toggled to true to run each individual part. This is located in my "src" folder. Everything should be run from the src folder. In the src folder, 16 | I currently have 1 "main" file, initialize.m which will do most of the "heavy-lifting." 17 | 18 | The file has boolean variables at the beginning which you can toggle. 19 | The two main variables are SAVE, GRAPH 20 | It is currently toggled to be: 21 | SAVE = false; # when toggled true, it will save to the corresponding output folders. 22 | GRAPH = false; 23 | 24 | 25 | ======== 26 | main.m 27 | ======== 28 | Run main.m which will run through each of the files listed above earlier. 29 | 30 | ============ 31 | initialize.m 32 | ============ 33 | Runs through the code where you can choose to use brute-force or jitter sampling methods 34 | 35 | ================= 36 | luminance_remap.m 37 | ================= 38 | Remaps luminance values from A to B - used formula 39 | 40 | =========================== 41 | Files for naive/brute-force 42 | =========================== 43 | image_colorization_brute_force.m - main file for doing brute-force method 44 | transform_fv.m - used to transfer AB channel values after finding best match 45 | compute_fv.m - concatenates fvs used 46 | sd_neighborhood.m - calculates std of each pixel in image around a certain neighborhood 47 | 48 | =========================== 49 | Files for jitter sampling 50 | =========================== 51 | image_colorization_jitter_sampling.m - main file for doing jitter sampling 52 | transfer_sample_fv.m - used to transfer AB channel values after finding best match 53 | 54 | ============================ 55 | Helper files for misc things 56 | ============================ 57 | load_images.m # loads appropriate images 58 | save_image.m # saves images 59 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Transferring Color to Greyscale Images 2 | 3 | This repository contains an attempt to imitate the results from: 4 | [http://dl.acm.org/citation.cfm?id=566576](http://dl.acm.org/citation.cfm?id=566576) 5 | `http://dl.acm.org/citation.cfm?id=566576` 6 | By Tomihisa Welsh, Michael Ashikhmin, Klaus Mueller 7 | 8 | ## Examples 9 | 10 | ### Color Source, Target, and Colorized Image 11 | ![Color source] (https://github.com/h-wang94/ImageColorization/blob/master/images/color_source/1.jpg) 12 | ![Target] (https://github.com/h-wang94/ImageColorization/blob/master/images/target/1.jpg) 13 | ![Colorized](https://github.com/h-wang94/ImageColorization/blob/master/output/1.jpg?raw=true) 14 | 15 | ### Color Source, Target, and Colorized Image 16 | ![Color_source] (https://raw.githubusercontent.com/h-wang94/ImageColorization/master/images/color_source/2.jpg) 17 | ![Target](https://github.com/h-wang94/ImageColorization/blob/master/images/target/2.jpg) 18 | ![Colorized](https://github.com/h-wang94/ImageColorization/blob/master/output/2.jpg?raw=true) 19 | -------------------------------------------------------------------------------- /images/color_source/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/1.jpg -------------------------------------------------------------------------------- /images/color_source/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/2.jpg -------------------------------------------------------------------------------- /images/color_source/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/3.jpg -------------------------------------------------------------------------------- /images/color_source/4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/4.jpg -------------------------------------------------------------------------------- /images/color_source/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/5.jpg -------------------------------------------------------------------------------- /images/color_source/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/color_source/6.jpg -------------------------------------------------------------------------------- /images/gray_source/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/1.jpg -------------------------------------------------------------------------------- /images/gray_source/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/2.jpg -------------------------------------------------------------------------------- /images/gray_source/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/3.jpg -------------------------------------------------------------------------------- /images/gray_source/4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/4.jpg -------------------------------------------------------------------------------- /images/gray_source/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/5.jpg -------------------------------------------------------------------------------- /images/gray_source/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/gray_source/6.jpg -------------------------------------------------------------------------------- /images/target/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/1.jpg -------------------------------------------------------------------------------- /images/target/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/2.jpg -------------------------------------------------------------------------------- /images/target/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/3.jpg -------------------------------------------------------------------------------- /images/target/4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/4.jpg -------------------------------------------------------------------------------- /images/target/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/5.jpg -------------------------------------------------------------------------------- /images/target/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/images/target/6.jpg -------------------------------------------------------------------------------- /output/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/1.jpg -------------------------------------------------------------------------------- /output/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/2.jpg -------------------------------------------------------------------------------- /output/bf_1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/bf_1.jpg -------------------------------------------------------------------------------- /output/bf_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/bf_2.jpg -------------------------------------------------------------------------------- /output/bf_4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/bf_4.jpg -------------------------------------------------------------------------------- /output/jitter_1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_1.jpg -------------------------------------------------------------------------------- /output/jitter_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_2.jpg -------------------------------------------------------------------------------- /output/jitter_3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_3.jpg -------------------------------------------------------------------------------- /output/jitter_4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_4.jpg -------------------------------------------------------------------------------- /output/jitter_5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_5.jpg -------------------------------------------------------------------------------- /output/jitter_6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/jitter_6.jpg -------------------------------------------------------------------------------- /output/overlay_1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/overlay_1.jpg -------------------------------------------------------------------------------- /output/overlay_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/h-wang94/ImageColorization/11b7e0b47d279eae0e1fbab04e31ad71acbb5829/output/overlay_2.jpg -------------------------------------------------------------------------------- /src/compute_best_match.m: -------------------------------------------------------------------------------- 1 | function best_match = compute_best_match(target_value, source) 2 | [y, x] = size(source); 3 | enlarged_target = repmat(target_value, y, 1); 4 | square_diff = (enlarged_target - source).^2; 5 | weight_1 = 0.5; 6 | weight_2 = 1-weight_1; 7 | weighted_sum = weight_1 * square_diff(:, 1) + weight_2 * square_diff(:, 2); 8 | [~, best_match] = min(weighted_sum); 9 | end -------------------------------------------------------------------------------- /src/compute_fv.m: -------------------------------------------------------------------------------- 1 | function fv = compute_fv(mat_struct) 2 | % currently use sds and luminance as FV; 3 | fv = [mat_struct.luminance(:), mat_struct.sds(:)]; 4 | end -------------------------------------------------------------------------------- /src/image_colorization_brute_force.m: -------------------------------------------------------------------------------- 1 | function transferred = image_colorization_brute_force(target, gsource, csource) 2 | %% get std in neighborhood 3 | csource.sds = sd_neighborhood(csource.luminance, 5); 4 | gsource.sds = sd_neighborhood(gsource.luminance, 5); 5 | target.sds = sd_neighborhood(target.luminance, 5); 6 | %% 7 | csource.fv = compute_fv(csource); 8 | gsource.fv = compute_fv(gsource); 9 | target.fv = compute_fv(target); 10 | 11 | transferred = transfer_fv(csource, gsource, target); 12 | end -------------------------------------------------------------------------------- /src/image_colorization_jitter_sampling.m: -------------------------------------------------------------------------------- 1 | function transferred = image_colorization_jitter_sampling(target, csource, GRAPH) 2 | jittered = sampling_jittered(csource.luminance, csource.lab, 256); 3 | if GRAPH 4 | imshow(csource.image); 5 | hold on; 6 | scatter(jittered.points(:,2), jittered.points(:,1), 'r'); 7 | end 8 | target.sds = sd_neighborhood(target.luminance, 5); 9 | target.fv = compute_fv(target); 10 | transferred = transfer_sample_fv(jittered, target); 11 | end -------------------------------------------------------------------------------- /src/image_colorization_pyramid.m: -------------------------------------------------------------------------------- 1 | function transferred = image_colorization_pyramid(target, gsource, csource) 2 | %% 3 | csource.gpyramid = create_gauss_pyramid(csource.image); 4 | gsource.gpyramid = create_gauss_pyramid(gsource.image); 5 | target.gpyramid = create_gauss_pyramid(target.image); 6 | 7 | 8 | end -------------------------------------------------------------------------------- /src/initialize.m: -------------------------------------------------------------------------------- 1 | %% 2 | clc; clear all; 3 | %% 4 | tic; 5 | GRAPH = false; 6 | SAVE = false; 7 | OPTION = 0; % 0 for brute-force, 1, for jitter-sampling 8 | 9 | %% load images 10 | img_name = '8.jpg'; 11 | imgs = load_images(img_name); 12 | target = {}; 13 | gsource = {}; 14 | csource = {}; 15 | target.image = imgs.target_image; 16 | gsource.image = imgs.gsource_image; 17 | csource.image = imgs.csource_image; 18 | tic; 19 | %% to LAB color space 20 | csource.lab = rgb2lab(csource.image); 21 | if ndims(gsource.image) == 3 22 | gsource.lab = rgb2lab(gsource.image); 23 | else 24 | gsource.lab = gsource.image; 25 | end 26 | if ndims(target.image) == 3 27 | target.lab = rgb2gray(target.image); 28 | else 29 | target.lab = target.image; 30 | end 31 | 32 | %% map luminance to target luminance 33 | csource.luminance = luminance_remap(csource.lab, target.lab); 34 | gsource.luminance = luminance_remap(gsource.lab, target.lab); 35 | target.luminance = target.lab; 36 | % pixel values are luminance 37 | 38 | %% 39 | if OPTION == 0 40 | transferred = image_colorization_brute_force(target, gsource, csource); 41 | new_name = strcat('bf_', img_name); 42 | elseif OPTION == 1 43 | transferred = image_colorization_jitter_sampling(target, csource, GRAPH); 44 | new_name = strcat('jitter_', img_name); 45 | end 46 | %% 47 | new_image = lab2rgb(transferred); 48 | 49 | toc; 50 | %% 51 | imshow(new_image); 52 | %% save images 53 | success = save_image(new_image, new_name, SAVE); 54 | toc; -------------------------------------------------------------------------------- /src/load_images.m: -------------------------------------------------------------------------------- 1 | function imgs = load_images(img_name, images_folder) 2 | %% load images 3 | if nargin < 2 4 | images_folder = '../images/'; 5 | end 6 | target_images_folder = strcat(images_folder, 'target/'); 7 | gsource_images_folder = strcat(images_folder, 'gray_source/'); 8 | csource_images_folder = strcat(images_folder, 'color_source/'); 9 | 10 | 11 | target_image_name = strcat(target_images_folder, img_name); 12 | gsource_image_name = strcat(gsource_images_folder, img_name); 13 | csource_image_name = strcat(csource_images_folder, img_name); 14 | target_image = im2double(imread(target_image_name)); 15 | gsource_image = im2double(imread(gsource_image_name)); 16 | csource_image = im2double(imread(csource_image_name)); 17 | imgs.target_image = target_image; 18 | imgs.gsource_image = gsource_image; 19 | imgs.csource_image = csource_image; 20 | end 21 | -------------------------------------------------------------------------------- /src/luminance_remap.m: -------------------------------------------------------------------------------- 1 | function remapped = luminance_remap(source, target) 2 | % assumes that the source is in LAB format ranging from 0-100 3 | % assumes that target is a grayscale image ranging from 0-1 4 | source_luminance = source(:,:,1) / 100; 5 | mu_a = mean(source_luminance(:)); 6 | mu_b = mean(target(:)); 7 | sigma_a = std(source_luminance(:)); 8 | sigma_b = std(target(:)); 9 | remapped = sigma_b / sigma_a * (source_luminance - mu_a) + mu_b; 10 | %source(:,:,1) = new_luminance; 11 | %remapped = source; 12 | end -------------------------------------------------------------------------------- /src/main.m: -------------------------------------------------------------------------------- 1 | % default to run through everything 2 | 3 | run_colorization = true; 4 | 5 | if run_colorization == true 6 | disp('Running Image Colorization'); 7 | run('initialize.m'); 8 | else 9 | disp('NOT running run_colorization. Edit main.m to toggle'); 10 | end 11 | 12 | disp('Done with script!'); 13 | -------------------------------------------------------------------------------- /src/sampling_jittered.m: -------------------------------------------------------------------------------- 1 | function jittered = sampling_jittered(image_luminance, image_lab, num_samples) 2 | % assumes num_samples is square number. num_samples = n^2 where n is an 3 | % integer 4 | fv = {}; 5 | jittered = {}; 6 | n = sqrt(num_samples); 7 | [y, x] = size(image_luminance); 8 | y_step = floor(y / n); 9 | x_step = floor(x / n); 10 | sample_idx = randi([1, y_step*x_step], 1, num_samples*2); 11 | fv.sds = zeros([1,num_samples]); 12 | fv.luminance = zeros([1, num_samples]); 13 | fv.ab = zeros([num_samples, 2]); 14 | jittered.points = zeros([num_samples, 2]); 15 | idx = 1; 16 | amt_to_pad = 2; 17 | padded = padarray(image_luminance, [amt_to_pad, amt_to_pad]); 18 | for j = 1:x_step:x-x_step-1 19 | for i = 1:y_step:y-y_step-1 20 | value = sample_idx(idx); 21 | % determines index inside block 22 | new_j = ceil(value / y_step); 23 | new_i = value - (new_j-1)*y_step; 24 | 25 | actual_i = i+new_i-1; 26 | actual_j = j+new_j-1; 27 | actual_pixel = image_luminance(actual_i, actual_j); 28 | ab = image_lab(actual_i, actual_j, 2:3); 29 | block = padded(actual_i:actual_i+amt_to_pad*2, actual_j:actual_j+amt_to_pad*2); 30 | fv.sds(idx) = std(block(:)); 31 | fv.luminance(idx) = actual_pixel(:, :); 32 | fv.ab(idx, :) = ab; 33 | jittered.points(idx, :) = [actual_i, actual_j]; 34 | idx = idx + 1; 35 | 36 | end 37 | end 38 | jittered.fv = [fv.luminance(:), fv.sds(:)]; 39 | jittered.ab = fv.ab; 40 | end -------------------------------------------------------------------------------- /src/save_image.m: -------------------------------------------------------------------------------- 1 | function success = save_image(img, img_name, SAVE, output_folder) 2 | if nargin < 3 3 | SAVE = true; 4 | output_folder = '../output/'; 5 | elseif nargin < 4 6 | output_folder = '../output/'; 7 | end 8 | 9 | success = false; 10 | if SAVE 11 | imwrite(img, strcat(output_folder, img_name)); 12 | success = true; 13 | end 14 | end -------------------------------------------------------------------------------- /src/sd_neighborhood.m: -------------------------------------------------------------------------------- 1 | function sds = sd_neighborhood(image, neighborhood_size) 2 | if nargin < 2 3 | neighborhood_size = 5; 4 | end 5 | amt_to_pad = (neighborhood_size - 1) / 2; 6 | [y, x] = size(image); 7 | sds = zeros([y, x]); 8 | padded = padarray(image, [amt_to_pad, amt_to_pad]); 9 | for i = amt_to_pad+1:y+2 10 | for j = amt_to_pad+1:x+2 11 | region = padded(i-2:i+2, j-2:j+2); 12 | sd = std(region(:)); 13 | sds(i-2, j-2) = sd; 14 | end 15 | end 16 | end -------------------------------------------------------------------------------- /src/transfer_fv.m: -------------------------------------------------------------------------------- 1 | function transferred = transfer_fv(csource, gsource, target) 2 | target_fv = target.fv; 3 | gsource_fv = gsource.fv; 4 | transferred = zeros(size(target.image)); 5 | transferred(:, :, 1) = target.lab; 6 | [c_y, c_x, ~] = size(csource.image); 7 | num_fv = size(target_fv, 2)-1; 8 | for j = 1:c_x 9 | for i = 1:c_y 10 | idx = (i-1) + (j-1)*c_y + 1; 11 | best_match = compute_best_match(target_fv(idx, :), gsource_fv); 12 | new_j = ceil(best_match/c_y); 13 | new_i = best_match - (new_j-1)*c_y; 14 | transferred(i, j, 2:3) = csource.lab(new_i, new_j, 2:3); 15 | end 16 | end 17 | transferred(:,:,1) = transferred(:,:,1) * 100; 18 | end -------------------------------------------------------------------------------- /src/transfer_sample_fv.m: -------------------------------------------------------------------------------- 1 | function transferred = transfer_sample_fv(jittered, target) 2 | target_fv = target.fv; 3 | transferred = zeros(size(target.image)); 4 | transferred(:, :, 1) = target.lab; 5 | [c_y, c_x] = size(target.lab); 6 | for j = 1:c_x 7 | for i = 1:c_y 8 | idx = (i-1) + (j-1)*c_y + 1; 9 | best_match = compute_best_match(target_fv(idx, :), jittered.fv); 10 | transferred(i, j, 2:3) = jittered.ab(best_match, :); 11 | end 12 | end 13 | transferred(:,:,1) = transferred(:,:,1) * 100; 14 | end --------------------------------------------------------------------------------