├── README.md └── image_segmentation ├── KMeansClustering.m ├── images ├── bg.jpg ├── bg2.jpg ├── bg3.jpg ├── cheetah.jpg ├── dog.jpg ├── gecko.jpg └── zebra.jpg ├── makeLMfilters.m ├── runfile.m ├── segmentImg.m ├── segmentImgExtra.m └── transferImg.m /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/README.md -------------------------------------------------------------------------------- /image_segmentation/KMeansClustering.m: -------------------------------------------------------------------------------- 1 | function idx = KMeansClustering(X, k, centers) 2 | % Run the k-means clustering algorithm. 3 | % 4 | % INPUTS 5 | % X - An array of size m x n containing the points to cluster. Each row is 6 | % an n-dimensional point, so X(i, :) gives the coordinates of the ith 7 | % point. 8 | % k - The number of clusters to compute. 9 | % centers - OPTIONAL parameter giving initial centers for the clusters. 10 | % If provided, centers should be a k x n matrix where 11 | % centers(c, :) is the center of the cth cluster. If not provided 12 | % then cluster centers will be initialized by selecting random 13 | % rows of X. You don't need to use this parameter; it is mainly 14 | % here to make your code more easily testable. 15 | % 16 | % OUTPUTS 17 | % idx - The assignments of points to clusters. idx(i) = c means that the 18 | % point X(i, :) has been assigned to cluster c. 19 | 20 | if ~isa(X, 'double') 21 | X = double(X); 22 | end 23 | m = size(X, 1); 24 | n = size(X, 2); 25 | 26 | 27 | % If initial cluster centers were not provided then initialize cluster 28 | % centers to random rows of X. Each row of the centers variable should 29 | % contain the center of a cluster, so that centers(c, :) is the center 30 | % of the cth cluster. 31 | if ~exist('centers', 'var') 32 | centers = zeros(k, n); 33 | random_number_array = zeros([k 1]); 34 | % Indranil : take k random Centers 35 | for i = 1 : k 36 | random_number = randi(m,1); 37 | % check for distinct random numbers 38 | while (any(random_number_array(:,:) == random_number)) 39 | random_number = randi(m,1); 40 | end 41 | random_number_array(k,1) = random_number; 42 | centers(i,:) = X(random_number,:); 43 | end 44 | end 45 | 46 | % The assignments of points to clusters. If idx(i) == c then the point 47 | % X(i, :) belongs to the cth cluster. 48 | idx = zeros(m, 1); 49 | 50 | % The number of iterations that we have performed. 51 | iter = 0; 52 | 53 | % If the assignments of points to clusters have not converged after 54 | % performing MAX_ITER iterations then we will break and just return the 55 | % current cluster assignments. 56 | MAX_ITER = 100; 57 | 58 | while true 59 | % Store old cluster assignments 60 | old_idx = idx; 61 | 62 | % Compute distances from each point to the centers and assign each 63 | % point to the closest cluster. 64 | for i = 1 : m 65 | min_dist = inf; 66 | for j = 1 : k 67 | distance = sqrt(sum((X(i,:) - centers(j,:)) .^ 2)); % EUCLIDEAN DISTANCE 68 | if (distance < min_dist) 69 | min_dist = distance; 70 | idx(i,:) = j; 71 | end 72 | end 73 | end 74 | 75 | % Break if cluster assignments didn't change 76 | if idx == old_idx 77 | break; 78 | end 79 | 80 | % Update the cluster centers 81 | % Indranil : get the average of all points in the same cluster and make 82 | % that the center 83 | for i = 1 : k 84 | temp_X = zeros(1,n); 85 | count = 0; 86 | for j = 1 : m 87 | if (idx(j) == i) 88 | temp_X = temp_X + X(j,:); 89 | count = count + 1; 90 | end 91 | centers(i,:) = temp_X./count; 92 | end 93 | end 94 | 95 | % Stop early if we have performed more than MAX_ITER iterations 96 | iter = iter + 1; 97 | if iter > MAX_ITER 98 | break; 99 | end 100 | end 101 | end 102 | -------------------------------------------------------------------------------- /image_segmentation/images/bg.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/bg.jpg -------------------------------------------------------------------------------- /image_segmentation/images/bg2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/bg2.jpg -------------------------------------------------------------------------------- /image_segmentation/images/bg3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/bg3.jpg -------------------------------------------------------------------------------- /image_segmentation/images/cheetah.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/cheetah.jpg -------------------------------------------------------------------------------- /image_segmentation/images/dog.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/dog.jpg -------------------------------------------------------------------------------- /image_segmentation/images/gecko.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/gecko.jpg -------------------------------------------------------------------------------- /image_segmentation/images/zebra.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ideb1988/Image_Segmentation/146b8a30ec40638140fe069045c074ac672432da/image_segmentation/images/zebra.jpg -------------------------------------------------------------------------------- /image_segmentation/makeLMfilters.m: -------------------------------------------------------------------------------- 1 | function F=makeLMfilters 2 | % Returns the LML filter bank of size 49x49x48 in F. To convolve an 3 | % image I with the filter bank you can either use the matlab function 4 | % conv2, i.e. responses(:,:,i)=conv2(I,F(:,:,i),'valid'), or use the 5 | % Fourier transform. 6 | 7 | SUP=49; % Support of the largest filter (must be odd) 8 | SCALEX=sqrt(2).^[1:3]; % Sigma_{x} for the oriented filters 9 | %SCALEX=[1 sqrt(2) 2 2*sqrt(2)]; 10 | NORIENT=6; % Number of orientations 11 | 12 | NROTINV=12; 13 | NBAR=length(SCALEX)*NORIENT; 14 | NEDGE=length(SCALEX)*NORIENT; 15 | NF=NBAR+NEDGE+NROTINV; 16 | F=zeros(SUP,SUP,NF); 17 | hsup=(SUP-1)/2; 18 | [x,y]=meshgrid([-hsup:hsup],[hsup:-1:-hsup]); 19 | orgpts=[x(:) y(:)]'; 20 | 21 | count=1; 22 | for scale=1:length(SCALEX), 23 | for orient=0:NORIENT-1, 24 | angle=pi*orient/NORIENT; % Not 2pi as filters have symmetry 25 | c=cos(angle);s=sin(angle); 26 | rotpts=[c -s;s c]*orgpts; 27 | F(:,:,count)=makefilter(SCALEX(scale),0,1,rotpts,SUP); 28 | F(:,:,count+NEDGE)=makefilter(SCALEX(scale),0,2,rotpts,SUP); 29 | count=count+1; 30 | end; 31 | end; 32 | 33 | count=NBAR+NEDGE+1; 34 | SCALES=sqrt(2).^[1:4]; 35 | for i=1:length(SCALES), 36 | F(:,:,count)=normalise(fspecial('gaussian',SUP,SCALES(i))); 37 | F(:,:,count+1)=normalise(fspecial('log',SUP,SCALES(i))); 38 | F(:,:,count+2)=normalise(fspecial('log',SUP,3*SCALES(i))); 39 | count=count+3; 40 | end; 41 | return 42 | 43 | function f=makefilter(scale,phasex,phasey,pts,sup) 44 | gx=gauss1d(3*scale,0,pts(1,:),phasex); 45 | gy=gauss1d(scale,0,pts(2,:),phasey); 46 | f=normalise(reshape(gx.*gy,sup,sup)); 47 | return 48 | 49 | function g=gauss1d(sigma,mean,x,ord) 50 | % Function to compute gaussian derivatives of order 0 <= ord < 3 51 | % evaluated at x. 52 | 53 | x=x-mean;num=x.*x; 54 | variance=sigma^2; 55 | denom=2*variance; 56 | g=exp(-num/denom)/(pi*denom)^0.5; 57 | switch ord, 58 | case 1, g=-g.*(x/variance); 59 | case 2, g=g.*((num-variance)/(variance^2)); 60 | end; 61 | return 62 | 63 | function f=normalise(f), f=f-mean(f(:)); f=f/sum(abs(f(:))); return -------------------------------------------------------------------------------- /image_segmentation/runfile.m: -------------------------------------------------------------------------------- 1 | % The runfile for ASSIGNMENT 2 2 | % Indranil Deb 50097062 3 | 4 | function [] = runfile(~) 5 | 6 | extracredit = true; % set false to use segmentImg, true uses segmentImgExtra 7 | 8 | % PLEASE SET WEIGHT TO 2 for the cheetah, 20 for gecko and 5 for the others 9 | weight = 20; 10 | % READ IN THE IMAGE TO BE INSERTED 11 | foreground_from_image = 'gecko.jpg'; 12 | % READ IN THE BACKGROUND 13 | background_image = 'bg2.jpg'; 14 | % SET THE VALUE OF k 15 | k = 2; 16 | 17 | % CHANGE TO THE IMAGES FOLDER 18 | main = cd('images'); 19 | sImg = imread(foreground_from_image); 20 | tImg = imread(background_image); 21 | % RETURN TO MAIN FOLDER 22 | cd(main); 23 | 24 | % SET FGS TO ALL POSSIBLE SEGMENTS (WE WILL CALCULATE THE FOREGROUND 25 | % SEGMENTS IN transferImg FUNCTION 26 | fgs = linspace(1,k,k); 27 | 28 | % CALCULATE idx 29 | if (extracredit == false) 30 | idx = segmentImg(sImg, k); 31 | else 32 | idx = segmentImgExtra(sImg, k, weight); 33 | end 34 | 35 | % USE THE transferImg FUNCTION TO GET THE FINAL OUTPUT 36 | newImg = transferImg(fgs, idx, sImg, tImg); 37 | 38 | % SHOW THE FINAL OUTPUT 39 | figure, imshow(newImg); 40 | end -------------------------------------------------------------------------------- /image_segmentation/segmentImg.m: -------------------------------------------------------------------------------- 1 | function idx = segmentImg(I, k) 2 | % function idx = segmentImage(img) 3 | % Returns the logical image containing the segment ids obtained from 4 | % segmenting the input image 5 | % 6 | % INPUTS 7 | % I - The input image contining textured foreground objects to be segmented 8 | % out. 9 | % k - The number of segments to compute (also the k-means parameter). 10 | % 11 | % OUTPUTS 12 | % idx - The logical image (same dimensions as the input image) contining 13 | % the segment ids after segmentation. The maximum value of idx is k. 14 | % 15 | 16 | % 1. Create your bank of filters using the given alogrithm; 17 | % 2. Compute the filter responses by convolving your input image with 18 | % each of the num_filters in the bank of filters F. 19 | % responses(:,:,i)=conv2(I,F(:,:,i),'same') 20 | % NOTE: we suggest to use 'same' instead of 'full' or 'valid'. 21 | % 3. Remember to take the absolute value of the filter responses (no 22 | % negative values should be used). 23 | % 4. Construct a matrix X of the points to be clustered, where 24 | % the rows of X = the total number of pixels in I (rows*cols); and 25 | % the columns of X = num_filters; 26 | % i.e. each pixel is transformed into a num_filters-dimensional 27 | % vector. 28 | % 5. Run kmeans to cluster the pixel features into k clusters, 29 | % returning a vector IDX of labels. 30 | % 6. Reshape IDX into an image with same dimensionality as I and return 31 | % the reshaped index image. 32 | % 33 | 34 | % PROCESSING OF IMAGE 35 | image = double(rgb2gray(I)); 36 | image=image(:,:,1); 37 | s1 = size(image,1); 38 | s2 = size(image,2); 39 | 40 | % GET THE BANK OF FILTERS 41 | F = makeLMfilters; 42 | num_filters = size(F, 3); 43 | 44 | % Compute the filter responses by convolving your input image with 45 | % each of the num_filters in the bank of filters F. 46 | responses = zeros(s1, s2, num_filters); 47 | for i = 1 : num_filters; 48 | responses(:,:,i) = conv2(image,F(:,:,i),'same'); 49 | end 50 | 51 | % Indranil : Get the absolute of responses 52 | responses = abs(responses); 53 | 54 | % Construct a matrix X of the points to be clustered, where 55 | % the rows of X = the total number of pixels in I (rows*cols); and 56 | % the columns of X = num_filters; 57 | % i.e. each pixel is transformed into a num_filters-dimensional 58 | % vector. 59 | 60 | % Flatten the responses 61 | X = reshape(responses, (s1*s2), num_filters); 62 | 63 | % Use my own clustering algorithm 64 | idx = KMeansClustering(X, k); 65 | 66 | % Reshape IDX into an image with same dimensionality as I and return 67 | % the reshaped index image. 68 | idx = reshape(idx, s1, s2); 69 | 70 | end 71 | -------------------------------------------------------------------------------- /image_segmentation/segmentImgExtra.m: -------------------------------------------------------------------------------- 1 | function idx = segmentImgExtra(I, k, weight) 2 | % function idx = segmentImage(img) 3 | % Returns the logical image containing the segment ids obtained from 4 | % segmenting the input image 5 | % 6 | % INPUTS 7 | % I - The input image contining textured foreground objects to be segmented 8 | % out. 9 | % k - The number of segments to compute (also the k-means parameter). 10 | % 11 | % OUTPUTS 12 | % idx - The logical image (same dimensions as the input image) contining 13 | % the segment ids after segmentation. The maximum value of idx is k. 14 | % 15 | 16 | % 1. Create your bank of filters using the given alogrithm; 17 | % 2. Compute the filter responses by convolving your input image with 18 | % each of the num_filters in the bank of filters F. 19 | % responses(:,:,i)=conv2(I,F(:,:,i),'same') 20 | % NOTE: we suggest to use 'same' instead of 'full' or 'valid'. 21 | % 3. Remember to take the absolute value of the filter responses (no 22 | % negative values should be used). 23 | % 4. Construct a matrix X of the points to be clustered, where 24 | % the rows of X = the total number of pixels in I (rows*cols); and 25 | % the columns of X = num_filters; 26 | % i.e. each pixel is transformed into a num_filters-dimensional 27 | % vector. 28 | % 5. Run kmeans to cluster the pixel features into k clusters, 29 | % returning a vector IDX of labels. 30 | % 6. Reshape IDX into an image with same dimensionality as I and return 31 | % the reshaped index image. 32 | % 33 | 34 | 35 | % PROCESSING OF IMAGE 36 | image = double(rgb2gray(I)); 37 | image=image(:,:,1); 38 | s1 = size(image,1); 39 | s2 = size(image,2); 40 | 41 | % GET THE BANK OF FILTERS 42 | F = makeLMfilters; 43 | num_filters = size(F, 3); 44 | 45 | % Compute the filter responses by convolving your input image with 46 | % each of the num_filters in the bank of filters F. 47 | responses = zeros(s1, s2, num_filters); 48 | for i = 1 : num_filters; 49 | responses(:,:,i) = conv2(image,F(:,:,i),'same'); 50 | end 51 | 52 | % Indranil : Get the absolute of responses 53 | responses = abs(responses); 54 | 55 | % Construct a matrix X of the points to be clustered, where 56 | % the rows of X = the total number of pixels in I (rows*cols); and 57 | % the columns of X = num_filters; 58 | % i.e. each pixel is transformed into a num_filters-dimensional 59 | % vector. 60 | 61 | % Additional for extra credit : colour based idx from 62 | % Convert RGB image to L*a*b* 63 | image_cform = makecform('srgb2lab'); 64 | responses_2 = applycform(I,image_cform); 65 | responses_2 = double(responses_2(:,:,2:3)); 66 | responses_2 = weight*responses_2; 67 | 68 | % Flatten the responses 69 | X = reshape(responses, (s1*s2), num_filters); 70 | X2 = reshape(responses_2, (s1*s2), 2); 71 | 72 | % CONCAT THE WEIGHTED COLOR RESPONSES TO THE TEXTURE FILTER RESPONSES 73 | X = horzcat(X,X2); 74 | 75 | % Use my own clustering algorithm kmeans 76 | idx = KMeansClustering(X, k); 77 | 78 | % Reshape IDX into an image with same dimensionality as I and return 79 | % the reshaped index image. 80 | idx = reshape(idx, s1, s2); 81 | 82 | end 83 | -------------------------------------------------------------------------------- /image_segmentation/transferImg.m: -------------------------------------------------------------------------------- 1 | function newImg = transferImg(fgs, idx, sImg, tImg) 2 | % function newimg = transferImg(fgs, idx, sImg, tImg) 3 | %fgs - array of foreground segments e.g [1 2 4 5] 4 | %idx - logical image containing indexes of segments (same size as sImg) 5 | %sImg - source image where object was segmented from (color) 6 | %tImg - target image where object will be transferred (color) 7 | 8 | %First crop sImg and idx to remove border problems 9 | [rows, cols,~]=size(sImg); 10 | sImg2= sImg(25:rows-25,25:cols-25,:); 11 | idx2 = idx(25:rows-25,25:cols-25); 12 | 13 | %These implementations are somewhat hardcoded to match 14 | % the images selected for the class assignment. You can 15 | % do more clever things with the resizing before transfer 16 | sImg2 = imresize(sImg2, 0.5, 'nearest'); 17 | idx2 = imresize(idx2, 0.5, 'nearest'); 18 | 19 | % Indranil : Calculate the no of pixels in the borders of each segment 20 | no_of_1_in_border = zeros(size(fgs)); 21 | for i=1:length(fgs) 22 | no_of_1_in_border(i) = no_in_borders(idx2,fgs(i)); 23 | end 24 | 25 | %Then create a mask of sImg with foreground segments 26 | [rows, cols,~]=size(sImg2); 27 | mask = false(rows, cols); 28 | 29 | for i=1:length(fgs) 30 | % Indranil : Do not take the mask with the highest number of pixels in the border 31 | if (no_of_1_in_border(i) == max(no_of_1_in_border)) 32 | mask(idx2==fgs(i))=false; 33 | else 34 | mask(idx2==fgs(i))=true; 35 | end 36 | end 37 | 38 | %Now embed the initial mask into a new mask that 39 | %is the same size as the target image 40 | [row_t, col_t,~]=size(tImg); 41 | newmask = false(row_t,col_t); 42 | start_r = floor(row_t*.50); 43 | start_c = floor(col_t*.25); 44 | newmask(start_r:start_r+rows-1, start_c:start_c+cols-1) = mask; 45 | 46 | %We must also embed the original image to match the new mask 47 | newimg = uint8(zeros(size(tImg))); 48 | newimg(start_r:start_r+rows-1, start_c:start_c+cols-1,:) = sImg2; 49 | 50 | for channel = 1:size(tImg, 3) 51 | sImgChannel = newimg(:, :, channel); 52 | bgChannel = tImg(:, :, channel); 53 | bgChannel(newmask) = sImgChannel(newmask); 54 | tImg(:, :, channel) = bgChannel; 55 | end 56 | newImg = tImg; 57 | 58 | end 59 | 60 | % Indranil: Calculate the no of pixels in the borders of each segment 61 | 62 | function [pixels_in_border] = no_in_borders(idx2,segment) 63 | 64 | pixels_in_border = 0; 65 | temp = (idx2==segment); 66 | [m n] = size(temp); 67 | 68 | for i = 1:n 69 | if (temp(1,i) == 1) 70 | pixels_in_border = pixels_in_border + 1; 71 | end 72 | if (temp(m,i) == 1) 73 | pixels_in_border = pixels_in_border + 1; 74 | end 75 | end 76 | 77 | for i = 1:m 78 | if (temp(i,1) == 1) 79 | pixels_in_border = pixels_in_border + 1; 80 | end 81 | if (temp(i,n) == 1) 82 | pixels_in_border = pixels_in_border + 1; 83 | end 84 | end 85 | 86 | end 87 | 88 | 89 | --------------------------------------------------------------------------------