├── ChannelNorm.m ├── Demo.m ├── GradientFusion.m ├── LICENSE ├── README.md ├── ReconstructGradient.m ├── descriptions └── ReadMe.pdf ├── getFusedGradients.m └── getGradientH.m /ChannelNorm.m: -------------------------------------------------------------------------------- 1 | function J = ChannelNorm(I,r) 2 | % function ChannelNorm.m 3 | % ------------------------------------------------------------------------- 4 | % This function normalizes the image I to a required range using a 5 | % non-linear transformation 6 | % 7 | % Inputs: 8 | % I = the image (two dimensional array, e.g. an image in grayscale 9 | % representation or one channel in a multi-channel image 10 | % in RGB or YCbCr representaion) 11 | % r = a vector with two elements representing the minimum and 12 | % maximum values of the range onto which the image range has to 13 | % be projected; r = [minimum_value maximum_value]; 14 | % Output: 15 | % J = the image I projected to the desired range r 16 | % 17 | % Written by : Sujoy Paul, Jadavpur University, 2014 18 | % At : University of Victoria, Canada 19 | % 20 | % Modified by: Ioana Sevcenco, University of Victoria, Canada 21 | % 22 | % Last updated: November 19, 2014 23 | Ran = range(r); 24 | L = range(I(:)); 25 | gam = log(Ran)/log(L); 26 | J = Ran*((I-min(I(:)))./(L)).^gam+r(1); 27 | end -------------------------------------------------------------------------------- /Demo.m: -------------------------------------------------------------------------------- 1 | %% Fusion of Multi-exposure and Multi-focus images 2 | % 3 | % Version 1.0 4 | % S. Paul(1), I. Sevcenco(2), P. Agathoklis(2) 5 | % (1) Department of Eletronics and Telecommunication Engineerin, 6 | % Jadavpur University, Kolkata, India 7 | % (2) Department of Electrical and Computer Engineering 8 | % University of Victoria, Victoria, B.C., Canada 9 | % 10 | % DESCRIPTION of the Code: 11 | % This is a demo file for executing the fusion of an arbitrary number 12 | % of grayscale and color images. 13 | % 14 | % The code can be used to fuse multi-exposure or multi-focus images. 15 | % The resulting image will have an increased amount of information 16 | % than the source images. 17 | % 18 | % NOTE: The images to be fused have to be all of the same type 19 | % (i.e., all in grayscale or all in RGB representation; all 20 | % multi exposure or all multi focus). 21 | % 22 | % GUIDELINES for running the Code: 23 | % The input images should be named as ImageName_ImageNumber and should 24 | % be kept in the same folder as the codes for image fusion. 25 | % 26 | % For example: An input image stack named 'office' containing 5 images 27 | % should be named 'office_1', 'office_2', .., 'office_5' 28 | % The 'office' image stack is part of the MATLAB distribution. 29 | % 30 | % Specifications to be mentioned in the code: 31 | % NameImg : A generic name for the set of input images to be fused 32 | % NumberOfImages : Number of images in the input images stack 33 | % Format : Format of the input images. 34 | % 35 | % Written by : Sujoy Paul, Jadavpur University, 2014 36 | % At : University of Victoria, Canada 37 | % Modified by: Ioana Sevcenco, University of Victoria, Canada 38 | % Last updated: Dec 12, 2017 39 | % 40 | % REFERENCES: 41 | % 42 | % [1] S. Paul, I.S. Sevcenco, P. Agathoklis, "Multi-exposure and 43 | % Multi-focus image fusion in gradient domain", Journal of Circuits, 44 | % Systems and Computers, 2016 45 | % 46 | % [2] I.S. Sevcenco, P.J. Hampton, P. Agathoklis, "A wavelet based method 47 | % for image reconstruction from gradient data with applications", 48 | % Multidimensional Systems and Signal Processing, November 2013 49 | 50 | clear; close all; 51 | %% Specifications of the set of input images 52 | NameImg = 'office'; % Name of the input image set 53 | NumberOfImages = 5; % Number of images in the input set 54 | Format = '.jpg'; % The format or type of the input images 55 | 56 | %% Preallocate stack where the images in the input set will be stored 57 | tmp = imread(strcat(NameImg,'_1',Format)); [s(1),s(2),s(3)]=size(tmp); clear tmp; 58 | I = zeros([s,NumberOfImages]); clear s 59 | %% Read the input images 60 | for i = 1:NumberOfImages 61 | I(:,:,:,i) = imread(strcat(NameImg,'_',num2str(i),Format)); 62 | end 63 | %% Call the function to fuse the input images 64 | G = GradientFusion(I); %Main function of image fusion 65 | %% Display the fused image 66 | %% 67 | figure, 68 | for i = 1:NumberOfImages, 69 | subplot(1,NumberOfImages,i), 70 | imshow(uint8(I(:,:,:,i)));title(['Input image ', num2str(i)]); 71 | end 72 | 73 | text(-1000,7500,'Histograms of grayscale vesions of input images') 74 | 75 | figure,imshow(G),title('Fused image') 76 | -------------------------------------------------------------------------------- /GradientFusion.m: -------------------------------------------------------------------------------- 1 | function K = GradientFusion(I) 2 | % function GradientFusion.m 3 | %-------------------------------------------------------------------------- 4 | % DESCRIPTION of the function: 5 | % This is the main function for fusion of images. It works for color 6 | % (RGB) as well as grayscale images, multi-exposure as well as multi-focus images. 7 | % 8 | % INPUT: 9 | % I = Set of input images. The image set should be a 4-D array of type 10 | % uint8 with the last dimension equal to the number of images in the set. 11 | % I(:,:,i,j) represents the ith channel of the jth image. 12 | % For RGB images, i varies from 1 to 3, and for grayscale images, it is 1. 13 | % All the images in I should be registered or aligned and should be of 14 | % the same kind (i.e., either all grayscale, or all color, all having 15 | % different exposures or different regions in focus) 16 | % 17 | % OUTPUT: 18 | % K = the fused image. It is a single image of the same 19 | % dimension as each of the input images. 20 | % 21 | % Written by : Sujoy Paul, Jadavpur University, 2014 22 | % At : University of Victoria, Canada 23 | % Modified by: Ioana Sevcenco, University of Victoria, 2014 24 | % 25 | % Last updated: Nov 19, 2014 26 | % REFERENCES: 27 | % 28 | % [1] S. Paul, I.S. Sevcenco, P. Agathoklis, "Multi-exposure and 29 | % Multi-focus image fusion in gradient domain", Journal of Circuits, 30 | % Systems and Computers, 2016 31 | % 32 | % [2] I.S. Sevcenco, P.J. Hampton, P. Agathoklis, "A wavelet based method 33 | % for image reconstruction from gradient data with applications", 34 | % Multidimensional Systems and Signal Processing, November 2013 35 | 36 | CG = size(I,3); %Number of channels 37 | if (CG ~= 1)&&(CG ~= 3), 38 | disp('Images must be single channels (grayscale) or in RGB representation'); 39 | return 40 | end 41 | 42 | NumberOfImages = size(I,4); %Number of images 43 | 44 | %% Transformation of the color images from RGB to YCbCr 45 | if CG == 3, 46 | for i = 1:NumberOfImages 47 | I(:,:,:,i)=rgb2ycbcr(uint8(I(:,:,:,i))); 48 | end 49 | end 50 | 51 | % Cast to double precision 52 | I = double(I); 53 | 54 | %% Get the Gradients of Luminance and separate the Chrominance 55 | % Preallocate 56 | xH = zeros(size(I,1),size(I,2),NumberOfImages); 57 | yH = xH; 58 | I_Cb = xH; 59 | I_Cr = xH; 60 | for i = 1:NumberOfImages 61 | [xH(:,:,i), yH(:,:,i)] = getGradientH(I(:,:,1,i),1); % Compute gradients of Luminance 62 | if CG == 3, 63 | I_Cb(:,:,i) = I(:,:,2,i); % Store Cb Chrominance 64 | I_Cr(:,:,i) = I(:,:,3,i); % Store Cr Chrominance 65 | end 66 | end 67 | 68 | %% Obtain the gradient of the Luminance component of the fused image 69 | [fxH, fyH] = getFusedGradients(xH,yH); 70 | 71 | %% Get the fused Chrominance components by weighted sum 72 | if CG == 3, 73 | I_Cb128 = abs(double(I_Cb)-128); 74 | I_Cr128 = abs(double(I_Cr)-128); 75 | I_CbNew = sum((I_Cb.*I_Cb128)./repmat(sum(I_Cb128,3),[1 1 NumberOfImages]),3); 76 | I_CrNew = sum((I_Cr.*I_Cr128)./repmat(sum(I_Cr128,3),[1 1 NumberOfImages]),3); 77 | I_CbNew(isnan(I_CbNew)) = 128; 78 | I_CrNew(isnan(I_CrNew)) = 128; 79 | end 80 | 81 | %% Reconstruct the luminance from luminance gradient 82 | PoissonOn = 1; % PoissonOn = 1 to activate the Poisson Solver at each resolution 83 | avg = 0; % avg represent the mean pixel gray level 84 | R = ReconstructGradient(fxH,fyH,avg,PoissonOn); % This function reconstructs the fused luminance from the gradient data 85 | 86 | %% Normalize and enhance the fused image 87 | if CG == 1 %if image is one channel (i.e., in grayscale representation) 88 | J1 = ChannelNorm(R,[0 255]); % Gamma Correction 89 | J1 = uint8(J1); 90 | K = J1; 91 | K = adapthisteq(K,'NumTiles',[8 8],'ClipLimit',0.003); % Adaptive histogram equalization 92 | else % if image is colour 93 | % Luminance and Chrominance comhination for color images 94 | J1(:,:,1)=ChannelNorm(R(:,:,1),[16 235]); %Gamma Correction 95 | J1 = uint8(J1); 96 | K = J1; 97 | K(:,:,2) = I_CbNew; 98 | K(:,:,3) = I_CrNew; 99 | K = double(K); 100 | O = (K(:,:,1)-min(min(K(:,:,1))))/(max(max(K(:,:,1)))-min(min(K(:,:,1)))); %Adaptive histogram equalization 101 | O = adapthisteq(O,'NumTiles',[8 8],'ClipLimit',0.003)*219+16; 102 | K(:,:,1) = uint8(O); 103 | K = ycbcr2rgb(uint8(K)); 104 | end 105 | K = uint8(K); 106 | end -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Sujoy Paul 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Multi-Exposure and Multi-Focus Image Fusion 2 | 3 | ### Overview 4 | This package is an implementation of the of the paper [Multi-exposure and Multi-focus Image Fusion in Gradient Domain](http://www.worldscientific.com/doi/pdf/10.1142/S0218126616501231), by [Sujoy Paul](http://www.ee.ucr.edu/~supaul/), [Ioana S. Sevcenco](http://www.ece.uvic.ca/~iss/) and [Panajotis Agathoklis](http://www.ece.uvic.ca/~panagath/) and published at [Journal of Circuits, Systems, and Computers 5 | ](http://www.worldscientific.com/worldscinet/jcsc) 6 | 7 | ### Dependencies 8 | The package depends on the [Wavelet Based Image Reconstruction](http://www.mathworks.com/matlabcentral/fileexchange/48066-wavelet-based-image-reconstruction-from-gradient-data), which is compiled into the file ReconstructGradient.m 9 | 10 | ### Data 11 | The package uses a multi-exposure image sequence available with Matlab namely the 'office' sequence for demo purpose. Any other image sequence can be used with the package after making necessary changes for loading them in Demo.m 12 | 13 | ### Running 14 | The package can be executed by running the Demo.m script which loads the example 'office' image sequence, applies the algorithm implemented in the package and displays the fused image along with the image sequence. 15 | 16 | ### Citation 17 | Please cite the following work if you use this package. 18 | ```javascript 19 | @article{paul2016multi, 20 | title={Multi-exposure and multi-focus image fusion in gradient domain}, 21 | author={Paul, Sujoy and Sevcenco, Ioana S and Agathoklis, Panajotis}, 22 | journal={Journal of Circuits, Systems and Computers}, 23 | volume={25}, 24 | number={10}, 25 | pages={1650123}, 26 | year={2016}, 27 | publisher={World Scientific} 28 | 29 | } 30 | ``` 31 | 32 | ### Contact 33 | Please contact Sujoy Paul (supaul@ece.ucr.edu) for any further queries regarding this package. 34 | 35 | 36 | 37 | 38 | -------------------------------------------------------------------------------- /ReconstructGradient.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sujoyp/gradient-domain-imagefusion/5c56b6118f02d63987e17443de4fac6aca076037/ReconstructGradient.m -------------------------------------------------------------------------------- /descriptions/ReadMe.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sujoyp/gradient-domain-imagefusion/5c56b6118f02d63987e17443de4fac6aca076037/descriptions/ReadMe.pdf -------------------------------------------------------------------------------- /getFusedGradients.m: -------------------------------------------------------------------------------- 1 | function [fxH, fyH] = getFusedGradients(gxH,gyH) 2 | % function getFusedGradients.m 3 | %------------------------------------------------------------------------- 4 | % DESCRIPTION of the Code: 5 | % This function fuses the gradients of several single channel 6 | % images by taking the maximum gradient magnitude at each pixel 7 | % location. 8 | % 9 | % INPUTS: 10 | % gxH and gyH = the x- and y- stacked gradient components of 11 | % a set of single channel (e.g., grayscale representation) 12 | % input images. 13 | % Both gxH and gyH should be of double format, with size: 14 | % Height x Width x NumberOfImagesInTheStack (where Height and 15 | % Width are the common dimensions of all images in the 16 | % stack) 17 | % 18 | % gxH(:,:,i) and gyH(:,:,i) represent the x- and y-gradient components 19 | % of the ith image in the stack; the index i varies from 20 | % 1 to the total number of images in the stack. 21 | % 22 | % OUTPUTS: 23 | % fxH and fyH = the horizontal and vertical gradient components 24 | % from which the single channel fused image will be reconstructed 25 | % (outside this function) 26 | % 27 | % Written by : Sujoy Paul, Jadavpur University, 2014 28 | % At : University of Victoria, Canada 29 | % Modified by: Ioana Sevcenco, University of Victoria, Canada 30 | % Last updated: Nov. 19, 2014 31 | 32 | v = (gxH.^2+gyH.^2).^0.5; % Compute gradient magnitudes 33 | N = size(gxH); 34 | [~, pos] = max(v,[],3); 35 | T = zeros(N); % preallocate T 36 | for i = 1:N(3) 37 | T(:,:,i)=ones(N(1:2))*i; 38 | end 39 | Index = (T==repmat(pos,[1 1 N(3)])); 40 | fxH = sum(gxH.*Index,3); 41 | fyH = sum(gyH.*Index,3); 42 | 43 | end -------------------------------------------------------------------------------- /getGradientH.m: -------------------------------------------------------------------------------- 1 | function [dxH, dyH] = getGradientH(ConservativeField,bd) 2 | % program getGradientH.m 3 | %------------------------------------------------------------------------- 4 | % This program computes the Hudgin derivatives of a 2D array; 5 | % Discretization used: forward difference approximation. 6 | % Input: 7 | % ConservativeField = 2d array, such as a grayscale digital image 8 | % bd = Optional input: a flag that allows user to opt for Neumann boundary conditions 9 | % set to 1 if Neumann boundary conditions are required 10 | % Output: 11 | % dxH, dyH = Hudgin gradient horizontal and vertical components 12 | % Written by: Ioana Sevcenco, University of Victoria, Canada 13 | % Last updated: May 25, 2014 14 | 15 | dxH = ConservativeField(:,2:end,:) - ConservativeField(:,1:end-1,:); 16 | dyH = ConservativeField(2:end,:,:) - ConservativeField(1:end-1,:,:); 17 | if nargin > 1, 18 | if bd, 19 | dxH = padarray(dxH,[0,1],'post'); 20 | dyH = padarray(dyH,[1,0],'post'); 21 | end 22 | end 23 | 24 | end --------------------------------------------------------------------------------