├── README.md ├── aLn.m ├── asetupLnormPrior.m ├── convsparseF.m ├── dispIm.m ├── example_featureextraction.m ├── example_trainkernels.m ├── exp_feature_learning_E2time.m ├── exp_feature_learning_convergence_f.m ├── localnormalize.m ├── mycg.m ├── normalize.m ├── out_mesto_4.mat ├── showmask.m ├── shrink.m └── tileims.m /README.md: -------------------------------------------------------------------------------- 1 | This is a Matlab code for convolutional sparse coding, implementing the method proposed in 2 | 3 | *Michal Sorel, Filip Sroubek, "Fast convolutional sparse coding using matrix inversion lemma", Digital Signal Processing, 4 | vol. 55, pp. 44-51, 2016* () 5 | 6 | Convolutional sparse coding is an alternative to standard sparse coding better suited for modelling shift-invariant signals. 7 | The most time-consuming part of both kernel learning and feature extraction is the inversion of a certain linear operator 8 | related to convolution. In this work we show how these inversions can be computed non-iteratively in 9 | the Fourier domain using the matrix inversion lemma even for multiple training signals. 10 | This greatly speeds up computation and makes convolutional sparse coding computationally feasible even for large problems. 11 | 12 | The matrix inversion lemma to speed up the convolutional sparse coding was already independently used in 13 | recent papers of B. Wohlberg and Heide et al. (see bellow) 14 | **The main problem of these methods is that they work efficiently only with one input image. 15 | In practice we would like the kernels to represent well not one particular signal/image but a 16 | set of examples. The main advantage of the proposed method is that it is fast even 17 | in such cases.** 18 | 19 | For a more detailed description see or the paper itself. 20 | 21 | **Files** 22 | 23 | *example_trainkernels.m* - example of training kernels 24 | 25 | *example_featureextraction.m* - example of computing feature maps, once kernels are trained 26 | 27 | *convsparseF.m* - main function implementing several variants of the proposed algorithm 28 | and our implementation of other methods (for details, see help included in the file). The code is 29 | implemented for 2D model, i.e. for 2D convolutions. The algorithm should work in any dimension 30 | but this code implements only the 2D variant. It can be probably used for 1D signals but not 31 | for 3D convolutions. 32 | 33 | + 3D variant of the proposed algorithm (method = -1). Note that the name "3D" relates to how the algorithm 34 | works. This code does not work with 3D input data. 35 | + Consensus variant of the proposed algorithm (method = -3) 36 | + Tiling variant of the proposed algorithm (method = -4) 37 | + *M.D. Zeiler, D. Krishnan, G.W. Taylor, R. Fergus, "Deconvolutional networks", 2010* (method = 3) 38 | + *H. Bristow, A. Eriksson, S. Lucey, "Fast convolutional sparse coding", 2013* (method = 1 and method = 2) 39 | + *B. Wohlberg, "Efficient Algorithms for Convolutional Sparse Representations", 2016* (method = 0) 40 | 41 | *out_mesto_4.mat* - data used as input images for learning kernels (images of Prague 42 | Old Town and Prague Castle). They are stored as a 4D array. 43 | 44 | *exp_feature_learning_E2time.m* comparison of convergence for the proposed algorithm and two 45 | older algorithms 46 | 47 | *exp_feature_learning_convergence_f.m* kernel learning experiment for multi-scale kernels. 48 | 49 | **Terms of Use** 50 | 51 | This code can be freely used for research purposes by academic and other non-profit organizations. 52 | If you use it in your research, please cite our paper *Michal Sorel, Filip Sroubek, "Fast convolutional 53 | sparse coding using matrix inversion lemma", Digital Signal Processing, vol. 55, pp. 44-51, 2016*. 54 | -------------------------------------------------------------------------------- /aLn.m: -------------------------------------------------------------------------------- 1 | function V = aLn(DU,normDU,u_star,k) 2 | % 3 | % V = aLn(DU,normDU,u_star,k) 4 | % 5 | % half-quadratic algorithm 6 | % additive version 7 | % 8 | % For prior function phi(s) = alpha*|s-t|^q, |s|>u_star 9 | % phi(s) = beta/2*s^2, |s|<=u_star 10 | % 11 | % Shrinkage formula: 12 | % min_v { (u-v)^2 + lambda*|v| } 13 | % v_min = sign(u)*max(|u|-lambda,0); 14 | % 15 | % This is generalized for problems with |v|^q, 0<=q<=1 16 | 17 | V = zeros(size(DU)); 18 | 19 | DUp = DU(normDU>u_star); 20 | normDUp = normDU(normDU>u_star); 21 | V(normDU>u_star) = DUp.*(normDUp-k)./normDUp; 22 | 23 | -------------------------------------------------------------------------------- /asetupLnormPrior.m: -------------------------------------------------------------------------------- 1 | function P = asetupLnormPrior(q,alpha,beta) 2 | 3 | 4 | 5 | switch q 6 | case 1 7 | v_star = 0; 8 | u_star = alpha/beta; 9 | case 0 10 | v_star = sqrt(2*alpha/beta); 11 | u_star = sqrt(2*alpha/beta); 12 | otherwise % for 0 in the denosing step 23 | % default: lambda = 0; no denoising 24 | % 25 | %The expression inverse formula above means using matrix inversion lemma (aka Woodbury 26 | %formula). For one image, methods 0,-1,-2,-3,-4 are equivalent. 27 | % 28 | % output: 29 | % U ... inferred image conv(A,H) (if lambda > 0 then denoised image) 30 | % A ... sparse representation for each filter (MxNxsize(iH)) 31 | % H ... learned filters; same structure as iH 32 | % report ... structure; field E: energy in every iteration 33 | % 34 | %Michal Sorel & Filip Sroubek 2014-16 35 | 36 | %% PARAMETERS 37 | if method == -4 % tile and run basic inverse formula algorithm 38 | method = 0; 39 | G = reshape(G,[size(G,1) size(G,2)*size(G,3)]); 40 | end 41 | % Number of iterations in each step 42 | if ~exist('iters','var') || isempty(iters) 43 | iters.maxiter_main = 50; %50; % max number of iterations in the main loop 44 | iters.maxiter_A = 10; % max number of iterations in the minAstep 45 | iters.maxiter_H = 1; % max number of iterations in the minHstep 46 | iters.showims = true; 47 | end 48 | maxiter_main = iters.maxiter_main; % max number of iterations in the main loop 49 | maxiter_A = iters.maxiter_A; % max number of iterations in the minAstep 50 | maxiter_H = iters.maxiter_H; % max number of iterations in the minHstep 51 | 52 | timeit_global = zeros(1,maxiter_main); 53 | E_global = zeros(1,maxiter_main); 54 | timeit_A = zeros(maxiter_main,maxiter_A); 55 | %timeit_H = zeros(1,maxiter_H); 56 | t0 = tic; 57 | 58 | %% MAIN PARAMETERS 59 | %(alpha, beta, xi, maxbeta, tau, gamma) 60 | % 1e1 1e1 1e1 1e3 1.1 1e0 61 | % 1e1 1e2 1e2 1e4 1.1 1e1 62 | Lp = 1; % type of sparse norm l_p 63 | if isfield(iters,'alpha'), alpha = iters.alpha; 64 | else 65 | alpha = 1e1; % sparse weight 66 | end 67 | if isfield(iters,'beta'), beta = iters.beta; 68 | else 69 | beta = 1e3; %1e1; %1e2; % beta || a - v ||^2 70 | end 71 | if isfield(iters,'xi'), xi = iters.xi; 72 | else 73 | xi = 1e3; %1e1; %1e2; % xi || a - w ||^2 74 | end 75 | if isfield(iters,'maxbeta'), maxbeta = iters.maxbeta; 76 | else 77 | maxbeta = 1e1; %1e3; %1e1; 78 | end 79 | if isfield(iters,'tau'), tau = iters.tau; 80 | else 81 | tau = 1.1; 82 | end 83 | if isfield(iters,'gamma'), gamma = iters.gamma; 84 | else 85 | gamma = 0.5e0; % data term weight 86 | end 87 | 88 | reltol = 1e-9; % CG reltol 89 | %% PARAMETERS END HERE 90 | % default method is our 91 | if ~exist('method','var') || isempty(method) 92 | method = 0; 93 | end 94 | 95 | if ~exist('lambda','var') || isempty(lambda) 96 | doDenoise = false; 97 | lambda = 0; % when performing denoising, ratio between G and inferred image U 98 | else 99 | doDenoise = true; 100 | end 101 | 102 | % set convolution kernels at different scales (initialization) 103 | if exist('iH','var') && ~isempty(iH) 104 | H = iH(:); 105 | S = length(H); 106 | hsize = zeros(S,3); 107 | for s = 1:S 108 | hsize(s,:) = [ size(H{s},1) size(H{s},2) size(H{s},3) ]; 109 | end 110 | else 111 | %% default initialization of convolution kernels H 112 | S = 7; % number of scales 113 | K = [1 1 1 1 1 1 1]; %1*ones(S,1); % number of kernels in each scale 114 | H = cell(S,1); 115 | hsize = zeros(S,3); 116 | for s = 1:S 117 | %H{s} = randn((s-1)*2+1,(s-1)*2+1,K(s)); 118 | H{s} = ones((s-1)*2+1,(s-1)*2+1,K(s)); 119 | %H{s} = rand(7,7,K(s)); 120 | hsize(s,:) = [size(H{s},1) size(H{s},2) size(H{s},3)]; 121 | end 122 | end 123 | 124 | gsize = [size(G,1),size(G,2),size(G,3)]; 125 | L = gsize(3); % number of input images 126 | 127 | % variables used in min_A 128 | asize = [gsize(1)*gsize(2),sum(hsize(:,3)),L]; 129 | A = zeros(asize); 130 | FA = zeros(asize); 131 | 132 | V = zeros(asize); % Auxiliar variable 133 | B = zeros(asize); % Lagrange multiplier 134 | %% additive 135 | Pr = asetupLnormPrior(Lp,alpha,beta); 136 | V = Pr.fh(A-B,abs(A-B)); 137 | 138 | % variables used in min_H 139 | FH = zeros(asize(1),asize(2)); 140 | i = 0; 141 | for s = 1:S 142 | H{s} = projSphere(H{s}); % assure that H satisfy the same condition on its magnitude as in H minimization 143 | FH(:,i+[1:hsize(s,3)]) = reshape(fft2(H{s},gsize(1),gsize(2)),[asize(1), hsize(s,3)]); 144 | i = i+hsize(s,3); 145 | end 146 | if method == -1 % pak zkontrolovat, jestli opravdu musim udrzovat 147 | FH3 = repmat(FH,[L 1]); 148 | W = zeros([gsize(1),gsize(2),L,asize(2)]); 149 | for i = 1:asize(2) 150 | W(:,:,:,i) = real(ifftn(reshape(FH3(:,i),[gsize(1),gsize(2),L]))); % Auxiliar variable 151 | end 152 | C = zeros(gsize(1),gsize(2),gsize(3),asize(2)); % Lagrange multiplier 153 | FG3 = repmat(reshape(fftn(G),[asize(1)*L,1]),[1 size(FH,2)]); 154 | elseif method == -3 % consensus (W is L x copy of initial H, C = 0) 155 | % FH3 = repmat(FH,[L 1]); ASI BUDU MUSET PRIDAT 156 | W = repmat(real(ifft2(reshape(FH,[gsize(1),gsize(2),asize(2)]))),[1 1 1 L]); % Auxiliary variable 157 | C = zeros(size(W)); % Lagrange multiplier N x K x L 158 | else 159 | W = real(ifft2(reshape(FH,[gsize(1),gsize(2),asize(2)]))); % Auxiliary variable 160 | C = zeros(gsize(1),gsize(2),asize(2)); % Lagrange multiplier 161 | end 162 | FG = repmat(reshape(fft2(G),[asize(1),1,L]),[1 size(FH,2)]); 163 | report.E = []; 164 | 165 | for i_main = 1:maxiter_main 166 | disp([num2str(i_main)]); 167 | t_g=tic;t_Astep=tic; 168 | % A = min_A E(A,H) 169 | minAstep; 170 | disp(['Time of A step: ' num2str(toc(t_Astep)) ' seconds']); 171 | % H = min_H E(A,H) 172 | minHstep; 173 | [E,E1,l1] = calcE; 174 | disp(['After minH: E=',num2str(E),', data=',num2str(E1),', l1=',num2str(l1)]); 175 | % update G (denoising step) 176 | if doDenoise 177 | minUstep; 178 | end 179 | 180 | % update beta, xi 181 | dstr = ''; 182 | if beta < maxbeta 183 | beta = beta*tau; 184 | dstr = [dstr,'beta=',num2str(beta),' ']; 185 | end 186 | if xi < maxbeta 187 | xi = xi*tau; 188 | dstr = [dstr,'xi=',num2str(xi)]; 189 | end 190 | disp(dstr); 191 | E_global(i_main) = E; 192 | timeit_global(i_main) = toc(t0); 193 | disp(['Time of iteration: ' num2str(toc(t_g)) ' seconds']); 194 | end 195 | report.timeit_global = timeit_global; 196 | report.E_global = E_global; 197 | report.timeit_A = timeit_A; 198 | A = reshape(A,[gsize(1),gsize(2),asize(2),asize(3)]); 199 | % if no denoising is selected return in U the synthesized image 200 | if ~doDenoise 201 | lambda = 1; 202 | minUstep; 203 | end 204 | 205 | if iters.showims 206 | figure; 207 | m = ceil((asize(2)+2)/2); 208 | subplot(m,2,1); dispIm(G(:,:,1)); title('input'); 209 | subplot(m,2,2); dispIm(U(:,:,1)); title(['synthesized: \alpha=',num2str(alpha),', \beta=',num2str(beta),', \gamma =',num2str(gamma)]); 210 | for s=1:asize(2) 211 | subplot(m,2,2+s); dispIm(A(:,:,s,1)); colorbar; %title(num2str(hsize(s,1))); 212 | end 213 | %if maxiter_H > 0 214 | figure; 215 | for s=1:S 216 | subplot(S,1,s); 217 | z = []; 218 | for i=1:hsize(s,3) 219 | z = [z,linscale(H{s}(:,:,i)),zeros(hsize(s,1),1)]; 220 | end; 221 | dispIm(z); 222 | end 223 | %end 224 | end 225 | %% END of main loop 226 | 227 | %% update A 228 | function minAstep 229 | % size asize(1) x asize(2) x L 230 | FHtG = repmat(conj(FH),[1 1 L]).*FG; 231 | 232 | % prepare HtH variable for Bristow's method 233 | if method == 2 234 | FHtH = zeros(asize(2),asize(2),asize(1)); 235 | for i = 1:asize(2) 236 | FHtH(i,i,:) = abs(FH(:,i)).^2; 237 | for j = i+1:asize(2) 238 | FHtH(i,j,:) = conj(FH(:,i)).*FH(:,j); 239 | FHtH(j,i,:) = conj(FHtH(i,j,:)); 240 | end 241 | end 242 | if maxiter_A > 1 % precompute inverses 243 | FHtHinv = zeros(size(FHtH));tic; 244 | for n=1:asize(1) 245 | FHtHinv(:,:,n) = inv(FHtH(:,:,n) + beta/gamma*eye(asize(2))); 246 | end; 247 | disp(['Time of inversion: ' num2str(toc) 's']); 248 | end 249 | end 250 | 251 | for i=1:maxiter_A 252 | % size asize(1) x asize(2) x L 253 | b = FHtG + beta/gamma*reshape(fft2(reshape(V + B,[gsize(1),gsize(2),asize(2),asize(3)])),asize); 254 | switch method 255 | case {-3,-2,-1,0, 1} 256 | if (L>1) 257 | rFH = repmat(FH,[1 1 L]); 258 | FA = gamma/beta*(b-conj(rFH).*... 259 | repmat(sum(rFH.*b,2)./(beta/gamma+sum(rFH.*conj(rFH),2)),[1 asize(2) 1])); 260 | else 261 | FA = gamma/beta*(b-conj(FH).*... 262 | repmat(sum(FH.*b,2)./(beta/gamma+sum(FH.*conj(FH),2)),[1 asize(2) 1])); 263 | end 264 | case 2 265 | b = reshape(shiftdim(b,1),[asize(2),L,asize(1)]); 266 | if maxiter_A > 1 267 | for n=1:asize(1) 268 | FA(n,:,:) = FHtHinv(:,:,n)*b(:,:,n); 269 | end 270 | else 271 | for n=1:asize(1) 272 | FA(n,:,:) = (FHtH(:,:,n) + beta/gamma*eye(asize(2)))\b(:,:,n); 273 | end 274 | end 275 | case 3 276 | [xmin,flag,relres,iter,resvec] = mycg(@gradA,vec(b),reltol,1000,[],vec(FA)); 277 | disp(['flag, iter:',num2str([flag iter])]); 278 | FA = reshape(xmin,asize); 279 | end 280 | 281 | A = reshape(real(ifft2(reshape(FA,[gsize(1),gsize(2),asize(2),asize(3)]))),asize); 282 | %% additive 283 | Pr = asetupLnormPrior(Lp,alpha,beta); 284 | %% additive 285 | V = Pr.fh(A-B,abs(A-B)); % Lp norm 286 | %V = A-B; % no contraint 287 | %V = beta*(A-B)/(alpha+beta); % L2 norm 288 | %V(V<0) = 0; 289 | % update Bregman variable (Lagrange variable) 290 | B = B + V - A; 291 | timeit_A(i_main,i) = toc(t0); 292 | end 293 | [E,E1,l1] = calcE; 294 | disp(['E=',num2str(E),', data=',num2str(E1),', l1=',num2str(l1)]); 295 | report.E = [report.E, E]; 296 | end 297 | 298 | function r = gradA(x) 299 | X = reshape(x,asize); 300 | R = repmat(sum(repmat(FH,[1 1 L]).*X,2),[1 asize(2)]).*repmat(conj(FH),[1 1 L]) ... 301 | + beta/gamma*X; 302 | r = vec(R); 303 | end 304 | 305 | %% update H 306 | function minHstep 307 | if maxiter_H == 0 308 | return; 309 | end 310 | if method == -1 % A'G for 3d convolution 311 | FA3 = FG3; %=zeros(size(FG3)); % could be faster like this 312 | for i = 1:size(FG3,2) 313 | FA3(:,i) = reshape(fftn(reshape(A(:,i,:),[gsize(1) gsize(2) L])),[size(FG3,1) 1]); 314 | end 315 | FAtG = conj(FA3).*FG3; %ve 3D verzi bych nemel scitat! 316 | else 317 | FAtG = sum(conj(FA).*FG,3); 318 | end 319 | 320 | % prepare AtA variable for Bristow's method and our method "1" 321 | if (method == 1) || (method == 2) 322 | FAtA = zeros(asize(2),asize(2),asize(1)); 323 | for i = 1:asize(2) 324 | FAtA(i,i,:) = sum(abs(FA(:,i,:)).^2,3); 325 | for j = i+1:asize(2) 326 | FAtA(i,j,:) = sum(conj(FA(:,i,:)).*FA(:,j,:),3); 327 | FAtA(j,i,:) = conj(FAtA(i,j,:)); 328 | end 329 | end 330 | end 331 | 332 | for i=1:maxiter_H 333 | if method == -1 % 3D inverse formula 334 | b = FAtG; 335 | for ii = 1:asize(2) 336 | b(:,ii) = b(:,ii) + xi/gamma*reshape(fftn(W(:,:,:,ii) + C(:,:,:,ii)),[asize(1)*asize(3) 1]); 337 | end 338 | FH3 = gamma/xi*(b-conj(FA3).*... 339 | repmat(sum(FA3.*b,2)./(xi/gamma+sum(FA3.*conj(FA3),2)),[1 asize(2)])); 340 | 341 | TH = zeros([gsize(1) gsize(2) gsize(3) asize(2)]); 342 | for ii = 1:asize(2) 343 | TH(:,:,:,ii) = real(ifftn(reshape(FH3(:,ii),[gsize(1),gsize(2),gsize(3)]))); 344 | end 345 | 346 | % update auxiliary variable 347 | W = TH-C; 348 | % zero W outside H support 349 | n = 0; 350 | for s = 1:S 351 | W(hsize(s,1)+1:end,:,1,n+[1:hsize(s,3)]) = 0; % everything bellow 352 | W(1:hsize(s,1),hsize(s,2)+1:end,1,n+[1:hsize(s,3)]) = 0; % to the right 353 | n = n+hsize(s,3); 354 | end 355 | W(:,:,2:end,:) = 0; % zero W outside of the main plane 356 | 357 | % project W into sphere 358 | W = projSphere(W); % function is in the same file under convsparseF() 359 | else 360 | if (method ~= -2 && method ~= -3) 361 | b = FAtG + xi/gamma*reshape(fft2(W + C),asize(1:2)); 362 | end 363 | switch method 364 | case -2 % consensus inverse formula - approximative 365 | b = conj(FA).*FG + xi/gamma*repmat(reshape(fft2(W+C),asize(1:2)),[1 1 L]); 366 | FH = mean(gamma/xi*(b-conj(FA).* ... 367 | repmat(sum(FA.*b,2)./(xi/gamma + sum(FA.*conj(FA),2)),[1 asize(2)])),3); 368 | case -3 % consensus inverse formula 369 | b = conj(FA).*FG + xi/gamma*reshape(fft2(W+C),asize); 370 | FH = gamma/xi*(b-conj(FA).* ... 371 | repmat(sum(FA.*b,2)./(xi/gamma + sum(FA.*conj(FA),2)),[1 asize(2)])); % jako 0, cili -2 bez prumerovani 372 | case 0 373 | if (L>1) 374 | % iterative inverse formula 375 | FH = calcInverse(b); 376 | else % L=1 or tiling 377 | FH = gamma/xi*(b-conj(FA).*... 378 | repmat(sum(FA.*b,2)./(xi/gamma+sum(FA.*conj(FA),2)),[1 asize(2)])); 379 | end 380 | case {1, 2} 381 | for n=1:asize(1) 382 | FH(n,:) = (FAtA(:,:,n) + xi/gamma*eye(asize(2)))\(b(n,:)).'; 383 | end 384 | case 3 385 | [xmin,flag,relres,iter,resvec] = mycg(@gradH,vec(b),reltol,1000,[],vec(FH)); 386 | disp(['flag, iter:',num2str([flag iter])]); 387 | FH = reshape(xmin,size(FH)); 388 | end 389 | if method == -3 390 | TH = real(ifft2(reshape(FH,[gsize(1),gsize(2),asize(2:3)]))); 391 | % update auxiliary variable 392 | W = mean(TH-C,4); 393 | else 394 | TH = real(ifft2(reshape(FH,[gsize(1),gsize(2),asize(2)]))); 395 | W = TH - C; 396 | end 397 | % zero W outside H support, average and project to sphere 398 | n = 0; 399 | for s = 1:S 400 | W(hsize(s,1)+1:end,:,n+(1:hsize(s,3))) = 0; 401 | W(1:hsize(s,1),hsize(s,2)+1:end,n+(1:hsize(s,3))) = 0; 402 | n = n+hsize(s,3); 403 | end 404 | % project W into sphere 405 | W = projSphere(W); % function is in the same file under convsparseF() 406 | end 407 | if method == -3 408 | W = repmat(W,[1 1 1 L]); 409 | end 410 | % update Bregman variable (Lagrange multiplier) 411 | C = C + W - TH; 412 | %calcE; 413 | end 414 | 415 | % update H - for now we update by the auxiliary variable 416 | if method == -1 % 3D 417 | n = 0; 418 | for s = 1:S 419 | %H{s} = TH(1:hsize(s,1),1:hsize(s,2),1,n+[1:hsize(s,3)]); 420 | H{s} = W(1:hsize(s,1),1:hsize(s,2),1,n+[1:hsize(s,3)]); 421 | FH(:,n+[1:hsize(s,3)]) = reshape(fft2(H{s},gsize(1),gsize(2)),[asize(1), hsize(s,3)]); 422 | n = n+hsize(s,3); 423 | end 424 | elseif method == -3 % true consensus 425 | n = 0; 426 | FH = zeros(asize(1),asize(2)); 427 | for s = 1:S 428 | H{s} = W(1:hsize(s,1),1:hsize(s,2),n+(1:hsize(s,3)),1); % take 1st, all are the same 429 | FH(:,n+[1:hsize(s,3)]) = reshape(fft2(H{s},gsize(1),gsize(2)),[asize(1), hsize(s,3)]); 430 | n = n+hsize(s,3); 431 | end 432 | else % all others 433 | n = 0; 434 | for s = 1:S 435 | %H{s} = TH(1:hsize(s,1),1:hsize(s,2),n+[1:hsize(s,3)]); 436 | H{s} = W(1:hsize(s,1),1:hsize(s,2),n+[1:hsize(s,3)]); 437 | FH(:,n+(1:hsize(s,3))) = reshape(fft2(H{s},gsize(1),gsize(2)),[asize(1), hsize(s,3)]); 438 | n = n+hsize(s,3); 439 | end 440 | end 441 | 442 | end % minHstep 443 | 444 | function W = projSphere(W) % projection into sphere in 2D (circle) - must be changed to work for other dims 445 | nW = squeeze(sum(sum(W.^2,1),2)); 446 | d = hsize(1,1)*hsize(1,2); %d = ones(asize(2),1)*hsize(1,1)*hsize(1,2); 447 | % n = 0; 448 | % for s = 1:S % d(n+[1:hsize(s,3)]) = hsize(s,1)*hsize(s,2); 449 | % d(n+[1:hsize(s,3)]) = (hsize(1,1)*hsize(1,2))^2/(hsize(s,1)*hsize(s,2)); 450 | % n = n+hsize(s,3); 451 | % end 452 | mask = nW>d; 453 | if sum(mask) ~= 0 454 | %W(:,:,mask) = W(:,:,mask)./repmat(reshape(sqrt(nW(mask)./d(mask)),1,1,[]),[gsize(1),gsize(2)]); 455 | W(:,:,mask) = W(:,:,mask)./repmat(reshape(sqrt(nW(mask)./d),1,1,[]),[size(W,1),size(W,2)]); 456 | end 457 | end 458 | 459 | function r = gradH(x) 460 | X = reshape(x,size(FH)); 461 | R = sum(repmat(sum(FA.*repmat(X,[1 1 L]),2),[1 asize(2)]).*conj(FA),3) ... 462 | + xi/gamma*X; 463 | r = vec(R); 464 | end 465 | 466 | function FH = calcInverse(b) 467 | iA = zeros(asize(1),asize(2),asize(2)); 468 | % inv(I + H_1^T H_1) 469 | ia = (gamma/xi)^2./(1 + gamma/xi*sum(abs(FA(:,:,1)).^2,2)); 470 | T1 = zeros(size(iA)); T2 = zeros(size(iA)); 471 | for k=1:asize(2) 472 | for l=1:asize(2) 473 | iA(:,k,l) = -FA(:,l,1).*conj(FA(:,k,1)).*ia; 474 | if k==l 475 | iA(:,k,l) = gamma/xi + iA(:,k,l); 476 | end 477 | end 478 | end 479 | % inverse of the rest 480 | for p = 2:L 481 | for k=1:asize(2) 482 | for l=1:asize(2); 483 | T1(:,k,l) = FA(:,k,p).*conj(FA(:,l,p)); 484 | end 485 | end 486 | ia = 1./(1 + sum(sum(T1.*iA,2),3)); 487 | for k=1:asize(2) 488 | for l=1:asize(2) 489 | T2(:,k,l) = T1(:,l,k).*ia; 490 | end 491 | end 492 | for k=1:asize(2) 493 | for l=1:asize(2) 494 | T1(:,k,l) = sum(squeeze(T2(:,k,:)).*iA(:,:,l),2); 495 | end 496 | end 497 | for k=1:asize(2) 498 | for l=1:asize(2) 499 | T2(:,k,l) = sum(squeeze(iA(:,k,:)).*T1(:,:,l),2); 500 | end 501 | end 502 | iA = iA - T2; 503 | end 504 | for k=1:asize(2) 505 | FH(:,k) = sum(squeeze(iA(:,k,:)).*b,2); 506 | end 507 | 508 | end 509 | 510 | %% update G 511 | function minUstep 512 | if lambda == 0 513 | U = G; 514 | return; 515 | end 516 | U = real(ifft2(reshape(sum(FA.*repmat(FH,[1 1 asize(3)]),2),gsize(1),gsize(2),asize(3)))); 517 | U = (1-lambda)*G + lambda*U; 518 | FG = repmat(reshape(fft2(U),[asize(1),1,L]),[1 size(FH,2)]); 519 | end 520 | 521 | 522 | function [E,E1,l1] = calcE 523 | U = real(ifft2(reshape(sum(FA.*repmat(FH,[1 1 asize(3)]),2),gsize(1),gsize(2),asize(3)))); 524 | l1 = sum(abs(A(:))); 525 | E1 = sum((U(:)-G(:)).^2); 526 | E = gamma*E1 + alpha*l1; 527 | %disp(['E=',num2str(E),', data=',num2str(E1),', l1=',num2str(l1)]); 528 | end 529 | 530 | function [E,E1,l1] = calcEcons 531 | U = real(ifft2(reshape(sum(FA.*repmat(FH,[1 1 asize(3)]),2),gsize(1),gsize(2),asize(3)))); 532 | l1 = sum(abs(A(:))); 533 | E1 = sum((U(:)-G(:)).^2); 534 | E = gamma*E1 + alpha*l1; 535 | %disp(['E=',num2str(E),', data=',num2str(E1),', l1=',num2str(l1)]); 536 | end 537 | 538 | function E1 = calcRes % residual 539 | U = real(ifft2(reshape(sum(FA.*repmat(FH,[1 1 asize(3)]),2),gsize(1),gsize(2),asize(3)))); 540 | E1 = sum((U(:)-G(:)).^2); 541 | end 542 | 543 | 544 | end 545 | 546 | -------------------------------------------------------------------------------- /dispIm.m: -------------------------------------------------------------------------------- 1 | function dispIm(y) 2 | 3 | imagesc(y); 4 | colormap(gray(255)); 5 | axis image; -------------------------------------------------------------------------------- /example_featureextraction.m: -------------------------------------------------------------------------------- 1 | % example of feature extraction - runs the same function as for training: convsparseF with 2 | % only one main iteration and no kernel optimization 3 | 4 | %% load data and set parameters 5 | load out_mesto_4 6 | 7 | nexp = 1; % experiment number 8 | q = 40; % figures to display results 9 | method = -4; % for feature extraction we could use any of 0,-1,-2,-3,-4 (all these are equivalent) 10 | 11 | image = double(rgb2gray(squeeze(im(:,:,1,:))))/65535; % let us try the first image 12 | % in this example we use grayscale but we could use RGB, which would work layer-wise: 13 | % image = double((squeeze(im(:,:,1,:))))/65535; 14 | image = localnormalize(image,4,4); 15 | 16 | load(['example_trainingresults_' num2str(nexp)],'H'); % loads variable 'H' 17 | K = {[8 32 64]}; % number of kernels for each scale (for one scale simply K = {8}) 18 | 19 | iters.maxiter_main = 1; % one main iteration 20 | iters.maxiter_A = 100; % max number of iterations in the minAstep 21 | iters.maxiter_H = 0; % no iterations in the minHstep 22 | iters.showims = false; 23 | 24 | tic;[Un, A, ~, report2_0] = convsparseF(image,H,0,iters);toc % Ours 25 | %tic;[Un, A, ~, report2_2] = convsparseF(image,H,2,iters);toc % Bristow 26 | %tic;[Un, A, ~, report2_3] = convsparseF(image,H,3,iters);toc % CG 27 | 28 | %% show results 29 | figure(q+1);imshow(image);title('Locally normalized image'); 30 | figure(q+2);imshow(Un);title('Image obtained from convolutional sparse model'); 31 | mapn = 10; % show feature map 10 but we could take any ... 32 | figure(q+3);imshow(abs(squeeze(A(:,:,mapn,:)))./max(max(max(abs(A(:,:,mapn,:))))));title(['Feature map n. ' num2str(mapn)]); 33 | 34 | %% Time of feature extraction 35 | % figure(q+6); 36 | % x = 1:iters.maxiter_A; 37 | % plot(...%x,report2_3.timeit_A,'k:',... %barvy: b,r,g,k,m,y,k 38 | % ...%x,report2_2.timeit_A,'b--',... 39 | % x,report2_0.timeit_A,'r-.',... 40 | % 'linewidth',2); 41 | % xlabel('Number of iterations'); 42 | % ylabel(['Computation time [s]']); 43 | % title('Time of feature extraction') 44 | % % legend('Zeiler et al. [21]','Bristow et al. [23]','Proposed','Location','NorthWest'); 45 | % % saveas(q+6,['time2iters' '.fig']); 46 | % %print( q+6, ['time2iters' '.eps'], '-deps2c', '-tiff' ); 47 | % %eps2pdf(['time2iters' '.eps'],'c:\program files\gs\gs9.10\bin\gswin64c.exe'); 48 | 49 | -------------------------------------------------------------------------------- /example_trainkernels.m: -------------------------------------------------------------------------------- 1 | % MULTISCALE EXAMPLE OF TRAINING CONVOLUTION KERNELS 2 | % 3 | disp('Warning: for large numbers of input images algorithm needs a lot of memory, be sure you have at least 16GB of RAM.'); 4 | disp('It also takes time, 1000 iterations can take a day or two, depending on your machine.'); 5 | load out_mesto_4 6 | 7 | nexp = 1; % experiment number 8 | method = -4; 9 | 10 | L=50; % number of input images 11 | %L = [1 10 50]; % multiple values can be used to compute several experiments at once 12 | K = {[8 32 64]}; % number of kernels for each scale (for one scale simply K = {8}) 13 | % again cell-array can have more entries to compute several experiments at once 14 | 15 | % Kernels are trained on 2D data, for this reason color images are first converted to grayscale 16 | % Data are locally normalized, it is necessary to get beautiful kernels! 17 | j = 1; 18 | clear In 19 | In = zeros(128,128,size(im,3)); 20 | for i=1:size(im,3) 21 | I = double(rgb2gray(squeeze(im(:,:,i,:))))/65535; 22 | I = I(101:228,101:228); % smaller crop 128x128 23 | [In(:,:,j)] = localnormalize(I,4,4); % for nature sigma = 5, otherwise sigma = 4 24 | j = j+1; 25 | end 26 | 27 | iters.maxiter_main = 1000; % max number of iterations in the main loop 28 | iters.maxiter_A = 10; % max number of iterations in the minAstep 29 | iters.maxiter_H = 10; % max number of iterations in the minHstep 30 | iters.showims = false; 31 | iters.beta = 1e3; 32 | iters.xi = 1e3; 33 | 34 | % Consensus algorithm works better with different parameters 35 | %itersConsensus = iters; 36 | %itersConsensus.beta = 1e2; 37 | %itersConsensus.xi = 1e2; 38 | 39 | rep = cell(length(L),length(K)); % reports 40 | t = zeros(length(L),numel(K)); % time per experiment 41 | for l = 1:length(L) 42 | if L(l) == 1 43 | warning('For one image, all inverse formula based methods are equivalent'); 44 | end 45 | input = In(:,:,1:L(l)); 46 | for k = 1:numel(K) 47 | iH = cell(length(K{k}),1); 48 | for i = 1:length(K{k}) 49 | iH{i} = randn(2^(i+2),2^(i+2),K{k}(i)); 50 | end 51 | tic; 52 | [Un, A, H, rep{l,k}] = convsparseF(input,iH,method,iters); 53 | t(l,k)=toc; 54 | disp(['L=' num2str(L(l)) ', K=' num2str(K{k}) ', time: ' num2str(toc) 's']); 55 | end 56 | end 57 | save(['example_trainingresults_' num2str(nexp)],'t','rep','H','A'); %tiling only 58 | 59 | % Energy vs. time for the last experiment 60 | figure(49); 61 | nK = length(K); 62 | nL = length(L); 63 | plot(rep{nL,nK}.timeit_global,rep{nL,nK}.E_global,'k-.','linewidth',2); 64 | xlabel('Time [s]'); 65 | ylabel(['Energy']); 66 | title(['Convergence, K=' num2str(K{nK}) ', L=' num2str(L(nL)) ', P=' num2str(iters.maxiter_A)]) 67 | 68 | % kernels 69 | ind = 1; 70 | for iii = 1:length(H) 71 | indend = ind + size(H{iii},3) - 1; 72 | [v i] = sort(squeeze(sum(sum(sum(abs(A(:,:,ind:indend)).^2,1),2),4)),'descend'); % only for one size - sort by A energy 73 | aux = reshape(mat2cell(squeeze(H{iii}),size(H{iii},1),size(H{iii},2),ones(1,size(H{iii},3))),[size(squeeze(H{iii}),3) 1]); 74 | figure(49+iii);w=showmask(tileims(aux(i),2,8,1,-5),-3); 75 | imwrite(w,['learnedkernels_L' num2str(size(H{iii},1)) '_' num2str(nexp) '.png']); 76 | ind = indend+1; 77 | end 78 | -------------------------------------------------------------------------------- /exp_feature_learning_E2time.m: -------------------------------------------------------------------------------- 1 | %im = read_out_mesto_4(true); 2 | %save out_mesto_4 im 3 | % Read input images stored in a .mat file 4 | load out_mesto_4 5 | 6 | j = 1; 7 | clear In 8 | In = zeros(128,128,size(im,3)); 9 | for i=1:size(im,3) 10 | I = double(rgb2gray(squeeze(im(:,:,i,:))))/255; 11 | I = I(101:228,101:228); % smaller crop 128x128 12 | [In(:,:,j) k q] = localnormalize(I,4,4); % for nature sigma=5 otherwise sigma=4 13 | j = j+1; 14 | end 15 | 16 | % Set numbers of iterations 17 | iters.maxiter_main = 100; %50; % max number of iterations in the main loop 18 | iters.maxiter_A = 5; % max number of iterations in the minAstep 19 | iters.maxiter_H = 5; % max number of iterations in the minHstep 20 | iters.showims = false; 21 | iters.beta = 1e3; 22 | iters.xi = 1e3; 23 | 24 | itersConsensus = iters; 25 | itersConsensus.beta = 1e2; 26 | itersConsensus.xi = 1e2; 27 | 28 | nexp = 50; % experiment number 29 | withBristow =false; 30 | if ~withBristow, disp('For speed the Bristow''s algorithm is skipped'); end 31 | L=10; %L = [1 10 50]; 32 | K=50; %K = [1:10 15:5:50 60:10:100]; Max. 59 33 | M=9; % kernel size is MxM 34 | rep1 = cell(length(L),length(K)); % reports 35 | rep2 = cell(size(rep1)); 36 | rep3 = cell(size(rep1)); 37 | rep4 = cell(size(rep1)); 38 | rep5 = cell(size(rep1)); 39 | kernels = randn(M,M,K(end)); 40 | t = zeros(length(L),length(K),4); 41 | for l = 1:length(L) 42 | input = In(:,:,1:L(l)); 43 | for k = 1:length(K) 44 | iH{1} = kernels(:,:,1:K(k)); 45 | if withBristow 46 | disp('---------------------------------------------------'); 47 | disp('Bristow'); 48 | disp('---------------------------------------------------'); 49 | tic;[Un, A1, H1, rep1{l,k}] = convsparseF(input,iH,2,iters);t(l,k,1)=toc; %Bristow 50 | end 51 | disp('---------------------------------------------------'); 52 | disp('Proposed 3D'); 53 | disp('---------------------------------------------------'); 54 | tic;[Un, A3, H3, rep3{l,k}] = convsparseF(input,iH,-1,iters);t(l,k,3)=toc; % 3D Woodbury 55 | if L == 1 56 | tic;[Un, A4, H4, rep4{l,k}] = convsparseF(input,iH,0,itersConsensus);t(l,k,4)=toc; % single image alg. 57 | %for L>1 this version uses recursion, which is extremely slow for more than just a few images 58 | else 59 | disp('---------------------------------------------------'); 60 | disp('Proposed consensus'); 61 | disp('---------------------------------------------------'); 62 | tic;[Un, A4, H4, rep4{l,k}] = convsparseF(input,iH,-3,itersConsensus);t(l,k,4)=toc; % consensus 63 | disp('---------------------------------------------------'); 64 | disp('Proposed consensus approximative'); 65 | disp('---------------------------------------------------'); 66 | tic;[Un, A4, H4, rep5{l,k}] = convsparseF(input,iH,-2,itersConsensus);t(l,k,4)=toc; % consensus approximative 67 | disp('---------------------------------------------------'); 68 | disp('Proposed tiling'); 69 | disp('---------------------------------------------------'); 70 | tic;[Un, A2, H2, rep2{l,k}] = ... 71 | convsparseF(reshape(input,[size(In,1) size(In,2)*size(input,3)]),... 72 | iH,0,iters);t(l,k,2)=toc; % Tiling - not computed for L=1 73 | end 74 | disp(['L=' num2str(L(l)) ', K=' num2str(K(k)) ', time: ' num2str(toc) 's']); 75 | end 76 | % with huge A's stored 77 | if withBristow 78 | save(['timegraph_' num2str(nexp)],'t','rep1','rep2','rep3','rep4',... 79 | 'H1','H2','H3','H4','A1','A2','A3','A4'); 80 | else 81 | save(['timegraph_' num2str(nexp)],'t','rep1','rep2','rep3','rep4',... 82 | 'H2','H3','H4','A2','A3','A4'); 83 | end 84 | end 85 | 86 | if withBristow 87 | % Energy vs. time 88 | q = 200; 89 | figure(q+10); 90 | %K = [1:10 15:5:50 60:10:100]; 91 | nK = length(K); % max. number of kernels 92 | nL = length(L); % max. number of input images 93 | plot( rep1{nL,nK}.timeit_global,rep1{nL,nK}.E_global,'b--',... % Bristow 94 | rep2{nL,nK}.timeit_global,rep2{nL,nK}.E_global,'k-.',... % Proposed tiling 95 | rep3{nL,nK}.timeit_global,rep3{nL,nK}.E_global,'r:',... % Proposed 3D convolution 96 | rep4{nL,nK}.timeit_global,rep4{nL,nK}.E_global,'g-',... % Proposed consensus 97 | 'linewidth',2); 98 | xlabel('Time'); 99 | ylabel(['Energy']); 100 | title(['Convergence, K=' num2str(K(nK)) ', L=' num2str(L(nL)) ', P=' num2str(iters.maxiter_A)]) 101 | legend('Bristow et al.','Proposed - tiling','Proposed - 3D','Proposed - consensus','Location','NorthEast'); 102 | end 103 | 104 | q = 100; % comparison of our methods (without approximative consensus) 105 | figure(q+11); 106 | nK = length(K); % max. number of kernels 107 | nL = length(L); % max. number of input images 108 | plot( ... %rep1{nL,nK}.timeit_global,rep1{nL,nK}.E_global,'b--',... % Bristow 109 | rep2{nL,nK}.timeit_global,rep2{nL,nK}.E_global,'k-.',... % Proposed tiling 110 | rep3{nL,nK}.timeit_global,rep3{nL,nK}.E_global,'r:',... % Proposed 3D convolution 111 | rep4{nL,nK}.timeit_global,rep4{nL,nK}.E_global,'g-',... % Proposed consensus 112 | 'linewidth',2); 113 | xlabel('Time [s]'); 114 | ylabel(['Energy']); 115 | title(['Convergence of proposed methods, K=' num2str(K(nK)) ', L=' num2str(L(nL)), ', P=' num2str(iters.maxiter_A)]) 116 | %legend('Bristow et al.','Proposed tiling','Proposed 3D','Proposed consensus','Location','NorthWest'); 117 | legend('Proposed - tiling','Proposed - 3D','Proposed - consensus','Location','NorthEast'); 118 | print( q+11, ['E2time_L10_' num2str(nexp) '.eps'], '-deps2c', '-tiff' ); 119 | %eps2pdf(['E2time_L10_' num2str(nexp) '.eps'],'c:\program files\gs\gs9.10\bin\gswin64c.exe'); 120 | 121 | q = 300; % comparison of our methods (with approximative consensus) 122 | figure(q+11); 123 | %K = [1:10 15:5:50 60:10:100]; 124 | nK = length(K); % max. number of kernels 125 | nL = length(L); % max. number of input images 126 | plot( ... %rep1{nL,nK}.timeit_global,rep1{nL,nK}.E_global,'b--',... % Bristow 127 | rep2{nL,nK}.timeit_global,rep2{nL,nK}.E_global,'k-.',... % Proposed tiling 128 | rep3{nL,nK}.timeit_global,rep3{nL,nK}.E_global,'r:',... % Proposed 3D convolution 129 | rep4{nL,nK}.timeit_global,rep4{nL,nK}.E_global,'g-',... % Proposed consensus 130 | rep5{nL,nK}.timeit_global,rep5{nL,nK}.E_global,'b--',... % Proposed consensus approximative 131 | 'linewidth',2); 132 | xlabel('Time [s]'); 133 | ylabel(['Energy']); 134 | title(['Convergence of proposed methods, K=' num2str(K(nK)) ', L=' num2str(L(nL)), ', P=' num2str(iters.maxiter_A)]) 135 | %legend('Bristow et al.','Proposed tiling','Proposed 3D','Proposed consensus','Location','NorthWest'); 136 | legend('Proposed - tiling','Proposed - 3D','Proposed - consensus','Proposed - consensus approx.','Location','NorthEast'); 137 | print( q+11, ['E2time_L10_' num2str(nexp) '.eps'], '-deps2c', '-tiff' ); 138 | %eps2pdf(['E2time_L10_' num2str(nexp) '.eps'],'c:\program files\gs\gs9.10\bin\gswin64c.exe'); 139 | 140 | %figure;plot(rep2{1,1}.E_global); - the same as our algorithm 141 | 142 | % kernel figure 143 | % Energy distribution 144 | [v i] = sort(squeeze(sum(sum(sum(abs(A3).^2,1),2),4)),'descend'); 145 | cs = cumsum(v'); 146 | figure(q+12);plot(cs./max(cs(:)),'linewidth',2); 147 | title('Cumulative kernel energy'); 148 | xlabel('Kernel'); 149 | ylabel('Energy'); 150 | print( q+12, ['KernelEnergyCum' num2str(nexp) '.eps'], '-deps2c', '-tiff' ); 151 | %eps2pdf(['KernelEnergyCum' num2str(nexp) '.eps'],'c:\program files\gs\gs9.10\bin\gswin64c.exe'); 152 | 153 | % kernels 154 | aux = reshape(mat2cell(squeeze(H3{1}),9,9,ones(1,50)),[size(squeeze(H3{1}),3) 1]); 155 | figure;w=showmask(tileims(aux(i),2,10,1,-5),-3); 156 | %figure;w=showmask(tileims(aux,2,10,1,-5),-3); % bez trideni 157 | imwrite(w,'tiledkernels_H1.png'); 158 | 159 | %M = mat2cell(reshape(H1{1},9,[]),[9],repmat([9],[1 50])); 160 | %figure;dispIm(cell2mat(M(i))) 161 | -------------------------------------------------------------------------------- /exp_feature_learning_convergence_f.m: -------------------------------------------------------------------------------- 1 | % MULTISCALE EXPERIMENT WITH TILING ONLY 2 | %im = read_out_mesto_4(true); 3 | %save out_mesto_4 im 4 | disp('Warning: this experiment needs a lot of memory - for machines with smaller memory can be extremely slow because of swaping'); 5 | load out_mesto_4 6 | 7 | j = 1; 8 | clear In 9 | In = zeros(128,128,size(im,3)); 10 | for i=1:size(im,3) 11 | I = double(rgb2gray(squeeze(im(:,:,i,:))))/65535; 12 | I = I(101:228,101:228); % smaller crop 128x128 13 | [In(:,:,j)] = localnormalize(I,4,4); % for nature sigma = 5, otherwise sigma = 4 14 | j = j+1; 15 | end 16 | 17 | iters.maxiter_main = 1000; % max number of iterations in the main loop 18 | iters.maxiter_A = 10; % max number of iterations in the minAstep 19 | iters.maxiter_H = 10; % max number of iterations in the minHstep 20 | iters.showims = false; 21 | iters.beta = 1e3; 22 | iters.xi = 1e3; 23 | 24 | %itersConsensus = iters; 25 | %itersConsensus.beta = 1e2; 26 | %itersConsensus.xi = 1e2; 27 | 28 | nexp = 35; % experiment number 29 | L=50; %L = [1 10 50]; 30 | K=30; %K = [1:10 15:5:50 60:10:100]; Max. 59 31 | Levels = 3; 32 | KK = [8 32 64]; 33 | 34 | rep1 = cell(length(L),length(K)); % reports 35 | rep2 = cell(size(rep1)); 36 | rep3 = cell(size(rep1)); 37 | rep4 = cell(size(rep1)); 38 | t = zeros(length(L),length(K),4); 39 | iH = cell(Levels,1); 40 | for l = 1:length(L) 41 | input = In(:,:,1:L(l)); 42 | for k = 1:length(K) 43 | for i = 1:Levels 44 | iH{i} = randn(2^(i+2),2^(i+2),KK(i)); %kernels(:,:,1:K(k)); 45 | end 46 | %tic;[Un, A1, H1, rep1{l,k}] = convsparseF(input,iH,2,iters);t(l,k,1)=toc; %Bristow 47 | %tic;[Un, A3, H3, rep3{l,k}] = convsparseF(input,iH,-1,iters);t(l,k,3)=toc; % 3D Woodbury 48 | %if L == 1 49 | % tic;[Un, A4, H4, rep4{l,k}] = convsparseF(input,iH,0,itersConsensus);t(l,k,4)=toc; % cons. Woodbury 50 | % %recursion is extremely slow for more than just a few images 51 | %else 52 | %tic;[Un, A4, H4, rep4{l,k}] = convsparseF(input,iH,-2,itersConsensus);t(l,k,4)=toc; % consensus Woodb. 53 | tic;[Un, A2, H2, rep2{l,k}] = ... 54 | convsparseF(reshape(input,[size(In,1) size(In,2)*size(input,3)]),... 55 | iH,0,iters);t(l,k,2)=toc; % Tiling - Variant 1 56 | %end 57 | disp(['L=' num2str(L(l)) ', K=' num2str(K(k)) ', time: ' num2str(toc) 's']); 58 | end 59 | save(['timegraph_' num2str(nexp)],'t','rep2','H2','A2'); %tiling only 60 | end 61 | 62 | % Energy vs. time 63 | q = 200; 64 | figure(q+10); 65 | %K = [1:10 15:5:50 60:10:100]; 66 | nK = length(K); % max. number of kernels 67 | nL = length(L); % max. number of input images 68 | plot( ...%rep1{nL,nK}.timeit_global,rep1{nL,nK}.E_global,'b--',... % Bristow 69 | rep2{nL,nK}.timeit_global,rep2{nL,nK}.E_global,'k-.',... % Proposed tiling 70 | ...%rep3{nL,nK}.timeit_global,rep3{nL,nK}.E_global,'r:',... % Proposed 3D convolution 71 | ...%rep4{nL,nK}.timeit_global,rep4{nL,nK}.E_global,'g-',... % Proposed consensus 72 | 'linewidth',2); 73 | xlabel('Time'); 74 | ylabel(['Energy']); 75 | title(['Convergence, K=' num2str(K(nK)) ', L=' num2str(L(nL)) ', P=' num2str(iters.maxiter_A)]) 76 | %legend('Bristow et al.','Proposed - tiling','Proposed - 3D','Proposed - consensus','Location','NorthEast'); 77 | legend('Proposed - tiling','Location','NorthEast'); 78 | 79 | %figure;plot(rep2{1,1}.E_global); - the same as our algorithm 80 | 81 | % kernel figure 82 | % Energy distribution 83 | [v i] = sort(squeeze(sum(sum(sum(abs(A2).^2,1),2),4)),'descend'); 84 | figure;plot(squeeze(sum(sum(abs(A2),1),2)));title('Energy distribution among kernels'); % energy distribution 85 | cs = cumsum(v'); 86 | figure(q+12);plot(cs./max(cs(:)),'linewidth',2); % cumulative energy distrib. 87 | title('Cumulative kernel energy'); 88 | xlabel('Kernel'); 89 | ylabel('Energy'); 90 | print( q+12, ['KernelEnergyCum' num2str(nexp) '.eps'], '-deps2c', '-tiff' ); 91 | %eps2pdf(['KernelEnergyCum' num2str(nexp) '.eps'],'c:\program files\gs\gs9.10\bin\gswin64c.exe'); 92 | 93 | ind = 1; 94 | for iii = 1:length(H2) 95 | indend = ind + size(H2{iii},3) - 1; 96 | [v i] = sort(squeeze(sum(sum(sum(abs(A2(:,:,ind:indend)).^2,1),2),4)),'descend'); % only for one size 97 | aux = reshape(mat2cell(squeeze(H2{iii}),size(H2{iii},1),size(H2{iii},2),ones(1,size(H2{iii},3))),[size(squeeze(H2{iii}),3) 1]); 98 | figure(50+iii);w=showmask(tileims(aux(i),2,8,1,-5),-3); 99 | imwrite(w,['learnedkernels_L' num2str(size(H2{iii},1)) '_' num2str(nexp) '.png']); 100 | ind = indend+1; 101 | end 102 | -------------------------------------------------------------------------------- /localnormalize.m: -------------------------------------------------------------------------------- 1 | function [ln, m, den] = localnormalize(IM,sigma1,sigma2) 2 | %LOCALNORMALIZE A local normalization algorithm that uniformizes the local 3 | %mean and variance of an image. 4 | % ln=localnormalize(IM,sigma1,sigma2) outputs local normalization effect of 5 | % image IM using local mean and standard deviation estimated by Gaussian 6 | % kernel with sigma1 and sigma2 respectively. 7 | % 8 | % Contributed by Guanglei Xiong (xgl99@mails.tsinghua.edu.cn) 9 | % at Tsinghua University, Beijing, China. 10 | % modified by F. Sroubek in order not to require statistic toolbox 11 | epsilon=1e-1; 12 | %halfsize1=ceil(-norminv(epsilon/2,0,sigma1)); 13 | halfsize1 = ceil(sqrt(2)*sigma1*erfcinv(2*epsilon/2)); 14 | size1=2*halfsize1+1; 15 | %halfsize2=ceil(-norminv(epsilon/2,0,sigma2)); 16 | halfsize2 = ceil(sqrt(2)*sigma2*erfcinv(2*epsilon/2)); 17 | size2=2*halfsize2+1; 18 | gaussian1=fspecial('gaussian',size1,sigma1); 19 | gaussian2=fspecial('gaussian',size2,sigma2); 20 | m = imfilter(IM,gaussian1); 21 | num=IM-m; 22 | den=sqrt(imfilter(num.^2,gaussian2)); 23 | ln=num./den; -------------------------------------------------------------------------------- /mycg.m: -------------------------------------------------------------------------------- 1 | function [x flag relres iter resvec] = mycg(fh,b,defrelres,noiter,ph,x0) 2 | % CG implementation 3 | % same input and output parameters as original MATLAB pcg 4 | flag = 1; 5 | nb = sqrt(b'*b); 6 | 7 | x = x0; 8 | r=b-fh(x); 9 | if isempty(ph) 10 | z = r; 11 | else 12 | z = ph(r); 13 | end; 14 | p=z; 15 | rzold=r'*z; 16 | for i=1:noiter 17 | Ap=fh(p); 18 | alpha=rzold/(p'*Ap); 19 | x=x+alpha*p; 20 | r=r-alpha*Ap; 21 | relres = sqrt(r'*r)/nb; 22 | if relres) 9 | % 10 | %Michal Sorel (c) 2006 11 | 12 | if ~exist('k','var'), k = -4; end 13 | if ~exist('scale','var'), scale = 1; end 14 | 15 | r = normalize(shrink(m,k)); 16 | imshow(r*scale); 17 | -------------------------------------------------------------------------------- /shrink.m: -------------------------------------------------------------------------------- 1 | function r = shrink(a,k) 2 | %SHRINK reduces image a to half of size (it is done k x when k specified) 3 | % 4 | % function r = shrink(a,k) 5 | % 6 | % a ... image (can be RGB), or cell aray of images 7 | % k ... > 0 reduces k-times 8 | % k ... < 0 doubles image -k-times 9 | % 10 | % r ... shrinked or enlarged image / cell array of images 11 | % 12 | %Remark: in case of odd size it removes odd columns/rows 13 | % 14 | %History: 7. 4. 2005 'same'-> 'valid' in shrinking(l.20) 15 | % 28.2.2006 a can be a cell array 16 | % 23.5.2006 made faster, works with uint variables 17 | % 18 | %Michal Sorel (c) 2004, 2005, 2006 19 | 20 | if nargin == 1 21 | k = 1; 22 | end 23 | 24 | if iscell(a) 25 | r = cell(size(a)); 26 | for j = 1:numel(a) 27 | r{j} = shraux(a{j},k); 28 | end 29 | else 30 | r = shraux(a,k); 31 | end 32 | 33 | function r = shraux(a,k) 34 | if k >= 0 % k x zmensim na pulku 35 | if ndims(a) < 3 36 | for i = 1:k 37 | %a = conv2(a,[1,1;1,1]/4,'valid'); 38 | %a = a(1:2:end,1:2:end); 39 | a = (a(1:2:end-1,1:2:end-1)+a(1:2:end-1,2:2:end)... 40 | +a(2:2:end,1:2:end-1)+a(2:2:end,2:2:end))/4; 41 | end 42 | else 43 | for i = 1:k 44 | %for ch = 1:size(a,3) 45 | %a(:,:,ch) = conv2(a(:,:,ch),[1,1;1,1]/4,'same'); 46 | a = (a(1:2:end-1,1:2:end-1,:)+a(1:2:end-1,2:2:end,:)... 47 | +a(2:2:end,1:2:end-1,:)+a(2:2:end,2:2:end,:))/4; 48 | %end 49 | %a = a(1:2:end-1,1:2:end-1,:); 50 | end 51 | end 52 | else % k < 0 ... enlarges image 53 | if ndims(a) < 3 54 | for i = 1:-k 55 | r = zeros(2*size(a)); 56 | r(1:2:end,1:2:end) = a; 57 | r(2:2:end,1:2:end) = a; 58 | r(1:2:end,2:2:end) = a; 59 | r(2:2:end,2:2:end) = a; 60 | a = r; 61 | end 62 | else 63 | for i = 1:-k 64 | r = zeros(2*size(a,1),2*size(a,2),size(a,3)); 65 | r(1:2:end,1:2:end,:) = a; 66 | r(2:2:end,1:2:end,:) = a; 67 | r(1:2:end,2:2:end,:) = a; 68 | r(2:2:end,2:2:end,:) = a; 69 | a = r; 70 | end 71 | end 72 | end 73 | r = a; -------------------------------------------------------------------------------- /tileims.m: -------------------------------------------------------------------------------- 1 | function u = tileims(su,dims,nsz,bd,val) 2 | %TILEIMS tiles images for displaying 3 | % 4 | % function u = tileims(su,dims,nsz,border) 5 | % 6 | % su ... cell array of images (1D or 2D) 7 | % dims ... 1 ... images tiled along x-axis 8 | % ... 2 ... tilex along both axes 9 | % nsz ... width of the tiling if su is 1D and dims==2 10 | % border ... white grid of border between images 11 | % 12 | % 13 | %Michal Sorel (c) 2007 14 | 15 | if ~exist('val','var'), val = 0.5; end 16 | 17 | [m,n,q] = size(su{1}); 18 | [mc, nc] = size(su); 19 | if mc==1 | nc==1 % 1D cell array input 20 | if dims == 1 %TADY TREBA UPRAVIT podle dims==2 21 | if q>1, u = zeros([m n*length(su) q]); 22 | else u = zeros([m n*length(su)]); end 23 | for i = 1:length(su) 24 | u(:,1+(i-1)*n:i*n,:) = su{i}; 25 | end 26 | else %dims == 2 27 | mc = ceil(length(su)/nsz);nc = nsz; 28 | m = m+bd;n=n+bd; 29 | if bd>0 30 | if q>1, u = val*ones([m*mc+1 n*nc+1 q]); 31 | else u = val*ones([m*mc+1 n*nc+1]); end 32 | else 33 | if q>1, u = val*ones([m*mc n*nc q]); 34 | else u = val*ones([m*mc n*nc]); end 35 | end 36 | rest = mc*nc-length(su); 37 | for i = 1:mc 38 | for j = 1:nc 39 | u(bd+1+(i-1)*m:i*m,bd+1+(j-1)*n:j*n,:) = su{(i-1)*nsz+j}; 40 | if i==mc & j+rest>=nc 41 | break; 42 | end 43 | end 44 | if i==mc, break; end; 45 | end 46 | end 47 | else % 2D array input (ignore nsz) 48 | if exist('nsz','var') 49 | if ~isempty(nsz), error('With 2D su don''t use parameter nsz.'); end;end; 50 | %u = zeros([m*mc n*nc]); 51 | m = m+bd;n=n+bd; 52 | if bd>0 53 | if q>1, u = val*ones([m*mc+1 n*nc+1 q]); 54 | else u = val*ones([m*mc+1 n*nc+1]); end 55 | else 56 | if q>1, u = val*ones([m*mc n*nc q]); 57 | else u = val*ones([m*mc n*nc]); end 58 | end 59 | for i = 1:mc 60 | for j = 1:nc 61 | u(bd+1+(i-1)*m:i*m,bd+1+(j-1)*n:j*n,:) = su{i,j}; 62 | end 63 | end 64 | end 65 | 66 | --------------------------------------------------------------------------------