├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── helper_functions ├── Blur.m ├── ComputePSNR.m ├── CostFunc.m ├── CreateBlurAndDecimationOperator.m ├── Denoiser.m ├── MergeChannels.m └── PrepareImage.m ├── images └── examples.png ├── main.m ├── minimizers ├── RunADMM.m ├── RunFP.m └── RunSD.m └── parameters ├── GetGaussianDeblurADMMParams.m ├── GetGaussianDeblurFPParams.m ├── GetGaussianDeblurSDParams.m ├── GetSuperResADMMParams.m ├── GetSuperResFPParams.m ├── GetSuperResSDParams.m ├── GetUniformDeblurADMMParams.m ├── GetUniformDeblurFPParams.m └── GetUniformDeblurSDParams.m /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to Contribute 2 | 3 | We'd love to accept your patches and contributions to this project. There are 4 | just a few small guidelines you need to follow. 5 | 6 | ## Contributor License Agreement 7 | 8 | Contributions to this project must be accompanied by a Contributor License 9 | Agreement. You (or your employer) retain the copyright to your contribution, 10 | this simply gives us permission to use and redistribute your contributions as 11 | part of the project. Head over to to see 12 | your current agreements on file or to sign a new one. 13 | 14 | You generally only need to submit a CLA once, so if you've already submitted one 15 | (even if it was for a different project), you probably don't need to do it 16 | again. 17 | 18 | ## Code reviews 19 | 20 | All submissions, including submissions by project members, require review. We 21 | use GitHub pull requests for this purpose. Consult 22 | [GitHub Help](https://help.github.com/articles/about-pull-requests/) for more 23 | information on using pull requests. 24 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # RED: version 1.0.0 2 | 3 | Demonstration of the image restoration experiments conducted in 4 | Y. Romano, M. Elad, and P. Milanfar, "[The Little Engine that Could: 5 | Regularization by Denoising (RED)](https://epubs.siam.org/doi/10.1137/16M1102884)", 6 | SIAM Journal on Imaging Sciences, 10(4), 1804–1844, 2017 7 | [[arXiv](https://arxiv.org/abs/1611.02862)]. 8 | 9 | ![Examples](images/examples.png?raw=true) 10 | 11 | The code was tested on Windows 7 and Windows 10, with Matlab R2016a. 12 | 13 | This is not an official Google product. 14 | 15 | 16 | ## Quick start 17 | 18 | 1. Clone or download this repository. 19 | 2. Download a lightweight version of TNRD denoising from 20 | https://drive.google.com/file/d/0B9L0NyTobx_3NHdJLUtPQWJmc2c/view?usp=sharing&resourcekey=0-nelqm_IDgT2cT3E6P-TKdw. 21 | 3. Unzip the file tnrd_denoising.zip (containing 7 files in total) to the folder 22 | located in `YOUR_LOCAL_PATH/RED/tnrd_denoising`. 23 | 4. Download test images from 24 | https://drive.google.com/drive/folders/0BzCe024Ewz8aZ3pNQXNqYXJseTQ?resourcekey=0-ZCqOZsrdjG-YNRJtrureOA&usp=sharing 25 | and copy to the folder located in `YOUR_LOCAL_PATH/RED/test_images`. 26 | 5. Open Matlab and change the current folder to `YOUR_LOCAL_PATH/RED` (use "cd" function). 27 | 6. In Matlab's command window run 28 | ``` 29 | >> main 30 | ``` 31 | to start a demo. 32 | 33 | 34 | ## Third-party software components needed 35 | 36 | The code uses an image denoising algorithm called TNRD, described in 37 | Yunjin Chen, and Thomas Pock, "Trainable Nonlinear Reaction Diffusion: 38 | A Flexible Framework for Fast and Effective Image Restoration", 39 | IEEE TPAMI 2016. The TNRD code is available in the authors website 40 | http://www.escience.cn/people/chenyunjin 41 | and downloaded from 42 | https://www.dropbox.com/s/8j6b880m6ddxtee/TNRD-Codes.zip?dl=0. 43 | 44 | Note1: For a fast execution, please enable "parfor" using Matlab command "matlabpool". 45 | 46 | Note2: The TNRD software contains mex files. If you encounter problems when running the 47 | denoiser, try to download the TNRD code from the above dropbox link. 48 | Then, copy the files from the directory 49 | `YOUR_LOCAL_PATH/TNRD-Codes/TestCodes(denoising-deblocking-SR)/GaussianDenoising` 50 | to `YOUR_LOCAL_PATH/RED/tnrd_denoising` directory. 51 | If you are using mex files for the first time, please run the following in Matlab's 52 | command window: 53 | 54 | ``` 55 | >> mex -setup 56 | You will see the following question: 57 | "Would you like mex to locate installed compilers [y]/n?" 58 | >> y 59 | Now you should choose a compiler. Simply choose the first one by clicking 60 | >> 1 61 | Then verify by 62 | >> y 63 | ``` 64 | 65 | The test images, used to degrade the image, are taken from on the NCSR software. 66 | For more details, please refer to Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li 67 | "Nonlocally Centralized Sparse Representation for Image Restoration", IEEE-TIP, 2013. 68 | The NCSR code is available in http://www4.comp.polyu.edu.hk/~cslzhang/NCSR.htm. 69 | Also, to have a fair comparison, we use a similar degradation process as done in NCSR. 70 | 71 | Good luck! 72 | -------------------------------------------------------------------------------- /helper_functions/Blur.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Blurs the image x with an input psf. This process is similar to the 16 | % degradation flow of the NCSR paper 17 | function out_im = Blur(in_im, psf) 18 | 19 | cntr = (size(psf)-1)/2; 20 | 21 | % treat the borders 22 | pad_im = [in_im(:,cntr:-1:1,:), in_im, in_im(:,end:-1:end-cntr+1,:)]; 23 | pad_im = [pad_im(cntr:-1:1,:,:); pad_im; pad_im(end:-1:end-cntr+1,:,:)]; 24 | 25 | if size(in_im,3) == 3 26 | out_im(:,:,1) = conv2(pad_im(:,:,1), psf, 'valid'); 27 | out_im(:,:,2) = conv2(pad_im(:,:,2), psf, 'valid'); 28 | out_im(:,:,3) = conv2(pad_im(:,:,3), psf, 'valid'); 29 | else 30 | out_im = conv2(pad_im, psf, 'valid'); 31 | end 32 | -------------------------------------------------------------------------------- /helper_functions/ComputePSNR.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Computes the PSNR of two single-channel images 16 | 17 | function out_psnr = ComputePSNR(orig_im,est_im) 18 | 19 | err = sqrt(mean((orig_im(:)-est_im(:)).^2)); 20 | out_psnr = 20*log10(255/err); 21 | 22 | return; -------------------------------------------------------------------------------- /helper_functions/CostFunc.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Objective: 16 | % compute the function 17 | % E(x) = 1/(2sigma^2)||Hx-y||_2^2 + 0.5*lambda*x'*(x-denoise(x)) 18 | % 19 | % Inputs: 20 | % y - the input image 21 | % x_est - the recovered image 22 | % input_sigma - noise level 23 | % ForwardFunc - the degradation operator H 24 | % lambda - regularization parameter 25 | % effective_sigma - the input noise level to the denoiser 26 | % 27 | % Output: 28 | % fun_val - the value of E(x_est) 29 | 30 | function fun_val = CostFunc(y, x_est, ForwardFunc, input_sigma,... 31 | lambda, effective_sigma) 32 | 33 | f_est = Denoiser(x_est, effective_sigma); 34 | 35 | fun_val1 = sum(sum((ForwardFunc(x_est)-y).^2))/(2*input_sigma^2); 36 | fun_val2 = lambda*sum(sum(x_est.*(x_est-f_est))); 37 | 38 | fun_val = fun_val1 + fun_val2; 39 | 40 | return; 41 | 42 | -------------------------------------------------------------------------------- /helper_functions/CreateBlurAndDecimationOperator.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % This function creates a sparse matrix H that blurs (according to the 16 | % input 'psf') and decimates (by factor 'scale') a high-resolution input 17 | % image of size lr_im_sz*scale, resulting in a low-resolution image 18 | % of size lr_im_sz 19 | 20 | % Note: This code is close to the function Set_blur_matrix, which 21 | % is provided in the NCSR package, given in: 22 | % link: http://www4.comp.polyu.edu.hk/~cslzhang/NCSR.htm 23 | 24 | function H = CreateBlurAndDecimationOperator(scale, lr_im_sz, psf) 25 | 26 | hr_im_sz = lr_im_sz*scale; 27 | 28 | offset = (size(psf, 1)-1)/2; 29 | cntr = ceil(size(psf, 1)/2); 30 | 31 | tot_numel = size(psf, 1)^2 * lr_im_sz(1)*lr_im_sz(2); 32 | row_locs = zeros(tot_numel, 1); 33 | col_locs = zeros(tot_numel, 1); 34 | values = zeros(tot_numel, 1); 35 | cnt_numel = 1; 36 | 37 | locs = 1 : 1 : (hr_im_sz(1)*hr_im_sz(2)); 38 | locs = reshape(locs, [hr_im_sz(1), hr_im_sz(2)]); 39 | 40 | for lr_row = 1 : 1 : lr_im_sz(1) 41 | 42 | for lr_col = 1 : 1 : lr_im_sz(2) 43 | 44 | % find locations 45 | hr_row = (lr_row-1)*scale + 1; 46 | hr_col = (lr_col-1)*scale + 1; 47 | 48 | row_idx = (lr_col-1)*lr_im_sz(1) + lr_row; 49 | rows_vec = max(hr_row - offset,1) : min(hr_row + offset,hr_im_sz(1)); 50 | cols_vec = max(hr_col - offset,1) : min(hr_col + offset,hr_im_sz(2)); 51 | col_idx = reshape(locs(rows_vec, cols_vec), [],1); 52 | cur_numel = size(col_idx,1); 53 | row_locs(cnt_numel : cnt_numel+cur_numel-1) = row_idx; 54 | col_locs(cnt_numel : cnt_numel+cur_numel-1) = col_idx; 55 | 56 | % fill the values of the psf in the desired matrix 57 | rows_vec = cntr - (hr_row - max(hr_row - offset,1)) : cntr + min(hr_row + offset,hr_im_sz(1)) - hr_row; 58 | cols_vec = cntr - (hr_col - max(hr_col - offset,1)) : cntr + min(hr_col + offset,hr_im_sz(2)) - hr_col; 59 | crop_psf = reshape( psf(rows_vec,cols_vec), [], 1); 60 | values(cnt_numel : cnt_numel+cur_numel-1) = crop_psf/sum(crop_psf); 61 | 62 | cnt_numel = cnt_numel + cur_numel; 63 | 64 | end 65 | 66 | end 67 | 68 | row_locs = row_locs(1:cnt_numel-1); 69 | col_locs = col_locs(1:cnt_numel-1); 70 | values = values(1:cnt_numel-1); 71 | 72 | H = sparse(... 73 | row_locs,... 74 | col_locs,... 75 | values,... 76 | lr_im_sz(1)*lr_im_sz(2),... 77 | hr_im_sz(1)*hr_im_sz(2)); 78 | 79 | return 80 | 81 | -------------------------------------------------------------------------------- /helper_functions/Denoiser.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Calls a denoiser and return the cleaned image f_est. 16 | % In this example the chosen denoising method is TNRD. 17 | % Note: The devision and multiplication by the constant 5 is done to handle 18 | % a different noise-level than a fixed one (which is equal to 5) 19 | 20 | function f_est = Denoiser(x_est,effective_sigma) 21 | 22 | f_est = ReactionDiffusion(5/effective_sigma*x_est); 23 | f_est = f_est*effective_sigma/5; 24 | 25 | return 26 | 27 | -------------------------------------------------------------------------------- /helper_functions/MergeChannels.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Replaces the luminance channel of in_im with enhanced_im 16 | 17 | function out_im = MergeChannels(in_im,enhanced_im) 18 | 19 | if size(in_im,3) == 3 20 | out_im = rgb2ycbcr(uint8(in_im)); 21 | out_im(:,:,1) = enhanced_im; 22 | out_im = ycbcr2rgb(uint8(out_im)); 23 | else 24 | out_im = enhanced_im; 25 | end 26 | 27 | -------------------------------------------------------------------------------- /helper_functions/PrepareImage.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % returns the luminance channel of the input_image 16 | 17 | function luma_image = PrepareImage(input_image) 18 | 19 | if size(input_image,3) == 3 20 | ycbcr_im = rgb2ycbcr(uint8(input_image)); 21 | luma_image = double(ycbcr_im(:,:,1)); 22 | else 23 | luma_image = input_image; 24 | end 25 | 26 | luma_image = double(luma_image); 27 | 28 | -------------------------------------------------------------------------------- /images/examples.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/RED/31142ab55ad37c25f6704f5bfe81e7fec39360f0/images/examples.png -------------------------------------------------------------------------------- /main.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Demonstration of the image restoration experiments conducted in 16 | % Y. Romano, M. Elad, and P. Milanfar, "The Little Engine that Could: 17 | % Regularization by Denoising (RED)", submitted to SIAM Journal on Imaging 18 | % Sciences, 2016. https://arxiv.org/abs/1611.02862 19 | % 20 | % This example reads a ground-truth image, degrades the image by 21 | % first blurring or downscaling it, followed by an addition of random white 22 | % Gaussian noise. Then it calls to RED in order to restore the image. 23 | % This example compares the input and output PSNR, shows and saves the 24 | % results. The suggested image-adaptive Laplacian-regularization functional 25 | % is minimized using the Fixed-Point, ADMM, and Steepest Descent methods. 26 | % Please refer to the paper for more details. 27 | % 28 | % The following are the degradation models that this example handles: 29 | % 'UniformBlur' - 9X9 uniform psf with noise-level equal to sqrt(2) 30 | % 'GaussianBlur' - 25X25 Gaussian psf with std 1.6 and noise-level 31 | % equal to sqrt(2) 32 | % 'Downscale' - 7X7 Gaussian psf with std 1.6 and noise-level equal to 5 33 | % 34 | % The denoising engine is TNRD: Yunjin Chen, and Thomas Pock, "Trainable 35 | % Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective 36 | % Image Restoration", IEEE TPAMI 2016. The code is available in 37 | % http://www.icg.tugraz.at/Members/Chenyunjin/about-yunjin-chen 38 | % Note: Enable parallel pool to reduce runtime. 39 | % 40 | % The degradation process is similar to the one suggested in NCSR paper: 41 | % Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li "Nonlocally 42 | % Centralized Sparse Representation for Image Restoration", IEEE-TIP, 2013. 43 | % The code is available in http://www4.comp.polyu.edu.hk/~cslzhang/NCSR.htm 44 | % 45 | 46 | clc; 47 | clear; 48 | close all; 49 | 50 | % configure the path 51 | % denoising functions 52 | addpath(genpath('./tnrd_denoising/')); 53 | % SD, FP, and ADMM methods 54 | addpath(genpath('./minimizers/')); 55 | % contains the default params 56 | addpath(genpath('./parameters/')); 57 | % contains basic functions 58 | addpath(genpath('./helper_functions/')); 59 | % test images for the debluring and super resolution problems, 60 | % taken from NCSR software package 61 | addpath(genpath('./test_images/')); 62 | 63 | % set light_mode = true to run the code in a sub optimal but faster mode 64 | % set light_mode = false to obtain the results reported in the RED paper 65 | light_mode = false; 66 | 67 | if light_mode 68 | fprintf('Running in light mode. '); 69 | fprintf('Turn off to obatain the results reported in RED paper.\n'); 70 | else 71 | fprintf('Light mode option is off. '); 72 | fprintf('Reproducing the result in RED paper.\n'); 73 | end 74 | 75 | %% read the original image 76 | 77 | file_name = 'starfish.tif'; 78 | 79 | fprintf('Reading %s image...', file_name); 80 | orig_im = imread(['./test_images/' file_name]); 81 | orig_im = double(orig_im); 82 | 83 | fprintf(' Done.\n'); 84 | 85 | 86 | %% define the degradation model 87 | 88 | % choose the secenrio: 'UniformBlur', 'GaussianBlur', or 'Downscale' 89 | degradation_model = 'UniformBlur'; 90 | 91 | fprintf('Test case: %s degradation model.\n', degradation_model); 92 | 93 | switch degradation_model 94 | case 'UniformBlur' 95 | % noise level 96 | input_sigma = sqrt(2); 97 | % filter size 98 | psf_sz = 9; 99 | % create uniform filter 100 | psf = fspecial('average', psf_sz); 101 | % use fft to solve a system of linear equations in closed form 102 | use_fft = true; 103 | % create a function-handle to blur the image 104 | ForwardFunc = ... 105 | @(in_im) imfilter(in_im,psf,'conv','same','circular'); 106 | % the psf is symmetric, i.e., the ForwardFunc and BackwardFunc 107 | % are the same 108 | BackwardFunc = ForwardFunc; 109 | % special initialization (e.g. the output of other method) 110 | % set to identity mapping 111 | InitEstFunc = @(in_im) in_im; 112 | 113 | case 'GaussianBlur' 114 | % noise level 115 | input_sigma = sqrt(2); 116 | % filter size 117 | psf_sz = 25; 118 | % std of the Gaussian filter 119 | gaussian_std = 1.6; 120 | % create gaussian filter 121 | psf = fspecial('gaussian', psf_sz, gaussian_std); 122 | % use fft to solve a system of linear equations in closed form 123 | use_fft = true; 124 | % create a function handle to blur the image 125 | ForwardFunc = ... 126 | @(in_im) imfilter(in_im,psf,'conv','same','circular'); 127 | % the psf is symmetric, i.e., the ForwardFunc and BackwardFunc 128 | % are the same 129 | BackwardFunc = ForwardFunc; 130 | % special initialization (e.g. the output of other method) 131 | % set to identity mapping 132 | InitEstFunc = @(in_im) in_im; 133 | 134 | case 'Downscale' 135 | % noise level 136 | input_sigma = 5; 137 | % filter size 138 | psf_sz = 7; 139 | % std of the Gaussian filter 140 | gaussian_std = 1.6; 141 | % create gaussian filter 142 | psf = fspecial('gaussian', psf_sz, gaussian_std); 143 | % scaling factor 144 | scale = 3; 145 | 146 | % compute the size of the low-res image 147 | lr_im_sz = [ceil(size(orig_im,1)/scale),... 148 | ceil(size(orig_im,2)/scale)]; 149 | % create the degradation operator 150 | H = CreateBlurAndDecimationOperator(scale,lr_im_sz,psf); 151 | % downscale 152 | ForwardFunc = @(in_im) reshape(H*in_im(:),lr_im_sz); 153 | % upscale 154 | BackwardFunc = @(in_im) reshape(H'*in_im(:),scale*lr_im_sz); 155 | % special initialization (e.g. the output of other method) 156 | % use bicubic upscaler 157 | InitEstFunc = @(in_im) imresize(in_im,scale,'bicubic'); 158 | 159 | otherwise 160 | error('Degradation model is not defined'); 161 | end 162 | 163 | 164 | %% degrade the original image 165 | 166 | switch degradation_model 167 | case {'UniformBlur', 'GaussianBlur'} 168 | fprintf('Blurring...'); 169 | % blur each channel using the ForwardFunc 170 | input_im = zeros( size(orig_im) ); 171 | for ch_id = 1:size(orig_im,3) 172 | input_im(:,:,ch_id) = ForwardFunc(orig_im(:,:,ch_id)); 173 | end 174 | % use 'seed' = 0 to be consistent with the experiments in NCSR 175 | randn('seed', 0); 176 | 177 | case 'Downscale' 178 | fprintf('Downscaling...'); 179 | % blur the image, similar to the degradation process of NCSR 180 | input_im = Blur(orig_im, psf); 181 | % decimate 182 | input_im = input_im(1:scale:end,1:scale:end,:); 183 | % use 'state' = 0 to be consistent with the experiments in NCSR 184 | randn('state', 0); 185 | 186 | otherwise 187 | error('Degradation model is not defined'); 188 | end 189 | 190 | % add noise 191 | fprintf(' Adding noise...'); 192 | input_im = input_im + input_sigma*randn(size(input_im)); 193 | 194 | % convert to YCbCr color space if needed 195 | input_luma_im = PrepareImage(input_im); 196 | orig_luma_im = PrepareImage(orig_im); 197 | 198 | if strcmp(degradation_model,'Downscale') 199 | % upscale using bicubic 200 | input_im = imresize(input_im,scale,'bicubic'); 201 | input_im = input_im(1:size(orig_im,1), 1:size(orig_im,2), :); 202 | end 203 | fprintf(' Done.\n'); 204 | psnr_input = ComputePSNR(orig_im, input_im); 205 | 206 | 207 | %% minimize the Laplacian regularization functional via Fixed Point 208 | 209 | fprintf('Restoring using RED: Fixed-Point method\n'); 210 | 211 | switch degradation_model 212 | case 'UniformBlur' 213 | params_fp = GetUniformDeblurFPParams(light_mode, psf, use_fft); 214 | case 'GaussianBlur' 215 | params_fp = GetGaussianDeblurFPParams(light_mode, psf, use_fft); 216 | case 'Downscale' 217 | assert(exist('use_fft','var') == 0); 218 | params_fp = GetSuperResFPParams(light_mode); 219 | otherwise 220 | error('Degradation model is not defined'); 221 | end 222 | 223 | [est_fp_im, psnr_fp] = RunFP(input_luma_im,... 224 | ForwardFunc,... 225 | BackwardFunc,... 226 | InitEstFunc,... 227 | input_sigma,... 228 | params_fp,... 229 | orig_luma_im); 230 | out_fp_im = MergeChannels(input_im,est_fp_im); 231 | 232 | fprintf('Done.\n'); 233 | 234 | 235 | %% minimize the Laplacian regularization functional via ADMM 236 | 237 | fprintf('Restoring using RED: ADMM method\n'); 238 | 239 | switch degradation_model 240 | case 'UniformBlur' 241 | params_admm = GetUniformDeblurADMMParams(light_mode, psf, use_fft); 242 | case 'GaussianBlur' 243 | params_admm = GetGaussianDeblurADMMParams(light_mode, psf, use_fft); 244 | case 'Downscale' 245 | assert(exist('use_fft','var') == 0); 246 | params_admm = GetSuperResADMMParams(light_mode); 247 | otherwise 248 | error('Degradation model is not defined'); 249 | end 250 | 251 | [est_admm_im, psnr_admm] = RunADMM(input_luma_im,... 252 | ForwardFunc,... 253 | BackwardFunc,... 254 | InitEstFunc,... 255 | input_sigma,... 256 | params_admm,... 257 | orig_luma_im); 258 | out_admm_im = MergeChannels(input_im,est_admm_im); 259 | 260 | fprintf('Done.\n'); 261 | 262 | 263 | %% minimize the laplacian regularization functional via Steepest Descent 264 | 265 | fprintf('Restoring using RED: Steepest-Descent method\n'); 266 | 267 | switch degradation_model 268 | case 'UniformBlur' 269 | params_sd = GetUniformDeblurSDParams(light_mode); 270 | case 'GaussianBlur' 271 | params_sd = GetGaussianDeblurSDParams(light_mode); 272 | case 'Downscale' 273 | params_sd = GetSuperResSDParams(light_mode); 274 | otherwise 275 | error('Degradation model is not defined'); 276 | end 277 | 278 | [est_sd_im, psnr_sd] = RunSD(input_luma_im,... 279 | ForwardFunc,... 280 | BackwardFunc,... 281 | InitEstFunc,... 282 | input_sigma,... 283 | params_sd,... 284 | orig_luma_im); 285 | % convert back to rgb if needed 286 | out_sd_im = MergeChannels(input_im,est_sd_im); 287 | 288 | fprintf('Done.\n'); 289 | 290 | 291 | %% display final results 292 | 293 | fprintf('Image name %s \n', file_name); 294 | fprintf('Input PSNR = %f \n', psnr_input); 295 | fprintf('RED: Fixed-Point PSNR = %f \n', psnr_fp); 296 | fprintf('RED: ADMM PSNR = %f \n', psnr_admm); 297 | fprintf('RED: Steepest-Decent PSNR = %f \n', psnr_sd); 298 | 299 | 300 | %% write images 301 | 302 | if ~exist('./results/','dir') 303 | mkdir('./results/'); 304 | end 305 | 306 | fprintf('Writing the images to ./results...'); 307 | 308 | imwrite(uint8(input_im),['./results/input_' file_name]); 309 | imwrite(uint8(out_fp_im),['./results/est_fp_' file_name]); 310 | imwrite(uint8(out_admm_im),['./results/est_admm_' file_name]); 311 | imwrite(uint8(out_sd_im),['./results/est_sd_' file_name]); 312 | 313 | fprintf(' Done.\n'); 314 | 315 | -------------------------------------------------------------------------------- /minimizers/RunADMM.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Objective: 16 | % Minimize E(x) = 1/(2sigma^2)||Hx-y||_2^2 + 0.5*lambda*x'*(x-denoise(x)) 17 | % via the ADMM method. 18 | % Please refer to Section 4.2 in the paper for more details: 19 | % "Deploying the Denoising Engine for Solving Inverse Problems -- ADMM". 20 | % 21 | % Inputs: 22 | % y - the input image 23 | % ForwardFunc - the degradation operator H 24 | % BackwardFunc - the transpose of the degradation operator H 25 | % InitEstFunc - special initialization (e.g. the output of other method) 26 | % input_sigma - noise level 27 | % params.lambda - regularization parameter 28 | % params.beta - ADMM parameter 29 | % params.outer_iters - number of total iterations 30 | % params.inner_iters - number of steps to minimize Part1 of ADMM 31 | % params.use_fft - solve the linear system Az = b using FFT rather than 32 | % running gradient descent for params.inner_iters. This 33 | % feature is suppoterd only for deblurring 34 | % params.psf - the Point Spread Function (used only when 35 | % use_fft == true). 36 | % params.inner_denoiser_iters - number of steps to minimize Part2 of ADMM 37 | % params.effective_sigma - the input noise level to the denoiser 38 | % orig_im - the original image, used for PSNR evaluation ONLY 39 | 40 | % Outputs: 41 | % im_out - the reconstructed image 42 | % psnr_out - PSNR measured between x_est and orig_im 43 | 44 | function [im_out, psnr_out] = RunADMM(y, ForwardFunc, BackwardFunc,... 45 | InitEstFunc, input_sigma, params, orig_im) 46 | 47 | % print info every PRINT_MOD steps 48 | QUIET = 0; 49 | PRINT_MOD = floor(params.outer_iters/10); 50 | if ~QUIET 51 | fprintf('%7s\t%10s\t%12s\n', 'iter', 'PSNR', 'objective'); 52 | end 53 | 54 | % parameters 55 | lambda = params.lambda; 56 | beta = params.beta; 57 | outer_iters = params.outer_iters; 58 | inner_iters = params.inner_iters; 59 | inner_denoiser_iters = params.inner_denoiser_iters; 60 | effective_sigma = params.effective_sigma; 61 | 62 | % initialization 63 | x_est = InitEstFunc(y); 64 | v_est = x_est; 65 | u_est = x_est*0; 66 | Ht_y = BackwardFunc(y)/(input_sigma^2); 67 | 68 | % compute the fft of the psf (useful for deblurring) 69 | if isfield(params,'use_fft') && params.use_fft == true 70 | [h, w, ~] = size(y); 71 | fft_psf = zeros(h, w); 72 | t = floor(size(params.psf, 1)/2); 73 | fft_psf(h/2+1-t:h/2+1+t, w/2+1-t:w/2+1+t) = params.psf; 74 | fft_psf = fft2( fftshift(fft_psf) ); 75 | 76 | fft_y = fft2(y); 77 | fft_Ht_y = conj(fft_psf).*fft_y / (input_sigma^2); 78 | fft_HtH = abs(fft_psf).^2 / (input_sigma^2); 79 | end 80 | 81 | for k = 1:1:outer_iters 82 | 83 | % Part1 of the ADMM, approximates the solution of: 84 | % x = argmin_z 1/(2sigma^2)||Hz-y||_2^2 + 0.5*beta||z - v + u||_2^2 85 | if isfield(params,'use_fft') && params.use_fft == true 86 | b = fft_Ht_y + beta*fft2(v_est-u_est); 87 | A = fft_HtH + beta; 88 | x_est = real(ifft2( b./A )); 89 | x_est = max( min(x_est, 255), 0); 90 | else % use gradient descent 91 | for j = 1:1:inner_iters 92 | b = Ht_y + beta*(v_est - u_est); 93 | A_x_est = BackwardFunc(ForwardFunc(x_est))/(input_sigma^2) + beta*x_est; 94 | res = b - A_x_est; 95 | a_res = BackwardFunc(ForwardFunc(res))/(input_sigma^2) + beta*res; 96 | mu_opt = mean(res(:).*res(:))/mean(res(:).*a_res(:)); 97 | x_est = x_est + mu_opt*res; 98 | x_est = max( min(x_est, 255), 0); 99 | end 100 | end 101 | 102 | % relaxation 103 | x_hat = params.alpha*x_est + (1-params.alpha)*v_est; 104 | 105 | 106 | % Part2 of the ADMM, approximates the solution of 107 | % v = argmin_z lambda*z'*(z-denoiser(z)) + 0.5*beta||z - x - u||_2^2 108 | % using gradient descent 109 | for j = 1:1:inner_denoiser_iters 110 | f_v_est = Denoiser(v_est, effective_sigma); 111 | v_est = (beta*(x_hat + u_est) + lambda*f_v_est)/(lambda + beta); 112 | end 113 | 114 | % Part3 of the ADMM, update the dual variable 115 | u_est = u_est + x_hat - v_est; 116 | 117 | if ~QUIET && (mod(k,PRINT_MOD) == 0 || k == outer_iters) 118 | % evaluate the cost function 119 | fun_val = CostFunc(y, x_est, ForwardFunc, input_sigma,... 120 | lambda, effective_sigma); 121 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 122 | psnr_out = ComputePSNR(orig_im, im_out); 123 | fprintf('%7i %12.5f %12.5f \n', k, psnr_out, fun_val); 124 | end 125 | end 126 | 127 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 128 | psnr_out = ComputePSNR(orig_im, im_out); 129 | 130 | return 131 | 132 | -------------------------------------------------------------------------------- /minimizers/RunFP.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Objective: 16 | % Minimize E(x) = 1/(2sigma^2)||Hx-y||_2^2 + 0.5*lambda*x'*(x-denoise(x)) 17 | % via the fixed-point method. 18 | % Please refer to Section 4.2 in the paper for more details: 19 | % "Deploying the Denoising Engine for Solving Inverse Problems -- 20 | % Fixed-Point Strategy". 21 | % 22 | % Inputs: 23 | % y - the input image 24 | % ForwardFunc - the degradation operator H 25 | % BackwardFunc - the transpose of the degradation operator H 26 | % InitEstFunc - special initialization (e.g. the output of other method) 27 | % input_sigma - noise level 28 | % params.lambda - regularization parameter 29 | % params.outer_iters - number of total iterations 30 | % params.inner_iters - number of inner FP iterations 31 | % params.use_fft - solve the linear system Az = b using FFT rather than 32 | % running gradient descent for params.inner_iters. This 33 | % feature is suppoterd only for deblurring 34 | % params.psf - the Point Spread Function (used only when 35 | % use_fft == true). 36 | % params.effective_sigma - the input noise level to the denoiser 37 | % orig_im - the original image, used for PSNR evaluation ONLY 38 | % 39 | % Outputs: 40 | % im_out - the reconstructed image 41 | % psnr_out - PSNR measured between x_est and orig_im 42 | 43 | function [im_out, psnr_out] = RunFP(y, ForwardFunc, BackwardFunc,... 44 | InitEstFunc, input_sigma, params, orig_im) 45 | 46 | % print infp every PRINT_MOD steps 47 | QUIET = 0; 48 | PRINT_MOD = floor(params.outer_iters/10); 49 | if ~QUIET 50 | fprintf('%7s\t%10s\t%12s\n', 'iter', 'PSNR', 'objective'); 51 | end 52 | 53 | % parameters 54 | lambda = params.lambda; 55 | outer_iters = params.outer_iters; 56 | inner_iters = params.inner_iters; 57 | effective_sigma = params.effective_sigma; 58 | 59 | % initialization 60 | x_est = InitEstFunc(y); 61 | Ht_y = BackwardFunc(y)/(input_sigma^2); 62 | 63 | % compute the fft of the psf (useful for deblurring) 64 | if isfield(params,'use_fft') && params.use_fft == true 65 | [h, w, ~] = size(y); 66 | fft_psf = zeros(h, w); 67 | t = floor(size(params.psf, 1)/2); 68 | fft_psf(h/2+1-t:h/2+1+t, w/2+1-t:w/2+1+t) = params.psf; 69 | fft_psf = fft2( fftshift(fft_psf) ); 70 | 71 | fft_y = fft2(y); 72 | fft_Ht_y = conj(fft_psf).*fft_y / (input_sigma^2); 73 | fft_HtH = abs(fft_psf).^2 / (input_sigma^2); 74 | end 75 | 76 | % outer iterations 77 | for k = 1:1:outer_iters 78 | 79 | % apply the denoising engine 80 | f_x_est = Denoiser(x_est, effective_sigma); 81 | 82 | % solve Az = b by a variant of the SD method, where 83 | % A = 1/(sigma^2)*H'*H + lambda*I, and 84 | % b = 1/(sigma^2)*H'*y + lambda*denoiser(x_est) 85 | if isfield(params,'use_fft') && params.use_fft == true 86 | b = fft_Ht_y + lambda*fft2(f_x_est); 87 | A = fft_HtH + lambda; 88 | x_est = real(ifft2( b./A )); 89 | x_est = max( min(x_est, 255), 0); 90 | else 91 | for j=1:1:inner_iters 92 | b = Ht_y + lambda*f_x_est; 93 | ht_h_x_est = BackwardFunc(ForwardFunc(x_est))/(input_sigma^2); 94 | res = b - ht_h_x_est - lambda*x_est; 95 | A_res = BackwardFunc(ForwardFunc(res))/(input_sigma^2) + lambda*res; 96 | mu_opt = mean(res(:).*A_res(:))/mean(A_res(:).*A_res(:)); 97 | x_est = x_est + mu_opt*res; 98 | x_est = max( min(x_est, 255), 0); 99 | end 100 | end 101 | if ~QUIET && (mod(k,PRINT_MOD) == 0 || k == outer_iters) 102 | % evaluate the cost function 103 | fun_val = CostFunc(y, x_est, ForwardFunc, input_sigma,... 104 | lambda, effective_sigma); 105 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 106 | psnr_out = ComputePSNR(orig_im, im_out); 107 | fprintf('%7i %12.5f %12.5f \n', k, psnr_out, fun_val); 108 | end 109 | end 110 | 111 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 112 | psnr_out = ComputePSNR(orig_im, im_out); 113 | 114 | return 115 | 116 | -------------------------------------------------------------------------------- /minimizers/RunSD.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Objective: 16 | % Minimize E(x) = 1/(2sigma^2)||Hx-y||_2^2 + 0.5*lambda*x'*(x-denoise(x)) 17 | % via steepest descend method. 18 | % Please refer to Section 4.2 in the paper for more details: 19 | % "Deploying the Denoising Engine for Solving Inverse Problems -- 20 | % Gradient Descent Methods". 21 | % 22 | % Inputs: 23 | % y - the input image 24 | % ForwardFunc - the degradation operator H 25 | % BackwardFunc - the transpose of the degradation operator H 26 | % InitEstFunc - special initialization (e.g. the output of other method) 27 | % input_sigma - noise level 28 | % params.lambda - regularization parameter 29 | % params.outer_iters - number of total iterations 30 | % params.effective_sigma - the input noise level to the denoiser 31 | % orig_im - the original image, used for PSNR evaluation ONLY 32 | % 33 | % Outputs: 34 | % im_out - the reconstructed image 35 | % psnr_out - PSNR measured between x_est and orig_im 36 | 37 | function [im_out, psnr_out] = RunSD(y, ForwardFunc, BackwardFunc,... 38 | InitEstFunc, input_sigma, params, orig_im) 39 | 40 | % print info every PRINT_MOD steps 41 | QUIET = 0; 42 | PRINT_MOD = floor(params.outer_iters/10); 43 | if ~QUIET 44 | fprintf('%7s\t%10s\t%12s\n', 'iter', 'PSNR', 'objective'); 45 | end 46 | 47 | % parameters 48 | lambda = params.lambda; 49 | outer_iters = params.outer_iters; 50 | effective_sigma = params.effective_sigma; 51 | 52 | % compute step size 53 | mu = 2/(1/(input_sigma^2) + lambda); 54 | 55 | % initialization 56 | x_est = InitEstFunc(y); 57 | 58 | for k = 1:1:outer_iters 59 | 60 | % denoise 61 | f_x_est = Denoiser(x_est, effective_sigma); 62 | 63 | % update the solution 64 | grad1 = BackwardFunc(ForwardFunc(x_est) - y)/(input_sigma^2); 65 | grad2 = lambda*(x_est - f_x_est); 66 | x_est = x_est - mu*(grad1 + grad2); 67 | 68 | % project to [0,255] 69 | x_est = max( min(x_est, 255), 0); 70 | 71 | if ~QUIET && (mod(k,PRINT_MOD) == 0 || k == outer_iters) 72 | % evaluate the cost function 73 | fun_val = CostFunc(y, x_est, ForwardFunc, input_sigma,... 74 | lambda, effective_sigma); 75 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 76 | psnr_out = ComputePSNR(orig_im, im_out); 77 | fprintf('%7i %12.5f %12.5f \n', k, psnr_out, fun_val); 78 | end 79 | end 80 | 81 | im_out = x_est(1:size(orig_im,1), 1:size(orig_im,2)); 82 | psnr_out = ComputePSNR(orig_im, im_out); 83 | 84 | return 85 | 86 | -------------------------------------------------------------------------------- /parameters/GetGaussianDeblurADMMParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | % psf - the Point Spread Function, used by the solver if use_fft == true 19 | % this feature is supported only for deblurring 20 | 21 | function params = GetGaussianDeblurADMMParams(light_mode, psf, use_fft) 22 | 23 | params = GetGaussianDeblurFPParams(light_mode, psf, use_fft); 24 | 25 | % admm parameter 26 | params.beta = 0.001; 27 | 28 | % number of denoising applications 29 | params.inner_denoiser_iters = 1; 30 | 31 | % relaxation parameter of ADMM 32 | params.alpha = 2; 33 | 34 | return 35 | 36 | -------------------------------------------------------------------------------- /parameters/GetGaussianDeblurFPParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | % psf - the Point Spread Function, used by the solver if use_fft == true 19 | % this feature is supported only for deblurring 20 | 21 | function params = GetGaussianDeblurFPParams(light_mode, psf, use_fft) 22 | 23 | params = GetGaussianDeblurSDParams(light_mode); 24 | 25 | % number of outer iterations 26 | if light_mode 27 | params.outer_iters = 100; 28 | else 29 | params.outer_iters = 200; 30 | end 31 | 32 | % use fft for solving a linear system in a closed form 33 | params.use_fft = use_fft; 34 | params.psf = psf; 35 | 36 | % number of inner iterations, set to 'nan' if use_fft == true 37 | if use_fft 38 | params.inner_iters = nan; 39 | else 40 | params.inner_iters = 200; 41 | end 42 | 43 | return 44 | 45 | -------------------------------------------------------------------------------- /parameters/GetGaussianDeblurSDParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | 19 | function params = GetGaussianDeblurSDParams(light_mode) 20 | 21 | % regularization factor 22 | params.lambda = 0.01; 23 | % number of outer iterations 24 | if light_mode 25 | params.outer_iters = 500; 26 | else 27 | params.outer_iters = 1500; 28 | end 29 | % level of noise assumed in the regularization-denoiser 30 | params.effective_sigma = 4.1; 31 | 32 | return 33 | 34 | -------------------------------------------------------------------------------- /parameters/GetSuperResADMMParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | 19 | function params = GetSuperResADMMParams(light_mode) 20 | 21 | params = GetSuperResFPParams(light_mode); 22 | 23 | % admm parameter 24 | params.beta = 1e-3; 25 | 26 | % number of denoising applications 27 | params.inner_denoiser_iters = 1; 28 | 29 | % relaxation parameter of ADMM 30 | params.alpha = 1; 31 | 32 | 33 | return 34 | 35 | -------------------------------------------------------------------------------- /parameters/GetSuperResFPParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | 19 | function params = GetSuperResFPParams(light_mode) 20 | 21 | params = GetSuperResSDParams(light_mode); 22 | 23 | % number of outer iterations 24 | if light_mode 25 | params.outer_iters = 100; 26 | else 27 | params.outer_iters = 200; 28 | end 29 | 30 | % number of inner iterations 31 | params.inner_iters = 200; 32 | 33 | return 34 | 35 | -------------------------------------------------------------------------------- /parameters/GetSuperResSDParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | 19 | function params = GetSuperResSDParams(light_mode) 20 | 21 | % regularization factor 22 | params.lambda = 0.008; 23 | 24 | % number of outer iterations 25 | if light_mode 26 | params.outer_iters = 500; 27 | else 28 | params.outer_iters = 1500; 29 | end 30 | 31 | % level of noise assumed in the regularization-denoiser 32 | params.effective_sigma = 3; 33 | 34 | return 35 | 36 | -------------------------------------------------------------------------------- /parameters/GetUniformDeblurADMMParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | % psf - the Point Spread Function, used by the solver if use_fft == true 19 | % this feature is supported only for deblurring 20 | 21 | function params = GetUniformDeblurADMMParams(light_mode, psf, use_fft) 22 | 23 | params = GetUniformDeblurFPParams(light_mode, psf, use_fft); 24 | 25 | % admm parameter 26 | params.beta = 1e-3; 27 | 28 | % number of denoising applications 29 | params.inner_denoiser_iters = 1; 30 | 31 | % relaxation parameter of ADMM 32 | params.alpha = 2; 33 | 34 | 35 | return 36 | 37 | 38 | -------------------------------------------------------------------------------- /parameters/GetUniformDeblurFPParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | % psf - the Point Spread Function, used by the solver if use_fft == true 19 | % this feature is supported only for deblurring 20 | 21 | function params = GetUniformDeblurFPParams(light_mode, psf, use_fft) 22 | 23 | params = GetUniformDeblurSDParams(light_mode); 24 | 25 | % number of outer iterations 26 | if light_mode 27 | params.outer_iters = 100; 28 | else 29 | params.outer_iters = 200; 30 | end 31 | 32 | % use fft for solving a linear system in a closed form 33 | params.use_fft = use_fft; 34 | params.psf = psf; 35 | 36 | % number of inner iterations, set to 'nan' if use_fft == true 37 | if use_fft 38 | params.inner_iters = nan; 39 | else 40 | params.inner_iters = 200; 41 | end 42 | 43 | return 44 | 45 | 46 | 47 | 48 | 49 | -------------------------------------------------------------------------------- /parameters/GetUniformDeblurSDParams.m: -------------------------------------------------------------------------------- 1 | % Copyright 2017 Google Inc. 2 | % 3 | % Licensed under the Apache License, Version 2.0 (the "License"); 4 | % you may not use this file except in compliance with the License. 5 | % You may obtain a copy of the License at 6 | % 7 | % https://www.apache.org/licenses/LICENSE-2.0 8 | % 9 | % Unless required by applicable law or agreed to in writing, software 10 | % distributed under the License is distributed on an "AS IS" BASIS, 11 | % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | % See the License for the specific language governing permissions and 13 | % limitations under the License. 14 | 15 | % Returns set of parameters 16 | % set light_mode = true to run the code in a sub optimal but faster mode 17 | % set light_mode = false to obtain the results reported in the RED paper 18 | 19 | function params = GetUniformDeblurSDParams(light_mode) 20 | 21 | % regularization factor 22 | params.lambda = 0.02; 23 | 24 | % number of outer iterations 25 | if light_mode 26 | params.outer_iters = 500; 27 | else 28 | params.outer_iters = 1500; 29 | end 30 | 31 | % level of noise assumed in the regularization-denoiser 32 | params.effective_sigma = 3.25; 33 | 34 | --------------------------------------------------------------------------------