├── LICENSE.txt ├── README.md ├── matlab ├── dataset │ ├── amle_clean.png │ ├── amle_input.png │ ├── amle_mask.png │ ├── cahn_hilliard_clean.png │ ├── cahn_hilliard_input.png │ ├── cahn_hilliard_mask.png │ ├── harmonic_clean.png │ ├── harmonic_input.png │ ├── harmonic_mask.png │ ├── mumford_shah_clean.png │ ├── mumford_shah_input.png │ ├── mumford_shah_mask.png │ ├── transport_clean.png │ ├── transport_input.png │ └── transport_mask.png ├── demo_html.m ├── html │ ├── amle_clean.png │ ├── amle_input.png │ ├── amle_mask.png │ ├── amle_output.png │ ├── cahn_hilliard_clean.png │ ├── cahn_hilliard_input.png │ ├── cahn_hilliard_mask.png │ ├── cahn_hilliard_output.png │ ├── demo_html.html │ ├── harmonic_clean.png │ ├── harmonic_input.png │ ├── harmonic_mask.png │ ├── harmonic_output.png │ ├── mumford_shah_clean.png │ ├── mumford_shah_input.png │ ├── mumford_shah_mask.png │ ├── mumford_shah_output.png │ ├── transport_clean.png │ ├── transport_input.png │ ├── transport_mask.png │ └── transport_output.png ├── lib │ ├── create_image_and_mask.m │ ├── inpainting_amle.m │ ├── inpainting_cahn_hilliard.m │ ├── inpainting_harmonic.m │ ├── inpainting_mumford_shah.m │ └── inpainting_transport.m ├── results │ ├── amle_output.png │ ├── cahn_hilliard_output.png │ ├── harmonic_output.png │ ├── mumford_shah_output.png │ └── transport_output.png └── runme.m └── python ├── dataset ├── amle_clean.png ├── amle_input.png ├── amle_mask.png ├── cahn_hilliard_clean.png ├── cahn_hilliard_input.png ├── cahn_hilliard_mask.png ├── harmonic_clean.png ├── harmonic_input.png ├── harmonic_mask.png ├── mumford_shah_clean.png ├── mumford_shah_input.png ├── mumford_shah_mask.png ├── output_harmonic.png ├── transport_clean.png ├── transport_harmonic.png ├── transport_input.png └── transport_mask.png ├── lib ├── inpainting.py └── operators.py ├── results ├── amle_output.png ├── cahn_hilliard_output.png ├── harmonic_output.png ├── mumford_shah_output.png └── transport_output.png └── runme.py /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016-2018, Simone Parisotto 2 | Copyright (c) 2011, Carola-Bibiane Schoenlieb 3 | All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are 7 | met: 8 | 9 | * Redistributions of source code must retain the above copyright 10 | notice, this list of conditions and the following disclaimer. 11 | * Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions and the following disclaimer in 13 | the documentation and/or other materials provided with the distribution 14 | * Neither the name of the University of Cambridge nor the names 15 | of its contributors may be used to endorse or promote products derived 16 | from this software without specific prior written permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 21 | ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 22 | LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 23 | CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 24 | SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 25 | INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 26 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 27 | ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28 | POSSIBILITY OF SUCH DAMAGE. 29 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MATLAB/Python Codes for the Image Inpainting Problem 2 | [![DOI](https://zenodo.org/badge/73518110.svg)](https://zenodo.org/badge/latestdoi/73518110) [![View MATLAB/Python Codes for the Image Inpainting Problem on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/55326-matlab-python-codes-for-the-image-inpainting-problem) 3 | 4 | This is a detailed MATLAB/Python implementation of classic inpainting methods. 5 | 6 | All the scripts provided are used in the book "[Partial Differential Equation Methods for Image Inpainting](https://www.cambridge.org/core/books/partial-differential-equation-methods-for-image-inpainting/F4750CEA96C35354A97E2161130E91DC)" by Carola-Bibiane Schönlieb, Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, 2015: 7 | 8 | Please use the following entry to cite this code: 9 | 10 | ``` 11 | @software{ParSch2016, 12 | author = {Simone Parisotto and 13 | Carola-Bibiane Sch\"{o}nlieb}, 14 | title = {{MATLAB/Python Codes for the Image Inpainting 15 | Problem}}, 16 | month = dec, 17 | year = 2020, 18 | publisher = {Zenodo}, 19 | version = {3.0.0}, 20 | doi = {10.5281/zenodo.4315173}, 21 | url = {https://doi.org/10.5281/zenodo.4315173} 22 | } 23 | ``` 24 | 25 |

1) Absolute Minimizing Lipschitz Extension Inpainting (AMLE)

26 | 27 | Used to reproduce Figure 4.10 in the book. 28 | 29 | 30 | 31 | **Bibliography**: 32 | - Caselles, V., Morel, J. M., & Sbert, C. (1998). An axiomatic approach to image interpolation. Image Processing, IEEE Transactions on, 7(3), 376-386. 33 | - Almansa, A. (2002). Echantillonnage, interpolation et detection: applications en imagerie satellitaire (Doctoral dissertation, Cachan, Ecole normale superieure). 34 | 35 |

2) Harmonic Inpainting

36 | 37 | Used to reproduce Figure 2.2 in the book. This program solves harmonic inpainting via a discrete heat flow. 38 | 39 | 40 | 41 | **Bibliography**: 42 | - Shen, J., & Chan, T. F. (2002). Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3), 1019-1043. 43 | 44 |

3) Mumford-Shah Inpainting with Ambrosio-Tortorelli approximation

45 | 46 | **Note**: Used to reproduce Figure 7.3 in the book. 47 | 48 | 49 | 50 | **Bibliography**: 51 | - Esedoglu, S., & Shen, J. (2002). Digital inpainting based on the Mumford-Shah-Euler image model. European Journal of Applied Mathematics, 13(04), 353-370. 52 | 53 |

4) Cahn-Hilliard Inpainting

54 | 55 | Used to reproduce Figure 5.9 in the book. 56 | 57 | 58 | 59 | **Bibliography**: 60 | - Bertozzi, A., Esedoglu, S. & Gillette, A. (2007). Inpainting of binary images using the Cahn-Hilliard equation, IEEE Transactions on image processing 16.1 pp. 285-291 (2007). 61 | - Schoenlieb, C.-B. & Bertozzi, A. (2011). Unconditionally stable schemes for higher order inpainting, Communications in Mathematical Sciences, Volume 9, Issue 2, pp. 413-457 (2011). 62 | 63 |

5) Transport Inpainting

64 | 65 | Refer to Section 6.1 in the book. (Both Grayscale / Color Images). 66 | 67 | 68 | 69 | **Bibliography**: 70 | - Bertalmio, M. (2001). Processing of flat and non-flat image information on arbitrary manifolds using partial differential equations.PhD Thesis, 2001. 71 | 72 | ### License 73 | [BSD-3-Clause](https://opensource.org/licenses/BSD-3-Clause) 74 | -------------------------------------------------------------------------------- /matlab/dataset/amle_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/amle_clean.png -------------------------------------------------------------------------------- /matlab/dataset/amle_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/amle_input.png -------------------------------------------------------------------------------- /matlab/dataset/amle_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/amle_mask.png -------------------------------------------------------------------------------- /matlab/dataset/cahn_hilliard_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/cahn_hilliard_clean.png -------------------------------------------------------------------------------- /matlab/dataset/cahn_hilliard_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/cahn_hilliard_input.png -------------------------------------------------------------------------------- /matlab/dataset/cahn_hilliard_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/cahn_hilliard_mask.png -------------------------------------------------------------------------------- /matlab/dataset/harmonic_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/harmonic_clean.png -------------------------------------------------------------------------------- /matlab/dataset/harmonic_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/harmonic_input.png -------------------------------------------------------------------------------- /matlab/dataset/harmonic_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/harmonic_mask.png -------------------------------------------------------------------------------- /matlab/dataset/mumford_shah_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/mumford_shah_clean.png -------------------------------------------------------------------------------- /matlab/dataset/mumford_shah_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/mumford_shah_input.png -------------------------------------------------------------------------------- /matlab/dataset/mumford_shah_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/mumford_shah_mask.png -------------------------------------------------------------------------------- /matlab/dataset/transport_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/transport_clean.png -------------------------------------------------------------------------------- /matlab/dataset/transport_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/transport_input.png -------------------------------------------------------------------------------- /matlab/dataset/transport_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/dataset/transport_mask.png -------------------------------------------------------------------------------- /matlab/demo_html.m: -------------------------------------------------------------------------------- 1 | %% MATLAB Codes for the Image Inpainting Problem 2 | % All the scripts provided are used in 3 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 4 | % 5 | % Authors: 6 | % Simone Parisotto (email: sp751 at cam dot ac dot uk) 7 | % Carola-Bibiane Schoenlieb (email: cbs31 at cam dot ac dot uk) 8 | % 9 | % Address: 10 | % Cambridge Image Analysis 11 | % Centre for Mathematical Sciences 12 | % Wilberforce Road 13 | % CB3 0WA, Cambridge, United Kingdom 14 | % 15 | % Date: 16 | % September, 2016 17 | % 18 | % Last update: 19 | % October, 2018 20 | % 21 | % Licence: BSD-3-Clause () 22 | % 23 | 24 | %% How to cite this work 25 | % Please use the following entry to cite this code: 26 | % 27 | %% 28 | % @Misc{MATLABinpainting2016, 29 | % author = {Parisotto, Simone and Sch\"{o}nlieb, Carola}, 30 | % title = {MATLAB Codes for the {Image} {Inpainting} {Problem}}, 31 | % howpublished = {GitHub repository, {MATLAB} Central File Exchange}, 32 | % month = {September}, 33 | % year = {2016} 34 | % } 35 | 36 | %% 1) AMLE (Absolute Minimizing Lipschitz Extension) Inpainting 37 | % Function used to reproduce Figure 4.10 in 38 | % 39 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 40 | 41 | cleanfilename = 'amle_clean.png'; 42 | %% 43 | % amle_clean.png : 44 | % 45 | % <> 46 | 47 | maskfilename = 'amle_mask.png'; 48 | %% 49 | % amle_mask.png : 50 | % 51 | % <> 52 | 53 | [u,mask] = create_image_and_mask(cleanfilename,maskfilename); 54 | imwrite(u,'./dataset/amle_input.png') 55 | %% 56 | % amle_input.png : 57 | % 58 | % <> 59 | 60 | % parameters 61 | lambda = 10^2; 62 | tol = 1e-8; 63 | maxiter = 40000; 64 | dt = 0.01; 65 | 66 | % inpainting 67 | inpainting_amle(u,mask,lambda,tol,maxiter,dt); 68 | %% 69 | % Result image: 70 | % 71 | % <> 72 | % 73 | 74 | %% 75 | % *Tic/Toc time*: Elapsed time is 34.93 seconds. 76 | % 77 | 78 | %% 79 | % See the code 80 | % . 81 | 82 | %% 83 | % *Bibliography* 84 | % 85 | % * Caselles, V., Morel, J. M., & Sbert, C. (1998). An axiomatic approach to image interpolation. Image Processing, IEEE Transactions on, 7(3), 376-386. 86 | % * Almansa, A. (2002). Echantillonnage, interpolation et detection: applications en imagerie satellitaire (Doctoral dissertation, Cachan, Ecole normale superieure). 87 | 88 | 89 | %% 2) Harmonic Inpainting 90 | % Function used to reproduce Figure 2.2 in 91 | % 92 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 93 | 94 | cleanfilename = 'harmonic_clean.png'; 95 | %% 96 | % harmonic_clean.png : 97 | % 98 | % <> 99 | 100 | maskfilename = 'harmonic_mask.png'; 101 | %% 102 | % harmonic_mask.png : 103 | % 104 | % <> 105 | [u,mask] = create_image_and_mask(cleanfilename,maskfilename); 106 | imwrite(u,'./dataset/harmonic_input.png') 107 | %% 108 | % harmonic_input.png : 109 | % 110 | % <> 111 | 112 | % parameters 113 | lambda = 10; 114 | tol = 1e-5; 115 | maxiter = 500; 116 | dt = 0.1; 117 | 118 | % inpainting 119 | inpainting_harmonic(u,mask,lambda,tol,maxiter,dt); 120 | %% 121 | % Result image: 122 | % 123 | % <> 124 | % 125 | 126 | %% 127 | % *Tic/Toc time*: Elapsed time is 2.14 seconds. 128 | % 129 | 130 | %% 131 | % See the code 132 | % . 134 | 135 | %% 136 | % *Bibliography* 137 | % 138 | % * Shen, J., & Chan, T. F. (2002). Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3), 1019-1043. 139 | 140 | %% 3) Mumford-Shah Inpainting 141 | % Function used to reproduce Figure 7.3 in 142 | % 143 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 144 | 145 | cleanfilename = 'mumford_shah_clean.png'; 146 | %% 147 | % mumford_shah_clean.png : 148 | % 149 | % <> 150 | % 151 | maskfilename = 'mumford_shah_mask.png'; 152 | %% 153 | % mumford_shah_mask.png : 154 | % 155 | % <> 156 | % 157 | [u,mask] = create_image_and_mask(cleanfilename,maskfilename); 158 | imwrite(u,'./dataset/mumford_shah_input.png') 159 | %% 160 | % mumford_shah_input.png : 161 | % 162 | % <> 163 | % 164 | 165 | % parameters 166 | maxiter = 20; 167 | tol = 1e-14; 168 | param.lambda = 10^9; % weight on data fidelity (should usually be large). 169 | param.alpha = 1; % regularisation parameters \alpha. 170 | param.gamma = 0.5; % regularisation parameters \gamma. 171 | param.epsilon = 0.05; % accuracy of Ambrosio-Tortorelli approximation of the edge set. 172 | 173 | % inpainting 174 | inpainting_mumford_shah(u,mask,maxiter,tol,param); 175 | %% 176 | % Result image: 177 | % 178 | % <> 179 | % 180 | 181 | %% 182 | % *Tic/Toc time*: Elapsed time is 48.99 seconds. 183 | % 184 | 185 | %% 186 | % See the code 187 | % . 189 | 190 | %% 191 | % *Bibliography* 192 | % 193 | % * Esedoglu, S., & Shen, J. (2002). Digital inpainting based on the Mumford-Shah-Euler image model. European Journal of Applied Mathematics, 13(04), 353-370. 194 | 195 | %% 4) Cahn-Hilliard Inpainting 196 | % Function used to reproduce Figure 5.9 in 197 | % 198 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 199 | 200 | cleanfilename = 'cahn_hilliard_clean.png'; 201 | %% 202 | % cahn_hilliard_clean.png : 203 | % 204 | % <> 205 | % 206 | maskfilename = 'cahn_hilliard_mask.png'; 207 | %% 208 | % cahn_hilliard_mask.png : 209 | % 210 | % <> 211 | % 212 | [u,mask] = create_image_and_mask(cleanfilename,maskfilename); 213 | imwrite(u,'./dataset/cahn_hilliard_input.png') 214 | %% 215 | % cahn_hilliard_input.png : 216 | % 217 | % <> 218 | % 219 | 220 | % parameters 221 | maxiter = 4000; 222 | param.epsilon = [100 1]; 223 | param.lambda = 10; 224 | param.dt = 1; 225 | 226 | % inpainting 227 | inpainting_cahn_hilliard(u,mask,maxiter,param); 228 | %% 229 | % Result image: 230 | % 231 | % <> 232 | % 233 | 234 | %% 235 | % *Tic/Toc time*: Elapsed time is 8.35 seconds. 236 | % 237 | 238 | %% 239 | % See the code 240 | % . 242 | 243 | %% 244 | % *Bibliography* 245 | % 246 | % * Bertozzi, A., Esedoglu, S. & Gillette, A. (2007). Inpainting of binary images using the Cahn-Hilliard equation, IEEE Transactions on image processing 16.1 pp. 285-291 (2007). 247 | % * Schoenlieb, C.-B. & Bertozzi, A. (2011). Unconditionally stable schemes for higher order inpainting, Communications in Mathematical Sciences, Volume 9, Issue 2, pp. 413-457 (2011). 248 | 249 | 250 | %% 5) Transport Inpainting 251 | % Function used to reproduce Figure 6.1 in 252 | % 253 | % (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015). 254 | 255 | cleanfilename = 'transport_clean.png'; 256 | %% 257 | % transport_clean.png : 258 | % 259 | % <> 260 | % 261 | maskfilename = 'transport_mask.png'; 262 | %% 263 | % transport_mask.png : 264 | % 265 | % <> 266 | % 267 | [u,mask] = create_image_and_mask(cleanfilename,maskfilename); 268 | imwrite(u,'./dataset/transport_input.png') 269 | %% 270 | % transport_input.png : 271 | % 272 | % <> 273 | % 274 | 275 | % parameters 276 | tol = 1e-5; 277 | maxiter = 50; 278 | dt = 0.1; 279 | param.M = 40; % number of steps of the inpainting procedure; 280 | param.N = 2; % number of steps of the anisotropic diffusion; 281 | param.eps = 1e-10; 282 | 283 | % inpainting 284 | inpainting_transport(u,mask,maxiter,tol,dt,param); 285 | %% 286 | % Results image: 287 | % 288 | % <> 289 | % 290 | 291 | %% 292 | % *Tic/Toc time*: Elapsed time is 151.69 seconds. 293 | % 294 | 295 | %% 296 | % See the code 297 | % . 299 | %% 300 | % *Bibliography* 301 | % 302 | % * Bertalmio, M. (2001). Processing of flat and non-flat image information on arbitrary manifolds using partial differential equations.PhD Thesis, 2001. 303 | 304 | %% M-file that created this page 305 | % <./demo_html.m demo_html.m> -------------------------------------------------------------------------------- /matlab/html/amle_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/amle_clean.png -------------------------------------------------------------------------------- /matlab/html/amle_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/amle_input.png -------------------------------------------------------------------------------- /matlab/html/amle_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/amle_mask.png -------------------------------------------------------------------------------- /matlab/html/amle_output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/amle_output.png -------------------------------------------------------------------------------- /matlab/html/cahn_hilliard_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/cahn_hilliard_clean.png -------------------------------------------------------------------------------- /matlab/html/cahn_hilliard_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/cahn_hilliard_input.png -------------------------------------------------------------------------------- /matlab/html/cahn_hilliard_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/cahn_hilliard_mask.png -------------------------------------------------------------------------------- /matlab/html/cahn_hilliard_output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/cahn_hilliard_output.png -------------------------------------------------------------------------------- /matlab/html/demo_html.html: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 6 | MATLAB Codes for the Image Inpainting Problem

MATLAB Codes for the Image Inpainting Problem

All the scripts provided are used in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

Authors: Simone Parisotto (email: sp751 at cam dot ac dot uk) Carola-Bibiane Schoenlieb (email: cbs31 at cam dot ac dot uk)

Address: Cambridge Image Analysis Centre for Mathematical Sciences Wilberforce Road CB3 0WA, Cambridge, United Kingdom

Date: September, 2016

Last update: October, 2018

Licence: BSD-3-Clause (https://opensource.org/licenses/BSD-3-Clause)

Contents

How to cite this work

Please use the following entry to cite this code:

@Misc{MATLABinpainting2016,
 70 |  author       = {Parisotto, Simone and Sch\"{o}nlieb, Carola},
 71 |  title        = {MATLAB Codes for the {Image} {Inpainting} {Problem}},
 72 |  howpublished = {GitHub repository, {MATLAB} Central File Exchange},
 73 |  month        = {September},
 74 |  year         = {2016}
 75 | }

1) AMLE Inpainting (Absolute Minimizing Lipschitz Extension Inpainting)

Function used to reproduce Figure 4.10 in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

cleanfilename = 'amle_clean.png';
 76 |

amle_clean.png :

maskfilename  = 'amle_mask.png';
 77 |

amle_mask.png :

[u,mask]      = create_image_and_mask(cleanfilename,maskfilename);
 78 | imwrite(u,'./dataset/amle_input.png')
 79 |

amle_input.png :

% parameters
 80 | lambda        = 10^2;
 81 | tol           = 1e-8;
 82 | maxiter       = 40000;
 83 | dt            = 0.01;
 84 | 
 85 | % inpainting
 86 | inpainting_amle(u,mask,lambda,tol,maxiter,dt);
 87 |

Result image:

Tic/Toc time: Elapsed time is 34.93 seconds.

See the code inpainting_amle.m.

Bibliography

  • Caselles, V., Morel, J. M., & Sbert, C. (1998). An axiomatic approach to image interpolation. Image Processing, IEEE Transactions on, 7(3), 376-386.
  • Almansa, A. (2002). Echantillonnage, interpolation et detection: applications en imagerie satellitaire (Doctoral dissertation, Cachan, Ecole normale superieure).

2) Harmonic Inpainting

Function used to reproduce Figure 2.2 in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

cleanfilename = 'harmonic_clean.png';
 88 |

harmonic_clean.png :

maskfilename  = 'harmonic_mask.png';
 89 |

harmonic_mask.png :

[u,mask]      = create_image_and_mask(cleanfilename,maskfilename);
 90 | imwrite(u,'./dataset/harmonic_input.png')
 91 |

harmonic_input.png :

% parameters
 92 | lambda        = 10;
 93 | tol           = 1e-5;
 94 | maxiter       = 500;
 95 | dt            = 0.1;
 96 | 
 97 | % inpainting
 98 | inpainting_harmonic(u,mask,lambda,tol,maxiter,dt);
 99 |

Result image:

Tic/Toc time: Elapsed time is 2.14 seconds.

See the code inpainting_harmonic.m.

Bibliography

  • Shen, J., & Chan, T. F. (2002). Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3), 1019-1043.

3) Mumford-Shah Inpainting

Function used to reproduce Figure 7.3 in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

cleanfilename = 'mumford_shah_clean.png';
100 |

mumford_shah_clean.png :

maskfilename  = 'mumford_shah_mask.png';
101 |

mumford_shah_mask.png :

[u,mask]      = create_image_and_mask(cleanfilename,maskfilename);
102 | imwrite(u,'./dataset/mumford_shah_input.png')
103 |

mumford_shah_input.png :

% parameters
104 | maxiter       = 20;
105 | tol           = 1e-14;
106 | param.lambda  = 10^9;   % weight on data fidelity (should usually be large).
107 | param.alpha   = 1;      % regularisation parameters \alpha.
108 | param.gamma   = 0.5;    % regularisation parameters \gamma.
109 | param.epsilon = 0.05;   % accuracy of Ambrosio-Tortorelli approximation of the edge set.
110 | 
111 | % inpainting
112 | inpainting_mumford_shah(u,mask,maxiter,tol,param);
113 |

Result image:

Tic/Toc time: Elapsed time is 48.99 seconds.

See the code inpainting_mumford_shah.m.

Bibliography

  • Esedoglu, S., & Shen, J. (2002). Digital inpainting based on the Mumford-Shah-Euler image model. European Journal of Applied Mathematics, 13(04), 353-370.

4) Cahn-Hilliard Inpainting

Function used to reproduce Figure 5.9 in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

cleanfilename = 'cahn_hilliard_clean.png';
114 |

cahn_hilliard_clean.png :

maskfilename  = 'cahn_hilliard_mask.png';
115 |

cahn_hilliard_mask.png :

[u,mask]      = create_image_and_mask(cleanfilename,maskfilename);
116 | imwrite(u,'./dataset/cahn_hilliard_input.png')
117 |

cahn_hilliard_input.png :

% parameters
118 | maxiter       = 4000;
119 | param.epsilon = [100 1];
120 | param.lambda  = 10;
121 | param.dt      = 1;
122 | 
123 | % inpainting
124 | inpainting_cahn_hilliard(u,mask,maxiter,param);
125 |

Result image:

Tic/Toc time: Elapsed time is 8.35 seconds.

See the code inpainting_cahn_hilliard.m.

Bibliography

  • Bertozzi, A., Esedoglu, S. & Gillette, A. (2007). Inpainting of binary images using the Cahn-Hilliard equation, IEEE Transactions on image processing 16.1 pp. 285-291 (2007).
  • Schoenlieb, C.-B. & Bertozzi, A. (2011). Unconditionally stable schemes for higher order inpainting, Communications in Mathematical Sciences, Volume 9, Issue 2, pp. 413-457 (2011).

5) Transport Inpainting

Function used to reproduce Figure 6.1 in Partial Differential Equation Methods for Image Inpainting (Carola-Bibiane Schoenlieb, Cambridge University Press, 2015).

cleanfilename = 'transport_clean.png';
126 |

transport_clean.png :

maskfilename  = 'transport_mask.png';
127 |

transport_mask.png :

[u,mask]      = create_image_and_mask(cleanfilename,maskfilename);
128 | imwrite(u,'./dataset/transport_input.png')
129 |

transport_input.png :

% parameters
130 | tol           = 1e-5;
131 | maxiter       = 50;
132 | dt            = 0.1;
133 | param.M       = 40; % number of steps of the inpainting procedure;
134 | param.N       = 2;  % number of steps of the anisotropic diffusion;
135 | param.eps     = 1e-10;
136 | 
137 | % inpainting
138 | inpainting_transport(u,mask,maxiter,tol,dt,param);
139 |

Results image:

Tic/Toc time: Elapsed time is 151.69 seconds.

See the code inpainting_transport.m.

Bibliography

  • Bertalmio, M. (2001). Processing of flat and non-flat image information on arbitrary manifolds using partial differential equations.PhD Thesis, 2001.

M-file that created this page

demo_html.m


Published with MATLAB® R2016b

-------------------------------------------------------------------------------- /matlab/html/harmonic_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/harmonic_clean.png -------------------------------------------------------------------------------- /matlab/html/harmonic_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/harmonic_input.png -------------------------------------------------------------------------------- /matlab/html/harmonic_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/harmonic_mask.png -------------------------------------------------------------------------------- /matlab/html/harmonic_output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/harmonic_output.png -------------------------------------------------------------------------------- /matlab/html/mumford_shah_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/mumford_shah_clean.png -------------------------------------------------------------------------------- /matlab/html/mumford_shah_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/mumford_shah_input.png -------------------------------------------------------------------------------- /matlab/html/mumford_shah_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/mumford_shah_mask.png -------------------------------------------------------------------------------- /matlab/html/mumford_shah_output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/mumford_shah_output.png -------------------------------------------------------------------------------- /matlab/html/transport_clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/transport_clean.png -------------------------------------------------------------------------------- /matlab/html/transport_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/transport_input.png -------------------------------------------------------------------------------- /matlab/html/transport_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/transport_mask.png -------------------------------------------------------------------------------- /matlab/html/transport_output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/simoneparisotto/MATLAB-Python-inpainting-codes/030bb3d6957a908210ab22522fbacb4166f86212/matlab/html/transport_output.png -------------------------------------------------------------------------------- /matlab/lib/create_image_and_mask.m: -------------------------------------------------------------------------------- 1 | %% Create data to be inpainted 2 | % part of "MATLAB Codes for the Image Inpainting Problem" 3 | % 4 | % Usage: 5 | % [u,mask,input] = create_image_and_mask(imagefilename,maskfilename) 6 | % 7 | % Authors: 8 | % Simone Parisotto (email: sp751 at cam dot ac dot uk) 9 | % Carola-Bibiane Schoenlieb (email: cbs31 at cam dot ac dot uk) 10 | % 11 | % Address: 12 | % Cambridge Image Analysis 13 | % Centre for Mathematical Sciences 14 | % Wilberforce Road 15 | % CB3 0WA, Cambridge, United Kingdom 16 | % 17 | % Date: 18 | % September, 2016 19 | % 20 | % Licence: BSD-3-Clause (https://opensource.org/licenses/BSD-3-Clause) 21 | % 22 | 23 | 24 | function [u,mask,input] = create_image_and_mask(imagefilename,maskfilename) 25 | 26 | % import a clean input to be corrupted with the mask 27 | input = im2double(imread(imagefilename)); 28 | 29 | % import the mask of the inpainting domain 30 | % mask = 1 intact part 31 | % mask = 0 missing domain 32 | mask = double( mat2gray( im2double(imread(maskfilename)) ) == 1 ); 33 | if size(mask,3)==1 && size(input,3)>1 34 | mask = repmat(mask,[1,1,size(input,3)]); 35 | end 36 | 37 | % create the image with the missin domain: 38 | noise = rand(size(input)); 39 | u = mask.*input + (1-mask).*noise; -------------------------------------------------------------------------------- /matlab/lib/inpainting_amle.m: -------------------------------------------------------------------------------- 1 | %% AMLE Inpainting 2 | % part of "MATLAB Codes for the Image Inpainting Problem" 3 | % 4 | % Usage: 5 | % u = inpainting_amle(u,mask,lambda,tol,maxiter,dt) 6 | % 7 | % Authors: 8 | % Simone Parisotto (email: sp751 at cam dot ac dot uk) 9 | % Carola-Bibiane Schoenlieb (email: cbs31 at cam dot ac dot uk) 10 | % 11 | % Address: 12 | % Cambridge Image Analysis 13 | % Centre for Mathematical Sciences 14 | % Wilberforce Road 15 | % CB3 0WA, Cambridge, United Kingdom 16 | % 17 | % Date: 18 | % September, 2016 19 | % 20 | % Licence: BSD-3-Clause (https://opensource.org/licenses/BSD-3-Clause) 21 | % 22 | 23 | function u = inpainting_amle(input,mask,lambda,tol,maxiter,dt) 24 | 25 | [M,N,C] = size(input); 26 | 27 | %% GRADIENT 28 | % GRID INTERVAL FOR AXIS ij 29 | h1 = 1; h2 = 1; 30 | 31 | % average upper (u_{i+1,j} - u_{ij})/hx and lower (u_{ij}-u_{i-1,j})/hx 32 | d1i_forward = spdiags([-ones(M,1),ones(M,1)],[0,1],M,M)/h1; 33 | d1i_backward = spdiags([-ones(M,1),ones(M,1)],[-1,0],M,M)/h1; 34 | % average upper (u_{i,j+1} - u_{ij})/hy and lower (u_{ij}-u_{i,j-1})/hy 35 | d1j_forward = spdiags([-ones(N,1),ones(N,1)],[0,1],N,N)/h2; 36 | d1j_backward = spdiags([-ones(N,1),ones(N,1)],[-1,0],N,N)/h2; 37 | 38 | % BACKWARD WITHOUT BOUNDARY CONDITIONS (FORWARD NOT OF INTEREST HERE) 39 | % DD1i_forward = kron(speye(N),d1i_forward); 40 | DD1i_backward = kron(speye(N),d1i_backward); 41 | % DD1j_forward = kron(d1j_forward,speye(M)); 42 | DD1j_backward = kron(d1j_backward,speye(M)); 43 | 44 | % NEUMANN BOUNDARY CONDITION 45 | d1i_forward(end,:) = 0; d1i_backward(1,:) = 0; 46 | d1j_forward(end,:) = 0; d1j_backward(1,:) = 0; 47 | 48 | % FORWARD AND BACKWARD WITH BOUNDARY CONDITIONS 49 | D1i_forward = kron(speye(N),d1i_forward); 50 | D1i_backward = kron(speye(N),d1i_backward); 51 | D1j_forward = kron(d1j_forward,speye(M)); 52 | D1j_backward = kron(d1j_backward,speye(M)); 53 | 54 | % CENTERED WITH BOUNDARY CONDITIONS 55 | D1i_centered = (D1i_forward+D1i_backward)/2; 56 | D1j_centered = (D1j_forward+D1j_backward)/2; 57 | 58 | %% FREE MEMORY 59 | clear d1i_forward d1i_backward d1j_forward d1j_backward 60 | 61 | %% INPAINTING ALGORITHM 62 | % INITIALIZATION 63 | u = reshape(input,M*N,C); 64 | input = reshape(input,M*N,C); 65 | mask = reshape(mask,M*N,C); 66 | 67 | v = zeros(M*N,2); 68 | 69 | % FOR EACH COLOR CHANNEL 70 | for c = 1:C 71 | 72 | % ITERATION 73 | for iter = 1:maxiter 74 | ux = D1i_forward*u(:,c); % forward differences along i 75 | uy = D1j_forward*u(:,c); % forward differences along j 76 | 77 | % second derivatives 78 | uxx = DD1i_backward*ux; 79 | uxy = DD1j_backward*ux; 80 | uyx = DD1i_backward*uy; 81 | uyy = DD1j_backward*uy; 82 | 83 | % create direction field Du/|Du| with central differences 84 | v(:,1) = D1i_centered*u(:,c); 85 | v(:,2) = D1j_centered*u(:,c); 86 | % normalize the direction field 87 | v = bsxfun(@rdivide,v,sqrt(sum(v.^2,2))); 88 | v(isnan(v)) = 0; 89 | 90 | % CORE ITERATION 91 | unew = u(:,c) + dt*(uxx.*v(:,1).^2+uyy.*v(:,2).^2 + (uxy+uyx) .* (v(:,1).*v(:,2)) + lambda*mask(:,c).*(input(:,c)-u(:,c))); 92 | 93 | % COMPUTE EXIT CONDITION 94 | diff = norm(unew-u(:,c))/norm(unew); 95 | 96 | % UPDATE 97 | u(:,c) = unew; 98 | 99 | % TEST EXIT CONDITION 100 | if diff D QuQ + QuQ D, where 60 | % Q is nonsingular, the matrix of eigenvectors of L and D is a diagonal matrix. 61 | % We have to compute QuQ. This we can do in a fast way by using the fft-transform: 62 | 63 | Lambda1 = spdiags(2*(cos(2*(0:M-1)'*pi/M)-1),0,M,M)/h1^2; 64 | Lambda2 = spdiags(2*(cos(2*(0:N-1)'*pi/N)-1),0,N,N)/h2^2; 65 | 66 | Denominator = Lambda1*ones(M,N) + ones(M,N)*Lambda2; 67 | 68 | % Now we can write the above equation in much simpler way and compute the 69 | % solution u_hat 70 | 71 | % FOR EACH COLOR CHANNEL 72 | for c = 1:C 73 | 74 | % Initialization of Fourier transform: 75 | u_hat = fft2(u(:,:,c)); 76 | lu0_hat = fft2(lambda(:,:,c).*u(:,:,c)); 77 | 78 | for it = 1:maxiter 79 | 80 | lu_hat = fft2(lambda(:,:,c).*u(:,:,c)); 81 | Fprime_hat = fft2(2*(2*u(:,:,c).^3-3*u(:,:,c).^2+u(:,:,c))); 82 | 83 | % CH-inpainting 84 | u_hat = (dt*(1+lambda(:,:,c)-Denominator/param.epsilon(2)).*u_hat... 85 | + dt/ep(it)*Denominator.*Fprime_hat... 86 | + dt*(lu0_hat-lu_hat))./(1+lambda(:,:,c)*dt+ep(it)*dt*Denominator.^2-dt*Denominator/param.epsilon(2)); 87 | 88 | u(:,:,c) = real(ifft2(u_hat)); 89 | 90 | end 91 | 92 | end 93 | 94 | %% WRITE IMAGE OUTPUTS 95 | imwrite(u,'./results/cahn_hilliard_output.png') 96 | -------------------------------------------------------------------------------- /matlab/lib/inpainting_harmonic.m: -------------------------------------------------------------------------------- 1 | %% HARMONIC Inpainting 2 | % part of "MATLAB Codes for the Image Inpainting Problem" 3 | % 4 | % Usage: 5 | % u = inpainting_harmonic(u,mask,lambda,tol,maxiter,dt) 6 | % 7 | % Authors: 8 | % Simone Parisotto (email: sp751 at cam dot ac dot uk) 9 | % Carola-Bibiane Schoenlieb (email: cbs31 at cam dot ac dot uk) 10 | % 11 | % Address: 12 | % Cambridge Image Analysis 13 | % Centre for Mathematical Sciences 14 | % Wilberforce Road 15 | % CB3 0WA, Cambridge, United Kingdom 16 | % 17 | % Date: 18 | % September, 2016 19 | % 20 | % Licence: BSD-3-Clause (https://opensource.org/licenses/BSD-3-Clause) 21 | % 22 | 23 | function u = inpainting_harmonic(input,mask,lambda,tol,maxiter,dt) 24 | 25 | [M,N,C] = size(input); 26 | 27 | %% LAPLACIAN 28 | % GRID INTERVAL FOR AXIS ij 29 | h1 = 1; h2 = 1; 30 | 31 | d2i = toeplitz(sparse([1,1],[1,2],[-2,1]/h1^2,1,M)); 32 | d2j = toeplitz(sparse([1,1],[1,2],[-2,1]/h2^2,1,N)); 33 | % NEUMANN BOUNDARY CONDITIONS 34 | d2i(1,[1 2]) = [-1 1]/h1; 35 | d2i(end,[end-1 end]) = [1 -1]/h1; 36 | d2j(1,[1 2]) = [-1 1]/h2; 37 | d2j(end,[end-1 end]) = [1 -1]/h2; 38 | % 2D domain LAPLACIAN 39 | L = kron(speye(N),d2i)+kron(d2j,speye(M)); 40 | 41 | %% ------------------------------------------------------------ FREE MEMORY 42 | clear d2i d2j 43 | 44 | %% INPAINTINH ALGORITHM 45 | 46 | % INITIALIZATION 47 | u = input; 48 | f = input; 49 | 50 | % FOR EACH COLOR CHANNEL 51 | for c = 1:C 52 | 53 | % ITERATION 54 | for iter = 1:maxiter 55 | 56 | % COMPUTE NEW SOLUTION 57 | laplacian = reshape( L*reshape(u(:,:,c),[],1) ,M,N); 58 | unew = u(:,:,c) + dt*( laplacian + lambda*mask(:,:,c).*(f(:,:,c)-u(:,:,c)) ); 59 | 60 | % COMPUTE EXIT CONDITION 61 | diff = norm(unew(:)-reshape(u(:,:,c),[],1))/norm(unew(:)); 62 | 63 | % UPDATE 64 | u(:,:,c) = unew; 65 | 66 | % TEST EXIT CONDITION 67 | if diff0); 126 | 127 | uxf = matrices.Dif*u(:,c); 128 | uxb = matrices.Dib*u(:,c); 129 | uyf = matrices.Djf*u(:,c); 130 | uyb = matrices.Djb*u(:,c); 131 | 132 | slopelim = betapos .* sqrt(min(uxb,0).^2 + max(uxf,0).^2 + min(uyb,0).^2 + max(uyf,0).^2)... 133 | + (~betapos) .* sqrt(max(uxb,0).^2 + min(uxf,0).^2 + max(uyb,0).^2 + min(uyf,0).^2); 134 | 135 | update = beta .* slopelim; 136 | 137 | % OPTIONAL: 138 | % Nonlinear scaling of the equation proposed by Bertalmio in 139 | % his thesis. It might cause instabilities when choosing dt 140 | % too large. 141 | % signo = sign(update); 142 | % update = signo.*sqrt(sqrt(signo.*update)); 143 | 144 | % UPADTE ONLY PIXELS INSIDE THE INPAINTING DOMAIN (BY MASK) 145 | u(:,c) = u(:,c) + dt * ~channel_mask(:,c).*update; 146 | end 147 | 148 | % param.N steps of anisotropic diffusion 149 | un = anisodiff(u(:,c),dt,param.eps,geps(:,c),param.N,matrices); 150 | 151 | diff = norm(un-u(:,c))/norm(un); 152 | 153 | % UPDATE 154 | u(:,c) = ~channel_mask(:,c).*un + channel_mask(:,c).*u(:,c); 155 | 156 | % TEST EXIT CONDITION 157 | if diff D QuQ + QuQ D, where 316 | # Q is nonsingular, the matrix of eigenvectors of L and D is a diagonal matrix. 317 | # We have to compute QuQ. This we can do in a fast way by using the fft-transform: 318 | 319 | l1 = scipy.array( 2*(scipy.cos( 2*scipy.array(range(0,M))*pi / M ) - 1) ) 320 | l2 = scipy.array( 2*(scipy.cos( 2*scipy.array(range(0,N))*pi / N ) - 1) ) 321 | Lambda1 = spdiags(l1,0,M,M)/(hi**2); 322 | Lambda2 = spdiags(l2,0,N,N)/(hj**2); 323 | 324 | Denominator1 = Lambda1.dot(scipy.ones((M,N))) 325 | Denominator2 = Lambda2.T.dot(scipy.ones((N,M))).T 326 | Denominator = Denominator1 + Denominator2 327 | 328 | u = scipy.ones((M,N,C)) 329 | 330 | for c in range(0,C): 331 | 332 | uu = input[:,:,c] 333 | u_hat = pyfftw.interfaces.numpy_fft.fft2( uu ) 334 | fidelity_u0_hat = pyfftw.interfaces.numpy_fft.fft2( FIDELITY * uu ) 335 | 336 | for iter in range(0,maxiter): 337 | 338 | fidelity_u_hat = pyfftw.interfaces.numpy_fft.fft2( FIDELITY * uu); 339 | 340 | Fprime_hat = pyfftw.interfaces.numpy_fft.fft2( 2*(2*uu**3 - 3*uu**2 + uu) ) 341 | 342 | # CH-inpainting 343 | u1 = (1 + dt*FIDELITY) * u_hat 344 | u2 = -( dt/epsilon[1] ) * Denominator * u_hat 345 | 346 | u3 = (dt / ep[iter])*Denominator * Fprime_hat 347 | 348 | u4 = dt * ( fidelity_u0_hat-fidelity_u_hat ) 349 | 350 | uden = 1.0 + dt*( FIDELITY + ep[iter]*(Denominator**2) - (Denominator / epsilon[1]) ) 351 | 352 | u_hat = (u1 + u2 + u3 + u4)/uden; 353 | 354 | uu = pyfftw.interfaces.numpy_fft.ifft2(u_hat) 355 | 356 | u[:,:,c] = uu.real 357 | 358 | if C==1: 359 | mpimg.imsave("./results/cahn_hilliard_output.png", u[:,:,0],cmap="gray") 360 | elif C==3: 361 | mpimg.imsave("./results/cahn_hilliard_output.png", u) 362 | 363 | return u 364 | 365 | ### Transport Inpainting 366 | 367 | # anisotropic diffusion 368 | def anisodiff(u,dt,eps,geps,iterations,Kfi,Kfj,Kbi,Kbj,Kci,Kcj): 369 | 370 | # M. Bertalmio, "Processing of flat and non-flat image information on 371 | # arbitrary manifolds using Partial Differential Equations", PhD Thesis, 2001. 372 | 373 | M,N = u.shape 374 | 375 | Kii = Kfi - Kbi 376 | Kjj = Kfj - Kbj 377 | ux = cv.filter2D(u,-1,Kci) 378 | uy = cv.filter2D(u,-1,Kcj) 379 | uxx = cv.filter2D(u,-1,Kii) 380 | uyy = cv.filter2D(u,-1,Kjj) 381 | uxy = cv.filter2D(cv.filter2D(u,-1,Kci),-1,Kcj) 382 | 383 | for i in range(0,iterations): 384 | squared_normgrad = ux**2 + uy**2 + eps; 385 | u = u + dt*geps*(uyy * (ux**2) + uxx * (uy**2) - 2*ux*uy*uxy) / squared_normgrad; 386 | 387 | return u 388 | 389 | def transport(input,mask,maxiter,tol,dt,iter_inpainting,iter_anisotropic,epsilon): 390 | 391 | if input.ndim==3: 392 | M,N,C = input.shape 393 | else: 394 | M,N = input.shape 395 | C = 1 396 | 397 | # KERNELS of derivatives 398 | Kfi,Kfj,Kbi,Kbj,Kci,Kcj = create_kernel_derivatives() 399 | 400 | # INITIALISATION 401 | laplacian_gepsilon = scipy.zeros((M,N,C)) 402 | Dlaplacian = scipy.zeros((M,N,2)) 403 | laplacian = scipy.zeros((M,N,C)) 404 | update = scipy.zeros((M,N,C)) 405 | zeros = scipy.zeros((M,N)) 406 | normal = scipy.zeros((M,N,2)) 407 | LAMBDAEPS = scipy.zeros((M,N,C)); 408 | 409 | u = input.copy() 410 | un = input.copy() 411 | MASKEPS = 1-mask.copy() 412 | gepsilon = 1-mask.copy(); 413 | 414 | # MORPHOLOGIC ELEMENT for the small epsilon-diffusion strip around the inpainting domain: 415 | SE = cv.getStructuringElement(cv.MORPH_ELLIPSE,(13,13)) 416 | 417 | for c in range(0,C): 418 | 419 | # INTERPOLATE g_{epsilon} with a few steps of linear diffusion within the strip 420 | LAMBDAEPS[:,:,c] = cv.dilate(MASKEPS[:,:,c], SE, iterations=1) - MASKEPS[:,:,c] 421 | for t in range(0,5): 422 | laplacian_gepsilon[:,:,c] = cv.filter2D(gepsilon[:,:,c],-1, Kfi-Kbi+Kfj-Kbj) 423 | gepsilon[:,:,c] = gepsilon[:,:,c] + dt*laplacian_gepsilon[:,:,c] + LAMBDAEPS[:,:,c]*( MASKEPS[:,:,c]-gepsilon[:,:,c] ); 424 | 425 | # ANISOTROPIC DIFFUSION PREPROCESSING STEP 426 | u[:,:,c] = anisodiff(u[:,:,c],dt,epsilon,gepsilon[:,:,c],1,Kfi,Kfj,Kbi,Kbj,Kci,Kcj); 427 | 428 | # INPAINTING 429 | for outer_iter in range(0,maxiter): 430 | 431 | for iter in range(0,iter_inpainting): 432 | 433 | laplacian[:,:,c] = cv.filter2D(u[:,:,c],-1, Kfi-Kbi+Kfj-Kbj) 434 | Dlaplacian[:,:,0] = cv.filter2D(laplacian[:,:,c],-1, Kci) 435 | Dlaplacian[:,:,1] = cv.filter2D(laplacian[:,:,c],-1, Kcj) 436 | normal[:,:,0] = cv.filter2D(u[:,:,c],-1, Kci) 437 | normal[:,:,1] = cv.filter2D(u[:,:,c],-1, Kcj) 438 | 439 | dennormal = scipy.sqrt( scipy.sum(normal**2,axis=2) + epsilon ) 440 | 441 | normal[:,:,0] = scipy.divide(normal[:,:,0],dennormal) 442 | normal[:,:,1] = scipy.divide(normal[:,:,1],dennormal) 443 | 444 | beta = Dlaplacian[:,:,0]*(-normal[:,:,1]) + Dlaplacian[:,:,1]*normal[:,:,0] 445 | betapos = beta>0 446 | 447 | uxf = cv.filter2D(u[:,:,c],-1, Kfi) 448 | uxb = cv.filter2D(u[:,:,c],-1, Kbi) 449 | uyf = cv.filter2D(u[:,:,c],-1, Kfj) 450 | uyb = cv.filter2D(u[:,:,c],-1, Kbj) 451 | 452 | sl1 = scipy.array([scipy.minimum(uxb,zeros), scipy.maximum(uxf,zeros), scipy.minimum(uyb,zeros), scipy.maximum(uyf,zeros)]) 453 | sl2 = scipy.array([scipy.maximum(uxb,zeros), scipy.minimum(uxf,zeros), scipy.maximum(uyb,zeros), scipy.minimum(uyf,zeros)]) 454 | slopelim1 = scipy.sum(sl1**2,axis=0) 455 | slopelim2 = scipy.sum(sl2**2,axis=0) 456 | 457 | slopelim = betapos*scipy.sqrt(slopelim1) + (1-betapos)*scipy.sqrt(slopelim2) 458 | update[:,:,c] = beta * slopelim; 459 | 460 | # UPADTE ONLY PIXELS INSIDE THE INPAINTING DOMAIN (BY MASK) 461 | u[:,:,c] = u[:,:,c] + dt * (1-mask[:,:,c])*update[:,:,c]; 462 | 463 | # param.N steps of anisotropic diffusion 464 | un[:,:,c] = anisodiff(u[:,:,c],dt,epsilon,gepsilon[:,:,c],iter_anisotropic,Kfi,Kfj,Kbi,Kbj,Kci,Kcj); 465 | 466 | # exit condition 467 | diff_u = np.linalg.norm(un[:,:,c].reshape(M*N,1)-u[:,:,c].reshape(M*N,1),2)/np.linalg.norm(un[:,:,c].reshape(M*N,1),2); 468 | 469 | # update 470 | u[:,:,c] = (1-mask[:,:,c])*un[:,:,c] + mask[:,:,c]*u[:,:,c]; 471 | 472 | # test exit condition 473 | if diff_u