├── favicon.png
├── comparison.png
├── img
    ├── mario.jpg
    ├── lighthouse.jpg
    ├── starrynight.jpg
    └── woody-allen.jpg
├── screenshot.png
├── LICENSE
├── README.md
└── index.html


/favicon.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/favicon.png


--------------------------------------------------------------------------------
/comparison.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/comparison.png


--------------------------------------------------------------------------------
/img/mario.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/img/mario.jpg


--------------------------------------------------------------------------------
/screenshot.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/screenshot.png


--------------------------------------------------------------------------------
/img/lighthouse.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/img/lighthouse.jpg


--------------------------------------------------------------------------------
/img/starrynight.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/img/starrynight.jpg


--------------------------------------------------------------------------------
/img/woody-allen.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/victorqribeiro/pixelRestorer/HEAD/img/woody-allen.jpg


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2018 Victor Ribeiro
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # pixelRestorer
 2 | 
 3 | Using statistics to restore pixel art images.
 4 | 
 5 | ![pixelRestorer](screenshot.png)
 6 | 
 7 | A live version of it is hosted [here](https://victorribeiro.com/pixelRestorer)
 8 | 
 9 | ## How to use
10 | 
11 | Select the image, the way of redrawing it ([mean, mode or median](https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/mean-median-basics/a/mean-median-and-mode-review)) and click Run or Hit Enter on the keyboard.
12 | 
13 | The processing is animated so you can see the algorithm working (it's not slow it was programmed like that).
14 | 
15 | ## About 
16 | 
17 | This is a project I've been idealizing from some time now. Is it possible to reconstruct a pixel art image that has been corrupted by scaling or compression, using statistics? Well, lets give it a shot.
18 | 
19 | ## The Algorithm
20 | 
21 | The first step is to determine the image's "pixel size". The algorithm does that by diving the image in windows and calculating the probability of that window being the pixel size.
22 | 
23 | ```
24 | window = imageSize / 2
25 | window = imageSize / 4
26 | window = imageSize / 6
27 | ...
28 | ```
29 | 
30 | The window is a square matrix. 3x3, 6x6, 8x8...
31 | 
32 | It stores all the probabilities in a dictionary then returns the most likely to be the right pixel size - the greater value.
33 | 
34 | The algorithm calculates the probability of the window by getting the mean - sum of pixels divided by total pixels - of the area and comparing it to the median - element on the center of the array - plus some threshold. That's a way I found to calculate the entropy of the window. A window with low entropy must mean that the pixels in there are smiliar, right?!
35 | 
36 | The second step is to redraw the image pixelSize by pixelSize getting the mean, mode or median of each pixelSize. E.g.: The algorithm determined that the pixelSize is 8x8. So, it gets a 8x8 patch of the image, take the mean, the mode or the median and redraw every real pixel (1x1) with that value. So now the pixelSize - the window of 8x8 - is now filled with pixels of the same value.
37 | 
38 | Here's a comparison:
39 | 
40 | ![pixelRestorer](comparison.png)
41 | 
42 | It got rid of most artefacts caused by scaling or compressing the original image.
43 | 
44 | ## Known problems
45 | 
46 | The algorithm does not work 100% of the cases nor with every possible image. For example, it doesn't work if the pixelSize is offsetted or if the image has a large area of the same color - as a background for instance.
47 | 
48 | If most of the image is similar, like a image with a large solid white color as background and just a small portion of it being the pixel art, it can't figure out the right size of the pixelSize. Cause every window will have a low entropy value (since its most white, no matter what).
49 | 
50 | It also blurs the images sometimes (most of the times), due to the fact that the image was scaled to some odd value (and the pixel seize its no a integer - like, it guessed the pixel size would be 6,5x6,5).
51 | 
52 | Also the code is a mess and has a lot of hard coded values in it. Sorry, I'll might refactor it later.
53 | 
54 | ## Disclaimer
55 | 
56 | I do not own the images used in the examples, I googled them. If you are or know the authors, contact me and will give proper credit.
57 | 


--------------------------------------------------------------------------------
/index.html:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE html>
  2 | <html lang="en">
  3 | <head>
  4 | <title> pixelRestorer </title>
  5 | <meta charset="utf-8" />
  6 | <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no, user-scalable=no">
  7 | <meta name="mobile-web-app-capable" content="yes">
  8 | <meta property="og:url" content="https://www.victorribeiro.com/pixelRestorer" />
  9 | <meta property="og:type" content="article" />
 10 | <meta property="og:title" content="Pixel Restorer" />
 11 | <meta property="og:author" content="Victor Ribeiro" />
 12 | <meta property="og:description" content="Using statistics to restore pixel art images." />
 13 | <meta property="og:image" content="https://www.victorribeiro.com/favicon.png" />
 14 | <link rel="icon" href="favicon.png" />
 15 | <style>
 16 | html, body{
 17 | 	height: 100%;
 18 | 	margin: 0;
 19 | 	padding: 0;
 20 | }
 21 | #canvas{
 22 | 	image-rendering: pixelated;
 23 | 	display: block;
 24 | 	z-index: 0;
 25 | 	max-width: 100%;
 26 | }
 27 | #buffer{
 28 | 	position: absolute;
 29 | 	top: 0;
 30 | 	left: 0;
 31 | 	z-index: 1;
 32 | 	max-width: 100%;
 33 | }
 34 | select, button{
 35 | 	display: block;
 36 | 	width: 100%;
 37 | 	padding: 0.3rem;
 38 | 	font-family: Arial, Helvetica, Sans;
 39 | 	font-size: 1rem;
 40 | }
 41 | #wrap{
 42 | 	height: 100%;
 43 | 	display: grid;
 44 | 	grid-template-rows: 1fr 1fr;
 45 | 	align-items: center;
 46 | 	justify-content: center;
 47 | }
 48 | </style>
 49 | </head>
 50 | <body>
 51 | <script>
 52 | let img, canvas, c, buffer, bc, probability, totalColors, size, pixelSize, btn, resize = true, render, _i, _j, method = 1;
 53 | 
 54 | let _sort = function(a,b){
 55 | 	return a - b;
 56 | }
 57 | 
 58 | let mode = function(array){
 59 | 	/* tem que tirar a moda da cor inteira e nao por canal. ex. [r,g,b] */
 60 | 	let mod = {};
 61 | 	for(let i = 0; i < array.length; i+=4){
 62 | 		let index = array[i]+","+array[i+1]+","+array[i+2];
 63 | 		if( mod[ index ] ){
 64 | 			mod[index] += 1;
 65 | 		}else{
 66 | 			mod[index] = 1;
 67 | 		}
 68 | 	}
 69 | 	let mode = [ array[ Math.floor(array.length/2) ],
 70 | 							 array[ Math.floor(array.length/2)+1],
 71 | 							 array[ Math.floor(array.length/2)+2 ] ], _max = 0;
 72 | 	for(let k in mod){
 73 | 		if( mod[k] > _max ){
 74 | 			_max = mod[k];
 75 | 			mode = k.split(",").map( _ => parseInt(_) );
 76 | 		}
 77 | 	}
 78 | 	return mode;
 79 | }
 80 | 
 81 | let median = function(array){
 82 | 	let index = Math.floor(array.length/2);
 83 | 	let median = [ array[index], array[index+1], array[index+2] ];
 84 | 	return median;
 85 | }
 86 | 
 87 | let mean = function(array){
 88 | 	let r = 0, g = 0, b = 0;
 89 | 	for(let i = 0; i < array.length; i+=4){
 90 | 		r += array[i];
 91 | 		g += array[i+1];
 92 | 		b += array[i+2];
 93 | 	}
 94 | 	let s = array.length / 4;
 95 | 	r = r / s;
 96 | 	g = g / s;
 97 | 	b = b / s;
 98 | 	return [r,g,b];
 99 | }
100 | function init(src,action){
101 | 	switch(action){
102 | 		case '1' :
103 | 				method = mean;
104 | 			break;
105 | 		case '2' :
106 | 				method = mode;
107 | 			break;
108 | 		case '3' :
109 | 				method = median
110 | 			break;
111 | 		default :
112 | 			method = mean;
113 | 	}
114 | 	if(render){
115 | 		cancelAnimationFrame( render );
116 | 		render = null;
117 | 	}
118 | 	_i = 0, _j = 0;
119 | 	btn = document.querySelector('#btn');
120 | 	btn.innerText = "Run";
121 | 	img = new Image();
122 | 	img.src = src || 'img/mario.jpg';
123 | 	img.onload = ()=>{
124 | 		canvas = document.querySelector('#canvas');
125 | 		canvas.width = img.width;
126 | 		canvas.height = img.height;
127 | 		buffer = document.querySelector('#buffer');
128 | 		buffer.width = canvas.width;
129 | 		buffer.height = canvas.height;
130 | 		bc = buffer.getContext('2d');
131 | 		c = canvas.getContext('2d',{alpha: false});
132 | 		c.drawImage(img,0,0);
133 | 		//getColors(10);
134 | 		getPixelSize(100,10);
135 | 	}
136 | }
137 | 
138 | function getPixelSize(times, threshold){
139 | 	times = times || 2;
140 | 	threshold = threshold || 5;
141 | 	probability = {};
142 | 	for(let t = 2; t < times; t+=2){
143 | 		if( img.width / t >= 3 ){
144 | 			let prob = 0;
145 | 			let sizeX = img.width / t ;
146 | 			let sizeY = img.height / t ;
147 | 			for(let i = 0, endI = img.height; i < endI; i+= sizeY){
148 | 				for(let j = 0, endJ = img.width; j < endJ; j+= sizeX){
149 | 					let patch = c.getImageData(j , i , sizeX, sizeY);
150 | 					let pathMean = mean(patch.data);
151 | 					rMean = pathMean[0];
152 | 					gMean = pathMean[1];
153 | 					bMean = pathMean[2];
154 | 					if( (rMean > patch.data[patch.data.length/2]-threshold && rMean < patch.data[patch.data.length/2]+threshold) &&
155 | 							(gMean > patch.data[patch.data.length/2+1]-threshold && gMean < patch.data[patch.data.length/2+1]+threshold) && 
156 | 							(bMean > patch.data[patch.data.length/2+2]-threshold && bMean < patch.data[patch.data.length/2+2]+threshold) ){
157 | 						prob += 1;
158 | 					}
159 | 				}
160 | 			}
161 | 			probability[t] = prob / (t*t);
162 | 		}
163 | 	}
164 | }
165 | 
166 | /*
167 | function correctColors(){
168 | 	let m = 0, size;
169 | 	for(key in probability){
170 | 		if( probability[key] > m ){
171 | 			m = probability[key];
172 | 			size = key;
173 | 		}
174 | 	}
175 | 	size = Math.floor(img.width / size);
176 | 	console.log( size );
177 | 	for(let i = 0; i < img.height; i+= size){
178 | 		for(let j = 0; j < img.width; j+= size){
179 | 			let data = c.getImageData(j,i,size,size);
180 | 			let r = [], g = [], b = [];
181 | 			for(let c = 0; c < data.data.length; c+=4){
182 | 				r.push(data.data[c]);
183 | 				g.push(data.data[c+1]);
184 | 				b.push(data.data[c+2]);
185 | 			}
186 | 			r.sort(_sort);
187 | 			g.sort(_sort);
188 | 			b.sort(_sort);
189 | 			r = r[r.length/2];
190 | 			g = g[g.length/2];
191 | 			b = b[b.length/2];
192 | 			for(let c = 0; c < data.data.length; c+=4){
193 | 				data.data[c] = r;
194 | 				data.data[c+1] = g;
195 | 				data.data[c+2] = b;
196 | 			}
197 | 			c.putImageData(data,j,i);
198 | 		}
199 | 	}
200 | }
201 | */
202 | 
203 | function correctColorsRender(){
204 | 	if( _i == 0 && _j == 0){
205 | 	
206 | 		let m = 0, size;
207 | 		for(key in probability){
208 | 			if( probability[key] > m ){
209 | 				m = probability[key];
210 | 				size = key;
211 | 			}
212 | 		}
213 | 		
214 | 		pixelSize = Math.floor(img.width / (Math.floor(size)+1));
215 | 		if(resize){
216 | 			let newW = size * pixelSize;
217 | 			let newH = size * pixelSize;
218 | 			canvas.width = newW;
219 | 			canvas.height = newH;
220 | 
221 | 			c.drawImage(img,0,0,newW,newH);
222 | 		}
223 | 		let pos = canvas.getBoundingClientRect();
224 | 		buffer.style.position = "absolute";
225 | 		buffer.style.left = pos.left +"px";
226 | 		buffer.style.top = pos.top + window.scrollY +"px";
227 | 		buffer.width = canvas.width;
228 | 		buffer.height = canvas.height;
229 | 		buffer.style.width = pos.width+"px";
230 | 		buffer.style.height = pos.height+"px";
231 | 	}
232 | 	
233 | 	let data = c.getImageData(_j,_i,pixelSize,pixelSize);
234 | 	let rgb = method( data.data );
235 | 
236 | 	for(let c = 0; c < data.data.length; c+=4){
237 | 		data.data[c] = rgb[0];
238 | 		data.data[c+1] = rgb[1];
239 | 		data.data[c+2] = rgb[2];
240 | 	}
241 | 	
242 | 	c.putImageData(data,_j,_i);
243 | 	bc.clearRect(0,0,buffer.width,buffer.height);
244 | 	bc.fillRect(_j,_i,pixelSize,pixelSize);
245 | 
246 | 	_j+=pixelSize;
247 | 	if( _j >= img.width ){
248 | 		_j = 0;
249 | 		_i+=pixelSize;
250 | 	}
251 | 	if(_i < img.width){
252 | 		render = requestAnimationFrame( correctColorsRender );
253 | 	}
254 | }
255 | 
256 | function correctColors(){
257 | 	let m = 0, size;
258 | 	for(key in probability){
259 | 		if( probability[key] > m ){
260 | 			m = probability[key];
261 | 			size = key;
262 | 		}
263 | 	}
264 | 	size = Math.floor(img.width / size);
265 | 
266 | 	for(let i = 0; i < img.height; i+= size){
267 | 		for(let j = 0; j < img.width; j+= size){
268 | 			let data = c.getImageData(j,i,size,size);
269 | 			
270 | 			let r = [], g = [], b = [];
271 | 			for(let c = 0; c < data.data.length; c+=4){
272 | 				r.push(data.data[c]);
273 | 				g.push(data.data[c+1]);
274 | 				b.push(data.data[c+2]);
275 | 			}
276 | 
277 | 			r.sort(_sort);
278 | 			g.sort(_sort);
279 | 			b.sort(_sort);
280 | 			r = r[Math.floor(r.length/2)];
281 | 			g = g[Math.floor(g.length/2)];
282 | 			b = b[Math.floor(b.length/2)];
283 | 			
284 | 			let minDist = Infinity, bestColor;
285 | 			
286 | 			for(let d = 0; d < totalColors.length; d++){
287 | 				let col = totalColors[d][0].split(",").map(v=>parseInt(v));
288 | 				let dist = distance([r,g,b],col);
289 | 				if( dist < minDist ){
290 | 					minDist = dist;
291 | 					bestColor = col;
292 | 				}
293 | 			}
294 | 			for(let c = 0; c < data.data.length; c+=4){
295 | 				data.data[c] = 0;
296 | 				data.data[c+1] = 0;
297 | 				data.data[c+2] = 0;
298 | 			}
299 | 			
300 | 			//c.fillRect(j,i,size,size);
301 | 			c.putImageData(data,j,i);
302 | 		}
303 | 	}
304 | }
305 | 
306 | function distance(a,b){
307 | 	let sum = 0;
308 | 	for(let s = 0; s < a.length; s++){
309 | 		sum += Math.abs( parseInt(a[s]) - parseInt(b[s]) );
310 | 	}
311 | 	return sum;
312 | }
313 | 
314 | function getColors(threshold){
315 | 	threshold = threshold || 30;
316 | 	let colors = {};
317 | 	let data = c.getImageData(0,0,img.width,img.height);
318 | 	for(let i = 0; i < data.data.length; i+=4){
319 | 		if(colors[data.data[i]+","+data.data[i+1]+","+data.data[i+2]]){
320 | 			colors[data.data[i]+","+data.data[i+1]+","+data.data[i+2]] += 1;
321 | 		}else{
322 | 			colors[data.data[i]+","+data.data[i+1]+","+data.data[i+2]] = 1;
323 | 		}
324 | 	}
325 | 	
326 | 	let items = Object.keys(colors).map(function(key) {
327 | 		return [key, colors[key]];
328 | 	});
329 | 
330 | 	items.sort(function(first, second) {
331 | 		return second[1] - first[1];
332 | 	});
333 | 
334 | 	for(let i = 0; i < items.length-1; i++){
335 | 		let valueA = items[i][0].split(",").map(v=>parseInt(v));
336 | 		for(let j = items.length-1; j > i+1; j--){
337 | 
338 | 			if(items[j][1] < 10){
339 | 				items.splice(j,1);
340 | 				continue;
341 | 			}
342 | 			
343 | 			let valueB = items[j][0].split(",").map(v=>parseInt(v));
344 | 
345 | 			if( distance(valueA,valueB) < threshold ){
346 | 				items[i][1] += items[j][1];
347 | 				items.splice(j,1);
348 | 			}
349 | 		}
350 | 	}
351 | 	
352 | 	items.sort(function(first, second) {
353 | 		return second[1] - first[1];
354 | 	});
355 | 
356 | 	//totalColors = items.slice(0, 20);
357 | 	totalColors = items;	
358 | 	
359 | }
360 | 
361 | function colorDistance(a,b){
362 | 	let minColorA = a[0], minColorB = b[0], indexA = 0, indexB = 0;
363 | 	for(let i = 1; i < a.length; i++){
364 | 		if( minColorA < a[i] ){
365 | 			minColorA = a[i];
366 | 			indexA = i;
367 | 		}
368 | 		if( minColorB < b[i] ){
369 | 			minColorB = b[i];
370 | 			indexB = i;
371 | 		}
372 | 	}
373 | 	let dist = 0;
374 | 	for(let i = 0; i < a.length; i++){
375 | 		dist += Math.abs( a[i] - b[i] );
376 | 	}
377 | 	
378 | 	if( indexA !== indexB ){
379 | 		dist += 0.25 * minColorB;
380 | 	}
381 | 	
382 | 	return dist;
383 | }
384 | 
385 | function change(){
386 | 	let src = document.querySelector('#src').value;
387 | 	let method = document.querySelector('#method').value;
388 | 	init(src,method);
389 | }
390 | 
391 | function toggleAnimation(){
392 | 	if(!render){
393 | 		correctColorsRender();
394 | 	}else{
395 | 		cancelAnimationFrame( render );
396 | 		render = null;	
397 | 	}
398 | }
399 | 
400 | window.onload = function(){
401 | 
402 | 	init();
403 | 
404 | 	document.body.addEventListener("keypress",function(e){
405 | 		e.preventDefault();
406 | 		if(e.keyCode == 13)
407 | 			toggleAnimation();
408 | 	});
409 | 
410 | 	btn.addEventListener("click", function(){
411 | 		toggleAnimation();
412 | 		btn.innerText = render ? "Pause" : "Run";
413 | 	});
414 | 	
415 | 	window.addEventListener("resize",function(){
416 | 		init();
417 | 	});
418 | 	
419 | }
420 | </script>
421 | <div id="wrap">
422 | 	<div id="screen">
423 | 		<canvas id="canvas"></canvas>
424 | 		<canvas id="buffer"></canvas>
425 | 	</div>
426 | 	<div id="tools">
427 | 		<select id="src" onchange="change()">
428 | 			<option value="img/mario.jpg"> Mario </option>
429 | 			<option value="img/starrynight.jpg"> Starry Night </option>
430 | 			<option value="img/lighthouse.jpg"> Lighthouse </option>
431 | 			<option value="img/woody-allen.jpg"> Woody Allen </option>
432 | 		</select>
433 | 		<select id="method" onchange="change()">
434 | 			<option value="1"> Mean </option>
435 | 			<option value="2"> Mode </option>
436 | 			<option value="3"> Median </option>
437 | 		</select>
438 | 		<button id="btn"> Run </button>
439 | 	</div>
440 | </div>
441 | </body>
442 | </html>
443 | 


--------------------------------------------------------------------------------