└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Super-Resolution.Benckmark 2 | A curated list of super-resolution resources and a benchmark for single image super-resolution algorithms. 3 | 4 | See my implementated super-resolution algorithms: 5 | 6 | + [SRGAN](https://github.com/huangzehao/torch-srgan) 7 | + [VDSR](https://github.com/huangzehao/caffe-vdsr) 8 | + [CSCN](https://github.com/huangzehao/SCN_Matlab) 9 | 10 | ## TODO 11 | Build a benckmark like [SelfExSR_Code](https://sites.google.com/site/jbhuang0604/publications/struct_sr) 12 | 13 | ## State-of-the-art algorithms 14 | #### Classical Sparse Coding Method 15 | * ScSR [[Web]](http://www.ifp.illinois.edu/~jyang29/ScSR.htm) 16 | * Image super-resolution as sparse representation of raw image patches (CVPR2008), Jianchao Yang et al. 17 | * Image super-resolution via sparse representation (TIP2010), Jianchao Yang et al. 18 | * Coupled dictionary training for image super-resolution (TIP2011), Jianchao Yang et al. 19 | 20 | #### Anchored Neighborhood Regression Method 21 | * ANR [[Web]](http://www.vision.ee.ethz.ch/~timofter/ICCV2013_ID1774_SUPPLEMENTARY/index.html) 22 | * Anchored Neighborhood Regression for Fast Example-Based Super-Resolution (ICCV2013), Radu Timofte et al. 23 | * A+ [[Web]](http://www.vision.ee.ethz.ch/~timofter/ACCV2014_ID820_SUPPLEMENTARY/) 24 | * A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution (ACCV2014), Radu Timofte et al. 25 | * IA [[Web]](http://www.vision.ee.ethz.ch/~timofter/CVPR2016_ID769_SUPPLEMENTARY/index.html) 26 | * Seven ways to improve example-based single image super resolution (CVPR2016), Radu Timofte et al. 27 | 28 | #### Self-Exemplars 29 | * SelfExSR [[Web]](https://sites.google.com/site/jbhuang0604/publications/struct_sr) 30 | * Single Image Super-Resolution from Transformed Self-Exemplars (CVPR2015), Jia-Bin Huang et al. 31 | 32 | #### Bayes 33 | * NBSRF [[Web]](http://jordisalvador-image.blogspot.com/2015/08/iccv-2015.html) 34 | * Naive Bayes Super-Resolution Forest (ICCV2015), Jordi Salvador et al. 35 | 36 | #### Deep Learning Method 37 | * SRCNN [[Web]](http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html) [[waifu2x by nagadomi]](https://github.com/nagadomi/waifu2x) 38 | * Image Super-Resolution Using Deep Convolutional Networks (ECCV2014), Chao Dong et al. 39 | * Image Super-Resolution Using Deep Convolutional Networks (TPAMI2015), Chao Dong et al. 40 | * CSCN [[Web]](http://www.ifp.illinois.edu/~dingliu2/iccv15/) 41 | * Deep Networks for Image Super-Resolution with Sparse Prior (ICCV2015), Zhaowen Wang et al. 42 | * Robust Single Image Super-Resolution via Deep Networks with Sparse Prior (TIP2016), Ding Liu et al. 43 | * VDSR [[Web]](http://cv.snu.ac.kr/research/VDSR/) [[Unofficial Implementation in Caffe]](https://github.com/huangzehao/caffe-vdsr) 44 | * Accurate Image Super-Resolution Using Very Deep Convolutional Networks (CVPR2016), Jiwon Kim et al. 45 | * DRCN [[Web]](http://cv.snu.ac.kr/research/DRCN/) 46 | * Deeply-Recursive Convolutional Network for Image Super-Resolution (CVPR2016), Jiwon Kim et al. 47 | * ESPCN [[PDF]](http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Shi_Real-Time_Single_Image_CVPR_2016_paper.pdf) 48 | * Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network (CVPR2016), Wenzhe Shi et al. 49 | * Is the deconvolution layer the same as a convolutional layer? [[PDF]](https://arxiv.org/ftp/arxiv/papers/1609/1609.07009.pdf) 50 | * Checkerboard artifact free sub-pixel convolution [[PDF]](https://arxiv.org/pdf/1707.02937.pdf) 51 | * FSRCNN [[Web]](http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html) 52 | * Acclerating the Super-Resolution Convolutional Neural Network (ECCV2016), Dong Chao et al. 53 | * LapSRN [[Web]](http://vllab1.ucmerced.edu/~wlai24/LapSRN/) 54 | * Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution (CVPR 2017), Wei-Sheng Lai et al. 55 | * EDSR [[PDF]](https://arxiv.org/pdf/1707.02921.pdf) 56 | * Enhanced Deep Residual Networks for Single Image Super-Resolution (Winner of NTIRE2017 Super-Resolution Challenge), Bee Lim et al. 57 | 58 | #### Perceptual Loss and GAN 59 | * Perceptual Loss [[PDF]](http://cs.stanford.edu/people/jcjohns/papers/eccv16/JohnsonECCV16.pdf) 60 | * Perceptual Losses for Real-Time Style Transfer and Super-Resolution (ECCV2016), Justin Johnson et al. 61 | * SRGAN [[PDF]](https://arxiv.org/abs/1609.04802) 62 | * Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (CVPR2017), Christian Ledig et al. 63 | * AffGAN [[PDF]](https://arxiv.org/pdf/1610.04490.pdf) 64 | * AMORTISED MAP INFERENCE FOR IMAGE SUPER-RESOLUTION (ICLR2017), Casper Kaae Sønderby et al. 65 | * EnhanceNet [[PDF]](https://arxiv.org/abs/1612.07919) 66 | * EnhanceNet: Single Image Super-Resolution through Automated Texture Synthesis, Mehdi S. M. Sajjadi et al. 67 | * neural-enchance [[Github]](https://github.com/alexjc/neural-enhance) 68 | 69 | #### Video SR 70 | * VESPCN [[PDF]](https://arxiv.org/pdf/1611.05250.pdf) 71 | * Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation (CVPR2017), Jose Caballero et al. 72 | 73 | ## Dicussion 74 | #### Deconvolution and Sub-Pixel Convolution 75 | * [Deconvolution and Checkerboard Artifacts](http://distill.pub/2016/deconv-checkerboard/) 76 | * [SubPixel](https://github.com/Tetrachrome/subpixel) 77 | 78 | 79 | ## Datasets 80 | 81 | | Test Dataset | Image source | 82 | |:----:|:----:| 83 | | **Set 5** | [Bevilacqua et al. BMVC 2012](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html) | 84 | | **Set 14** | [Zeyde et al. LNCS 2010](https://sites.google.com/site/romanzeyde/research-interests) | 85 | | **BSD 100** | [Martin et al. ICCV 2001](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/) | 86 | | **Urban 100** | [Huang et al. CVPR 2015](https://sites.google.com/site/jbhuang0604/publications/struct_sr) | 87 | 88 | | Train Dataset | Image source | 89 | |:----:|:----:| 90 | | **Yang 91** | [Yang et al. CVPR 2008](http://www.ifp.illinois.edu/~jyang29/ScSR.htm) | 91 | | **BSD 200** | [Martin et al. ICCV 2001](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/) | 92 | | **General 100** | [Dong et al. ECCV 2016](http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html) | 93 | | **ImageNet** | [Olga Russakovsky et al. IJCV 2015](http://www.image-net.org/) | 94 | | **COCO**| [Tsung-Yi Lin et al. ECCV 2014](http://mscoco.org/) 95 | 96 | ## Quantitative comparisons 97 | Results from papers of VDSR, DRCN, CSCN and IA. 98 | 99 | **Note:** IA use enchanced prediction trick to improve result. 100 | ##### Results on Set 5 101 | 102 | | Scale | Bicubic | A+ | SRCNN | SelfExSR | CSCN | VDSR | DRCN | IA | 103 | |:---------:|:-------:|:--------:|:------:|:----:|:----:|:----:|:----:|:----:| 104 | | **2x** - PSNR/SSIM| 33.66/0.9929 | 36.54/0.9544 | 36.66/0.9542 | 36.49/0.9537 | 36.93/0.9552 | 37.53/0.9587 | **37.63/0.9588** |37.39/| 105 | | **3x** - PSNR/SSIM| 30.39/0.8682 | 32.59/0.9088 | 32.75/0.9090 | 32.58/0.9093 | 33.10/0.9144 | 33.66/0.9213 | **33.82/0.9226** |33.46/| 106 | | **4x** - PSNR/SSIM| 28.42/0.8104 | 30.28/0.8603 | 30.48/0.8628 | 30.31/0.8619 | 30.86/0.8732 | 31.35/0.8838 | **31.53/0.8854** |31.10/| 107 | 108 | ##### Results on Set 14 109 | 110 | | Scale | Bicubic | A+ | SRCNN | SelfExSR | CSCN | VDSR | DRCN | IA | 111 | |:---------:|:-------:|:--------:|:------:|:----:|:----:|:----:|:----:|:----:| 112 | | **2x** - PSNR/SSIM| 30.24/0.8688 | 32.28/0.9056 | 32.42/0.9063 | 32.22/0.9034 | 32.56/0.9074 | 33.03/**0.9124** | **33.04**/0.9118 |32.87/| 113 | | **3x** - PSNR/SSIM| 27.55/0.7742 | 29.13/0.8188 | 29.28/0.8209 | 29.16/0.8196 | 29.41/0.8238 | **29.77/0.8314** | 29.76/0.8311 |29.69/| 114 | | **4x** - PSNR/SSIM| 26.00/0.7027 | 27.32/0.7491 | 27.49/0.7503 | 27.40/0.7518 | 27.64/0.7587 | 28.01/**0.7674** | **28.02**/0.7670 |27.88/| 115 | 116 | ##### Results on BSD 100 117 | 118 | | Scale | Bicubic | A+ | SRCNN | SelfExSR | CSCN | VDSR | DRCN | IA | 119 | |:---------:|:-------:|:--------:|:------:|:----:|:----:|:----:|:----:|:----:| 120 | | **2x** - PSNR/SSIM| 29.56/0.8431 | 31.21/0.8863 | 31.36/0.8879 | 31.18/0.8855 | 31.40/0.8884 | **31.90/0.8960** | 31.85/0.8942 |31.79/| 121 | | **3x** - PSNR/SSIM| 27.21/0.7385 | 28.29/0.7835 | 28.41/0.7863 | 28.29/0.7840 | 28.50/0.7885 | **28.82/0.7976** | 28.80/0.7963 |28.76/| 122 | | **4x** - PSNR/SSIM| 25.96/0.6675 | 26.82/0.7087 | 26.90/0.7101 | 26.84/0.7106 | 27.03/0.7161 | **27.29/0.7251** | 27.23/0.7233 |27.25/| 123 | --------------------------------------------------------------------------------