├── README.md ├── code ├── .idea │ ├── code.iml │ ├── encodings.xml │ ├── misc.xml │ ├── modules.xml │ └── workspace.xml ├── __init__.py ├── data │ ├── __init__.py │ ├── benchmark.py │ ├── common.py │ ├── demo.py │ ├── div2k.py │ └── srdata.py ├── dataloader.py ├── demo.sh ├── loss │ ├── __init__.py │ ├── adversarial.py │ ├── discriminator.py │ └── vgg.py ├── main.py ├── model │ ├── __init__.py │ ├── common.py │ ├── ddbpn.py │ ├── edsr.py │ ├── mdsr.py │ └── physics_sr.py ├── option.py ├── plot_epoches.py ├── plot_psnr_in_log_txt.py ├── psnr_refine1_refine2.pdf ├── psnr_refine1_refine2_x3.pdf ├── psnr_refine1_refine2_x4.pdf ├── template.py ├── tools │ ├── downsampling_jpeg.m │ └── jpeg2binary.py ├── trainer.py └── utility.py ├── experiment └── training results can be found here ├── models └── please put all the models here └── testset └── please put the testset here /README.md: -------------------------------------------------------------------------------- 1 | # PHYSICS_SR 2 | 3 | This repository is an official PyTorch implementation of the paper "Image Formation Model Guided Deep Image Super-Resolution" from AAAI 2020. 4 | The code is built on EDSR (PyTorch) and tested on Ubuntu 16.04 environment (Python3.6, PyTorch_0.4.1, CUDA8.0, cuDNN5.1) with Tesla V100/1080Ti GPUs. 5 | 6 | 7 | ## Dependencies 8 | * ubuntu16.04 9 | * Python 3.6(Recommend to use Anaconda) 10 | * PyTorch0.4.1 11 | * numpy 12 | * skimage 13 | * imageio 14 | * matplotlib 15 | * tqdm 16 | * cv2 17 | 18 | ## Get started 19 | 20 | #### Training dataset: 21 | We use the DIV2K dataset to train our models. You can download it from [here](https://data.vision.ee.ethz.ch/cvl/DIV2K/) 22 | 23 | #### Benechmarks: 24 | You can evaluate our models with widely-used benchmark datasets: 25 | 26 | [Set5 - Bevilacqua et al. BMVC 2012](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html), 27 | 28 | [Set14 - Zeyde et al. LNCS 2010](https://sites.google.com/site/romanzeyde/research-interests), 29 | 30 | [B100 - Martin et al. ICCV 2001](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/), 31 | 32 | [Urban100 - Huang et al. CVPR 2015](https://sites.google.com/site/jbhuang0604/publications/struct_sr). 33 | 34 | #### Models 35 | All the models(X2, X3, X4) can be downloaded from [GoogleDrive](https://drive.google.com/open?id=1ns0zFBZgOCFxafBALRq7_wO-jrUZqNfH). 36 | 37 | ## Quicktest with benchmark 38 | You can test our super-resolution algorithm with benchmarks. Please organize the testset in ``testset`` folder like this: 39 | ``` 40 | |--testset 41 | |--Set5 42 | |--LR 43 | |--X2 44 | |--babyx2.png 45 | ： 46 | ： 47 | |--X3 48 | |--X4 49 | |--HR 50 | |--baby.png 51 | ： 52 | ： 53 | ``` 54 | 55 | Then, run the following commands: 56 | ```bash 57 | cd code 58 | python main.py --dir_data ../testset --data_test Set5 --model physics_sr --pre_train ../models/X2/model_best.pt --scale 2 --save physics_sr_x2 --save_results --test_only 59 | ``` 60 | And generated results can be found in ``./experiment/physics_sr_x2/results/`` 61 | * To test all scales, you can modify the options(pre_train, scale) of the command above. 62 | * To test other benchmarks, you can modify the option(data_test) of the command above. 63 | * To change the save root, you can modify the option(save) of the command above. 64 | 65 | 66 | ## How to train 67 | If you have downloaded the trainset, please make sure that the trainset has been organized as follows: 68 | ``` 69 | |--DIV2K 70 | |--train 71 | |--DIV2K_train_LR_bicubic 72 | |--X2 73 | |--0001x2.png 74 | ： 75 | ： 76 | |--X3 77 | |--X4 78 | |--DIV2K_train_HR 79 | |--0001.png 80 | ： 81 | ： 82 | ``` 83 | The command for training is as follow: 84 | ``` 85 | cd code 86 | python main.py --dir_data --data_test DIV2K --model physics_sr --scale 2 --save physics_sr_x2 --save_results 87 | ``` 88 | The trained model can be found in ``./experiment/physics_sr_x2/model`` 89 | * To train all scales, you can modify the option(scale) of the command above. 90 | * To test other benchmarks, you can modify the option(data_test) of the command above. 91 | * To change the save root, you can modify the option(save) of the command above. 92 | 93 | 94 | ## Citation 95 | If our work is useful in your research or publication, please cite our work: 96 | ``` 97 | 98 | @inproceedings{pan2020physics_sr, 99 | title={Image Formation Model Guided Deep Image Super-Resolution}, 100 | author={Jinshan Pan, Yang Liu, Deqing Sun, Jimmy Ren, Ming-Ming Cheng, Jian Yang, and Jinhui Tang}, 101 | booktitle={AAAI}, 102 | year={2020} 103 | } 104 | ``` 105 | 106 | ## Acknowledgements 107 | This code is built on EDSR(PyTorch). We thank the authors for sharing their codes of EDSR. 108 | -------------------------------------------------------------------------------- /code/.idea/code.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 11 | -------------------------------------------------------------------------------- /code/.idea/encodings.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /code/.idea/misc.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 6 | -------------------------------------------------------------------------------- /code/.idea/modules.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /code/.idea/workspace.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 |