├── LICENSE ├── README.md ├── img ├── highres.png ├── lowres.png ├── pg_10x12.png ├── pg_1x120.png ├── pg_2x60.png ├── pg_3x40.png ├── pg_4x15.png ├── pg_4x23.png ├── pg_4x3.png ├── pg_4x30.png ├── pg_4x5.png ├── pg_4x8.png ├── pg_6x20.png ├── pg_8x15.png ├── std_10.png ├── std_120.png ├── std_20.png ├── std_30.png ├── std_60.png └── std_90.png └── scripts └── hires_fix_pg.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Armit 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # stable-diffusion-webui-hires-fix-progressive 2 | 3 | A progressive upscaling-img2img version of hires.fix, extension script for AUTOMATIC1111/stable-diffusion-webui. 4 | 5 | ---- 6 | 7 | Highres.fix pipline gives us an inspirable way to sketch-and-refine an image, we could make it even further~ 8 | 9 | ⚠ This is possibly obsolete, try use [multidiffusion-upscaler-for-automatic1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) for img2img upscale instead :) 10 | 11 | 12 | ### Examples 13 | 14 | ⚪ progressive vs. official 15 | 16 | | official | ![std_10](img/std_10.png) | ![std_20](img/std_20.png) | ![std_30](img/std_30.png) | ![std_60](img/std_60.png) | ![std_90](img/std_90.png) | ![std_120](img/std_120.png) | 17 | | :-: | :-: | :-: | :-: | :-: | :-: | :-: | 18 | | hires steps | 10 | 20 | 30 | 60 | 90 | 120 | 19 | | **progressive** | ![pg_4x3](img/pg_4x3.png) | ![pg_4x5](img/pg_4x5.png) | ![pg_4x8](img/pg_4x8.png) | ![pg_4x15](img/pg_4x15.png) | ![pg_4x23](img/pg_4x23.png) | ![pg_4x30](img/pg_4x30.png) | 20 | | iter \* step | 4\*3=12 | 4\*5=20 | 4\*8=32 | 4\*15=60 | 4\*23=92 | 4\*30=120 | 21 | 22 | ℹ It is found that original hires.fix sometimes suffers from color red-shift, on the contratry, the progressive version suffers from blue-shift. I don't know why though... 23 | 24 | ⚪ balancing iter and step 25 | 26 | | progressive | ![pg_1x120](img/pg_1x120.png) | ![pg_2x60](img/pg_2x60.png) | ![pg_3x40](img/pg_3x40.png) | ![pg_4x30](img/pg_4x30.png) | ![pg_6x20](img/pg_6x20.png) | ![pg_8x15](img/pg_8x15.png) | ![pg_10x12](img/pg_10x12.png) | 27 | | :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: | 28 | | iter \* step = 120 | 1 \* 120 | 2 \* 60 | 3 \* 40 | 4 \* 30 | 6 \* 20 | 8 \* 15 | 10 \* 12 | 29 | 30 | 1x120: 31 | 32 | ![pg_1x120](img/pg_1x120.png) 33 | 34 | 2x60: 35 | 36 | ![pg_2x60](img/pg_2x60.png) 37 | 38 | 3x40: 39 | 40 | ![pg_3x40](img/pg_3x40.png) 41 | 42 | 4x30: 43 | 44 | ![pg_4x30](img/pg_4x30.png) 45 | 46 | 6x20: 47 | 48 | ![pg_6x20](img/pg_6x20.png) 49 | 50 | 8x15: 51 | 52 | ![pg_8x15](img/pg_8x15.png) 53 | 54 | 10x12: 55 | 56 | ![pg_10x12](img/pg_10x12.png) 57 | 58 | | raw lowres (txt2img ouput) | raw highres (upscale only) | 59 | | :-: | :-: | 60 | | ![lowres](img/lowres.png) | ![highres](img/highres.png) | 61 | 62 | Commom parameters for above demo images: 63 | 64 | ```text 65 | Steps: 20 66 | Sampler: Euler a 67 | CFG scale: 7 68 | Seed: 1919810 69 | Size: 512 x 512 70 | Target size: 1024 x 1024 71 | Upscaler: Lanczos 72 | Denoising strength: 0.5 73 | ``` 74 | 75 | 76 | ### How it works 77 | 78 | ⚪ How the original hires.fix works? 79 | 80 | It takes three steps: 81 | 82 | - Several steps of `txt2img` to sketch a low resolution image 83 | - One step `upscale` the low resolution image to a higher target resoultion 84 | - Yet another several steps of `img2img` to refine the high resolution image 85 | 86 | ⚪ How a **progressive** hires.fix works? 87 | 88 | Just make the upscale & img2img step repeatible and progressive: 89 | 90 | - Several steps of txt2img to sketch a low resolution image 91 | - Repeat the follwing steps until reaching the target resoultion 92 | - Upscale low resolution image to some higher itermediate resoultion 93 | - Small steps of img2img on the higher resolution image 94 | 95 | 96 | ### Options 97 | 98 | - upscale_* 99 | - upscaler: (categorical), upscaleing method same as in hires.fix 100 | - resize_mode: 101 | - target_(width|height): (int), final image resolution 102 | - img2img_* 103 | - iter: (int), the repeated upscale-img2img iteration 104 | - step: (int), img2img steps per iteration 105 | - denoise_strength: (float), denoise strength for img2img 106 | - prompt: (string), use another prompt for img2img, default to the main prompt box 107 | 108 | 109 | ### Installation 110 | 111 | Easiest way to install it is to: 112 | 1. Go to the "Extensions" tab in the webui, switch to the "Install from URL" tab 113 | 2. Paste https://github.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive.git into "URL for extension's git repository" and click install 114 | 115 | Manual install: 116 | 1. Copy this repo folder to the 'extensions' folder of https://github.com/AUTOMATIC1111/stable-diffusion-webui 117 | 118 | ---- 119 | 120 | by Armit 121 | 2023/01/12 122 | -------------------------------------------------------------------------------- /img/highres.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/highres.png -------------------------------------------------------------------------------- /img/lowres.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/lowres.png -------------------------------------------------------------------------------- /img/pg_10x12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_10x12.png -------------------------------------------------------------------------------- /img/pg_1x120.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_1x120.png -------------------------------------------------------------------------------- /img/pg_2x60.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_2x60.png -------------------------------------------------------------------------------- /img/pg_3x40.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_3x40.png -------------------------------------------------------------------------------- /img/pg_4x15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x15.png -------------------------------------------------------------------------------- /img/pg_4x23.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x23.png -------------------------------------------------------------------------------- /img/pg_4x3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x3.png -------------------------------------------------------------------------------- /img/pg_4x30.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x30.png -------------------------------------------------------------------------------- /img/pg_4x5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x5.png -------------------------------------------------------------------------------- /img/pg_4x8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_4x8.png -------------------------------------------------------------------------------- /img/pg_6x20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_6x20.png -------------------------------------------------------------------------------- /img/pg_8x15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/pg_8x15.png -------------------------------------------------------------------------------- /img/std_10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_10.png -------------------------------------------------------------------------------- /img/std_120.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_120.png -------------------------------------------------------------------------------- /img/std_20.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_20.png -------------------------------------------------------------------------------- /img/std_30.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_30.png -------------------------------------------------------------------------------- /img/std_60.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_60.png -------------------------------------------------------------------------------- /img/std_90.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Kahsolt/stable-diffusion-webui-hires-fix-progressive/049255d43efe1b94bebf4c712c559170ef16a589/img/std_90.png -------------------------------------------------------------------------------- /scripts/hires_fix_pg.py: -------------------------------------------------------------------------------- 1 | from PIL.Image import Image 2 | from typing import List 3 | 4 | import gradio as gr 5 | import numpy as np 6 | 7 | from modules.scripts import Script 8 | from modules.shared import state, sd_upscalers 9 | from modules.processing import process_images, StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img, Processed 10 | from modules.images import resize_image 11 | 12 | DEFAULT_TARGET_WIDTH = 1024 13 | DEFAULT_TARGET_HEIGHT = 1024 14 | DEFAULT_ITER = 2 15 | DEFAULT_STEP = 30 16 | DEFAULT_DENOISE_W = 0.5 17 | DEFAULT_ALTER_PROMPT = None 18 | DEFAULT_UPSCALER = 'Lanczos' 19 | DEFAULT_RESIZE_MODE = 'Adjust' 20 | DEFAULT_SAVE_INTERM = False 21 | 22 | CHOICE_UPSCALER = [x.name for x in sd_upscalers if x.name not in [None, 'None']] 23 | CHOICE_RESIZE_MODE = ['Adjust', 'Crop', 'Fill'] 24 | 25 | 26 | def txt2img_to_img2img(p:StableDiffusionProcessingTxt2Img) -> StableDiffusionProcessingImg2Img: 27 | KNOWN_KEYS = [ # see `StableDiffusionProcessing.__init__()` 28 | 'sd_model', 29 | 'outpath_samples', 30 | 'outpath_grids', 31 | 'prompt', 32 | 'styles', 33 | 'seed', 34 | 'subseed', 35 | 'subseed_strength', 36 | 'seed_resize_from_h', 37 | 'seed_resize_from_w', 38 | 'seed_enable_extras', 39 | 'sampler_name', 40 | 'batch_size', 41 | 'n_iter', 42 | 'steps', 43 | 'cfg_scale', 44 | 'width', 45 | 'height', 46 | 'restore_faces', 47 | 'tiling', 48 | 'do_not_save_samples', 49 | 'do_not_save_grid', 50 | 'extra_generation_params', 51 | 'overlay_images', 52 | 'negative_prompt', 53 | 'eta', 54 | 'do_not_reload_embeddings', 55 | 'denoising_strength', 56 | 'ddim_discretize', 57 | 's_churn', 58 | 's_tmax', 59 | 's_tmin', 60 | 's_noise', 61 | 'override_settings', 62 | 'override_settings_restore_afterwards', 63 | 'sampler_index', 64 | 'script_args', 65 | ] 66 | kwargs = { k: getattr(p, k) for k in dir(p) if k in KNOWN_KEYS } # inherit params 67 | return StableDiffusionProcessingImg2Img(**kwargs) 68 | 69 | 70 | class Script(Script): 71 | 72 | def title(self): 73 | return 'Hires.fix Progressive' 74 | 75 | def describe(self): 76 | return "A progressive version of hires.fix implementation." 77 | 78 | def show(self, is_img2img): 79 | return not is_img2img 80 | 81 | def ui(self, is_img2img): 82 | with gr.Row(): 83 | upscaler = gr.Dropdown(label='Upscaler', value=lambda: DEFAULT_UPSCALER, choices=CHOICE_UPSCALER) 84 | resize_mode = gr.Dropdown(label='Resize mode', value=lambda: DEFAULT_RESIZE_MODE, choices=CHOICE_RESIZE_MODE) 85 | target_width = gr.Slider(label='Target width', value=lambda: DEFAULT_TARGET_WIDTH, minimum=512, maximum=2048, step=8) 86 | target_height = gr.Slider(label='Target height', value=lambda: DEFAULT_TARGET_HEIGHT, minimum=512, maximum=2048, step=8) 87 | 88 | with gr.Group(): 89 | with gr.Row(): 90 | iters = gr.Slider(label='Iteration', value=lambda: DEFAULT_ITER, minimum=0, maximum=30, step=1) 91 | steps = gr.Slider(label='Img2img steps (per iter)', value=lambda: DEFAULT_STEP, minimum=1, maximum=150, step=1) 92 | denoising_strength = gr.Slider(label='Denoising strength', value=lambda: DEFAULT_DENOISE_W, minimum=0.0, maximum=1.0, step=0.01) 93 | with gr.Row(): 94 | alter_prompt = gr.TextArea(label='Img2img prompt', value=lambda: DEFAULT_ALTER_PROMPT, lines=2) 95 | 96 | with gr.Row(): 97 | save_interm = gr.Checkbox(label='Save intermediate images', value=lambda: DEFAULT_SAVE_INTERM) 98 | 99 | return [iters, 100 | upscaler, resize_mode, target_width, target_height, 101 | steps, denoising_strength, alter_prompt, 102 | save_interm] 103 | 104 | def run(self, p:StableDiffusionProcessingTxt2Img, 105 | iters:int, 106 | upscaler:str, resize_mode:str, target_width:int, target_height:int, 107 | steps:int, denoising_strength:float, alter_prompt:str, 108 | save_interm:bool): 109 | 110 | if 'force disable original hires.fix': 111 | p.enable_hr = False 112 | p.denoising_strength = None 113 | p.extra_generation_params.pop("Hires upscale", None) 114 | p.extra_generation_params.pop("Hires resize", None) 115 | 116 | if 'reuse to fix violate things': 117 | seed = p.seed 118 | subseed = p.subseed 119 | subseed_strength = p.subseed_strength 120 | 121 | images: List[Image] = [] 122 | 123 | ''' Txt2Img ''' 124 | if not save_interm: 125 | p.do_not_save_grid = iters > 0 126 | p.do_not_save_samples = iters > 0 127 | proc = process_images(p) 128 | info: str = proc.info 129 | imgs = proc.images 130 | images.extend(imgs) 131 | 132 | resize_mode: int = CHOICE_RESIZE_MODE.index(resize_mode) 133 | widths: np.array = np.linspace(p.width, target_width , iters + 1)[1:].round().astype(int) 134 | heights: np.array = np.linspace(p.height, target_height, iters + 1)[1:].round().astype(int) 135 | 136 | state.nextjob() 137 | 138 | p:StableDiffusionProcessingImg2Img = txt2img_to_img2img(p) 139 | p.steps = int(steps / denoising_strength) # NOTE: to work alike original hires.fix 140 | p.prompt = alter_prompt or p.prompt 141 | p.denoising_strength = denoising_strength 142 | p.resize_mode = resize_mode 143 | 144 | for i in range(iters): 145 | if state.interrupted: break 146 | 147 | ''' Upscale ''' 148 | imgs = [resize_image(resize_mode, img, widths[i], heights[i], upscaler_name=upscaler) for img in imgs] 149 | 150 | if steps <= 0: continue # upscale only, no img2img 151 | 152 | ''' Img2Img ''' 153 | rw, rh = imgs[0].size 154 | p.width = rw 155 | p.height = rh 156 | p.init_images = imgs 157 | p.seed = seed 158 | p.subseed = subseed 159 | p.subseed_strength = subseed_strength 160 | 161 | if not save_interm: 162 | p.do_not_save_grid = i < iters - 1 163 | p.do_not_save_samples = i < iters - 1 164 | proc = process_images(p) 165 | info = proc.info 166 | imgs = proc.images 167 | images.extend(imgs) 168 | 169 | ret_imgs = images if save_interm else imgs 170 | 171 | return Processed(p, ret_imgs, p.seed, info) 172 | --------------------------------------------------------------------------------