├── LICENSE ├── README.md └── scripts └── loopback_scaler.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Elldreth 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # loopback_scaler 2 | Automatic1111 python script to enhance image quality 3 | 4 | ## Overview 5 | The Loopback Scaler is an Automatic1111 Python script that enhances image resolution and quality using an iterative process. The code takes an input image and performs a series of image processing steps, including denoising, resizing, and applying various filters. The algorithm loops through these steps multiple times, with user-defined parameters controlling how the image evolves at each iteration. The result is an improved image, often with more detail, better color balance, and fewer artifacts than the original. 6 | 7 | ## Key features 8 | - **Iterative enhancement**: The script processes the input image in several loops, with each loop increasing the resolution and refining the image quality. The image result from one loop is then inserted as the input image for the next loop which continually builds on what has been created. 9 | - **Denoise Change**: The denoising strength can be adjusted for each loop, allowing users to strike a balance between preserving details and reducing artifacts. 10 | - **Dimension Increase**: The script adjusts the amount of resolution increase per loop based on the easing option chosen. This helps to improve image quality at different points in the looping. 11 | - **Image filters**: Users can apply various PIL Image Filters to the final image, including detail enhancement, blur, smooth, and contour filters. 12 | - **Image adjustments**: The script provides sliders to fine-tune the sharpness, brightness, color, and contrast of the final image. 13 | 14 | Recommended settings for img2img processing are provided in the script, including resize mode, sampling method, width/height, CFG scale, denoising strength, and seed. 15 | 16 | Please note that the performance of the Loopback Scaler depends on the GPU, input image, and user-defined parameters. Experimenting with different settings can help you achieve the desired results. 17 | 18 | ## Tips, Tricks, and Advice 19 | - Do **NOT** expect to recreate images with prompts using this method. 20 | - You can start from txt2img with a prompt. Generate your image and then send it over to img2img. When creating images for this process, shoot for lower resolution images (512x768, 340x512, etc.) 21 | - **ALWAYS** have a prompt in your img2img tab when doing this process, unless you are interested in creating chaos :D. Your results will usually be poor, but you CAN put a different prompt in img2img than what you created the source image with. Pretty interesting results come from this method. 22 | - When using models that require VAE, keep the number of loops lower than normal because it will cause the image to fade each iteration. Luckily you can add Color and Sharpness back in with the PIL enhancements if you need. 23 | - Don't set your maximum Width/Height higher than what you can normally generate. This script is not an upscaler model and isn't intended to make giant images. It is intended to give you detailed quality images that you can send to an upscaler. 24 | - Once installed, there is an Info panel at the bottom of the script interface to help you understand the settings and what they do. 25 | 26 | ## Manual Installation 27 | 1. Unzip the `loopback_scaler.py` script. 28 | 2. Move the script to the `\stable-diffusion-webui\scripts` folder. 29 | 3. Close the Automatic1111 webui console window. 30 | 4. Relaunch the webui by running the `webui-user.bat` file. 31 | 5. Open your web browser and navigate to the Automatic1111 page or refresh the page if it's already open. 32 | 33 | # Settings Guide 34 | 35 | ## Loops 36 | The number of times the script will inference your image and increase the resolution in increments. The amount the resolution is increased each loop is determined by this number and the maximum image width/height. The more loops, the more chances of your image picking up more detail, but also artifacts. 4 to 10 is what I find to work best, but you may like more or less. 37 | 38 | ## Denoise change 39 | This setting will increase or decrease the denoising strength every loop. A higher value will increase the denoising strength, while a lower value will decrease it. A setting of 1 keeps the denoising strength as it is set on the img2img settings. 40 | 41 | ## Dimension Increase 42 | This setting changes the amount of resolution increase per loop, keeping the changes from being linear. You will get non-linear increases in image size based on which easing option you choose. To increase the image size earlier in the process, choose one of the 'Ease Out' options, to increase the image size later in the process, choose an 'Ease In' option, to place image increases more toward the center of the process, use an 'Ease InOut' option. 43 | 44 | ## Maximum Image Width/Height 45 | These parameters set the maximum width and height of the final image. Always start with an image smaller than these dimensions. The smaller you start, the more impressive the results. I usually start at either 340x512 or 512x768. 46 | 47 | ## Detail, Blur, Smooth, Contour 48 | These parameters are checkboxes that apply a PIL Image Filter to the final image. 49 | 50 | ## Sharpness, Brightness, Color, Contrast 51 | These parameters are sliders that adjust the sharpness, brightness, color, and contrast of the image. 1 will result in no adjustments, less than one reduces these settings for the final image and greater than 1 increases these settings. 52 | 53 | ## Img2Img Settings 54 | I recommend creating an image with txt2img and then sending the result to img2img with the prompt and settings. For this script, I use these settings: 55 | 56 | - **Resize mode**: Crop and resize 57 | - **Sampling method**: DDIM 58 | - **Sampling steps**: 30 59 | - **Width/Height**: 340x512 or 512x768. I’d try to keep to the aspect ratio of the original image, but these can be set lower than the resolution of the original image. 60 | - **CFG Scale**: 6 to 8 61 | - **Denoising strength**: 0.2 to 0.4 is usual. The lower you go, the less change between loops. The higher you go, the less the end result will look like the original image. 62 | - **Seed**: This doesn’t matter too much, I usually keep it at -1 63 | -------------------------------------------------------------------------------- /scripts/loopback_scaler.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import math 3 | import modules.scripts as scripts 4 | import gradio as gr 5 | import time 6 | from modules import processing, images 7 | from modules.processing import Processed 8 | from modules.shared import opts, state 9 | 10 | # Import PIL libraries 11 | from PIL import ImageFilter, ImageEnhance 12 | 13 | # This is a modification of the Loopback script. Thank you to the original author for making this available. 14 | # This modification came from a process that I learned from the AI community to improve details and prepare an 15 | # image for post-processing. 16 | 17 | class Script(scripts.Script): 18 | def title(self): 19 | return "Loopback Scaler" 20 | 21 | def show(self, is_img2img): 22 | return is_img2img 23 | help_text = "Loops: The number of times the script will inference your image and increase the resolution in increments. The amount the resolution is increased each loop is determined by this number and the maximum image width/height. The more loops, the more chances of your image picking up more detail, but also artifacts. 4 to 10 is what I find to work best, but you may like more or less.

Denoise change: This setting will increase or decrease the denoising strength every loop. A higher value will increase the denoising strength, while a lower value will decrease it. A setting of 1 keeps the denoising strength as it is set on the img2img settings.

Dimension change: This setting changes the amount of resolution increase or decrease per loop, keeping the changes from being linear. You will get non-linear increases in image size based on which easing option you choose. To increase the image size earlier in the process, choose one of the 'Ease Out' options, to increase the image size later in the process, choose an 'Ease In' option, to place image increases more toward the center of the process, use an 'Ease InOut' option.

Maximum Image Width/Height: These parameters set the maximum width and height of the final image. Always start with an image smaller than these dimensions. The smaller you start, the more impressive the results. I usually start at either 340x512 or 512x768

Detail, Blur, Smooth, Contour: These parameters are checkboxes that apply a PIL Image Filter to the final image.

Sharpness, Brightness, Color, Contrast: These parameters are sliders that adjust the sharpness, brightness, color, and contrast of the image. 1 will result in no adjustments, less than one reduces these settings for the final image and greater than 1 increases these settings.

Img2Img Settings: I recommend creating an image with txt2img and then sending the result to img2img with the prompt and settings. For this script I use these settings..

Resize mode - Crop and resize
Sampling method - DDIM
Sampling steps - 30
Width/Height - 340x512 or 512x768. I’d try to keep to the aspect ratio of the original image but these can be set lower than the resolution of the original image
CFG Scale - 6 to 8
Denoising strength - 0.2 to 0.4 is usual. The lower you go, the less change between loops. The higher you go the less the end result will look like the original image.
Seed - This doesn’t matter too much, I usually keep it at -1

" 24 | detail_choices = ["None", "Low", "Medium", "High"] 25 | dim_increase_options = ["Linear", 26 | "Ease In: Sine", 27 | "Ease In: Cubic", 28 | "Ease In: Quint", 29 | "Ease In: Circ", 30 | "Ease Out: Sine", 31 | "Ease Out: Cubic", 32 | "Ease Out: Quint", 33 | "Ease Out: Circ", 34 | "Ease InOut: Sine", 35 | "Ease InOut: Cubic", 36 | "Ease InOut: Quint", 37 | "Ease InOut: Circ"] 38 | def ui(self, is_img2img): 39 | with gr.Blocks(): 40 | with gr.Box(): 41 | with gr.Row(): 42 | dimension_increment_factor = gr.Dropdown(label='Dimension Increase:', choices=self.dim_increase_options, value="Linear", elem_id=self.elem_id("dimension_increment_factor")) 43 | with gr.Row(): 44 | loops = gr.Slider(minimum=1, maximum=32, step=1, label='Loops:', value=4, elem_id=self.elem_id("loops")) 45 | with gr.Row(): 46 | denoising_strength_change_factor = gr.Slider(minimum=0.9, maximum=1.1, step=0.01, label='Denoise Change:', value=1, elem_id=self.elem_id("denoising_strength_change_factor")) 47 | with gr.Box(): 48 | with gr.Row(): 49 | max_width = gr.Slider(minimum=512, maximum=4096, step=64, label='Maximum Image Width:', value=1024, elem_id=self.elem_id("max_width")) 50 | max_height = gr.Slider(minimum=512, maximum=4096, step=64, label='Maximum Image Height:', value=1024, elem_id=self.elem_id("max_height")) 51 | with gr.Row(): 52 | use_scale = gr.Checkbox(label='Use Scale', value=False, elem_id=self.elem_id("use_scale")) 53 | scale = gr.Slider(minimum=.5, maximum=4, step=.1, label='Scale Final Image:', value=1, elem_id=self.elem_id("final_image_scale")) 54 | with gr.Accordion("Final Image Filters"): 55 | with gr.Box(): 56 | with gr.Row(): 57 | detail_strength = gr.Dropdown(label='Add Detail', choices=self.detail_choices, value="None", elem_id=self.elem_id("detail_strength")) 58 | blur_strength = gr.Dropdown(label='Add Blur', choices=self.detail_choices, value="None", elem_id=self.elem_id("blur_bool")) 59 | smooth_strength = gr.Dropdown(label='Smoothing', choices=self.detail_choices, value="None", elem_id=self.elem_id("smooth_strength")) 60 | contour_bool = gr.Checkbox(label='Contour', value=False, elem_id=self.elem_id("contour_bool")) 61 | with gr.Box(): 62 | with gr.Row(): 63 | sharpness_strength = gr.Slider(minimum=0.1, maximum=2.0, step=0.01, label='Sharpness:', value=1.0, elem_id=self.elem_id("sharpness_strength")) 64 | brightness_strength = gr.Slider(minimum=0.1, maximum=2.0, step=0.01, label='Brightness:', value=1.0, elem_id=self.elem_id("brightness_strength")) 65 | with gr.Row(): 66 | color_strength = gr.Slider(minimum=0.1, maximum=2.0, step=0.01, label='Color:', value=1.0, elem_id=self.elem_id("color_strength")) 67 | contrast_strength = gr.Slider(minimum=0.1, maximum=2.0, step=0.01, label='Contrast:', value=1.0, elem_id=self.elem_id("contrast_strength")) 68 | with gr.Accordion("Info - Loopback Scaler", open=False): 69 | helpinfo = gr.HTML("

{}

".format(self.help_text)) 70 | return [helpinfo, loops, denoising_strength_change_factor, max_width, max_height, scale, use_scale, detail_strength, blur_strength, contour_bool, smooth_strength, sharpness_strength, brightness_strength, color_strength, contrast_strength, dimension_increment_factor] 71 | 72 | def __get_width_from_ratio(self, height, ratio): 73 | new_width = math.floor(height / ratio) 74 | return new_width 75 | 76 | def __get_height_from_ratio(self, width, ratio): 77 | new_height = math.floor(width * ratio) 78 | return new_height 79 | 80 | def __get_strength_iterations(self, strength): 81 | if strength == "None": return 0 82 | elif strength == "Low": return 1 83 | elif strength == "Medium": return 2 84 | elif strength == "High": return 3 85 | return 0 86 | 87 | def __get_dimension_increment(self, option, perc): 88 | if option == "Linear": return perc 89 | elif option == "Ease In: Sine": return 1 - math.cos((perc * math.pi)/2) 90 | elif option == "Ease In: Cubic": return perc * perc * perc 91 | elif option == "Ease In: Quint": return perc * perc * perc * perc 92 | elif option == "Ease In: Circ": return 1 - math.sqrt(1 - math.pow(perc, 2)) 93 | elif option == "Ease Out: Sine": return math.sin((perc * math.pi) / 2) 94 | elif option == "Ease Out: Cubic": return 1 - pow(1 - perc, 3) 95 | elif option == "Ease Out: Quint": return 1 - pow(1 - perc, 5) 96 | elif option == "Ease Out: Circ": return math.sqrt(1 - math.pow(perc - 1, 2)) 97 | elif option == "Ease InOut: Sine": return -(math.cos(math.pi * perc) - 1) / 2 98 | elif option == "Ease InOut: Cubic": return 4 * perc * perc * perc if perc < 0.5 else 1 - math.pow(-2 * perc + 2, 3) / 2 99 | elif option == "Ease InOut: Quint": return 16 * perc * perc * perc * perc * perc if perc < 0.5 else 1 - math.pow(-2 * perc + 2, 5) / 2 100 | elif option == "Ease InOut: Circ": return (1 - math.sqrt(1 - math.pow(2 * perc, 2))) / 2 if perc < 0.5 else (math.sqrt(1 - math.pow(-2 * perc + 2, 2)) + 1) / 2 101 | return perc 102 | 103 | def __resize_to_nearest_multiple_of_m(self, width, height, m=8): 104 | aspect_ratio = width / height 105 | if width < height: 106 | new_width = math.ceil(width / m) * m 107 | new_height = round(new_width / aspect_ratio) 108 | new_height = math.ceil(new_height / m) * m 109 | else: 110 | new_height = math.ceil(height / m) * m 111 | new_width = round(new_height * aspect_ratio) 112 | new_width = math.ceil(new_width / m) * m 113 | 114 | return int(new_width), int(new_height) 115 | 116 | def run(self, p, _, loops, denoising_strength_change_factor, max_width, max_height, scale, use_scale, detail_strength, blur_strength, contour_bool, smooth_strength, sharpness_strength, brightness_strength, color_strength, contrast_strength, dimension_increment_factor): 117 | start_time = time.time() 118 | processing.fix_seed(p) 119 | batch_count = p.n_iter 120 | p.extra_generation_params = { 121 | "Denoising strength change factor": denoising_strength_change_factor, 122 | "Dimension increment factor": dimension_increment_factor, 123 | "Add Detail": detail_strength, 124 | "Add Blur": blur_strength, 125 | "Smoothing": smooth_strength, 126 | "Contour": contour_bool, 127 | "Sharpness": sharpness_strength, 128 | "Brightness": brightness_strength, 129 | "Color Strength": color_strength, 130 | "Contrast": contrast_strength, 131 | } 132 | 133 | p.batch_size = 1 134 | p.n_iter = 1 135 | 136 | initial_seed = None 137 | initial_info = None 138 | 139 | all_images = [] 140 | original_init_image = p.init_images 141 | original_prompt = p.prompt 142 | state.job_count = loops * batch_count 143 | 144 | initial_color_corrections = [processing.setup_color_correction(p.init_images[0])] 145 | 146 | #determine oritinal image h/w ratio and max h/w ratio 147 | base_ratio = p.height / p.width 148 | 149 | final_height = math.floor(p.height * scale) if use_scale else max_height 150 | final_width = math.floor(p.width * scale) if use_scale else max_width 151 | 152 | orig_height_diff = final_height - p.height 153 | orig_width_diff = final_width - p.width 154 | 155 | orig_height = p.height 156 | orig_width = p.width 157 | 158 | max_ratio = final_height / final_width 159 | use_height = base_ratio >= max_ratio 160 | 161 | print("Starting Loopback Scaler") 162 | print(f"Original size: {p.width}x{p.height}") 163 | print(f"Final size: {final_width}x{final_height}") 164 | print(f"Denoising: {denoising_strength_change_factor}") 165 | print(f"Dimension change: {dimension_increment_factor}") 166 | 167 | for n in range(batch_count): 168 | history = [] 169 | 170 | # Reset to original init image at the start of each batch 171 | p.init_images = original_init_image 172 | 173 | for i in range(loops): 174 | p.n_iter = 1 175 | p.batch_size = 1 176 | p.do_not_save_grid = True 177 | loop_fraction = i/loops 178 | easing_factor = self.__get_dimension_increment(dimension_increment_factor, loop_fraction) 179 | 180 | last_image = i == loops - 1 181 | 182 | calc_height = final_height if last_image else (int((orig_height_diff * easing_factor) + orig_height )) 183 | calc_width = final_width if last_image else (int((orig_width_diff * easing_factor) + orig_width)) 184 | 185 | if use_height: 186 | p.width, p.height = self.__resize_to_nearest_multiple_of_m(width=self.__get_width_from_ratio(calc_height, base_ratio), height=calc_height) 187 | else: 188 | p.width, p.height = self.__resize_to_nearest_multiple_of_m(width=calc_width, height=self.__get_height_from_ratio(calc_height, base_ratio)) 189 | 190 | print() 191 | print(f"Loopback Scaler: {i+1}/{loops}") 192 | print(f"Iteration size: {p.width}x{p.height}") 193 | print(f"Denoising strength: {p.denoising_strength}") 194 | 195 | if opts.img2img_color_correction: 196 | p.color_corrections = initial_color_corrections 197 | 198 | state.job = f"Iteration {i + 1}/{loops}, batch {n + 1}/{batch_count}" 199 | 200 | # Processing image 201 | processed = processing.process_images(p) 202 | 203 | if last_image: 204 | processed.images[0] = ImageEnhance.Sharpness(processed.images[0]).enhance(sharpness_strength) 205 | processed.images[0] = ImageEnhance.Brightness(processed.images[0]).enhance(brightness_strength) 206 | processed.images[0] = ImageEnhance.Color(processed.images[0]).enhance(color_strength) 207 | processed.images[0] = ImageEnhance.Contrast(processed.images[0]).enhance(contrast_strength) 208 | 209 | for j in range(self.__get_strength_iterations(detail_strength)): 210 | processed.images[0] = processed.images[0].filter(ImageFilter.DETAIL) 211 | 212 | for j in range(self.__get_strength_iterations(smooth_strength)): 213 | processed.images[0] = processed.images[0].filter(ImageFilter.SMOOTH) 214 | 215 | for j in range(self.__get_strength_iterations(blur_strength)): 216 | processed.images[0] = processed.images[0].filter(ImageFilter.BLUR) 217 | 218 | if contour_bool == True: 219 | processed.images[0] = processed.images[0].filter(ImageFilter.CONTOUR) 220 | 221 | images.save_image(processed.images[0], p.outpath_samples, "img2img", initial_seed, original_prompt, opts.samples_format, info=processed.info, short_filename=False,p=p) 222 | history.append(processed.images[0]) 223 | 224 | if initial_seed is None: 225 | initial_seed = processed.seed 226 | initial_info = processed.info 227 | 228 | init_img = processed.images[0] 229 | 230 | p.init_images = [init_img] 231 | p.seed = processed.seed + 1 232 | p.all_seeds.append(p.seed) 233 | p.all_subseeds.append(p.subseed) 234 | p.all_prompts.append(p.prompt) 235 | 236 | p.denoising_strength = min(max(p.denoising_strength * denoising_strength_change_factor, 0.1), 1) 237 | 238 | all_images += history 239 | 240 | end_time = time.time() 241 | print("Loopback Scaler: All Done!") 242 | print(f"LS: {round(end_time - start_time)}s elapsed") 243 | print() 244 | processed = Processed(p, all_images, p.all_seeds, initial_info,) 245 | 246 | return processed --------------------------------------------------------------------------------