├── .github └── workflows │ └── publish.yml ├── LICENSE ├── README.md ├── __init__.py ├── custom_samplers.py ├── dress_example.json ├── presets_to_add.py ├── pyproject.toml └── sampler_explaination.md /.github/workflows/publish.yml: -------------------------------------------------------------------------------- 1 | name: Publish to Comfy registry 2 | on: 3 | workflow_dispatch: 4 | push: 5 | branches: 6 | - main 7 | - master 8 | paths: 9 | - "pyproject.toml" 10 | 11 | permissions: 12 | issues: write 13 | 14 | jobs: 15 | publish-node: 16 | name: Publish Custom Node to registry 17 | runs-on: ubuntu-latest 18 | if: ${{ github.repository_owner == 'Extraltodeus' }} 19 | steps: 20 | - name: Check out code 21 | uses: actions/checkout@v4 22 | - name: Publish Custom Node 23 | uses: Comfy-Org/publish-node-action@v1 24 | with: 25 | ## Add your own personal access token to your Github Repository secrets and reference it here. 26 | personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }} 27 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DistanceSampler 2 | 3 | A custom experimental sampler based on relative distances. The first few steps are slower and then the sampler accelerates (the end is made with Heun). 4 | 5 | The idea is to get a more precise start since this is when most of the work is being done. 6 | 7 | [A more technical explaination](https://github.com/Extraltodeus/DistanceSampler/blob/main/sampler_explaination.md) 8 | 9 | Pros: 10 | 11 | - Less body horror / merged fused people. 12 | - Little steps required (4-10, recommanded general use: 7 with beta or AYS) 13 | - Can sample simple subjects without unconditional prediction (meaning with a CFG scale at 1) with a good quality. 14 | 15 | Cons: 16 | 17 | - Slow, which is also a plus. Relax, the image is generating ⛱ (but really since it requires little amounts of steps while giving a lesser percentage of horrors I personally prefer it) 18 | 19 | Variations: 20 | 21 | - The variation having a "n" in the name stands for "negative" and makes use of the unconditional prediction so to determin the best output. The results may vary depending on your negative prompt. In general it seems to make less mistakes. This is what I sample with in general. 22 | 23 | - The "p" variation uses a comparison with each previous step so to enhance the result. In general things become smoother / less messy. 24 | 25 | --- 26 | 27 | ### Potential compatibility issue: 28 | 29 | If any error was to relate to tensors shape, uncomment these two lines in the file "presets_to_add.py": 30 | 31 | extra_samplers["Distance_fast"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False) 32 | extra_samplers["Distance_fast_n"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_negative=True) 33 | 34 | These are basically the same except they don't use a spherical interpolation at the end. The interpolation was made with latent spaces such as those used in Stable Diffusion in mind. These two alternatives use a weighted average instead (the difference is barely noticeable from my testing). 35 | 36 | --- 37 | 38 | ## Comparisons 39 | 40 | Examples below are using the beta scheduler. **The amount of steps has been adjusted to match the duration** has this sampler is quite slow, yet requires little amounts of steps. 41 | 42 | left: Distance, 7 steps 43 | 44 | right: dpmpp2m, 20 steps 45 | 46 | ![combined_side_by_side](https://github.com/user-attachments/assets/65a66eba-d038-45fc-9648-79084cc1e011) 47 | 48 | 49 | 50 | Distance, 10 steps: 51 | 52 | ![distance_10_steps](https://github.com/user-attachments/assets/32d7cf21-4c6e-45e1-892f-adc08a0cfa49) 53 | 54 | Distance n, 10 steps: 55 | 56 | ![distance_n_10_steps](https://github.com/user-attachments/assets/8d41657a-7e21-4909-b03f-01afa532edf7) 57 | 58 | DPM++SDE (gpu), 14 steps: 59 | 60 | ![dpmppsder_14steps](https://github.com/user-attachments/assets/8a7eab3d-8948-4df6-b51a-8f456ecc6980) 61 | 62 | 63 | ## Disabled Guidance (CFG=1) 64 | 65 | 66 | CFG scale at 1 on a normal SDXL model (works for simple subjects): 67 | 68 | ![ComfyUI_00645_](https://github.com/user-attachments/assets/c9676d09-2c66-4d48-86b0-f0cc7c82569c) 69 | 70 | ![ComfyUI_00640_](https://github.com/user-attachments/assets/daf59ad3-4abf-4a0f-abdd-6e7cf423e6b7) 71 | 72 | ![ComfyUI_00632_](https://github.com/user-attachments/assets/515ad683-d841-4c95-b452-9263fdeb46f1) 73 | 74 | Distance p with a CFG at 1 and 6 steps: 75 | 76 | ![ComfyUI_00695_](https://github.com/user-attachments/assets/4ff194ac-a0ad-4e10-9cd4-c8d6aa4e3d57) 77 | 78 | ![ComfyUI_00692_](https://github.com/user-attachments/assets/a5bfc880-b7a3-45b3-867d-82ca7560bf34) 79 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | from .custom_samplers import SamplerDistanceAdvanced 2 | from .presets_to_add import extra_samplers 3 | 4 | def add_samplers(): 5 | from comfy.samplers import KSampler, k_diffusion_sampling 6 | added = 0 7 | samplers_names = [n for n in extra_samplers][::-1] 8 | for sampler in samplers_names: 9 | if sampler not in KSampler.SAMPLERS: 10 | try: 11 | idx = KSampler.SAMPLERS.index("uni_pc_bh2") # Last item in the samplers list 12 | KSampler.SAMPLERS.insert(idx+1, sampler) # Add our custom samplers 13 | setattr(k_diffusion_sampling, "sample_{}".format(sampler), extra_samplers[sampler]) 14 | added += 1 15 | except ValueError as _err: 16 | pass 17 | if added > 0: 18 | import importlib 19 | importlib.reload(k_diffusion_sampling) 20 | 21 | add_samplers() 22 | 23 | NODE_CLASS_MAPPINGS = { 24 | "SamplerDistance": SamplerDistanceAdvanced, 25 | } -------------------------------------------------------------------------------- /custom_samplers.py: -------------------------------------------------------------------------------- 1 | import torch 2 | from comfy.k_diffusion.sampling import trange, to_d 3 | import comfy.model_patcher 4 | import comfy.samplers 5 | from math import pi 6 | mmnorm = lambda x: (x - x.min()) / (x.max() - x.min()) 7 | selfnorm = lambda x: x / x.norm() 8 | EPSILON = 1e-4 9 | 10 | @torch.no_grad() 11 | def matrix_batch_slerp(t, tn, w): 12 | dots = torch.mul(tn.unsqueeze(0), tn.unsqueeze(1)).sum(dim=[-1,-2], keepdim=True).clamp(min=-1.0 + EPSILON, max=1.0 - EPSILON) 13 | mask = ~torch.eye(tn.shape[0], dtype=torch.bool, device=tn.device) 14 | A, B, C, D, E = dots.shape 15 | dots = dots[mask].reshape(A, B - 1, C, D, E) 16 | omegas = dots.acos() 17 | sin_omega = omegas.sin() 18 | res = t.unsqueeze(1).repeat(1, B - 1, 1, 1, 1) * torch.sin(w.div(B - 1).unsqueeze(1).repeat(1, B - 1, 1, 1, 1) * omegas) / sin_omega 19 | res = res.sum(dim=[0, 1]).unsqueeze(0) 20 | return res 21 | 22 | @torch.no_grad() 23 | def fast_distance_weights(t, use_softmax=False, use_slerp=False, uncond=None): 24 | norm = torch.linalg.matrix_norm(t, keepdim=True) 25 | n = t.shape[0] 26 | tn = t.div(norm) 27 | 28 | distances = (tn.unsqueeze(0) - tn.unsqueeze(1)).abs().sum(dim=0) 29 | distances = distances.max(dim=0, keepdim=True).values - distances 30 | 31 | if uncond != None: 32 | uncond = uncond.div(torch.linalg.matrix_norm(uncond, keepdim=True)) 33 | distances += tn.sub(uncond).abs() #.div(n) 34 | 35 | if use_softmax: 36 | distances = distances.mul(n).softmax(dim=0) 37 | else: 38 | distances = distances.div(distances.max(dim=0).values).pow(2) 39 | distances = distances / distances.sum(dim=0) 40 | 41 | if use_slerp: 42 | res = matrix_batch_slerp(t, tn, distances) 43 | else: 44 | res = (t * distances).sum(dim=0).unsqueeze(0) 45 | res = res.div(torch.linalg.matrix_norm(res, keepdim=True)).mul(norm.mul(distances).sum(dim=0).unsqueeze(0)) 46 | return res 47 | 48 | @torch.no_grad() 49 | def normalize_adjust(a,b,strength=1): 50 | c = a.clone() 51 | norm_a = a.norm(dim=1,keepdim=True) 52 | a = a / norm_a 53 | b = b / b.norm(dim=1,keepdim=True) 54 | d = mmnorm((a - b).abs()) 55 | a = a - b * d * strength 56 | a = a * norm_a / a.norm(dim=1,keepdim=True) 57 | if a.isnan().any(): 58 | a[~torch.isfinite(a)] = c[~torch.isfinite(a)] 59 | return a 60 | 61 | # Euler and CFGpp part taken from comfy_extras/nodes_advanced_samplers 62 | def distance_wrap(resample,resample_end=-1,cfgpp=False,sharpen=False,use_softmax=False,first_only=False,use_slerp=False,perp_step=False,smooth=False,use_negative=False): 63 | @torch.no_grad() 64 | def sample_distance_advanced(model, x, sigmas, extra_args=None, callback=None, disable=None): 65 | extra_args = {} if extra_args is None else extra_args 66 | uncond = None 67 | 68 | if cfgpp or use_negative: 69 | uncond = None 70 | def post_cfg_function(args): 71 | nonlocal uncond 72 | uncond = args["uncond_denoised"] 73 | return args["denoised"] 74 | model_options = extra_args.get("model_options", {}).copy() 75 | extra_args["model_options"] = comfy.model_patcher.set_model_options_post_cfg_function(model_options, post_cfg_function) 76 | 77 | s_min, s_max = sigmas[sigmas > 0].min(), sigmas.max() 78 | progression = lambda x, y=0.5: max(0,min(1,((x - s_min) / (s_max - s_min)) ** y)) 79 | d_prev = None 80 | 81 | if resample == -1: 82 | current_resample = min(10, sigmas.shape[0] // 2) 83 | else: 84 | current_resample = resample 85 | total = 0 86 | s_in = x.new_ones([x.shape[0]]) 87 | for i in trange(len(sigmas) - 1, disable=disable): 88 | sigma_hat = sigmas[i] 89 | 90 | res_mul = progression(sigma_hat) 91 | if resample_end >= 0: 92 | resample_steps = max(min(current_resample,resample_end),min(max(current_resample,resample_end),int(current_resample * res_mul + resample_end * (1 - res_mul)))) 93 | else: 94 | resample_steps = current_resample 95 | 96 | denoised = model(x, sigma_hat * s_in, **extra_args) 97 | total += 1 98 | 99 | if cfgpp and torch.any(uncond): 100 | d = to_d(x - denoised + uncond, sigmas[i], denoised) 101 | else: 102 | d = to_d(x, sigma_hat, denoised) 103 | 104 | if callback is not None: 105 | callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigma_hat, 'denoised': denoised}) 106 | dt = sigmas[i + 1] - sigma_hat 107 | 108 | if sigmas[i + 1] == 0 or resample_steps == 0 or (i > 0 and first_only): 109 | # Euler method 110 | x = x + d * dt 111 | else: 112 | # not Euler method 113 | x_n = [d] 114 | for re_step in range(resample_steps): 115 | x_new = x + d * dt 116 | new_denoised = model(x_new, sigmas[i + 1] * s_in, **extra_args) 117 | if smooth: 118 | new_denoised = new_denoised.abs().pow(1 / new_denoised.std().sqrt()) * new_denoised.sign() 119 | new_denoised = new_denoised.div(new_denoised.std().sqrt()) 120 | total += 1 121 | if cfgpp and torch.any(uncond): 122 | new_d = to_d(x_new - new_denoised + uncond, sigmas[i + 1], new_denoised) 123 | else: 124 | new_d = to_d(x_new, sigmas[i + 1] * s_in, new_denoised) 125 | x_n.append(new_d) 126 | if re_step == 0: 127 | d = (new_d + d) / 2 128 | else: 129 | u = uncond if (use_negative and uncond is not None and torch.any(uncond)) else None 130 | d = fast_distance_weights(torch.stack(x_n).squeeze(1), use_softmax=use_softmax, use_slerp=use_slerp, uncond=u) 131 | if sharpen or perp_step: 132 | if sharpen and d_prev is not None: 133 | d = normalize_adjust(d, d_prev, 1) 134 | elif perp_step and d_prev is not None: 135 | d = diff_step(d, d_prev, 0.5) 136 | d_prev = d.clone() 137 | x_n.append(d) 138 | x = x + d * dt 139 | return x 140 | return sample_distance_advanced 141 | 142 | def blend_add(t,v,s): 143 | tn = torch.linalg.norm(t) 144 | vn = torch.linalg.norm(v) 145 | vp = (v / vn - torch.dot(v / vn, t / tn) * t / tn) * tn 146 | return t + vp * s / 2 147 | 148 | @torch.no_grad() 149 | def diff_step(a, b, s): 150 | n = torch.linalg.matrix_norm(a, keepdim=True) 151 | x = a.div(n) 152 | y = b.div(torch.linalg.matrix_norm(b, keepdim=True)) 153 | y = n * y.sub(x.mul(torch.mul(x, y).sum().clamp(min=-1.0, max=1.0))) 154 | return a - y * s 155 | 156 | def perp_step_wrap(s=0.5): 157 | @torch.no_grad() 158 | def perp_step(model, x, sigmas, extra_args=None, callback=None, disable=None): 159 | """Implements Algorithm 2 (Euler steps) from Karras et al. (2022).""" 160 | extra_args = {} if extra_args is None else extra_args 161 | s_in = x.new_ones([x.shape[0]]) 162 | previous_step = None 163 | for i in trange(len(sigmas) - 1, disable=disable): 164 | sigma_hat = sigmas[i] 165 | denoised = model(x, sigma_hat * s_in, **extra_args) 166 | d = to_d(x, sigma_hat, denoised) 167 | dt = sigmas[i + 1] - sigma_hat 168 | if previous_step is not None and sigmas[i + 1] != 0: 169 | d = diff_step(d, previous_step, s) 170 | previous_step = d.clone() 171 | if callback is not None: 172 | callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigma_hat, 'denoised': denoised}) 173 | x = x + d * dt 174 | return x 175 | return perp_step 176 | 177 | # as a reference 178 | @torch.no_grad() 179 | def simplified_euler(model, x, sigmas, extra_args=None, callback=None, disable=None): 180 | extra_args = {} if extra_args is None else extra_args 181 | s_in = x.new_ones([x.shape[0]]) 182 | for i in trange(len(sigmas) - 1, disable=disable): 183 | sigma_hat = sigmas[i] 184 | denoised = model(x, sigma_hat * s_in, **extra_args) 185 | d = to_d(x, sigma_hat, denoised) 186 | if callback is not None: 187 | callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigma_hat, 'denoised': denoised}) 188 | dt = sigmas[i + 1] - sigma_hat 189 | # Euler method 190 | x = x + d * dt 191 | return x 192 | 193 | class SamplerDistanceAdvanced: 194 | @classmethod 195 | def INPUT_TYPES(s): 196 | return {"required": {"resample": ("INT", {"default": 3, "min": -1, "max": 32, "step": 1, 197 | "tooltip":"0 all along gives Euler. 1 gives Heun.\nAnything starting from 2 will use the distance method.\n-1 will do remaining steps + 1 as the resample value. This can be pretty slow."}), 198 | "resample_end": ("INT", {"default": -1, "min": -1, "max": 32, "step": 1, "tooltip":"How many resamples for the end. -1 means constant."}), 199 | "cfgpp" : ("BOOLEAN", {"default": True}), 200 | }} 201 | RETURN_TYPES = ("SAMPLER",) 202 | CATEGORY = "sampling/custom_sampling/samplers" 203 | FUNCTION = "get_sampler" 204 | 205 | def get_sampler(self,resample,resample_end,cfgpp): 206 | sampler = comfy.samplers.KSAMPLER( 207 | distance_wrap(resample=resample,cfgpp=cfgpp,resample_end=resample_end)) 208 | return (sampler, ) 209 | -------------------------------------------------------------------------------- /dress_example.json: -------------------------------------------------------------------------------- 1 | { 2 | "last_node_id": 22, 3 | "last_link_id": 40, 4 | "nodes": [ 5 | { 6 | "id": 10, 7 | "type": "SamplerCustomAdvanced", 8 | "pos": [ 9 | 2040, 10 | 760 11 | ], 12 | "size": [ 13 | 355.20001220703125, 14 | 106 15 | ], 16 | "flags": {}, 17 | "order": 15, 18 | "mode": 0, 19 | "inputs": [ 20 | { 21 | "name": "noise", 22 | "type": "NOISE", 23 | "link": 4, 24 | "slot_index": 0 25 | }, 26 | { 27 | "name": "guider", 28 | "type": "GUIDER", 29 | "link": 5, 30 | "slot_index": 1 31 | }, 32 | { 33 | "name": "sampler", 34 | "type": "SAMPLER", 35 | "link": 6, 36 | "slot_index": 2 37 | }, 38 | { 39 | "name": "sigmas", 40 | "type": "SIGMAS", 41 | "link": 40, 42 | "slot_index": 3 43 | }, 44 | { 45 | "name": "latent_image", 46 | "type": "LATENT", 47 | "link": 29, 48 | "slot_index": 4 49 | } 50 | ], 51 | "outputs": [ 52 | { 53 | "name": "output", 54 | "type": "LATENT", 55 | "links": [ 56 | 12 57 | ], 58 | "shape": 3, 59 | "slot_index": 0 60 | }, 61 | { 62 | "name": "denoised_output", 63 | "type": "LATENT", 64 | "links": null, 65 | "shape": 3 66 | } 67 | ], 68 | "properties": { 69 | "Node name for S&R": "SamplerCustomAdvanced" 70 | }, 71 | "widgets_values": [] 72 | }, 73 | { 74 | "id": 16, 75 | "type": "VAEDecode", 76 | "pos": [ 77 | 2540, 78 | 760 79 | ], 80 | "size": [ 81 | 210, 82 | 46 83 | ], 84 | "flags": {}, 85 | "order": 16, 86 | "mode": 0, 87 | "inputs": [ 88 | { 89 | "name": "samples", 90 | "type": "LATENT", 91 | "link": 12 92 | }, 93 | { 94 | "name": "vae", 95 | "type": "VAE", 96 | "link": 13 97 | } 98 | ], 99 | "outputs": [ 100 | { 101 | "name": "IMAGE", 102 | "type": "IMAGE", 103 | "links": [ 104 | 3 105 | ], 106 | "shape": 3, 107 | "slot_index": 0 108 | } 109 | ], 110 | "properties": { 111 | "Node name for S&R": "VAEDecode" 112 | }, 113 | "widgets_values": [] 114 | }, 115 | { 116 | "id": 17, 117 | "type": "Reroute", 118 | "pos": [ 119 | 520, 120 | 520 121 | ], 122 | "size": [ 123 | 82, 124 | 26 125 | ], 126 | "flags": {}, 127 | "order": 6, 128 | "mode": 0, 129 | "inputs": [ 130 | { 131 | "name": "", 132 | "type": "*", 133 | "link": 30 134 | } 135 | ], 136 | "outputs": [ 137 | { 138 | "name": "MODEL", 139 | "type": "MODEL", 140 | "links": [ 141 | 18 142 | ] 143 | } 144 | ], 145 | "properties": { 146 | "showOutputText": true, 147 | "horizontal": false 148 | } 149 | }, 150 | { 151 | "id": 18, 152 | "type": "Reroute", 153 | "pos": [ 154 | 520, 155 | 700 156 | ], 157 | "size": [ 158 | 75, 159 | 26 160 | ], 161 | "flags": {}, 162 | "order": 7, 163 | "mode": 0, 164 | "inputs": [ 165 | { 166 | "name": "", 167 | "type": "*", 168 | "link": 31 169 | } 170 | ], 171 | "outputs": [ 172 | { 173 | "name": "CLIP", 174 | "type": "CLIP", 175 | "links": [ 176 | 1, 177 | 2 178 | ] 179 | } 180 | ], 181 | "properties": { 182 | "showOutputText": true, 183 | "horizontal": false 184 | } 185 | }, 186 | { 187 | "id": 19, 188 | "type": "Reroute", 189 | "pos": [ 190 | 480, 191 | 920 192 | ], 193 | "size": [ 194 | 75, 195 | 26 196 | ], 197 | "flags": {}, 198 | "order": 8, 199 | "mode": 0, 200 | "inputs": [ 201 | { 202 | "name": "", 203 | "type": "*", 204 | "link": 32 205 | } 206 | ], 207 | "outputs": [ 208 | { 209 | "name": "VAE", 210 | "type": "VAE", 211 | "links": [ 212 | 19 213 | ] 214 | } 215 | ], 216 | "properties": { 217 | "showOutputText": true, 218 | "horizontal": false 219 | } 220 | }, 221 | { 222 | "id": 21, 223 | "type": "Reroute", 224 | "pos": [ 225 | 1400, 226 | 520 227 | ], 228 | "size": [ 229 | 82, 230 | 26 231 | ], 232 | "flags": {}, 233 | "order": 9, 234 | "mode": 0, 235 | "inputs": [ 236 | { 237 | "name": "", 238 | "type": "*", 239 | "link": 18 240 | } 241 | ], 242 | "outputs": [ 243 | { 244 | "name": "MODEL", 245 | "type": "MODEL", 246 | "links": [ 247 | 9, 248 | 17 249 | ] 250 | } 251 | ], 252 | "properties": { 253 | "showOutputText": true, 254 | "horizontal": false 255 | } 256 | }, 257 | { 258 | "id": 22, 259 | "type": "Reroute", 260 | "pos": [ 261 | 2380, 262 | 920 263 | ], 264 | "size": [ 265 | 75, 266 | 26 267 | ], 268 | "flags": {}, 269 | "order": 12, 270 | "mode": 0, 271 | "inputs": [ 272 | { 273 | "name": "", 274 | "type": "*", 275 | "link": 19 276 | } 277 | ], 278 | "outputs": [ 279 | { 280 | "name": "VAE", 281 | "type": "VAE", 282 | "links": [ 283 | 13 284 | ] 285 | } 286 | ], 287 | "properties": { 288 | "showOutputText": true, 289 | "horizontal": false 290 | } 291 | }, 292 | { 293 | "id": 6, 294 | "type": "EmptyLatentImage", 295 | "pos": [ 296 | 1540, 297 | 1240 298 | ], 299 | "size": [ 300 | 315, 301 | 106 302 | ], 303 | "flags": {}, 304 | "order": 0, 305 | "mode": 0, 306 | "inputs": [], 307 | "outputs": [ 308 | { 309 | "name": "LATENT", 310 | "type": "LATENT", 311 | "links": [], 312 | "slot_index": 0 313 | } 314 | ], 315 | "properties": { 316 | "Node name for S&R": "EmptyLatentImage" 317 | }, 318 | "widgets_values": [ 319 | 512, 320 | 512, 321 | 1 322 | ] 323 | }, 324 | { 325 | "id": 9, 326 | "type": "SaveImage", 327 | "pos": [ 328 | 2780, 329 | 760 330 | ], 331 | "size": [ 332 | 210, 333 | 270 334 | ], 335 | "flags": { 336 | "collapsed": true 337 | }, 338 | "order": 17, 339 | "mode": 0, 340 | "inputs": [ 341 | { 342 | "name": "images", 343 | "type": "IMAGE", 344 | "link": 3 345 | } 346 | ], 347 | "outputs": [], 348 | "properties": {}, 349 | "widgets_values": [ 350 | "ComfyUI" 351 | ] 352 | }, 353 | { 354 | "id": 15, 355 | "type": "EmptyLatentImage", 356 | "pos": [ 357 | 1540, 358 | 1100 359 | ], 360 | "size": [ 361 | 315, 362 | 106 363 | ], 364 | "flags": {}, 365 | "order": 1, 366 | "mode": 0, 367 | "inputs": [], 368 | "outputs": [ 369 | { 370 | "name": "LATENT", 371 | "type": "LATENT", 372 | "links": [ 373 | 29 374 | ], 375 | "shape": 3, 376 | "slot_index": 0 377 | } 378 | ], 379 | "properties": { 380 | "Node name for S&R": "EmptyLatentImage" 381 | }, 382 | "widgets_values": [ 383 | 1024, 384 | 1024, 385 | 1 386 | ] 387 | }, 388 | { 389 | "id": 5, 390 | "type": "CheckpointLoaderSimple", 391 | "pos": [ 392 | 30.752971649169922, 393 | 691.9783935546875 394 | ], 395 | "size": [ 396 | 315, 397 | 98 398 | ], 399 | "flags": {}, 400 | "order": 2, 401 | "mode": 0, 402 | "inputs": [], 403 | "outputs": [ 404 | { 405 | "name": "MODEL", 406 | "type": "MODEL", 407 | "links": [ 408 | 30 409 | ], 410 | "slot_index": 0 411 | }, 412 | { 413 | "name": "CLIP", 414 | "type": "CLIP", 415 | "links": [ 416 | 31 417 | ], 418 | "slot_index": 1 419 | }, 420 | { 421 | "name": "VAE", 422 | "type": "VAE", 423 | "links": [ 424 | 32 425 | ], 426 | "slot_index": 2 427 | } 428 | ], 429 | "properties": { 430 | "Node name for S&R": "CheckpointLoaderSimple" 431 | }, 432 | "widgets_values": [ 433 | "SDXL\\Iris_Lux_v1257_msa.safetensors" 434 | ] 435 | }, 436 | { 437 | "id": 13, 438 | "type": "CFGGuider", 439 | "pos": [ 440 | 1540, 441 | 560 442 | ], 443 | "size": [ 444 | 315, 445 | 98 446 | ], 447 | "flags": {}, 448 | "order": 14, 449 | "mode": 0, 450 | "inputs": [ 451 | { 452 | "name": "model", 453 | "type": "MODEL", 454 | "link": 9 455 | }, 456 | { 457 | "name": "positive", 458 | "type": "CONDITIONING", 459 | "link": 10 460 | }, 461 | { 462 | "name": "negative", 463 | "type": "CONDITIONING", 464 | "link": 11 465 | } 466 | ], 467 | "outputs": [ 468 | { 469 | "name": "GUIDER", 470 | "type": "GUIDER", 471 | "links": [ 472 | 5 473 | ], 474 | "shape": 3 475 | } 476 | ], 477 | "properties": { 478 | "Node name for S&R": "CFGGuider" 479 | }, 480 | "widgets_values": [ 481 | 8 482 | ] 483 | }, 484 | { 485 | "id": 8, 486 | "type": "CLIPTextEncode", 487 | "pos": [ 488 | 760, 489 | 860 490 | ], 491 | "size": [ 492 | 425.27801513671875, 493 | 180.6060791015625 494 | ], 495 | "flags": {}, 496 | "order": 11, 497 | "mode": 0, 498 | "inputs": [ 499 | { 500 | "name": "clip", 501 | "type": "CLIP", 502 | "link": 2 503 | } 504 | ], 505 | "outputs": [ 506 | { 507 | "name": "CONDITIONING", 508 | "type": "CONDITIONING", 509 | "links": [ 510 | 11 511 | ], 512 | "slot_index": 0 513 | } 514 | ], 515 | "properties": { 516 | "Node name for S&R": "CLIPTextEncode" 517 | }, 518 | "widgets_values": [ 519 | "kindly change the style of this photo into something horrible, unrealistic and disgusting, uncanny cgi like wednesday adams bathing in oranges with fat demons all around her in a videogame about molten wax people made of pure smooth bright light in black and white.\n\ntags: cgi, text, watermark, simple, smooth, videogame, hyper saturated," 520 | ] 521 | }, 522 | { 523 | "id": 12, 524 | "type": "AlignYourStepsScheduler", 525 | "pos": [ 526 | 1540, 527 | 960 528 | ], 529 | "size": [ 530 | 315, 531 | 106 532 | ], 533 | "flags": {}, 534 | "order": 3, 535 | "mode": 0, 536 | "inputs": [], 537 | "outputs": [ 538 | { 539 | "name": "SIGMAS", 540 | "type": "SIGMAS", 541 | "links": [], 542 | "shape": 3, 543 | "slot_index": 0 544 | } 545 | ], 546 | "properties": { 547 | "Node name for S&R": "AlignYourStepsScheduler" 548 | }, 549 | "widgets_values": [ 550 | "SDXL", 551 | 10, 552 | 1 553 | ] 554 | }, 555 | { 556 | "id": 20, 557 | "type": "BasicScheduler", 558 | "pos": [ 559 | 1542, 560 | 802 561 | ], 562 | "size": [ 563 | 315, 564 | 106 565 | ], 566 | "flags": {}, 567 | "order": 13, 568 | "mode": 0, 569 | "inputs": [ 570 | { 571 | "name": "model", 572 | "type": "MODEL", 573 | "link": 17 574 | } 575 | ], 576 | "outputs": [ 577 | { 578 | "name": "SIGMAS", 579 | "type": "SIGMAS", 580 | "links": [ 581 | 40 582 | ], 583 | "shape": 3, 584 | "slot_index": 0 585 | } 586 | ], 587 | "properties": { 588 | "Node name for S&R": "BasicScheduler" 589 | }, 590 | "widgets_values": [ 591 | "beta", 592 | 10, 593 | 1 594 | ] 595 | }, 596 | { 597 | "id": 7, 598 | "type": "CLIPTextEncode", 599 | "pos": [ 600 | 760, 601 | 660 602 | ], 603 | "size": [ 604 | 422.84503173828125, 605 | 164.31304931640625 606 | ], 607 | "flags": {}, 608 | "order": 10, 609 | "mode": 0, 610 | "inputs": [ 611 | { 612 | "name": "clip", 613 | "type": "CLIP", 614 | "link": 1 615 | } 616 | ], 617 | "outputs": [ 618 | { 619 | "name": "CONDITIONING", 620 | "type": "CONDITIONING", 621 | "links": [ 622 | 10 623 | ], 624 | "slot_index": 0 625 | } 626 | ], 627 | "properties": { 628 | "Node name for S&R": "CLIPTextEncode" 629 | }, 630 | "widgets_values": [ 631 | "close detailed upper body high quality photo of a princess, nightly analog photo, low contrast, low exposure noise, crown, jewels and lace princess dress. macro skin details, Soft colors." 632 | ] 633 | }, 634 | { 635 | "id": 14, 636 | "type": "RandomNoise", 637 | "pos": [ 638 | 1540, 639 | 440 640 | ], 641 | "size": [ 642 | 315, 643 | 82 644 | ], 645 | "flags": {}, 646 | "order": 4, 647 | "mode": 0, 648 | "inputs": [], 649 | "outputs": [ 650 | { 651 | "name": "NOISE", 652 | "type": "NOISE", 653 | "links": [ 654 | 4 655 | ], 656 | "shape": 3 657 | } 658 | ], 659 | "properties": { 660 | "Node name for S&R": "RandomNoise" 661 | }, 662 | "widgets_values": [ 663 | 750937431712078, 664 | "fixed" 665 | ] 666 | }, 667 | { 668 | "id": 11, 669 | "type": "KSamplerSelect", 670 | "pos": [ 671 | 1540, 672 | 700 673 | ], 674 | "size": [ 675 | 315, 676 | 58 677 | ], 678 | "flags": {}, 679 | "order": 5, 680 | "mode": 0, 681 | "inputs": [], 682 | "outputs": [ 683 | { 684 | "name": "SAMPLER", 685 | "type": "SAMPLER", 686 | "links": [ 687 | 6 688 | ], 689 | "shape": 3, 690 | "slot_index": 0 691 | } 692 | ], 693 | "properties": { 694 | "Node name for S&R": "KSamplerSelect" 695 | }, 696 | "widgets_values": [ 697 | "Distance" 698 | ] 699 | } 700 | ], 701 | "links": [ 702 | [ 703 | 1, 704 | 18, 705 | 0, 706 | 7, 707 | 0, 708 | "CLIP" 709 | ], 710 | [ 711 | 2, 712 | 18, 713 | 0, 714 | 8, 715 | 0, 716 | "CLIP" 717 | ], 718 | [ 719 | 3, 720 | 16, 721 | 0, 722 | 9, 723 | 0, 724 | "IMAGE" 725 | ], 726 | [ 727 | 4, 728 | 14, 729 | 0, 730 | 10, 731 | 0, 732 | "NOISE" 733 | ], 734 | [ 735 | 5, 736 | 13, 737 | 0, 738 | 10, 739 | 1, 740 | "GUIDER" 741 | ], 742 | [ 743 | 6, 744 | 11, 745 | 0, 746 | 10, 747 | 2, 748 | "SAMPLER" 749 | ], 750 | [ 751 | 9, 752 | 21, 753 | 0, 754 | 13, 755 | 0, 756 | "MODEL" 757 | ], 758 | [ 759 | 10, 760 | 7, 761 | 0, 762 | 13, 763 | 1, 764 | "CONDITIONING" 765 | ], 766 | [ 767 | 11, 768 | 8, 769 | 0, 770 | 13, 771 | 2, 772 | "CONDITIONING" 773 | ], 774 | [ 775 | 12, 776 | 10, 777 | 0, 778 | 16, 779 | 0, 780 | "LATENT" 781 | ], 782 | [ 783 | 13, 784 | 22, 785 | 0, 786 | 16, 787 | 1, 788 | "VAE" 789 | ], 790 | [ 791 | 17, 792 | 21, 793 | 0, 794 | 20, 795 | 0, 796 | "MODEL" 797 | ], 798 | [ 799 | 18, 800 | 17, 801 | 0, 802 | 21, 803 | 0, 804 | "*" 805 | ], 806 | [ 807 | 19, 808 | 19, 809 | 0, 810 | 22, 811 | 0, 812 | "*" 813 | ], 814 | [ 815 | 29, 816 | 15, 817 | 0, 818 | 10, 819 | 4, 820 | "LATENT" 821 | ], 822 | [ 823 | 30, 824 | 5, 825 | 0, 826 | 17, 827 | 0, 828 | "*" 829 | ], 830 | [ 831 | 31, 832 | 5, 833 | 1, 834 | 18, 835 | 0, 836 | "*" 837 | ], 838 | [ 839 | 32, 840 | 5, 841 | 2, 842 | 19, 843 | 0, 844 | "*" 845 | ], 846 | [ 847 | 40, 848 | 20, 849 | 0, 850 | 10, 851 | 3, 852 | "SIGMAS" 853 | ] 854 | ], 855 | "groups": [], 856 | "config": {}, 857 | "extra": {}, 858 | "version": 0.4 859 | } -------------------------------------------------------------------------------- /presets_to_add.py: -------------------------------------------------------------------------------- 1 | from .custom_samplers import distance_wrap, perp_step_wrap 2 | 3 | extra_samplers = {} 4 | 5 | """ 6 | 7 | To add a sampler to the list of samplers you can do it this way (outside of this comment): 8 | 9 | extra_samplers["the_name_that_you_want"] = distance_wrap(resample=3,resample_end=-1,cfgpp=False) 10 | 11 | "resample" is the starting value, how many more inferences it will use. More is slower. 12 | For resample_end "-1" means constant. 13 | Constant resample at 0 gives Euler, 1 gives Heun. 14 | Other variables: 15 | - use_negative: will use the negative prediction to prepare the distance scores. This tends to give images with less errors from my testing. 16 | - cfgpp: will determin if you want it or not. True or False. 17 | - use_slerp: will slerp the predictions instead of doing a weighted average. The difference is more obvious when using use_negative. 18 | - use_softmax: rather than using a min/max normalization and an exponent will use a softmax instead. 19 | - sharpen: not recommanded, attempts to sharpen the results but instead tends to make things fuzzy. 20 | - smooth: not recommanded, will make everything brighter. Not smoother. 21 | - perp step: experimental, not yet recommanded. 22 | PerpStep is a test sampler, uncomment if you want to try. 23 | """ 24 | # Distance_fast_slerp 25 | extra_samplers["Distance"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True) 26 | # Distance_fast_slerp_n 27 | extra_samplers["Distance_n"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True,use_negative=True) 28 | # Distance_fast_slerp_p 29 | extra_samplers["Distance_p"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True,perp_step=True) 30 | 31 | # extra_samplers["PerpStep"] = perp_step_wrap(s=0.5) 32 | # extra_samplers["euler_test"] = distance_wrap(resample=0,resample_end=-1) 33 | # extra_samplers["heun_test"] = distance_wrap(resample=1,resample_end=1) 34 | # extra_samplers["Distance_fast"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False) 35 | # extra_samplers["Distance_fast_n"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_negative=True) 36 | # extra_samplers["Distance_fast_slerp"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True) 37 | # extra_samplers["Distance_fast_slerp_n"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True,use_negative=True) 38 | # extra_samplers["Distance_fast_slerp_p"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True,perp_step=True) 39 | # extra_samplers["Distance_fast_slerp_np"] = distance_wrap(resample=3,resample_end=1,cfgpp=False,sharpen=False,use_slerp=True,use_negative=True,perp_step=True) 40 | 41 | # # constant 42 | # extra_samplers["distance_c2"] = distance_wrap(resample=2,resample_end=-1,cfgpp=False,sharpen=False) 43 | # extra_samplers["distance_c3"] = distance_wrap(resample=3,resample_end=-1,cfgpp=False,sharpen=False) 44 | # extra_samplers["distance_c4"] = distance_wrap(resample=4,resample_end=-1,cfgpp=False,sharpen=False) 45 | # extra_samplers["Distance_cfgpp"] = distance_wrap(resample=4,resample_end=-1,sharpen=False,cfgpp=True) 46 | 47 | # cfgpp 48 | # extra_samplers["distance_step_cfg_pp"] = distance_wrap(resample=-1,resample_end=1,cfgpp=True) 49 | extra_samplers["euler_cfg_pp_alt"] = distance_wrap(resample=0,cfgpp=True) 50 | extra_samplers["heun_cfg_pp"] = distance_wrap(resample=1,cfgpp=True) 51 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [project] 2 | name = "distancesampler" 3 | description = "Heuristic modification of the Heun sampler using a custom function based on normalized distances. For ComfyUI." 4 | version = "2.0.0" 5 | license = {file = "LICENSE"} 6 | 7 | [project.urls] 8 | Repository = "https://github.com/Extraltodeus/DistanceSampler" 9 | # Used by Comfy Registry https://comfyregistry.org 10 | 11 | [tool.comfy] 12 | PublisherId = "extraltodeus" 13 | DisplayName = "distancesampler" 14 | Icon = "" 15 | -------------------------------------------------------------------------------- /sampler_explaination.md: -------------------------------------------------------------------------------- 1 | # **Explanation of a Custom Diffusion Model Sampling Method in ComfyUI** 2 | 3 | (made using Gemini (and read by myself so you don't just hurt yourself reading potential garbage) and re-touched a little so it's readable) 4 | 5 | I recommand to have the code on the side to follow. 6 | 7 | But in short: 8 | 9 | Similarily to Heun's method or some other samplers, this sampler uses the last prediction to re-compute a new one. 10 | 11 | Here it loops and in each iteration uses the distances in between each values to create a new result. 12 | 13 | The new result is made from a weighted average (or a slerp using the same weights) where the weights are related to the proportions of proximity. 14 | 15 | --- 16 | 17 | The fast\_distance\_weights function calculates weights for a batch of tensors based on their pairwise distances in a normalized space. 18 | 19 | It takes the input tensor t, boolean flags use\_softmax and use\_slerp, and an optional unconditional tensor uncond as arguments. 20 | 21 | The function begins by calculating the matrix norm of the input tensor t: 22 | 23 | norm = torch.linalg.matrix_norm(t, keepdim=True). 24 | 25 | The batch size is then obtained: 26 | 27 | n = t.shape[0] 28 | 29 | The input tensor is normalized by dividing it by its norm: 30 | 31 | tn = t.div(norm) 32 | 33 | This step projects the tensors onto a unit hypersphere. 34 | 35 | (Note: that's a super fancy way to say I divide by the norm/radius, juste to ensure a similar scale, also usable by the slerp after) 36 | 37 | Next, the function calculates a distance metric between all pairs of normalized tensors: 38 | 39 | distances = (tn.unsqueeze(0) - tn.unsqueeze(1)).abs().sum(dim=0) 40 | 41 | This involves creating all pairwise combinations, calculating the element-wise absolute difference, and then summing along the first dimension. 42 | 43 | This results in a measure of dissimilarity between each pair of normalized tensors. This distance is then transformed: 44 | 45 | distances = distances.max(dim=0, keepdim=True).values - distances 46 | 47 | This operation inverts the distances, so that smaller original distances result in larger values after the subtraction. 48 | 49 | If an unconditional tensor uncond is provided (which is relevant for Classifier-Free Guidance), it is also normalized: 50 | 51 | uncond = uncond.div(torch.linalg.matrix_norm(uncond, keepdim=True)) 52 | 53 | Then, the distance of each normalized tensor tn from the normalized unconditional tensor is calculated and added to the existing distances: 54 | 55 | distances += tn.sub(uncond).abs().div(n) 56 | 57 | This step incorporates information about how far each conditional sample is from the unconditional sample into the weighting scheme. 58 | 59 | The function then proceeds to normalize the distances to obtain weights. If use\_softmax is True, the softmax function is applied: 60 | 61 | distances = distances.mul(n).softmax(dim=0) 62 | 63 | Softmax converts the distances into a probability distribution. If use\_softmax is False, the distances are first normalized by dividing by their maximum value and then squared: 64 | 65 | distances = distances.div(distances.max(dim=0).values).pow(2). 66 | 67 | These squared values are then further normalized by dividing by their sum: 68 | 69 | distances = distances / distances.sum(dim=0). 70 | 71 | This provides an alternative way to obtain weights that sum to 1\. 72 | 73 | Finally, the function combines the original tensors t using the calculated weights. If use\_slerp is True, the matrix\_batch\_slerp function is called: 74 | 75 | res = matrix_batch_slerp(t, tn, distances) 76 | 77 | Otherwise, a simple weighted sum is performed, followed by a normalization and scaling step: 78 | 79 | res = (t * distances).sum(dim=0).unsqueeze(0); res = res.div(torch.linalg.matrix_norm(res, keepdim=True)).mul(norm.mul(distances).sum(dim=0).unsqueeze(0)). 80 | 81 | This function provides a flexible way to compute weights based on the relationships between tensors and their proximity to an optional unconditional sample, and it offers a choice between spherical interpolation and a weighted sum for combining the tensors based on these weights. The inclusion of the uncond parameter strongly suggests a connection to Classifier-Free Guidance principles. 82 | 83 | --- 84 | 85 | The function _matrix\_batch\_slerp_ implements batched spherical linear interpolation (SLERP). SLERP is a technique used to interpolate between two points on a unit sphere along the great circle that connects them, maintaining a constant angular velocity. This is particularly useful for interpolating rotations or, more generally, directions in high-dimensional spaces. 86 | 87 | The function is decorated with @torch.no\_grad(), indicating that the operations within it should not be tracked for gradient computation, as it is likely used during inference or sampling. The function takes three arguments: t, which likely represents the original batch of tensors; tn, which is likely the normalized version of t; and w, which probably contains the weights or interpolation factors for each tensor in the batch. 88 | 89 | The first step inside matrix\_batch\_slerp is the calculation of dot products between all pairs of normalized tensors in tn: 90 | 91 | dots = torch.mul(tn.unsqueeze(0), tn.unsqueeze(1)).sum(dim=[-1,-2], keepdim=True).clamp(min=-1.0 + EPSILON, max=1.0 - EPSILON) 92 | 93 | tn.unsqueeze(0) and tn.unsqueeze(1) add dimensions to create all pairwise combinations for element-wise multiplication. 94 | 95 | The .sum(dim=\[-1,-2\], keepdim=True) operation then calculates the dot product between these pairs along their feature dimensions. 96 | 97 | The .clamp() operation ensures that the resulting dot product values stay within the range of -1 to 1, which is necessary for the subsequent acos() function. 98 | 99 | The dot product of two unit vectors is the cosine of the angle between them, so this step effectively calculates the cosine of the angles between all pairs of normalized tensors. 100 | 101 | Next, a mask is created to exclude the diagonal elements of the dot product matrix: 102 | 103 | mask = ~torch.eye(tn.shape, dtype=torch.bool, device=tn.device) 104 | 105 | torch.eye() creates an identity matrix, and the bitwise NOT operator \~ inverts it, resulting in a mask where the diagonal elements are False and all others are True. This mask is then used to select only the off-diagonal elements from the dots tensor: 106 | 107 | dots = dots[mask].reshape(A, B - 1, C, D, E) 108 | 109 | This step focuses the interpolation on the relationships between *different* tensors in the batch, not a tensor with itself. 110 | 111 | The dimensions of the dots tensor are then unpacked and reshaped to prepare for the SLERP calculation. 112 | 113 | The angles between the normalized tensors are calculated using the arccosine function: omegas \= dots.acos(). The sine of these angles is then computed: 114 | 115 | sin_omega = omegas.sin() 116 | 117 | The core of the SLERP implementation follows: 118 | 119 | res = t.unsqueeze(1).repeat(1, B - 1, 1, 1, 1) * torch.sin(w.div(B - 1).unsqueeze(1).repeat(1, B - 1, 1, 1, 1) * omegas) / sin_omega 120 | 121 | This formula calculates a weighted sum based on the angles and the provided weights w. 122 | 123 | The original tensor t is prepared for batch interpolation by adding and repeating dimensions. 124 | 125 | The weights w are also manipulated to match the dimensions and are divided by B \- 1, suggesting a normalization of the weights across the other tensors in the batch. 126 | 127 | The final result is obtained by summing the interpolated values across the first two dimensions and then unsqueezing to restore the expected shape: 128 | 129 | res = res.sum(dim=).unsqueeze(0) 130 | 131 | This function essentially performs a smooth, spherical interpolation between multiple matrices in a batch, using the provided weights to determine the contribution of each interpolated direction. 132 | 133 | The exclusion of self-pairs in the dot product calculation indicates that the function is designed to aggregate or combine information from distinct entities within the batch. 134 | 135 | --- 136 | 137 | The distance\_wrap function acts as a decorator that takes several parameters to configure a custom sampling process. 138 | 139 | These parameters include resample, resample\_end, cfgpp, sharpen, use\_softmax, first\_only, use\_slerp, perp\_step, smooth, and use\_negative. 140 | 141 | These flags and integer values control various aspects of the sampling algorithm implemented in the inner function sample\_distance\_advanced. 142 | 143 | The decorator returns this inner function, effectively creating a customized sampling function based on the provided configuration. 144 | 145 | The inner function sample\_distance\_advanced implements the core custom sampling logic. 146 | 147 | It takes the diffusion model (model), the initial noisy latent tensor (x), a tensor of noise levels (sigmas), optional extra arguments (extra\_args), a callback function (callback), and a disable flag for the progress bar (disable). It begins by initializing extra\_args if it's None and sets the unconditional output variable uncond to None. 148 | 149 | If either the cfgpp or use\_negative flag is True, the function defines an inner function post\_cfg\_function. This function is designed to capture the unconditional denoised output from the model after it has been called. It takes a dictionary of arguments and stores the unconditional denoised output in the uncond variable before returning the conditional denoised output. This mechanism is then integrated into the model's options within the ComfyUI framework using comfy.model\_patcher.set\_model\_options\_post\_cfg\_function. This setup allows the sampler to access the unconditional output needed for Classifier-Free Guidance or related techniques. 150 | 151 | The function then finds the minimum and maximum positive sigma values and defines a progression function. This function likely controls how the number of resampling steps changes during the sampling process based on the current noise level. A variable d\_prev is initialized to None, which will likely store the denoised direction from the previous step for use in sharpening or perpendicular step operations. The number of resampling steps for each main sampling step is determined based on the resample and resample\_end parameters. A counter total is initialized to track the number of model evaluations, and a tensor of ones s\_in is created for scaling sigma values. 152 | 153 | The main sampling loop iterates through the sequence of sigma values. In each iteration, the current sigma value is obtained, and the number of resampling steps is determined. The diffusion model is called to denoise the current latent tensor. Based on the cfgpp flag and the availability of the unconditional output, the denoised direction d is calculated using either a modified CFG approach or the standard method via the to\_d function. If a callback function is provided, it is called with the current state. The step size to the next sigma value is calculated. 154 | 155 | Depending on whether the next sigma is zero, no resampling steps are to be performed, or if it's after the first step and first\_only is True, an Euler step is applied to update the latent tensor x. Otherwise, a resampling process is initiated. A list x\_n is created to store the denoised directions from the resampling steps. An inner loop runs for the specified number of resampling steps. In each inner step, an initial Euler step is taken, the model is called to denoise the new latent state, and an optional smoothing operation is applied if the smooth flag is True. The new denoised direction new\_d is calculated, and it is appended to the x\_n list. For the first resampling step, the denoised direction d is updated by averaging it with new\_d. For subsequent resampling steps, the fast\_distance\_weights function is called to combine the denoised directions in x\_n using the specified use\_softmax and use\_slerp flags and the optional unconditional output. If the sharpen or perp\_step flags are True and a previous denoised direction d\_prev exists, the corresponding (undefined in the snippet) functions normalize\_adjust or diff\_step are called. The current denoised direction d is then stored in d\_prev, and it is also appended to x\_n. After the resampling loop, the latent tensor x is updated using the combined denoised direction d and the step size dt. Finally, the function returns the fully denoised latent tensor. 156 | 157 | | Parameter | Type | Description | Potential Impact on Sampling | 158 | | :---- | :---- | :---- | :---- | 159 | | resample | Integer | Number of resampling steps to perform at each main sampling step. \-1 for dynamic adjustment. | Controls the level of refinement at each step; higher values increase computation time but potentially quality. | 160 | | resample\_end | Integer | End value for dynamic resampling step calculation. | Influences how the number of resampling steps changes over the course of the sampling process. | 161 | | cfgpp | Boolean | Enables Classifier-Free Guidance++. | Improves adherence to the conditioning prompt, potentially enhancing output quality and coherence. | 162 | | sharpen | Boolean | Enables a sharpening operation on the denoised direction. | Likely enhances the details and sharpness of the generated image. | 163 | | use\_softmax | Boolean | Uses the softmax function for calculating distance weights. | Affects the distribution of weights assigned to different tensors based on their distances. | 164 | | first\_only | Boolean | If true, only performs resampling at the first sampling step. | Could be used for specific optimization strategies or early-stage refinement. | 165 | | use\_slerp | Boolean | Uses Spherical Linear Interpolation to combine weighted tensors in fast\_distance\_weights. | Provides a smooth interpolation between tensors, potentially leading to more coherent results. | 166 | | perp\_step | Boolean | Enables a perpendicular step operation on the denoised direction. | Likely influences the direction of the denoising step, potentially avoiding artifacts or improving consistency. | 167 | | smooth | Boolean | Enables a smoothing operation on the denoised output during resampling. | Likely reduces noise or artifacts in the intermediate denoised results. | 168 | | use\_negative | Boolean | Incorporates the unconditional output into the distance weight calculation, potentially related to negative prompting strategies. | Influences the weighting based on the distance from the unconditional sample. | 169 | 170 | In conclusion, the provided Python code defines a custom sampling method for diffusion models within the ComfyUI framework that incorporates several advanced techniques. The method utilizes spherical linear interpolation (matrix\_batch\_slerp) and a distance-based weighting scheme (fast\_distance\_weights) to refine the denoising process. The distance\_wrap decorator allows for extensive configuration of the sampling behavior through various parameters, and the inner sample\_distance\_advanced function orchestrates the iterative denoising, including dynamic resampling steps and the integration of Classifier-Free Guidance. This custom sampler likely aims to improve the quality and control over the generated samples by considering the geometric relationships between latent tensors and by iteratively refining the denoising direction using information from multiple model evaluations within each main sampling step. Further research and experimentation with the various parameters and the potentially integrated sharpening, perpendicular step, and smoothing operations would be necessary to fully understand the capabilities and benefits of this sophisticated sampling method. 171 | 172 | #### **Sources des citations** 173 | 174 | 1. KSampler | ComfyUI Wiki, consulté le avril 27, 2025, [https://comfyui-wiki.com/en/comfyui-nodes/sampling/k-sampler](https://comfyui-wiki.com/en/comfyui-nodes/sampling/k-sampler) 175 | 2. SamplerCustom \- ComfyUI Wiki, consulté le avril 27, 2025, [https://comfyui-wiki.com/en/comfyui-nodes/sampling/custom-sampling/sampler-custom](https://comfyui-wiki.com/en/comfyui-nodes/sampling/custom-sampling/sampler-custom) 176 | 3. k-diffusion \- PyPI, consulté le avril 27, 2025, [https://pypi.org/project/k-diffusion/](https://pypi.org/project/k-diffusion/) 177 | --------------------------------------------------------------------------------