├── docs ├── default_panel ├── example1_1.png ├── synthmap.png ├── comparison.jpeg ├── panel_opened.png ├── default_panel.jpg ├── scale_textures.png └── Suzanne_synthesis.png ├── licenses.md ├── README.md ├── uv_synthesize.py ├── uv_prepare.py ├── __init__.py ├── LICENSE └── tex_synthesize.py /docs/default_panel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/default_panel -------------------------------------------------------------------------------- /docs/example1_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/example1_1.png -------------------------------------------------------------------------------- /docs/synthmap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/synthmap.png -------------------------------------------------------------------------------- /docs/comparison.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/comparison.jpeg -------------------------------------------------------------------------------- /docs/panel_opened.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/panel_opened.png -------------------------------------------------------------------------------- /docs/default_panel.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/default_panel.jpg -------------------------------------------------------------------------------- /docs/scale_textures.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/scale_textures.png -------------------------------------------------------------------------------- /docs/Suzanne_synthesis.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yeus/syntexmex/HEAD/docs/Suzanne_synthesis.png -------------------------------------------------------------------------------- /licenses.md: -------------------------------------------------------------------------------- 1 | software used: 2 | 3 | # commercial license 4 | 5 | syntexmex, this version: GPL 6 | 7 | # other licenses: 8 | 9 | python: PSF license 10 | scikit-learn: BSD-new License 11 | skimage: BSD License 12 | shapely: BSD License 13 | scipy: BSD-new License 14 | pynndescent: BSD2-License 15 | Pillow: Python Imaging Library license (MIT-like) 16 | numba: BSD 2-Clause "Simplified" License 17 | networkx: BSD License 18 | decorator: BSD 2-Clause "Simplified" License 19 | llvmlite: BSD 2-Clause "Simplified" License 20 | numpy: BSD 21 | 22 | blender: GPL 23 | 24 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Syntexmex V0.2 3 | subtitle: UV Texture Synthesis based on syntexmex for blender 4 | date: 2020-03-20 5 | tags: ["plugins", "blender", "texture", "PBR", "synthesis"] 6 | bigimg: [{src: "/doc/unwrappedhead.jpg", desc: "unwrapped head"}] 7 | --- 8 | 9 | Syntexmex is an addon for blender which helps generating 10 | UV-textures (PBR textures work as well) based on examples. 11 | 12 | taking an image such as this one: 13 | 14 |

15 | 16 | we can synthesize new texture from the example image: 17 | 18 |

19 | 20 | or make a texture seamless, also using an example image (example 21 | image can be the texture itself): 22 | 23 |

24 | 25 | 27 | 28 | 29 | 30 | 37 | 38 | ## Installation 39 | 40 | ### Option 1: 41 | 42 | Download the zip file from this location: 43 | 44 | -> [prebuilt-packages (Linux/Windows)](https://github.com/yeus/syntexmex/releases) 45 | 46 | After downloading, open blender and go to: 47 | 48 | Edit -> Preferences -> Add-ons -> Install... 49 | 50 | Choose the downloaded zip file and press 51 | 52 | "Install Add-on from File..." 53 | 54 | Afterwards in Preferences, 55 | search for the **Syntexmex** plugin and 56 | put a checkmark in the box to enable the addon.nn 57 | 58 | 65 | ### Option 2 (not recommended): 66 | 67 | building the addon form source is quiet complicated as it involves 68 | a lot of 3rd party libraries. 69 | 70 | These are: 71 | 72 | skimage, shapely, pynndescent, Pillow, decorator, sklearn, scipy, numba, 73 | networkx, PIL, llvmlite 74 | 75 | after the add-on directory has been copied into 76 | the blender addons directory (TODO; where?), 77 | these libraries have to be copied into a "lib/" directory 78 | inside of the blender plugin. 79 | 80 | ## How to Use the Add-on 81 | 82 | ### Where to find it 83 | 84 | The Addon can be accessed through the properties sidebar in 85 | 3D View (Access with shortcut key: 'N') in a the *syntexmex* tab 86 | 87 |

118 | 119 | Example of Suzanne *before* making edges seamless and *after*: 120 | 121 | withseams

122 |

123 | 124 | 125 | ### Buttons 126 | 127 |

128 | 129 | **Synthesize UV example based texture** 130 | : Synthesize textures for UV islands and make edges between them seamless. 131 | 132 | **Synthesize textures to UV islands** 133 | : Only synthesize texture for each UV island without making edges seamless. 134 | 135 | **Make UV seams seamless** 136 | : Make edges between UV islands seamless. 137 | 138 | ### Parameters 139 | 140 | **source material** 141 | : Choose the source material which should get used for synthesizing 142 | a new material without seems on a given UV map 143 | 144 | **source texture** 145 | : choose one of the textures within the material as the basis for 146 | the synthesis 147 | 148 | **ex. scaling** 149 | : the example texture can be scaled up /down by a factor. Use this 150 | if you want to achieve a different scaling of the example texture 151 | in the final result. 152 | 153 | **library size** 154 | : This controls the quality of the algorithm. It defines the size of 155 | the search-index that the algorithm builds. Higher values will 156 | need more calculation time and memory. The quality is limited 157 | by the amount of available memory on your system. Normally values 158 | up to 100000 should not be a problem for most systems (~1-2GB). 159 | 160 | **patch size ratio** 161 | : The algorithm stitches together patches from the original example 162 | image. The size of the patches in relation to the example image 163 | can be controlled with this parameter. Larger sizes tend to 164 | improve the quality (although not always) and the coherence 165 | of the final result. 166 | 167 | **synth-res** 168 | : The resolution of the target texture. The higher this value 169 | the "smaller" synthesized features will become and vice versa. 170 | 171 | **seed value** 172 | : The algorithm makes use of some random internal parameters 173 | this value enables an option to get different versions of a 174 | synthesized texture 175 | 176 | ### Advanced Settings 177 | 178 | **debugging options** 179 | : Turn detailed debugging in console on/off. 180 | 181 | ## Some hints 182 | 183 | - usually its a good idea to first make a test run with a low value 184 | for library size to get a quick preview of what it might look like 185 | then once you are satisfied with the patch size, example scaling etc... 186 | turn library size up to the maximum value possible with the 187 | available memory. 188 | 189 | - make Sure to unwrap the UVs of your model. You will want 190 | to make sure that there are no overlapping UVs. The same rules apply here 191 | as for every other UV-unwrapping process. Good results can be achieved 192 | when UVs aren't too warped, and areas of UV faces correspond to the actual 193 | face areas. Also there should be a little gap of at least a couple pixels 194 | between each UV island to prevent textures from bleeding into 195 | other uv faces. 196 | 197 | - make sure when using "subdivision modifier" to put Options on "Sharp", 198 | because otherwise seams are going to be visible again due to the smoothing 199 | of UV coordinates 200 | 201 | - increase patch size for better adherence to the patterns found 202 | in the example image 203 | 204 | 205 | - alorithm automatically iterates over the longer of two edges and 206 | writes pixels from there into the face of the shorter edge 207 | -------------------------------------------------------------------------------- /uv_synthesize.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | ################################################################################ 5 | Copyright (C) 2020 Thomas Meschede a.k.a. yeus 6 | 7 | This program is free software: you can redistribute it and/or modify 8 | it under the terms of the GNU General Public License as published by 9 | the Free Software Foundation, either version 3 of the License, or 10 | (at your option) any later version. 11 | 12 | This program is distributed in the hope that it will be useful, 13 | but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | GNU General Public License for more details. 16 | 17 | You should have received a copy of the GNU General Public License 18 | along with this program. If not, see . 19 | ################################################################################ 20 | 21 | 22 | @author: Thomas Meschede a.k.a. yeus (yeusblender@gmail.com) 23 | 24 | This texture synthesis algorithm takes inspiration from three papers and 25 | combines their papers into a new algorithm: 26 | 27 | - Image Quilting for Texture Synthesis and Transfer [Efros, Freeman] 28 | - taking the optimal-patch seam strategy and 29 | - Fast Texture Synthesis using Tree-structured Vector Quantization [Wei, Levoy] 30 | - iterations, non-causal buildup local neighbourhood search 31 | - Real-Time Texture Synthesis by Patch-Based Sampling [Liang et al.] 32 | - building a gaussian image pyramid combined with KD-Trees for 33 | fast searches 34 | 35 | and some more inspiration from: 36 | - wfc synthesis algorithm 37 | 38 | TODO: optionally in the future: 39 | - Graphcut Textures: Image and Video Synthesis Using Graph Cuts [Kwatra, Schödl] 40 | """ 41 | 42 | __author__ = "Thomas Meschede / Thomas Meschede Rose" 43 | __copyright__ = "Copyright (C) 2020 Thomas Rose Meschede" 44 | __license__ = "Proprietary" 45 | __version__ = "1.0" 46 | 47 | import random 48 | import numpy as np 49 | import skimage 50 | import skimage.io 51 | import skimage.transform 52 | import threading 53 | #import gc 54 | import math 55 | import functools 56 | sign = functools.partial(math.copysign, 1) # either of these 57 | import logging 58 | logger = logging.getLogger(__name__) 59 | try: 60 | from . import tex_synthesize as ts 61 | except ImportError: 62 | import tex_synthesize as ts 63 | #import tex_synthesize as ts 64 | import pickle 65 | 66 | 67 | #def norm(x): return np.sqrt(x.dot(x)) 68 | def norm(x): return np.sqrt((x*x).sum(-1)) 69 | #need to be transposed for correct ultiplcation along axis 1 70 | def normalized(x): return (x.T /norm(x)).T 71 | 72 | def calc_angle_vec(u, v): 73 | """ 74 | >>> u = vec((1.0,1.0,0.0)) 75 | >>> v = vec((1.0,0.0,0.0)) 76 | >>> calc_angle_vec(u,v)*rad 77 | 45.00000000000001 78 | >>> u = vec((1.0,0.0,0.0)) 79 | >>> v = vec((-1.0,0.0,0.0)) 80 | >>> calc_angle_vec(u,v)*rad 81 | 180.0 82 | >>> u = vec([-9.38963669e-01, 3.44016319e-01, 1.38777878e-17]) 83 | >>> v = vec([-0.93896367, 0.34401632, 0.]) 84 | >>> u @ v / (norm(v)*norm(u)) 85 | 1.0000000000000002 86 | >>> calc_angle_vec(u,v)*rad 87 | 0.0 88 | """ 89 | #angle = np.arctan2(norm(np.cross(u,v)), np.dot(u,v)) 90 | res = np.sum(u*v) / (norm(u) * norm(v)) 91 | t = np.clip(res,-1.0,1.0) 92 | angle = np.arccos(t) 93 | return angle 94 | 95 | def tqdm(iterator, *args, **kwargs): 96 | return iterator 97 | 98 | GB = 1.0/1024**3 #GB factor 99 | 100 | def normalized_synthmap(synthmap, example): 101 | return synthmap/(*example.shape[:2][::-1],1) 102 | 103 | @ts.timing 104 | def synthesize_textures_on_uvs(synth_tex=False, 105 | seamless_UVs=False, 106 | msg_queue=None, 107 | stop_event=None, 108 | edge_iterations=0, 109 | *argv, **kwargs): 110 | """ 111 | msg_queue lets the algorithm share intermediate steps 112 | when using threaded calculations (queue.Queue) 113 | """ 114 | target = kwargs['target'] 115 | example = kwargs['example'] 116 | patch_ratio = kwargs['patch_ratio'] 117 | libsize = kwargs.get('libsize',128*128) 118 | face_uvs = kwargs['face_uvs'] 119 | islands = kwargs['islands'] 120 | edge_infos = kwargs['edge_infos'] 121 | seed = kwargs.get('seed_value', 0) 122 | logger.info(f"seed: {seed}") 123 | if msg_queue is None: msg_queue=False 124 | if stop_event is None: stop_event=threading.Event() 125 | 126 | #set random seed for algorithm 127 | np.random.seed(seed) 128 | random.seed(seed) 129 | 130 | ta_map = None 131 | 132 | #TODO: check whether we have "left or right" sided coordinate system 133 | 134 | if synth_tex: #generate initial textures 135 | #TODO: make sure all islands are taken into account 136 | logger.info("synthesize uv islands") 137 | res = target.shape[:2] 138 | for island in islands: 139 | island_uvs = [face_uvs[i] for i in island] 140 | island_uvs_px = np.array([uv[...,::-1] * res[:2] for uv in island_uvs]) 141 | #get a boundingbox for the entire island 142 | isl_mins = np.array([isl_px.min(axis=0) for isl_px in island_uvs_px]) 143 | ymin,xmin = isl_mins.min(axis=0).astype(int)#-(1,1) 144 | ymin,xmin = max(ymin,0),max(xmin,0) 145 | isl_mins = np.array([isl_px.max(axis=0) for isl_px in island_uvs_px]) 146 | ymax,xmax = isl_mins.max(axis=0).astype(int)#+(1,1) 147 | ymax,xmax = min(ymax,res[0]),min(xmax,res[1]) 148 | 149 | #add .5 so that uv coordinates refer to the middle of a pixel 150 | # this has to be done after the "mins" where found 151 | island_uvs_px = [isl + (-0.5,-0.5) for isl in island_uvs_px] 152 | 153 | island_mask = np.zeros(target.shape[:2]) 154 | for uvs in island_uvs_px: 155 | island_mask[skimage.draw.polygon(*uvs.T)]=1.0 156 | island_mask = island_mask[ymin:ymax,xmin:xmax]>0 157 | 158 | target, ta_map = ts.fill_area_with_texture(target, example, ta_map, 159 | patch_ratio=patch_ratio, libsize = libsize, 160 | bounding_box=(ymin,xmin,ymax,xmax), 161 | mask = island_mask) 162 | if msg_queue: msg_queue.put((target,normalized_synthmap(ta_map,example))) 163 | 164 | if stop_event.is_set(): 165 | logger.info("stopping_thread") 166 | return 167 | 168 | if seamless_UVs: 169 | tree_info = None 170 | if ta_map is None: 171 | ta_map = np.ones([*target.shape[:2],3])*-1 172 | for i,(e1,e2) in enumerate(edge_infos): 173 | logger.info(f"making edge seamless: #{i}") 174 | #TODO: add pre-calculated island mask to better find "valid" uv pixels 175 | edge1 = e1[0],face_uvs[e1[1]][:,::-1]*target.shape[:2] 176 | edge2 = e2[0],face_uvs[e2[1]][:,::-1]*target.shape[:2] 177 | target, ta_map, tree_info = ts.make_seamless_edge(edge1, edge2, 178 | target, example, ta_map, 179 | patch_ratio, libsize, 180 | tree_info=tree_info, 181 | debug_level=0) 182 | if msg_queue: msg_queue.put((target,normalized_synthmap(ta_map,example))) 183 | if (edge_iterations != 0) and (i >= edge_iterations): break 184 | if stop_event.is_set(): 185 | logger.info("stopping_thread") 186 | return 187 | #debug_image(target2) 188 | #import ipdb; ipdb.set_trace() # BREAKPOINT 189 | 190 | return (target, 191 | normalized_synthmap(ta_map,example), 192 | ta_map) 193 | 194 | def check_face_orientation(face): 195 | edge_vecs = np.roll(face,1,0)-face 196 | return np.cross(np.roll(edge_vecs,1,0),edge_vecs) 197 | 198 | def paint_uv_dots(faces, target): 199 | for f in faces.values(): 200 | for v in f[:,::-1]*target.shape[:2]: 201 | #v=v[::-1] 202 | target[skimage.draw.circle(v[0],v[1],2)]=(1,0,0,1) 203 | 204 | def reconstruct_synthmap(synthmap,example,mode="coordinates"): 205 | """ 206 | TODO: reconstruct from multiple examples as well (third channel in ta_map) 207 | 208 | modes: "coordinates", "normalized" 209 | """ 210 | if mode=="coordinates": 211 | return example[synthmap[:,:,1],synthmap[:,:,0]].copy() 212 | elif mode=='normalized': 213 | synthmap = synthmap * (*example.shape[:2][::-1],1) 214 | synthmap = synthmap.astype(int) 215 | return example[synthmap[:,:,1],synthmap[:,:,0]].copy() 216 | 217 | if __name__=="__main__": 218 | logging.basicConfig(level=logging.INFO) 219 | logger.setLevel(logging.INFO) 220 | logging.getLogger('tex_synthesize').setLevel(logging.INFO) 221 | 222 | #logging.get 223 | 224 | with open('uv_test_island.pickle', 'rb') as handle: 225 | uv_info = pickle.load(handle) 226 | 227 | #skimage.io.imshow_collection([uv_info["target"],uv_info["example"]]) 228 | 229 | target=uv_info['target'] 230 | example=uv_info['example'] 231 | #skimage.io.imshow_collection([uv_info['target']]) 232 | #skimage.io.imshow_collection([example]) 233 | #paint_uv_dots(uv_info['face_uvs'],target) 234 | 235 | 236 | ta, ta_map1, ta_map2 = synthesize_textures_on_uvs(synth_tex=False, 237 | seamless_UVs=True, 238 | edge_iterations=0, 239 | **uv_info) 240 | logger.info("finished test!") 241 | skimage.io.imshow_collection([target, ta_map1, ta_map2]) 242 | 243 | skimage.io.imsave("test.png",ta_map1) 244 | 245 | ta_map_recon = ta_map1*(*example.shape[:2],1) 246 | ((ta_map_recon-ta_map2)**2).sum() #test the rounding-error (its pretty good) 247 | 248 | ta_map = ta_map2.astype(int) 249 | 250 | conv = reconstruct_synthmap(ta_map1,example, mode="normalized") 251 | skimage.io.imshow_collection([ta,conv, ((ta-conv)**2)[...,:3]]) 252 | mask = np.all(ta_map1[:,:,:2]>0,axis=2) 253 | edge_seams = ts.copy_img(ta_map1.copy(),conv[:,:,:3],pos=(0,0),mask=mask) 254 | skimage.io.imshow_collection([target,ta, edge_seams]) 255 | 256 | #uv_info['edge_infos'][0] 257 | 258 | #faces = uv_info['island_uvs'] 259 | 260 | #[check_face_orientation(f) for f in faces] 261 | 262 | -------------------------------------------------------------------------------- /uv_prepare.py: -------------------------------------------------------------------------------- 1 | """ 2 | ################################################################################ 3 | Copyright (C) 2020 Thomas Meschede a.k.a. yeus 4 | 5 | This program is free software: you can redistribute it and/or modify 6 | it under the terms of the GNU General Public License as published by 7 | the Free Software Foundation, either version 3 of the License, or 8 | (at your option) any later version. 9 | 10 | This program is distributed in the hope that it will be useful, 11 | but WITHOUT ANY WARRANTY; without even the implied warranty of 12 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 13 | GNU General Public License for more details. 14 | 15 | You should have received a copy of the GNU General Public License 16 | along with this program. If not, see . 17 | ################################################################################ 18 | """ 19 | 20 | import bpy 21 | import bmesh 22 | import numpy as np 23 | import itertools 24 | import skimage 25 | import skimage.io 26 | import skimage.transform 27 | import networkx as nx 28 | import logging 29 | import pickle 30 | logger = logging.getLogger(__name__) 31 | 32 | #def norm(x): return np.sqrt(x.dot(x)) 33 | def norm(x): return np.sqrt((x*x).sum(-1)) 34 | #need to be transposed for correct ultiplcation along axis 1 35 | def normalized(x): return (x.T /norm(x)).T 36 | 37 | def debug_image(img, name = None): 38 | if img.shape[2]<4:#add alpha channel 39 | img = np.pad(img,((0,0),(0,0),(0,1)),constant_values=1) 40 | 41 | new_tex = bpy.data.images.new("debug", width=img.shape[1], 42 | height=img.shape[0]) 43 | new_tex.pixels[:] = img.flatten() 44 | 45 | def create_bmesh_from_active_object(): 46 | if bpy.context.active_object.mode == 'OBJECT': 47 | ob = bpy.context.selected_objects[0] 48 | me = ob.data 49 | bm = bmesh.new() # create an empty BMesh 50 | bm.from_mesh(me) # fill it in from a Mesh 51 | else: 52 | ob = bpy.context.edit_object 53 | me = ob.data 54 | bm = bmesh.from_edit_mesh(me) 55 | 56 | # the next step makes operations possible in non-edit mode 57 | bm.faces.ensure_lookup_table() 58 | return ob,bm 59 | 60 | #TODO: find_uv_origin(uv): #so that we know where and how 61 | # to start the texture synthesis 62 | def find_uv_origin(uv): 63 | """ find out orientation of the uv face by 64 | calculating the normal in the direction of the loops 65 | we want the "left" of the two uv vertices 66 | to be the starting point. and that depends on the 67 | orientation of the uv face""" 68 | uv_n = (uv[1] - uv[0]).cross(uv[2] - uv[1]) 69 | if uv_n < 0: #means loop direction is clockwise 70 | origin = uv[1] 71 | else: #means loop direction is counter clockwise 72 | origin = uv[0] 73 | 74 | def find_minmax(uv): 75 | y_max,idx = max((v[1],i) for i,v in enumerate(uv)) 76 | x_max,idx = max((v[0],i) for i,v in enumerate(uv)) 77 | 78 | y_min,idx = min((v[1],i) for i,v in enumerate(uv)) 79 | x_min,idx = min((v[0],i) for i,v in enumerate(uv)) 80 | return x_min,x_max,y_min,y_max 81 | 82 | def create_initial_image(res): 83 | import skimage as skim 84 | #TODO: add option to remove "random shapes" 85 | tmp_img,_ = random_shapes((res[1],res[0]), 86 | max_shapes=20, 87 | intensity_range=((100, 255),)) 88 | 89 | img = skim.util.img_as_float(tmp_img) 90 | #add padding 91 | img = np.pad(img,((res[1],res[1]),(res[0],res[0]),(0,0)),constant_values=0) 92 | return img 93 | 94 | def add_uvs(img, xy): 95 | # draw uvs in the image for debugging purposes: 96 | for x,y in xy: 97 | rr,cc = skim.draw.circle(y, x, 5) 98 | img[rr,cc,:]=(0,0,1) 99 | rr,cc = skim.draw.circle(y, x, 0.5) 100 | img[rr,cc,:]=(1,0,0) 101 | 102 | def add_face_shadow(img, xy): 103 | x,y = zip(*xy) 104 | rr,cc = skim.draw.polygon(y, x) 105 | img[rr,cc,2] = 0 106 | img = np.clip(img,0.0,1.0) 107 | 108 | def copy_img(target, src, pos, mask=None): 109 | """ 110 | copy image src to target at pos 111 | """ 112 | #TODO: handle border clipping problems 113 | # - when copying images that exten over "left" and "top" edges 114 | sh,sw,sch = src.shape 115 | th,tw,tch = target.shape 116 | 117 | i0x = pos[0] 118 | i0y = pos[1] 119 | i1x = i0x+sw 120 | i1y = i0y+sh 121 | t_ix0 = max(i0x, 0) 122 | t_iy0 = max(i0y, 0) 123 | t_ix1 = min(i1x, tw) 124 | t_iy1 = min(i1y, th) 125 | 126 | #cut patch to right size 127 | pw, ph = t_ix1 - t_ix0, t_iy1 - t_iy0 128 | 129 | if mask is None: 130 | tch = sch 131 | #print(pos) 132 | #import ipdb; ipdb.set_trace() # BREAKPOINT 133 | target[t_iy0:t_iy1, t_ix0:t_ix1, 0:tch] = src[0:ph, 0:pw] 134 | else: 135 | m = mask 136 | target[t_iy0:t_iy1, t_ix0:t_ix1, 0:tch][m] = src[m] 137 | 138 | 139 | def create_mask(vecs): 140 | #TODO: create a "soft" mask 141 | #rr,cc = skim.draw.polygon(y, x) 142 | #mask = np.zeros_like(img) 143 | #mask[rr,cc,0] = 0.5 144 | mask = skim.draw.polygon2mask(img.shape[:2], vecs) 145 | return mask 146 | 147 | def draw_face_outline(img, uv_p, uv_vp): 148 | #color only the corners 149 | abs = ((0,0),(0,1),(1,0),(1,1)) 150 | for a,b in abs: 151 | #corners 152 | cbase = uv_p[0] + a * uv_vp[0] + b * uv_vp[3] 153 | rr,cc = skim.draw.circle(cbase[1], cbase[0], 5) 154 | target[rr,cc,:3]=(0,0,1) 155 | 156 | #edge1 157 | a_length = mh.norm(uv_vp[0]) 158 | for i in np.arange(0,a_length,0.3): 159 | a = i/a_length 160 | b = 0 161 | base = uv_p[0] + a * uv_vp[0] + b * uv_vp[3] 162 | idx = base.astype(int) 163 | img[idx[1],idx[0],:3]=(0,1,0) 164 | #import ipdb; ipdb.set_trace() # BREAKPOINT 165 | 166 | b_length = mh.norm(uv_vp[3]) 167 | for i in np.arange(0,a_length,0.3): 168 | a = 0 169 | b = i/a_length 170 | base = uv_p[0] + a * uv_vp[0] + b * uv_vp[3] 171 | idx = base.astype(int) 172 | img[idx[1],idx[0],:3]=(0,1,0) 173 | #import ipdb; ipdb.set_trace() # BREAKPOINT 174 | 175 | target[cbase[1].astype(int), cbase[0].astype(int) 176 | ,:3]=(1,0,0) 177 | 178 | def draw_triangle_outline(img, uv_p, uv_vp): 179 | a_length = mh.norm(uv_vp[0]) 180 | b_length = mh.norm(uv_vp[1]) 181 | for b in np.arange(0,b_length,0.5): 182 | #for a in np.arange() 183 | a = 1-b/b_length 184 | b = b/b_length 185 | x = uv_p[0] + a * uv_vp[0] + b * uv_vp[3] 186 | img[x[1].astype(int),x[0].astype(int),:3]=(1,1,0) 187 | 188 | 189 | def sign(p1, p2, p3): 190 | return (p1[0] - p3[0]) * (p2[1] - p3[1]) - (p2[0] - p3[0]) * (p1[1] - p3[1]); 191 | 192 | def PointInTriangle(pt, v1, v2, v3): 193 | d1 = sign(pt, v1, v2) 194 | d2 = sign(pt, v2, v3) 195 | d3 = sign(pt, v3, v1) 196 | 197 | has_neg = (d1 < 0) or (d2 < 0) or (d3 < 0); 198 | has_pos = (d1 > 0) or (d2 > 0) or (d3 > 0); 199 | 200 | return not (has_neg and has_pos); 201 | 202 | def idx_sanitize(co, img): 203 | x_max = img.shape[:2][::-1] 204 | x_min = (0,0) 205 | z = np.min([x_max,co], axis=0) 206 | return np.max([z,x_min], axis=0) 207 | 208 | def idx_check(co, img): 209 | x_max = img.shape[:2][::-1] 210 | x_min = np.array((0,0)) 211 | return np.all((co=x_min)) 212 | 213 | def init_face_map(img, bm): 214 | #TODO: get rid of empty pixels 215 | #TODO: make algorithm more efficient 216 | for face in bm.faces:#[:1]: 217 | uv = np.array([l[uv_layer].uv for l in face.loops]) 218 | uv_p = uv * res[:2] #transform to pixel space 219 | 220 | #get four uv edge vectors: 221 | uv_v = np.array([uv[1]-uv[0], 222 | uv[2]-uv[1], 223 | uv[3]-uv[2], 224 | uv[0]-uv[3]]) 225 | 226 | 227 | #TODO: do the followgin for both face triangles 228 | tris = [(0,1,2),(2,3,0)] 229 | img = face_map 230 | for tr in tris: 231 | base = uv_p[tr[1]] 232 | uv_v = np.array([uv[tr[0]]-uv[tr[1]], 233 | uv[tr[2]]-uv[tr[1]]]) 234 | 235 | #transform to pixel space 236 | uv_vp = uv_v * res[:2] 237 | 238 | #draw patches 239 | a_length = mh.norm(uv_vp[0]) 240 | b_length = mh.norm(uv_vp[1]) 241 | st_len = 0.8 242 | for b in np.arange(0,b_length,st_len): 243 | #for a in np.arange() 244 | b_ = b/b_length 245 | c_ = a_length * (1-b_) 246 | #import ipdb; ipdb.set_trace() # BREAKPOINT 247 | for a_ in np.arange(0,c_,st_len): 248 | a_ = a_/a_length 249 | x = base + a_ * uv_vp[0] + b_ * uv_vp[1] 250 | x = x.astype(int) 251 | #if x[1]==300: 252 | # import ipdb; ipdb.set_trace() # BREAKPOINT 253 | if idx_check(x, img): 254 | img[x[1],x[0]]=face.index 255 | 256 | #draw_face_outline(target, uv_p, uv_vp) 257 | #draw_triangle_outline(target, uv_p, uv_vp) 258 | 259 | #import ipdb; ipdb.set_trace() # BREAKPOINT 260 | 261 | def empty_copy(img, dtype): 262 | #return np.zeros(img.shape[:2], dtype = dtype) 263 | return np.full(img.shape[:2], -1, dtype = dtype) 264 | 265 | 266 | def blimage2array(blender_source): 267 | res = np.array((blender_source.size[1],blender_source.size[0])) 268 | return np.array(list(blender_source.pixels)).reshape(*res,4) 269 | 270 | def init_texture_buffers(example_image, target_texture, example_scaling): 271 | # numpy handles shapes in a different way then images in blender, 272 | # because of this, the convention, when indexing looks like this: 273 | # shape = (y from bottom, x from left, alpha) 274 | res_ex = np.array((example_image.size[1],example_image.size[0])) 275 | nt = target_texture 276 | res = np.array((nt.size[1], nt.size[0])) 277 | # define numpy buffers for image data access 278 | target = np.array(list(nt.pixels)).reshape(*res,4) # create an editable copy 279 | example = np.array(example_image.pixels).reshape(*res_ex,4) 280 | example = skimage.transform.rescale(example, example_scaling, 281 | multichannel=True) 282 | 283 | return example, target 284 | 285 | #import ipdb; ipdb.set_trace() # BREAKPOINT 286 | 287 | # TODO: calculate face area and scale example 288 | # accordingly 289 | 290 | #TODO: 291 | #find center, so that we can either start the 292 | #algorithm in the center or extend a little bit beyond the 293 | # borders of the face: 294 | #uv_center = np.sum(uv, axis=0)/4 295 | 296 | """ 297 | face_map = empty_copy(target, int) 298 | init_face_map(face_map, bm) 299 | 300 | #convert facemap into image 301 | num_faces = len(bm.faces) 302 | #create random colors for faces (+1 because index starts with 0) 303 | rand_colors = np.random.rand(num_faces+1,3) 304 | map2 = np.array([rand_colors[px] if px != -1 else (0,0,0) for px in face_map.flatten()]) 305 | shape = (*target.shape[0:2],3) 306 | map2 = map2.reshape(shape) 307 | #if display_face_map: 308 | target[:,:,0:3] = map2 309 | #import ipdb; ipdb.set_trace() # BREAKPOINT 310 | """ 311 | 312 | #debug_img(tmp0) 313 | #import ipdb; ipdb.set_trace() # BREAKPOINT 314 | 315 | #randomly copy patch to target image 316 | #coords = np.random.rand(2)*res[:2]/2 + res[:2]/4 317 | #copy_img(target, patch, coords.astype(int)) 318 | 319 | #TODO: get rid of empty pixels 320 | #TODO: make algorithm more efficient 321 | 322 | #TODO: make a map of "remaining" pixels which did not get painted 323 | # to get rid of holes 324 | 325 | """ 326 | img = target 327 | for face in bm.faces[:]: 328 | uv = np.array([l[uv_layer].uv for l in face.loops]) 329 | uv_p = uv * res[:2] #transform to pixel space 330 | 331 | #get four uv edge vectors: 332 | uv_v = np.array([uv[1]-uv[0], 333 | uv[2]-uv[1], 334 | uv[3]-uv[2], 335 | uv[0]-uv[3]]) 336 | 337 | 338 | #TODO: do the followgin for both face triangles 339 | tris = [(0,1,2),(2,3,0)] 340 | for tr in tris: 341 | base = uv_p[tr[1]] 342 | uv_v = np.array([uv[tr[0]]-uv[tr[1]], 343 | uv[tr[2]]-uv[tr[1]]]) 344 | 345 | #transform to pixel space 346 | uv_vp = uv_v * res[:2] 347 | 348 | #draw patches 349 | a_length = mh.norm(uv_vp[0]) 350 | b_length = mh.norm(uv_vp[1]) 351 | st_len = 20.0 352 | for b in np.arange(0,b_length,st_len): 353 | #for a in np.arange() 354 | b_ = b/b_length 355 | c_ = a_length * (1-b_) 356 | #import ipdb; ipdb.set_trace() # BREAKPOINT 357 | for a_ in np.arange(0,c_,st_len): 358 | a_ = a_/a_length 359 | x = base + a_ * uv_vp[0] + b_ * uv_vp[1] 360 | patch = random.choice(patches) 361 | coords = mh.vec((x[0]-res_patch[0]*0.5, 362 | x[1]-res_patch[1]*0.5)).astype(int) 363 | #if all(coords == ( -8, 1013)): 364 | #import ipdb; ipdb.set_trace() # BREAKPOINT 365 | copy_img(target, patch, coords) 366 | #target[coords[1],coords[0],:3]=(0.5,0.5,1.0) 367 | #img[x[1].astype(int),x[0].astype(int)]=patch 368 | """ 369 | 370 | def get_uv_levelset(uvs): 371 | from shapely.geometry import Polygon, Point 372 | 373 | uv_p = uvs * res[:2] #transform to pixel space 374 | face = Polygon(uv_p) 375 | boundary = face.boundary 376 | face_container = face.buffer(+10.0) #add two pixels on the container 377 | bbox = face_container.bounds 378 | minx, miny, maxx, maxy = bbox_px = np.round(np.array(bbox)).astype(int) 379 | w,h = maxx - minx, maxy-miny 380 | 381 | def distance(x,y): 382 | d = boundary.distance(Point(x,y)) 383 | if face.contains(Point(x,y)): return d 384 | else: return -d 385 | 386 | bbcoords = itertools.product(range(miny,maxy), range(minx, maxx)) 387 | levelset = np.array([distance(x,y) for y,x in bbcoords]).reshape(h,w) 388 | #normalize levelset: 389 | #levelset = np.maximum(levelset/levelset.max(),0.0) 390 | return levelset, bbox_px 391 | 392 | #get all UVs per face: 393 | 394 | def get_uvs(bm): 395 | uvs = {}#defaultdict(list) 396 | for face in bm.faces:#[:1]: 397 | uvs[face.index] = get_face_uvs(face,bm) 398 | return uvs 399 | 400 | def get_face_uvs(face, bm): 401 | uv_layer = bm.loops.layers.uv['UVMap'] 402 | uv = np.array([l[uv_layer].uv for l in face.loops]) 403 | return uv 404 | 405 | def get_edge_uvs(edge, bm): 406 | uv_layer = bm.loops.layers.uv['UVMap'] 407 | #if len(edge.link_loops) == 2: 408 | l = edge.link_loops #the two "opposite loops" 409 | ln = [l.link_loop_next for l in edge.link_loops] #for the next uv 410 | uv_edges = np.array([[l[uv_layer].uv for l in lln] for lln in zip(l,ln)]) 411 | #uv_edge2 = np.array([l[uv_layer].uv for l in [l2,l2n]]) 412 | return uv_edges 413 | 414 | 415 | def loops_connected(edge, bm): 416 | """check if the uv coordinates of an edge are the 417 | same for both connected faces""" 418 | uv_edges = get_edge_uvs(edge, bm) 419 | if len(uv_edges)<2: 420 | return False 421 | else: 422 | if np.all(uv_edges[0]==uv_edges[1][::-1]): return True 423 | else: return False 424 | 425 | def is_border_edge(edge, bm): 426 | if len(edge.link_loops)<2: 427 | return True 428 | else: return False 429 | 430 | def generate_edge_loop_uvs(bm_edges, res, bm): 431 | """generates an edge_info structure that can be used 432 | as input to make seamless edges""" 433 | edge_uvs = [get_edge_uvs(e, bm) for e in bm_edges] #get uvs 434 | #[len(e) for e in edge_uvs] 435 | #import ipdb; ipdb.set_trace() # BREAKPOINT 436 | #switching x and y coordinates to adapt to numpy column-row convention 437 | edge_uvs = np.array(edge_uvs)[...,::-1]*res + (-0.5,-0.5) #switch xy to numpy yx convention and transform into pixel space 438 | #import ipdb; ipdb.set_trace() # BREAKPOINT 439 | face_uvs = np.array([(e.link_faces[0].index, 440 | e.link_faces[1].index) for e in bm_edges]) 441 | #import ipdb; ipdb.set_trace() 442 | #for face its a little more complicated switching the coordinates 443 | #as there can have different number of vertices and thus 444 | # can not be put into a homogenous numpy array 445 | #face_uvs = [[f[...,::-1]*res + (-0.5,-0.5) for f in fs] for fs in face_uvs] 446 | #face_uvs = [np.array(face_uvs)[...,::-1]*res + (-0.5,-0.5)] 447 | #import ipdb; ipdb.set_trace() # BREAKPOINT 448 | edge_infos1 = tuple(zip(edge_uvs[:,0,:,:],face_uvs[:,0])) 449 | edge_infos2 = tuple(zip(edge_uvs[:,1,:,:],face_uvs[:,1])) 450 | return edge_infos1,edge_infos2 451 | 452 | def prepare_uv_synth_info(example, 453 | target, bm, 454 | patch_ratio, 455 | libsize): 456 | logger.info("generate uv synth info") 457 | #import ipdb; ipdb.set_trace() # BREAKPOINT 458 | #generate a list of connected faces 459 | bm.edges.index_update() 460 | connected_edges = [e for e in bm.edges if loops_connected(e, bm)] 461 | #build a graph of connected faces 462 | connected_faces = [(e.link_faces[0].index,e.link_faces[1].index, e.index) 463 | for e in connected_edges] 464 | G = nx.Graph() 465 | G.add_weighted_edges_from(connected_faces, weight='index') 466 | islands = list(nx.connected_components(G)) 467 | 468 | res = np.array(target.shape[:2]) 469 | #res_ex 470 | 471 | #import ipdb; ipdb.set_trace() # BREAKPOINT 472 | 473 | #build a list of edge pairs 474 | #iterate through edges 475 | #https://b3d.interplanety.org/en/learning-loops/ 476 | bm.edges.index_update() 477 | continuous_edges = [edge for edge in bm.edges if not is_border_edge(edge,bm)] 478 | unconnected_edges = [edge for edge in continuous_edges if not 479 | loops_connected(edge, bm)] 480 | 481 | #import ipdb; ipdb.set_trace() # BREAKPOINT 482 | edge_infos1, edge_infos2 = generate_edge_loop_uvs(unconnected_edges, 483 | res, bm) 484 | uv_info = { 485 | "target":target, 486 | "example":example, 487 | "patch_ratio":patch_ratio, 488 | "libsize":libsize, 489 | "face_uvs":get_uvs(bm), 490 | "islands":islands, 491 | "edge_infos":list(zip(edge_infos1, edge_infos2)) 492 | } 493 | 494 | #uncomment, if picklig is needed for debugging the senthesis algorithm 495 | #needs to be debugged outside of blender 496 | if False: 497 | logger.info("saving pickle") 498 | with open('uv_test_island.pickle', 'wb') as handle: 499 | pickle.dump(uv_info, handle, protocol=pickle.HIGHEST_PROTOCOL) 500 | 501 | return uv_info 502 | 503 | 504 | """for face in bm.faces[:1]: 505 | uv = np.array([l[uv_layer].uv for l in face.loops]) 506 | 507 | import ipdb; ipdb.set_trace() # BREAKPOINT 508 | """ 509 | #for uvs in list(get_uvs(bm).values())[:]: 510 | # #iterate through each edge 511 | # import ipdb; ipdb.set_trace() # BREAKPOINT 512 | 513 | 514 | #TODO: find out which edges are already "connected", because 515 | 516 | #make UVs seamless 517 | 518 | #for these we don't need to copy the pixels 519 | 520 | #TODO: for directly connected faces we can omit the whole 521 | #seamless procedure and render the entire "metaface" as a whole 522 | #and thn concentrate on indiviual edges oly afterwards 523 | 524 | #debug_image(cospxs) 525 | #debug_image(f2) 526 | #debug_image(f1) 527 | #import ipdb; ipdb.set_trace() # BREAKPOINT 528 | 529 | 530 | """ 531 | #get four uv edge vectors: 532 | uv_v = np.array([uv[1]-uv[0], 533 | uv[2]-uv[1], 534 | uv[3]-uv[2], 535 | uv[0]-uv[3]]) 536 | """ 537 | 538 | #find points on the "outside" of uv_edges: which correspond to certain 539 | #points in other faces according to some rule (probably try to resemble a 540 | # straight line as much as possible) 541 | 542 | 543 | #debug_image(face_map) 544 | 545 | #bmesh.update_edit_mesh(me, True) 546 | 547 | 548 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Syntexmex 3 | 4 | - a plugin for texture synthesis on 3d meshes and removing edge seams 5 | 6 | Copyright (C) 2020 Thomas Meschede a.k.a. yeus 7 | 8 | This program is free software: you can redistribute it and/or modify 9 | it under the terms of the GNU General Public License as published by 10 | the Free Software Foundation, either version 3 of the License, or 11 | (at your option) any later version. 12 | 13 | This program is distributed in the hope that it will be useful, 14 | but WITHOUT ANY WARRANTY; without even the implied warranty of 15 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 16 | GNU General Public License for more details. 17 | 18 | You should have received a copy of the GNU General Public License 19 | along with this program. If not, see . 20 | """ 21 | 22 | import importlib 23 | import bpy 24 | import os 25 | import sys 26 | import logging 27 | import numpy as np 28 | import threading, queue, time 29 | import functools 30 | logger = logging.getLogger(__name__) 31 | logger.setLevel(logging.INFO) 32 | 33 | """ 34 | To set up the logging module to your needs, create a file $HOME/.config/blender/{version}/scripts/startup/setup_logging.py (this is on Linux, but you’re likely a developer, so you’ll know where to find this directory). If the folder doesn’t exist yet, create it. I like to put something like this in there: 35 | 36 | import logging 37 | 38 | logging.basicConfig(level=logging.INFO, 39 | format='%(asctime)-15s %(levelname)8s %(name)s %(message)s') 40 | 41 | for name in ('blender_id', 'blender_cloud'): 42 | logging.getLogger(name).setLevel(logging.DEBUG) 43 | 44 | def register(): 45 | pass 46 | """ 47 | 48 | 49 | # 50 | #blend_dir = os.path.dirname(bpy.data.filepath) 51 | # if blend_dir not in sys.path: 52 | # sys.path.append(blend_dir) 53 | # temporarily appends the folder containing this file into sys.path 54 | #main_dir = os.path.dirname(bpy.data.filepath) #blender directory 55 | #sys.path.append(main_dir) 56 | main_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'lib') #addon dir "__init__.py" + lib 57 | sys.path.append(main_dir) 58 | logger.info(f"add library directory to syntexmex: {main_dir}") 59 | 60 | from . import uv_prepare as up 61 | importlib.reload(up) 62 | from . import uv_synthesize as us 63 | importlib.reload(us) 64 | importlib.reload(us.ts) 65 | 66 | __author__ = "yeus " 67 | __status__ = "alpha" 68 | __version__ = "0.9.2" 69 | __date__ = "2020 Feb 29th" 70 | 71 | #TODO: fill this out 72 | bl_info = { 73 | "name": "Syntexmex Blender", 74 | "author": "yeus Properties > syntexmex", 78 | "description": "Generate example-based seamless textures", 79 | "warning": "", 80 | "wiki_url": "http://yeus.gitlab.io/syntexmex TODO", 81 | "category": "View3D", 82 | "support": "COMMUNITY", 83 | } 84 | 85 | #from operator import * 86 | 87 | # TODO: remove tabs 88 | # https://blender.stackexchange.com/questions/97502/removing-tabs-from-tool-shelf-t-key/97503#97503 89 | 90 | synth_progress = 0.0 91 | 92 | def multiline_label(layout,text): 93 | for t in text.split("\n"): 94 | layout.label(text=t) 95 | 96 | class syntexmex(bpy.types.Operator): 97 | """This operator synthesizes texture in various ways on UV textures""" 98 | bl_idname = "texture.syntexmex" 99 | bl_label = "Synthesize Operations on UV Textures" 100 | bl_category = 'syntexmex' 101 | bl_options = {'REGISTER', 'UNDO'} 102 | 103 | patch_size: bpy.props.FloatProperty( 104 | name="Patch Size Ratio", 105 | description="Set width of patches as a ratio of shortest edge of an image", 106 | min=0.0, 107 | max=0.5, 108 | default=0.1, 109 | precision=3, 110 | step=0.1 111 | ) 112 | example_image: bpy.props.StringProperty() 113 | example_scaling: bpy.props.FloatProperty( 114 | name="Example Scaling", 115 | description="""Scale Example to a certain size which will be used 116 | to generate the texture""", 117 | min=0.0, 118 | max=1.0, 119 | default=1.0, 120 | precision=3, 121 | step=1 122 | ) 123 | target_image: bpy.props.StringProperty() 124 | target_resolution: bpy.props.IntVectorProperty(name="synth-res.", 125 | size=2, 126 | min=0, 127 | max=10000, 128 | default=(1024,1024)) 129 | synth_tex: bpy.props.BoolProperty(name="synthesize textures") 130 | seamless_UVs: bpy.props.BoolProperty(name="seamless UV islands") 131 | libsize: bpy.props.IntProperty(name="patch library size") 132 | seed_value: bpy.props.IntProperty( 133 | name="Seed Value", 134 | description="Seed value for predictable texture generation", 135 | default = 0 136 | ) 137 | replace_material: bpy.props.BoolProperty(name="replace material", 138 | default=False) 139 | 140 | _timer = None 141 | 142 | @classmethod 143 | def poll(cls, context): 144 | #TODO: this operator should also be able to execute in other 145 | # windows as well 146 | return context.space_data.type in {'VIEW_3D'} 147 | 148 | def run_algorithm(self, context): 149 | logger.info("start synthesizing algorithm") 150 | #scene = context.scene 151 | 152 | #TODO: select specific object in menu 153 | obj, bm = up.create_bmesh_from_active_object() 154 | uv_layer = bm.loops.layers.uv['UVMap'] 155 | 156 | logging.info(self.example_image) 157 | logging.info(self.target_image) 158 | 159 | example_image = bpy.data.images[self.example_image] 160 | if self.target_image is "": 161 | self.target_image = 'synthtarget' 162 | 163 | bpy.data.images.new(self.target_image, 164 | self.target_resolution[0], 165 | self.target_resolution[1], 166 | alpha=False,float_buffer=True) 167 | self.target = bpy.data.images[self.target_image] 168 | 169 | ta_width,ta_height=self.target.size 170 | self.ta_map = bpy.data.images.new("ta_map",ta_width,ta_height, 171 | alpha=False,float_buffer=True, 172 | is_data=True) 173 | #BlendDataImages.new(name, width, height, alpha=False, float_buffer=False, stereo3d=False, is_data=False, tiled=False) 174 | 175 | examplebuf, targetbuf = up.init_texture_buffers(example_image, 176 | self.target, self.example_scaling) 177 | 178 | self.msg_queue = queue.Queue() 179 | args = (examplebuf, 180 | targetbuf, 181 | bm, 182 | self.patch_size, 183 | self.libsize) 184 | kwargs = dict() 185 | 186 | uv_info = up.prepare_uv_synth_info(*args,**kwargs) 187 | uv_info['seed_value']=self.seed_value 188 | self.algorithm_steps=self.synth_tex+len(uv_info['edge_infos']*self.seamless_UVs) 189 | global synth_progress 190 | synth_progress=0.01 191 | context.scene.syntexmexsettings.synth_progress=0.01 192 | self._killthread = threading.Event() 193 | synthtex = functools.partial(us.synthesize_textures_on_uvs, 194 | synth_tex=self.synth_tex, 195 | seamless_UVs=self.seamless_UVs, 196 | msg_queue=self.msg_queue, 197 | stop_event=self._killthread, 198 | **uv_info) 199 | self._thread = threading.Thread(target = synthtex, 200 | daemon=True, 201 | args = [], 202 | kwargs = {}) 203 | self._thread.start() 204 | 205 | def write_images(self,target,ta_map=None): 206 | # Write back to blender image. 207 | self.target.pixels[:] = target.flatten() 208 | self.target.update() 209 | 210 | #import ipdb; ipdb.set_trace() # BREAKPOINT 211 | 212 | if ta_map is not None: 213 | ta_map = np.dstack((ta_map,np.ones(ta_map.shape[:2]))) 214 | self.ta_map.pixels[:] = ta_map.flatten() 215 | self.ta_map.update() 216 | 217 | logger.info("synced images!") 218 | 219 | def execute(self, context): 220 | def testworker(conn): 221 | for i in range(10): 222 | time.sleep(1.0) 223 | logger.info(f"working thread at {i}") 224 | conn.put(i) 225 | logger.info("working thread finished!") 226 | 227 | self.run_algorithm(context) 228 | 229 | # start timer to check thread status 230 | wm = context.window_manager 231 | self._timer = wm.event_timer_add(0.5, window=context.window) 232 | wm.modal_handler_add(self) 233 | self._region = context.region 234 | self._area = context.area 235 | return {'RUNNING_MODAL'} 236 | 237 | def modal(self, context, event): 238 | #TODO: https://stackoverflow.com/questions/21409683/passing-numpy-arrays-through-multiprocessing-queue for passing images from the other (sub) process 239 | if event.type in {'ESC'}:#{'RIGHTMOUSE', 'ESC'}: 240 | self.cancel(context) 241 | logger.info("thread was cancelled!") 242 | return {'CANCELLED'} 243 | 244 | if event.type == 'TIMER': 245 | # change theme color, silly! 246 | if self._thread.is_alive(): 247 | self.receive(context) 248 | #print(".", end = '') 249 | else: 250 | logger.info("thread seems to have finished!") 251 | self.receive(context) 252 | self.ta_map.pack() 253 | self.target.pack() 254 | if self.replace_material: 255 | bpy.ops.texture.syntexmex_pbr_texture( 256 | synth_map=self.ta_map.name, 257 | source_material=context.scene.syntexmexsettings.source_material.name 258 | ) 259 | 260 | self.cancel(context) 261 | return {'FINISHED'} 262 | 263 | return {'PASS_THROUGH'} 264 | 265 | def receive(self, context): 266 | try: 267 | msg = self.msg_queue.get_nowait() 268 | except queue.Empty: 269 | msg = None 270 | else: 271 | self.msg_queue.task_done() 272 | logger.info(f"received a new msg!!") 273 | if msg is not None: 274 | target,ta_map = msg 275 | self.write_images(target,ta_map) 276 | global synth_progress 277 | synth_progress += 1.0/self.algorithm_steps 278 | context.scene.syntexmexsettings.synth_progress += 1.0/self.algorithm_steps 279 | self._region.tag_redraw() 280 | self._area.tag_redraw() 281 | return msg 282 | 283 | def cancel(self, context): 284 | logger.info("cleaning up timer!") 285 | self._killthread.set() 286 | global synth_progress 287 | synth_progress = 0.0 288 | context.scene.syntexmexsettings.synth_progress=0.0 289 | self._region.tag_redraw() 290 | self._area.tag_redraw() 291 | wm = context.window_manager 292 | wm.event_timer_remove(self._timer) 293 | 294 | class clear_target_texture(bpy.types.Operator): 295 | """Clear target texture to make it black""" 296 | bl_idname = "texture.clear_target_texture" 297 | bl_label = "Clear target texture (black)" 298 | bl_options = {'REGISTER', 'UNDO'} 299 | 300 | def execute(self, context): 301 | logger.info("clear texture") 302 | ta = context.scene.syntexmexsettings.target_image 303 | 304 | target = np.zeros((ta.size[1], ta.size[0],4)) 305 | target[...,3]=1.0 306 | 307 | ta.pixels[:] = target.flatten() 308 | ta.update() 309 | ta.pack() 310 | 311 | return {'FINISHED'} 312 | 313 | def calc_syntexmex_info(ex_img, props): 314 | res_ex = np.array((ex_img.size[1],ex_img.size[0])) * props.example_scaling 315 | if props.target_image: 316 | ta_img = props.target_image 317 | res_ta = np.array((ta_img.size[1],ta_img.size[0])) 318 | else: 319 | res_ta = np.array((props.target_resolution[1], 320 | props.target_resolution[0])) 321 | scaling = us.ts.calc_lib_scaling(res_ex, props.libsize) 322 | (res_patch, res_patch0, 323 | overlap, overlap0) = us.ts.create_patch_params2(res_ex, 324 | scaling, 325 | 1/6.0, props.patch_size) 326 | #mem_reqs = tu.ts.check_memory_requirements2(res_ex, 327 | # res_patch, ch_num, ) 328 | mem_reqs = 2*us.ts.calculate_memory_consumption( 329 | res_ex*scaling, 330 | res_patch, 331 | ch_num = ex_img.channels, 332 | itemsize = 8) #itemsize= 8 comes from the floatingpoint size of 8 bytes in numpy arrays 333 | 334 | maxmem=us.ts.get_mem_limit() 335 | 336 | return (res_ex,res_ta,scaling,res_patch, res_patch0, 337 | overlap, overlap0, mem_reqs, maxmem) 338 | 339 | #TODO: ame everything according to here: 340 | #https://wiki.blender.org/wiki/Reference/Release_Na.otes/2.80/Python_API/Addons 341 | class syntexmex_panel(bpy.types.Panel): 342 | bl_label = "Syntexmex Configuration" 343 | bl_idname = "SYNTEXMEX_PT_syntexmex_panel" 344 | bl_space_type = 'VIEW_3D' 345 | bl_region_type = 'UI' 346 | bl_category = 'syntexmex' 347 | #bl_context = "tool" 348 | 349 | def copy_to_operator(self, op, settings): 350 | op.example_image = settings.example_image.name 351 | if settings.target_image: 352 | op.target_image = settings.target_image.name 353 | op.example_scaling = settings.example_scaling 354 | op.libsize = settings.libsize 355 | op.patch_size = settings.patch_size 356 | op.synth_tex=True 357 | op.seamless_UVs=True 358 | op.seed_value=settings.seed_value 359 | op.replace_material=settings.replace_material 360 | 361 | def draw(self, context): 362 | global synth_progress 363 | scene = context.scene 364 | props = scene.syntexmexsettings 365 | 366 | layout = self.layout 367 | layout.use_property_split = True 368 | 369 | 370 | ex_img = props.example_image 371 | 372 | #calculate algorithm properties and display some help how to set 373 | #it up correctly 374 | col2 = layout.column() 375 | if ex_img: 376 | (res_ex,res_ta,scaling, 377 | res_patch, res_patch0, 378 | overlap, overlap0, 379 | mem_reqs, maxmem) = calc_syntexmex_info(ex_img, props) 380 | if synth_progress > 0.0001: 381 | showtext=f"synthesize texture...\npress 'ESC' to cancel." 382 | elif mem_reqs > maxmem: 383 | showtext=f"algorithm uses too much memory: \n~{mem_reqs:.2f} GB\nmore than the available {maxmem:.2f} GB.\nMaybe close down other \nprograms to free up memory?" 384 | canrun=False 385 | col2.alert=True 386 | elif np.any(np.array(res_patch)<2): 387 | showtext=f"algorithm needs patch\nsizes >2 to run!" 388 | col2.alert=True 389 | canrun=False 390 | elif np.any(np.array(overlap)<1): 391 | showtext=f"algorithm needs overlap\nsizes > 1" 392 | col2.alert=True 393 | canrun=False 394 | else: 395 | showtext="algorithm is ready..." 396 | canrun=True 397 | elif not ex_img: 398 | showtext="choose an example img.!" 399 | col2.alert=True 400 | elif synth_progress > 0.0001: 401 | showtext="press 'ESC' to stop texture synthesis" 402 | 403 | #display helptext 404 | col2.scale_y = 0.7 405 | multiline_label(col2,"> "+showtext) 406 | 407 | layout.separator(factor=2.0) 408 | if synth_progress < 0.0001: #if algorithm isnt running 409 | ###### algorithm start buttons for different run modes 410 | col3 = layout.column() 411 | if ex_img and canrun: col3.enabled=True 412 | else: col3.enabled=False 413 | col2 = col3.column() 414 | col2.scale_y = 2.0 415 | op1 = col2.operator("texture.syntexmex", 416 | text = "Synthesize UV example based texture") 417 | 418 | #col.scale_y = 1.0 419 | col3.separator(factor=2.0) 420 | op2 = col3.operator("texture.syntexmex", 421 | text = "Synthesize textures to UV islands") 422 | 423 | op3 = col3.operator("texture.syntexmex", 424 | text = "Make UV seams seamless") 425 | 426 | if ex_img and canrun: 427 | self.copy_to_operator(op1,props) 428 | self.copy_to_operator(op2,props) 429 | op2.synth_tex=True 430 | op2.seamless_UVs=False 431 | self.copy_to_operator(op3,props) 432 | op3.synth_tex=False 433 | op3.seamless_UVs=True 434 | 435 | 436 | #####algorithm properties 437 | layout.separator(factor=2.0) 438 | layout.prop(props, "replace_material") 439 | #for prop in scene.syntexmexsettings.__annotations__.keys(): 440 | #layout.label(text="Example Material") 441 | layout.prop_search(props,"source_material", bpy.data, "materials", 442 | text="Source Mat.") 443 | if props.source_material: 444 | layout.prop(props, "material_textures",text="Source Tex.") 445 | else: 446 | layout.prop_search(props, "example_image", bpy.data, "images", 447 | text="Source Tex.") 448 | #layout.prop_search(props,"example_image", props, "material_textures", 449 | # text="Source Tex.") 450 | layout.prop(props, "example_scaling", text='Ex. Scaling') 451 | layout.prop(props, "libsize") 452 | #TODO: make patch_size dependend on target_texture 453 | layout.prop(props, "patch_size") 454 | layout.prop(props, "target_resolution") 455 | layout.prop(props, "seed_value") 456 | 457 | 458 | class syntexmex_info_panel(bpy.types.Panel): 459 | bl_label = "Info Panel" 460 | bl_idname = "SYNTEXMEX_PT_syntexmex_info_panel" 461 | bl_space_type = 'VIEW_3D' 462 | bl_region_type = 'UI' 463 | bl_category = 'syntexmex' 464 | #bl_context = "tool" 465 | #bl_options = {'DEFAULT_CLOSED'} 466 | 467 | def draw(self, context): 468 | layout = self.layout 469 | scene = context.scene 470 | props = scene.syntexmexsettings 471 | 472 | ex_img = props.example_image 473 | 474 | if ex_img: 475 | (res_ex,res_ta,scaling, 476 | res_patch, res_patch0, 477 | overlap, overlap0, 478 | mem_reqs, maxmem) = calc_syntexmex_info(ex_img, props) 479 | 480 | #layout.box() 481 | layout.prop(props, "synth_progress") 482 | layout.enabled=False 483 | 484 | layout.label(text="Algorithm Data:") 485 | b = layout.box() 486 | b.scale_y = 0.3 487 | 488 | multiline_label(b,f"""memory requirements: {mem_reqs:.2f}/{maxmem:.2f} GB 489 | source scaling: {props.example_scaling*100:.0f}% 490 | source resolution: [{res_ex[1]:.0f} {res_ex[0]:.0f}] px 491 | target resolution: {res_ta[::-1]} px 492 | synth resolution scaling: {scaling:.2f} 493 | patches (highres): {res_patch0[::-1]} px 494 | patches (lowres): {res_patch[::-1]} px 495 | overlap (highres): {overlap0[::-1]} 496 | overlap (lowres): {overlap[::-1]}""") 497 | 498 | class syntexmex_advanced_panel(bpy.types.Panel): 499 | bl_label = "Advanced Settings" 500 | bl_idname = "SYNTEXMEX_PT_syntexmex_advanced_panel" 501 | bl_space_type = 'VIEW_3D' 502 | bl_region_type = 'UI' 503 | bl_category = 'syntexmex' 504 | #bl_context = "tool" 505 | bl_options = {'DEFAULT_CLOSED'} 506 | 507 | def draw(self, context): 508 | layout = self.layout 509 | col = layout.column() 510 | 511 | scene = context.scene 512 | props = scene.syntexmexsettings 513 | 514 | col.label(text="debugging options:") 515 | col.prop(props,"advanced_debugging", 516 | text="enable advanced console logs") 517 | 518 | 519 | def set_debugging(on=True): 520 | logging.basicConfig(level=logging.INFO) 521 | if on: 522 | logging.disable(logging.NOTSET) 523 | else: 524 | logging.disable(logging.INFO) 525 | 526 | 527 | def update_debugging(self, context): 528 | scene = context.scene 529 | props = scene.syntexmexsettings 530 | logger.info(f"Set advanced infos to: {props.advanced_debugging}") 531 | set_debugging(props.advanced_debugging) 532 | return None 533 | 534 | 535 | class synth_PBR_texture(bpy.types.Operator): 536 | """This operator synthesizes PBR textures from a synthmap 537 | (an image in which the pixel values are coordinates to an example image)""" 538 | bl_idname = "texture.syntexmex_pbr_texture" 539 | bl_label = "Synthesize PBR textures using a synth-map" 540 | bl_category = 'syntexmex' 541 | bl_options = {'REGISTER', 'UNDO'} 542 | 543 | synth_map: bpy.props.StringProperty() 544 | source_material : bpy.props.StringProperty() 545 | source_image : bpy.props.StringProperty() 546 | 547 | def execute(self, context): 548 | logger.info("start PBR synthesis") 549 | 550 | logger.info(f"synthesize textures: {self.synth_map},"\ 551 | f"{self.source_material},{self.source_image}") 552 | 553 | synthmap = up.blimage2array(bpy.data.images[self.synth_map])[...,:3] 554 | mask = np.all(synthmap[:,:,:2]>=0,axis=2) 555 | 556 | #scn = context.scene 557 | obj = context.active_object 558 | 559 | if self.source_material is not None: 560 | logger.info("create new seamless material") 561 | obj.active_material = mat = bpy.data.materials[self.source_material].copy() 562 | 563 | #duplicate texture and duplicate images inside 564 | #TODO: make "original" texture permanently saved 565 | imgnodes = [n for n in mat.node_tree.nodes if n.type=='TEX_IMAGE'] 566 | images = [up.blimage2array(n.image)[...,:3] for n in imgnodes] 567 | synth_images = [us.reconstruct_synthmap(synthmap, 568 | img, 569 | mode='normalized') 570 | for img in images] 571 | 572 | #import ipdb; ipdb.set_trace() # BREAKPOINT 573 | if not np.all(mask): #only do this if we need a masked copy 574 | logger.info("copying seams to original textures") 575 | scaled_images = [us.skimage.transform.resize( 576 | img, synthmap.shape[:2]) 577 | for img in images] 578 | synth_images = [us.ts.copy_img(img,simg,mask=mask) 579 | for img, simg in zip(scaled_images,synth_images)] 580 | 581 | 582 | 583 | #functionality not implemented right now 584 | #elif self.source_image is not None: 585 | # images = [up.blimage2array(self.source_image)[...,:3]] 586 | 587 | for simg,node in zip(synth_images,imgnodes): 588 | logger.info(f"processing image: {node.image.name}") 589 | if node.image.colorspace_settings.name=='Non-Color': is_data=True 590 | else: is_data=False 591 | new_img = bpy.data.images.new(node.image.name+"_synth", 592 | synthmap.shape[1],synthmap.shape[0], 593 | alpha=False,float_buffer=node.image.is_float, 594 | is_data=is_data) 595 | 596 | #add alpha channel and upload into blender 597 | new_img.pixels[:] = np.dstack((simg,np.ones(simg.shape[:2]))).flatten() 598 | 599 | #new_img.colorspace_settings.name = node.image.colorspace_settings.name 600 | node.image = new_img 601 | new_img.update() 602 | new_img.pack() 603 | 604 | 605 | #us.reconstruct_synthmap(self.) 606 | return {'FINISHED'} 607 | 608 | 609 | 610 | #TODO: if we want a list o something: 611 | #https://sinestesia.co/blog/tutorials/using-uilists-in-blender/ 612 | class syntexmex_texture_operations_panel(bpy.types.Panel): 613 | bl_label = "texture operations" 614 | bl_idname = "SYNTEXMEX_PT_syntexmex_texture_operations_panel" 615 | bl_space_type = 'VIEW_3D' 616 | bl_region_type = 'UI' 617 | bl_category = 'syntexmex' 618 | #bl_context = "tool" 619 | bl_options = {'DEFAULT_CLOSED'} 620 | 621 | def draw(self, context): 622 | layout = self.layout 623 | layout.use_property_decorate = False 624 | ob = context.active_object 625 | 626 | scene = context.scene 627 | props = scene.syntexmexsettings 628 | op = layout.operator("texture.syntexmex_pbr_texture", 629 | text = "Synthesize UV example based texture") 630 | 631 | ex_img = props.example_image 632 | ta_img = props.target_image 633 | img_init=(ex_img is not None) and (ta_img is not None) 634 | 635 | if props.active_synthmap: 636 | op.synth_map = props.active_synthmap.name 637 | if props.source_material: 638 | op.source_material = props.source_material.name 639 | if props.source_image: 640 | op.source_image = props.source_image.name 641 | 642 | layout.use_property_split = True 643 | layout.label(text="active synthmap:") 644 | layout.template_ID(props,"active_synthmap", open="image.open") 645 | layout.prop_search(props,"source_material", bpy.data, "materials") 646 | layout.label(text="source image:") 647 | layout.template_ID(props,"source_image", open="image.open") 648 | 649 | 650 | # if prop in images: 651 | layout.label(text="Example Image:") 652 | layout.template_ID(props, "example_image", open="image.open") 653 | layout.prop(props, "example_scaling") 654 | layout.prop(props, "libsize") 655 | #TODO: make patch_size dependend on target_texture 656 | layout.prop(props, "patch_size") 657 | 658 | 659 | layout.separator(factor=2.0) 660 | layout.label(text="Target Image:") 661 | layout.template_ID(props, "target_image", 662 | new="image.new",open="image.open") 663 | if (ta_img is not None): 664 | layout.operator("texture.clear_target_texture") 665 | 666 | def get_textures_from_material(self, context): 667 | mat = self.source_material 668 | texlist = [] 669 | if mat is not None: 670 | for i,n in enumerate(mat.node_tree.nodes): 671 | if n.type=="TEX_IMAGE": 672 | if n.image: 673 | texlist.append((n.image.name,n.image.name,"",i)) 674 | #print(f"updating images!{n.image.name},{i}") 675 | return texlist 676 | #images = [n.image for n in mat.node_tree.nodes if n.type=='TEX_IMAGE'] 677 | #return None#images 678 | 679 | def update_source_image_from_enum(self, context): 680 | props = context.scene.syntexmexsettings 681 | sel_tex = props.material_textures 682 | props.example_image = bpy.data.images[sel_tex] 683 | 684 | class SyntexmexSettings(bpy.types.PropertyGroup): 685 | #https://docs.blender.org/api/current/bpy.props.html 686 | synth_progress: bpy.props.FloatProperty( 687 | name="synthetization progress", 688 | description="synthetization progress", 689 | min=0.0,max=1.0, 690 | default=0.0, 691 | subtype = "PERCENTAGE", 692 | options = {'SKIP_SAVE'} 693 | ) 694 | patch_size: bpy.props.FloatProperty( 695 | name="Patch Size Ratio", 696 | description="Set width of patches as a ratio of shortest edge of an image", 697 | min=0.0001, 698 | max=0.5, 699 | default=0.1, 700 | precision=3, 701 | step=0.1, 702 | ) 703 | example_image: bpy.props.PointerProperty(name="Ex.Tex.", type=bpy.types.Image) 704 | example_scaling: bpy.props.FloatProperty( 705 | name="Example Scaling", 706 | description="""Scale Example to a certain size which will be used 707 | to generate the texture""", 708 | min=0.0001, 709 | max=1.0, 710 | default=1.0, 711 | precision=3, 712 | step=1 713 | ) 714 | target_image: bpy.props.PointerProperty(name="target tex", type=bpy.types.Image) 715 | target_resolution: bpy.props.IntVectorProperty(name="synth-res.", 716 | size=2, 717 | min=0, 718 | max=10000, 719 | default=(1024,1024)) 720 | libsize: bpy.props.IntProperty(name="Library Size", 721 | description="defines the quality of the texture (higher=better, but needs more memory)", 722 | min=10*10, default = 128*128, 723 | step=10#TODO: currently not implemented in blender 724 | ) 725 | seed_value: bpy.props.IntProperty( 726 | name="Seed Value", 727 | description="Seed value for predictable texture generation", 728 | default = 0, 729 | ) 730 | source_material : bpy.props.PointerProperty(name="source material", 731 | type=bpy.types.Material) 732 | #material_textures : bpy.props.CollectionProperty(type=TextureList) 733 | #material_textures : bpy.props.CollectionProperty(type=bpy.types.Image) 734 | material_textures : bpy.props.EnumProperty(items = get_textures_from_material, 735 | update = update_source_image_from_enum) 736 | source_image : bpy.props.PointerProperty(name="source image", 737 | type=bpy.types.Image) 738 | active_synthmap : bpy.props.PointerProperty(name="active synthmap", 739 | type=bpy.types.Image) 740 | advanced_debugging: bpy.props.BoolProperty( 741 | name="Advanced Debugging", 742 | description="Enable Advanced Debugging (in console)", 743 | default = True, 744 | update = update_debugging 745 | ) 746 | replace_material : bpy.props.BoolProperty( 747 | name="replace material", 748 | description=("replace all textures in material with" 749 | "a seamless version"), 750 | default=True 751 | ) 752 | 753 | 754 | classes = ( 755 | syntexmex_info_panel, 756 | syntexmex_panel, 757 | #TODO: enable this again: syntexmex_texture_operations_panel, 758 | syntexmex_advanced_panel, 759 | syntexmex, 760 | synth_PBR_texture, 761 | clear_target_texture 762 | ) 763 | register_panel, unregister_panel = bpy.utils.register_classes_factory(classes) 764 | 765 | @bpy.app.handlers.persistent 766 | def on_load(dummy): 767 | print("initializing syntexmex!") 768 | if bpy.data.scenes[0].syntexmexsettings.advanced_debugging: 769 | print("turn on advanced debugging!") 770 | set_debugging(True) 771 | else: 772 | print("turn off advanced debugging!") 773 | set_debugging(False) 774 | 775 | def register(): 776 | #syntexmex_panel.register() 777 | bpy.utils.register_class(SyntexmexSettings) 778 | bpy.types.Scene.syntexmexsettings = bpy.props.PointerProperty(type=SyntexmexSettings) 779 | register_panel() 780 | global synth_progress 781 | synth_progress = 0.0 782 | #bpy.app.timers.register(in_5_seconds, first_interval=5) 783 | bpy.app.handlers.load_post.append(on_load) 784 | #TODO: initialize/deiniailize debuggin according to how the variable was set 785 | #init propertygroup 786 | 787 | def unregister(): 788 | #syntexmex_panel.unregister() 789 | unregister_panel() 790 | bpy.utils.unregister_class(SyntexmexSettings) 791 | del(bpy.types.Scene.syntexmexsettings) 792 | #del(bpy.context.scene['syntexmexsettings']) 793 | 794 | if __name__ == "__main__": 795 | logger.info("register syntexmex") 796 | # pipe_operator.register() 797 | # bpy.ops.mesh.add_pipes() 798 | register() 799 | # debugging 800 | #bpy.ops.mesh.add_pipes(number=5, mode='skin', seed = 11) 801 | 802 | #obj = bpy.context.selected_objects[0] 803 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | piperator 635 | Copyright (C) 2019 Thomas Meschede 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | piperator Copyright (C) 2019 Thomas Meschede 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /tex_synthesize.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Sun Jan 5 13:39:03 2020 5 | 6 | ################################################################################ 7 | Copyright (C) 2020 Thomas Meschede a.k.a. yeus 8 | 9 | This program is free software: you can redistribute it and/or modify 10 | it under the terms of the GNU General Public License as published by 11 | the Free Software Foundation, either version 3 of the License, or 12 | (at your option) any later version. 13 | 14 | This program is distributed in the hope that it will be useful, 15 | but WITHOUT ANY WARRANTY; without even the implied warranty of 16 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 17 | GNU General Public License for more details. 18 | 19 | You should have received a copy of the GNU General Public License 20 | along with this program. If not, see . 21 | ################################################################################ 22 | 23 | @author: Thomas Meschede a.k.a. yeus (yeusblender@gmail.com) 24 | 25 | This texture synthesis algorithm takes inspiration from three papers and 26 | combines their papers into a new algorithm: 27 | 28 | - Image Quilting for Texture Synthesis and Transfer [Efros, Freeman] 29 | - taking the optimal-patch seam strategy and 30 | - Fast Texture Synthesis using Tree-structured Vector Quantization [Wei, Levoy] 31 | - iterations, non-causal buildup local neighbourhood search 32 | - Real-Time Texture Synthesis by Patch-Based Sampling [Liang et al.] 33 | - building a gaussian image pyramid combined with KD-Trees for 34 | fast searches 35 | """ 36 | 37 | import random 38 | import numpy as np 39 | import skimage 40 | import skimage.io 41 | import skimage.transform 42 | import itertools 43 | from functools import wraps 44 | import time 45 | #from pynndescent import NNDescent 46 | #import gc 47 | import math 48 | import os, sys 49 | import functools 50 | sign = functools.partial(math.copysign, 1) # either of these 51 | #import scipy 52 | import shapely 53 | import shapely.geometry 54 | import logging 55 | #import psutil #TODO: not easy in blender, because of a lacking Python.h 56 | logger = logging.getLogger(__name__) 57 | 58 | #ann_library = "pynndescent" 59 | ann_library = "sklearn" 60 | use_pynnd, use_sklearn=False,False 61 | if ann_library=="pynndescent": 62 | import pynndescent as pynnd 63 | use_pynnd=True 64 | elif ann_library=='sklearn': 65 | import sklearn 66 | import sklearn.neighbors 67 | use_sklearn=True 68 | 69 | #def norm(x): return np.sqrt(x.dot(x)) 70 | def norm(x): return np.sqrt((x*x).sum(-1)) 71 | #need to be transposed for correct ultiplcation along axis 1 72 | def normalized(x): return (x.T /norm(x)).T 73 | 74 | def calc_angle_vec(u, v): 75 | """ 76 | >>> u = vec((1.0,1.0,0.0)) 77 | >>> v = vec((1.0,0.0,0.0)) 78 | >>> calc_angle_vec(u,v)*rad 79 | 45.00000000000001 80 | >>> u = vec((1.0,0.0,0.0)) 81 | >>> v = vec((-1.0,0.0,0.0)) 82 | >>> calc_angle_vec(u,v)*rad 83 | 180.0 84 | >>> u = vec([-9.38963669e-01, 3.44016319e-01, 1.38777878e-17]) 85 | >>> v = vec([-0.93896367, 0.34401632, 0.]) 86 | >>> u @ v / (norm(v)*norm(u)) 87 | 1.0000000000000002 88 | >>> calc_angle_vec(u,v)*rad 89 | 0.0 90 | """ 91 | #angle = np.arctan2(norm(np.cross(u,v)), np.dot(u,v)) 92 | res = np.sum(u*v) / (norm(u) * norm(v)) 93 | t = np.clip(res,-1.0,1.0) 94 | angle = np.arccos(t) 95 | return angle 96 | 97 | #consider replacing sklearn KDTree with scipy KDTree 98 | #https://jakevdp.github.io/blog/2013/04/29/benchmarking-nearest-neighbor-searches-in-python/ 99 | #from tqdm import tqdm 100 | def tqdm(iterator, *args, **kwargs): 101 | return iterator 102 | 103 | GB = 1.0/1024**3 #GB factor 104 | 105 | def timing(f): 106 | @wraps(f) 107 | def wrap(*args, **kw): 108 | ts = time.time() 109 | result = f(*args, **kw) 110 | te = time.time() 111 | logger.info(f'func:{f.__name__} took: {te-ts:2.4f} sec') 112 | return result 113 | return wrap 114 | 115 | @timing 116 | def init_ann_index(data): 117 | """ 118 | TODO: parameterize quality of ANN search 119 | """ 120 | #metric='euclidean' 121 | metric='manhattan' 122 | if use_pynnd: 123 | index = pynnd.NNDescent(data,metric, 124 | n_neighbors=3,#steers the quality of the algorithm 125 | n_jobs=-1 #-1: use all processors 126 | ) 127 | elif use_sklearn: 128 | index = sklearn.neighbors.KDTree(data, metric=metric)#'l2' 129 | return index 130 | 131 | def query_index(index, data, k): 132 | if use_pynnd: 133 | idx, e = index.query([data], k=k) 134 | return e,idx[0] 135 | elif sklearn: 136 | e, idx = index.query([data], k=k) 137 | e, idx = e.flatten(), idx.flatten() 138 | return e,idx 139 | 140 | def get_mem_limit(factor=0.5): 141 | #stats = psutil.virtual_memory() # returns a named tuple 142 | #avilable_memory = getattr(stats, 'available')/1024**3 # available memory in GB 143 | #mem_limit = avilable_memory*factor 144 | #tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:]) 145 | if sys.platform=='linux': 146 | free_m = int(os.popen('free -t -m').readlines()[1].split()[-1]) 147 | else: #TODO: include windows/mac 148 | free_m = 2000 #MB 149 | return free_m * factor / 1024 #in GB 150 | 151 | def copy_img(target, src, pos=(0,0), mask=None): 152 | """ 153 | copy image src to target at pos 154 | 155 | careful! x & y are switched around here (normal order) in contrast to other 156 | functions of this library. order: pos=(x,y) 157 | """ 158 | #TODO: handle border clipping problems 159 | # - when copying images that exten over "left" and "top" edges 160 | sh,sw,sch = src.shape 161 | th,tw,tch = target.shape 162 | 163 | i0x = np.clip(pos[0],0,tw) 164 | i0y = np.clip(pos[1],0,th) 165 | i1x = np.clip(pos[0]+sw,0,tw) 166 | i1y = np.clip(pos[1]+sh,0,th) 167 | 168 | #cut source patch to right size 169 | pw, ph = max(i1x - i0x,0), max(i1y - i0y,0) 170 | 171 | if mask is None: 172 | tch = sch 173 | #print(pos) 174 | #import ipdb; ipdb.set_trace() # BREAKPOINT 175 | target[i0y:i1y, i0x:i1x, 0:tch] = src[0:ph, 0:pw] 176 | else: 177 | m = mask 178 | target[i0y:i1y, i0x:i1x, 0:tch][m] = src[m] 179 | 180 | return target 181 | 182 | def mask_blend(m, img1, img2): 183 | """ 184 | this blends two images by blending them using a mask 185 | """ 186 | mT = np.expand_dims(m,axis=2) 187 | b1 = img1 * (1 - mT) 188 | b2 = img2 * mT 189 | new_img = b1+b2 190 | return new_img 191 | 192 | @timing 193 | def transform_patch_grid_to_tex(res, res_patch, pg, example, 194 | overlap, 195 | use_quilting=True): 196 | """ 197 | synthesize texture from pre-calculated grid 198 | 199 | overlap = (horizontal_ovelap, vertical_overlap) 200 | """ 201 | #TODO: create a generate- "info" function so that information doesn't have 202 | # to be exchanged so often 203 | #TODO: adaptive overlap (wih larger patch sizes it might not 204 | #make enough sense anymore, because textures become too repetetive ...) 205 | #overlap = np.ceil((res_patch/6)).astype(int) 206 | #TODO: the overlap can actually be different from the overlap when searching 207 | # for the images. This might make some things more efficient 208 | # or we can just generate a standad overlap in all functions 209 | 210 | ch_num = example.shape[-1] 211 | 212 | #def draw_patch_grid(): 213 | rpg = np.array(res_patch) - overlap 214 | if res is None: res = rpg * pg.shape[:2] + overlap 215 | target = np.zeros((res[0],res[1],ch_num), dtype = example.dtype) 216 | ta_map = np.zeros((res[0],res[1],3)) 217 | 218 | target_map_patch_base = gen_coordinate_map(res_patch) 219 | onemask = np.ones(res_patch) 220 | for iy,ix in np.ndindex(pg.shape[:2]): 221 | #if (ix, iy) == (3,2): break 222 | x = ix * rpg[1] 223 | y = iy * rpg[0] 224 | y0,x0 = pa_coords = pg[iy,ix] #get coords of patch in example texture 225 | 226 | #TODO: searching new patches based on the already existing image also 227 | # helps when having all sorts of constraints 228 | #TODO: maybe pass a "constraint-error-function" for the search function? 229 | #TODO: get rid of resolution parameters to make dynamic sized patches 230 | # possible --> this doesn#t work with a "grid" function 231 | if all(pa_coords == (-1,-1)): # --> a part of the grid that is not assigned 232 | pa = np.zeros((res_patch[0],res_patch[1],4)) 233 | pa[:,:,:]=(0, 0.31, 0.22, 1) #TODO: make different fill colors possible 234 | 235 | #get corresponding overlaps: 236 | if iy==0: ovs=(overlap[1],0,0,0) #first row 237 | elif ix==0: ovs=(0,overlap[0],0,0) #first column 238 | else: ovs=(overlap[1],overlap[0],0,0) #rest of the grid 239 | 240 | if (iy==0 and ix==0) or (not use_quilting): 241 | pa = example[y0:y0+res_patch[0],x0:x0+res_patch[1]].copy() 242 | mask = onemask 243 | else: 244 | ta_patch = target[y:y+res_patch[0],x:x+res_patch[1]] 245 | pa, _, _, mask = optimal_patch(ta_patch, example, 246 | res_patch, ovs, (y0,x0), (y,x)) 247 | 248 | #skimage.io.imshow_collection([pa, ov_h[0], b_h, ov_v[0], b_v]) 249 | copy_img(target, pa, (x,y)) 250 | #print((ix,iy),pg[iy,ix]) 251 | 252 | ta_map_patch = target_map_patch_base + (x0,y0,0) 253 | #TODO: find a better method how to use "partial" coordinate transfer 254 | #or in other words: "mixes" which appear for example at smoothed 255 | #out and blended optimal borders 256 | copy_img(ta_map, ta_map_patch, (x,y), mask=mask>0) 257 | #copy_img(ta_map, ta_map_patch, (x,y)) 258 | 259 | 260 | return target, ta_map 261 | 262 | def overlap_slices(overlap): 263 | """define slices for overlaps""" 264 | ovs = [np.s_[:,:overlap[0]],#left 265 | np.s_[:overlap[1],:],#top 266 | np.s_[:,-overlap[2]:],#right 267 | np.s_[-overlap[3]:,:]]#bottom 268 | return ovs 269 | 270 | def create_optimal_patch(pa, ta, overlaps): 271 | """ 272 | this function creates an optimal patch out of 1 tile with 273 | given number of border tiles where two overlapping regions 274 | are replaced by an optimal boundary from the other patches 275 | """ 276 | mask = np.ones(pa.shape[:2]) 277 | ovs = overlap_slices(overlaps) 278 | for ov, orient, sl, mirrored in zip(overlaps,["h","v","h","v"], ovs, 279 | [False,False,True,True]): 280 | if ov>0: #if this boundary is used 281 | m = minimum_error_boundary_cut((pa[sl],ta[sl]), orient) 282 | if mirrored: m = 1-m 283 | mask[sl] = np.minimum(m,mask[sl]) 284 | 285 | new_pa = mask_blend(mask, ta, pa) 286 | #TODO: only return mask 287 | return new_pa, mask 288 | 289 | def optimal_patch(ta_patch, example, res_patch, overlap, pos_ex, pos_ta): 290 | """ 291 | this creates optimal patches for reacangular patches with overlaps 292 | given in "overlap" with indics as follows: 293 | 2 294 | ----- 295 | 1| |3 296 | ----- 297 | 4 298 | 299 | the number of in "overlap" specifies the size of the overlap in pixels 300 | TODO: expand to arbitrarily sized boundaries 301 | TODO: make it possible to "move" the source patch a couple pixels 302 | to find a better fit 303 | """ 304 | #TODO use np.s_ as indices 305 | y,x = pos_ta 306 | y0,x0 = pos_ex 307 | pa = example[y0:y0+res_patch[0],x0:x0+res_patch[1]].copy() 308 | optimpatch, mask = create_optimal_patch(pa, ta_patch, overlap) 309 | return optimpatch, pa, ta_patch, mask 310 | 311 | def find_match(data, index, tol = 0.1, k=5): 312 | #get a horizonal overlap match 313 | e,idx = query_index(index,data,k) 314 | #e,idx = query_index(index,data,k) 315 | #TODO: keep an index with "weights" to make sure 316 | # that patches are selected equally oftern OR according 317 | # to a predefined distribution 318 | # TODO: make sure patches can get selected with an additional error-term 319 | # TODO: for very large textures it might make sense 320 | # to only query a close radius 321 | # TODO: implement theshold function 322 | #or choose ithin a certain tolerance 323 | if k>1: 324 | min_err = e[e>0].min() #find minimum error except for the exact matching pic 325 | #i2 = err.argmin() 326 | th = min_err * (1+tol) #threshold 327 | idx = [i for i,e in zip(idx,e) if e1: print(f"{len(idx)},",) 332 | 333 | @timing 334 | def synthesize_grid(example, res_patch, res_grid, overlap, tol = 0.1, k=5): 335 | """ 336 | synthesize a grid of patch indices 337 | by searching match pairs & triples in the 338 | overlaps database 339 | 340 | tolerance -> this basically controls the randomness of 341 | the algorithm 342 | """ 343 | 344 | res_patch = np.array(res_patch).astype(int) 345 | res_ex = example.shape[:2] 346 | #TODO: decrease overlap to mitigate memory_problems 347 | max_co = res_ex - res_patch 348 | ch = example.shape[-1] 349 | cellsize = example.itemsize 350 | #TODO: replace by a better KD Tree implementation or a Ball Tree 351 | cellsize = 8 #bytes (this is because of scikit-learns KDTree implementation 352 | # which will convert the supplied data into a float 353 | # representation) 354 | #ex_img_ratio = res_ex[0]/res_ex[1] 355 | 356 | #calculate maximum possible patch size for given number of patches and 357 | rp = res_patch 358 | pnum = max_co.prod() 359 | 360 | mem_limit = get_mem_limit() 361 | #check memory consumption for overlap database: 362 | #TODO: check memory consumption instructions from sklearn webpage 363 | #single_overlap_memory_horizontal = \ 364 | # overlap[0] * res_patch[1] * ch * cellsize #in byte 365 | #single_overlap_memory_vertical = \ 366 | # overlap[1] * res_patch[0] * ch * cellsize #in byte 367 | 368 | #factor 2 comes in here, because we have a 3rd database with the combined overlaps 369 | #totalmemoryGB = pnum * GB \ 370 | # * (2 * single_overlap_memory_vertical + 2 * single_overlap_memory_horizontal) 371 | #augmentation_multiplicator = 2 #(for mirroring, and rotating, the data gets 372 | #) multiplied 373 | data_memoryGB = check_memory_requirements(example,res_patch, maxmem=mem_limit, 374 | disable_safety_check=True) 375 | logger.info(f"using approx. {3*data_memoryGB:2f} GB in RAM.") 376 | 377 | #TODO: build my own search algorithm which doesn't consume as much memory 378 | # and can find things in an image much quicker 379 | #TODO: the "overlaps" are the same for left & right and top & bottom 380 | # patch generation can be completly left out as they can be 381 | # taken directly from the texture. For the search, only the 382 | # overlaps are important. 383 | #TODO: maybe save the coordinates of the overlaps in a second database 384 | #TODO: augmented patches(mirrored, ) 385 | #TODO: check memory consumption of patch overlaps 386 | #TODO: if memory consumption reaches a certain thresold, lower resolution 387 | # of overlap databases (this can be done in multiple ways: 388 | # - only take patches from every nth pixel, 389 | # - take overlaps with 1/k the resolution of the original in the database) 390 | # - use smaller patch sizes 391 | # - lower the reoslution of the original image but later keep the 392 | # original rsolution when stitching it back together 393 | # - take a random sample of patches from the original source image 394 | # TODO: add patch augmentation mirroring, rotaton 395 | # the problem here is: when augmenting horiz. or vert. 396 | # images, the "combined" would becom very large 397 | # basically squared. so if we mirror vert. overlaps, 398 | # only horizontally, we also have twice as many combined. 399 | # If we mirror vertically & horozontally we have 3 times as 400 | # many (all data). 401 | # If we also mirror horiz. overlaps in th same way we have 402 | # 3x3 = 9x as much data in combined and 3+3=6 times as much in 403 | # vert. or horiz. 404 | # we can find out whether something is mirrored, rotated or 405 | # whatever by analyzing the index module by the number of 406 | # augentations 407 | #lm = [] 408 | try: 409 | logger.info("init kdtree1") 410 | ld = create_patch_data(example, (rp[0], overlap[1]), max_co) 411 | l = init_ann_index(ld) 412 | logger.info("init kdtree2") 413 | td = create_patch_data(example, (overlap[0],rp[1]), max_co) 414 | t = init_ann_index(td) 415 | logger.info("init kdtree3") 416 | lt = init_ann_index(np.hstack((ld,td))) 417 | #TODO: check memory consumption of KDTrees 418 | #sklearn.neighbors.KDTree.valid_metrics 419 | #ov_db = [sklearn.neighbors.KDTree(i, metric='euclidean') for i in (ov_l, ov_t, ov_lt)] 420 | logger.info("KD-Tree initialization done") 421 | except MemoryError as err: 422 | logger.info(err) 423 | logger.info("example texture too large, algorithm needs " 424 | "too much RAM: {totalmemoryGB:.2f}GB") 425 | raise 426 | 427 | pg = np.full(shape=(*res_grid,2), fill_value=-1, dtype=int) 428 | pg[0,0] = idx2co(random.randrange(pnum), max_co) #get first patch 429 | #pg[0,0]=(0,0) 430 | 431 | #TODO: synthesize an arbitrary pic with the overlap database generated from example 432 | # but using a different source texture to "mix" two textures 433 | 434 | for i in tqdm(range(1,res_grid[1]),"find matches: first row"): 435 | y,x = pg[0,i-1] 436 | #patch = example[y:y+rp[1],x:x+rp[0]] 437 | #((patches[idx] - patch)**2).sum() == 0 #they have to be the same! 438 | ovl = example[y:y+rp[0],x:x+rp[1]][:,-overlap[0]:].flatten() 439 | #ov_idx = idx + (res_patch[0] - overlap[0])*max_co[1]#move towards the right by res_grid 440 | #(ovl-ov_l[ov_idx]) 441 | new_idx = find_match(ovl, l , tol=tol, k=k) 442 | #if new_idx 443 | pg[0,i] = idx2co(new_idx, max_co) 444 | 445 | for i in tqdm(range(1,res_grid[0]),"find matches: first column"): 446 | y,x = pg[i-1,0] 447 | ovt = example[y:y+rp[1],x:x+rp[0]][-overlap[1]:,:].flatten() 448 | pg[i,0] = idx2co(find_match(ovt, t, tol=tol, k=k), max_co) 449 | 450 | for ix,iy in tqdm(itertools.product(range(1,res_grid[1]), 451 | range(1,res_grid[0])), 452 | total = (res_grid[1]-1)*(res_grid[0]-1), 453 | desc = "find matches: complete grid"): 454 | y,x = pg[iy,ix-1] 455 | ovl = example[y:y+rp[0],x:x+rp[1]][:,-overlap[1]:].flatten() 456 | y,x = pg[iy-1,ix] 457 | ovt = example[y:y+rp[0],x:x+rp[1]][-overlap[0]:,:].flatten() 458 | ovlt = np.hstack((ovl, ovt)) 459 | pg[iy,ix] = idx2co(find_match(ovlt, lt, tol = tol, k=k), max_co) 460 | 461 | return pg 462 | 463 | def minimum_error_boundary_cut(overlaps, direction): 464 | """ 465 | create an optimal boundary cut from 466 | an error matrix calculated from overlaps 467 | """ 468 | #TODO: create minimum boundary for very small overlaps (for example just 1 pixel) 469 | ol1, ol2 = overlaps 470 | #calculate error and convert to grayscale 471 | err = ((ol1 - ol2)**2).mean(2) 472 | 473 | if direction == "v": err = err.T 474 | minIndex = [] 475 | E = [list(err[0])] 476 | for i in range(1, err.shape[0]): 477 | # Get min values and args, -1 = left, 0 = middle, 1 = right 478 | e = [np.inf] + E[-1] + [np.inf] 479 | e = np.array([e[:-2], e[1:-1], e[2:]]) 480 | # Get minIndex 481 | minArr = e.min(0) 482 | minArg = e.argmin(0) - 1 483 | minIndex.append(minArg) 484 | # Set Eij = e_ij + min_ 485 | Eij = err[i] + minArr 486 | E.append(list(Eij)) 487 | 488 | # Check the last element and backtrack to find path 489 | path = [] 490 | minArg = np.argmin(E[-1]) 491 | path.append(minArg) 492 | 493 | # Backtrack to min path 494 | for idx in minIndex[::-1]: 495 | minArg = minArg + idx[minArg] 496 | path.append(minArg) 497 | 498 | # Reverse to find full path 499 | path = path[::-1] 500 | m = np.zeros(err.shape, dtype=float)#define mask 501 | for i,pi in enumerate(path): 502 | #p1[i,-overlap[0] + pi,0]*=0.5 503 | m[i,pi:]=True 504 | m[i,pi]=0.5 #set a "smooth" boundary 505 | #err[i,pi]+=0.05 506 | 507 | if direction=="v": return m.T 508 | else: return m 509 | 510 | def create_patch_data(example, res_patch, max_co=None): 511 | """max_co is needed in the case where only overlap 512 | areas of patches are of interest. In this case we want 513 | the overlap areas to not extend beyond the area of 514 | a contiguos patch 515 | """ 516 | if max_co is None: max_co = np.array(example.shape[:2]) - res_patch 517 | rp = res_patch 518 | data = np.ascontiguousarray([example[y:y+rp[0],x:x+rp[1]].flatten() 519 | for y,x in tqdm(np.ndindex(*max_co), "create_patch_data")]) 520 | return data 521 | 522 | def idx2co(idx, max_co): 523 | yp = int(idx/max_co[1]) 524 | xp = idx - yp * max_co[1] 525 | return yp,xp 526 | 527 | #create patches 528 | def gen_patches(image, res_patch): 529 | res_img = np.array(image.shape[:2]) 530 | max_co = res_img - res_patch 531 | patches = np.array([image[y:y+res_patch[0],x:x+res_patch[1]] 532 | for y,x in np.ndindex(*max_co)]) 533 | return patches 534 | 535 | #create patches 536 | def gen_patches_from_mask(image, mask): 537 | res_img = np.array(image.shape[:2]) 538 | m_res = mask.shape[:2] 539 | max_co = res_img - m_res + (1,1) 540 | patches = np.array([image[y:y+m_res[0],x:x+m_res[1]][mask].flatten() 541 | for y,x in np.ndindex(*max_co)]) 542 | def idx2co(idx): 543 | yp = int(idx/max_co[1]) 544 | xp = idx - yp * max_co[1] 545 | return xp,yp 546 | 547 | return patches, max_co, idx2co 548 | 549 | #TODO: checkout the quality of the results for small image resolutions. 550 | # - it might make sense toreduce the resolution for images based 551 | # on certain criteria (for example patch size) anyways 552 | def build_gaussian_pyramid(example0, min_res=8): 553 | #build a gaussian image pyramid 554 | py = [im for im in skimage.transform.pyramid_gaussian(example0, multichannel=True)] 555 | #filter for minimum resolution 556 | #min_res = 8 557 | #py = [im for im in py if min(im.shape[:2]) >= min_res] 558 | return list(reversed(py)) 559 | 560 | 561 | def create_mask_tree(img, kind="causal5x3"): 562 | if kind == "causal5x3": #generate causal mask 563 | mask_res = (3,5) 564 | mask = np.ones(mask_res, dtype=bool) 565 | mask[2,2:]=False 566 | mask_center = (2,2) #starting index from 0 567 | elif kind == "noncausal5x5": #generate non-causal mask: 568 | mask_res = (5,5) 569 | mask = np.ones(mask_res, dtype=bool) 570 | mask_center = (2,2) #starting index from 0 571 | mask[mask_center]=False 572 | elif kind == "noncausal3x3": #generate non-causal mask: 573 | mask_res = (3,3) 574 | mask = np.ones(mask_res, dtype=bool) 575 | mask_center = (1,1) #starting index from 0 576 | else: raise ValueError(f"kind: {kind} is unknown") 577 | 578 | logger.info("generating patches") 579 | m_patches, max_co, idx2co = gen_patches_from_mask(img,mask) 580 | #build local neighbourdhood KDTree for pyramid level 581 | logger.info("generating tree from patches") 582 | index = init_ann_index(m_patches) 583 | return index, mask, mask_center, idx2co 584 | 585 | 586 | #generate image with noisy borders which later be cut off 587 | def local_neighbourhood_enhance(target, ex, mask, tree, 588 | mask_center, idx2co, initial=False, 589 | seed = 0): 590 | #TODO: make tileable texture by copying the right sections from the 591 | # texture to the other sides 592 | 593 | np.random.seed(seed) 594 | 595 | target_res=np.array(target.shape[:2]) 596 | m_res = np.array(mask.shape) 597 | image_range=((mask_center[0], mask_center[0] + target_res[0]), 598 | (mask_center[1], mask_center[0] + target_res[1])) 599 | mc = mask_center 600 | if initial: 601 | t = np.random.random((target_res[0] + m_res[0]-1, 602 | target_res[1] + m_res[1]-1,4)) 603 | t[...,3] = 1.0 604 | t[image_range[0][0]: image_range[0][1], 605 | image_range[1][0]: image_range[1][1]] = target 606 | else: 607 | t = np.random.random((target_res[0] + m_res[0]-1, 608 | target_res[1] + m_res[1]-1,4)) 609 | #target_tmp[:target_cut[0],:target_cut[1]//2] = target[] 610 | t[...,3] = 1.0 611 | t[image_range[0][0]: image_range[0][1], 612 | image_range[1][0]: image_range[1][1]] = target 613 | 614 | #skimage.io.imshow_collection([target, t]) 615 | #return 616 | 617 | if not initial: t2 = t.copy() 618 | else: t2 = t 619 | coords = itertools.product(range(image_range[0][0], image_range[0][1]), 620 | range(image_range[1][0], image_range[1][1])) 621 | #TODO: do some caching of similar mask queries to speed up the 622 | # process quiet a bit 623 | for y,x in tqdm(coords, "iterating over image", 624 | total = target_res.prod(), 625 | smoothing=0.01): 626 | p = t[y-mc[0]:y-mc[0]+m_res[0],x-mc[1]:x-mc[1]+m_res[1]] 627 | pf = p[mask].flatten() 628 | [[err]], [[idx]] = tree.query([pf], k=1) 629 | xp,yp = idx2co(idx) 630 | pixel = ex[yp+mc[0],xp+mc[1]] 631 | t2[y,x] = pixel 632 | 633 | if False: #if debugging 634 | if (x,y)==(4,4): 635 | #py[-1][yp:yp+3,xp:xp+5] 636 | p2 = ex[-1][yp:yp+m_res[0],xp:xp+m_res[1]].copy() 637 | p2[~mask]=(0,0,0,1) 638 | p3 = np.zeros((*mask.shape,4)) 639 | p3[mask] = m_patches[idx].reshape(-1,4) 640 | #skimage.io.imshow_collection([t]) 641 | skimage.io.imshow_collection([p,target, p2,p3, t]) 642 | break 643 | 644 | 645 | return t2[image_range[0][0]: image_range[0][1], 646 | image_range[1][0]: image_range[1][1]] 647 | 648 | 649 | def mask_synthesize_pixel(level, example, seed, pyramid, 650 | final_res, target = None): 651 | img = pyramid[level] 652 | levelscale = img.shape[:2]/np.array(example.shape[:2]) 653 | target_res = np.rint(final_res*levelscale).astype(int) 654 | if target is not None: 655 | target = skimage.transform.resize(target, target_res, anti_aliasing=False,order=0) 656 | kind = "noncausal5x5" 657 | else: 658 | target = np.zeros((*target_res,4), dtype=float) 659 | target[...,3] = 1.0 660 | kind = "causal5x3" 661 | 662 | 663 | tree, mask, mask_center, idx2co = create_mask_tree(img,kind) 664 | target = local_neighbourhood_enhance(target, img, mask, 665 | tree, mask_center, idx2co, 666 | initial=True, seed=seed) 667 | return target 668 | 669 | 670 | 671 | def pixel_synthesize_texture(final_res, scale = 1/2**3, seed = 15): 672 | #TODO: choose output resolution level with a dedicated (2D-)scale-factor 673 | # or a given resolution parameter (original texture will be 674 | # up/down-scaled) 675 | example = skimage.transform.rescale(example0, scale, preserve_range=False, 676 | multichannel=True, anti_aliasing=True) 677 | py = build_gaussian_pyramid(example) 678 | start_level=5 679 | 680 | tas=[] #save images for debug information 681 | 682 | target = mask_synthesize_pixel(start_level, example, seed, py, final_res) 683 | tas.append(target) 684 | for level in range(start_level+1,len(py)): 685 | logger.info(f"\nstarting next level calculation: {level}\n") 686 | target = mask_synthesize_pixel(level, example, seed, py, final_res, target) 687 | tas.append(target) 688 | 689 | return target, tas 690 | 691 | 692 | 693 | def calc_lib_scaling(res_ex, max_pixels): 694 | ex_pixels = np.prod(res_ex) 695 | px_ratio = ex_pixels/max_pixels 696 | scaling = 1/math.sqrt(px_ratio) 697 | if scaling<1.0: return scaling 698 | else: return 1.0 699 | 700 | def normalize_picture(example0, max_pixels = 256*256): 701 | """ 702 | scale picture down to a maximum number of pixels 703 | """ 704 | scaling = 1.0 705 | res_ex = example0.shape[:2] 706 | if max_pixels < np.prod(res_ex): 707 | #max_pixels basically defines whats possible with the avialable 708 | # memory & CPU power 256x256 has proved to be effective on modern systems 709 | scaling = calc_lib_scaling(res_ex, max_pixels) 710 | logger.info(f"resizing with scaling {scaling}") 711 | example = skimage.transform.rescale(example0, scaling, 712 | #example = skimage.transform.resize(example0, (256,256), 713 | anti_aliasing=True, 714 | multichannel=True, 715 | preserve_range=True)#.astype(np.uint8) 716 | #search_res = example.shape[:2] 717 | else: example = example0 718 | 719 | return example, scaling 720 | 721 | def create_patch_params2(res_ex, scaling, 722 | overlap_ratio, patch_ratio): 723 | """patch_ratio = #size of patches in comparison with original 724 | """ 725 | #TODO: define minimum size for patches 726 | res_patch0 = int(min(res_ex)*patch_ratio) 727 | res_patch0 = np.array([res_patch0]*2) 728 | res_patch = np.round(res_patch0*scaling).astype(int) 729 | overlap = np.ceil(res_patch*overlap_ratio).astype(int) 730 | #res_patch2 = np.round(np.array(res_patch)/scaling).astype(int) 731 | overlap0 = np.ceil(res_patch0*overlap_ratio).astype(int) 732 | 733 | return res_patch, res_patch0, overlap, overlap0 734 | 735 | def create_patch_params(example0, scaling, 736 | overlap_ratio = 1/6, patch_ratio = 0.05): 737 | return create_patch_params2(example0.shape[:2], scaling, 738 | overlap_ratio, patch_ratio) 739 | 740 | #create test-target to fill with mask: 741 | def generate_test_target_with_fill_mask(example): 742 | target = example.copy() 743 | target[...,3]=1.0 744 | 745 | verts = [np.array(((0.1,0.1),(0.4,0.15),(0.41,0.4),(0.2,0.38))), 746 | np.array(((0.5,0.55),(0.85,0.53),(0.8,0.7),(0.51,0.71)))] 747 | masks = [] 748 | pxverts = [] 749 | for v in verts: 750 | pxverts.append(v * target.shape[:2]) 751 | rr,cc = skimage.draw.polygon(*v.T) 752 | mask = np.zeros(target.shape[:2]) 753 | mask[rr,cc]=1.0 754 | target[rr,cc]=(1,0,0,1) 755 | masks.append(mask) 756 | 757 | return target, masks, pxverts 758 | 759 | 760 | def draw_polygon_mask(verts, size): 761 | rr,cc = skimage.draw.polygon(*verts.T) 762 | mask = np.zeros(size) 763 | mask[rr,cc]=1.0 764 | return mask 765 | 766 | def edge_distance(poly, x,y): 767 | d = poly.boundary.distance(shapely.geometry.Point(x,y)) 768 | if poly.contains(shapely.geometry.Point(x,y)): return d 769 | else: return -d 770 | 771 | 772 | def get_poly_levelset(verts, width=10): 773 | poly = shapely.geometry.Polygon(verts) 774 | poly_box = poly.buffer(+width) #add two pixels on the container 775 | bbox = poly_box.bounds 776 | miny, minx, maxy, maxx = bbox_px = np.round(np.array(bbox)).astype(int) 777 | w,h = maxx - minx, maxy-miny 778 | 779 | 780 | bbcoords = itertools.product(range(miny,maxy), range(minx, maxx)) 781 | levelset = np.array([edge_distance(poly,y,x) for y,x in bbcoords]).reshape(h,w) 782 | #normalize levelset: 783 | #levelset = np.maximum(levelset/levelset.max(),0.0) 784 | return levelset, bbox_px 785 | 786 | @timing 787 | def fill_area_with_texture(target, example0, ta_map_final=None, 788 | patch_ratio=0.1, libsize = 128*128, 789 | verts=None, mask = None, bounding_box = None): 790 | if bounding_box is None: 791 | area = shapely.geometry.Polygon(verts) 792 | ov = 1 #overlap 793 | y0,x0,y1,x1 = np.array(area.bounds).astype(int) + (-ov,-ov,ov,ov) 794 | else: 795 | y0,x0,y1,x1 = bounding_box 796 | #print("create levelset") 797 | #levelset, (minx, miny, maxx, maxy) = get_poly_levelset(verts, width=ov) 798 | bbox = target[y0:y1,x0:x1] 799 | #bmask = levelset>0 800 | if mask is None: 801 | mask = draw_polygon_mask(verts,target.shape[:2]) 802 | bmask = mask[y0:y1,x0:x1]>0 803 | else: 804 | bmask = mask 805 | #bmask2 = mask[y0:y1,x0:x1]>0 806 | #area.boundary.buffer(100) 807 | 808 | logger.info("synthesize texture") 809 | fill1, ta_map = synth_patch_tex(bbox, example0, k=1, 810 | patch_ratio=patch_ratio, 811 | libsize=libsize) 812 | copy_img(target, fill1, (x0,y0), bmask) 813 | #ta_map_final = np.full((*target.shape[:2],3),[0,0,0]) 814 | #ta_map_final = np.full([*target.shape[:2],3],0) 815 | if ta_map_final is None: ta_map_final = np.zeros([*target.shape[:2],3]) 816 | #copy_img(ta_map_final, ta_map, (x0,y0), bmask) 817 | copy_img(ta_map_final, ta_map, (x0,y0), bmask) 818 | #TODO: somehow the copy operation doesnt work here 819 | #import ipdb; ipdb.set_trace() # BREAKPOINT 820 | 821 | return target, ta_map_final 822 | 823 | def calculate_memory_consumption(res_ex, res_patch, 824 | ch_num, itemsize): 825 | patch_num=np.product(np.array(res_ex) - res_patch) 826 | #import ipdb; ipdb.set_trace() # BREAKPOINT 827 | data_memoryGB = patch_num*ch_num*itemsize*np.product(res_patch)*GB 828 | #print(f"using approx. {data_memoryGB:2f} GB in RAM.") 829 | return data_memoryGB 830 | 831 | def check_memory_requirements(example, res_patch, maxmem = 1.0, 832 | disable_safety_check=False): 833 | data_memoryGB = calculate_memory_consumption( 834 | np.array(example.shape[:2]),res_patch, 835 | ch_num = example.shape[2], 836 | itemsize = example.itemsize) 837 | logger.info(f"using {data_memoryGB:.4f} of {maxmem} GB for synthesis") 838 | if not disable_safety_check: 839 | if data_memoryGB > maxmem: 840 | raise MemoryError("the algorithm would exceed the " 841 | "maximum amount of Memory: " 842 | f"{data_memoryGB:2f} GB,; max: {maxmem}") 843 | 844 | return data_memoryGB 845 | 846 | @timing 847 | def prepare_tree(example0, lib_size, overlap_ratio, patch_ratio, 848 | mode=None): 849 | example, scaling = normalize_picture(example0, lib_size) 850 | res_patch, res_patch0, overlap, overlap0 = create_patch_params(example0, scaling, 851 | overlap_ratio, 852 | patch_ratio) 853 | max_co = np.array(example.shape[:2]) - res_patch 854 | 855 | data_memoryGB = check_memory_requirements(example,res_patch, 856 | maxmem=get_mem_limit(), 857 | disable_safety_check=True) 858 | logger.info(f"using approx. {data_memoryGB:2f} GB in RAM.") 859 | try: 860 | if mode==None: 861 | data = create_patch_data(example, res_patch, max_co) 862 | index = init_ann_index(data) 863 | else: 864 | if ('both' in mode) or ('horizontal' in mode): 865 | logger.info("init kdtree1") 866 | ld = create_patch_data(example, (res_patch[0], overlap[1]), max_co) 867 | l = init_ann_index(ld) 868 | if ('both' in mode) or ('vertical' in mode): 869 | logger.info("init kdtree2") 870 | td = create_patch_data(example, (overlap[0],res_patch[1]), max_co) 871 | t = init_ann_index(td) 872 | if 'both:': 873 | logger.info("init kdtree3") 874 | lt = init_ann_index(np.hstack((ld,td))) 875 | index = [l,t,lt] 876 | except MemoryError as err: 877 | logger.info(err) 878 | logger.info("example texture too large, algorithm needs " 879 | "too much RAM: {totalmemoryGB:.2f}GB") 880 | raise 881 | return index, res_patch, res_patch0, overlap, overlap0, max_co, scaling 882 | 883 | def gen_coordinate_grid(shape, flatten=False): 884 | x = np.arange(0,shape[1]) 885 | y = np.arange(0,shape[0]) 886 | #get coordinate grid 887 | #TODO: this might be more elegant using np.dstack 888 | grid = np.stack(np.meshgrid(y,x),axis = 2) 889 | if flatten: grid = grid.reshape(-1,2) 890 | return grid 891 | 892 | def gen_coordinate_map(shape): 893 | grid2ch=gen_coordinate_grid(shape) 894 | #TODO: this might be more elegant using np.dstack 895 | grid3ch=np.stack((grid2ch[...,0], 896 | grid2ch[...,1], 897 | np.zeros(shape)),axis=2)#,axis=2) 898 | return grid3ch 899 | 900 | """ 901 | #TODO: high quality optimization 902 | This function doesn't work yet... 903 | 904 | the main problem is that when changing the resolution of textures 905 | only for single patches, we get a different result for that area 906 | then if we would change it for the entire texture at once. The reason 907 | for this are the rounding errors when going to smaller resolution 908 | pictures: some of the pixels of the smaller sized image will include 909 | the values of multiple pixels also from the neighbouring patches. if 910 | we decrease the resolution of only a patch, this does not happen. 911 | 912 | This 913 | makes it impossible to incoporate 914 | @timing 915 | def synth_patch_tex(target0, example0, 916 | lib_size = 10000, 917 | k=1, 918 | patch_ratio=0.1, 919 | overlap_ratio = 1/6, 920 | tol=0.1 921 | ): 922 | ''' 923 | lib_size = 10000 924 | k=1 925 | patch_ratio=0.1 926 | overlap_ratio = 1/6 927 | tol=0.1 928 | ''' 929 | #TODO: merge this function with the "search" and the "optimal patch" 930 | # functionality to make it similar to the "synthesize_tex_patches" function 931 | chan = 3 932 | #target = target.copy() 933 | example0 = example0[...,:chan] 934 | target_map = target0.copy() 935 | (trees, res_patch, 936 | res_patch0, overlap, 937 | overlap0, max_co, 938 | scaling) = prepare_tree(example0, lib_size, overlap_ratio, patch_ratio, 939 | mode=["horizontal","vertical","both"]) 940 | logger.info(f"patch_size: {res_patch0}; initial scaling: {scaling}, ") 941 | 942 | #define search grid 943 | res_target0 = target0.shape[:2] 944 | rpg0 = np.array(res_patch0) - overlap0 945 | rpg = np.array(res_patch) - overlap 946 | res_grid0 = np.ceil((res_target0-overlap0)/rpg0).astype(int) 947 | 948 | co_map_base = gen_coordinate_map(res_patch0) 949 | 950 | #resize target to the same scale as the scaled example 951 | #this is actually important and can NOT be done "on the fly" for 952 | #each 953 | target = skimage.transform.rescale(target0, scaling, 954 | anti_aliasing=True, 955 | multichannel=True, 956 | preserve_range=True)#.astype(np.uint8) 957 | 958 | example = skimage.transform.rescale(example0, scaling, 959 | anti_aliasing=True, 960 | multichannel=True, 961 | preserve_range=True)#.astype(np.uint8) 962 | 963 | #left corner (0,0) 964 | for coords in tqdm(np.ndindex(*res_grid0),"iterate over image"): 965 | yp,xp=np.array(coords) * rpg 966 | search_area0 = target[yp:yp+res_patch[0], 967 | xp:xp+res_patch[1]].copy() 968 | if coords==(0,0): 969 | pa_coords_idx = pa_y,pa_x = np.array(idx2co( 970 | random.randrange(np.product(max_co)), 971 | max_co)) #get first patch 972 | 973 | #skimage.io.imshow_collection([patch, co_map/(*example.shape[:2],1)]) 974 | elif coords[0]==0: #first row 975 | ovl = search_area0[:,:overlap[1]] 976 | #ovl = skimage.transform.resize(ovl, 977 | # (res_patch[0],overlap[1]), 978 | # preserve_range=True) 979 | pa_idx = find_match(ovl.flatten(), trees[0] , tol=tol, k=k) 980 | pa_coords_idx = pa_y,pa_x = np.array(idx2co(pa_idx, max_co)) 981 | 982 | pa = example[pa_y:pa_y+res_patch[0],pa_x:pa_x+res_patch[1]] 983 | copy_img(target,pa,(xp,yp)) 984 | pa_y0,pa_x0 = np.round(pa_coords_idx / scaling).astype(int) 985 | pa0 = example0[pa_y0:pa_y0+res_patch0[0],pa_x0:pa_x0+res_patch0[1]] 986 | copy_img(target0,pa0,(xp,yp)) 987 | co_map = co_map_base + (pa_y0,pa_x0,0) 988 | copy_img(target_map,co_map,(xp,yp)) 989 | 990 | #ovl = example[y:y+rp[0],x:x+rp[1]][:,-overlap[0]:].flatten() 991 | 992 | #if False: 993 | if coords==(0,10): 994 | #patch_idx = trees[0].get_arrays()[0][pa_idx].reshape(res_patch[0],-1,3) 995 | patch_idx = trees[0]._raw_data[pa_idx].reshape(res_patch[0],-1,3) 996 | skimage.io.imshow_collection([ovl,search_area0, pa0, patch_idx]) 997 | skimage.io.imshow_collection([pa0, target_map/(*example0.shape[:2],1), target]) 998 | break 999 | 1000 | return target_map, target 1001 | """ 1002 | 1003 | 1004 | def synth_patch_tex(target, example0, k=1, patch_ratio=0.1, libsize = 256*256): 1005 | #TODO: merge this function with the "search" and the "optimal patch" 1006 | # functionality to make it similar to the "synthesize_tex_patches" function 1007 | example, scaling = normalize_picture(example0, libsize) 1008 | res_target = target.shape[:2] 1009 | res_patch, res_patch2, overlap, overlap2 = create_patch_params(example0, scaling, 1010 | patch_ratio=patch_ratio) 1011 | res_grid = np.ceil(res_target/(res_patch2 - overlap2)).astype(int) 1012 | 1013 | logger.info(f"patch_size: {res_patch2}; initial scaling: {scaling}, ") 1014 | 1015 | #time.sleep(10.0) 1016 | 1017 | pg = synthesize_grid(example, res_patch, res_grid, overlap, tol=.1, k=k) 1018 | 1019 | #transform pg coordinates into original source texture 1020 | pg2 = np.round(pg / scaling).astype(int) 1021 | pgimage = (pg2/example0.shape[:2])[...,::-1] 1022 | pgimage = np.dstack((pgimage,np.zeros(pgimage.shape[:2]))) 1023 | 1024 | target, ta_map = transform_patch_grid_to_tex(None, res_patch2, pg2, example0, 1025 | overlap2, 1026 | use_quilting=True) 1027 | 1028 | #target2, ta_map = transform_patch_grid_to_tex(None, res_patch, pg, example, 1029 | # overlap, 1030 | # use_quilting=True) 1031 | 1032 | return (target[:res_target[0],:res_target[1]], 1033 | ta_map[:res_target[0],:res_target[1]]) 1034 | 1035 | 1036 | 1037 | @timing 1038 | def synthesize_tex_patches(target0, example0, 1039 | lib_size = 200*200, 1040 | patch_ratio = 0.07, 1041 | overlap_ratio = 1/3): 1042 | """ 1043 | TODO: make target texture use the already synthesized patch as an option 1044 | --> this means to take a patch from the high-res image and scale it down 1045 | "on the fly". This way can avoid creating a scaled down version of the 1046 | target. It also makes the produced patch more "precise" as we are not 1047 | restricted to the target-pixels anymore 1048 | """ 1049 | 1050 | target_new = target0.copy() 1051 | (tree, res_patch, 1052 | res_patch0, overlap, 1053 | overlap0, max_co, 1054 | scaling) = prepare_tree(example0, lib_size, overlap_ratio, patch_ratio) 1055 | 1056 | res_target0 = target0.shape[:2] 1057 | rpg0 = np.array(res_patch0) - overlap0 1058 | res_grid0 = np.ceil((res_target0-overlap0)/rpg0).astype(int) 1059 | 1060 | rp = res_patch 1061 | #resize target to the same scale as the scaled example 1062 | #TODO: replace this with a "instant-scale-down" method (have a look 1063 | #at function documentation) 1064 | target = skimage.transform.rescale(target0, scaling, 1065 | anti_aliasing=True, 1066 | multichannel=True, 1067 | preserve_range=True)#.astype(np.uint8) 1068 | 1069 | for coords in tqdm(np.ndindex(*res_grid0),"iterate over image"): 1070 | #TODO: only iterate over "necessary" pixels indicated by a mask 1071 | #to get "whole" patches, we need the last row to have the same 1072 | #border as the target image thats why we use "minimum": 1073 | #print(coords) 1074 | yp0,xp0 = co_target0 = np.minimum(res_target0-res_patch0,np.array(coords) * rpg0) 1075 | yp,xp = np.round(co_target0*scaling).astype(int) 1076 | yp,xp = np.minimum(np.array(target.shape[:2])-res_patch, (yp,xp)) 1077 | 1078 | #TODO: replace with "real" target and scale down. This makes 1079 | # the search more precise as pixel rounding effects can be 1080 | # mitigated this way 1081 | search_area = target[yp:yp+rp[0],xp:xp+rp[1]].copy() 1082 | new_idx = find_match(search_area.flatten(), tree , tol=0.1, k=1) 1083 | #find patch from original image 1084 | co_p = np.array(idx2co(new_idx, max_co)) 1085 | co_p0 = np.round(co_p / scaling).astype(int) 1086 | ovs = np.r_[overlap0,0,0]#,overlap0] 1087 | pa, pa0, ta0, mask = optimal_patch(target_new, example0, res_patch0, 1088 | ovs, co_p0, co_target0) 1089 | copy_img(target_new, pa, co_target0[::-1]) 1090 | 1091 | return target_new 1092 | 1093 | def calculate_face_normal(f): 1094 | v1,v2 = f[1]-f[0], f[2]-f[1] 1095 | n = sign(np.cross(v1,v2)) 1096 | return n 1097 | 1098 | #@timing 1099 | def transfer_patch_pixelwise(target, search_area0, 1100 | yp,xp, 1101 | edge_info, 1102 | fromtarget, 1103 | face_source, 1104 | ta_map = None, 1105 | sub_pixels = 1, 1106 | mask = None, 1107 | pa = None, 1108 | ta_pa = None, 1109 | tol=0.0): 1110 | """This function takes an edge and pixel, and searches for the corresponding 1111 | pixel at another edge. 1112 | 1113 | TODO: cythonize or do other stuff with this function (nuitka, scipy, numba) 1114 | """ 1115 | e1,e2,v1,v2,e2_perp_left = edge_info 1116 | """ 1117 | TODO: the below version is still present, as it might 1118 | be preferable to do a loop-based version using cython 1119 | for patch_index in np.ndindex(search_area0.shape[:2]): 1120 | for sub_pix in np.ndindex(sub_pixels,sub_pixels): 1121 | sub_idx = np.array(patch_index) + np.array(sub_pix)/sub_pixels 1122 | coords = np.array((yp,xp)) + sub_idx - e1[0] 1123 | c = coords.dot(v1)/(v1.dot(v1))#orthogonal projection of pixel on edge 1124 | d = c*v1-coords #orthogonal distance from edge 1125 | #pixel on left or right side of edge? 1126 | isleft = (v1[1]*d[0]-d[1]*v1[0])>0 1127 | d_len = norm(d) 1128 | if isleft or (d_len0: 1140 | #copy pixel from generated patch back to target 1141 | target[tuple(tmp)] = pa[patch_index]""" 1142 | 1143 | y1,x1 = search_area0.shape[:2] 1144 | x = np.arange(0,x1,1.0/sub_pixels) 1145 | y = np.arange(0,y1,1.0/sub_pixels) 1146 | #get coordinate grid 1147 | sub_pix = np.stack(np.meshgrid(y,x),axis = 2).reshape(-1,2) 1148 | #vector from uv edge to sub_pixel in patch with corner yp,xp 1149 | coords_matrix = (yp,xp) + sub_pix - e1[0] 1150 | 1151 | 1152 | proj = proj = coords_matrix.dot(v1)/(v1.dot(v1)) #orthogonal projection of pixels on edge 1153 | orth_d = np.outer(proj,v1)-coords_matrix #orthogonal distance vector from edge 1154 | 1155 | isleft = (v1[1]*orth_d[:,0]-orth_d[:,1]*v1[0])>0 1156 | d_len = norm(orth_d)#distance of pixel from edge 1157 | 1158 | #calculate corresponding pixel at other edge in reverse direction 1159 | # (this is why we use (1-proj)) 1160 | p_e2s = np.outer((1-proj),(v2)) + e2[0] #projecton on corresponding edge 1161 | #import ipdb; ipdb.set_trace() # BREAKPOINT 1162 | px2_cos = p_e2s + (np.outer(d_len,e2_perp_left).T*(isleft * 2 - 1)).T#orthogonal pixel from that projection point on the edge 1163 | px2_cos = np.round(px2_cos).astype(int) 1164 | 1165 | #choose to transfer pixels from the left soude + a little overlap 1166 | #transfer_pixels = (isleft | (d_len0: 1190 | #copy pixel from generated patch back to target 1191 | #target[tuple(px2_coords)][:3] = pa[patch_index][:3]*mask[patch_index] 1192 | target[tuple(px2_coords)] = pa[patch_index] 1193 | if ta_map is not None: 1194 | ta_map[tuple(px2_coords)] = ta_pa[patch_index] 1195 | 1196 | def check_inside_convex_quadrilateral(corners, p, tol=0): 1197 | """be careful!, because this function is based 1198 | on whether the coordinate system is left- or right-handed""" 1199 | #TODO: vectorize this function or implement some cython-magic 1200 | sv = np.roll(corners, 1, 0) - corners 1201 | pv = p-corners 1202 | 1203 | #normalized normal vetors on all edges: 1204 | n = normalized(sv[:,::-1]*(1,-1)) 1205 | #orthogonal distance for all edges 1206 | orth_d = (pv*n).sum(axis=1) 1207 | #calculate distance from all edges 1208 | 1209 | return np.all(orth_d < tol) 1210 | # else: 1211 | # #check if point lies on left side of every side vector 1212 | #return np.all(sv[:,0] * pv[:,1] - sv[:,1] * pv[:,0] >= 0) # > 0 1213 | #return True 1214 | 1215 | #cache this function as it will be very similar for many points in the 1216 | #polygon. The function cache should be reset when the algorithm gets rerun though 1217 | def check_inside_face(polygon, point, tol=0.0): 1218 | face = shapely.geometry.Polygon(polygon).buffer(tol) 1219 | return face.contains(shapely.geometry.Point(*point)) 1220 | 1221 | @timing 1222 | def make_seamless_edge(edge1,edge2, target, example0, ta_map, 1223 | patch_ratio, 1224 | lib_size, debug_level=0, 1225 | tree_info = None): 1226 | #TODO: make sure that the "longer" edge is defined as 1227 | #"e1" --> this is so that we don't have huge undefined patch_sizes 1228 | #in the final result. Additionally, this make sure that we always have a 1229 | #"high-resolution"-edge where the patchesa re used. 1230 | 1231 | 1232 | (e1,verts1),(e2,verts2) = edge1,edge2 1233 | v1 = e1[1]-e1[0] 1234 | v2 = e2[1]-e2[0] 1235 | if norm(v1)0: #for debugging 1320 | search_area0[:,:,0:2]=(1,1) 1321 | pa0_r = pa0.copy() 1322 | pa0[:,:,0]*=0.5 #green -> "inside" 1323 | pa[:,:,1]*=0.5 #red -> "outside" 1324 | 1325 | #copy one side of the patch back to its respective face 1326 | #and also create a left/rght mask for masking the second part 1327 | #import ipdb; ipdb.set_trace() # BREAKPOINT 1328 | #print("copy back!") 1329 | transfer_patch_pixelwise(target_new, search_area0, 1330 | yp,xp, 1331 | edge_info = (e1,e2,v1,v2,e2_perp_left), 1332 | fromtarget = False, 1333 | face_source = verts2, 1334 | sub_pixels = 2, 1335 | mask = mask, 1336 | pa = pa, 1337 | ta_pa = ta_pa, 1338 | ta_map = ta_map, 1339 | tol=tol) 1340 | 1341 | 1342 | #TODO: copy only the part thats "inside" face 1 1343 | mask_inside = np.zeros(search_area0.shape[:2]) 1344 | for patch_index in np.ndindex(search_area0.shape[:2]): 1345 | coords = patch_index + np.array((yp,xp)) 1346 | mask_inside[patch_index] = check_inside_convex_quadrilateral(verts1,coords, tol=tol) 1347 | 1348 | #copy only the right side to its place 1349 | #mask_right_optimal = mask_sides==0 1350 | #mask_right_optimal = np.minimum(mask_sides==0, mask>0) 1351 | mask_right_optimal = np.minimum(mask>0, mask_inside>0) 1352 | copy_img(target_new, pa0, (xp,yp), mask_right_optimal) 1353 | 1354 | copy_img(ta_map, ta_pa, (xp,yp), mask_right_optimal) 1355 | #copy_img(target_new.copy(),) 1356 | 1357 | #if counter == 2: 1358 | if False:#debug_level>0:#counter == 2: 1359 | #patch_from_data = data[new_idx].reshape(*res_patch,4) 1360 | skimage.io.imshow_collection([search_area0, pa, mask, pa0, mask_inside]) 1361 | #skimage.io.imshow_collection([search_area0, search_area, target1, 1362 | # patch_from_data, pa,pa0,ta0]) 1363 | #break 1364 | 1365 | #import ipdb; ipdb.set_trace() # BREAKPOINT 1366 | return (target_new[res_patch0[0]:-res_patch0[0], 1367 | res_patch0[1]:-res_patch0[1]], 1368 | ta_map[res_patch0[0]:-res_patch0[0], 1369 | res_patch0[1]:-res_patch0[1]], 1370 | tree_info) 1371 | 1372 | if __name__ == "__main__": 1373 | logging.basicConfig(level=logging.INFO) 1374 | logger.setLevel(logging.INFO) 1375 | logging.getLogger('tex_synthesize').setLevel(logging.INFO) 1376 | #example0 = example = skimage.io.imread("textures/3.gif") #load example texture 1377 | example0 = skimage.io.imread("textures/rpitex.png") 1378 | example0 = example0/255 1379 | #example0 = skimage.transform.resize(example0, (500,1000)) 1380 | example0 = skimage.transform.rescale(example0, 0.3, multichannel=True) 1381 | #example0 = example = skimage.io.imread("RASP_03_05.png") #load example texture 1382 | #TODO: more sophisticated memory reduction techniques (such as 1383 | # a custom KDTree) This KDTree could be based on different, hierarchical 1384 | # image resolutions for the error-search 1385 | 1386 | #TODO: test "overhaul" of images with correspondance maps 1387 | 1388 | 1389 | #test image synthesis function 1390 | if True: 1391 | seed = 32 1392 | np.random.seed(seed) 1393 | random.seed(seed)#2992)#25 is chip + original img 1394 | 1395 | channels = 3 #TODO: prepare function for 1,3 and 4 channels 1396 | target0 = np.full((800,1000,channels),0.0) 1397 | target, ta_map = synth_patch_tex(target0,example0, 1398 | libsize=256*256, 1399 | patch_ratio=0.1) 1400 | ta_map = ta_map/np.array([*example0.shape[:2][::-1],1]) 1401 | skimage.io.imshow_collection([target, ta_map]) 1402 | 1403 | if False: 1404 | np.random.seed(10) 1405 | random.seed(50)#2992)#25 is chip + original img 1406 | target1, mask, verts = generate_test_target_with_fill_mask(example0) 1407 | skimage.io.imshow_collection([target1,mask]) 1408 | target1[:]=(0,0.5,0,1) 1409 | for v in verts: 1410 | y0,x0,y1,x1 = np.array(shapely.geometry.Polygon(v).bounds).astype(int) 1411 | #target1[y0:y1,x0:x1]*=(0.5,0.5,0.5,1)#mark bounding box for debugging 1412 | target1, fill1, fill2, pgimg, bmask = fill_area_with_texture(target1, example0, v) 1413 | 1414 | check_inside_face(verts[0],(55,100)) 1415 | 1416 | 1417 | #select two corresponding edges: 1418 | edges = ((verts[0][:2],verts[1][:2]), 1419 | (verts[0][1:3],verts[1][1:3])) 1420 | 1421 | target_new = target1 1422 | for e1,e2 in edges[:1]: 1423 | logger.info("alter next edge") 1424 | target_new = make_seamless_edge((e1,verts[0]),(e2,verts[1]), 1425 | target_new, example0) 1426 | 1427 | skimage.io.imshow_collection([target_new, target1]) 1428 | 1429 | #test image copying with "wrong" boundaries 1430 | if False: 1431 | tmp = np.full((100,100,4),(1.0,0,0,1)) 1432 | search_area0 = np.random.random((28,28,4)) 1433 | pos = (5,10) 1434 | copy_img(target=search_area0,src=tmp,pos=pos) 1435 | skimage.io.imshow_collection([search_area0,tmp]) 1436 | 1437 | #skimage.io.imshow_collection([target_new]) 1438 | #edge_area = np.array(edgebox.boundary.coords)[:-1] 1439 | #target1[skimage.draw.polygon(*x.boundary.xy)] *= (0.5,0.5,1.0,1.0) 1440 | #skimage.io.imshow_collection([target1]) 1441 | 1442 | #edgebox.buffer(15).bounds 1443 | 1444 | 1445 | #area around edge to get edge straight 1446 | #calculate area around edge: 1447 | 1448 | 1449 | #edge = 10 1450 | #pos = y0-edge,x0-edge 1451 | #target0 = target1[y0-edge:y1+edge,x0-edge:x1+edge].copy() 1452 | 1453 | #skimage.io.imshow_collection([target0]) 1454 | #TODO: cut out the stripes between the two uvs 1455 | 1456 | 1457 | 1458 | #skimage.io.imshow_collection([target0_start, fill1, fill2, pgimg])#,target0]) 1459 | 1460 | #targets=[] 1461 | #targets.append(synthesize_tex_patches(target0, example)) 1462 | #targets.append(synthesize_tex_patches(targets[-1], example,patch_ratio = 0.05)) 1463 | #targets.append(synthesize_tex_patches(targets[-1], example,patch_ratio = 0.04)) 1464 | 1465 | #copy_img(target1, targets[-1], pos[::-1]) 1466 | 1467 | #skimage.io.imshow_collection(targets) 1468 | #skimage.io.imshow_collection([target0, target]) 1469 | #skimage.io.imshow_collection([pgimg]) 1470 | #skimage.io.imshow_collection([target1]) 1471 | #skimage.io.imshow_collection([example]) 1472 | 1473 | #tree, mask, mask_center, idx2co = create_mask_tree(example0,"noncausal5x5") 1474 | 1475 | #skimage.io.imshow_collection([target, target2, pgimg, example0, target1, mask]) 1476 | #skimage.io.imshow_collection([target1, fill1, fill2, pgimg]) 1477 | #skimage.io.imshow_collection([target1]) 1478 | #skimage.io.imshow_collection([example0]) 1479 | #skimage.io.imsave("debug/synth.jpg", target0[...,:3]) 1480 | 1481 | #analyze resulting patchgrid (og): 1482 | #import pandas as pd 1483 | #allcells = pg.view([('f0', pg.dtype), ('f1', pg.dtype)]).squeeze(axis=-1) 1484 | #allcells = pd.DataFrame(np.unique(allcells.flatten(), return_counts=True)).T 1485 | #allcells.sort_values(1) 1486 | #pd.DataFrame.from_items(allcells) 1487 | 1488 | --------------------------------------------------------------------------------