├── README.md ├── clean.py ├── extract.py ├── generation.py ├── interpolate.py ├── iterate.py ├── prompt.txt ├── settings.py └── spritesheet.py /README.md: -------------------------------------------------------------------------------- 1 | # Stable Diffusion Animation Utilities 2 | 3 | This repo contains a suite of utility programs written in Python to help create animations using Stable Diffusion 4 | 5 | All of the programs read constants from settings.py - if anything is not where it expects it will let you know (most likely) and you either need to rename your files, or change a value in settings.py 6 | 7 | ## Getting Started 8 | 9 | Install the required python packages 10 | ``` 11 | pip install opencv-python 12 | pip install numpy 13 | pip install rembg 14 | ``` 15 | If you want to run frame interpolation locally, you will also need some tensorflow packages. For this I used a conda env with python=3.11 and used these installs 16 | ``` 17 | pip install tensorflow 18 | pip install tensorflow_hub 19 | ``` 20 | 21 | ## Initializing a Workspace 22 | 23 | To get a workspace started, you need to do these x things: 24 | 1. Create a folder in the workspaces directory, and change WORKSPACE in settings.py to that folder location 25 | 2. Create a poses folder in your workspace, and put all of your openpose pngs in there 26 | 3. Create a turntable.png pose picture, put it into the poses folder 27 | 4. Create a reference image following the turntable pose, and put it in the root of your workspace named source.png 28 | 5. Run `python generation.py` to initialize your first iteraction 29 | 30 | After running step 5, if everything went well you should see a iter000 folder appear in your workspace. If you look inside, it should house a suite of folders, all named the same as all of the pose images in your poses folder, minus the turntable. 31 | 32 | You can now run, folder by folder, the images and poses through stable diffusion to generate an initial version for all of your characters in each of their poses. Once you found an image you are satisfied with for a given pose, throw that image into the corresponding folder, alongside input.png, pose.png, and data.json (you should now have 4 files in that folder). 33 | 34 | ## Iterating a Workspace 35 | 36 | To clean up the images and increase coherency, the iterate.py script will pull in surrounding images for the refinement process. For control over what gets pulled in, check out the KEEP_TURNTABLE, IMGS_LEFT, IMGS_RIGHT, and RANDOMIZE_ORDER variables at the top of settings.py. 37 | 38 | Once you have the settings the way you want, run iterate.py. If everything went well, you should see an iter001 folder appear with all of your settings. If you do not like what you got, you can always change the settings, delete the iter folder, and run iterate.py again. 39 | 40 | The process of running images through SD is exactly the same as before. Pull the images into your webui, run them through SD, and pull the result in the same folder the inputs came from. 41 | 42 | To keep iterating, run iterate.py to generate a iter002, iter003, and so on folders to really refine your animation. 43 | 44 | ## Extracting and Cleaning 45 | 46 | Once you are satisfied with the result, you can run extract.py. This will run through the latest iter folder, and pull out your subjects, placing them into and extracted folder in your workspace. 47 | 48 | Running clean.py will take all of the images in the extracted folder, remove the background along with creating a variant with a clean background, and saving both copies into a cleaned folder. 49 | 50 | ## Interpolating (Optional) 51 | 52 | I also included a script to interpolate between frames. This is a little more complex, required a more finicky python install library, along with source code changes to direct what animations to interpolate between. 53 | 54 | Note: you can also use 3rd party interpolation. I have not used any of them, but RunwayML seems to have a decent one at first glance. 55 | 56 | Heding down to Line 131, you will see 4 variables and a function. 57 | 1. ITERATIONS: how many iterations this program will run between frames. For every iteration, the number of frames will be doubled. For example if you had 4 animation frames, 1 iteration would create 4 new frames (8 total), 2 iterations would create 12 new frames (16 total), 3 iterations would create 28 new frames (32 total), etc. 58 | 2. LOOP_INTERP: whether the program will try to interpolate between the last and first frame. Set this to true if your animation is supposed to loop (like a run animation) and false if not (like an attack animation). 59 | 3. interp_folder: where it will scan for the files to interpolate between (default is the cleaned folder in your workspace). If you don't feel comfortable changing code, put the files you want to interpolate into a folder within cleaned and add that folder name to this variable. 60 | 4. to_interpolate: a function to determine if a file within interp_folder will be considered for animation. Default is return True while runs all files in that folder, but I left some examples for how one could limit files to certain animation segments. 61 | 62 | Run iterpolate.py and you should see your interpolated frames getting generated. 63 | 64 | ## Spritesheet Generation 65 | 66 | Once you have everything you want, you can run spritesheet.py to generate a spritesheet of your animation frames (without a background). There are a few settings within spritesheet.py to control some aspects of generation. 67 | 1. SKIP_EXISTING: the program automatically runs the remove background on images that do not have a rembg version. Setting this variable to True will have it run it for ALL images, even if one already exists. Mostly there for if you run interpolate a second time, and need the rembg version of iterpolated frame regenerated. 68 | 2. MAX_WIDTH: will create a second and so on row as the size of the spritesheet exceeds this value. Is there since Godot has a max image width of 16k for sprites. 69 | 3. RESIZE: by what factor images are resized when put into the spritesheet. Default is 1 (unchanged), but a value of 0.5 would downsize the image by 1/2 for both width and height (1/4 the pixels). This does NOT use AI, so I recommend avoiding upscaling using this feature. 70 | -------------------------------------------------------------------------------- /clean.py: -------------------------------------------------------------------------------- 1 | ################################################## 2 | # # 3 | # Cleans the extraction run and adds back in # 4 | # a stable background # 5 | # # 6 | ################################################## 7 | 8 | from settings import * 9 | 10 | import subprocess 11 | import numpy as np 12 | import cv2 13 | import os 14 | 15 | if not os.path.exists(CLEAN_PATH): os.mkdir(CLEAN_PATH) 16 | 17 | for file in os.listdir(EXTRACT_PATH): 18 | if file.endswith("_s.png") or file == SHEET_NAME or not file.endswith('.png'): continue 19 | print(file) 20 | 21 | clean_img_path = f"{EXTRACT_PATH}/{file}" 22 | stripped_img_path = f"{CLEAN_PATH}/{file.replace('.png', '_s.png')}" 23 | subprocess.call(["rembg", "i", clean_img_path, stripped_img_path]) 24 | 25 | img = cv2.imread(stripped_img_path, cv2.IMREAD_UNCHANGED) 26 | output = np.ones((img.shape[0], img.shape[1], 3)) 27 | output *= BG_COLOR 28 | for c in range(3): 29 | output[:, :, c] = (1-img[:,:,3]/255)*output[:,:,c] + (img[:,:,3]/255)*img[:,:,c] 30 | cv2.imwrite(f"{CLEAN_PATH}/{file}", output) 31 | -------------------------------------------------------------------------------- /extract.py: -------------------------------------------------------------------------------- 1 | ################################################## 2 | # # 3 | # Extracts all of your images from the last # 4 | # iteration that was run # 5 | # # 6 | ################################################## 7 | 8 | from settings import * 9 | 10 | import json 11 | import cv2 12 | import os 13 | 14 | def folder_name(i): 15 | return f"{WORKSPACE}/{ITER_PREFIX}{i:03}" 16 | 17 | curr_folder = "" 18 | next_folder = "" 19 | for i in range(1000): 20 | next_folder = folder_name(i) 21 | if not os.path.exists(next_folder): 22 | curr_folder = folder_name(i-1) 23 | break 24 | 25 | print(f"extracting: {curr_folder}") 26 | assert len(curr_folder) > 0, "could not find an iter folder, run generation.py to first create a workspace" 27 | 28 | if not os.path.exists(EXTRACT_PATH): os.mkdir(EXTRACT_PATH) 29 | 30 | root_folders = [f for f in os.listdir(curr_folder) if f != TOUCH_FILE_NAME] 31 | exclusion_list = [DATA_JSON_NAME, INPUT_IMG_NAME, POSE_IMG_NAME] 32 | 33 | imgs = [] 34 | 35 | for root_folder in root_folders: 36 | root = f"{curr_folder}/{root_folder}" 37 | 38 | data_path = f"{root}/{DATA_JSON_NAME}" 39 | input_paths = [f for f in os.listdir(root) if f not in exclusion_list] 40 | assert len(input_paths) == 1, f"expected 1 extra file in {root_folder}, got {len(input_paths)}: {input_paths}" 41 | input_path = f"{root}/{input_paths[0]}" 42 | 43 | assert os.path.exists(data_path), f"could not find data_path for {root_folder}, searched {data_path}" 44 | assert os.path.exists(input_path), f"could not find input_path for {root_folder}, searched {input_path}" 45 | 46 | with open(data_path) as f: 47 | data = json.loads(f.read()) 48 | assert "start" in data, f"expected 'start' key in data json file for {root_folder} at {data_path}" 49 | assert "end" in data, f"expected 'end' key in data json file for {root_folder} at {data_path}" 50 | 51 | input = cv2.imread(input_path) 52 | assert input is not None, f"input img was None for '{input_path}', expected value" 53 | assert input.shape[1] >= data["end"], f"input img must have width of at least data.json['end'], found {input.shape[1]} < {data['end']}" 54 | input = input[:,data["start"]:data["end"]] 55 | 56 | clean_img_path = f"{EXTRACT_PATH}/{root_folder}.png" 57 | cv2.imwrite(clean_img_path, input) 58 | -------------------------------------------------------------------------------- /generation.py: -------------------------------------------------------------------------------- 1 | ################################################## 2 | # # 3 | # Very first file to run to init a workspace # 4 | # # 5 | ################################################## 6 | 7 | from settings import * 8 | 9 | import subprocess 10 | import numpy as np 11 | import json 12 | import cv2 13 | import os 14 | 15 | if not os.path.exists(WORKSPACE): os.makedirs(WORKSPACE) 16 | 17 | assert os.path.exists(POSES_FOLDER), f"poses folder must exist, settings.py says it is '{POSES_FOLDER}'" 18 | assert os.path.exists(TURNTABLE_PATH), f"turntable image must exist, settings.py says it is '{TURNTABLE_PATH}'" 19 | assert os.path.exists(SOURCE_PATH), f"source image must exist, settings.py says it is '{SOURCE_PATH}'" 20 | subprocess.call(["rembg", "i", SOURCE_PATH, SOURCE_S_PATH]) 21 | assert os.path.exists(SOURCE_S_PATH), f"error creating stripped source image" 22 | 23 | imgs = [f for f in os.listdir(POSES_FOLDER) if f != TURNTABLE_NAME] 24 | print(f"found pose images: {imgs}") 25 | 26 | tt = cv2.imread(TURNTABLE_PATH) 27 | assert tt is not None, f"error loading turntable image '{TURNTABLE_PATH}', cv2 returned null" 28 | 29 | source = cv2.imread(SOURCE_S_PATH, cv2.IMREAD_UNCHANGED) 30 | assert source is not None, f"error loading source image '{SOURCE_S_PATH}', cv2 returned null" 31 | 32 | assert tt.shape[0] == source.shape[0], f"source and turntable images must have the same height, {source.shape[0]} != {tt.shape[0]}" 33 | assert tt.shape[1] == source.shape[1], f"source and turntable images must have the same width, {source.shape[1]} != {tt.shape[1]}" 34 | 35 | iter_name = f"{ITER_PREFIX}000" 36 | 37 | first = True 38 | for img in imgs: 39 | assert img.endswith(".png"), f"pose images must be of type png, invalid for {img}" 40 | 41 | path = f"{POSES_FOLDER}/{img}" 42 | i = cv2.imread(path) 43 | 44 | assert i is not None, f"error loading pose image '{path}', cv2 returned null" 45 | assert tt.shape[0] == i.shape[0], f"turntable and pose images must have the same height, was not the case for {img}: {tt.shape[0]} != {i.shape[0]}" 46 | assert tt.shape[2] == i.shape[2], f"turntable and pose images must have the same number of color channels, was not the case for {img}: {tt.shape[2]} != {i.shape[2]}" 47 | 48 | pose = np.zeros((tt.shape[0], tt.shape[1] + i.shape[1], tt.shape[2])) 49 | pose[:, :tt.shape[1]] = tt 50 | pose[:, tt.shape[1]:] = i 51 | 52 | output = np.ones(pose.shape) 53 | output *= BG_COLOR 54 | for c in range(3): 55 | output[:, :tt.shape[1], c] = (1-source[:,:,3]/255)*output[:,:tt.shape[1],c] + (source[:,:,3]/255)*source[:,:,c] 56 | 57 | if first: 58 | source_g = output[:, :tt.shape[1]] 59 | cv2.imwrite(SOURCE_G_PATH, source_g) 60 | first = False 61 | 62 | folder = f'{WORKSPACE}/{iter_name}/{img.replace(".png", "")}' 63 | if not os.path.exists(folder): os.makedirs(folder) 64 | 65 | cv2.imwrite(f"{folder}/{POSE_IMG_NAME}", pose) 66 | cv2.imwrite(f"{folder}/{INPUT_IMG_NAME}", output) 67 | with open(f"{folder}/{DATA_NAME}", "w") as f: 68 | f.write(json.dumps({ "start": tt.shape[1], "end": tt.shape[1] + i.shape[1] })) 69 | 70 | with open(f"{WORKSPACE}/{iter_name}/{TOUCH_FILE_NAME}", "w") as f: 71 | f.write("_") 72 | -------------------------------------------------------------------------------- /interpolate.py: -------------------------------------------------------------------------------- 1 | ################################################## 2 | # # 3 | # Interpolates between frames that have been # 4 | # cleaned (with background reinstated) # 5 | # # 6 | ################################################## 7 | 8 | # This code has been adapted from: https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf_hub_film_example.ipynb 9 | 10 | import tensorflow as tf 11 | import tensorflow_hub as hub 12 | import numpy as np 13 | 14 | model = hub.load("https://tfhub.dev/google/film/1") 15 | 16 | _UINT8_MAX_F = float(np.iinfo(np.uint8).max) 17 | 18 | def load_image(img_url: str): 19 | """Returns an image with shape [height, width, num_channels], with pixels in [0..1] range, and type np.float32.""" 20 | image_data = tf.io.read_file(img_url) 21 | image = tf.io.decode_image(image_data, channels=3) 22 | image_numpy = tf.cast(image, dtype=tf.float32).numpy() 23 | return image_numpy / _UINT8_MAX_F 24 | 25 | 26 | """A wrapper class for running a frame interpolation based on the FILM model on TFHub 27 | 28 | Usage: 29 | interpolator = Interpolator() 30 | result_batch = interpolator(image_batch_0, image_batch_1, batch_dt) 31 | Where image_batch_1 and image_batch_2 are numpy tensors with TF standard 32 | (B,H,W,C) layout, batch_dt is the sub-frame time in range [0..1], (B,) layout. 33 | """ 34 | 35 | def _pad_to_align(x, align): 36 | """Pads image batch x so width and height divide by align. 37 | 38 | Args: 39 | x: Image batch to align. 40 | align: Number to align to. 41 | 42 | Returns: 43 | 1) An image padded so width % align == 0 and height % align == 0. 44 | 2) A bounding box that can be fed readily to tf.image.crop_to_bounding_box 45 | to undo the padding. 46 | """ 47 | # Input checking. 48 | assert np.ndim(x) == 4 49 | assert align > 0, 'align must be a positive number.' 50 | 51 | height, width = x.shape[-3:-1] 52 | height_to_pad = (align - height % align) if height % align != 0 else 0 53 | width_to_pad = (align - width % align) if width % align != 0 else 0 54 | 55 | bbox_to_pad = { 56 | 'offset_height': height_to_pad // 2, 57 | 'offset_width': width_to_pad // 2, 58 | 'target_height': height + height_to_pad, 59 | 'target_width': width + width_to_pad 60 | } 61 | padded_x = tf.image.pad_to_bounding_box(x, **bbox_to_pad) 62 | bbox_to_crop = { 63 | 'offset_height': height_to_pad // 2, 64 | 'offset_width': width_to_pad // 2, 65 | 'target_height': height, 66 | 'target_width': width 67 | } 68 | return padded_x, bbox_to_crop 69 | 70 | 71 | class Interpolator: 72 | """A class for generating interpolated frames between two input frames. 73 | 74 | Uses the Film model from TFHub 75 | """ 76 | 77 | def __init__(self, align: int = 64) -> None: 78 | """Loads a saved model. 79 | 80 | Args: 81 | align: 'If >1, pad the input size so it divides with this before 82 | inference.' 83 | """ 84 | self._model = hub.load("https://tfhub.dev/google/film/1") 85 | self._align = align 86 | 87 | def __call__(self, x0: np.ndarray, x1: np.ndarray, dt: np.ndarray) -> np.ndarray: 88 | """Generates an interpolated frame between given two batches of frames. 89 | 90 | All inputs should be np.float32 datatype. 91 | 92 | Args: 93 | x0: First image batch. Dimensions: (batch_size, height, width, channels) 94 | x1: Second image batch. Dimensions: (batch_size, height, width, channels) 95 | dt: Sub-frame time. Range [0,1]. Dimensions: (batch_size,) 96 | 97 | Returns: 98 | The result with dimensions (batch_size, height, width, channels). 99 | """ 100 | if self._align is not None: 101 | x0, bbox_to_crop = _pad_to_align(x0, self._align) 102 | x1, _ = _pad_to_align(x1, self._align) 103 | 104 | inputs = {'x0': x0, 'x1': x1, 'time': dt[..., np.newaxis]} 105 | result = self._model(inputs, training=False) 106 | image = result['image'] 107 | 108 | if self._align is not None: 109 | image = tf.image.crop_to_bounding_box(image, **bbox_to_crop) 110 | return image.numpy() 111 | 112 | 113 | 114 | 115 | #################### 116 | # === NEW CODE === # 117 | #################### 118 | interp = Interpolator() 119 | def run_interp(frame1: np.ndarray, frame2: np.ndarray): 120 | time = np.full(shape=(1,), fill_value=0.5, dtype=np.float32) 121 | return interp(np.expand_dims(frame1, axis=0), np.expand_dims(frame2, axis=0), time)[0] 122 | 123 | from settings import * 124 | 125 | import cv2 126 | import os 127 | 128 | def raw_check(filename): 129 | return not filename.endswith("_s.png") and filename != SHEET_NAME and filename.endswith('.png') and 'interp' not in filename 130 | 131 | ITERATIONS = 2 # This will run 2^ITERATIONS times, (4 iterations = 16 new images per input pair) 132 | LOOP_INTERP = True 133 | interp_folder = f"{CLEAN_PATH}" 134 | def to_interpolate(filename): 135 | return True 136 | # return "srun" in filename 137 | # return "frun" in filename 138 | # return "brun" in filename 139 | 140 | assert os.path.exists(interp_folder), f"The interp folder does not exist: '{interp_folder}'" 141 | 142 | files = [f for f in os.listdir(interp_folder) if raw_check(f) and to_interpolate(f)] 143 | 144 | assert len(files) > 1, f"Found {len(files)} files in interp_folder, expected >= 2" 145 | print(f"Interpolating between: {files}") 146 | 147 | all_imgs = [load_image(f"{interp_folder}/{f}") for f in files] 148 | 149 | for pair_i in range(len(files) - (0 if LOOP_INTERP else 1)): 150 | curr_imgs = [ all_imgs[pair_i], all_imgs[pair_i + 1 if pair_i+1 0, "could not find an iter folder, run generation.py to first create a workspace" 29 | 30 | if not os.path.exists(next_folder): 31 | os.makedirs(next_folder) 32 | 33 | if KEEP_TURNTABLE: 34 | assert os.path.exists(SOURCE_G_PATH), f"turntable source image not found, expected file at '{SOURCE_G_PATH}'" 35 | assert os.path.exists(TURNTABLE_PATH), f"turntable pose image not found, expected file at '{TURNTABLE_PATH}'" 36 | turntable_pair = [ cv2.imread(SOURCE_G_PATH), cv2.imread(TURNTABLE_PATH) ] 37 | 38 | img_pose_pairs = {} 39 | 40 | root_folders = [f for f in os.listdir(curr_folder) if f != TOUCH_FILE_NAME] 41 | exclusion_list = [DATA_JSON_NAME, INPUT_IMG_NAME, POSE_IMG_NAME] 42 | 43 | for root_folder in root_folders: 44 | root = f"{curr_folder}/{root_folder}" 45 | 46 | data_path = f"{root}/{DATA_JSON_NAME}" 47 | pose_path = f"{POSES_FOLDER}/{root_folder}.png" 48 | input_paths = [f for f in os.listdir(root) if f not in exclusion_list] 49 | assert len(input_paths) == 1, f"expected 1 extra file in {root_folder}, got {len(input_paths)}: {input_paths}" 50 | input_path = f"{root}/{input_paths[0]}" 51 | 52 | assert os.path.exists(data_path), f"could not find data_path for {root_folder}, searched {data_path}" 53 | assert os.path.exists(pose_path), f"could not find pose_path for {root_folder}, searched {pose_path}" 54 | assert os.path.exists(input_path), f"could not find input_path for {root_folder}, searched {input_path}" 55 | 56 | with open(data_path) as f: 57 | data = json.loads(f.read()) 58 | assert "start" in data, f"expected 'start' key in data json file for {root_folder} at {data_path}" 59 | assert "end" in data, f"expected 'end' key in data json file for {root_folder} at {data_path}" 60 | 61 | input = cv2.imread(input_path) 62 | assert input is not None, f"input img was None for '{input_path}', expected value" 63 | assert input.shape[1] >= data["end"], f"input img must have width of at least data.json['end'], found {input.shape[1]} < {data['end']}" 64 | input = input[:,data["start"]:data["end"]] 65 | 66 | pose = cv2.imread(pose_path) 67 | assert pose is not None, f"pose img was None for '{pose_path}', expected value" 68 | 69 | assert input.shape[0] == pose.shape[0], f"input and pose need to have height, got {input.shape[0]} != {pose.shape[0]}" 70 | assert input.shape[1] == pose.shape[1], f"input and pose need to have width, got {input.shape[1]} != {pose.shape[1]}" 71 | assert input.shape[2] == pose.shape[2], f"input and pose need to have color depth, got {input.shape[2]} != {pose.shape[2]}" 72 | 73 | img_pose_pairs[root_folder] = [input, pose] 74 | 75 | keys = list(img_pose_pairs.keys()) 76 | if RANDOMIZE_ORDER: 77 | random.seed(curr_folder + next_folder) 78 | random.shuffle(keys) 79 | print(f"keys order: {keys}") 80 | 81 | assert type(IMGS_LEFT) == type(0), f"settings.IMGS_LEFT must be of type int, got {type(IMGS_LEFT)}" 82 | assert type(IMGS_RIGHT) == type(0), f"settings.IMGS_RIGHT must be of type int, got {type(IMGS_RIGHT)}" 83 | assert IMGS_LEFT >= 0, f"settings.IMGS_LEFT must be non-negative, got {IMGS_LEFT}" 84 | assert IMGS_RIGHT >= 0, f"settings.IMGS_RIGHT must be non-negative, got {IMGS_RIGHT}" 85 | 86 | def wrap(x): 87 | w = len(keys) 88 | while x >= w: x -= w 89 | while x < 0: x += w 90 | return x 91 | 92 | def get_pair(i): 93 | return img_pose_pairs[keys[wrap(i)]] 94 | 95 | for i in range(len(keys)): 96 | pairs_to_use = [] 97 | 98 | if KEEP_TURNTABLE: pairs_to_use.append(turntable_pair) 99 | for d in range(-IMGS_LEFT, 0): pairs_to_use.append(get_pair(i + d)) 100 | pairs_to_use.append(get_pair(i)) 101 | for d in range(1, IMGS_RIGHT + 1): pairs_to_use.append(get_pair(i + d)) 102 | 103 | x = 0 104 | output_img = np.zeros((pairs_to_use[0][0].shape[0], sum([p[0].shape[1] for p in pairs_to_use]), pairs_to_use[0][0].shape[2])) 105 | output_pose = np.zeros(output_img.shape) 106 | for img, pose in pairs_to_use: 107 | assert img.shape[0] == output_img.shape[0], f"found mismatched image height when combining, found {img.shape[0]} != {output_img.shape[0]}" 108 | dx = img.shape[1] 109 | output_img[ :,x:x+dx] = img 110 | output_pose[:,x:x+dx] = pose 111 | x += dx 112 | assert x == output_img.shape[1], f"FATAL: expected shifted x to equal output width, found {x} != {output_img.shape[1]}" 113 | 114 | target_index = (1 if KEEP_TURNTABLE else 0) + IMGS_LEFT 115 | data = { 116 | "start": sum([p[0].shape[1] for p in pairs_to_use[:target_index ]]), 117 | "end": sum([p[0].shape[1] for p in pairs_to_use[:target_index+1]]), 118 | } 119 | 120 | folder = f"{next_folder}/{keys[i]}" 121 | if not os.path.exists(folder): os.mkdir(folder) 122 | cv2.imwrite(f"{folder}/{INPUT_IMG_NAME}", output_img) 123 | cv2.imwrite(f"{folder}/{POSE_IMG_NAME}", output_pose) 124 | with open(f"{folder}/{DATA_JSON_NAME}", "w") as f: 125 | f.write(json.dumps(data)) 126 | 127 | with open(f"{next_folder}/{TOUCH_FILE_NAME}", "w") as f: 128 | f.write("_") 129 | 130 | -------------------------------------------------------------------------------- /prompt.txt: -------------------------------------------------------------------------------- 1 | pos: , (mulitple view of the same character and same outfits:1.0), character reference sheet, side view, front view, %%PROMPT%%, high fantasy, dnd, solid gray background, same character, identical characters, identical outfit, identical hair, identical face, multiple of the same person 2 | neg: text, watermark, logo, blurry, weapon, sword, shield, holding things, items, spears, blades, clubs, handles, lighting, edge lighting, harsh lighting, deformed face, sexy, skimpy, revealing clothes, (multiple legs:1.2), (multiple limbs:1.2) 3 | 4 | 5 | %%PROMPT%% examples: 6 | female necromancer, short and slight with pale skin and jet black hair styled in intricate braids, wearing dark flowing robes 7 | male warlock, tall and gaunt with pale skin and dark, piercing eyes, dressed in black robes 8 | female paladin, short and muscular with short cropped blonde hair, wearing shiny plate armor 9 | male wizard, tall and thin with a prominent nose and long white beard, draped in flowing blue robes adorned with intricate silver symbols 10 | female druid, petite and agile with piercing blue eyes and wild, curly hair, wearing a cloak made of leaves 11 | male fighter, broad-shouldered and muscular with short, cropped hair and a rugged face, dressed in heavy plate armor -------------------------------------------------------------------------------- /settings.py: -------------------------------------------------------------------------------- 1 | WORKSPACE = "workspaces/000" 2 | 3 | # VRAM sensitive constants 4 | KEEP_TURNTABLE = True 5 | IMGS_LEFT = 0 6 | IMGS_RIGHT = 1 7 | RANDOMIZE_ORDER = False 8 | 9 | BG_COLOR = (127, 127, 127) # in BlueGreenRed, not RedGreenBlue! 10 | 11 | ITER_PREFIX = "iter" 12 | DATA_NAME = "data.json" 13 | 14 | POSE_IMG_NAME = "pose.png" 15 | INPUT_IMG_NAME = "input.png" 16 | DATA_JSON_NAME = "data.json" 17 | TOUCH_FILE_NAME = ".created" 18 | 19 | SOURCE_NAME = "source.png" 20 | SOURCE_PATH = f"{WORKSPACE}/{SOURCE_NAME}" 21 | SOURCE_S_NAME = "source_s.png" 22 | SOURCE_S_PATH = f"{WORKSPACE}/{SOURCE_S_NAME}" 23 | SOURCE_G_NAME = "source_g.png" 24 | SOURCE_G_PATH = f"{WORKSPACE}/{SOURCE_G_NAME}" 25 | 26 | POSES_NAME = "poses" 27 | POSES_FOLDER = f"{WORKSPACE}/{POSES_NAME}" 28 | TURNTABLE_NAME = "turntable.png" 29 | TURNTABLE_PATH = f"{POSES_FOLDER}/{TURNTABLE_NAME}" 30 | 31 | EXTRACT_FOLDER = "extracted" 32 | EXTRACT_PATH = f"{WORKSPACE}/{EXTRACT_FOLDER}" 33 | CLEAN_FOLDER = "cleaned" 34 | CLEAN_PATH = f"{WORKSPACE}/{CLEAN_FOLDER}" 35 | SHEET_NAME = "sheet.png" 36 | SHEET_PATH = f"{CLEAN_PATH}/{SHEET_NAME}" -------------------------------------------------------------------------------- /spritesheet.py: -------------------------------------------------------------------------------- 1 | ################################################## 2 | # # 3 | # Removes background for all files without a # 4 | # BG-less version and creates spritesheet # 5 | # # 6 | ################################################## 7 | 8 | from settings import * 9 | 10 | import math 11 | import subprocess 12 | import numpy as np 13 | import cv2 14 | import os 15 | 16 | SKIP_EXISTING = True 17 | MAX_WIDTH = 16384 18 | RESIZE = 1 19 | 20 | def check_file(file): 21 | return file.endswith("_s.png") or file == SHEET_NAME or not file.endswith('.png') 22 | 23 | imgs = [] 24 | 25 | raw_files = [f for f in os.listdir(CLEAN_PATH) if not check_file(f)] 26 | for i in range(32): 27 | files = [] 28 | hold = -1 29 | for i in range(len(raw_files)): 30 | if hold < 0: 31 | if i + 1 >= len(raw_files): 32 | files.append(raw_files[i]) 33 | continue 34 | 35 | t1, t2 = raw_files[i].replace(".png", ""), raw_files[i + 1].replace(".png", "") 36 | if t1 in t2 or t2 in t1: 37 | cmp = len(t2) - len(t1) 38 | # print(f"1 comparing {t1} and {t2}, value {cmp}") 39 | if cmp < 0: 40 | hold = i 41 | files.append(raw_files[i + 1]) 42 | else: 43 | files.append(raw_files[i]) 44 | else: 45 | files.append(raw_files[i]) 46 | else: 47 | if i + 1 >= len(raw_files): 48 | files.append(raw_files[hold]) 49 | continue 50 | 51 | t1, t2 = raw_files[hold].replace(".png", ""), raw_files[i + 1].replace(".png", "") 52 | if t1 in t2 or t2 in t1: 53 | cmp = len(t2) - len(t1) 54 | # print(f"2 comparing {t1} and {t2}, value {cmp}") 55 | if cmp < 0: 56 | files.append(raw_files[i + 1]) 57 | else: 58 | files.append(raw_files[hold]) 59 | hold = -1 60 | else: 61 | files.append(raw_files[hold]) 62 | hold = -1 63 | raw_files = files 64 | 65 | 66 | for file in files: 67 | 68 | print(file) 69 | 70 | clean_img_path = f"{CLEAN_PATH}/{file}" 71 | stripped_img_path = f"{CLEAN_PATH}/{file.replace('.png', '_s.png')}" 72 | 73 | if not SKIP_EXISTING or not os.path.exists(stripped_img_path): 74 | subprocess.call(["rembg", "i", clean_img_path, stripped_img_path]) 75 | 76 | input_img = cv2.imread(stripped_img_path, cv2.IMREAD_UNCHANGED) 77 | imgs.append(cv2.resize(input_img, (int(input_img.shape[1]*RESIZE), int(input_img.shape[0]*RESIZE)))) 78 | 79 | row_count = 1 80 | row_size = len(imgs) 81 | h, w = imgs[0].shape[0], sum([img.shape[1] for img in imgs]) 82 | dy = h 83 | 84 | new_w = w 85 | while new_w > MAX_WIDTH: 86 | row_count *= 2 87 | row_size = math.ceil(len(imgs) / float(row_count)) 88 | new_w = row_size * imgs[0].shape[0] 89 | h, w = h * row_count, new_w 90 | 91 | output_img = np.zeros((h, w, imgs[0].shape[2])) 92 | img_i = 0 93 | for row_i in range(row_count): 94 | x = 0 95 | for col_i in range(row_size): 96 | img = imgs[img_i] 97 | dx = img.shape[1] 98 | 99 | output_img[dy*row_i:dy*(row_i+1),x:x+dx] = img 100 | 101 | x += dx 102 | img_i += 1 103 | if img_i >= len(imgs): break 104 | if img_i >= len(imgs): break 105 | 106 | cv2.imwrite(SHEET_PATH, output_img) --------------------------------------------------------------------------------