├── README.md
├── media
├── SuGaR_x_Frosting_Logo_dark.png
└── tuto_images
│ ├── add_sugar_mesh_panel.PNG
│ ├── mesh_added.PNG
│ ├── mesh_editing.PNG
│ ├── render.png
│ ├── render_panel.PNG
│ └── scene_in_supersplat.png
└── sugar_addon.py
/README.md:
--------------------------------------------------------------------------------
1 |
18 |
19 |
20 | Using our methods SuGaR (CVPR 2024) or Gaussian Frosting (ECCV 2024), you can reconstruct editable meshes from RGB images or RGB videos, and render them using realistic Gaussian Splatting rendering.
21 | We now provide a Blender Add-On to edit, sculpt, combine or animate 3D scenes reconstructed with SuGaR of Frosting and render them with Gaussian Splatting without a single line of code.
22 |
23 |
24 |
25 | Installation
26 |
27 | 1. Please start by cloning this repo.
28 |
29 | 2. Then, install Blender (version 4.0.2 is recommended but not mandatory).
30 |
31 | 3. Open Blender, and go to `Edit` > `Preferences` > `Add-ons` > `Install`, and select the file `sugar_addon.py` located in this repo.
32 |
33 | You have now installed the SuGaR x Frosting addon for Blender!
34 |
35 | Usage
36 |
37 | 1. Please start by installing either SuGaR or Gaussian Frosting from the respective repositories.
38 |
39 | 2. Follow the instructions from the repositories to optimize a SuGaR or Frosting model.
40 |
41 | 3. Open a new scene in Blender, and go to the `Render` tab in the Properties. You should see a panel named `Add SuGaR or Frosting mesh`. The panel is not necessary at the top of the tab, so you may need to scroll down to find it.
42 |
43 |
44 |
45 | Use the `Add SuGaR or Frosting mesh` panel to load a mesh
reconstructed with SuGaR or Frosting in Blender.
46 |
47 |
49 |
50 | 4. **(a) Select a mesh.** Enter the path to the final mesh extracted from SuGaR or Frosting in the `Path to OBJ file` field. You can also click on the folder icon to select the file. The mesh should be located in `SuGaR/output/refined_mesh/` or in `Frosting/output/refined_frosting_base_mesh/` depending on the model you used.
51 | **(b) Select a checkpoint.** Similarly, enter the path to the final checkpoint of the optimization in the `Path to PT file` field. You can also click on the folder icon to select the file. The checkpoint should be located in `SuGaR/output/refined/` or in `Frosting/output/refined_frosting/` depending on the model you used.
52 | **(c) Load the mesh.** Finally, click on `Add mesh` to load the mesh in Blender. Feel free to rotate the mesh and change the shading mode to better visualize the mesh and its colors.
53 |
54 |
55 |
56 |
57 | Please rotate the mesh and change the shading mode to get a better view of the mesh.
58 |
59 |
61 |
62 | 5. **Now, feel free to edit your mesh using Blender!**
63 |
You can segment it into different pieces, sculpt it, rig it, animate it using a parent armature, *etc*. You can also add other SuGaR or Frosting meshes to the scene, and combine elements from different scenes.
64 | Feel free to set a camera in the scene and prepare an animation: You can animate the camera, the mesh, *etc*.
65 | Please avoid using `Apply Location`, `Apply Rotation`, or `Apply Scale` on the edited mesh, as I am still not sure how it will affect the correspondence between the mesh and the optimized checkpoint.
66 |
67 |
68 |
69 |
70 | You can edit the mesh as you want, and prepare an animation with it.
71 | You can also add other SuGaR or Frosting meshes to the scene.
72 |
73 |
75 |
76 | 6. Once you're done with your editing, you can prepare a rendering package ready to be rendered with SuGaR or Frosting. To do so, go to the `Render` tab in the Properties again, and select the main directory of your model in the `Path to SuGaR/Frosting directory` field.
77 | If you installed the repo of SuGaR or Frosting from GitHub, this directory should be named either `SuGaR` or `Frosting`.
78 | Finally, click on `Render Image` or `Render Animation` to render the scene.
79 |
80 |
81 |
82 |
83 | In this example, we first animated the camera and the character.
84 | Then, we added another Frosting mesh to get a background for our character.
85 |
86 |
88 |
89 | `Render Image` will render a single image of the scene, with the current camera position and mesh editions/poses.
90 | `Render Animation` will render a full animation of the scene, from the first frame to the last frame you set in the Blender Timeline.
91 |
92 | The package should be located in `/blender/packages/`.
93 |
94 | 7. Finally, you can render the package with SuGaR or Frosting. You just need to go to the root directory of the SuGaR or Frosting repo and run the following command:
95 | ```shell
96 | python render_blender_scene.py -p
97 | ```
98 |
99 |
100 |
101 |
102 | With SuGaR or Frosting, you can get a realistic, high-quality rendering of your scene!
103 |
104 |
106 |
107 | Please check the documentation of SuGaR or Frosting and the `render_blender_scene.py` scripts for more information on the additional arguments.
108 | If you get artifacts in the rendering, you can try to switch the automatic adjustment method of the Gaussians from 'complex' to 'simple', as the simple method is less accurate but faster and more robust to extreme deformations:
109 | ```shell
110 | python render_blender_scene.py -p --adaptation_method simple
111 | ```
112 | If you notice that some Gaussians are not rendered (especially if you perform extreme deformations on the meshes), you can also change the `--deformation_threshold` argument. All Gaussians belonging to triangles with a deformation factor higher than this threshold will be culled during rendering. The default value is 2., but you can try to increase it to 5. or 10:
113 | ```shell
114 | python render_blender_scene.py -p --deformation_threshold 10
115 | ```
116 |
117 | 8. Instead of rendering the frames, you can also export a PLY file of the Frosting representation at a specific frame. This PLY file can be used to visualize the Frosting representation in any 3D Gaussian Splatting viewer, such as SuperSplat. To do so, add the argument `--export_frame_as_ply` followed by the frame number you want to export. For example, for exporting a PLY file of the representation at frame 10:
118 | ```shell
119 | python render_blender_scene.py -p --export_frame_as_ply 10
120 | ```
121 |
122 |
123 |
124 |
125 | You can visualize the edited SuGaR or Frosting representation at a specific frame
126 | in any 3D Gaussian Splatting viewer, such as SuperSplat!
127 |
128 |
--------------------------------------------------------------------------------
/media/SuGaR_x_Frosting_Logo_dark.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/SuGaR_x_Frosting_Logo_dark.png
--------------------------------------------------------------------------------
/media/tuto_images/add_sugar_mesh_panel.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/add_sugar_mesh_panel.PNG
--------------------------------------------------------------------------------
/media/tuto_images/mesh_added.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/mesh_added.PNG
--------------------------------------------------------------------------------
/media/tuto_images/mesh_editing.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/mesh_editing.PNG
--------------------------------------------------------------------------------
/media/tuto_images/render.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/render.png
--------------------------------------------------------------------------------
/media/tuto_images/render_panel.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/render_panel.PNG
--------------------------------------------------------------------------------
/media/tuto_images/scene_in_supersplat.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Anttwo/sugar_frosting_blender_addon/229afc5cbe178b7964e1ac4cf7ea97373fb29f4b/media/tuto_images/scene_in_supersplat.png
--------------------------------------------------------------------------------
/sugar_addon.py:
--------------------------------------------------------------------------------
1 | bl_info = {
2 | "name": "SuGaR x Frosting Editing Tool",
3 | "author": "Antoine Guedon",
4 | "blender": (4, 0, 2),
5 | "description": "Can be used to import and edit meshes reconstructed with SuGaR or Frosting, as well as to render edited scenes with SuGaR or Frosting.",
6 | }
7 |
8 | import os
9 | import json
10 | import numpy as np
11 | import bpy
12 | from bpy_extras.io_utils import ExportHelper, ImportHelper
13 |
14 |
15 | def get_mesh_vertex_idx(mesh):
16 | vert_idx_values = np.zeros(len(mesh.vertices), dtype=int)
17 | # vert_idx_values = np.empty(len(mesh.vertices), dtype=int)
18 | mesh.attributes['index'].data.foreach_get("value", vert_idx_values)
19 | return vert_idx_values
20 |
21 |
22 | def get_mesh_vertex_xyz(mesh):
23 | vert_xyz = np.zeros((len(mesh.vertices), 3)).flatten()
24 | mesh.attributes['position'].data.foreach_get("vector", vert_xyz)
25 | return vert_xyz.reshape(-1, 3)
26 |
27 |
28 | def get_mesh_vertex_metadata(mesh):
29 | vert_idx_values = np.zeros(len(mesh.vertices), dtype=int)
30 | mesh.attributes['metadata'].data.foreach_get("value", vert_idx_values)
31 | return vert_idx_values
32 |
33 |
34 | def get_text(text_data):
35 | return text_data.body
36 |
37 |
38 | def set_text(text_data, new_text:str):
39 | text_data.body = new_text
40 |
41 |
42 | def get_sugar_metadata(metadata_name:str):
43 | metadata_object = bpy.data.objects[metadata_name]
44 | metadata = {}
45 | for child in metadata_object.children:
46 | metadata_text = get_text(child.data)
47 | metadata_dict = {}
48 | metadata_parsed_txt = metadata_text.split(';')
49 | for parsed_txt in metadata_parsed_txt:
50 | metadata_dict[parsed_txt.split(':::')[0]] = parsed_txt.split(':::')[-1]
51 | metadata_idx = str(child.name)
52 | metadata[metadata_idx] = metadata_dict
53 | return metadata
54 |
55 |
56 | def is_sugar_mesh(mesh):
57 | return ('metadata' in mesh.attributes)
58 |
59 |
60 | def is_windows_path(path:str):
61 | if '\\' in path:
62 | return True
63 |
64 |
65 | def convert_path_to_linux(path:str):
66 | return path.replace('\\\\', '\\').replace('\\', '/')
67 |
68 |
69 | def get_matrix_world(obj):
70 | return [[obj.matrix_world[i][j] for j in range(4)] for i in range(4)]
71 |
72 |
73 | class QueryProps(bpy.types.PropertyGroup):
74 | # env: bpy.props.StringProperty(
75 | # name="Conda env name", default="sugar",
76 | # description='Conda environment to use for running SuGaR or Frosting',
77 | # )
78 |
79 | n_checkpoints: bpy.props.IntProperty(name="Number of checkpoints", default=5, min=1, max=10)
80 |
81 | sugar_dir: bpy.props.StringProperty(
82 | name="Path to SuGaR or Frosting directory",
83 | default="./sugar",
84 | description="Path to the directory cloned from SuGaR's or Frosting's repo",
85 | )
86 |
87 | # output_dir: bpy.props.StringProperty(
88 | # name="Path to output directory",
89 | # default="./output",
90 | # description="Path to the output directory in which rendered images will be saved",
91 | # )
92 |
93 | mesh_file_to_load: bpy.props.StringProperty(
94 | name="Path to OBJ file",
95 | default="",
96 | description="Path to the mesh file (in OBJ format) to load",
97 | )
98 |
99 | checkpoint_to_load: bpy.props.StringProperty(
100 | name="Path to PT file",
101 | default="",
102 | description="Path to the checkpoint file (in PT format) to use for rendering",
103 | )
104 |
105 |
106 | class WMSuGaRSelector(bpy.types.Operator, ImportHelper):
107 | """Select SuGaR or Frosting directory"""
108 | bl_idname = "something.sugar_selector"
109 | bl_label = "Select SuGaR or Frosting folder"
110 |
111 | filename: bpy.props.StringProperty()
112 | filter_glob: bpy.props.StringProperty(
113 | default="",
114 | options={'HIDDEN'},
115 | )
116 |
117 | def execute(self, context):
118 | fdir = self.properties.filepath
119 | bpy.context.scene.QueryProps.sugar_dir = fdir
120 | return{'FINISHED'}
121 |
122 | def invoke(self, context, event):
123 | self.filename = ""
124 | wm = context.window_manager.fileselect_add(self)
125 | return {'RUNNING_MODAL'}
126 |
127 |
128 | # class WMOutputSelector(bpy.types.Operator, ImportHelper):
129 | # """Select output directory"""
130 | # bl_idname = "something.output_selector"
131 | # bl_label = "Select output folder"
132 |
133 | # filename: bpy.props.StringProperty()
134 | # filter_glob: bpy.props.StringProperty(
135 | # default="",
136 | # options={'HIDDEN'},
137 | # )
138 |
139 | # def execute(self, context):
140 | # fdir = self.properties.filepath
141 | # bpy.context.scene.QueryProps.output_dir = fdir
142 | # return{'FINISHED'}
143 |
144 | # def invoke(self, context, event):
145 | # self.filename = ""
146 | # wm = context.window_manager.fileselect_add(self)
147 | # return {'RUNNING_MODAL'}
148 |
149 |
150 | class WMSuGaRMeshSelector(bpy.types.Operator, ImportHelper):
151 | """Select mesh file"""
152 | bl_idname = "something.mesh_selector"
153 | bl_label = "Select mesh file"
154 |
155 | filename_ext = ".obj"
156 | filter_glob: bpy.props.StringProperty(
157 | default="*.obj",
158 | options={'HIDDEN'},
159 | )
160 |
161 | def execute(self, context):
162 | fdir = self.properties.filepath
163 | bpy.context.scene.QueryProps.mesh_file_to_load = fdir
164 | return{'FINISHED'}
165 |
166 |
167 | class WMSuGaRCheckpointSelector(bpy.types.Operator, ImportHelper):
168 | """Select checkpoint file"""
169 | bl_idname = "something.checkpoint_selector"
170 | bl_label = "Select checkpoint file"
171 |
172 | filename_ext = ".pt"
173 | filter_glob: bpy.props.StringProperty(
174 | default="*.pt",
175 | options={'HIDDEN'},
176 | )
177 |
178 | def execute(self, context):
179 | fdir = self.properties.filepath
180 | bpy.context.scene.QueryProps.checkpoint_to_load = fdir
181 | return{'FINISHED'}
182 |
183 |
184 | def create_render_package(
185 | query_props,
186 | sugar_metadata,
187 | start_frame,
188 | end_frame,
189 | just_render_current_screen=False,
190 | ):
191 | # Camera object
192 | camera_data = {}
193 | camera_data['matrix_world'] = []
194 | camera_data['lens'] = []
195 | camera_data['angle'] = []
196 | camera_data['angle_x'] = []
197 | camera_data['angle_y'] = []
198 | camera_data['clip_start'] = []
199 | camera_data['clip_end'] = []
200 | cam_obj = bpy.context.scene.camera
201 |
202 | camera_data['image_width'] = bpy.context.scene.render.resolution_x
203 | camera_data['image_height'] = bpy.context.scene.render.resolution_y
204 | print(f"Output images have resolution: {camera_data['image_width']} x {camera_data['image_height']}")
205 |
206 | # Create Meshes data
207 | meshes_data = []
208 | n_mesh_owners = 0
209 | n_posable_mesh_owners = 0
210 | mesh_owners = []
211 | posable_mesh_owners = []
212 | for ob in bpy.data.objects:
213 | if (ob.type == 'MESH') and (not ob.hide_render) and is_sugar_mesh(ob.data):
214 | n_mesh_owners += 1
215 | mesh_owners.append(ob)
216 | if ob.parent and ob.parent.type=='ARMATURE':
217 | n_posable_mesh_owners += 1
218 | posable_mesh_owners.append(ob)
219 | plural_print = '' if n_mesh_owners<=1 else 'es'
220 | print(f"\n{n_mesh_owners} SuGaR/Frosting mesh{plural_print} detected in the scene,")
221 | plural_print = '' if n_posable_mesh_owners<=1 else 'es'
222 | print(f"including {n_posable_mesh_owners} posable SuGaR/Frosting mesh{plural_print}.")
223 |
224 | for i_mesh in range(n_mesh_owners):
225 | mesh = mesh_owners[i_mesh].data
226 | mesh_metadata = get_mesh_vertex_metadata(mesh)
227 | mesh_metadata_dict = sugar_metadata[str(mesh_metadata[0])]
228 | mesh_name = mesh_metadata_dict['mesh_name']
229 | checkpoint_name = mesh_metadata_dict['checkpoint_name']
230 | mesh_data = {
231 | 'mesh_name': mesh_name,
232 | 'checkpoint_name': checkpoint_name,
233 | 'matrix_world': np.array(mesh_owners[i_mesh].matrix_world).tolist(),
234 | 'xyz': get_mesh_vertex_xyz(mesh).tolist(),
235 | 'idx': get_mesh_vertex_idx(mesh).tolist(),
236 | 'metadata': get_mesh_vertex_metadata(mesh).tolist(),
237 | }
238 | meshes_data.append(mesh_data)
239 |
240 | # Create rest bones data
241 | bones_data = []
242 | for i_mesh in range(n_mesh_owners):
243 | mesh_obj = mesh_owners[i_mesh]
244 | mesh = mesh_obj.data
245 | vertices = mesh.vertices
246 | if mesh_obj in posable_mesh_owners:
247 | armature_obj = mesh_obj.parent
248 | armature = armature_obj.data
249 | armature.pose_position = 'REST'
250 |
251 | # Armature data
252 | armature_dict = {}
253 | armature_dict['matrix_world'] = [[armature_obj.matrix_world[i][j] for j in range(4)] for i in range(4)]
254 | armature_dict['rest_bones'] = {}
255 | armature_dict['pose_bones'] = {}
256 | for bone in armature.bones:
257 | mat = armature_obj.matrix_world @ bone.matrix_local
258 | mat_list = [[mat[i][j] for j in range(4)] for i in range(4)]
259 | armature_dict['rest_bones'][bone.name] = mat_list
260 | armature_dict['pose_bones'][bone.name] = []
261 |
262 | # Vertex group data
263 | vertex_dict = {}
264 | vertex_dict['matrix_world'] = get_matrix_world(mesh_obj)
265 | vertex_dict['tpose_points'] = []
266 | vertex_dict['groups'] = []
267 | vertex_dict['weights'] = []
268 | vertex_group_names = {}
269 | for i in range(len(mesh_obj.vertex_groups)):
270 | group = mesh_obj.vertex_groups[i]
271 | vertex_group_names[str(group.index)] = group.name
272 | for i in range(len(vertices)):
273 | v = mesh_obj.matrix_world @ vertices[i].co
274 | vertex_dict['tpose_points'].append([v[0], v[1], v[2]])
275 | group_list = []
276 | weight_list = []
277 | for group in vertices[i].groups:
278 | group_list.append(vertex_group_names[str(group.group)])
279 | weight_list.append(group.weight)
280 | vertex_dict['groups'].append(group_list)
281 | vertex_dict['weights'].append(weight_list)
282 |
283 | armature.pose_position = 'POSE'
284 |
285 | bones_dict = {
286 | "armature": armature_dict,
287 | "vertex": vertex_dict
288 | }
289 | else:
290 | bones_dict = None
291 | bones_data.append(bones_dict)
292 |
293 | if just_render_current_screen:
294 | start_frame, end_frame = 1, 1
295 |
296 | for i_frame in range(start_frame, end_frame+1):
297 | # Set frame
298 | if not just_render_current_screen:
299 | bpy.context.scene.frame_set(i_frame)
300 |
301 | # Create camera data
302 | camera_data['matrix_world'].append([[cam_obj.matrix_world[i][j] for j in range(4)] for i in range(4)])
303 | camera_data['lens'].append(cam_obj.data.lens)
304 | camera_data['angle'].append(cam_obj.data.angle)
305 | camera_data['angle_x'].append(cam_obj.data.angle_x)
306 | camera_data['angle_y'].append(cam_obj.data.angle_y)
307 | camera_data['clip_start'].append(cam_obj.data.clip_start)
308 | camera_data['clip_end'].append(cam_obj.data.clip_end)
309 |
310 | # Create Bones data
311 | for i_mesh in range(n_mesh_owners):
312 | mesh_obj = mesh_owners[i_mesh]
313 | if mesh_obj in posable_mesh_owners:
314 | armature_obj = mesh_obj.parent
315 | pose = armature_obj.pose
316 | armature = armature_obj.data
317 | for bone in pose.bones:
318 | mat = armature_obj.matrix_world @ bone.matrix
319 | mat_list = [[mat[i][j] for j in range(4)] for i in range(4)]
320 | bones_data[i_mesh]['armature']['pose_bones'][bone.name].append(mat_list)
321 |
322 | # Build and save render package
323 | # TODO
324 | render_package = {
325 | 'camera': camera_data,
326 | 'meshes': meshes_data,
327 | 'bones': bones_data
328 | }
329 |
330 | return render_package
331 |
332 |
333 | class RenderSuGaROperator(bpy.types.Operator):
334 | """Render using SuGaR or Frosting"""
335 | bl_idname = "object.sugar_render"
336 | bl_label = "SuGaR Renderer"
337 | bl_options = {'REGISTER', 'UNDO'}
338 |
339 | def execute(self, context):
340 | query_props = bpy.context.scene.QueryProps
341 | print("\nStart rendering with SuGaR or Frosting...")
342 | # print("---> Conda env:", query_props.env)
343 | print("---> SuGaR/Frosting directory:", query_props.sugar_dir)
344 | # print("---> Output directory:", query_props.output_dir)
345 |
346 | sugar_metadata_name = "SuGaR x Frosting metadata (do not delete)"
347 | sugar_metadata = get_sugar_metadata(sugar_metadata_name)
348 | print("\nSuGaR/Frosting metadata:", sugar_metadata)
349 |
350 | scene = context.scene
351 | cursor = scene.cursor.location
352 | obj = context.active_object
353 |
354 | start_frame = bpy.context.scene.frame_start
355 | end_frame = bpy.context.scene.frame_end
356 | print("\nStart frame:", start_frame)
357 | print("End frame:", end_frame)
358 |
359 | # Output path (TO CHECK)
360 | scene_name = os.path.splitext(os.path.basename(bpy.data.filepath))[0] + '.json'
361 | # output_dir = os.path.join(query_props.output_dir, 'output', 'blender')
362 | # output_dir = query_props.output_dir
363 | package_output_dir = os.path.join(query_props.sugar_dir, 'output', 'blender', 'package')
364 | os.makedirs(package_output_dir, exist_ok=True)
365 | package_output_file_path = os.path.join(package_output_dir, scene_name)
366 |
367 | render_package = create_render_package(
368 | query_props,
369 | sugar_metadata,
370 | start_frame,
371 | end_frame,
372 | just_render_current_screen=False,
373 | )
374 |
375 | with open(package_output_file_path, "w") as outfile:
376 | json.dump(render_package, outfile)
377 | print(f'Results saved to "{package_output_file_path}".')
378 |
379 | return {'FINISHED'}
380 |
381 |
382 | class RenderSuGaROperatorSingleImage(bpy.types.Operator):
383 | """Render using SuGaR or Frosting"""
384 | bl_idname = "object.sugar_render_single"
385 | bl_label = "SuGaR Renderer"
386 | bl_options = {'REGISTER', 'UNDO'}
387 |
388 | def execute(self, context):
389 | query_props = bpy.context.scene.QueryProps
390 | print("\nStart rendering with SuGaR or Frosting...")
391 | # print("---> Conda env:", query_props.env)
392 | print("---> SuGaR/Frosting directory:", query_props.sugar_dir)
393 | # print("---> Output directory:", query_props.output_dir)
394 |
395 | sugar_metadata_name = "SuGaR x Frosting metadata (do not delete)"
396 | sugar_metadata = get_sugar_metadata(sugar_metadata_name)
397 | print("\nSuGaR/Frosting metadata:", sugar_metadata)
398 |
399 | scene = context.scene
400 | cursor = scene.cursor.location
401 | obj = context.active_object
402 |
403 | start_frame = bpy.context.scene.frame_start
404 | end_frame = bpy.context.scene.frame_end
405 | print("\nStart frame:", start_frame)
406 | print("End frame:", end_frame)
407 |
408 | # Output path (TO CHECK)
409 | scene_name = os.path.splitext(os.path.basename(bpy.data.filepath))[0] + '.json'
410 | # output_dir = os.path.join(query_props.output_dir, 'output', 'blender')
411 | # output_dir = query_props.output_dir
412 | package_output_dir = os.path.join(query_props.sugar_dir, 'output', 'blender', 'package')
413 | os.makedirs(package_output_dir, exist_ok=True)
414 | package_output_file_path = os.path.join(package_output_dir, scene_name)
415 |
416 | render_package = create_render_package(
417 | query_props,
418 | sugar_metadata,
419 | start_frame,
420 | end_frame,
421 | just_render_current_screen=True,
422 | )
423 |
424 | with open(package_output_file_path, "w") as outfile:
425 | json.dump(render_package, outfile)
426 | print(f'Results saved to "{package_output_file_path}".')
427 |
428 | return {'FINISHED'}
429 |
430 |
431 | class AddSuGaRMeshOperator(bpy.types.Operator):
432 | """Add a mesh reconstructed with SuGaR or Frosting to the scene"""
433 | bl_idname = "object.add_sugar_mesh"
434 | bl_label = "Add SuGaR Mesh"
435 | bl_options = {'REGISTER', 'UNDO'}
436 |
437 | def execute(self, context):
438 | sugar_metadata_name = "SuGaR x Frosting metadata (do not delete)"
439 |
440 | scene = context.scene
441 | cursor = scene.cursor.location
442 | obj = context.active_object
443 |
444 | # Create metadata object if needed
445 | if sugar_metadata_name in bpy.data.objects:
446 | metadata_object = bpy.data.objects[sugar_metadata_name]
447 | else:
448 | bpy.ops.object.empty_add(
449 | type='PLAIN_AXES',
450 | align='WORLD',
451 | location=(0, 0, 0),
452 | scale=(1, 1, 1)
453 | )
454 | metadata_object = bpy.context.active_object
455 | metadata_object.name = sugar_metadata_name
456 | metadata_object.hide_viewport = True
457 | metadata_object.hide_render = True
458 |
459 | query_props = bpy.context.scene.QueryProps
460 |
461 | print("\nStart loading SuGaR/Frosting mesh...")
462 | print("---> Mesh to load:", query_props.mesh_file_to_load)
463 |
464 | # ---Load mesh---
465 | bpy.ops.wm.obj_import(filepath=query_props.mesh_file_to_load)
466 | obj = bpy.context.selected_objects[-1]
467 |
468 | # ---Set Rotation for better visualization---
469 | obj.rotation_euler[0] = -np.pi / 2
470 |
471 | # ---Material---
472 | # Set up
473 | material = obj.active_material
474 | material.use_nodes = True
475 | nodes = material.node_tree.nodes
476 | p_bsdf_inputs = nodes['Principled BSDF'].inputs
477 |
478 | # Build new Base Color node
479 | rgb_node = nodes.new('ShaderNodeRGB')
480 | rgb_node.outputs[0].default_value = (0., 0., 0., 1.)
481 |
482 | # Set up new Emission Color node
483 | emission_rgb_node = nodes['Image Texture']
484 | emission_rgb_node.interpolation = 'Closest'
485 |
486 | # Links nodes
487 | material.node_tree.links.new(rgb_node.outputs[0], p_bsdf_inputs['Base Color'])
488 | material.node_tree.links.new(emission_rgb_node.outputs[0], p_bsdf_inputs['Emission Color'])
489 | p_bsdf_inputs['Emission Strength'].default_value = 1.
490 |
491 | # ---Create index data---
492 | mesh = obj.data
493 | vert_idx_values = np.arange(len(mesh.vertices)).tolist()
494 | vert_idx_attribute = mesh.attributes.new(name="index", type="INT", domain="POINT")
495 | vert_idx_attribute.data.foreach_set("value", vert_idx_values)
496 |
497 | # ---Write metadata---
498 | if True:
499 | mesh_name = convert_path_to_linux(query_props.mesh_file_to_load)
500 | checkpoint_name = convert_path_to_linux(query_props.checkpoint_to_load)
501 | metadata_string = ''
502 | metadata_string = metadata_string + 'mesh_name:::' + mesh_name
503 | metadata_string = metadata_string + ';checkpoint_name:::' + checkpoint_name
504 |
505 | max_idx = -1
506 | for child in metadata_object.children:
507 | tmp_idx = int(child.name)
508 | if tmp_idx > max_idx:
509 | max_idx = tmp_idx
510 | new_idx = max_idx + 1
511 |
512 | # Text object
513 | bpy.ops.object.text_add(enter_editmode=False, align='WORLD')
514 | text_obj = bpy.context.active_object
515 | text_obj.name = str(new_idx)
516 | set_text(text_obj.data, metadata_string)
517 | text_obj.hide_render = True
518 | text_obj.hide_viewport = True
519 | text_obj.parent = metadata_object
520 |
521 | # Give to every vertex the corresponding metadata idx
522 | vert_idx_values = (new_idx + np.zeros(len(mesh.vertices), dtype=int)).tolist()
523 | vert_idx_attribute = mesh.attributes.new(name="metadata", type="INT", domain="POINT")
524 | vert_idx_attribute.data.foreach_set("value", vert_idx_values)
525 |
526 | # ---Set the viewport shading for better visualization---
527 | area = next(area for area in bpy.context.screen.areas if area.type == 'VIEW_3D')
528 | space = next(space for space in area.spaces if space.type == 'VIEW_3D')
529 | space.shading.type = 'RENDERED'
530 |
531 | return {'FINISHED'}
532 |
533 |
534 | class AddSuGaRMeshPanel(bpy.types.Panel):
535 | """ Display panel in 3D view"""
536 | bl_label = "Add SuGaR or Frosting mesh"
537 |
538 | bl_space_type = 'PROPERTIES'
539 | bl_region_type = 'WINDOW'
540 | bl_context = 'render'
541 |
542 | def draw(self, context):
543 | layout = self.layout
544 |
545 | # Layout
546 | row = layout.row(align=True)
547 | row.alignment = 'EXPAND'
548 |
549 | text_col = row.column(align=True)
550 | text_col_row0 = text_col.row(align=True)
551 | text_col_row0.alignment = 'RIGHT'
552 | text_col_row1 = text_col.row(align=True)
553 | text_col_row1.alignment = 'RIGHT'
554 |
555 | field_col = row.column(align=True)
556 | field_col_row0 = field_col.row(align=True)
557 | field_col_row1 = field_col.row(align=True)
558 |
559 | main_button = layout.column(align=True)
560 |
561 | # Props
562 | query_props = bpy.context.scene.QueryProps
563 |
564 | # Select mesh
565 | text_col_row0.label(text='Path to OBJ file ')
566 | field_col_row0.prop(query_props, 'mesh_file_to_load', text='')
567 | field_col_row0.operator("something.mesh_selector", icon="FILE_FOLDER", text="")
568 |
569 | # Select checkpoint
570 | text_col_row1.label(text='Path to PT file ')
571 | field_col_row1.prop(query_props, 'checkpoint_to_load', text='')
572 | field_col_row1.operator("something.checkpoint_selector", icon="FILE_FOLDER", text="")
573 |
574 | # Add mesh
575 | props = main_button.operator("object.add_sugar_mesh",
576 | text='Add mesh',
577 | emboss=True,
578 | icon="MESH_CUBE"
579 | )
580 |
581 |
582 | class RenderSuGaRPanel(bpy.types.Panel):
583 | """ Display panel in 3D view"""
584 | bl_label = "Render SuGaR or Frosting scene"
585 |
586 | if True:
587 | bl_space_type = 'PROPERTIES'
588 | bl_region_type = 'WINDOW'
589 | bl_context = 'render'
590 | else:
591 | bl_region_type = "UI"
592 | bl_space_type = "VIEW_3D"
593 |
594 | bl_options = {'HEADER_LAYOUT_EXPAND'}
595 |
596 | def draw(self, context):
597 | layout = self.layout
598 |
599 | # Parameters
600 | row = layout.row(align=True)
601 | row.alignment = 'EXPAND'
602 |
603 | if False:
604 | empty_col = row.column(align=False)
605 | empty_col.label(text='')
606 | empty_col.scale_x = 0.2
607 |
608 | text_col = row.column(align=True)
609 | text_col_row0 = text_col.row(align=True)
610 | text_col_row1 = text_col.row(align=True)
611 | text_col_row2 = text_col.row(align=True)
612 | text_col_row0.alignment = 'RIGHT'
613 | text_col_row1.alignment = 'RIGHT'
614 | text_col_row2.alignment = 'RIGHT'
615 |
616 | field_col = row.column(align=True)
617 | field_col_row0 = field_col.row(align=True)
618 | field_col_row1 = field_col.row(align=True)
619 | field_col_row2 = field_col.row(align=True)
620 |
621 | # Main button
622 | main_button = layout.column(align=True)
623 | main_button2 = layout.column(align=True)
624 |
625 | # Props
626 | query_props = bpy.context.scene.QueryProps
627 | props = main_button.operator("object.sugar_render_single",
628 | text='Render Image',
629 | emboss=True,
630 | icon="RENDER_STILL"
631 | )
632 |
633 | props2 = main_button2.operator("object.sugar_render",
634 | text='Render Animation',
635 | emboss=True,
636 | icon="RENDER_ANIMATION"
637 | )
638 |
639 | # text_col_row0.label(text='Conda env name ')
640 | text_col_row1.label(text='Path to SuGaR/Frosting directory ')
641 | # text_col_row2.label(text='Path to output directory ')
642 |
643 | # field_col_row0.prop(query_props, 'env', text='')
644 |
645 | field_col_row1.prop(query_props, 'sugar_dir', text='')
646 | field_col_row1.operator("something.sugar_selector", icon="FILE_FOLDER", text="")
647 |
648 | # field_col_row2.prop(query_props, 'output_dir', text='')
649 | # field_col_row2.operator("something.output_selector", icon="FILE_FOLDER", text="")
650 |
651 | # field_col.prop(query_props, 'n_checkpoints', slider=True, text='')
652 |
653 |
654 | def menu_func(self, context):
655 | self.layout.operator(RenderSuGaROperator.bl_idname)
656 |
657 |
658 | classes = (
659 | QueryProps,
660 | WMSuGaRSelector,
661 | # WMOutputSelector,
662 | WMSuGaRMeshSelector,
663 | WMSuGaRCheckpointSelector,
664 | AddSuGaRMeshPanel,
665 | AddSuGaRMeshOperator,
666 | RenderSuGaRPanel,
667 | RenderSuGaROperator,
668 | RenderSuGaROperatorSingleImage,
669 | )
670 |
671 |
672 | def register():
673 | for cls in classes:
674 | bpy.utils.register_class(cls)
675 | bpy.types.Scene.QueryProps = bpy.props.PointerProperty(type=QueryProps)
676 |
677 |
678 | def unregister():
679 | for cls in classes:
680 | bpy.utils.unregister_class(cls)
681 | del(bpy.types.Scene.QueryProps)
682 |
683 |
684 | if __name__ == "__main__":
685 | register()
686 |
--------------------------------------------------------------------------------
4 |
5 |
6 | Antoine Guédon
7 | Vincent Lepetit
8 |
9 |