├── .gitignore
├── CHANGELOG.md
├── CHANGELOG.md.meta
├── Editor.meta
├── Editor
├── Doji.SNeRG.Editor.asmdef
├── Doji.SNeRG.Editor.asmdef.meta
├── ImportContext.cs
├── ImportContext.cs.meta
├── RaymarchShader.cs
├── RaymarchShader.cs.meta
├── SNeRGLoader.cs
├── SNeRGLoader.cs.meta
├── SNeRGScene.cs
├── SNeRGScene.cs.meta
├── SceneParams.cs
├── SceneParams.cs.meta
├── WebRequestAsyncUtility.cs
└── WebRequestAsyncUtility.cs.meta
├── License.txt
├── License.txt.meta
├── Runtime.meta
├── Runtime
├── Doji.SNeRG.asmdef
├── Doji.SNeRG.asmdef.meta
├── EnableDepthTexture.cs
└── EnableDepthTexture.cs.meta
├── package.json
├── package.json.meta
├── readme.md
└── readme.md.meta
/.gitignore:
--------------------------------------------------------------------------------
1 | # This .gitignore file should be placed at the root of your Unity project directory
2 | #
3 | # Get latest from https://github.com/github/gitignore/blob/master/Unity.gitignore
4 | #
5 | /[Ll]ibrary/
6 | /[Tt]emp/
7 | /[Oo]bj/
8 | /[Bb]uild/
9 | /[Bb]uilds/
10 | /[Ll]ogs/
11 | /[Uu]ser[Ss]ettings/
12 |
13 | # MemoryCaptures can get excessive in size.
14 | # They also could contain extremely sensitive data
15 | /[Mm]emoryCaptures/
16 |
17 | # Asset meta data should only be ignored when the corresponding asset is also ignored
18 | !/[Aa]ssets/**/*.meta
19 |
20 | # Uncomment this line if you wish to ignore the asset store tools plugin
21 | # /[Aa]ssets/AssetStoreTools*
22 |
23 | # Autogenerated Jetbrains Rider plugin
24 | /[Aa]ssets/Plugins/Editor/JetBrains*
25 |
26 | # Visual Studio cache directory
27 | .vs/
28 |
29 | # Gradle cache directory
30 | .gradle/
31 |
32 | # Autogenerated VS/MD/Consulo solution and project files
33 | ExportedObj/
34 | .consulo/
35 | *.csproj
36 | *.unityproj
37 | *.sln
38 | *.suo
39 | *.tmp
40 | *.user
41 | *.userprefs
42 | *.pidb
43 | *.booproj
44 | *.svd
45 | *.pdb
46 | *.mdb
47 | *.opendb
48 | *.VC.db
49 | *.editorconfig
50 | *.vsconfig
51 |
52 | # Unity3D generated meta files
53 | *.pidb.meta
54 | *.pdb.meta
55 | *.mdb.meta
56 |
57 | # Unity3D generated file on crash reports
58 | sysinfo.txt
59 |
60 | # Builds
61 | *.apk
62 | *.aab
63 | *.unitypackage
64 |
65 | # Crashlytics generated file
66 | crashlytics-build.properties
67 |
68 | # Packed Addressables
69 | /[Aa]ssets/[Aa]ddressable[Aa]ssets[Dd]ata/*/*.bin*
70 |
71 | # Temporary auto-generated Android Assets
72 | /[Aa]ssets/[Ss]treamingAssets/aa.meta
73 | /[Aa]ssets/[Ss]treamingAssets/aa/*
74 |
75 |
76 | ----------------------------------------------------
77 |
78 |
79 | # ============ #
80 | # OS generated #
81 | # ============ #
82 | .DS_Store
83 | .DS_Store?
84 | ._*
85 | .Spotlight-V100
86 | .Trashes
87 | ehthumbs.db
88 | Thumbs.db
89 |
--------------------------------------------------------------------------------
/CHANGELOG.md:
--------------------------------------------------------------------------------
1 | # Changelog
2 |
3 | All notable changes to this project will be documented in this file.
4 |
5 | The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
6 | and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
7 |
8 | ## [1.0.2] - 2023-06-01
9 |
10 | ### Fix
11 |
12 | - Fixed incorrect volume position/size for some scenes. Thanks to [@54way](https://github.com/54way)
13 |
14 | ## [1.0.1] - 2023-05-30
15 |
16 | ### Fix
17 |
18 | - Fixed incorrect block grid calculation. Thanks to [@54way](https://github.com/54way)
19 |
20 | ## [1.0.0] - 2023-04-08
21 |
22 | ### Fix
23 |
24 | - Fixed volumes being overdrawn due to rendering before skybox
25 |
26 | ## [x.x.x] - 2022-04-02
27 |
28 | ### Added
29 |
30 | - Add support for importing your own trained datasets
31 |
32 | ## [x.x.x] - 2022-03-14
33 |
34 | ### Fixed
35 |
36 | - Fixed rendering issues in certain demo scenes
37 |
38 | ## [x.x.x] - 2022-03-13
39 |
40 | ### Added
41 |
42 | - Add depth testing
43 | - Add support for 'Linear' color space
44 |
45 | ## [x.x.x] - 023-03-09
46 |
47 | ### Added
48 |
49 | - Add support for Single Pass Instanced rendering
50 |
51 | ## [x.x.x] - 2023-03-07
52 | Initial Release
--------------------------------------------------------------------------------
/CHANGELOG.md.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: bc4f71bf658bace4da5df6273d79d4da
3 | TextScriptImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------
/Editor.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: a2cdc21ba07cdd244b87d30f685d6c35
3 | folderAsset: yes
4 | DefaultImporter:
5 | externalObjects: {}
6 | userData:
7 | assetBundleName:
8 | assetBundleVariant:
9 |
--------------------------------------------------------------------------------
/Editor/Doji.SNeRG.Editor.asmdef:
--------------------------------------------------------------------------------
1 | {
2 | "name": "Doji.SNeRG.Editor",
3 | "rootNamespace": "",
4 | "references": [
5 | "GUID:f2ef7ba7c4c2fc44680f667d61ba9966"
6 | ],
7 | "includePlatforms": [
8 | "Editor"
9 | ],
10 | "excludePlatforms": [],
11 | "allowUnsafeCode": false,
12 | "overrideReferences": false,
13 | "precompiledReferences": [],
14 | "autoReferenced": true,
15 | "defineConstraints": [],
16 | "versionDefines": [],
17 | "noEngineReferences": false
18 | }
--------------------------------------------------------------------------------
/Editor/Doji.SNeRG.Editor.asmdef.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 1fad94f956f8bd94d921786515ade67b
3 | AssemblyDefinitionImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------
/Editor/ImportContext.cs:
--------------------------------------------------------------------------------
1 | using System.IO;
2 |
3 | namespace SNeRG.Editor {
4 |
5 | public class ImportContext {
6 |
7 | ///
8 | /// True if we are we currently importing a custom scene,
9 | /// false if it is one of the demo scenes.
10 | ///
11 | public bool CustomScene;
12 |
13 | ///
14 | /// The demo scene being imported.
15 | ///
16 | public SNeRGScene Scene;
17 |
18 | ///
19 | /// The path to the source files for custom scene imports.
20 | ///
21 | public string CustomScenePath;
22 |
23 | public string SceneName {
24 | get {
25 | if (CustomScene) {
26 | return new DirectoryInfo(CustomScenePath).Name.ToLower();
27 | } else {
28 | return Scene.LowerCaseName();
29 | }
30 |
31 | }
32 | }
33 |
34 | public string SceneNameUpperCase {
35 | get {
36 | if (CustomScene) {
37 | return new DirectoryInfo(CustomScenePath).Name;
38 | } else {
39 | return Scene.Name();
40 | }
41 |
42 | }
43 | }
44 | }
45 | }
--------------------------------------------------------------------------------
/Editor/ImportContext.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: d28747095943d9a438f48b3d58abf4ac
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/Editor/RaymarchShader.cs:
--------------------------------------------------------------------------------
1 | namespace SNeRG.Editor {
2 | public static class RaymarchShader {
3 | public const string Template = @"Shader ""SNeRG/RayMarchShader_OBJECT_NAME"" {
4 | Properties {
5 | mapAlpha(""Alpha Map"", 3D) = """" {}
6 | mapColor(""Color Map"", 3D) = """" {}
7 | mapFeatures(""Feature Map"", 3D) = """" {}
8 | mapIndex(""Index Map"", 3D) = """" {}
9 |
10 | weightsZero (""Weights Zero"", 2D) = ""white"" {}
11 | weightsOne (""Weights One"", 2D) = ""white"" {}
12 | weightsTwo (""Weights Two"", 2D) = ""white"" {}
13 |
14 | displayMode(""Display Mode"", Integer) = 0
15 | ndc(""NDC"", Integer) = 0
16 |
17 | minPosition (""Min Position"", Vector) = (0, 0, 0, 0)
18 | gridSize (""Grid Size"", Vector) = (0, 0, 0, 0)
19 | atlasSize (""Atlas Size"", Vector) = (0, 0, 0, 0)
20 | voxelSize (""Voxel Size"", Float) = 0.0
21 | blockSize (""Block Size"", Float) = 0.0
22 |
23 | maxStep (""Max Step"", Integer) = 0.0
24 | }
25 | SubShader {
26 | Tags { ""Queue"" = ""Transparent"" }
27 |
28 | Cull Front
29 | ZWrite Off
30 | ZTest Always
31 | Blend SrcAlpha OneMinusSrcAlpha
32 |
33 | Pass {
34 | CGPROGRAM
35 |
36 | #pragma vertex vert
37 | #pragma fragment frag
38 |
39 | #include ""UnityCG.cginc""
40 |
41 | int displayMode;
42 | int ndc;
43 |
44 | float3 minPosition;
45 | float3 gridSize;
46 | float3 atlasSize;
47 | float voxelSize;
48 | float blockSize;
49 | int maxStep;
50 |
51 | UNITY_DECLARE_TEX3D(mapAlpha);
52 | UNITY_DECLARE_TEX3D(mapColor);
53 | UNITY_DECLARE_TEX3D(mapFeatures);
54 | UNITY_DECLARE_TEX3D(mapIndex);
55 |
56 | UNITY_DECLARE_TEX2D(weightsZero);
57 | UNITY_DECLARE_TEX2D(weightsOne);
58 | UNITY_DECLARE_TEX2D(weightsTwo);
59 |
60 | UNITY_DECLARE_DEPTH_TEXTURE(_CameraDepthTexture);
61 |
62 | struct appdata {
63 | float4 vertex : POSITION;
64 | UNITY_VERTEX_INPUT_INSTANCE_ID
65 | };
66 |
67 | struct v2f {
68 | float4 vertex : SV_POSITION;
69 | float3 origin : TEXCOORD1;
70 | float3 direction : TEXCOORD2;
71 | float4 projPos : TEXCOORD3;
72 | float3 camRelativeWorldPos : TEXCOORD4;
73 | UNITY_VERTEX_OUTPUT_STEREO
74 | };
75 |
76 | v2f vert (appdata v) {
77 | v2f o;
78 |
79 | UNITY_SETUP_INSTANCE_ID(v);
80 | UNITY_INITIALIZE_OUTPUT(v2f, o);
81 | UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
82 |
83 | o.vertex = UnityObjectToClipPos(v.vertex);
84 |
85 | // set up values to raymarch in object space
86 | o.origin = mul(unity_WorldToObject, float4(_WorldSpaceCameraPos, 1));
87 | o.direction = -ObjSpaceViewDir(v.vertex);
88 |
89 | // set up values to test against scene depth for proper composition with opaque scene objects
90 | o.projPos = ComputeScreenPos(o.vertex);
91 | o.camRelativeWorldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz - _WorldSpaceCameraPos;
92 |
93 | return o;
94 | }
95 |
96 | half indexToPosEnc (half3 dir, uint index) {
97 | half coordinate =
98 | (index % 3 == 0) ? dir.x : (
99 | (index % 3 == 1) ? dir.y : dir.z);
100 | if (index < 3) {
101 | return coordinate;
102 | }
103 | int scaleExponent = ((index - 3) % (3 * 4)) / 3;
104 | coordinate *= pow(2.0, float(scaleExponent));
105 | if ((index - 3) >= 3 * 4) {
106 | const float kHalfPi = 1.57079632679489661923;
107 | coordinate += kHalfPi;
108 | }
109 | return sin(coordinate);
110 | }
111 |
112 | half3 evaluateNetwork(fixed3 color, fixed4 features, fixed3 viewdir) {
113 | half intermediate_one[NUM_CHANNELS_ONE] = { BIAS_LIST_ZERO };
114 | uint i = 0;
115 | uint j = 0;
116 |
117 | for (j = 0; j < NUM_CHANNELS_ZERO; ++j) {
118 | half input_value = 0.0;
119 | if (j < 27) {
120 | input_value = indexToPosEnc(viewdir, j);
121 | }
122 | else if (j < 30) {
123 | input_value =
124 | (j % 3 == 0) ? color.r : (
125 | (j % 3 == 1) ? color.g : color.b);
126 | }
127 | else {
128 | input_value =
129 | (j == 30) ? features.r : (
130 | (j == 31) ? features.g : (
131 | (j == 32) ? features.b : features.a));
132 | }
133 | if (abs(input_value) < 0.1 / 255.0) {
134 | continue;
135 | }
136 | for (int i = 0; i < NUM_CHANNELS_ONE; ++i) {
137 | intermediate_one[i] += input_value * weightsZero.Load(int3(j, i, 0)).x;
138 | }
139 | }
140 |
141 | half intermediate_two[NUM_CHANNELS_TWO] = { BIAS_LIST_ONE };
142 |
143 | for (j = 0; j < NUM_CHANNELS_ONE; ++j) {
144 | if (intermediate_one[j] <= 0.0) {
145 | continue;
146 | }
147 | for (i = 0; i < NUM_CHANNELS_TWO; ++i) {
148 | intermediate_two[i] += intermediate_one[j] * weightsOne.Load(int3(j, i, 0)).x;
149 | }
150 | }
151 |
152 | half result[NUM_CHANNELS_THREE] = { BIAS_LIST_TWO };
153 |
154 | for (j = 0; j < NUM_CHANNELS_TWO; ++j) {
155 | if (intermediate_two[j] <= 0.0) {
156 | continue;
157 | }
158 | for (i = 0; i < NUM_CHANNELS_THREE; ++i) {
159 | result[i] += intermediate_two[j] * weightsTwo.Load(int3(j, i, 0)).x;
160 | }
161 | }
162 | for (i = 0; i < NUM_CHANNELS_THREE; ++i) {
163 | result[i] = 1.0 / (1.0 + exp(-result[i]));
164 | }
165 |
166 | return half3(result[0], result[1], result[2]);
167 | }
168 |
169 | half3 convertOriginToNDC(float3 origin, float3 direction) {
170 | // We store the NDC scenes flipped, so flip back.
171 | origin.z *= -1.0;
172 | direction.z *= -1.0;
173 |
174 | const float near = 1.0;
175 | float t = -(near + origin.z) / direction.z;
176 | origin = origin * t + direction;
177 |
178 | // Hardcoded, worked out using approximate iPhone FOV of 67.3 degrees
179 | // and an image width of 1006 px.
180 | const float focal = 755.644;
181 | const float W = 1006.0;
182 | const float H = 756.0;
183 | float o0 = 1.0 / (W / (2.0 * focal)) * origin.x / origin.z;
184 | float o1 = -1.0 / (H / (2.0 * focal)) * origin.y / origin.z;
185 | float o2 = 1.0 + 2.0 * near / origin.z;
186 |
187 | origin = float3(o0, o1, o2);
188 | origin.z *= -1.0;
189 | return origin;
190 | }
191 |
192 | half3 convertDirectionToNDC(float3 origin, float3 direction) {
193 | // We store the NDC scenes flipped, so flip back.
194 | origin.z *= -1.0;
195 | direction.z *= -1.0;
196 |
197 | const float near = 1.0;
198 | float t = -(near + origin.z) / direction.z;
199 | origin = origin * t + direction;
200 |
201 | // Hardcoded, worked out using approximate iPhone FOV of 67.3 degrees
202 | // and an image width of 1006 px.
203 | const float focal = 755.6440;
204 | const float W = 1006.0;
205 | const float H = 756.0;
206 |
207 | float d0 = 1.0 / (W / (2.0 * focal)) *
208 | (direction.x / direction.z - origin.x / origin.z);
209 | float d1 = -1.0 / (H / (2.0 * focal)) *
210 | (direction.y / direction.z - origin.y / origin.z);
211 | float d2 = -2.0 * near / origin.z;
212 |
213 | direction = normalize(float3(d0, d1, d2));
214 | direction.z *= -1.0;
215 | return direction;
216 | }
217 |
218 | // Compute the atlas block index for a point in the scene using pancake
219 | // 3D atlas packing.
220 | half3 pancakeBlockIndex(half3 posGrid, float blockSize, float3 iBlockGridBlocks) {
221 | float3 iBlockIndex = floor(posGrid / blockSize);
222 | float3 iAtlasBlocks = atlasSize / (blockSize + 2.0);
223 | float linearIndex = iBlockIndex.x + iBlockGridBlocks.x *
224 | (iBlockIndex.z + iBlockGridBlocks.z * iBlockIndex.y);
225 |
226 | half3 atlasBlockIndex = half3(
227 | linearIndex % iAtlasBlocks.x,
228 | (linearIndex / iAtlasBlocks.x) % iAtlasBlocks.y,
229 | linearIndex / (iAtlasBlocks.x * iAtlasBlocks.y));
230 |
231 | // If we exceed the size of the atlas, indicate an empty voxel block.
232 | if (atlasBlockIndex.z >= float(iAtlasBlocks.z)) {
233 | atlasBlockIndex = half3(-1.0, -1.0, -1.0);
234 | }
235 |
236 | return atlasBlockIndex;
237 | }
238 |
239 | half2 rayAabbIntersection(half3 aabbMin, half3 aabbMax, half3 origin, half3 invDirection) {
240 | half3 t1 = (aabbMin - origin) * invDirection;
241 | half3 t2 = (aabbMax - origin) * invDirection;
242 | half3 tMin = min(t1, t2);
243 | half3 tMax = max(t1, t2);
244 | return half2(max(tMin.x, max(tMin.y, tMin.z)), min(tMax.x, min(tMax.y, tMax.z)));
245 | }
246 |
247 | fixed4 frag (v2f i) : SV_Target {
248 | UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i);
249 |
250 | // Runs the full model with view dependence.
251 | const int DISPLAY_NORMAL = 0;
252 | // Disables the view-dependence network.
253 | const int DISPLAY_DIFFUSE = 1;
254 | // Only shows the latent features.
255 | const int DISPLAY_FEATURES = 2;
256 | // Only shows the view dependent component.
257 | const int DISPLAY_VIEW_DEPENDENT = 3;
258 | // Only shows the coarse block grid.
259 | const int DISPLAY_COARSE_GRID = 4;
260 | // Only shows the 3D texture atlas.
261 | const int DISPLAY_3D_ATLAS = 5;
262 |
263 | // Set up the ray parameters in object space..
264 | float nearPlane = _ProjectionParams.y;
265 | half3 origin = i.origin;
266 | half3 directionWorld = normalize(i.direction);
267 | if (ndc != 0) {
268 | nearPlane = 0.0;
269 | origin = convertOriginToNDC(i.origin, normalize(i.direction));
270 | directionWorld = convertDirectionToNDC(i.origin, normalize(i.direction));
271 | }
272 |
273 | // Now transform them to the voxel grid coordinate system.
274 | half3 originGrid = (origin - minPosition) / voxelSize;
275 | half3 directionGrid = directionWorld;
276 | half3 invDirectionGrid = 1.0 / directionGrid;
277 |
278 | float3 iBlockGridBlocks = floor((gridSize + blockSize - 1) / blockSize);
279 | float3 blockGridSize = iBlockGridBlocks * blockSize;
280 |
281 | half2 tMinMax = rayAabbIntersection(half3(0.0, 0.0, 0.0), gridSize, originGrid, invDirectionGrid);
282 |
283 | // Skip any rays that miss the scene bounding box.
284 | if (tMinMax.x > tMinMax.y) {
285 | discard;
286 | }
287 |
288 | float t = max(nearPlane / voxelSize, tMinMax.x) + 0.5;
289 | half3 posGrid = originGrid + directionGrid * t;
290 |
291 | half3 blockMin = floor(posGrid / blockSize) * blockSize;
292 | half3 blockMax = blockMin + blockSize;
293 | half2 tBlockMinMax = rayAabbIntersection(
294 | blockMin, blockMax, originGrid, invDirectionGrid);
295 | half3 atlasBlockIndex;
296 |
297 | if (displayMode == DISPLAY_3D_ATLAS) {
298 | atlasBlockIndex = pancakeBlockIndex(posGrid, blockSize, iBlockGridBlocks);
299 | } else {
300 | atlasBlockIndex = 255.0 * UNITY_SAMPLE_TEX3D(mapIndex, (blockMin + blockMax) / (2.0 * blockGridSize)).xyz;
301 | }
302 |
303 | half visibility = 1.0;
304 | half3 color = half3(0.0, 0.0, 0.0);
305 | half4 features = half4(0.0, 0.0, 0.0, 0.0);
306 | int step = 0;
307 |
308 | // sample scene depth and calculate maximum ray distance in voxel grid space
309 | float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, i.projPos.xy / i.projPos.w).r;
310 | float linearDepth = LinearEyeDepth(depth);
311 |
312 | // reconstruct world position in voxel grid space
313 | float3 viewPlane = (i.camRelativeWorldPos) / dot((i.camRelativeWorldPos), mul(float3(0.0,0.0,-1.0), UNITY_MATRIX_V));
314 | float3 sceneDepthWorld = viewPlane * linearDepth + _WorldSpaceCameraPos;
315 | float3 sceneDepthObj = mul(unity_WorldToObject, float4(sceneDepthWorld, 1.0));
316 | float3 sceneDepthGrid = ((sceneDepthObj - minPosition) / voxelSize) + 0.5;
317 |
318 | // maximum ray distance to march in voxel grid space
319 | float maxDist = distance(sceneDepthGrid, originGrid);
320 |
321 | [loop]
322 | while (step < maxStep && t < maxDist && t < tMinMax.y && visibility > 1.0 / 255.0) {
323 | // Skip empty macroblocks.
324 | if (atlasBlockIndex.x > 254.0) {
325 | t = 0.5 + tBlockMinMax.y;
326 | } else { // Otherwise step through them and fetch RGBA and Features.
327 | half3 posAtlas = clamp(posGrid - blockMin, 0.0, blockSize);
328 |
329 | posAtlas += atlasBlockIndex * (blockSize + 2.0);
330 | posAtlas += 1.0; // Account for the one voxel padding in the atlas.
331 |
332 | if (displayMode == DISPLAY_COARSE_GRID) {
333 | color = atlasBlockIndex * (blockSize + 2.0) / atlasSize;
334 | features.rgb = atlasBlockIndex * (blockSize + 2.0) / atlasSize;
335 | features.a = 1.0;
336 | visibility = 0.0;
337 | continue;
338 | }
339 |
340 | // Do a conservative fetch for alpha!=0 at a lower resolution,
341 | // and skip any voxels which are empty. First, this saves bandwidth
342 | // since we only fetch one byte instead of 8 (trilinear) and most
343 | // fetches hit cache due to low res. Second, this is conservative,
344 | // and accounts for any possible alpha mass that the high resolution
345 | // trilinear would find.
346 | const int skipMipLevel = 2;
347 | const float miniBlockSize = float(1 << skipMipLevel);
348 |
349 | // Only fetch one byte at first, to conserve memory bandwidth in
350 | // empty space.
351 | float atlasAlpha = mapAlpha.Load(int4(int3(posAtlas / miniBlockSize), skipMipLevel)).x;
352 |
353 | if (atlasAlpha > 0.0) {
354 | // OK, we hit something, do a proper trilinear fetch at high res.
355 | half3 atlasUvw = posAtlas / atlasSize;
356 | atlasAlpha = UNITY_SAMPLE_TEX3D_LOD(mapAlpha, atlasUvw, 0.0).x;
357 |
358 | // Only worth fetching the content if high res alpha is non-zero.
359 | if (atlasAlpha > 0.5 / 255.0) {
360 | half4 atlasRgba = half4(0.0, 0.0, 0.0, atlasAlpha);
361 | atlasRgba.rgb = UNITY_SAMPLE_TEX3D(mapColor, atlasUvw).rgb;
362 | if (displayMode != DISPLAY_DIFFUSE) {
363 | half4 atlasFeatures = UNITY_SAMPLE_TEX3D(mapFeatures, atlasUvw);
364 | features += visibility * atlasFeatures;
365 | }
366 | color += visibility * atlasRgba.rgb;
367 | visibility *= 1.0 - atlasRgba.a;
368 | }
369 | }
370 | t += 1.0;
371 | }
372 |
373 | posGrid = originGrid + directionGrid * t;
374 | if (t > tBlockMinMax.y) {
375 | blockMin = floor(posGrid / blockSize) * blockSize;
376 | blockMax = blockMin + blockSize;
377 | tBlockMinMax = rayAabbIntersection(blockMin, blockMax, originGrid, invDirectionGrid);
378 |
379 | if (displayMode == DISPLAY_3D_ATLAS) {
380 | atlasBlockIndex = pancakeBlockIndex(posGrid, blockSize, iBlockGridBlocks);
381 | } else {
382 | atlasBlockIndex = 255.0 * UNITY_SAMPLE_TEX3D(mapIndex, (blockMin + blockMax) / (2.0 * blockGridSize)).xyz;
383 | }
384 | }
385 | step++;
386 | }
387 |
388 | if (displayMode == DISPLAY_VIEW_DEPENDENT) {
389 | color = half3(0.0, 0.0, 0.0) * visibility;
390 | } else if (displayMode == DISPLAY_FEATURES) {
391 | color = features.rgb;
392 | }
393 |
394 | // For forward-facing scenes, we partially unpremultiply alpha to fill
395 | // tiny holes in the rendering.
396 | half alpha = 1.0 - visibility;
397 | if (ndc != 0 && alpha > 0.0) {
398 | half filledAlpha = min(1.0, alpha * 1.5);
399 | color *= filledAlpha / alpha;
400 | alpha = filledAlpha;
401 | visibility = 1.0 - filledAlpha;
402 | }
403 |
404 | // convert from Unity's right handed coordinate system
405 | // to OpenGL coordinates used in the MLP evaluation
406 | i.direction.xz = -i.direction.xz;
407 | i.direction.yz = i.direction.zy;
408 |
409 | // Compute the final color, to save compute only compute view-dependence
410 | // for rays that intersected something in the scene.
411 | color = half3(1.0, 1.0, 1.0) * visibility + color;
412 | const float kVisibilityThreshold = 254.0 / 255.0;
413 | if (visibility <= kVisibilityThreshold &&
414 | (displayMode == DISPLAY_NORMAL ||
415 | displayMode == DISPLAY_VIEW_DEPENDENT)) {
416 | color += evaluateNetwork(color, features, normalize(i.direction));
417 | }
418 |
419 | #if !UNITY_COLORSPACE_GAMMA
420 | color = GammaToLinearSpace(color);
421 | #endif
422 |
423 | return fixed4(color, alpha);
424 | }
425 | ENDCG
426 | }
427 | }
428 | }";
429 | }
430 | }
--------------------------------------------------------------------------------
/Editor/RaymarchShader.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 34c0bfff7aac44b4cbb7f9d3bf2521f5
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/Editor/SNeRGLoader.cs:
--------------------------------------------------------------------------------
1 | using Newtonsoft.Json;
2 | using System.IO;
3 | using System.Text;
4 | using System.Text.RegularExpressions;
5 | using System.Threading.Tasks;
6 | using Unity.Collections;
7 | using UnityEditor;
8 | using UnityEngine;
9 | using static SNeRG.Editor.WebRequestAsyncUtility;
10 |
11 | namespace SNeRG.Editor {
12 |
13 | public class SNeRGLoader {
14 |
15 | private static readonly string LoadingTitle = "Loading Assets";
16 | private static readonly string ProcessingTitle = "Processing Assets";
17 | private static readonly string DownloadInfo = "Loading Assets for ";
18 | private static readonly string FolderTitle = "Select folder with SNeRG source files";
19 | private static readonly string FolderExistsTitle = "Folder already exists";
20 | private static readonly string FolderExistsMsg = "A folder for this asset already exists in the Unity project. Overwrite?";
21 | private static readonly string OK = "OK";
22 | private static readonly string ImportErrorTitle = "Error importing SNeRG assets";
23 |
24 | [MenuItem("SNeRG/Asset Downloads/-- Synthetic Rendered Scenes --", false, -1)]
25 | public static void Separator0() { }
26 | [MenuItem("SNeRG/Asset Downloads/-- Synthetic Rendered Scenes --", true, -1)]
27 | public static bool Separator0Validate() {
28 | return false;
29 | }
30 | [MenuItem("SNeRG/Asset Downloads/-- Real Captured Scenes --", false, 49)]
31 | public static void Separator1() { }
32 | [MenuItem("SNeRG/Asset Downloads/-- Real Captured Scenes --", true, 49)]
33 | public static bool Separator1Validate() {
34 | return false;
35 | }
36 |
37 | [MenuItem("SNeRG/Import from disk", false, 0)]
38 | public async static void ImportAssetsFromDisk() {
39 | // select folder with custom data
40 | string path = EditorUtility.OpenFolderPanel(FolderTitle, "", "");
41 | if (string.IsNullOrEmpty(path) || !Directory.Exists(path)) {
42 | return;
43 | }
44 |
45 | // ask whether to overwrite existing folder
46 | string objName = new DirectoryInfo(path).Name;
47 | if (Directory.Exists($"{BASE_FOLDER}{objName}")) {
48 | if (!EditorUtility.DisplayDialog(FolderExistsTitle, FolderExistsMsg, OK)) {
49 | return;
50 | }
51 | }
52 |
53 | await ImportCustomScene(path);
54 | }
55 |
56 | [MenuItem("SNeRG/Asset Downloads/Lego", false, 0)]
57 | public async static void DownloadLegoAssets() {
58 | await ImportDemoSceneAsync(SNeRGScene.Lego);
59 | }
60 | [MenuItem("SNeRG/Asset Downloads/Chair", false, 0)]
61 | public async static void DownloadChairAssets() {
62 | await ImportDemoSceneAsync(SNeRGScene.Chair);
63 | }
64 | [MenuItem("SNeRG/Asset Downloads/Drums", false, 0)]
65 | public async static void DownloadDrumsAssets() {
66 | await ImportDemoSceneAsync(SNeRGScene.Drums);
67 | }
68 | [MenuItem("SNeRG/Asset Downloads/Hotdog", false, 0)]
69 | public async static void DownloadHotdogAssets() {
70 | await ImportDemoSceneAsync(SNeRGScene.Hotdog);
71 | }
72 | [MenuItem("SNeRG/Asset Downloads/Ship", false, 0)]
73 | public async static void DownloadShipsAssets() {
74 | await ImportDemoSceneAsync(SNeRGScene.Ship);
75 | }
76 | [MenuItem("SNeRG/Asset Downloads/Mic", false, 0)]
77 | public async static void DownloadMicAssets() {
78 | await ImportDemoSceneAsync(SNeRGScene.Mic);
79 | }
80 | [MenuItem("SNeRG/Asset Downloads/Ficus", false, 0)]
81 | public async static void DownloadFicusAssets() {
82 | await ImportDemoSceneAsync(SNeRGScene.Ficus);
83 | }
84 | [MenuItem("SNeRG/Asset Downloads/Materials", false, 0)]
85 | public async static void DownloadMaterialsAssets() {
86 | await ImportDemoSceneAsync(SNeRGScene.Materials);
87 | }
88 |
89 | [MenuItem("SNeRG/Asset Downloads/Spheres", false, 50)]
90 | public async static void DownloadSpheresAssets() {
91 | await ImportDemoSceneAsync(SNeRGScene.Spheres);
92 | }
93 | [MenuItem("SNeRG/Asset Downloads/Vase Deck", false, 50)]
94 | public async static void DownloadVaseDeckAssets() {
95 | await ImportDemoSceneAsync(SNeRGScene.VaseDeck);
96 | }
97 | [MenuItem("SNeRG/Asset Downloads/Pine Cone", false, 50)]
98 | public async static void DownloadPineConeAssets() {
99 | await ImportDemoSceneAsync(SNeRGScene.PineCone);
100 | }
101 | [MenuItem("SNeRG/Asset Downloads/Toy Car", false, 50)]
102 | public async static void DownloadToyCarAssets() {
103 | await ImportDemoSceneAsync(SNeRGScene.ToyCar);
104 | }
105 |
106 | private const string BASE_URL_SYNTH = "https://storage.googleapis.com/snerg/750/";
107 | private const string BASE_URL_REAL = "https://storage.googleapis.com/snerg/real_1000/";
108 |
109 | private static readonly string BASE_FOLDER = Path.Combine("Assets", "SNeRG Data");
110 | private static readonly string BASE_LIB_FOLDER = Path.Combine("Library", "Cached SNeRG Data");
111 |
112 | private static ImportContext _context;
113 |
114 | private static string GetBasePath(string sceneName) {
115 | return Path.Combine(BASE_FOLDER, sceneName);
116 | }
117 | private static string GetCacheLocation(string sceneName) {
118 | return Path.Combine(BASE_LIB_FOLDER, sceneName);
119 | }
120 | private static string GetBaseUrl(SNeRGScene scene) {
121 | if (scene.IsSynthetic()) {
122 | return $"{BASE_URL_SYNTH}{scene.LowerCaseName()}";
123 | } else {
124 | return $"{BASE_URL_REAL}{scene.LowerCaseName()}";
125 | }
126 | }
127 |
128 | private static string GetSceneParamsUrl() {
129 | return $"{GetBaseUrl(_context.Scene)}/scene_params.json";
130 | }
131 | private static string GetAtlasIndexUrl() {
132 | return $"{GetBaseUrl(_context.Scene)}/atlas_indices.png";
133 | }
134 | private static string GetRGBVolumeUrl(int i) {
135 | return $"{GetBaseUrl(_context.Scene)}/rgba_{i:D3}.png";
136 | }
137 | private static string GetFeatureVolumeUrl(int i) {
138 | return $"{GetBaseUrl(_context.Scene)}/feature_{i:D3}.png";
139 | }
140 |
141 | private static string GetSceneParamsAssetPath() {
142 | return GetAssetPath("SceneParams", $"{_context.SceneName}.asset");
143 | }
144 | private static string GetRGBTextureAssetPath() {
145 | return GetAssetPath("Textures", $"{_context.SceneName} RGB Volume Texture.asset");
146 | }
147 | private static string GetAlphaTextureAssetPath() {
148 | return GetAssetPath("Textures", $"{_context.SceneName} Alpha Volume Texture.asset");
149 | }
150 | private static string GetFeatureTextureAssetPath() {
151 | return GetAssetPath("Textures", $"{_context.SceneName} Feature Volume Texture.asset");
152 | }
153 | private static string GetAtlasTextureAssetPath() {
154 | return GetAssetPath("Textures", $"{_context.SceneName} Atlas Index Texture.asset");
155 | }
156 | private static string GetShaderAssetPath() {
157 | return GetAssetPath("Shaders", $"RayMarchShader_{_context.SceneName}.shader");
158 | }
159 | private static string GetMaterialAssetPath() {
160 | return GetAssetPath("Materials", $"Material_{_context.SceneName}.mat");
161 | }
162 | private static string GetWeightsAssetPath(int i) {
163 | return GetAssetPath("SceneParams", $"weightsTex{i}.asset");
164 | }
165 | private static string GetPrefabAssetPath() {
166 | return GetAssetPath("", $"{_context.SceneName}.prefab");
167 | }
168 | private static string GetMeshAssetPath() {
169 | return GetAssetPath("Volume Mesh", $"{_context.SceneName} Volume Mesh.asset");
170 | }
171 | ///
172 | /// This returns a path in the asset directory to store the specific asset into.
173 | ///
174 | private static string GetAssetPath(string subFolder, string assetName) {
175 | string path = Path.Combine(GetBasePath(_context.SceneName), subFolder, assetName);
176 | Directory.CreateDirectory(Path.GetDirectoryName(path));
177 | return path;
178 | }
179 |
180 | private static string GetAtlasIndexCachePath() {
181 | return GetCachePath("atlas_indices.png");
182 | }
183 | private static string GetRGBVolumeCachePath(int i) {
184 | return GetCachePath($"rgba_{i:D3}.png");
185 | }
186 | private static string GetFeatureVolumeCachePath(int i) {
187 | return GetCachePath($"feature_{i:D3}.png");
188 | }
189 | ///
190 | /// This is either the location where demo scenes are first downloaded to,
191 | /// or the path to the source files for custom scene imports.
192 | ///
193 | private static string GetCachePath(string assetName) {
194 | string path;
195 | if (_context.CustomScene) {
196 | path = Path.Combine(_context.CustomScenePath, assetName);
197 | } else {
198 | path = Path.Combine(GetCacheLocation(_context.SceneName), assetName);
199 | Directory.CreateDirectory(Path.GetDirectoryName(path));
200 | }
201 | return path;
202 | }
203 |
204 | ///
205 | /// Creates Unity assets for the given SNeRG assets on disk.
206 | ///
207 | /// The path to the folder with the SNeRG assets (PNGs & mlp.json)
208 | private static async Task ImportCustomScene(string path) {
209 | _context = new ImportContext() {
210 | CustomScene = true,
211 | CustomScenePath = path,
212 | Scene = SNeRGScene.Custom
213 | };
214 | string objName = new DirectoryInfo(path).Name;
215 |
216 | SceneParams sceneParams = CopySceneParamsFromPath(path);
217 | if (sceneParams == null) {
218 | return;
219 | }
220 | if (!ValidatePNGs(path, sceneParams)) {
221 | return;
222 | }
223 |
224 | AssetDatabase.SaveAssets();
225 | AssetDatabase.Refresh();
226 |
227 | await ProcessAssets(objName);
228 | }
229 |
230 | ///
231 | /// Downloads the source files for the given SNeRG demo scene
232 | /// into the folder. Then in the
233 | /// Unity project, creates the Unity assets necessary to display it.
234 | ///
235 | private static async Task ImportDemoSceneAsync(SNeRGScene scene) {
236 | _context = new ImportContext() {
237 | CustomScene = false,
238 | Scene = scene
239 | };
240 |
241 | EditorUtility.DisplayProgressBar(LoadingTitle, $"{DownloadInfo}'{scene.Name()}'...", 0.1f);
242 | await DownloadSceneParamsAsync();
243 |
244 | await ProcessAssets(scene.Name());
245 | }
246 |
247 | ///
248 | /// Set specific import settings on OBJs/PNGs.
249 | /// Creates Weight Textures, Materials and Shader from MLP data.
250 | /// Creates a convenient prefab for the SNeRG object.
251 | ///
252 | private static async Task ProcessAssets(string sceneName) {
253 | var sceneParams = GetSceneParams();
254 |
255 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Raymarch Shader...", 0.2f);
256 | CreateRayMarchShader(sceneParams);
257 |
258 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Material...", 0.3f);
259 | CreateMaterial(sceneParams);
260 |
261 | // load 3D slices from web or cacbe directory, then create 3D volume textures from that data
262 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Atlas Index Texture...", 0.4f);
263 | Texture2D atlasIndexData = await LoadAtlasIndexDataAsync();
264 | CreateAtlasIndexTexture(atlasIndexData, sceneParams);
265 |
266 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' RGB Volume Texture...", 0.5f);
267 | Texture2D[] rgbImages = await LoadRGBVolumeDataAsync(sceneParams);
268 | CreateRgbVolumeTexture(rgbImages, sceneParams);
269 |
270 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Feature Volume Texture...", 0.6f);
271 | Texture2D[] featureImages = await LoadFeatureVolumeDataAsync(sceneParams);
272 | CreateFeatureVolumeTexture(featureImages, sceneParams);
273 |
274 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Weight Textures...", 0.7f);
275 | CreateWeightTextures(sceneParams);
276 |
277 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Finishing '{sceneName}' assets..", 0.8f);
278 | VerifyMaterial(sceneParams);
279 |
280 | CreatePrefab(sceneParams);
281 |
282 | EditorUtility.ClearProgressBar();
283 | }
284 |
285 | ///
286 | /// Looks for a scene_params.json at and imports it.
287 | ///
288 | private static SceneParams CopySceneParamsFromPath(string path) {
289 | string[] sceneParamsPaths = Directory.GetFiles(path, "scene_params.json", SearchOption.AllDirectories);
290 | if (sceneParamsPaths.Length > 1) {
291 | EditorUtility.DisplayDialog(ImportErrorTitle, "Multiple scene_params.json files found", OK);
292 | return null;
293 | }
294 | if (sceneParamsPaths.Length <= 0) {
295 | EditorUtility.DisplayDialog(ImportErrorTitle, "No scene_params.json files found", OK);
296 | return null;
297 | }
298 |
299 | string sceneParamsJson = File.ReadAllText(sceneParamsPaths[0]);
300 | TextAsset sceneParamsTextAsset = new TextAsset(sceneParamsJson);
301 | AssetDatabase.CreateAsset(sceneParamsTextAsset, GetSceneParamsAssetPath());
302 | SceneParams sceneParams = JsonConvert.DeserializeObject(sceneParamsJson);
303 | return sceneParams;
304 | }
305 |
306 | private static async Task DownloadSceneParamsAsync() {
307 | string url = GetSceneParamsUrl();
308 | string sceneParamsJson = await WebRequestSimpleAsync.SendWebRequestAsync(url);
309 | TextAsset sceneParamsTextAsset = new TextAsset(sceneParamsJson);
310 | AssetDatabase.CreateAsset(sceneParamsTextAsset, GetSceneParamsAssetPath());
311 | }
312 |
313 | private static SceneParams GetSceneParams() {
314 | string mlpJson = AssetDatabase.LoadAssetAtPath(GetSceneParamsAssetPath()).text;
315 | return JsonConvert.DeserializeObject(mlpJson);
316 | }
317 |
318 | ///
319 | /// Checks if all necessary textures for a given SNeRG scene are present in the given folder.
320 | ///
321 | private static bool ValidatePNGs(string path, SceneParams mlp) {
322 | string objName = new DirectoryInfo(path).Name;
323 | int numSlicePNGs = mlp.NumSlices;
324 |
325 | string[] pngPaths;
326 | pngPaths = Directory.GetFiles(path, "feature_*.png", SearchOption.TopDirectoryOnly);
327 | if (pngPaths.Length != numSlicePNGs) {
328 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Invalid number of feature textures found. Expected: {numSlicePNGs}. Actual: {pngPaths.Length}", OK);
329 | return false;
330 | }
331 | pngPaths = Directory.GetFiles(path, "rgba_*.png", SearchOption.TopDirectoryOnly);
332 | if (pngPaths.Length != numSlicePNGs) {
333 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Invalid number of feature textures found. Expected: {numSlicePNGs}. Actual: {pngPaths.Length}", OK);
334 | return false;
335 | }
336 |
337 | string atlasPath = Path.Combine(path, "atlas_indices.png");
338 | if (!File.Exists(atlasPath)) {
339 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Could not find atlas indices texture.", OK);
340 | return false;
341 | }
342 | return true;
343 | }
344 |
345 | private static async Task LoadAtlasIndexDataAsync() {
346 | string path = GetAtlasIndexCachePath();
347 | byte[] atlasIndexData;
348 |
349 | if (File.Exists(path)) {
350 | // file is already downloaded
351 | atlasIndexData = File.ReadAllBytes(path);
352 | } else {
353 | string url = GetAtlasIndexUrl();
354 | atlasIndexData = await WebRequestBinaryAsync.SendWebRequestAsync(url);
355 | File.WriteAllBytes(path, atlasIndexData);
356 | }
357 |
358 | // !!! Unity's LoadImage() does NOT respect the texture format specified in the input texture!
359 | // It always loads this as ARGB32, no matter the format specified here.
360 | // Ideally we'd directly load an RGB24 texture.
361 | Texture2D atlasIndexImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true);
362 | atlasIndexImage.filterMode = FilterMode.Point;
363 | atlasIndexImage.wrapMode = TextureWrapMode.Clamp;
364 | atlasIndexImage.LoadImage(atlasIndexData);
365 |
366 | return atlasIndexImage;
367 | }
368 |
369 | private static async Task LoadRGBVolumeDataAsync(SceneParams sceneParams) {
370 | Texture2D[] rgbVolumeArray = new Texture2D[sceneParams.NumSlices];
371 | for (int i = 0; i < sceneParams.NumSlices; i++) {
372 | string path = GetRGBVolumeCachePath(i);
373 | byte[] rgbVolumeData;
374 |
375 | if (File.Exists(path)) {
376 | // file is already downloaded
377 | rgbVolumeData = File.ReadAllBytes(path);
378 | } else {
379 | string url = GetRGBVolumeUrl(i);
380 | rgbVolumeData = await WebRequestBinaryAsync.SendWebRequestAsync(url);
381 | File.WriteAllBytes(path, rgbVolumeData);
382 | }
383 |
384 | // Unity's LoadImage() always loads this as ARGB32, no matter the format specified here
385 | Texture2D rgbVolumeImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true);
386 | rgbVolumeImage.filterMode = FilterMode.Point;
387 | rgbVolumeImage.wrapMode = TextureWrapMode.Clamp;
388 | rgbVolumeImage.alphaIsTransparency = true;
389 | rgbVolumeImage.LoadImage(rgbVolumeData);
390 | rgbVolumeArray[i] = rgbVolumeImage;
391 | }
392 |
393 | return rgbVolumeArray;
394 | }
395 |
396 | private static async Task LoadFeatureVolumeDataAsync(SceneParams sceneParams) {
397 | Texture2D[] featureVolumeArray = new Texture2D[sceneParams.NumSlices];
398 |
399 | for (int i = 0; i < sceneParams.NumSlices; i++) {
400 | string path = GetFeatureVolumeCachePath(i);
401 | byte[] featureVolumeData;
402 |
403 | if (File.Exists(path)) {
404 | // file is already downloaded
405 | featureVolumeData = File.ReadAllBytes(path);
406 | } else {
407 | string url = GetFeatureVolumeUrl(i);
408 | featureVolumeData = await WebRequestBinaryAsync.SendWebRequestAsync(url);
409 | File.WriteAllBytes(path, featureVolumeData);
410 | }
411 |
412 | // Unity's LoadImage() always loads this as ARGB32, no matter the format specified here
413 | Texture2D featureVolumeImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true);
414 | featureVolumeImage.filterMode = FilterMode.Point;
415 | featureVolumeImage.wrapMode = TextureWrapMode.Clamp;
416 | featureVolumeImage.LoadImage(featureVolumeData);
417 | featureVolumeArray[i] = featureVolumeImage;
418 | }
419 |
420 | return featureVolumeArray;
421 | }
422 |
423 | private static void CreateRgbVolumeTexture(Texture2D[] rgbaData, SceneParams sceneParams) {
424 | string rgbAssetPath = GetRGBTextureAssetPath();
425 | string alphaAssetPath = GetAlphaTextureAssetPath();
426 |
427 | // already exists
428 | if (File.Exists(rgbAssetPath) && File.Exists(alphaAssetPath)) {
429 | return;
430 | }
431 |
432 | // initialize 3D textures
433 | Texture3D rgbVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.RGB24, mipChain: false) {
434 | filterMode = FilterMode.Bilinear,
435 | wrapMode = TextureWrapMode.Clamp,
436 | name = Path.GetFileNameWithoutExtension(rgbAssetPath)
437 | };
438 |
439 | Texture3D alphaVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.R8, mipChain: true) {
440 | filterMode = FilterMode.Trilinear, // original code uses different min mag filters, which Unity doesn't let us do, tbd
441 | wrapMode = TextureWrapMode.Clamp,
442 | name = Path.GetFileNameWithoutExtension(alphaAssetPath)
443 | };
444 |
445 | // load data into 3D textures
446 | loadSplitVolumeTexture(rgbaData, alphaVolumeTexture, rgbVolumeTexture, sceneParams);
447 |
448 | // destroy intermediate textures to free memory
449 | for (int i = rgbaData.Length - 1; i >= 0; i--) {
450 | UnityEngine.Object.DestroyImmediate(rgbaData[i]);
451 | }
452 | Resources.UnloadUnusedAssets();
453 |
454 | AssetDatabase.CreateAsset(rgbVolumeTexture, rgbAssetPath);
455 | AssetDatabase.CreateAsset(alphaVolumeTexture, alphaAssetPath);
456 |
457 | string materialAssetPath = GetMaterialAssetPath();
458 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
459 | material.SetTexture("mapColor", rgbVolumeTexture);
460 | material.SetTexture("mapAlpha", alphaVolumeTexture);
461 |
462 | AssetDatabase.SaveAssets();
463 | }
464 |
465 | private static void CreateFeatureVolumeTexture(Texture2D[] featureData, SceneParams sceneParams) {
466 | string featureAssetPath = GetFeatureTextureAssetPath();
467 |
468 | // already exists
469 | if (File.Exists(featureAssetPath)) {
470 | return;
471 | }
472 |
473 | // initialize 3D texture
474 | Texture3D featureVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.ARGB32, mipChain: false) {
475 | filterMode = FilterMode.Bilinear,
476 | wrapMode = TextureWrapMode.Clamp,
477 | name = Path.GetFileNameWithoutExtension(featureAssetPath)
478 | };
479 |
480 | // load data into 3D textures
481 | LoadVolumeTexture(featureData, featureVolumeTexture, sceneParams);
482 |
483 | // destroy intermediate textures to free memory
484 | for (int i = featureData.Length - 1; i >= 0; i--) {
485 | UnityEngine.Object.DestroyImmediate(featureData[i]);
486 | }
487 | Resources.UnloadUnusedAssets();
488 |
489 | AssetDatabase.CreateAsset(featureVolumeTexture, featureAssetPath);
490 |
491 | string materialAssetPath = GetMaterialAssetPath();
492 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
493 | material.SetTexture("mapFeatures", featureVolumeTexture);
494 | }
495 |
496 | private static void CreateAtlasIndexTexture(Texture2D atlasIndexImage, SceneParams sceneParams) {
497 | int width = (int)Mathf.Ceil(sceneParams.GridWidth / (float)sceneParams.BlockSize);
498 | int height = (int)Mathf.Ceil(sceneParams.GridHeight / (float)sceneParams.BlockSize);
499 | int depth = (int)Mathf.Ceil(sceneParams.GridDepth / (float)sceneParams.BlockSize);
500 |
501 | string atlasAssetPath = GetAtlasTextureAssetPath();
502 |
503 | // already exists
504 | if (File.Exists(atlasAssetPath)) {
505 | return;
506 | }
507 |
508 | // initialize 3D texture
509 | Texture3D atlasIndexTexture = new Texture3D(width, height, depth, TextureFormat.ARGB32, mipChain: false) {
510 | filterMode = FilterMode.Point,
511 | wrapMode = TextureWrapMode.Clamp,
512 | name = Path.GetFileNameWithoutExtension(atlasAssetPath),
513 | };
514 |
515 | // load data into 3D textures
516 | NativeArray rawAtlasIndexData = atlasIndexImage.GetRawTextureData();
517 | atlasIndexTexture.SetPixelData(rawAtlasIndexData, 0);
518 |
519 | // flip the y axis for each depth slice
520 | FlipY(atlasIndexTexture);
521 |
522 | // reverse the slices of the 3D texture
523 | //FlipZ(atlasIndexTexture);
524 |
525 | atlasIndexTexture.Apply();
526 |
527 | // destroy intermediate texture to free memory
528 | UnityEngine.Object.DestroyImmediate(atlasIndexImage);
529 | Resources.UnloadUnusedAssets();
530 |
531 | AssetDatabase.CreateAsset(atlasIndexTexture, atlasAssetPath);
532 |
533 | string materialAssetPath = GetMaterialAssetPath();
534 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
535 | material.SetTexture("mapIndex", atlasIndexTexture);
536 | }
537 |
538 | ///
539 | /// Fills two distinct 3D textures from a set of atlased PNGs.
540 | /// This method flips both y and z axes of the 3D texture, because the original code assumes we're
541 | /// indexing from top left, but Unity loaded the PNGs starting bottom right.
542 | ///
543 | private static void loadSplitVolumeTexture(Texture2D[] rgbaArray, Texture3D alphaVolumeTexture, Texture3D rgbVolumeTexture, SceneParams sceneParams) {
544 | Debug.Assert(sceneParams.NumSlices == rgbaArray.Length, "Expected " + sceneParams.NumSlices + " RGBA slices, but found " + rgbaArray.Length);
545 |
546 | int volumeWidth = sceneParams.AtlasWidth;
547 | int volumeHeight = sceneParams.AtlasHeight;
548 | int volumeDepth = sceneParams.AtlasDepth;
549 |
550 | int sliceDepth = 4; // slices packed into one atlased texture
551 | int numSlices = sceneParams.NumSlices; // number of slice atlases
552 | int ppAtlas = volumeWidth * volumeHeight * sliceDepth; // pixels per atlased texture
553 | int ppSlice = volumeWidth * volumeHeight; // pixels per volume slice
554 |
555 | NativeArray rgbPixels = rgbVolumeTexture.GetPixelData(0);
556 | NativeArray alphaPixels = alphaVolumeTexture.GetPixelData(0);
557 |
558 | Debug.Assert(rgbPixels.Length == numSlices * ppAtlas * 3, "Mismatching RGB Texture Data. Expected: " + numSlices * ppAtlas * 3 + ". Actual: + " + rgbPixels.Length);
559 | Debug.Assert(alphaPixels.Length == numSlices * ppAtlas, "Mismatching alpha Texture Data. Expected: " + numSlices * ppAtlas + ". Actual: + " + alphaPixels.Length);
560 |
561 | for (int i = 0; i < numSlices; i++) {
562 | // rgba images are in ARGB format!
563 | NativeArray rgbaImageFourSlices = rgbaArray[i].GetPixelData(0);
564 | Debug.Assert(rgbaImageFourSlices.Length == ppAtlas * 4, "Mismatching RGBA Texture Data. Expected: " + ppAtlas * 4 + ". Actual: + " + rgbaImageFourSlices.Length);
565 |
566 | for (int s_r = sliceDepth - 1, s = 0; s_r >= 0; s_r--, s++) {
567 |
568 | int baseIndexRGB = (i * ppAtlas + s * ppSlice) * 3;
569 | int baseIndexAlpha = (i * ppAtlas + s * ppSlice);
570 | for (int j = 0; j < ppSlice; j++) {
571 | rgbPixels[baseIndexRGB + (j * 3)] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 1];
572 | rgbPixels[baseIndexRGB + (j * 3) + 1] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 2];
573 | rgbPixels[baseIndexRGB + (j * 3) + 2] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 3];
574 | alphaPixels[baseIndexAlpha + j] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4)];
575 | }
576 | }
577 | }
578 |
579 | FlipY(rgbVolumeTexture);
580 | FlipZ(rgbVolumeTexture, sceneParams.AtlasBlocksZ);
581 | FlipY(alphaVolumeTexture);
582 | FlipZ(alphaVolumeTexture, sceneParams.AtlasBlocksZ);
583 |
584 | rgbVolumeTexture.Apply(updateMipmaps: false, makeNoLongerReadable: true);
585 | alphaVolumeTexture.Apply(updateMipmaps: true, makeNoLongerReadable: true);
586 | }
587 |
588 | ///
589 | /// Fills an existing 3D RGBA texture from atlased PNGs.
590 | /// This method flips both y and z axes of the 3D texture, because the original code assumes we're
591 | /// indexing from top left, but Unity loaded the PNGs starting bottom right.
592 | ///
593 | private static void LoadVolumeTexture(Texture2D[] featureImages, Texture3D featureVolumeTexture, SceneParams sceneParams) {
594 | Debug.Assert(sceneParams.NumSlices == featureImages.Length, "Expected " + sceneParams.NumSlices + " feature slices, but found " + featureImages.Length);
595 |
596 | int volumeWidth = sceneParams.AtlasWidth;
597 | int volumeHeight = sceneParams.AtlasHeight;
598 | int volumeDepth = sceneParams.AtlasDepth;
599 |
600 | int sliceDepth = 4; // slices packed into one atlased texture
601 | long numSlices = sceneParams.NumSlices; // number of slice atlases
602 | int ppAtlas = volumeWidth * volumeHeight * sliceDepth; // pixels per atlased feature texture
603 | int ppSlice = volumeWidth * volumeHeight; // pixels per volume slice
604 |
605 | NativeArray featurePixels = featureVolumeTexture.GetPixelData(0);
606 | Debug.Assert(featurePixels.Length == numSlices * ppAtlas, "Mismatching RGB Texture Data. Expected: " + numSlices * ppAtlas + ". Actual: + " + featurePixels.Length);
607 |
608 | for (int i = 0; i < numSlices; i++) {
609 |
610 | NativeArray _featureImageFourSlices = featureImages[i].GetRawTextureData();
611 | Debug.Assert(_featureImageFourSlices.Length == ppAtlas, "Mismatching feature Texture Data. Expected: " + ppAtlas + ". Actual: + " + _featureImageFourSlices.Length);
612 |
613 | for (int s_r = sliceDepth - 1, s = 0; s_r >= 0; s_r--, s++) {
614 | int targetIndex = (i * ppAtlas) + (s * ppSlice);
615 | NativeSlice dst = new NativeSlice(featurePixels, targetIndex, ppSlice);
616 | NativeSlice src = new NativeSlice(_featureImageFourSlices, s_r * ppSlice, ppSlice);
617 |
618 | dst.CopyFrom(src);
619 | }
620 | }
621 | FlipY(featureVolumeTexture);
622 | FlipZ(featureVolumeTexture, sceneParams.AtlasBlocksZ);
623 |
624 | featureVolumeTexture.Apply(updateMipmaps: false, makeNoLongerReadable: true);
625 | }
626 |
627 | ///
628 | /// Vertically flips each depth slice in the given 3D texture.
629 | ///
630 | private static void FlipY(Texture3D texture) where T : struct {
631 | int width = texture.width;
632 | int height = texture.height;
633 | int depth = texture.depth;
634 | NativeArray data = texture.GetPixelData(0);
635 | for (int z = 0; z < depth; z++) {
636 | for (int y = 0; y < height / 2; y++) {
637 | for (int x = 0; x < width; x++) {
638 | int flippedY = height - y - 1;
639 | int source = z * (width * height) + (flippedY * width) + x;
640 | int target = z * (width * height) + (y * width) + x;
641 | (data[target], data[source]) = (data[source], data[target]);
642 | }
643 | }
644 | }
645 | }
646 |
647 | ///
648 | /// Flips z - Reverses the order of the depth slices of the given 3D texture.
649 | /// Atlases might be divided in macro blocks that are treated individually here.
650 | /// I.e. depth slices are only reversed within a block.
651 | ///
652 | private static void FlipZ(Texture3D texture, int atlasBlocksZ) where T : struct {
653 | int width = texture.width;
654 | int height = texture.height;
655 | int depth = texture.depth;
656 | int stride = depth / atlasBlocksZ;
657 | int blockSize = width * height * stride;
658 | int sliceSize = width * height;
659 |
660 | NativeArray data = texture.GetPixelData(0);
661 | NativeArray tmp = new NativeArray(sliceSize, Allocator.Temp);
662 |
663 | for (int z = 0; z < atlasBlocksZ; z++) {
664 | for (int s = 0; s < stride / 2; s++) {
665 | int atlasBlock = z * blockSize;
666 | int slice1Index = atlasBlock + s * sliceSize;
667 | int slice2Index = atlasBlock + ((stride - s - 1) * sliceSize);
668 |
669 | NativeSlice slice1 = new NativeSlice(data, slice1Index, sliceSize);
670 | NativeSlice slice2 = new NativeSlice(data, slice2Index, sliceSize);
671 |
672 | slice1.CopyTo(tmp);
673 | slice1.CopyFrom(slice2);
674 | slice2.CopyFrom(tmp);
675 | }
676 | }
677 |
678 | tmp.Dispose();
679 | }
680 |
681 | private struct Color24 {
682 | public byte r;
683 | public byte g;
684 | public byte b;
685 | }
686 |
687 | private static void CreateRayMarchShader(SceneParams sceneParams) {
688 | double[][] Weights_0 = sceneParams._0Weights;
689 | double[][] Weights_1 = sceneParams._1Weights;
690 | double[][] Weights_2 = sceneParams._2Weights;
691 | double[] Bias_0 = sceneParams._0Bias;
692 | double[] Bias_1 = sceneParams._1Bias;
693 | double[] Bias_2 = sceneParams._2Bias;
694 |
695 | StringBuilder biasListZero = toBiasList(Bias_0);
696 | StringBuilder biasListOne = toBiasList(Bias_1);
697 | StringBuilder biasListTwo = toBiasList(Bias_2);
698 |
699 | int channelsZero = Weights_0.Length;
700 | int channelsOne = Bias_0.Length;
701 | int channelsTwo = Bias_1.Length;
702 | int channelsThree = Bias_2.Length;
703 | int posEncScales = 4;
704 |
705 | string shaderSource = RaymarchShader.Template;
706 | shaderSource = new Regex("OBJECT_NAME").Replace(shaderSource, $"{_context.SceneNameUpperCase}");
707 | shaderSource = new Regex("NUM_CHANNELS_ZERO").Replace(shaderSource, $"{channelsZero}");
708 | shaderSource = new Regex("NUM_POSENC_SCALES").Replace(shaderSource, $"{posEncScales}");
709 | shaderSource = new Regex("NUM_CHANNELS_ONE").Replace(shaderSource, $"{channelsOne}");
710 | shaderSource = new Regex("NUM_CHANNELS_TWO").Replace(shaderSource, $"{channelsTwo}");
711 | shaderSource = new Regex("NUM_CHANNELS_THREE").Replace(shaderSource, $"{channelsThree}");
712 | shaderSource = new Regex("BIAS_LIST_ZERO").Replace(shaderSource, $"{biasListZero}");
713 | shaderSource = new Regex("BIAS_LIST_ONE").Replace(shaderSource, $"{biasListOne}");
714 | shaderSource = new Regex("BIAS_LIST_TWO").Replace(shaderSource, $"{biasListTwo}");
715 |
716 | string shaderAssetPath = GetShaderAssetPath();
717 | File.WriteAllText(shaderAssetPath, shaderSource);
718 | AssetDatabase.SaveAssets();
719 | AssetDatabase.Refresh();
720 | }
721 |
722 | private static void CreateMaterial(SceneParams sceneParams) {
723 | string materialAssetPath = GetMaterialAssetPath();
724 | Material material;
725 |
726 | if (File.Exists(GetMaterialAssetPath())) {
727 | material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
728 | } else {
729 | string shaderAssetPath = GetShaderAssetPath();
730 | Shader raymarchShader = AssetDatabase.LoadAssetAtPath(shaderAssetPath);
731 | material = new Material(raymarchShader);
732 | }
733 |
734 | material.SetInteger("displayMode", 0);
735 | material.SetInteger("ndc", sceneParams.Ndc ? 1 : 0);
736 |
737 | material.SetVector("minPosition", new Vector4(
738 | (float)sceneParams.MinX,
739 | (float)sceneParams.MinY,
740 | (float)sceneParams.MinZ
741 | )
742 | );
743 | material.SetVector("gridSize", new Vector4(
744 | sceneParams.GridWidth,
745 | sceneParams.GridHeight,
746 | sceneParams.GridDepth
747 | )
748 | );
749 | material.SetVector("atlasSize", new Vector4(
750 | sceneParams.AtlasWidth,
751 | sceneParams.AtlasHeight,
752 | sceneParams.AtlasDepth
753 | )
754 | );
755 | material.SetFloat("voxelSize", (float)sceneParams.VoxelSize);
756 | material.SetFloat("blockSize", (float)sceneParams.BlockSize);
757 | int maxStep = Mathf.CeilToInt(new Vector3(sceneParams.GridWidth, sceneParams.GridHeight, sceneParams.GridDepth).magnitude);
758 | material.SetInteger("maxStep", maxStep);
759 |
760 | // volume texture properties will be assigned when creating the 3D textures to avoid having to load them into memory here
761 |
762 | if (!File.Exists(GetMaterialAssetPath())) {
763 | AssetDatabase.CreateAsset(material, materialAssetPath);
764 | }
765 | AssetDatabase.SaveAssets();
766 | AssetDatabase.Refresh();
767 | }
768 |
769 | ///
770 | /// Assign volume textures again in case material got recreated.
771 | ///
772 | private static void VerifyMaterial(SceneParams sceneParams) {
773 | string materialAssetPath = GetMaterialAssetPath();
774 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
775 |
776 | string rgbAssetPath = GetRGBTextureAssetPath();
777 | string alphaAssetPath = GetAlphaTextureAssetPath();
778 | string featureAssetPath = GetFeatureTextureAssetPath();
779 | string atlasAssetPath = GetAtlasTextureAssetPath();
780 |
781 | if (material.GetTexture("mapColor") == null) {
782 | Texture3D rgb = AssetDatabase.LoadAssetAtPath(rgbAssetPath);
783 | material.SetTexture("mapColor", rgb);
784 | }
785 | if (material.GetTexture("mapAlpha") == null) {
786 | Texture3D alpha = AssetDatabase.LoadAssetAtPath(alphaAssetPath);
787 | material.SetTexture("mapAlpha", alpha);
788 | }
789 | if (material.GetTexture("mapFeatures") == null) {
790 | Texture3D feature = AssetDatabase.LoadAssetAtPath(featureAssetPath);
791 | material.SetTexture("mapFeatures", feature);
792 | }
793 | if (material.GetTexture("mapIndex") == null) {
794 | Texture3D atlas = AssetDatabase.LoadAssetAtPath(atlasAssetPath);
795 | material.SetTexture("mapIndex", atlas);
796 | }
797 | }
798 |
799 | private static void CreateWeightTextures(SceneParams sceneParams) {
800 | Texture2D weightsTexZero = createFloatTextureFromData(sceneParams._0Weights);
801 | Texture2D weightsTexOne = createFloatTextureFromData(sceneParams._1Weights);
802 | Texture2D weightsTexTwo = createFloatTextureFromData(sceneParams._2Weights);
803 | AssetDatabase.CreateAsset(weightsTexZero, GetWeightsAssetPath(0));
804 | AssetDatabase.CreateAsset(weightsTexOne, GetWeightsAssetPath(1));
805 | AssetDatabase.CreateAsset(weightsTexTwo, GetWeightsAssetPath(2));
806 |
807 | string materialAssetPath = GetMaterialAssetPath();
808 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
809 | material.SetTexture("weightsZero", weightsTexZero);
810 | material.SetTexture("weightsOne", weightsTexOne);
811 | material.SetTexture("weightsTwo", weightsTexTwo);
812 | }
813 |
814 | ///
815 | /// Creates a float32 texture from an 2D array of weights.
816 | ///
817 | private static Texture2D createFloatTextureFromData(double[][] weights) {
818 | int width = weights.Length;
819 | int height = weights[0].Length;
820 |
821 | Texture2D texture = new Texture2D(width, height, TextureFormat.RFloat, mipChain: false, linear: true);
822 | texture.filterMode = FilterMode.Point;
823 | texture.wrapMode = TextureWrapMode.Clamp;
824 | NativeArray textureData = texture.GetRawTextureData();
825 | FillTexture(textureData, weights);
826 | texture.Apply(updateMipmaps: false, makeNoLongerReadable: true);
827 |
828 | return texture;
829 | }
830 |
831 | private static void FillTexture(NativeArray textureData, double[][] data) {
832 | int width = data.Length;
833 | int height = data[0].Length;
834 |
835 | for (int co = 0; co < height; co++) {
836 | for (int ci = 0; ci < width; ci++) {
837 | int index = co * width + ci;
838 | double weight = data[ci][co];
839 | textureData[index] = (float)weight;
840 | }
841 | }
842 | }
843 |
844 | private static StringBuilder toBiasList(double[] biases) {
845 | System.Globalization.CultureInfo culture = System.Globalization.CultureInfo.InvariantCulture;
846 | int width = biases.Length;
847 | StringBuilder biasList = new StringBuilder(width * 12);
848 | for (int i = 0; i < width; i++) {
849 | double bias = biases[i];
850 | biasList.Append(bias.ToString("F7", culture));
851 | if (i + 1 < width) {
852 | biasList.Append(", ");
853 | }
854 | }
855 | return biasList;
856 | }
857 |
858 | ///
859 | /// Creates a convenient prefab for the SNeRG.
860 | ///
861 | private static void CreatePrefab(SceneParams sceneParams) {
862 | GameObject prefabObject = GameObject.CreatePrimitive(PrimitiveType.Cube);
863 | prefabObject.name = _context.SceneName;
864 | MeshRenderer renderer = prefabObject.GetComponent();
865 | string materialAssetPath = GetMaterialAssetPath();
866 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath);
867 | renderer.material = material;
868 | MeshFilter meshFilter = prefabObject.GetComponent();
869 | meshFilter.sharedMesh = CreateMesh(meshFilter.sharedMesh, sceneParams);
870 | meshFilter.sharedMesh.name = $"{_context.SceneName}_volume_mesh";
871 | AssetDatabase.CreateAsset(meshFilter.sharedMesh, GetMeshAssetPath());
872 | prefabObject.AddComponent();
873 | PrefabUtility.SaveAsPrefabAsset(prefabObject, GetPrefabAssetPath());
874 | GameObject.DestroyImmediate(prefabObject);
875 | }
876 |
877 | ///
878 | /// Creates a copy of the default cube mesh
879 | /// and scales by volume extents to be able to raymarch in object space.
880 | ///
881 | ///
882 | ///
883 | ///
884 | private static Mesh CreateMesh(Mesh mesh, SceneParams sceneParams) {
885 | var vertices = mesh.vertices;
886 | for (int i = 0; i < vertices.Length; i++) {
887 |
888 | Vector3 v = vertices[i];
889 | Vector3 size = new Vector3(
890 | Mathf.Abs((float)sceneParams.GridWidth),
891 | Mathf.Abs((float)sceneParams.GridHeight),
892 | Mathf.Abs((float)sceneParams.GridDepth))
893 | * (float)sceneParams.VoxelSize;
894 | Vector3 minPos = new Vector3(
895 | (float)sceneParams.MinX,
896 | (float)sceneParams.MinY,
897 | (float)sceneParams.MinZ
898 | );
899 | Vector3 offset = minPos + size / 2;
900 | v.Scale(size);
901 | v += offset;
902 | vertices[i] = v;
903 | }
904 | Mesh newMesh = new Mesh {
905 | vertices = vertices,
906 | triangles = mesh.triangles,
907 | uv = mesh.uv,
908 | normals = mesh.normals,
909 | colors = mesh.colors,
910 | tangents = mesh.tangents
911 | };
912 | return newMesh;
913 | }
914 | }
915 | }
--------------------------------------------------------------------------------
/Editor/SNeRGLoader.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 8878a305d5f21a743ae14219b389fd01
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/Editor/SNeRGScene.cs:
--------------------------------------------------------------------------------
1 | using System;
2 |
3 | namespace SNeRG.Editor {
4 |
5 | public enum SNeRGScene {
6 | Custom = -1,
7 | Lego,
8 | Chair,
9 | Drums,
10 | Hotdog,
11 | Ship,
12 | Mic,
13 | Ficus,
14 | Materials,
15 | Spheres,
16 | VaseDeck,
17 | PineCone,
18 | ToyCar
19 | }
20 |
21 | public static class SNeRGSceneExtensions {
22 | public static string Name(this SNeRGScene scene) {
23 | return scene.ToString();
24 | }
25 |
26 | public static string LowerCaseName(this SNeRGScene scene) {
27 | return scene.ToString().ToLower();
28 | }
29 |
30 | public static SNeRGScene ToEnum(string value) {
31 | return (SNeRGScene)Enum.Parse(typeof(SNeRGScene), value, true);
32 | }
33 |
34 | public static bool IsSynthetic(this SNeRGScene scene) {
35 | switch (scene) {
36 | case SNeRGScene.Lego:
37 | case SNeRGScene.Chair:
38 | case SNeRGScene.Drums:
39 | case SNeRGScene.Hotdog:
40 | case SNeRGScene.Ship:
41 | case SNeRGScene.Mic:
42 | case SNeRGScene.Ficus:
43 | case SNeRGScene.Materials:
44 | return true;
45 | case SNeRGScene.Spheres:
46 | case SNeRGScene.VaseDeck:
47 | case SNeRGScene.PineCone:
48 | case SNeRGScene.ToyCar:
49 | return false;
50 | default:
51 | throw new InvalidOperationException();
52 | }
53 | }
54 | }
55 | }
--------------------------------------------------------------------------------
/Editor/SNeRGScene.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 91924917c19a3df4a83fa27b27f09380
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/Editor/SceneParams.cs:
--------------------------------------------------------------------------------
1 | using Newtonsoft.Json;
2 |
3 | namespace SNeRG.Editor {
4 | public partial class SceneParams {
5 | [JsonProperty("voxel_size")]
6 | public double VoxelSize { get; set; }
7 |
8 | [JsonProperty("block_size")]
9 | public int BlockSize { get; set; }
10 |
11 | [JsonProperty("grid_width")]
12 | public int GridWidth { get; set; }
13 |
14 | [JsonProperty("grid_height")]
15 | public int GridHeight { get; set; }
16 |
17 | [JsonProperty("grid_depth")]
18 | public int GridDepth { get; set; }
19 |
20 | [JsonProperty("atlas_width")]
21 | public int AtlasWidth { get; set; }
22 |
23 | [JsonProperty("atlas_height")]
24 | public int AtlasHeight { get; set; }
25 |
26 | [JsonProperty("atlas_depth")]
27 | public int AtlasDepth { get; set; }
28 |
29 | [JsonProperty("num_slices")]
30 | public int NumSlices { get; set; }
31 |
32 | [JsonProperty("min_x")]
33 | public double MinX { get; set; }
34 |
35 | [JsonProperty("min_y")]
36 | public double MinY { get; set; }
37 |
38 | [JsonProperty("min_z")]
39 | public double MinZ { get; set; }
40 |
41 | [JsonProperty("atlas_blocks_x")]
42 | public int AtlasBlocksX { get; set; }
43 |
44 | [JsonProperty("atlas_blocks_y")]
45 | public int AtlasBlocksY { get; set; }
46 |
47 | [JsonProperty("atlas_blocks_z")]
48 | public int AtlasBlocksZ { get; set; }
49 |
50 | [JsonProperty("worldspace_T_opengl")]
51 | public float[][] WorldspaceTOpengl { get; set; }
52 |
53 | [JsonProperty("ndc")]
54 | public bool Ndc { get; set; }
55 |
56 | [JsonProperty("0_weights")]
57 | public double[][] _0Weights { get; set; }
58 |
59 | [JsonProperty("1_weights")]
60 | public double[][] _1Weights { get; set; }
61 |
62 | [JsonProperty("2_weights")]
63 | public double[][] _2Weights { get; set; }
64 |
65 | [JsonProperty("0_bias")]
66 | public double[] _0Bias { get; set; }
67 |
68 | [JsonProperty("1_bias")]
69 | public double[] _1Bias { get; set; }
70 |
71 | [JsonProperty("2_bias")]
72 | public double[] _2Bias { get; set; }
73 |
74 | [JsonProperty("format")]
75 | public string Format { get; set; }
76 | }
77 | }
--------------------------------------------------------------------------------
/Editor/SceneParams.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: faed614a21ec17842b601e8b33bea6c3
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/Editor/WebRequestAsyncUtility.cs:
--------------------------------------------------------------------------------
1 | using UnityEngine;
2 | using Newtonsoft.Json;
3 | using UnityEngine.Networking;
4 | using System;
5 | using System.Threading.Tasks;
6 |
7 | namespace SNeRG.Editor {
8 |
9 | public static class WebRequestAsyncUtility {
10 |
11 | public static class WebRequestAsync {
12 |
13 | ///
14 | /// Performs a WebRequest with the given URL. The response is expected to be a JSON serialized
15 | /// object of type and is automatically being deserialized and returned.
16 | ///
17 | /// the type that this web request returns.
18 | public static async Task SendWebRequestAsync(string url) {
19 | UnityWebRequest wr = UnityWebRequest.Get(url);
20 | if (wr == null) {
21 | return default;
22 | }
23 |
24 | try {
25 | var asyncOp = wr.SendWebRequest();
26 | while (!asyncOp.webRequest.isDone) {
27 | await Task.Yield();
28 | }
29 |
30 | switch (asyncOp.webRequest.result) {
31 | case UnityWebRequest.Result.InProgress:
32 | break;
33 | case UnityWebRequest.Result.Success:
34 | return JsonConvert.DeserializeObject(asyncOp.webRequest.downloadHandler.text);
35 | case UnityWebRequest.Result.ConnectionError:
36 | case UnityWebRequest.Result.ProtocolError:
37 | case UnityWebRequest.Result.DataProcessingError:
38 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}");
39 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}");
40 | break;
41 | default:
42 | break;
43 | }
44 |
45 | } catch (Exception e) {
46 | Debug.LogError(e);
47 | } finally {
48 | wr.Dispose();
49 | }
50 | return default;
51 | }
52 | }
53 |
54 | ///
55 | /// Simple WebRequest without JSON parsing. Returns plain text.
56 | ///
57 | public class WebRequestSimpleAsync {
58 |
59 | public static async Task SendWebRequestAsync(string url) {
60 | UnityWebRequest wr = UnityWebRequest.Get(url);
61 | if (wr == null) {
62 | return default;
63 | }
64 |
65 | try {
66 | var asyncOp = wr.SendWebRequest();
67 | while (!asyncOp.webRequest.isDone) {
68 | await Task.Yield();
69 | }
70 |
71 | switch (asyncOp.webRequest.result) {
72 | case UnityWebRequest.Result.InProgress:
73 | break;
74 | case UnityWebRequest.Result.Success:
75 | return asyncOp.webRequest.downloadHandler.text;
76 | case UnityWebRequest.Result.ConnectionError:
77 | case UnityWebRequest.Result.ProtocolError:
78 | case UnityWebRequest.Result.DataProcessingError:
79 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}");
80 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}");
81 | break;
82 | default:
83 | break;
84 | }
85 |
86 | } catch (Exception e) {
87 | Debug.LogError(e);
88 | } finally {
89 | wr.Dispose();
90 | }
91 | return default;
92 | }
93 | }
94 |
95 | ///
96 | /// WebRequest for binary data.
97 | ///
98 | public class WebRequestBinaryAsync {
99 |
100 | public static async Task SendWebRequestAsync(string url) {
101 | UnityWebRequest wr = UnityWebRequest.Get(url);
102 | if (wr == null) {
103 | return default;
104 | }
105 |
106 | try {
107 | var asyncOp = wr.SendWebRequest();
108 | while (!asyncOp.webRequest.isDone) {
109 | await Task.Yield();
110 | }
111 |
112 | switch (asyncOp.webRequest.result) {
113 | case UnityWebRequest.Result.InProgress:
114 | break;
115 | case UnityWebRequest.Result.Success:
116 | return asyncOp.webRequest.downloadHandler.data;
117 | case UnityWebRequest.Result.ConnectionError:
118 | case UnityWebRequest.Result.ProtocolError:
119 | case UnityWebRequest.Result.DataProcessingError:
120 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}");
121 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}");
122 | break;
123 | default:
124 | break;
125 | }
126 |
127 | } catch (Exception e) {
128 | Debug.LogError(e);
129 | } finally {
130 | wr.Dispose();
131 | }
132 | return default;
133 | }
134 | }
135 |
136 | ///
137 | /// WebRequest for texture data.
138 | ///
139 | public class WebRequestTextureAsync {
140 |
141 | public static async Task SendWebRequestAsync(string url) {
142 | UnityWebRequest wr = UnityWebRequestTexture.GetTexture(url);
143 | if (wr == null) {
144 | return default;
145 | }
146 |
147 | try {
148 | var asyncOp = wr.SendWebRequest();
149 | while (!asyncOp.webRequest.isDone) {
150 | await Task.Yield();
151 | }
152 |
153 | switch (asyncOp.webRequest.result) {
154 | case UnityWebRequest.Result.InProgress:
155 | break;
156 | case UnityWebRequest.Result.Success:
157 | return DownloadHandlerTexture.GetContent(asyncOp.webRequest);
158 | case UnityWebRequest.Result.ConnectionError:
159 | case UnityWebRequest.Result.ProtocolError:
160 | case UnityWebRequest.Result.DataProcessingError:
161 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}");
162 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}");
163 | break;
164 | default:
165 | break;
166 | }
167 |
168 | } catch (Exception e) {
169 | Debug.LogError(e);
170 | } finally {
171 | wr.Dispose();
172 | }
173 | return default;
174 | }
175 | }
176 | }
177 | }
--------------------------------------------------------------------------------
/Editor/WebRequestAsyncUtility.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 730f94c3031e12b4793062e6319c01ac
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/License.txt:
--------------------------------------------------------------------------------
1 | Apache License
2 | Version 2.0, January 2004
3 | http://www.apache.org/licenses/
4 |
5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6 |
7 | 1. Definitions.
8 |
9 | "License" shall mean the terms and conditions for use, reproduction,
10 | and distribution as defined by Sections 1 through 9 of this document.
11 |
12 | "Licensor" shall mean the copyright owner or entity authorized by
13 | the copyright owner that is granting the License.
14 |
15 | "Legal Entity" shall mean the union of the acting entity and all
16 | other entities that control, are controlled by, or are under common
17 | control with that entity. For the purposes of this definition,
18 | "control" means (i) the power, direct or indirect, to cause the
19 | direction or management of such entity, whether by contract or
20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
21 | outstanding shares, or (iii) beneficial ownership of such entity.
22 |
23 | "You" (or "Your") shall mean an individual or Legal Entity
24 | exercising permissions granted by this License.
25 |
26 | "Source" form shall mean the preferred form for making modifications,
27 | including but not limited to software source code, documentation
28 | source, and configuration files.
29 |
30 | "Object" form shall mean any form resulting from mechanical
31 | transformation or translation of a Source form, including but
32 | not limited to compiled object code, generated documentation,
33 | and conversions to other media types.
34 |
35 | "Work" shall mean the work of authorship, whether in Source or
36 | Object form, made available under the License, as indicated by a
37 | copyright notice that is included in or attached to the work
38 | (an example is provided in the Appendix below).
39 |
40 | "Derivative Works" shall mean any work, whether in Source or Object
41 | form, that is based on (or derived from) the Work and for which the
42 | editorial revisions, annotations, elaborations, or other modifications
43 | represent, as a whole, an original work of authorship. For the purposes
44 | of this License, Derivative Works shall not include works that remain
45 | separable from, or merely link (or bind by name) to the interfaces of,
46 | the Work and Derivative Works thereof.
47 |
48 | "Contribution" shall mean any work of authorship, including
49 | the original version of the Work and any modifications or additions
50 | to that Work or Derivative Works thereof, that is intentionally
51 | submitted to Licensor for inclusion in the Work by the copyright owner
52 | or by an individual or Legal Entity authorized to submit on behalf of
53 | the copyright owner. For the purposes of this definition, "submitted"
54 | means any form of electronic, verbal, or written communication sent
55 | to the Licensor or its representatives, including but not limited to
56 | communication on electronic mailing lists, source code control systems,
57 | and issue tracking systems that are managed by, or on behalf of, the
58 | Licensor for the purpose of discussing and improving the Work, but
59 | excluding communication that is conspicuously marked or otherwise
60 | designated in writing by the copyright owner as "Not a Contribution."
61 |
62 | "Contributor" shall mean Licensor and any individual or Legal Entity
63 | on behalf of whom a Contribution has been received by Licensor and
64 | subsequently incorporated within the Work.
65 |
66 | 2. Grant of Copyright License. Subject to the terms and conditions of
67 | this License, each Contributor hereby grants to You a perpetual,
68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69 | copyright license to reproduce, prepare Derivative Works of,
70 | publicly display, publicly perform, sublicense, and distribute the
71 | Work and such Derivative Works in Source or Object form.
72 |
73 | 3. Grant of Patent License. Subject to the terms and conditions of
74 | this License, each Contributor hereby grants to You a perpetual,
75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76 | (except as stated in this section) patent license to make, have made,
77 | use, offer to sell, sell, import, and otherwise transfer the Work,
78 | where such license applies only to those patent claims licensable
79 | by such Contributor that are necessarily infringed by their
80 | Contribution(s) alone or by combination of their Contribution(s)
81 | with the Work to which such Contribution(s) was submitted. If You
82 | institute patent litigation against any entity (including a
83 | cross-claim or counterclaim in a lawsuit) alleging that the Work
84 | or a Contribution incorporated within the Work constitutes direct
85 | or contributory patent infringement, then any patent licenses
86 | granted to You under this License for that Work shall terminate
87 | as of the date such litigation is filed.
88 |
89 | 4. Redistribution. You may reproduce and distribute copies of the
90 | Work or Derivative Works thereof in any medium, with or without
91 | modifications, and in Source or Object form, provided that You
92 | meet the following conditions:
93 |
94 | (a) You must give any other recipients of the Work or
95 | Derivative Works a copy of this License; and
96 |
97 | (b) You must cause any modified files to carry prominent notices
98 | stating that You changed the files; and
99 |
100 | (c) You must retain, in the Source form of any Derivative Works
101 | that You distribute, all copyright, patent, trademark, and
102 | attribution notices from the Source form of the Work,
103 | excluding those notices that do not pertain to any part of
104 | the Derivative Works; and
105 |
106 | (d) If the Work includes a "NOTICE" text file as part of its
107 | distribution, then any Derivative Works that You distribute must
108 | include a readable copy of the attribution notices contained
109 | within such NOTICE file, excluding those notices that do not
110 | pertain to any part of the Derivative Works, in at least one
111 | of the following places: within a NOTICE text file distributed
112 | as part of the Derivative Works; within the Source form or
113 | documentation, if provided along with the Derivative Works; or,
114 | within a display generated by the Derivative Works, if and
115 | wherever such third-party notices normally appear. The contents
116 | of the NOTICE file are for informational purposes only and
117 | do not modify the License. You may add Your own attribution
118 | notices within Derivative Works that You distribute, alongside
119 | or as an addendum to the NOTICE text from the Work, provided
120 | that such additional attribution notices cannot be construed
121 | as modifying the License.
122 |
123 | You may add Your own copyright statement to Your modifications and
124 | may provide additional or different license terms and conditions
125 | for use, reproduction, or distribution of Your modifications, or
126 | for any such Derivative Works as a whole, provided Your use,
127 | reproduction, and distribution of the Work otherwise complies with
128 | the conditions stated in this License.
129 |
130 | 5. Submission of Contributions. Unless You explicitly state otherwise,
131 | any Contribution intentionally submitted for inclusion in the Work
132 | by You to the Licensor shall be under the terms and conditions of
133 | this License, without any additional terms or conditions.
134 | Notwithstanding the above, nothing herein shall supersede or modify
135 | the terms of any separate license agreement you may have executed
136 | with Licensor regarding such Contributions.
137 |
138 | 6. Trademarks. This License does not grant permission to use the trade
139 | names, trademarks, service marks, or product names of the Licensor,
140 | except as required for reasonable and customary use in describing the
141 | origin of the Work and reproducing the content of the NOTICE file.
142 |
143 | 7. Disclaimer of Warranty. Unless required by applicable law or
144 | agreed to in writing, Licensor provides the Work (and each
145 | Contributor provides its Contributions) on an "AS IS" BASIS,
146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 | implied, including, without limitation, any warranties or conditions
148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 | PARTICULAR PURPOSE. You are solely responsible for determining the
150 | appropriateness of using or redistributing the Work and assume any
151 | risks associated with Your exercise of permissions under this License.
152 |
153 | 8. Limitation of Liability. In no event and under no legal theory,
154 | whether in tort (including negligence), contract, or otherwise,
155 | unless required by applicable law (such as deliberate and grossly
156 | negligent acts) or agreed to in writing, shall any Contributor be
157 | liable to You for damages, including any direct, indirect, special,
158 | incidental, or consequential damages of any character arising as a
159 | result of this License or out of the use or inability to use the
160 | Work (including but not limited to damages for loss of goodwill,
161 | work stoppage, computer failure or malfunction, or any and all
162 | other commercial damages or losses), even if such Contributor
163 | has been advised of the possibility of such damages.
164 |
165 | 9. Accepting Warranty or Additional Liability. While redistributing
166 | the Work or Derivative Works thereof, You may choose to offer,
167 | and charge a fee for, acceptance of support, warranty, indemnity,
168 | or other liability obligations and/or rights consistent with this
169 | License. However, in accepting such obligations, You may act only
170 | on Your own behalf and on Your sole responsibility, not on behalf
171 | of any other Contributor, and only if You agree to indemnify,
172 | defend, and hold each Contributor harmless for any liability
173 | incurred by, or claims asserted against, such Contributor by reason
174 | of your accepting any such warranty or additional liability.
175 |
176 | END OF TERMS AND CONDITIONS
177 |
178 | APPENDIX: How to apply the Apache License to your work.
179 |
180 | To apply the Apache License to your work, attach the following
181 | boilerplate notice, with the fields enclosed by brackets "[]"
182 | replaced with your own identifying information. (Don't include
183 | the brackets!) The text should be enclosed in the appropriate
184 | comment syntax for the file format. We also recommend that a
185 | file or class name and description of purpose be included on the
186 | same "printed page" as the copyright notice for easier
187 | identification within third-party archives.
188 |
189 | Copyright [yyyy] [name of copyright owner]
190 |
191 | Licensed under the Apache License, Version 2.0 (the "License");
192 | you may not use this file except in compliance with the License.
193 | You may obtain a copy of the License at
194 |
195 | http://www.apache.org/licenses/LICENSE-2.0
196 |
197 | Unless required by applicable law or agreed to in writing, software
198 | distributed under the License is distributed on an "AS IS" BASIS,
199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 | See the License for the specific language governing permissions and
201 | limitations under the License.
202 |
--------------------------------------------------------------------------------
/License.txt.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 793c8006049ee10459442c8078265df3
3 | TextScriptImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------
/Runtime.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 32a81c4270a18314bb8906203e15cc6c
3 | folderAsset: yes
4 | DefaultImporter:
5 | externalObjects: {}
6 | userData:
7 | assetBundleName:
8 | assetBundleVariant:
9 |
--------------------------------------------------------------------------------
/Runtime/Doji.SNeRG.asmdef:
--------------------------------------------------------------------------------
1 | {
2 | "name": "Doji.SNeRG"
3 | }
4 |
--------------------------------------------------------------------------------
/Runtime/Doji.SNeRG.asmdef.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: f2ef7ba7c4c2fc44680f667d61ba9966
3 | AssemblyDefinitionImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------
/Runtime/EnableDepthTexture.cs:
--------------------------------------------------------------------------------
1 | using UnityEngine;
2 |
3 | namespace SNeRG {
4 |
5 | ///
6 | /// Enables on the main camera.
7 | /// ExecuteInEditMode is needed, so that this setting also gets
8 | /// applied on the scene camera when not in Play Mode.
9 | ///
10 | [ExecuteInEditMode]
11 | public class EnableDepthTexture : MonoBehaviour {
12 |
13 | private void OnEnable() {
14 | Camera camera = Camera.main;
15 |
16 | if (camera == null) {
17 | Debug.LogError("Could not find main camera." +
18 | " For proper depth composition between SNeRGs and other opaque objects in the scene, " +
19 | "ensure that DeptextureMode.Depth is active on the camera used for rendering.",
20 | camera.gameObject);
21 | return;
22 | }
23 |
24 | camera.depthTextureMode = DepthTextureMode.Depth;
25 | }
26 | }
27 | }
--------------------------------------------------------------------------------
/Runtime/EnableDepthTexture.cs.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: 54254b92b9c44824c80d2696944c6671
3 | MonoImporter:
4 | externalObjects: {}
5 | serializedVersion: 2
6 | defaultReferences: []
7 | executionOrder: 0
8 | icon: {instanceID: 0}
9 | userData:
10 | assetBundleName:
11 | assetBundleVariant:
12 |
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "com.doji.snerg",
3 | "version": "1.0.2",
4 | "displayName": "SNeRG",
5 | "description": "An unofficial Unity port of the SNeRG viewer.",
6 | "dependencies":
7 | {
8 | "com.unity.nuget.newtonsoft-json": "3.0.2"
9 | },
10 | "author":
11 | {
12 | "name": "Doji Technologies",
13 | "email": "support@doji-tech.com",
14 | "url": "https://www.doji-tech.com/"
15 | }
16 | }
--------------------------------------------------------------------------------
/package.json.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: c94a1344b832e8d4f9bee4c16c004653
3 | PackageManifestImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------
/readme.md:
--------------------------------------------------------------------------------
1 | # SNeRG Unity Viewer
2 |
3 | https://user-images.githubusercontent.com/26555424/224686209-113c4a0f-860e-4b8c-9158-430b4ba2d2a9.mp4
4 |
5 | This repository contains the source code for a Unity port of the web viewer for [Baking Neural Radiance Fields for Real-Time View Synthesis](https://phog.github.io/snerg/)[^1]
6 |
7 | *Please note, that this is an unofficial port. I am not affiliated with the original authors or their institution.*
8 |
9 | ## Requirements
10 |
11 | I recommend 32GB of RAM and a graphics card with 4GB of VRAM.
12 |
13 | ## Usage
14 |
15 | ### Installation
16 |
17 | Go to the [releases section](https://github.com/julienkay/SNeRG-Unity-Viewer/releases/latest), download the Unity Package, and import it into any Unity project. This is a 'Hybrid Package' that will install into your project as a local package.
18 |
19 | ##### Alternatives
20 |
21 |
22 | UPM Package via OpenUPM
23 |
24 | In `Edit -> Project Settings -> Package Manager`, add a new scoped registry:
25 |
26 | Name: Doji
27 | URL: https://package.openupm.com
28 | Scope(s): com.doji
29 |
30 | In the Package Manager install 'com.doji.snerg' either by name or via `Package Manager -> My Registries`
31 |
32 |
33 |
34 | UPM Package via Git URL
35 |
36 | In `Package Manager -> Add package from git URL...` paste `https://github.com/julienkay/SNeRG-Unity-Viewer.git` [as described here](https://docs.unity3d.com/Manual/upm-ui-giturl)
37 |
38 |
39 | ### Importing sample scenes
40 |
41 | After succesful installation, you can use the menu *SNeRG -> Asset Downloads* to download any of the sample scenes available.
42 | In each scene folder there will be a convenient prefab, that you can then drag into the scene and you're good to go.
43 |
44 | ### Updating
45 |
46 | Since the initial release a number of issues have been fixed in the asset generation code.
47 | That means, that if you have already imported some scenes before, you'll have to delete these source files and regenerate them by going to *SNeRG -> Asset Downloads* again.
48 |
49 | ### Importing self-trained scenes
50 |
51 | If you have successfully trained your own SNeRG scenes using the [official code release](https://github.com/google-research/google-research/tree/master/snerg) and want to render them in Unity, you can use the menu *SNeRG -> Import from disk*.
52 | This lets you choose a folder that must contain the output files of your training process.
53 |
54 | More specifically, the following assets are required:
55 | - a file named 'scene_params.json'
56 | - a PNG file for the 3D atlas texture: atlas_indices.png
57 | - several PNG files for the 3D feature texture: feature_XXX.png - feature_YYY.png
58 | - several PNG files for the 3D RGB and alpha texture: rgba_XXX.png - rgba_YYY.png
59 |
60 | ## Details
61 |
62 | This project was created with Unity 2021.3 LTS using the Built-in Render Pipeline.
63 |
64 | Deviations from the official viewer:
65 | - Rather than doing the raymarching by drawing a full-screen quad, I opted to use a frontface-culled cube in world space instead. From a rendering perspective, there should not be too much of a difference, but it has some workflow advantages in the Unity Editor I think. It is a bit more straightforward to render the volume in Unity's scene view. It's also easier to integrate into existing projects because it allows other components to work directly with that GameObject's transform hierarchy (e.g. Physics, scripts from various XR Interaction SDKs, ...).
66 | - I added manual depth testing, so the volume correctly interacts with other (opaque) mesh-based objects in the scene. This usually requires a [depth pre-pass](https://docs.unity3d.com/Manual/SL-CameraDepthTexture.html) though.
67 |
68 | ## Known Issues
69 |
70 | - Memory requirements for working with 3D textures in Unity are quite high. The way the import process works right now is that Texture3D assets are created in-Editor. The memory that Unity uses when serializing these as assets on disk is a bit too high for my liking. Granted, the size of these scenes can go up to Gigabytes. However, I've found that 16GB of RAM was often not enough to handle the import/creation of 2GB of uncompressed 3D textures due to inefficiencies. I wasn’t even able to import some of the larger scenes until upgrading to 32GB RAM.
71 |
72 | - Unity straight-up errors out when trying to create 3D textures with sizes of 2GB or larger. Some of the feature volumes like the ficus scene (2048x2048x128) hit that limit and can not be imported.
73 |
74 | ### Possible Workarounds
75 |
76 | - One possible solution to handling the data in-Editor might be to defer the 3D texture creation and import the volumes into Unity as individual 2D slices, (similar to how the official web viewer distributes the data) and only assemble the 3D textures at runtime. For development, it was quite useful to preview the volume without having to enter Play Mode though. But some scenes are currently just very difficult to handle, even when your GPU could fit the data into VRAM just fine.
77 |
78 | - I should have probably looked more into texture compression options for 3D textures
79 |
80 | - I've set the Asset Serialization Mode to 'Force Binary' with some success of reducing memory consumption during import. Unfortunately, the Asset Serialization Mode is a project-wide setting, which is a bit inconvenient, because in real-world projects text serialization is more commonly used I suppose.
81 |
82 | For future improvements, [MERF](https://merf42.github.io/) looks promising in terms of reducing memory usage.
83 |
84 | ## References
85 |
86 | Other projects exploring NeRFs and related techniques in Unity:
87 | - [MobileNeRF (Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures)](https://github.com/julienkay/MobileNeRF-Unity-Viewer)
88 | - [MERF (Memory Efficient Radiance Fields)](https://github.com/julienkay/MERF-Unity-Viewer)
89 |
90 | [^1]: [Peter Hedman and Pratul P. Srinivasan and Ben Mildenhall and Jonathan T. Barron and Paul Debevec. Baking Neural Radiance Fields for Real-Time View Synthesis. ICCV, 2021](https://phog.github.io/snerg/)
91 |
--------------------------------------------------------------------------------
/readme.md.meta:
--------------------------------------------------------------------------------
1 | fileFormatVersion: 2
2 | guid: d30e1e83d9e306e4ab72d3168baf29d0
3 | TextScriptImporter:
4 | externalObjects: {}
5 | userData:
6 | assetBundleName:
7 | assetBundleVariant:
8 |
--------------------------------------------------------------------------------