├── .gitignore ├── CHANGELOG.md ├── CHANGELOG.md.meta ├── Editor.meta ├── Editor ├── Doji.SNeRG.Editor.asmdef ├── Doji.SNeRG.Editor.asmdef.meta ├── ImportContext.cs ├── ImportContext.cs.meta ├── RaymarchShader.cs ├── RaymarchShader.cs.meta ├── SNeRGLoader.cs ├── SNeRGLoader.cs.meta ├── SNeRGScene.cs ├── SNeRGScene.cs.meta ├── SceneParams.cs ├── SceneParams.cs.meta ├── WebRequestAsyncUtility.cs └── WebRequestAsyncUtility.cs.meta ├── License.txt ├── License.txt.meta ├── Runtime.meta ├── Runtime ├── Doji.SNeRG.asmdef ├── Doji.SNeRG.asmdef.meta ├── EnableDepthTexture.cs └── EnableDepthTexture.cs.meta ├── package.json ├── package.json.meta ├── readme.md └── readme.md.meta /.gitignore: -------------------------------------------------------------------------------- 1 | # This .gitignore file should be placed at the root of your Unity project directory 2 | # 3 | # Get latest from https://github.com/github/gitignore/blob/master/Unity.gitignore 4 | # 5 | /[Ll]ibrary/ 6 | /[Tt]emp/ 7 | /[Oo]bj/ 8 | /[Bb]uild/ 9 | /[Bb]uilds/ 10 | /[Ll]ogs/ 11 | /[Uu]ser[Ss]ettings/ 12 | 13 | # MemoryCaptures can get excessive in size. 14 | # They also could contain extremely sensitive data 15 | /[Mm]emoryCaptures/ 16 | 17 | # Asset meta data should only be ignored when the corresponding asset is also ignored 18 | !/[Aa]ssets/**/*.meta 19 | 20 | # Uncomment this line if you wish to ignore the asset store tools plugin 21 | # /[Aa]ssets/AssetStoreTools* 22 | 23 | # Autogenerated Jetbrains Rider plugin 24 | /[Aa]ssets/Plugins/Editor/JetBrains* 25 | 26 | # Visual Studio cache directory 27 | .vs/ 28 | 29 | # Gradle cache directory 30 | .gradle/ 31 | 32 | # Autogenerated VS/MD/Consulo solution and project files 33 | ExportedObj/ 34 | .consulo/ 35 | *.csproj 36 | *.unityproj 37 | *.sln 38 | *.suo 39 | *.tmp 40 | *.user 41 | *.userprefs 42 | *.pidb 43 | *.booproj 44 | *.svd 45 | *.pdb 46 | *.mdb 47 | *.opendb 48 | *.VC.db 49 | *.editorconfig 50 | *.vsconfig 51 | 52 | # Unity3D generated meta files 53 | *.pidb.meta 54 | *.pdb.meta 55 | *.mdb.meta 56 | 57 | # Unity3D generated file on crash reports 58 | sysinfo.txt 59 | 60 | # Builds 61 | *.apk 62 | *.aab 63 | *.unitypackage 64 | 65 | # Crashlytics generated file 66 | crashlytics-build.properties 67 | 68 | # Packed Addressables 69 | /[Aa]ssets/[Aa]ddressable[Aa]ssets[Dd]ata/*/*.bin* 70 | 71 | # Temporary auto-generated Android Assets 72 | /[Aa]ssets/[Ss]treamingAssets/aa.meta 73 | /[Aa]ssets/[Ss]treamingAssets/aa/* 74 | 75 | 76 | ---------------------------------------------------- 77 | 78 | 79 | # ============ # 80 | # OS generated # 81 | # ============ # 82 | .DS_Store 83 | .DS_Store? 84 | ._* 85 | .Spotlight-V100 86 | .Trashes 87 | ehthumbs.db 88 | Thumbs.db 89 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | All notable changes to this project will be documented in this file. 4 | 5 | The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) 6 | and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). 7 | 8 | ## [1.0.2] - 2023-06-01 9 | 10 | ### Fix 11 | 12 | - Fixed incorrect volume position/size for some scenes. Thanks to [@54way](https://github.com/54way) 13 | 14 | ## [1.0.1] - 2023-05-30 15 | 16 | ### Fix 17 | 18 | - Fixed incorrect block grid calculation. Thanks to [@54way](https://github.com/54way) 19 | 20 | ## [1.0.0] - 2023-04-08 21 | 22 | ### Fix 23 | 24 | - Fixed volumes being overdrawn due to rendering before skybox 25 | 26 | ## [x.x.x] - 2022-04-02 27 | 28 | ### Added 29 | 30 | - Add support for importing your own trained datasets 31 | 32 | ## [x.x.x] - 2022-03-14 33 | 34 | ### Fixed 35 | 36 | - Fixed rendering issues in certain demo scenes 37 | 38 | ## [x.x.x] - 2022-03-13 39 | 40 | ### Added 41 | 42 | - Add depth testing 43 | - Add support for 'Linear' color space 44 | 45 | ## [x.x.x] - 023-03-09 46 | 47 | ### Added 48 | 49 | - Add support for Single Pass Instanced rendering 50 | 51 | ## [x.x.x] - 2023-03-07 52 | Initial Release -------------------------------------------------------------------------------- /CHANGELOG.md.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: bc4f71bf658bace4da5df6273d79d4da 3 | TextScriptImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | -------------------------------------------------------------------------------- /Editor.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: a2cdc21ba07cdd244b87d30f685d6c35 3 | folderAsset: yes 4 | DefaultImporter: 5 | externalObjects: {} 6 | userData: 7 | assetBundleName: 8 | assetBundleVariant: 9 | -------------------------------------------------------------------------------- /Editor/Doji.SNeRG.Editor.asmdef: -------------------------------------------------------------------------------- 1 | { 2 | "name": "Doji.SNeRG.Editor", 3 | "rootNamespace": "", 4 | "references": [ 5 | "GUID:f2ef7ba7c4c2fc44680f667d61ba9966" 6 | ], 7 | "includePlatforms": [ 8 | "Editor" 9 | ], 10 | "excludePlatforms": [], 11 | "allowUnsafeCode": false, 12 | "overrideReferences": false, 13 | "precompiledReferences": [], 14 | "autoReferenced": true, 15 | "defineConstraints": [], 16 | "versionDefines": [], 17 | "noEngineReferences": false 18 | } -------------------------------------------------------------------------------- /Editor/Doji.SNeRG.Editor.asmdef.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 1fad94f956f8bd94d921786515ade67b 3 | AssemblyDefinitionImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | -------------------------------------------------------------------------------- /Editor/ImportContext.cs: -------------------------------------------------------------------------------- 1 | using System.IO; 2 | 3 | namespace SNeRG.Editor { 4 | 5 | public class ImportContext { 6 | 7 | /// 8 | /// True if we are we currently importing a custom scene, 9 | /// false if it is one of the demo scenes. 10 | /// 11 | public bool CustomScene; 12 | 13 | /// 14 | /// The demo scene being imported. 15 | /// 16 | public SNeRGScene Scene; 17 | 18 | /// 19 | /// The path to the source files for custom scene imports. 20 | /// 21 | public string CustomScenePath; 22 | 23 | public string SceneName { 24 | get { 25 | if (CustomScene) { 26 | return new DirectoryInfo(CustomScenePath).Name.ToLower(); 27 | } else { 28 | return Scene.LowerCaseName(); 29 | } 30 | 31 | } 32 | } 33 | 34 | public string SceneNameUpperCase { 35 | get { 36 | if (CustomScene) { 37 | return new DirectoryInfo(CustomScenePath).Name; 38 | } else { 39 | return Scene.Name(); 40 | } 41 | 42 | } 43 | } 44 | } 45 | } -------------------------------------------------------------------------------- /Editor/ImportContext.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: d28747095943d9a438f48b3d58abf4ac 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /Editor/RaymarchShader.cs: -------------------------------------------------------------------------------- 1 | namespace SNeRG.Editor { 2 | public static class RaymarchShader { 3 | public const string Template = @"Shader ""SNeRG/RayMarchShader_OBJECT_NAME"" { 4 | Properties { 5 | mapAlpha(""Alpha Map"", 3D) = """" {} 6 | mapColor(""Color Map"", 3D) = """" {} 7 | mapFeatures(""Feature Map"", 3D) = """" {} 8 | mapIndex(""Index Map"", 3D) = """" {} 9 | 10 | weightsZero (""Weights Zero"", 2D) = ""white"" {} 11 | weightsOne (""Weights One"", 2D) = ""white"" {} 12 | weightsTwo (""Weights Two"", 2D) = ""white"" {} 13 | 14 | displayMode(""Display Mode"", Integer) = 0 15 | ndc(""NDC"", Integer) = 0 16 | 17 | minPosition (""Min Position"", Vector) = (0, 0, 0, 0) 18 | gridSize (""Grid Size"", Vector) = (0, 0, 0, 0) 19 | atlasSize (""Atlas Size"", Vector) = (0, 0, 0, 0) 20 | voxelSize (""Voxel Size"", Float) = 0.0 21 | blockSize (""Block Size"", Float) = 0.0 22 | 23 | maxStep (""Max Step"", Integer) = 0.0 24 | } 25 | SubShader { 26 | Tags { ""Queue"" = ""Transparent"" } 27 | 28 | Cull Front 29 | ZWrite Off 30 | ZTest Always 31 | Blend SrcAlpha OneMinusSrcAlpha 32 | 33 | Pass { 34 | CGPROGRAM 35 | 36 | #pragma vertex vert 37 | #pragma fragment frag 38 | 39 | #include ""UnityCG.cginc"" 40 | 41 | int displayMode; 42 | int ndc; 43 | 44 | float3 minPosition; 45 | float3 gridSize; 46 | float3 atlasSize; 47 | float voxelSize; 48 | float blockSize; 49 | int maxStep; 50 | 51 | UNITY_DECLARE_TEX3D(mapAlpha); 52 | UNITY_DECLARE_TEX3D(mapColor); 53 | UNITY_DECLARE_TEX3D(mapFeatures); 54 | UNITY_DECLARE_TEX3D(mapIndex); 55 | 56 | UNITY_DECLARE_TEX2D(weightsZero); 57 | UNITY_DECLARE_TEX2D(weightsOne); 58 | UNITY_DECLARE_TEX2D(weightsTwo); 59 | 60 | UNITY_DECLARE_DEPTH_TEXTURE(_CameraDepthTexture); 61 | 62 | struct appdata { 63 | float4 vertex : POSITION; 64 | UNITY_VERTEX_INPUT_INSTANCE_ID 65 | }; 66 | 67 | struct v2f { 68 | float4 vertex : SV_POSITION; 69 | float3 origin : TEXCOORD1; 70 | float3 direction : TEXCOORD2; 71 | float4 projPos : TEXCOORD3; 72 | float3 camRelativeWorldPos : TEXCOORD4; 73 | UNITY_VERTEX_OUTPUT_STEREO 74 | }; 75 | 76 | v2f vert (appdata v) { 77 | v2f o; 78 | 79 | UNITY_SETUP_INSTANCE_ID(v); 80 | UNITY_INITIALIZE_OUTPUT(v2f, o); 81 | UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); 82 | 83 | o.vertex = UnityObjectToClipPos(v.vertex); 84 | 85 | // set up values to raymarch in object space 86 | o.origin = mul(unity_WorldToObject, float4(_WorldSpaceCameraPos, 1)); 87 | o.direction = -ObjSpaceViewDir(v.vertex); 88 | 89 | // set up values to test against scene depth for proper composition with opaque scene objects 90 | o.projPos = ComputeScreenPos(o.vertex); 91 | o.camRelativeWorldPos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1.0)).xyz - _WorldSpaceCameraPos; 92 | 93 | return o; 94 | } 95 | 96 | half indexToPosEnc (half3 dir, uint index) { 97 | half coordinate = 98 | (index % 3 == 0) ? dir.x : ( 99 | (index % 3 == 1) ? dir.y : dir.z); 100 | if (index < 3) { 101 | return coordinate; 102 | } 103 | int scaleExponent = ((index - 3) % (3 * 4)) / 3; 104 | coordinate *= pow(2.0, float(scaleExponent)); 105 | if ((index - 3) >= 3 * 4) { 106 | const float kHalfPi = 1.57079632679489661923; 107 | coordinate += kHalfPi; 108 | } 109 | return sin(coordinate); 110 | } 111 | 112 | half3 evaluateNetwork(fixed3 color, fixed4 features, fixed3 viewdir) { 113 | half intermediate_one[NUM_CHANNELS_ONE] = { BIAS_LIST_ZERO }; 114 | uint i = 0; 115 | uint j = 0; 116 | 117 | for (j = 0; j < NUM_CHANNELS_ZERO; ++j) { 118 | half input_value = 0.0; 119 | if (j < 27) { 120 | input_value = indexToPosEnc(viewdir, j); 121 | } 122 | else if (j < 30) { 123 | input_value = 124 | (j % 3 == 0) ? color.r : ( 125 | (j % 3 == 1) ? color.g : color.b); 126 | } 127 | else { 128 | input_value = 129 | (j == 30) ? features.r : ( 130 | (j == 31) ? features.g : ( 131 | (j == 32) ? features.b : features.a)); 132 | } 133 | if (abs(input_value) < 0.1 / 255.0) { 134 | continue; 135 | } 136 | for (int i = 0; i < NUM_CHANNELS_ONE; ++i) { 137 | intermediate_one[i] += input_value * weightsZero.Load(int3(j, i, 0)).x; 138 | } 139 | } 140 | 141 | half intermediate_two[NUM_CHANNELS_TWO] = { BIAS_LIST_ONE }; 142 | 143 | for (j = 0; j < NUM_CHANNELS_ONE; ++j) { 144 | if (intermediate_one[j] <= 0.0) { 145 | continue; 146 | } 147 | for (i = 0; i < NUM_CHANNELS_TWO; ++i) { 148 | intermediate_two[i] += intermediate_one[j] * weightsOne.Load(int3(j, i, 0)).x; 149 | } 150 | } 151 | 152 | half result[NUM_CHANNELS_THREE] = { BIAS_LIST_TWO }; 153 | 154 | for (j = 0; j < NUM_CHANNELS_TWO; ++j) { 155 | if (intermediate_two[j] <= 0.0) { 156 | continue; 157 | } 158 | for (i = 0; i < NUM_CHANNELS_THREE; ++i) { 159 | result[i] += intermediate_two[j] * weightsTwo.Load(int3(j, i, 0)).x; 160 | } 161 | } 162 | for (i = 0; i < NUM_CHANNELS_THREE; ++i) { 163 | result[i] = 1.0 / (1.0 + exp(-result[i])); 164 | } 165 | 166 | return half3(result[0], result[1], result[2]); 167 | } 168 | 169 | half3 convertOriginToNDC(float3 origin, float3 direction) { 170 | // We store the NDC scenes flipped, so flip back. 171 | origin.z *= -1.0; 172 | direction.z *= -1.0; 173 | 174 | const float near = 1.0; 175 | float t = -(near + origin.z) / direction.z; 176 | origin = origin * t + direction; 177 | 178 | // Hardcoded, worked out using approximate iPhone FOV of 67.3 degrees 179 | // and an image width of 1006 px. 180 | const float focal = 755.644; 181 | const float W = 1006.0; 182 | const float H = 756.0; 183 | float o0 = 1.0 / (W / (2.0 * focal)) * origin.x / origin.z; 184 | float o1 = -1.0 / (H / (2.0 * focal)) * origin.y / origin.z; 185 | float o2 = 1.0 + 2.0 * near / origin.z; 186 | 187 | origin = float3(o0, o1, o2); 188 | origin.z *= -1.0; 189 | return origin; 190 | } 191 | 192 | half3 convertDirectionToNDC(float3 origin, float3 direction) { 193 | // We store the NDC scenes flipped, so flip back. 194 | origin.z *= -1.0; 195 | direction.z *= -1.0; 196 | 197 | const float near = 1.0; 198 | float t = -(near + origin.z) / direction.z; 199 | origin = origin * t + direction; 200 | 201 | // Hardcoded, worked out using approximate iPhone FOV of 67.3 degrees 202 | // and an image width of 1006 px. 203 | const float focal = 755.6440; 204 | const float W = 1006.0; 205 | const float H = 756.0; 206 | 207 | float d0 = 1.0 / (W / (2.0 * focal)) * 208 | (direction.x / direction.z - origin.x / origin.z); 209 | float d1 = -1.0 / (H / (2.0 * focal)) * 210 | (direction.y / direction.z - origin.y / origin.z); 211 | float d2 = -2.0 * near / origin.z; 212 | 213 | direction = normalize(float3(d0, d1, d2)); 214 | direction.z *= -1.0; 215 | return direction; 216 | } 217 | 218 | // Compute the atlas block index for a point in the scene using pancake 219 | // 3D atlas packing. 220 | half3 pancakeBlockIndex(half3 posGrid, float blockSize, float3 iBlockGridBlocks) { 221 | float3 iBlockIndex = floor(posGrid / blockSize); 222 | float3 iAtlasBlocks = atlasSize / (blockSize + 2.0); 223 | float linearIndex = iBlockIndex.x + iBlockGridBlocks.x * 224 | (iBlockIndex.z + iBlockGridBlocks.z * iBlockIndex.y); 225 | 226 | half3 atlasBlockIndex = half3( 227 | linearIndex % iAtlasBlocks.x, 228 | (linearIndex / iAtlasBlocks.x) % iAtlasBlocks.y, 229 | linearIndex / (iAtlasBlocks.x * iAtlasBlocks.y)); 230 | 231 | // If we exceed the size of the atlas, indicate an empty voxel block. 232 | if (atlasBlockIndex.z >= float(iAtlasBlocks.z)) { 233 | atlasBlockIndex = half3(-1.0, -1.0, -1.0); 234 | } 235 | 236 | return atlasBlockIndex; 237 | } 238 | 239 | half2 rayAabbIntersection(half3 aabbMin, half3 aabbMax, half3 origin, half3 invDirection) { 240 | half3 t1 = (aabbMin - origin) * invDirection; 241 | half3 t2 = (aabbMax - origin) * invDirection; 242 | half3 tMin = min(t1, t2); 243 | half3 tMax = max(t1, t2); 244 | return half2(max(tMin.x, max(tMin.y, tMin.z)), min(tMax.x, min(tMax.y, tMax.z))); 245 | } 246 | 247 | fixed4 frag (v2f i) : SV_Target { 248 | UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i); 249 | 250 | // Runs the full model with view dependence. 251 | const int DISPLAY_NORMAL = 0; 252 | // Disables the view-dependence network. 253 | const int DISPLAY_DIFFUSE = 1; 254 | // Only shows the latent features. 255 | const int DISPLAY_FEATURES = 2; 256 | // Only shows the view dependent component. 257 | const int DISPLAY_VIEW_DEPENDENT = 3; 258 | // Only shows the coarse block grid. 259 | const int DISPLAY_COARSE_GRID = 4; 260 | // Only shows the 3D texture atlas. 261 | const int DISPLAY_3D_ATLAS = 5; 262 | 263 | // Set up the ray parameters in object space.. 264 | float nearPlane = _ProjectionParams.y; 265 | half3 origin = i.origin; 266 | half3 directionWorld = normalize(i.direction); 267 | if (ndc != 0) { 268 | nearPlane = 0.0; 269 | origin = convertOriginToNDC(i.origin, normalize(i.direction)); 270 | directionWorld = convertDirectionToNDC(i.origin, normalize(i.direction)); 271 | } 272 | 273 | // Now transform them to the voxel grid coordinate system. 274 | half3 originGrid = (origin - minPosition) / voxelSize; 275 | half3 directionGrid = directionWorld; 276 | half3 invDirectionGrid = 1.0 / directionGrid; 277 | 278 | float3 iBlockGridBlocks = floor((gridSize + blockSize - 1) / blockSize); 279 | float3 blockGridSize = iBlockGridBlocks * blockSize; 280 | 281 | half2 tMinMax = rayAabbIntersection(half3(0.0, 0.0, 0.0), gridSize, originGrid, invDirectionGrid); 282 | 283 | // Skip any rays that miss the scene bounding box. 284 | if (tMinMax.x > tMinMax.y) { 285 | discard; 286 | } 287 | 288 | float t = max(nearPlane / voxelSize, tMinMax.x) + 0.5; 289 | half3 posGrid = originGrid + directionGrid * t; 290 | 291 | half3 blockMin = floor(posGrid / blockSize) * blockSize; 292 | half3 blockMax = blockMin + blockSize; 293 | half2 tBlockMinMax = rayAabbIntersection( 294 | blockMin, blockMax, originGrid, invDirectionGrid); 295 | half3 atlasBlockIndex; 296 | 297 | if (displayMode == DISPLAY_3D_ATLAS) { 298 | atlasBlockIndex = pancakeBlockIndex(posGrid, blockSize, iBlockGridBlocks); 299 | } else { 300 | atlasBlockIndex = 255.0 * UNITY_SAMPLE_TEX3D(mapIndex, (blockMin + blockMax) / (2.0 * blockGridSize)).xyz; 301 | } 302 | 303 | half visibility = 1.0; 304 | half3 color = half3(0.0, 0.0, 0.0); 305 | half4 features = half4(0.0, 0.0, 0.0, 0.0); 306 | int step = 0; 307 | 308 | // sample scene depth and calculate maximum ray distance in voxel grid space 309 | float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, i.projPos.xy / i.projPos.w).r; 310 | float linearDepth = LinearEyeDepth(depth); 311 | 312 | // reconstruct world position in voxel grid space 313 | float3 viewPlane = (i.camRelativeWorldPos) / dot((i.camRelativeWorldPos), mul(float3(0.0,0.0,-1.0), UNITY_MATRIX_V)); 314 | float3 sceneDepthWorld = viewPlane * linearDepth + _WorldSpaceCameraPos; 315 | float3 sceneDepthObj = mul(unity_WorldToObject, float4(sceneDepthWorld, 1.0)); 316 | float3 sceneDepthGrid = ((sceneDepthObj - minPosition) / voxelSize) + 0.5; 317 | 318 | // maximum ray distance to march in voxel grid space 319 | float maxDist = distance(sceneDepthGrid, originGrid); 320 | 321 | [loop] 322 | while (step < maxStep && t < maxDist && t < tMinMax.y && visibility > 1.0 / 255.0) { 323 | // Skip empty macroblocks. 324 | if (atlasBlockIndex.x > 254.0) { 325 | t = 0.5 + tBlockMinMax.y; 326 | } else { // Otherwise step through them and fetch RGBA and Features. 327 | half3 posAtlas = clamp(posGrid - blockMin, 0.0, blockSize); 328 | 329 | posAtlas += atlasBlockIndex * (blockSize + 2.0); 330 | posAtlas += 1.0; // Account for the one voxel padding in the atlas. 331 | 332 | if (displayMode == DISPLAY_COARSE_GRID) { 333 | color = atlasBlockIndex * (blockSize + 2.0) / atlasSize; 334 | features.rgb = atlasBlockIndex * (blockSize + 2.0) / atlasSize; 335 | features.a = 1.0; 336 | visibility = 0.0; 337 | continue; 338 | } 339 | 340 | // Do a conservative fetch for alpha!=0 at a lower resolution, 341 | // and skip any voxels which are empty. First, this saves bandwidth 342 | // since we only fetch one byte instead of 8 (trilinear) and most 343 | // fetches hit cache due to low res. Second, this is conservative, 344 | // and accounts for any possible alpha mass that the high resolution 345 | // trilinear would find. 346 | const int skipMipLevel = 2; 347 | const float miniBlockSize = float(1 << skipMipLevel); 348 | 349 | // Only fetch one byte at first, to conserve memory bandwidth in 350 | // empty space. 351 | float atlasAlpha = mapAlpha.Load(int4(int3(posAtlas / miniBlockSize), skipMipLevel)).x; 352 | 353 | if (atlasAlpha > 0.0) { 354 | // OK, we hit something, do a proper trilinear fetch at high res. 355 | half3 atlasUvw = posAtlas / atlasSize; 356 | atlasAlpha = UNITY_SAMPLE_TEX3D_LOD(mapAlpha, atlasUvw, 0.0).x; 357 | 358 | // Only worth fetching the content if high res alpha is non-zero. 359 | if (atlasAlpha > 0.5 / 255.0) { 360 | half4 atlasRgba = half4(0.0, 0.0, 0.0, atlasAlpha); 361 | atlasRgba.rgb = UNITY_SAMPLE_TEX3D(mapColor, atlasUvw).rgb; 362 | if (displayMode != DISPLAY_DIFFUSE) { 363 | half4 atlasFeatures = UNITY_SAMPLE_TEX3D(mapFeatures, atlasUvw); 364 | features += visibility * atlasFeatures; 365 | } 366 | color += visibility * atlasRgba.rgb; 367 | visibility *= 1.0 - atlasRgba.a; 368 | } 369 | } 370 | t += 1.0; 371 | } 372 | 373 | posGrid = originGrid + directionGrid * t; 374 | if (t > tBlockMinMax.y) { 375 | blockMin = floor(posGrid / blockSize) * blockSize; 376 | blockMax = blockMin + blockSize; 377 | tBlockMinMax = rayAabbIntersection(blockMin, blockMax, originGrid, invDirectionGrid); 378 | 379 | if (displayMode == DISPLAY_3D_ATLAS) { 380 | atlasBlockIndex = pancakeBlockIndex(posGrid, blockSize, iBlockGridBlocks); 381 | } else { 382 | atlasBlockIndex = 255.0 * UNITY_SAMPLE_TEX3D(mapIndex, (blockMin + blockMax) / (2.0 * blockGridSize)).xyz; 383 | } 384 | } 385 | step++; 386 | } 387 | 388 | if (displayMode == DISPLAY_VIEW_DEPENDENT) { 389 | color = half3(0.0, 0.0, 0.0) * visibility; 390 | } else if (displayMode == DISPLAY_FEATURES) { 391 | color = features.rgb; 392 | } 393 | 394 | // For forward-facing scenes, we partially unpremultiply alpha to fill 395 | // tiny holes in the rendering. 396 | half alpha = 1.0 - visibility; 397 | if (ndc != 0 && alpha > 0.0) { 398 | half filledAlpha = min(1.0, alpha * 1.5); 399 | color *= filledAlpha / alpha; 400 | alpha = filledAlpha; 401 | visibility = 1.0 - filledAlpha; 402 | } 403 | 404 | // convert from Unity's right handed coordinate system 405 | // to OpenGL coordinates used in the MLP evaluation 406 | i.direction.xz = -i.direction.xz; 407 | i.direction.yz = i.direction.zy; 408 | 409 | // Compute the final color, to save compute only compute view-dependence 410 | // for rays that intersected something in the scene. 411 | color = half3(1.0, 1.0, 1.0) * visibility + color; 412 | const float kVisibilityThreshold = 254.0 / 255.0; 413 | if (visibility <= kVisibilityThreshold && 414 | (displayMode == DISPLAY_NORMAL || 415 | displayMode == DISPLAY_VIEW_DEPENDENT)) { 416 | color += evaluateNetwork(color, features, normalize(i.direction)); 417 | } 418 | 419 | #if !UNITY_COLORSPACE_GAMMA 420 | color = GammaToLinearSpace(color); 421 | #endif 422 | 423 | return fixed4(color, alpha); 424 | } 425 | ENDCG 426 | } 427 | } 428 | }"; 429 | } 430 | } -------------------------------------------------------------------------------- /Editor/RaymarchShader.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 34c0bfff7aac44b4cbb7f9d3bf2521f5 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /Editor/SNeRGLoader.cs: -------------------------------------------------------------------------------- 1 | using Newtonsoft.Json; 2 | using System.IO; 3 | using System.Text; 4 | using System.Text.RegularExpressions; 5 | using System.Threading.Tasks; 6 | using Unity.Collections; 7 | using UnityEditor; 8 | using UnityEngine; 9 | using static SNeRG.Editor.WebRequestAsyncUtility; 10 | 11 | namespace SNeRG.Editor { 12 | 13 | public class SNeRGLoader { 14 | 15 | private static readonly string LoadingTitle = "Loading Assets"; 16 | private static readonly string ProcessingTitle = "Processing Assets"; 17 | private static readonly string DownloadInfo = "Loading Assets for "; 18 | private static readonly string FolderTitle = "Select folder with SNeRG source files"; 19 | private static readonly string FolderExistsTitle = "Folder already exists"; 20 | private static readonly string FolderExistsMsg = "A folder for this asset already exists in the Unity project. Overwrite?"; 21 | private static readonly string OK = "OK"; 22 | private static readonly string ImportErrorTitle = "Error importing SNeRG assets"; 23 | 24 | [MenuItem("SNeRG/Asset Downloads/-- Synthetic Rendered Scenes --", false, -1)] 25 | public static void Separator0() { } 26 | [MenuItem("SNeRG/Asset Downloads/-- Synthetic Rendered Scenes --", true, -1)] 27 | public static bool Separator0Validate() { 28 | return false; 29 | } 30 | [MenuItem("SNeRG/Asset Downloads/-- Real Captured Scenes --", false, 49)] 31 | public static void Separator1() { } 32 | [MenuItem("SNeRG/Asset Downloads/-- Real Captured Scenes --", true, 49)] 33 | public static bool Separator1Validate() { 34 | return false; 35 | } 36 | 37 | [MenuItem("SNeRG/Import from disk", false, 0)] 38 | public async static void ImportAssetsFromDisk() { 39 | // select folder with custom data 40 | string path = EditorUtility.OpenFolderPanel(FolderTitle, "", ""); 41 | if (string.IsNullOrEmpty(path) || !Directory.Exists(path)) { 42 | return; 43 | } 44 | 45 | // ask whether to overwrite existing folder 46 | string objName = new DirectoryInfo(path).Name; 47 | if (Directory.Exists($"{BASE_FOLDER}{objName}")) { 48 | if (!EditorUtility.DisplayDialog(FolderExistsTitle, FolderExistsMsg, OK)) { 49 | return; 50 | } 51 | } 52 | 53 | await ImportCustomScene(path); 54 | } 55 | 56 | [MenuItem("SNeRG/Asset Downloads/Lego", false, 0)] 57 | public async static void DownloadLegoAssets() { 58 | await ImportDemoSceneAsync(SNeRGScene.Lego); 59 | } 60 | [MenuItem("SNeRG/Asset Downloads/Chair", false, 0)] 61 | public async static void DownloadChairAssets() { 62 | await ImportDemoSceneAsync(SNeRGScene.Chair); 63 | } 64 | [MenuItem("SNeRG/Asset Downloads/Drums", false, 0)] 65 | public async static void DownloadDrumsAssets() { 66 | await ImportDemoSceneAsync(SNeRGScene.Drums); 67 | } 68 | [MenuItem("SNeRG/Asset Downloads/Hotdog", false, 0)] 69 | public async static void DownloadHotdogAssets() { 70 | await ImportDemoSceneAsync(SNeRGScene.Hotdog); 71 | } 72 | [MenuItem("SNeRG/Asset Downloads/Ship", false, 0)] 73 | public async static void DownloadShipsAssets() { 74 | await ImportDemoSceneAsync(SNeRGScene.Ship); 75 | } 76 | [MenuItem("SNeRG/Asset Downloads/Mic", false, 0)] 77 | public async static void DownloadMicAssets() { 78 | await ImportDemoSceneAsync(SNeRGScene.Mic); 79 | } 80 | [MenuItem("SNeRG/Asset Downloads/Ficus", false, 0)] 81 | public async static void DownloadFicusAssets() { 82 | await ImportDemoSceneAsync(SNeRGScene.Ficus); 83 | } 84 | [MenuItem("SNeRG/Asset Downloads/Materials", false, 0)] 85 | public async static void DownloadMaterialsAssets() { 86 | await ImportDemoSceneAsync(SNeRGScene.Materials); 87 | } 88 | 89 | [MenuItem("SNeRG/Asset Downloads/Spheres", false, 50)] 90 | public async static void DownloadSpheresAssets() { 91 | await ImportDemoSceneAsync(SNeRGScene.Spheres); 92 | } 93 | [MenuItem("SNeRG/Asset Downloads/Vase Deck", false, 50)] 94 | public async static void DownloadVaseDeckAssets() { 95 | await ImportDemoSceneAsync(SNeRGScene.VaseDeck); 96 | } 97 | [MenuItem("SNeRG/Asset Downloads/Pine Cone", false, 50)] 98 | public async static void DownloadPineConeAssets() { 99 | await ImportDemoSceneAsync(SNeRGScene.PineCone); 100 | } 101 | [MenuItem("SNeRG/Asset Downloads/Toy Car", false, 50)] 102 | public async static void DownloadToyCarAssets() { 103 | await ImportDemoSceneAsync(SNeRGScene.ToyCar); 104 | } 105 | 106 | private const string BASE_URL_SYNTH = "https://storage.googleapis.com/snerg/750/"; 107 | private const string BASE_URL_REAL = "https://storage.googleapis.com/snerg/real_1000/"; 108 | 109 | private static readonly string BASE_FOLDER = Path.Combine("Assets", "SNeRG Data"); 110 | private static readonly string BASE_LIB_FOLDER = Path.Combine("Library", "Cached SNeRG Data"); 111 | 112 | private static ImportContext _context; 113 | 114 | private static string GetBasePath(string sceneName) { 115 | return Path.Combine(BASE_FOLDER, sceneName); 116 | } 117 | private static string GetCacheLocation(string sceneName) { 118 | return Path.Combine(BASE_LIB_FOLDER, sceneName); 119 | } 120 | private static string GetBaseUrl(SNeRGScene scene) { 121 | if (scene.IsSynthetic()) { 122 | return $"{BASE_URL_SYNTH}{scene.LowerCaseName()}"; 123 | } else { 124 | return $"{BASE_URL_REAL}{scene.LowerCaseName()}"; 125 | } 126 | } 127 | 128 | private static string GetSceneParamsUrl() { 129 | return $"{GetBaseUrl(_context.Scene)}/scene_params.json"; 130 | } 131 | private static string GetAtlasIndexUrl() { 132 | return $"{GetBaseUrl(_context.Scene)}/atlas_indices.png"; 133 | } 134 | private static string GetRGBVolumeUrl(int i) { 135 | return $"{GetBaseUrl(_context.Scene)}/rgba_{i:D3}.png"; 136 | } 137 | private static string GetFeatureVolumeUrl(int i) { 138 | return $"{GetBaseUrl(_context.Scene)}/feature_{i:D3}.png"; 139 | } 140 | 141 | private static string GetSceneParamsAssetPath() { 142 | return GetAssetPath("SceneParams", $"{_context.SceneName}.asset"); 143 | } 144 | private static string GetRGBTextureAssetPath() { 145 | return GetAssetPath("Textures", $"{_context.SceneName} RGB Volume Texture.asset"); 146 | } 147 | private static string GetAlphaTextureAssetPath() { 148 | return GetAssetPath("Textures", $"{_context.SceneName} Alpha Volume Texture.asset"); 149 | } 150 | private static string GetFeatureTextureAssetPath() { 151 | return GetAssetPath("Textures", $"{_context.SceneName} Feature Volume Texture.asset"); 152 | } 153 | private static string GetAtlasTextureAssetPath() { 154 | return GetAssetPath("Textures", $"{_context.SceneName} Atlas Index Texture.asset"); 155 | } 156 | private static string GetShaderAssetPath() { 157 | return GetAssetPath("Shaders", $"RayMarchShader_{_context.SceneName}.shader"); 158 | } 159 | private static string GetMaterialAssetPath() { 160 | return GetAssetPath("Materials", $"Material_{_context.SceneName}.mat"); 161 | } 162 | private static string GetWeightsAssetPath(int i) { 163 | return GetAssetPath("SceneParams", $"weightsTex{i}.asset"); 164 | } 165 | private static string GetPrefabAssetPath() { 166 | return GetAssetPath("", $"{_context.SceneName}.prefab"); 167 | } 168 | private static string GetMeshAssetPath() { 169 | return GetAssetPath("Volume Mesh", $"{_context.SceneName} Volume Mesh.asset"); 170 | } 171 | /// 172 | /// This returns a path in the asset directory to store the specific asset into. 173 | /// 174 | private static string GetAssetPath(string subFolder, string assetName) { 175 | string path = Path.Combine(GetBasePath(_context.SceneName), subFolder, assetName); 176 | Directory.CreateDirectory(Path.GetDirectoryName(path)); 177 | return path; 178 | } 179 | 180 | private static string GetAtlasIndexCachePath() { 181 | return GetCachePath("atlas_indices.png"); 182 | } 183 | private static string GetRGBVolumeCachePath(int i) { 184 | return GetCachePath($"rgba_{i:D3}.png"); 185 | } 186 | private static string GetFeatureVolumeCachePath(int i) { 187 | return GetCachePath($"feature_{i:D3}.png"); 188 | } 189 | /// 190 | /// This is either the location where demo scenes are first downloaded to, 191 | /// or the path to the source files for custom scene imports. 192 | /// 193 | private static string GetCachePath(string assetName) { 194 | string path; 195 | if (_context.CustomScene) { 196 | path = Path.Combine(_context.CustomScenePath, assetName); 197 | } else { 198 | path = Path.Combine(GetCacheLocation(_context.SceneName), assetName); 199 | Directory.CreateDirectory(Path.GetDirectoryName(path)); 200 | } 201 | return path; 202 | } 203 | 204 | /// 205 | /// Creates Unity assets for the given SNeRG assets on disk. 206 | /// 207 | /// The path to the folder with the SNeRG assets (PNGs & mlp.json) 208 | private static async Task ImportCustomScene(string path) { 209 | _context = new ImportContext() { 210 | CustomScene = true, 211 | CustomScenePath = path, 212 | Scene = SNeRGScene.Custom 213 | }; 214 | string objName = new DirectoryInfo(path).Name; 215 | 216 | SceneParams sceneParams = CopySceneParamsFromPath(path); 217 | if (sceneParams == null) { 218 | return; 219 | } 220 | if (!ValidatePNGs(path, sceneParams)) { 221 | return; 222 | } 223 | 224 | AssetDatabase.SaveAssets(); 225 | AssetDatabase.Refresh(); 226 | 227 | await ProcessAssets(objName); 228 | } 229 | 230 | /// 231 | /// Downloads the source files for the given SNeRG demo scene 232 | /// into the folder. Then in the 233 | /// Unity project, creates the Unity assets necessary to display it. 234 | /// 235 | private static async Task ImportDemoSceneAsync(SNeRGScene scene) { 236 | _context = new ImportContext() { 237 | CustomScene = false, 238 | Scene = scene 239 | }; 240 | 241 | EditorUtility.DisplayProgressBar(LoadingTitle, $"{DownloadInfo}'{scene.Name()}'...", 0.1f); 242 | await DownloadSceneParamsAsync(); 243 | 244 | await ProcessAssets(scene.Name()); 245 | } 246 | 247 | /// 248 | /// Set specific import settings on OBJs/PNGs. 249 | /// Creates Weight Textures, Materials and Shader from MLP data. 250 | /// Creates a convenient prefab for the SNeRG object. 251 | /// 252 | private static async Task ProcessAssets(string sceneName) { 253 | var sceneParams = GetSceneParams(); 254 | 255 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Raymarch Shader...", 0.2f); 256 | CreateRayMarchShader(sceneParams); 257 | 258 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Material...", 0.3f); 259 | CreateMaterial(sceneParams); 260 | 261 | // load 3D slices from web or cacbe directory, then create 3D volume textures from that data 262 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Atlas Index Texture...", 0.4f); 263 | Texture2D atlasIndexData = await LoadAtlasIndexDataAsync(); 264 | CreateAtlasIndexTexture(atlasIndexData, sceneParams); 265 | 266 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' RGB Volume Texture...", 0.5f); 267 | Texture2D[] rgbImages = await LoadRGBVolumeDataAsync(sceneParams); 268 | CreateRgbVolumeTexture(rgbImages, sceneParams); 269 | 270 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Feature Volume Texture...", 0.6f); 271 | Texture2D[] featureImages = await LoadFeatureVolumeDataAsync(sceneParams); 272 | CreateFeatureVolumeTexture(featureImages, sceneParams); 273 | 274 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Creating '{sceneName}' Weight Textures...", 0.7f); 275 | CreateWeightTextures(sceneParams); 276 | 277 | EditorUtility.DisplayProgressBar(ProcessingTitle, $"Finishing '{sceneName}' assets..", 0.8f); 278 | VerifyMaterial(sceneParams); 279 | 280 | CreatePrefab(sceneParams); 281 | 282 | EditorUtility.ClearProgressBar(); 283 | } 284 | 285 | /// 286 | /// Looks for a scene_params.json at and imports it. 287 | /// 288 | private static SceneParams CopySceneParamsFromPath(string path) { 289 | string[] sceneParamsPaths = Directory.GetFiles(path, "scene_params.json", SearchOption.AllDirectories); 290 | if (sceneParamsPaths.Length > 1) { 291 | EditorUtility.DisplayDialog(ImportErrorTitle, "Multiple scene_params.json files found", OK); 292 | return null; 293 | } 294 | if (sceneParamsPaths.Length <= 0) { 295 | EditorUtility.DisplayDialog(ImportErrorTitle, "No scene_params.json files found", OK); 296 | return null; 297 | } 298 | 299 | string sceneParamsJson = File.ReadAllText(sceneParamsPaths[0]); 300 | TextAsset sceneParamsTextAsset = new TextAsset(sceneParamsJson); 301 | AssetDatabase.CreateAsset(sceneParamsTextAsset, GetSceneParamsAssetPath()); 302 | SceneParams sceneParams = JsonConvert.DeserializeObject(sceneParamsJson); 303 | return sceneParams; 304 | } 305 | 306 | private static async Task DownloadSceneParamsAsync() { 307 | string url = GetSceneParamsUrl(); 308 | string sceneParamsJson = await WebRequestSimpleAsync.SendWebRequestAsync(url); 309 | TextAsset sceneParamsTextAsset = new TextAsset(sceneParamsJson); 310 | AssetDatabase.CreateAsset(sceneParamsTextAsset, GetSceneParamsAssetPath()); 311 | } 312 | 313 | private static SceneParams GetSceneParams() { 314 | string mlpJson = AssetDatabase.LoadAssetAtPath(GetSceneParamsAssetPath()).text; 315 | return JsonConvert.DeserializeObject(mlpJson); 316 | } 317 | 318 | /// 319 | /// Checks if all necessary textures for a given SNeRG scene are present in the given folder. 320 | /// 321 | private static bool ValidatePNGs(string path, SceneParams mlp) { 322 | string objName = new DirectoryInfo(path).Name; 323 | int numSlicePNGs = mlp.NumSlices; 324 | 325 | string[] pngPaths; 326 | pngPaths = Directory.GetFiles(path, "feature_*.png", SearchOption.TopDirectoryOnly); 327 | if (pngPaths.Length != numSlicePNGs) { 328 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Invalid number of feature textures found. Expected: {numSlicePNGs}. Actual: {pngPaths.Length}", OK); 329 | return false; 330 | } 331 | pngPaths = Directory.GetFiles(path, "rgba_*.png", SearchOption.TopDirectoryOnly); 332 | if (pngPaths.Length != numSlicePNGs) { 333 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Invalid number of feature textures found. Expected: {numSlicePNGs}. Actual: {pngPaths.Length}", OK); 334 | return false; 335 | } 336 | 337 | string atlasPath = Path.Combine(path, "atlas_indices.png"); 338 | if (!File.Exists(atlasPath)) { 339 | EditorUtility.DisplayDialog(ImportErrorTitle, $"Could not find atlas indices texture.", OK); 340 | return false; 341 | } 342 | return true; 343 | } 344 | 345 | private static async Task LoadAtlasIndexDataAsync() { 346 | string path = GetAtlasIndexCachePath(); 347 | byte[] atlasIndexData; 348 | 349 | if (File.Exists(path)) { 350 | // file is already downloaded 351 | atlasIndexData = File.ReadAllBytes(path); 352 | } else { 353 | string url = GetAtlasIndexUrl(); 354 | atlasIndexData = await WebRequestBinaryAsync.SendWebRequestAsync(url); 355 | File.WriteAllBytes(path, atlasIndexData); 356 | } 357 | 358 | // !!! Unity's LoadImage() does NOT respect the texture format specified in the input texture! 359 | // It always loads this as ARGB32, no matter the format specified here. 360 | // Ideally we'd directly load an RGB24 texture. 361 | Texture2D atlasIndexImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true); 362 | atlasIndexImage.filterMode = FilterMode.Point; 363 | atlasIndexImage.wrapMode = TextureWrapMode.Clamp; 364 | atlasIndexImage.LoadImage(atlasIndexData); 365 | 366 | return atlasIndexImage; 367 | } 368 | 369 | private static async Task LoadRGBVolumeDataAsync(SceneParams sceneParams) { 370 | Texture2D[] rgbVolumeArray = new Texture2D[sceneParams.NumSlices]; 371 | for (int i = 0; i < sceneParams.NumSlices; i++) { 372 | string path = GetRGBVolumeCachePath(i); 373 | byte[] rgbVolumeData; 374 | 375 | if (File.Exists(path)) { 376 | // file is already downloaded 377 | rgbVolumeData = File.ReadAllBytes(path); 378 | } else { 379 | string url = GetRGBVolumeUrl(i); 380 | rgbVolumeData = await WebRequestBinaryAsync.SendWebRequestAsync(url); 381 | File.WriteAllBytes(path, rgbVolumeData); 382 | } 383 | 384 | // Unity's LoadImage() always loads this as ARGB32, no matter the format specified here 385 | Texture2D rgbVolumeImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true); 386 | rgbVolumeImage.filterMode = FilterMode.Point; 387 | rgbVolumeImage.wrapMode = TextureWrapMode.Clamp; 388 | rgbVolumeImage.alphaIsTransparency = true; 389 | rgbVolumeImage.LoadImage(rgbVolumeData); 390 | rgbVolumeArray[i] = rgbVolumeImage; 391 | } 392 | 393 | return rgbVolumeArray; 394 | } 395 | 396 | private static async Task LoadFeatureVolumeDataAsync(SceneParams sceneParams) { 397 | Texture2D[] featureVolumeArray = new Texture2D[sceneParams.NumSlices]; 398 | 399 | for (int i = 0; i < sceneParams.NumSlices; i++) { 400 | string path = GetFeatureVolumeCachePath(i); 401 | byte[] featureVolumeData; 402 | 403 | if (File.Exists(path)) { 404 | // file is already downloaded 405 | featureVolumeData = File.ReadAllBytes(path); 406 | } else { 407 | string url = GetFeatureVolumeUrl(i); 408 | featureVolumeData = await WebRequestBinaryAsync.SendWebRequestAsync(url); 409 | File.WriteAllBytes(path, featureVolumeData); 410 | } 411 | 412 | // Unity's LoadImage() always loads this as ARGB32, no matter the format specified here 413 | Texture2D featureVolumeImage = new Texture2D(2, 2, TextureFormat.ARGB32, mipChain: false, linear: true); 414 | featureVolumeImage.filterMode = FilterMode.Point; 415 | featureVolumeImage.wrapMode = TextureWrapMode.Clamp; 416 | featureVolumeImage.LoadImage(featureVolumeData); 417 | featureVolumeArray[i] = featureVolumeImage; 418 | } 419 | 420 | return featureVolumeArray; 421 | } 422 | 423 | private static void CreateRgbVolumeTexture(Texture2D[] rgbaData, SceneParams sceneParams) { 424 | string rgbAssetPath = GetRGBTextureAssetPath(); 425 | string alphaAssetPath = GetAlphaTextureAssetPath(); 426 | 427 | // already exists 428 | if (File.Exists(rgbAssetPath) && File.Exists(alphaAssetPath)) { 429 | return; 430 | } 431 | 432 | // initialize 3D textures 433 | Texture3D rgbVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.RGB24, mipChain: false) { 434 | filterMode = FilterMode.Bilinear, 435 | wrapMode = TextureWrapMode.Clamp, 436 | name = Path.GetFileNameWithoutExtension(rgbAssetPath) 437 | }; 438 | 439 | Texture3D alphaVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.R8, mipChain: true) { 440 | filterMode = FilterMode.Trilinear, // original code uses different min mag filters, which Unity doesn't let us do, tbd 441 | wrapMode = TextureWrapMode.Clamp, 442 | name = Path.GetFileNameWithoutExtension(alphaAssetPath) 443 | }; 444 | 445 | // load data into 3D textures 446 | loadSplitVolumeTexture(rgbaData, alphaVolumeTexture, rgbVolumeTexture, sceneParams); 447 | 448 | // destroy intermediate textures to free memory 449 | for (int i = rgbaData.Length - 1; i >= 0; i--) { 450 | UnityEngine.Object.DestroyImmediate(rgbaData[i]); 451 | } 452 | Resources.UnloadUnusedAssets(); 453 | 454 | AssetDatabase.CreateAsset(rgbVolumeTexture, rgbAssetPath); 455 | AssetDatabase.CreateAsset(alphaVolumeTexture, alphaAssetPath); 456 | 457 | string materialAssetPath = GetMaterialAssetPath(); 458 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 459 | material.SetTexture("mapColor", rgbVolumeTexture); 460 | material.SetTexture("mapAlpha", alphaVolumeTexture); 461 | 462 | AssetDatabase.SaveAssets(); 463 | } 464 | 465 | private static void CreateFeatureVolumeTexture(Texture2D[] featureData, SceneParams sceneParams) { 466 | string featureAssetPath = GetFeatureTextureAssetPath(); 467 | 468 | // already exists 469 | if (File.Exists(featureAssetPath)) { 470 | return; 471 | } 472 | 473 | // initialize 3D texture 474 | Texture3D featureVolumeTexture = new Texture3D(sceneParams.AtlasWidth, sceneParams.AtlasHeight, sceneParams.AtlasDepth, TextureFormat.ARGB32, mipChain: false) { 475 | filterMode = FilterMode.Bilinear, 476 | wrapMode = TextureWrapMode.Clamp, 477 | name = Path.GetFileNameWithoutExtension(featureAssetPath) 478 | }; 479 | 480 | // load data into 3D textures 481 | LoadVolumeTexture(featureData, featureVolumeTexture, sceneParams); 482 | 483 | // destroy intermediate textures to free memory 484 | for (int i = featureData.Length - 1; i >= 0; i--) { 485 | UnityEngine.Object.DestroyImmediate(featureData[i]); 486 | } 487 | Resources.UnloadUnusedAssets(); 488 | 489 | AssetDatabase.CreateAsset(featureVolumeTexture, featureAssetPath); 490 | 491 | string materialAssetPath = GetMaterialAssetPath(); 492 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 493 | material.SetTexture("mapFeatures", featureVolumeTexture); 494 | } 495 | 496 | private static void CreateAtlasIndexTexture(Texture2D atlasIndexImage, SceneParams sceneParams) { 497 | int width = (int)Mathf.Ceil(sceneParams.GridWidth / (float)sceneParams.BlockSize); 498 | int height = (int)Mathf.Ceil(sceneParams.GridHeight / (float)sceneParams.BlockSize); 499 | int depth = (int)Mathf.Ceil(sceneParams.GridDepth / (float)sceneParams.BlockSize); 500 | 501 | string atlasAssetPath = GetAtlasTextureAssetPath(); 502 | 503 | // already exists 504 | if (File.Exists(atlasAssetPath)) { 505 | return; 506 | } 507 | 508 | // initialize 3D texture 509 | Texture3D atlasIndexTexture = new Texture3D(width, height, depth, TextureFormat.ARGB32, mipChain: false) { 510 | filterMode = FilterMode.Point, 511 | wrapMode = TextureWrapMode.Clamp, 512 | name = Path.GetFileNameWithoutExtension(atlasAssetPath), 513 | }; 514 | 515 | // load data into 3D textures 516 | NativeArray rawAtlasIndexData = atlasIndexImage.GetRawTextureData(); 517 | atlasIndexTexture.SetPixelData(rawAtlasIndexData, 0); 518 | 519 | // flip the y axis for each depth slice 520 | FlipY(atlasIndexTexture); 521 | 522 | // reverse the slices of the 3D texture 523 | //FlipZ(atlasIndexTexture); 524 | 525 | atlasIndexTexture.Apply(); 526 | 527 | // destroy intermediate texture to free memory 528 | UnityEngine.Object.DestroyImmediate(atlasIndexImage); 529 | Resources.UnloadUnusedAssets(); 530 | 531 | AssetDatabase.CreateAsset(atlasIndexTexture, atlasAssetPath); 532 | 533 | string materialAssetPath = GetMaterialAssetPath(); 534 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 535 | material.SetTexture("mapIndex", atlasIndexTexture); 536 | } 537 | 538 | /// 539 | /// Fills two distinct 3D textures from a set of atlased PNGs. 540 | /// This method flips both y and z axes of the 3D texture, because the original code assumes we're 541 | /// indexing from top left, but Unity loaded the PNGs starting bottom right. 542 | /// 543 | private static void loadSplitVolumeTexture(Texture2D[] rgbaArray, Texture3D alphaVolumeTexture, Texture3D rgbVolumeTexture, SceneParams sceneParams) { 544 | Debug.Assert(sceneParams.NumSlices == rgbaArray.Length, "Expected " + sceneParams.NumSlices + " RGBA slices, but found " + rgbaArray.Length); 545 | 546 | int volumeWidth = sceneParams.AtlasWidth; 547 | int volumeHeight = sceneParams.AtlasHeight; 548 | int volumeDepth = sceneParams.AtlasDepth; 549 | 550 | int sliceDepth = 4; // slices packed into one atlased texture 551 | int numSlices = sceneParams.NumSlices; // number of slice atlases 552 | int ppAtlas = volumeWidth * volumeHeight * sliceDepth; // pixels per atlased texture 553 | int ppSlice = volumeWidth * volumeHeight; // pixels per volume slice 554 | 555 | NativeArray rgbPixels = rgbVolumeTexture.GetPixelData(0); 556 | NativeArray alphaPixels = alphaVolumeTexture.GetPixelData(0); 557 | 558 | Debug.Assert(rgbPixels.Length == numSlices * ppAtlas * 3, "Mismatching RGB Texture Data. Expected: " + numSlices * ppAtlas * 3 + ". Actual: + " + rgbPixels.Length); 559 | Debug.Assert(alphaPixels.Length == numSlices * ppAtlas, "Mismatching alpha Texture Data. Expected: " + numSlices * ppAtlas + ". Actual: + " + alphaPixels.Length); 560 | 561 | for (int i = 0; i < numSlices; i++) { 562 | // rgba images are in ARGB format! 563 | NativeArray rgbaImageFourSlices = rgbaArray[i].GetPixelData(0); 564 | Debug.Assert(rgbaImageFourSlices.Length == ppAtlas * 4, "Mismatching RGBA Texture Data. Expected: " + ppAtlas * 4 + ". Actual: + " + rgbaImageFourSlices.Length); 565 | 566 | for (int s_r = sliceDepth - 1, s = 0; s_r >= 0; s_r--, s++) { 567 | 568 | int baseIndexRGB = (i * ppAtlas + s * ppSlice) * 3; 569 | int baseIndexAlpha = (i * ppAtlas + s * ppSlice); 570 | for (int j = 0; j < ppSlice; j++) { 571 | rgbPixels[baseIndexRGB + (j * 3)] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 1]; 572 | rgbPixels[baseIndexRGB + (j * 3) + 1] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 2]; 573 | rgbPixels[baseIndexRGB + (j * 3) + 2] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4) + 3]; 574 | alphaPixels[baseIndexAlpha + j] = rgbaImageFourSlices[((s_r * ppSlice + j) * 4)]; 575 | } 576 | } 577 | } 578 | 579 | FlipY(rgbVolumeTexture); 580 | FlipZ(rgbVolumeTexture, sceneParams.AtlasBlocksZ); 581 | FlipY(alphaVolumeTexture); 582 | FlipZ(alphaVolumeTexture, sceneParams.AtlasBlocksZ); 583 | 584 | rgbVolumeTexture.Apply(updateMipmaps: false, makeNoLongerReadable: true); 585 | alphaVolumeTexture.Apply(updateMipmaps: true, makeNoLongerReadable: true); 586 | } 587 | 588 | /// 589 | /// Fills an existing 3D RGBA texture from atlased PNGs. 590 | /// This method flips both y and z axes of the 3D texture, because the original code assumes we're 591 | /// indexing from top left, but Unity loaded the PNGs starting bottom right. 592 | /// 593 | private static void LoadVolumeTexture(Texture2D[] featureImages, Texture3D featureVolumeTexture, SceneParams sceneParams) { 594 | Debug.Assert(sceneParams.NumSlices == featureImages.Length, "Expected " + sceneParams.NumSlices + " feature slices, but found " + featureImages.Length); 595 | 596 | int volumeWidth = sceneParams.AtlasWidth; 597 | int volumeHeight = sceneParams.AtlasHeight; 598 | int volumeDepth = sceneParams.AtlasDepth; 599 | 600 | int sliceDepth = 4; // slices packed into one atlased texture 601 | long numSlices = sceneParams.NumSlices; // number of slice atlases 602 | int ppAtlas = volumeWidth * volumeHeight * sliceDepth; // pixels per atlased feature texture 603 | int ppSlice = volumeWidth * volumeHeight; // pixels per volume slice 604 | 605 | NativeArray featurePixels = featureVolumeTexture.GetPixelData(0); 606 | Debug.Assert(featurePixels.Length == numSlices * ppAtlas, "Mismatching RGB Texture Data. Expected: " + numSlices * ppAtlas + ". Actual: + " + featurePixels.Length); 607 | 608 | for (int i = 0; i < numSlices; i++) { 609 | 610 | NativeArray _featureImageFourSlices = featureImages[i].GetRawTextureData(); 611 | Debug.Assert(_featureImageFourSlices.Length == ppAtlas, "Mismatching feature Texture Data. Expected: " + ppAtlas + ". Actual: + " + _featureImageFourSlices.Length); 612 | 613 | for (int s_r = sliceDepth - 1, s = 0; s_r >= 0; s_r--, s++) { 614 | int targetIndex = (i * ppAtlas) + (s * ppSlice); 615 | NativeSlice dst = new NativeSlice(featurePixels, targetIndex, ppSlice); 616 | NativeSlice src = new NativeSlice(_featureImageFourSlices, s_r * ppSlice, ppSlice); 617 | 618 | dst.CopyFrom(src); 619 | } 620 | } 621 | FlipY(featureVolumeTexture); 622 | FlipZ(featureVolumeTexture, sceneParams.AtlasBlocksZ); 623 | 624 | featureVolumeTexture.Apply(updateMipmaps: false, makeNoLongerReadable: true); 625 | } 626 | 627 | /// 628 | /// Vertically flips each depth slice in the given 3D texture. 629 | /// 630 | private static void FlipY(Texture3D texture) where T : struct { 631 | int width = texture.width; 632 | int height = texture.height; 633 | int depth = texture.depth; 634 | NativeArray data = texture.GetPixelData(0); 635 | for (int z = 0; z < depth; z++) { 636 | for (int y = 0; y < height / 2; y++) { 637 | for (int x = 0; x < width; x++) { 638 | int flippedY = height - y - 1; 639 | int source = z * (width * height) + (flippedY * width) + x; 640 | int target = z * (width * height) + (y * width) + x; 641 | (data[target], data[source]) = (data[source], data[target]); 642 | } 643 | } 644 | } 645 | } 646 | 647 | /// 648 | /// Flips z - Reverses the order of the depth slices of the given 3D texture. 649 | /// Atlases might be divided in macro blocks that are treated individually here. 650 | /// I.e. depth slices are only reversed within a block. 651 | /// 652 | private static void FlipZ(Texture3D texture, int atlasBlocksZ) where T : struct { 653 | int width = texture.width; 654 | int height = texture.height; 655 | int depth = texture.depth; 656 | int stride = depth / atlasBlocksZ; 657 | int blockSize = width * height * stride; 658 | int sliceSize = width * height; 659 | 660 | NativeArray data = texture.GetPixelData(0); 661 | NativeArray tmp = new NativeArray(sliceSize, Allocator.Temp); 662 | 663 | for (int z = 0; z < atlasBlocksZ; z++) { 664 | for (int s = 0; s < stride / 2; s++) { 665 | int atlasBlock = z * blockSize; 666 | int slice1Index = atlasBlock + s * sliceSize; 667 | int slice2Index = atlasBlock + ((stride - s - 1) * sliceSize); 668 | 669 | NativeSlice slice1 = new NativeSlice(data, slice1Index, sliceSize); 670 | NativeSlice slice2 = new NativeSlice(data, slice2Index, sliceSize); 671 | 672 | slice1.CopyTo(tmp); 673 | slice1.CopyFrom(slice2); 674 | slice2.CopyFrom(tmp); 675 | } 676 | } 677 | 678 | tmp.Dispose(); 679 | } 680 | 681 | private struct Color24 { 682 | public byte r; 683 | public byte g; 684 | public byte b; 685 | } 686 | 687 | private static void CreateRayMarchShader(SceneParams sceneParams) { 688 | double[][] Weights_0 = sceneParams._0Weights; 689 | double[][] Weights_1 = sceneParams._1Weights; 690 | double[][] Weights_2 = sceneParams._2Weights; 691 | double[] Bias_0 = sceneParams._0Bias; 692 | double[] Bias_1 = sceneParams._1Bias; 693 | double[] Bias_2 = sceneParams._2Bias; 694 | 695 | StringBuilder biasListZero = toBiasList(Bias_0); 696 | StringBuilder biasListOne = toBiasList(Bias_1); 697 | StringBuilder biasListTwo = toBiasList(Bias_2); 698 | 699 | int channelsZero = Weights_0.Length; 700 | int channelsOne = Bias_0.Length; 701 | int channelsTwo = Bias_1.Length; 702 | int channelsThree = Bias_2.Length; 703 | int posEncScales = 4; 704 | 705 | string shaderSource = RaymarchShader.Template; 706 | shaderSource = new Regex("OBJECT_NAME").Replace(shaderSource, $"{_context.SceneNameUpperCase}"); 707 | shaderSource = new Regex("NUM_CHANNELS_ZERO").Replace(shaderSource, $"{channelsZero}"); 708 | shaderSource = new Regex("NUM_POSENC_SCALES").Replace(shaderSource, $"{posEncScales}"); 709 | shaderSource = new Regex("NUM_CHANNELS_ONE").Replace(shaderSource, $"{channelsOne}"); 710 | shaderSource = new Regex("NUM_CHANNELS_TWO").Replace(shaderSource, $"{channelsTwo}"); 711 | shaderSource = new Regex("NUM_CHANNELS_THREE").Replace(shaderSource, $"{channelsThree}"); 712 | shaderSource = new Regex("BIAS_LIST_ZERO").Replace(shaderSource, $"{biasListZero}"); 713 | shaderSource = new Regex("BIAS_LIST_ONE").Replace(shaderSource, $"{biasListOne}"); 714 | shaderSource = new Regex("BIAS_LIST_TWO").Replace(shaderSource, $"{biasListTwo}"); 715 | 716 | string shaderAssetPath = GetShaderAssetPath(); 717 | File.WriteAllText(shaderAssetPath, shaderSource); 718 | AssetDatabase.SaveAssets(); 719 | AssetDatabase.Refresh(); 720 | } 721 | 722 | private static void CreateMaterial(SceneParams sceneParams) { 723 | string materialAssetPath = GetMaterialAssetPath(); 724 | Material material; 725 | 726 | if (File.Exists(GetMaterialAssetPath())) { 727 | material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 728 | } else { 729 | string shaderAssetPath = GetShaderAssetPath(); 730 | Shader raymarchShader = AssetDatabase.LoadAssetAtPath(shaderAssetPath); 731 | material = new Material(raymarchShader); 732 | } 733 | 734 | material.SetInteger("displayMode", 0); 735 | material.SetInteger("ndc", sceneParams.Ndc ? 1 : 0); 736 | 737 | material.SetVector("minPosition", new Vector4( 738 | (float)sceneParams.MinX, 739 | (float)sceneParams.MinY, 740 | (float)sceneParams.MinZ 741 | ) 742 | ); 743 | material.SetVector("gridSize", new Vector4( 744 | sceneParams.GridWidth, 745 | sceneParams.GridHeight, 746 | sceneParams.GridDepth 747 | ) 748 | ); 749 | material.SetVector("atlasSize", new Vector4( 750 | sceneParams.AtlasWidth, 751 | sceneParams.AtlasHeight, 752 | sceneParams.AtlasDepth 753 | ) 754 | ); 755 | material.SetFloat("voxelSize", (float)sceneParams.VoxelSize); 756 | material.SetFloat("blockSize", (float)sceneParams.BlockSize); 757 | int maxStep = Mathf.CeilToInt(new Vector3(sceneParams.GridWidth, sceneParams.GridHeight, sceneParams.GridDepth).magnitude); 758 | material.SetInteger("maxStep", maxStep); 759 | 760 | // volume texture properties will be assigned when creating the 3D textures to avoid having to load them into memory here 761 | 762 | if (!File.Exists(GetMaterialAssetPath())) { 763 | AssetDatabase.CreateAsset(material, materialAssetPath); 764 | } 765 | AssetDatabase.SaveAssets(); 766 | AssetDatabase.Refresh(); 767 | } 768 | 769 | /// 770 | /// Assign volume textures again in case material got recreated. 771 | /// 772 | private static void VerifyMaterial(SceneParams sceneParams) { 773 | string materialAssetPath = GetMaterialAssetPath(); 774 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 775 | 776 | string rgbAssetPath = GetRGBTextureAssetPath(); 777 | string alphaAssetPath = GetAlphaTextureAssetPath(); 778 | string featureAssetPath = GetFeatureTextureAssetPath(); 779 | string atlasAssetPath = GetAtlasTextureAssetPath(); 780 | 781 | if (material.GetTexture("mapColor") == null) { 782 | Texture3D rgb = AssetDatabase.LoadAssetAtPath(rgbAssetPath); 783 | material.SetTexture("mapColor", rgb); 784 | } 785 | if (material.GetTexture("mapAlpha") == null) { 786 | Texture3D alpha = AssetDatabase.LoadAssetAtPath(alphaAssetPath); 787 | material.SetTexture("mapAlpha", alpha); 788 | } 789 | if (material.GetTexture("mapFeatures") == null) { 790 | Texture3D feature = AssetDatabase.LoadAssetAtPath(featureAssetPath); 791 | material.SetTexture("mapFeatures", feature); 792 | } 793 | if (material.GetTexture("mapIndex") == null) { 794 | Texture3D atlas = AssetDatabase.LoadAssetAtPath(atlasAssetPath); 795 | material.SetTexture("mapIndex", atlas); 796 | } 797 | } 798 | 799 | private static void CreateWeightTextures(SceneParams sceneParams) { 800 | Texture2D weightsTexZero = createFloatTextureFromData(sceneParams._0Weights); 801 | Texture2D weightsTexOne = createFloatTextureFromData(sceneParams._1Weights); 802 | Texture2D weightsTexTwo = createFloatTextureFromData(sceneParams._2Weights); 803 | AssetDatabase.CreateAsset(weightsTexZero, GetWeightsAssetPath(0)); 804 | AssetDatabase.CreateAsset(weightsTexOne, GetWeightsAssetPath(1)); 805 | AssetDatabase.CreateAsset(weightsTexTwo, GetWeightsAssetPath(2)); 806 | 807 | string materialAssetPath = GetMaterialAssetPath(); 808 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 809 | material.SetTexture("weightsZero", weightsTexZero); 810 | material.SetTexture("weightsOne", weightsTexOne); 811 | material.SetTexture("weightsTwo", weightsTexTwo); 812 | } 813 | 814 | /// 815 | /// Creates a float32 texture from an 2D array of weights. 816 | /// 817 | private static Texture2D createFloatTextureFromData(double[][] weights) { 818 | int width = weights.Length; 819 | int height = weights[0].Length; 820 | 821 | Texture2D texture = new Texture2D(width, height, TextureFormat.RFloat, mipChain: false, linear: true); 822 | texture.filterMode = FilterMode.Point; 823 | texture.wrapMode = TextureWrapMode.Clamp; 824 | NativeArray textureData = texture.GetRawTextureData(); 825 | FillTexture(textureData, weights); 826 | texture.Apply(updateMipmaps: false, makeNoLongerReadable: true); 827 | 828 | return texture; 829 | } 830 | 831 | private static void FillTexture(NativeArray textureData, double[][] data) { 832 | int width = data.Length; 833 | int height = data[0].Length; 834 | 835 | for (int co = 0; co < height; co++) { 836 | for (int ci = 0; ci < width; ci++) { 837 | int index = co * width + ci; 838 | double weight = data[ci][co]; 839 | textureData[index] = (float)weight; 840 | } 841 | } 842 | } 843 | 844 | private static StringBuilder toBiasList(double[] biases) { 845 | System.Globalization.CultureInfo culture = System.Globalization.CultureInfo.InvariantCulture; 846 | int width = biases.Length; 847 | StringBuilder biasList = new StringBuilder(width * 12); 848 | for (int i = 0; i < width; i++) { 849 | double bias = biases[i]; 850 | biasList.Append(bias.ToString("F7", culture)); 851 | if (i + 1 < width) { 852 | biasList.Append(", "); 853 | } 854 | } 855 | return biasList; 856 | } 857 | 858 | /// 859 | /// Creates a convenient prefab for the SNeRG. 860 | /// 861 | private static void CreatePrefab(SceneParams sceneParams) { 862 | GameObject prefabObject = GameObject.CreatePrimitive(PrimitiveType.Cube); 863 | prefabObject.name = _context.SceneName; 864 | MeshRenderer renderer = prefabObject.GetComponent(); 865 | string materialAssetPath = GetMaterialAssetPath(); 866 | Material material = AssetDatabase.LoadAssetAtPath(materialAssetPath); 867 | renderer.material = material; 868 | MeshFilter meshFilter = prefabObject.GetComponent(); 869 | meshFilter.sharedMesh = CreateMesh(meshFilter.sharedMesh, sceneParams); 870 | meshFilter.sharedMesh.name = $"{_context.SceneName}_volume_mesh"; 871 | AssetDatabase.CreateAsset(meshFilter.sharedMesh, GetMeshAssetPath()); 872 | prefabObject.AddComponent(); 873 | PrefabUtility.SaveAsPrefabAsset(prefabObject, GetPrefabAssetPath()); 874 | GameObject.DestroyImmediate(prefabObject); 875 | } 876 | 877 | /// 878 | /// Creates a copy of the default cube mesh 879 | /// and scales by volume extents to be able to raymarch in object space. 880 | /// 881 | /// 882 | /// 883 | /// 884 | private static Mesh CreateMesh(Mesh mesh, SceneParams sceneParams) { 885 | var vertices = mesh.vertices; 886 | for (int i = 0; i < vertices.Length; i++) { 887 | 888 | Vector3 v = vertices[i]; 889 | Vector3 size = new Vector3( 890 | Mathf.Abs((float)sceneParams.GridWidth), 891 | Mathf.Abs((float)sceneParams.GridHeight), 892 | Mathf.Abs((float)sceneParams.GridDepth)) 893 | * (float)sceneParams.VoxelSize; 894 | Vector3 minPos = new Vector3( 895 | (float)sceneParams.MinX, 896 | (float)sceneParams.MinY, 897 | (float)sceneParams.MinZ 898 | ); 899 | Vector3 offset = minPos + size / 2; 900 | v.Scale(size); 901 | v += offset; 902 | vertices[i] = v; 903 | } 904 | Mesh newMesh = new Mesh { 905 | vertices = vertices, 906 | triangles = mesh.triangles, 907 | uv = mesh.uv, 908 | normals = mesh.normals, 909 | colors = mesh.colors, 910 | tangents = mesh.tangents 911 | }; 912 | return newMesh; 913 | } 914 | } 915 | } -------------------------------------------------------------------------------- /Editor/SNeRGLoader.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 8878a305d5f21a743ae14219b389fd01 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /Editor/SNeRGScene.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | 3 | namespace SNeRG.Editor { 4 | 5 | public enum SNeRGScene { 6 | Custom = -1, 7 | Lego, 8 | Chair, 9 | Drums, 10 | Hotdog, 11 | Ship, 12 | Mic, 13 | Ficus, 14 | Materials, 15 | Spheres, 16 | VaseDeck, 17 | PineCone, 18 | ToyCar 19 | } 20 | 21 | public static class SNeRGSceneExtensions { 22 | public static string Name(this SNeRGScene scene) { 23 | return scene.ToString(); 24 | } 25 | 26 | public static string LowerCaseName(this SNeRGScene scene) { 27 | return scene.ToString().ToLower(); 28 | } 29 | 30 | public static SNeRGScene ToEnum(string value) { 31 | return (SNeRGScene)Enum.Parse(typeof(SNeRGScene), value, true); 32 | } 33 | 34 | public static bool IsSynthetic(this SNeRGScene scene) { 35 | switch (scene) { 36 | case SNeRGScene.Lego: 37 | case SNeRGScene.Chair: 38 | case SNeRGScene.Drums: 39 | case SNeRGScene.Hotdog: 40 | case SNeRGScene.Ship: 41 | case SNeRGScene.Mic: 42 | case SNeRGScene.Ficus: 43 | case SNeRGScene.Materials: 44 | return true; 45 | case SNeRGScene.Spheres: 46 | case SNeRGScene.VaseDeck: 47 | case SNeRGScene.PineCone: 48 | case SNeRGScene.ToyCar: 49 | return false; 50 | default: 51 | throw new InvalidOperationException(); 52 | } 53 | } 54 | } 55 | } -------------------------------------------------------------------------------- /Editor/SNeRGScene.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 91924917c19a3df4a83fa27b27f09380 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /Editor/SceneParams.cs: -------------------------------------------------------------------------------- 1 | using Newtonsoft.Json; 2 | 3 | namespace SNeRG.Editor { 4 | public partial class SceneParams { 5 | [JsonProperty("voxel_size")] 6 | public double VoxelSize { get; set; } 7 | 8 | [JsonProperty("block_size")] 9 | public int BlockSize { get; set; } 10 | 11 | [JsonProperty("grid_width")] 12 | public int GridWidth { get; set; } 13 | 14 | [JsonProperty("grid_height")] 15 | public int GridHeight { get; set; } 16 | 17 | [JsonProperty("grid_depth")] 18 | public int GridDepth { get; set; } 19 | 20 | [JsonProperty("atlas_width")] 21 | public int AtlasWidth { get; set; } 22 | 23 | [JsonProperty("atlas_height")] 24 | public int AtlasHeight { get; set; } 25 | 26 | [JsonProperty("atlas_depth")] 27 | public int AtlasDepth { get; set; } 28 | 29 | [JsonProperty("num_slices")] 30 | public int NumSlices { get; set; } 31 | 32 | [JsonProperty("min_x")] 33 | public double MinX { get; set; } 34 | 35 | [JsonProperty("min_y")] 36 | public double MinY { get; set; } 37 | 38 | [JsonProperty("min_z")] 39 | public double MinZ { get; set; } 40 | 41 | [JsonProperty("atlas_blocks_x")] 42 | public int AtlasBlocksX { get; set; } 43 | 44 | [JsonProperty("atlas_blocks_y")] 45 | public int AtlasBlocksY { get; set; } 46 | 47 | [JsonProperty("atlas_blocks_z")] 48 | public int AtlasBlocksZ { get; set; } 49 | 50 | [JsonProperty("worldspace_T_opengl")] 51 | public float[][] WorldspaceTOpengl { get; set; } 52 | 53 | [JsonProperty("ndc")] 54 | public bool Ndc { get; set; } 55 | 56 | [JsonProperty("0_weights")] 57 | public double[][] _0Weights { get; set; } 58 | 59 | [JsonProperty("1_weights")] 60 | public double[][] _1Weights { get; set; } 61 | 62 | [JsonProperty("2_weights")] 63 | public double[][] _2Weights { get; set; } 64 | 65 | [JsonProperty("0_bias")] 66 | public double[] _0Bias { get; set; } 67 | 68 | [JsonProperty("1_bias")] 69 | public double[] _1Bias { get; set; } 70 | 71 | [JsonProperty("2_bias")] 72 | public double[] _2Bias { get; set; } 73 | 74 | [JsonProperty("format")] 75 | public string Format { get; set; } 76 | } 77 | } -------------------------------------------------------------------------------- /Editor/SceneParams.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: faed614a21ec17842b601e8b33bea6c3 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /Editor/WebRequestAsyncUtility.cs: -------------------------------------------------------------------------------- 1 | using UnityEngine; 2 | using Newtonsoft.Json; 3 | using UnityEngine.Networking; 4 | using System; 5 | using System.Threading.Tasks; 6 | 7 | namespace SNeRG.Editor { 8 | 9 | public static class WebRequestAsyncUtility { 10 | 11 | public static class WebRequestAsync { 12 | 13 | /// 14 | /// Performs a WebRequest with the given URL. The response is expected to be a JSON serialized 15 | /// object of type and is automatically being deserialized and returned. 16 | /// 17 | /// the type that this web request returns. 18 | public static async Task SendWebRequestAsync(string url) { 19 | UnityWebRequest wr = UnityWebRequest.Get(url); 20 | if (wr == null) { 21 | return default; 22 | } 23 | 24 | try { 25 | var asyncOp = wr.SendWebRequest(); 26 | while (!asyncOp.webRequest.isDone) { 27 | await Task.Yield(); 28 | } 29 | 30 | switch (asyncOp.webRequest.result) { 31 | case UnityWebRequest.Result.InProgress: 32 | break; 33 | case UnityWebRequest.Result.Success: 34 | return JsonConvert.DeserializeObject(asyncOp.webRequest.downloadHandler.text); 35 | case UnityWebRequest.Result.ConnectionError: 36 | case UnityWebRequest.Result.ProtocolError: 37 | case UnityWebRequest.Result.DataProcessingError: 38 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}"); 39 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}"); 40 | break; 41 | default: 42 | break; 43 | } 44 | 45 | } catch (Exception e) { 46 | Debug.LogError(e); 47 | } finally { 48 | wr.Dispose(); 49 | } 50 | return default; 51 | } 52 | } 53 | 54 | /// 55 | /// Simple WebRequest without JSON parsing. Returns plain text. 56 | /// 57 | public class WebRequestSimpleAsync { 58 | 59 | public static async Task SendWebRequestAsync(string url) { 60 | UnityWebRequest wr = UnityWebRequest.Get(url); 61 | if (wr == null) { 62 | return default; 63 | } 64 | 65 | try { 66 | var asyncOp = wr.SendWebRequest(); 67 | while (!asyncOp.webRequest.isDone) { 68 | await Task.Yield(); 69 | } 70 | 71 | switch (asyncOp.webRequest.result) { 72 | case UnityWebRequest.Result.InProgress: 73 | break; 74 | case UnityWebRequest.Result.Success: 75 | return asyncOp.webRequest.downloadHandler.text; 76 | case UnityWebRequest.Result.ConnectionError: 77 | case UnityWebRequest.Result.ProtocolError: 78 | case UnityWebRequest.Result.DataProcessingError: 79 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}"); 80 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}"); 81 | break; 82 | default: 83 | break; 84 | } 85 | 86 | } catch (Exception e) { 87 | Debug.LogError(e); 88 | } finally { 89 | wr.Dispose(); 90 | } 91 | return default; 92 | } 93 | } 94 | 95 | /// 96 | /// WebRequest for binary data. 97 | /// 98 | public class WebRequestBinaryAsync { 99 | 100 | public static async Task SendWebRequestAsync(string url) { 101 | UnityWebRequest wr = UnityWebRequest.Get(url); 102 | if (wr == null) { 103 | return default; 104 | } 105 | 106 | try { 107 | var asyncOp = wr.SendWebRequest(); 108 | while (!asyncOp.webRequest.isDone) { 109 | await Task.Yield(); 110 | } 111 | 112 | switch (asyncOp.webRequest.result) { 113 | case UnityWebRequest.Result.InProgress: 114 | break; 115 | case UnityWebRequest.Result.Success: 116 | return asyncOp.webRequest.downloadHandler.data; 117 | case UnityWebRequest.Result.ConnectionError: 118 | case UnityWebRequest.Result.ProtocolError: 119 | case UnityWebRequest.Result.DataProcessingError: 120 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}"); 121 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}"); 122 | break; 123 | default: 124 | break; 125 | } 126 | 127 | } catch (Exception e) { 128 | Debug.LogError(e); 129 | } finally { 130 | wr.Dispose(); 131 | } 132 | return default; 133 | } 134 | } 135 | 136 | /// 137 | /// WebRequest for texture data. 138 | /// 139 | public class WebRequestTextureAsync { 140 | 141 | public static async Task SendWebRequestAsync(string url) { 142 | UnityWebRequest wr = UnityWebRequestTexture.GetTexture(url); 143 | if (wr == null) { 144 | return default; 145 | } 146 | 147 | try { 148 | var asyncOp = wr.SendWebRequest(); 149 | while (!asyncOp.webRequest.isDone) { 150 | await Task.Yield(); 151 | } 152 | 153 | switch (asyncOp.webRequest.result) { 154 | case UnityWebRequest.Result.InProgress: 155 | break; 156 | case UnityWebRequest.Result.Success: 157 | return DownloadHandlerTexture.GetContent(asyncOp.webRequest); 158 | case UnityWebRequest.Result.ConnectionError: 159 | case UnityWebRequest.Result.ProtocolError: 160 | case UnityWebRequest.Result.DataProcessingError: 161 | Debug.LogError($"{asyncOp.webRequest.result}: {asyncOp.webRequest.error}\nURL: {asyncOp.webRequest.url}"); 162 | Debug.LogError($"{asyncOp.webRequest.downloadHandler.text}"); 163 | break; 164 | default: 165 | break; 166 | } 167 | 168 | } catch (Exception e) { 169 | Debug.LogError(e); 170 | } finally { 171 | wr.Dispose(); 172 | } 173 | return default; 174 | } 175 | } 176 | } 177 | } -------------------------------------------------------------------------------- /Editor/WebRequestAsyncUtility.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 730f94c3031e12b4793062e6319c01ac 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /License.txt: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /License.txt.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 793c8006049ee10459442c8078265df3 3 | TextScriptImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | -------------------------------------------------------------------------------- /Runtime.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 32a81c4270a18314bb8906203e15cc6c 3 | folderAsset: yes 4 | DefaultImporter: 5 | externalObjects: {} 6 | userData: 7 | assetBundleName: 8 | assetBundleVariant: 9 | -------------------------------------------------------------------------------- /Runtime/Doji.SNeRG.asmdef: -------------------------------------------------------------------------------- 1 | { 2 | "name": "Doji.SNeRG" 3 | } 4 | -------------------------------------------------------------------------------- /Runtime/Doji.SNeRG.asmdef.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: f2ef7ba7c4c2fc44680f667d61ba9966 3 | AssemblyDefinitionImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | -------------------------------------------------------------------------------- /Runtime/EnableDepthTexture.cs: -------------------------------------------------------------------------------- 1 | using UnityEngine; 2 | 3 | namespace SNeRG { 4 | 5 | /// 6 | /// Enables on the main camera. 7 | /// ExecuteInEditMode is needed, so that this setting also gets 8 | /// applied on the scene camera when not in Play Mode. 9 | /// 10 | [ExecuteInEditMode] 11 | public class EnableDepthTexture : MonoBehaviour { 12 | 13 | private void OnEnable() { 14 | Camera camera = Camera.main; 15 | 16 | if (camera == null) { 17 | Debug.LogError("Could not find main camera." + 18 | " For proper depth composition between SNeRGs and other opaque objects in the scene, " + 19 | "ensure that DeptextureMode.Depth is active on the camera used for rendering.", 20 | camera.gameObject); 21 | return; 22 | } 23 | 24 | camera.depthTextureMode = DepthTextureMode.Depth; 25 | } 26 | } 27 | } -------------------------------------------------------------------------------- /Runtime/EnableDepthTexture.cs.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: 54254b92b9c44824c80d2696944c6671 3 | MonoImporter: 4 | externalObjects: {} 5 | serializedVersion: 2 6 | defaultReferences: [] 7 | executionOrder: 0 8 | icon: {instanceID: 0} 9 | userData: 10 | assetBundleName: 11 | assetBundleVariant: 12 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "com.doji.snerg", 3 | "version": "1.0.2", 4 | "displayName": "SNeRG", 5 | "description": "An unofficial Unity port of the SNeRG viewer.", 6 | "dependencies": 7 | { 8 | "com.unity.nuget.newtonsoft-json": "3.0.2" 9 | }, 10 | "author": 11 | { 12 | "name": "Doji Technologies", 13 | "email": "support@doji-tech.com", 14 | "url": "https://www.doji-tech.com/" 15 | } 16 | } -------------------------------------------------------------------------------- /package.json.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: c94a1344b832e8d4f9bee4c16c004653 3 | PackageManifestImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # SNeRG Unity Viewer 2 | 3 | https://user-images.githubusercontent.com/26555424/224686209-113c4a0f-860e-4b8c-9158-430b4ba2d2a9.mp4 4 | 5 | This repository contains the source code for a Unity port of the web viewer for [Baking Neural Radiance Fields for Real-Time View Synthesis](https://phog.github.io/snerg/)[^1] 6 | 7 | *Please note, that this is an unofficial port. I am not affiliated with the original authors or their institution.* 8 | 9 | ## Requirements 10 | 11 | I recommend 32GB of RAM and a graphics card with 4GB of VRAM. 12 | 13 | ## Usage 14 | 15 | ### Installation 16 | 17 | Go to the [releases section](https://github.com/julienkay/SNeRG-Unity-Viewer/releases/latest), download the Unity Package, and import it into any Unity project. This is a 'Hybrid Package' that will install into your project as a local package. 18 | 19 | ##### Alternatives 20 | 21 |
22 | UPM Package via OpenUPM 23 | 24 | In `Edit -> Project Settings -> Package Manager`, add a new scoped registry: 25 | 26 | Name: Doji 27 | URL: https://package.openupm.com 28 | Scope(s): com.doji 29 | 30 | In the Package Manager install 'com.doji.snerg' either by name or via `Package Manager -> My Registries` 31 |
32 | 33 |
34 | UPM Package via Git URL 35 | 36 | In `Package Manager -> Add package from git URL...` paste `https://github.com/julienkay/SNeRG-Unity-Viewer.git` [as described here](https://docs.unity3d.com/Manual/upm-ui-giturl) 37 |
38 | 39 | ### Importing sample scenes 40 | 41 | After succesful installation, you can use the menu *SNeRG -> Asset Downloads* to download any of the sample scenes available. 42 | In each scene folder there will be a convenient prefab, that you can then drag into the scene and you're good to go. 43 | 44 | ### Updating 45 | 46 | Since the initial release a number of issues have been fixed in the asset generation code. 47 | That means, that if you have already imported some scenes before, you'll have to delete these source files and regenerate them by going to *SNeRG -> Asset Downloads* again. 48 | 49 | ### Importing self-trained scenes 50 | 51 | If you have successfully trained your own SNeRG scenes using the [official code release](https://github.com/google-research/google-research/tree/master/snerg) and want to render them in Unity, you can use the menu *SNeRG -> Import from disk*. 52 | This lets you choose a folder that must contain the output files of your training process. 53 | 54 | More specifically, the following assets are required: 55 | - a file named 'scene_params.json' 56 | - a PNG file for the 3D atlas texture: atlas_indices.png 57 | - several PNG files for the 3D feature texture: feature_XXX.png - feature_YYY.png 58 | - several PNG files for the 3D RGB and alpha texture: rgba_XXX.png - rgba_YYY.png 59 | 60 | ## Details 61 | 62 | This project was created with Unity 2021.3 LTS using the Built-in Render Pipeline. 63 | 64 | Deviations from the official viewer: 65 | - Rather than doing the raymarching by drawing a full-screen quad, I opted to use a frontface-culled cube in world space instead. From a rendering perspective, there should not be too much of a difference, but it has some workflow advantages in the Unity Editor I think. It is a bit more straightforward to render the volume in Unity's scene view. It's also easier to integrate into existing projects because it allows other components to work directly with that GameObject's transform hierarchy (e.g. Physics, scripts from various XR Interaction SDKs, ...). 66 | - I added manual depth testing, so the volume correctly interacts with other (opaque) mesh-based objects in the scene. This usually requires a [depth pre-pass](https://docs.unity3d.com/Manual/SL-CameraDepthTexture.html) though. 67 | 68 | ## Known Issues 69 | 70 | - Memory requirements for working with 3D textures in Unity are quite high. The way the import process works right now is that Texture3D assets are created in-Editor. The memory that Unity uses when serializing these as assets on disk is a bit too high for my liking. Granted, the size of these scenes can go up to Gigabytes. However, I've found that 16GB of RAM was often not enough to handle the import/creation of 2GB of uncompressed 3D textures due to inefficiencies. I wasn’t even able to import some of the larger scenes until upgrading to 32GB RAM. 71 | 72 | - Unity straight-up errors out when trying to create 3D textures with sizes of 2GB or larger. Some of the feature volumes like the ficus scene (2048x2048x128) hit that limit and can not be imported. 73 | 74 | ### Possible Workarounds 75 | 76 | - One possible solution to handling the data in-Editor might be to defer the 3D texture creation and import the volumes into Unity as individual 2D slices, (similar to how the official web viewer distributes the data) and only assemble the 3D textures at runtime. For development, it was quite useful to preview the volume without having to enter Play Mode though. But some scenes are currently just very difficult to handle, even when your GPU could fit the data into VRAM just fine. 77 | 78 | - I should have probably looked more into texture compression options for 3D textures 79 | 80 | - I've set the Asset Serialization Mode to 'Force Binary' with some success of reducing memory consumption during import. Unfortunately, the Asset Serialization Mode is a project-wide setting, which is a bit inconvenient, because in real-world projects text serialization is more commonly used I suppose. 81 | 82 | For future improvements, [MERF](https://merf42.github.io/) looks promising in terms of reducing memory usage. 83 | 84 | ## References 85 | 86 | Other projects exploring NeRFs and related techniques in Unity: 87 | - [MobileNeRF (Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures)](https://github.com/julienkay/MobileNeRF-Unity-Viewer) 88 | - [MERF (Memory Efficient Radiance Fields)](https://github.com/julienkay/MERF-Unity-Viewer) 89 | 90 | [^1]: [Peter Hedman and Pratul P. Srinivasan and Ben Mildenhall and Jonathan T. Barron and Paul Debevec. Baking Neural Radiance Fields for Real-Time View Synthesis. ICCV, 2021](https://phog.github.io/snerg/) 91 | -------------------------------------------------------------------------------- /readme.md.meta: -------------------------------------------------------------------------------- 1 | fileFormatVersion: 2 2 | guid: d30e1e83d9e306e4ab72d3168baf29d0 3 | TextScriptImporter: 4 | externalObjects: {} 5 | userData: 6 | assetBundleName: 7 | assetBundleVariant: 8 | --------------------------------------------------------------------------------