├── LICENSE ├── README.md └── images ├── UE_mesh_draw_calls.jpg └── test.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Tom 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Notes on VR Performance 2 | Performance is crucial for developing high-quality VR content, as it helps prevent user discomfort, motion sickness, and enhances realism. Ensuring smooth operation and prioritizing optimal frame rates are essential aspects of VR development that are often overlooked. Inadequate frame rates or inconsistent frame times can result in subpar user experiences and contribute to motion sickness.
3 | 4 | Crafting high-performance content is no easy task, PCVR (powerful gaming PCs) and standalone VR (modest mobile headsets) demand different approaches. These notes focus on standalone optimisation, specifically for the Oculus Quest ecosystem. Note, different headsets carry unique requirements. Performance hinges on numerous factors: the need for high resolutions, double frames, power constraints, limited processing, rendering power, and the weight of thermal throttling.
5 | 6 | Extended build times can hinder productivity. Despite its distinct challenges, VR development benefits from numerous performance-enhancing solutions that are commonly employed in the flat game world.
7 | 8 | ## When to Optimise? 9 | * Optimise from the beginning! 10 | * Be performance conscious during development - think about scene complexity, do I need to add this in? is it worth the performance hit? 11 | * A good time to optimise is when you are not hitting the desired frame rate. 12 | * Some features look amazing, but are they worth it? (each frame has a budget) 13 | * At the time of writing standalone VR is very very performance limited. 14 | ## Frame Rate Vs Frame Time 15 | * **Frame rate** is the number of frames displayed per second (FPS). It is calculated by dividing the total number of frames displayed by the total amount of time taken to display them.
16 | * **Frame time** is calculated by dividing the total time it takes to render a frame by the number of frames rendered. It is typically measured in milliseconds (ms).
17 | * Let's say a game runs at 60 frames per second (fps), taking 16.67 milliseconds to render each frame. The frame time is 16.67 milliseconds per frame. This is also known as the frame budget. Here we have a frame budget of 16.67 ms for both the CPU and GPU. It's super easy to spend your frame budget on a standalone headset (start thinking of other variables like thermal throttling).
18 |
19 | | Frame Rate (FPS) | Frame Time (ms) | 20 | |------------------|-----------------| 21 | |30|33.3| 22 | |60|16.67| 23 | |90|8.3| 24 | * Frame time is a better measure of performance than frame rate because it takes into account the consistency of the frame rate. A high frame rate does not necessarily mean that the game will be smooth, if the frame time is inconsistent. For example, a game may have a frame rate of 60fps, but if the frame time varies from 16ms to 33ms, the game will not be smooth.
25 | * Frame time is useful for diagnosing performance issues. If you are experiencing stuttering or other performance problems, you can use frame time to identify the source of the problem.
26 | * Headsets have different required frame rates. When the frame rates drop, it's likely to cause disorientation and nausea for the user. Maintaining a minimum frame rate (reducing frame times) can be particularly challenging on a standalone headset.
27 | * Most PCVR runs at 90fps. Standalone is usually 72fps (but can be as high as 120fps).
28 |
29 | | Headset | Frame Rate | 30 | |---------|--------------------------------| 31 | |HTC Vive|90| 32 | |Quest 1 |72| 33 | |Quest 2 |upto 120| 34 | |Quest Pro|90| 35 | |Valve Index|upto 144| 36 | ## Profiling 37 | * Profiling tools can help identify where a game or application is experiencing slow performance by highlighting the frame times that are taking longer to render. There are a variety of profiling solutions available to choose from. Personally, as someone who develops using Unreal Engine (UE) with a Quest headset, I prefer to use RenderDoc and UE's built-in performance suite, Unreal Insights.
38 | * Profiling performance reveals many reasons for increased frame times, which can be classified into two categories: CPU-bound or GPU-bound.
39 | ### Profiling Tools (lots of options - but can be dependent on headset and graphic chipset) 40 | [🆓⚒️ Oculus Debug Tool](https://developer.oculus.com/documentation/native/pc/dg-debug-tool/?locale=en_GB)
41 | [🆓⚒️ RenderDoc](https://renderdoc.org/)
42 | [🆓⚒️ OVR Metrics Tools](https://developer.oculus.com/downloads/package/ovr-metrics-tool/)
43 | [🆓⚒️ Android GPU Inspector (AGI) - for Android based HMDs](https://developer.android.com/agi)
44 | [🆓⚒️ ARM: Streamline Performance Analyzer](https://developer.arm.com/Tools%20and%20Software/Streamline%20Performance%20Analyzer)
45 | [🆓⚒️ SteamVR Frame Timing](https://developer.valvesoftware.com/wiki/SteamVR/Frame_Timing)
46 | [🆓⚒️ NVIDIA Visual Profiler](https://developer.nvidia.com/nvidia-visual-profiler)
47 | [🆓⚒️ NVIDIA Nsight Graphics](https://developer.nvidia.com/nsight-graphics)
48 | [🆓⚒️ NVIDIA VR Capture & Replay - Early Access](https://developer.nvidia.com/vcr-early-access)
49 | [🆓⚒️ NVIDIA Performance Analysis Tools](https://developer.nvidia.com/performance-analysis-tools)
50 | [🆓⚒️ AMD Radeon GPU Profiler](https://gpuopen.com/rgp/)
51 | [🆓⚒️ Intel® Graphics Performance Analyzers:](https://www.intel.com/content/www/us/en/developer/tools/graphics-performance-analyzers/overview.html)
52 | [🆓⚒️ Intel® VTune™ Profiler](https://www.intel.com/content/www/us/en/developer/tools/oneapi/vtune-profiler.html#gs.qs0mup)
53 | [🆓⚒️ PVRTune](https://developer.imaginationtech.com/pvrtune/)
54 | [⚒️ Superluminal](https://superluminal.eu/)
55 | 56 | ⭐ [See my resourses on performance](https://github.com/authorTom/ultimate-XR-dev-guide/blob/main/README.md#performance-and-benchmarking) 57 |
58 | ### CPU or GPU Bound? 59 | The first questions to ask yourself; Whats causing the problem? is it CPU or GPU bound?
60 | 61 | **CPU Bound**
62 | Performance primarily limited by the CPU's processing power can be caused by:
63 | * AI/physics calculations 64 | * Animation/character controller logic 65 | * Collision detection 66 | * Level/asset streaming 67 | 68 | A straightforward method to assess this is by temporarily disabling the rendering process, which eliminates the cost of the render pipeline. Monitor the application's framerate and the time taken to complete each frame in milliseconds. If performance remains unchanged, it is likely that the issue is CPU-bound. However, if performance is significantly impacted, the GPU is likely the cause.
69 |
70 | [Getting started with Unreal Insights (UE5)](https://docs.unrealengine.com/5.0/en-US/trace-in-unreal-engine-5/) 71 | 72 | **GPU Bound**
73 | Performance limited by the GPU's rendering capabilities can be caused by:
74 | * Scene complexity (vertex) 75 | * Shader complexity (fragment) 76 | * Texture quality/resolution 77 | * Particle effects 78 | 79 | To determine whether the performance issue stems from a vertex or fragment-related problem, attempt rendering fewer pixels by setting the render scale to 0.01. This approach will reduce the number of fragments rendered while maintaining the complexity.
80 | ``` 81 | vr.PixelDensity 0.01 82 | ``` 83 | Common causes of vertex bound performance
84 | * Culling objects are taking too long 85 | * Too many draw calls are being issued 86 | * Too many vertices are being rendered 87 | 88 | Possible solutions
89 | * Simplify complex geometry 90 | * Reduce draw calls (batching draw calls) 91 | * Implement a Level of Detail (LOD) system 92 | 93 | Use a profiler to isolate problems.
94 | Change one thing at a time and re-profile. Test, test and test some more. Making small iterations is a good strategy for identifying problems.
95 | If fragment bound, optimise shaders to address pixel complexity.
96 | 97 | ## Performance on Standalone VR 98 | When developing for standalone VR get a feel for the numbers and look at headset recommendations and limitations. Recommended draw calls for PCVR is around 1000.
99 | 100 | | Platform | Draw Calls | Description | 101 | |-----|-----|-----| 102 | | Quest 1 | 50-150 |Busy Simulation| 103 | | Quest 1 | 150-250 |Medium Simulation| 104 | | Quest 1 | 200-400 |Light Simulation| 105 | | Quest 2 | 80-200 |Busy Simulation| 106 | | Quest 2 | 200-300 |Medium Simulation| 107 | | Quest 2 | 400-600 | Light Simulation| 108 | 109 | Both Quest 1 & 2 have a recommended minimum of 72FPS.
110 | 111 | | Platform | Triangle Count | 112 | |-----|-----| 113 | | Quest 1 | 350k-500k | 114 | | Quest 2 | 750k-1M | 115 | 116 | Data taken from [Meta](https://developer.oculus.com/documentation/unreal/unreal-debug-android/) 117 |
118 | ## Useful profiling commands (Unreal Engine) 119 | | Command | Function | 120 | |-----|-----| 121 | | Stat Unit | Displays performance information for the project's frame, game, draw, GPU, RHIT, and DynRes threads | 122 | | Stat FPS | Displays frames per second and average frame time | 123 | | Stat RHI | Displays RHI memory and performance statistics) 124 | | Stat SceneRendering | Displays general rendering statistics. This is a good starting point to find general areas of slow performance in the rendering process | 125 | | Stat GPU | Displays GPU statistics for the frame | 126 |
127 | 128 | ## Scene and VR Project Settings Within Unreal Engine 129 | ``` 130 | Project Settings > Engine > Rendering > VR 131 | 132 | Dynamic Fixed Foveation (enable) 133 | Instanced stereo (enable) 134 | Mobile Multiview (enable) 135 | Round Robin Occlusion Queries (enable) 136 | 137 | Project Settings > Engine > Rendering 138 | 139 | Forward shading (enable) 140 | Vertex fogging for opaque (enable) 141 | Anti-aliasing = MSAA 142 | MSAA sample count = 4x 143 | 144 | Project Settings > Project > Target Hardware 145 | 146 | Optimise project settings for 'mobile' and 'scalable' 147 | 148 | *These settings are optimal for mobile/standalone 149 | ``` 150 |
151 | Mobile HDR allows for post processing in a standalone headset but can be very expensive and not all headsets support this. Round Robin Occlusion (RRO) improves VR performance by alternating rendering occlusion between one eye each frame instead of both, at the cost of a single frame of latency. To optimise stereo rendering on the CPU, use instanced stereo for desktop and mobile multiview for mobile. Forward rendering has improved performance over deferred rendering if most lighting is baked. MSAA anti-aliasing produces less blur that TAA.
152 | 153 | ## Tip 1 - Reduce Draw Calls 154 | * A draw call is a command to the GPU, instructing it to render a specific set of graphical elements, such as textures or polygons. 155 | * GPU handles most graphics computations, but CPU controls it. 156 | * Too many draw calls can bottleneck performance. 157 | * Batching combines calls for identical meshes and parameters. 158 | * Static batching is effective for unchanging, static scene parts. 159 | * Unity engine uses static batching; can affect culling negatively if overused. 160 | * Unreal Engine sorts objects by mesh and shader parameters for more flexibility. 161 | * Effectiveness depends on unique meshes and materials in scene. 162 | * Having modular environment art reduces draw calls. 163 | * Less is better. Thresholds are dependent on headset specifications. 164 | * For Quest 2 some recommend targeting draw calls <100 per frame. 165 | 166 | ``` 167 | Quick Tip: Unreal Engine - to check draw call counts in the editor run the "stat SceneRendering" command. 168 | ``` 169 | ![An image of Unreal Engine 5 UI displaying the draw call count](images/UE_mesh_draw_calls.jpg) 170 | #### Solution - Batching draw calls 171 | * Static batching 172 | * Dynamic batching 173 | #### Solution - GPU instancing 174 | * GPU instancing is an extreme form of batching. 175 | * Single mesh loaded, multiple copies stamped. 176 | * Minimal CPU overhead. 177 | * Limitations: 178 | * Position, rotation, scale can change. 179 | * Same material/properties for all instances. 180 | * Specifics can depend on the engine. 181 | 182 |
183 | 184 | ## Tip 2 - Textures & Materials 185 | ## Tip 3 - Lighting 186 | ## Tip 4 - Transparency & Blending 187 | ## Tip 5 - Shaders 188 | ## Tip 6 - Antialiasing 189 | ## Tip 7 - Occlusion culling 190 | ## Tip 8 - Fixed Foveated Rendering 191 | ## Resources 192 | [➡️ My list of XR performance resources](https://github.com/authorTom/ultimate-XR-dev-guide#performance-and-benchmarking) 193 | ## Issues 194 | This project is actively maintained, please raise an issue if you feel the need to.
195 |
196 | **Like what has been done? please give the repository a star ⭐**
197 | ## My Other Projects 198 | [➡️ The Ultimate XR Developers Resource Guide](https://github.com/authorTom/ultimate-XR-dev-guide) 199 | -------------------------------------------------------------------------------- /images/UE_mesh_draw_calls.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/authorTom/notes-on-VR-performance/12025c4198099413f36a90fef9a0a528e0535b6b/images/UE_mesh_draw_calls.jpg -------------------------------------------------------------------------------- /images/test.md: -------------------------------------------------------------------------------- 1 | 2 | --------------------------------------------------------------------------------