├── .gitattributes ├── CODE_OF_CONDUCT.md ├── Docs └── Images │ ├── Banner.png │ └── InvalidVersion.png ├── LICENSE ├── MsftOpenXRGame ├── .clang-format ├── .gitignore ├── Build │ └── HoloLens │ │ ├── Resources │ │ ├── Logo.png │ │ ├── SmallLogo.png │ │ ├── SplashScreen.png │ │ ├── StoreLogo.png │ │ └── WideLogo.png │ │ └── SigningCertificate.pfx ├── Config │ ├── DefaultEditor.ini │ ├── DefaultEngine.ini │ ├── DefaultGame.ini │ ├── DefaultInput.ini │ └── HoloLens │ │ └── HoloLensEngine.ini ├── Content │ ├── AzureObjectAnchorActor.uasset │ ├── AzureObjectAnchorQueryModifiers.uasset │ ├── AzureObjectAnchors │ │ └── cube_a03ddc87-441f-4d1f-b1c8-d915245d827c.ou │ ├── Blueprints │ │ └── Pivot.uasset │ ├── CamCapture.uasset │ ├── EyeTrackingComponent.uasset │ ├── HandTrackingComponent.uasset │ ├── MRGameMode.uasset │ ├── MRPlayerPawn.uasset │ ├── Maps │ │ ├── DefaultMap.umap │ │ └── DefaultMap_BuiltData.uasset │ ├── Materials │ │ ├── AzureObjectAnchorMaterial.uasset │ │ ├── CamTextureMaterial.uasset │ │ ├── DefaultTextMaterialOpaque.uasset │ │ ├── Floor.uasset │ │ ├── InteractionSphere.uasset │ │ ├── MI_GlowPlastic.uasset │ │ ├── M_BasicUnlit.uasset │ │ ├── M_GlowPlastic.uasset │ │ ├── M_Plastic.uasset │ │ ├── MaterialLayers │ │ │ ├── T_ML_Aluminum01.uasset │ │ │ └── T_ML_Aluminum01_N.uasset │ │ ├── SceneUnderstanding │ │ │ ├── CeilingMaterial.uasset │ │ │ ├── FloorMaterial.uasset │ │ │ ├── SpatialMappingMaterial.uasset │ │ │ ├── TableMaterial.uasset │ │ │ └── WallMaterial.uasset │ │ └── SpeechMaterials │ │ │ ├── Blue.uasset │ │ │ ├── ColorEnum.uasset │ │ │ ├── ColorMat.uasset │ │ │ ├── Green.uasset │ │ │ ├── Orange.uasset │ │ │ ├── Pink.uasset │ │ │ ├── Purple.uasset │ │ │ ├── Red.uasset │ │ │ ├── White.uasset │ │ │ └── Yellow.uasset │ ├── Meshes │ │ ├── SpawnArrow_Arrow.uasset │ │ ├── SpawnArrow_Base.uasset │ │ ├── SpawnArrow_GreenCircle.uasset │ │ └── SpawnArrow_RedEx.uasset │ ├── QRTracker.uasset │ ├── SpatialMappingActor.uasset │ ├── SpeechActor.uasset │ ├── TextActor.uasset │ ├── TextWidget.uasset │ └── WMRConfig.uasset ├── MsftOpenXRGame.png ├── MsftOpenXRGame.uproject ├── Plugins │ └── MicrosoftOpenXR │ │ ├── Config │ │ └── FilterPlugin.ini │ │ ├── MicrosoftOpenXR.uplugin │ │ ├── Resources │ │ └── Icon128.png │ │ └── Source │ │ ├── AzureSpatialAnchorsForOpenXR │ │ ├── AzureSpatialAnchorsForOpenXR.Build.cs │ │ ├── Private │ │ │ └── AzureSpatialAnchorsForOpenXR.cpp │ │ └── Public │ │ │ └── AzureSpatialAnchorsForOpenXR.h │ │ ├── MicrosoftOpenXR │ │ ├── MicrosoftOpenXR.Build.cs │ │ ├── Private │ │ │ ├── AzureObjectAnchorsPlugin.cpp │ │ │ ├── AzureObjectAnchorsPlugin.h │ │ │ ├── External │ │ │ │ └── openxr │ │ │ │ │ ├── openxr.h │ │ │ │ │ ├── openxr_platform.h │ │ │ │ │ ├── openxr_platform_defines.h │ │ │ │ │ └── openxr_reflection.h │ │ │ ├── HandMeshPlugin.cpp │ │ │ ├── HandMeshPlugin.h │ │ │ ├── HolographicRemotingPlugin.cpp │ │ │ ├── HolographicRemotingPlugin.h │ │ │ ├── HolographicWindowAttachmentPlugin.cpp │ │ │ ├── HolographicWindowAttachmentPlugin.h │ │ │ ├── LocatableCamPlugin.cpp │ │ │ ├── LocatableCamPlugin.h │ │ │ ├── MicrosoftOpenXR.cpp │ │ │ ├── OpenXRCameraImageTexture.cpp │ │ │ ├── OpenXRCameraImageTexture.h │ │ │ ├── OpenXRCameraImageTexture_UE426.cpp │ │ │ ├── OpenXRCommon.cpp │ │ │ ├── OpenXRCommon.h │ │ │ ├── QRTrackingPlugin.cpp │ │ │ ├── QRTrackingPlugin.h │ │ │ ├── SceneUnderstandingBase.cpp │ │ │ ├── SceneUnderstandingBase.h │ │ │ ├── SceneUnderstandingPlugin.cpp │ │ │ ├── SceneUnderstandingPlugin.h │ │ │ ├── SceneUnderstandingUtility.h │ │ │ ├── SecondaryViewConfiguration.cpp │ │ │ ├── SecondaryViewConfiguration.h │ │ │ ├── SpatialAnchorPlugin.cpp │ │ │ ├── SpatialAnchorPlugin.h │ │ │ ├── SpatialMappingPlugin.cpp │ │ │ ├── SpatialMappingPlugin.h │ │ │ ├── SpeechPlugin.cpp │ │ │ ├── SpeechPlugin.h │ │ │ ├── TrackedGeometryCollision.cpp │ │ │ ├── TrackedGeometryCollision.h │ │ │ ├── UniqueHandle.h │ │ │ └── WindowsMixedRealityInteropUtility.h │ │ └── Public │ │ │ ├── AzureObjectAnchorTypes.h │ │ │ └── MicrosoftOpenXR.h │ │ ├── MicrosoftOpenXREditor │ │ ├── MicrosoftOpenXREditor.Build.cs │ │ └── Private │ │ │ ├── MicrosoftOpenXRDetails.cpp │ │ │ ├── MicrosoftOpenXRDetails.h │ │ │ └── MicrosoftOpenXREditorModule.cpp │ │ ├── MicrosoftOpenXRRuntimeSettings │ │ ├── MicrosoftOpenXRRuntimeSettings.Build.cs │ │ ├── Private │ │ │ ├── MicrosoftOpenXRRuntimeSettings.cpp │ │ │ └── MicrosoftOpenXRRuntimeSettingsModule.cpp │ │ └── Public │ │ │ └── MicrosoftOpenXRRuntimeSettings.h │ │ └── NuGetModule │ │ ├── NuGetModule.Build.cs │ │ ├── Private │ │ ├── NuGetModule.cpp │ │ └── NuGetModule.h │ │ └── packages.config └── Source │ ├── MsftOpenXRGame.Target.cs │ ├── MsftOpenXRGame │ ├── MsftOpenXRGame.Build.cs │ ├── MsftOpenXRGame.cpp │ ├── MsftOpenXRGame.h │ ├── MsftOpenXRGameGameModeBase.cpp │ └── MsftOpenXRGameGameModeBase.h │ └── MsftOpenXRGameEditor.Target.cs ├── README.md └── SECURITY.md /.gitattributes: -------------------------------------------------------------------------------- 1 | # Batch scripts need CR/LF line endings 2 | *.bat eol=crlf 3 | 4 | # Shell scripts need unix line endings 5 | *.sh text eol=lf 6 | 7 | # Binary files 8 | *.png binary 9 | *.jpg binary 10 | *.umap binary 11 | *.uasset binary 12 | *.dll binary -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Microsoft Open Source Code of Conduct 2 | 3 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 4 | 5 | Resources: 6 | 7 | - [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) 8 | - [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) 9 | - Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns 10 | -------------------------------------------------------------------------------- /Docs/Images/Banner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/Docs/Images/Banner.png -------------------------------------------------------------------------------- /Docs/Images/InvalidVersion.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/Docs/Images/InvalidVersion.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Microsoft Corporation. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE 22 | -------------------------------------------------------------------------------- /MsftOpenXRGame/.clang-format: -------------------------------------------------------------------------------- 1 | --- 2 | # File originally from https://gist.github.com/intinig/9bba3a3faee80250b781bf916a4ab8b7 3 | Language: Cpp 4 | AccessModifierOffset: -4 5 | AlignAfterOpenBracket: DontAlign 6 | AlignConsecutiveAssignments: false 7 | AlignConsecutiveDeclarations: false 8 | AlignEscapedNewlines: Left 9 | AlignOperands: true 10 | AlignTrailingComments: true 11 | AllowAllParametersOfDeclarationOnNextLine: true 12 | AllowShortBlocksOnASingleLine: false 13 | AllowShortCaseLabelsOnASingleLine: false 14 | AllowShortFunctionsOnASingleLine: None 15 | AllowShortIfStatementsOnASingleLine: false 16 | AllowShortLoopsOnASingleLine: false 17 | AlwaysBreakAfterReturnType: None 18 | AlwaysBreakBeforeMultilineStrings: true 19 | AlwaysBreakTemplateDeclarations: Yes 20 | BinPackArguments: true 21 | BinPackParameters: true 22 | BreakBeforeBinaryOperators: None 23 | BreakBeforeBraces: Allman 24 | BreakBeforeInheritanceComma: false 25 | BreakInheritanceList: BeforeColon 26 | BreakBeforeTernaryOperators: true 27 | BreakConstructorInitializersBeforeComma: false 28 | BreakConstructorInitializers: BeforeComma 29 | BreakAfterJavaFieldAnnotations: false 30 | BreakStringLiterals: true 31 | ColumnLimit: 132 32 | CommentPragmas: '^ IWYU pragma:' 33 | CompactNamespaces: true 34 | ConstructorInitializerAllOnOneLineOrOnePerLine: true 35 | ConstructorInitializerIndentWidth: 4 36 | ContinuationIndentWidth: 4 37 | Cpp11BracedListStyle: true 38 | DerivePointerAlignment: false 39 | DisableFormat: false 40 | FixNamespaceComments: true 41 | ForEachMacros: 42 | - for 43 | IncludeBlocks: Regroup 44 | IncludeCategories: 45 | - Regex: '.*\.generated\.h' 46 | Priority: 100 47 | - Regex: '.*(PCH).*' 48 | Priority: -1 49 | - Regex: '".*"' 50 | Priority: 1 51 | - Regex: '^<.*\.(h)>' 52 | Priority: 3 53 | - Regex: '^<.*>' 54 | Priority: 4 55 | IncludeIsMainRegex: '([-_](test|unittest))?$' 56 | IndentCaseLabels: true 57 | IndentPPDirectives: None 58 | IndentWidth: 4 59 | IndentWrappedFunctionNames: false 60 | JavaScriptQuotes: Leave 61 | JavaScriptWrapImports: true 62 | KeepEmptyLinesAtTheStartOfBlocks: false 63 | MacroBlockBegin: '' 64 | MacroBlockEnd: '' 65 | MaxEmptyLinesToKeep: 1 66 | NamespaceIndentation: All 67 | ObjCBinPackProtocolList: Never 68 | ObjCBlockIndentWidth: 2 69 | ObjCSpaceAfterProperty: false 70 | ObjCSpaceBeforeProtocolList: true 71 | PenaltyBreakAssignment: 2 72 | PenaltyBreakBeforeFirstCallParameter: 1 73 | PenaltyBreakComment: 300 74 | PenaltyBreakFirstLessLess: 120 75 | PenaltyBreakString: 1000 76 | PenaltyBreakTemplateDeclaration: 10 77 | PenaltyExcessCharacter: 1000000 78 | PenaltyReturnTypeOnItsOwnLine: 200 79 | PointerAlignment: Left 80 | RawStringFormats: 81 | - Language: Cpp 82 | Delimiters: 83 | - cc 84 | - CC 85 | - cpp 86 | - Cpp 87 | - CPP 88 | - 'c++' 89 | - 'C++' 90 | CanonicalDelimiter: '' 91 | BasedOnStyle: google 92 | - Language: TextProto 93 | Delimiters: 94 | - pb 95 | - PB 96 | - proto 97 | - PROTO 98 | EnclosingFunctions: 99 | - EqualsProto 100 | - EquivToProto 101 | - PARSE_PARTIAL_TEXT_PROTO 102 | - PARSE_TEST_PROTO 103 | - PARSE_TEXT_PROTO 104 | - ParseTextOrDie 105 | - ParseTextProtoOrDie 106 | CanonicalDelimiter: '' 107 | ReflowComments: true 108 | SortIncludes: true 109 | SortUsingDeclarations: true 110 | SpaceAfterCStyleCast: true 111 | SpaceAfterTemplateKeyword: true 112 | SpaceBeforeAssignmentOperators: true 113 | SpaceBeforeCpp11BracedList: false 114 | SpaceBeforeCtorInitializerColon: true 115 | SpaceBeforeInheritanceColon: true 116 | SpaceBeforeParens: ControlStatements 117 | SpaceBeforeRangeBasedForLoopColon: true 118 | SpaceInEmptyParentheses: false 119 | SpacesBeforeTrailingComments: 4 120 | SpacesInAngles: false 121 | SpacesInContainerLiterals: true 122 | SpacesInCStyleCastParentheses: false 123 | SpacesInParentheses: false 124 | SpacesInSquareBrackets: false 125 | Standard: Auto 126 | TabWidth: 4 127 | UseTab: Always 128 | ... 129 | -------------------------------------------------------------------------------- /MsftOpenXRGame/.gitignore: -------------------------------------------------------------------------------- 1 | # Visual Studio 2015 user specific files 2 | .vs/ 3 | 4 | # These project files can be generated by the engine 5 | *.sln 6 | 7 | # Precompiled Assets 8 | SourceArt/**/*.png 9 | SourceArt/**/*.tga 10 | 11 | # Binary Files 12 | Binaries/* 13 | Plugins/*/Binaries/* 14 | 15 | # Builds 16 | Build/* 17 | 18 | # Whitelist PakBlacklist-.txt files 19 | !Build/*/ 20 | Build/*/** 21 | !Build/*/PakBlacklist*.txt 22 | 23 | # Don't ignore icon files in Build 24 | !Build/**/*.ico 25 | 26 | # Built data for maps 27 | # Not ignoring this beacuse it forces each person to regenerate it locally (by building lighting) and that leads to the correspoding .umap file being modified. 28 | # *_BuiltData.uasset 29 | 30 | # Configuration files generated by the Editor 31 | Saved/* 32 | 33 | # Compiled source files for the engine to use 34 | Intermediate/* 35 | Plugins/*/Intermediate/* 36 | 37 | # Cache files for the editor to use 38 | DerivedDataCache/* 39 | 40 | Hololens/* 41 | WindowsNoEditor/* 42 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/Resources/Logo.png: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:63e70c898d49938c6920670806bc47263b652aec48b1728e511de12ddc0ca6a4 3 | size 35735 4 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/Resources/SmallLogo.png: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:e007f415760022ae05201c8d72cf3eadca634c872fd1820e30e504b67324992e 3 | size 4345 4 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/Resources/SplashScreen.png: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:6215495eccf992ddd5d27ad27a742379189a9e14712bdfe565a2812bbb2eda0c 3 | size 240279 4 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/Resources/StoreLogo.png: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:284f073d7edd69de5ab4410bdb101967bd658c3cc321ddf449ec22f0c7ebd874 3 | size 5351 4 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/Resources/WideLogo.png: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:fc5550f7e77df19eb040f7727313f896167965e635b6b8af6a3c1426a4badd06 3 | size 68457 4 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Build/HoloLens/SigningCertificate.pfx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Build/HoloLens/SigningCertificate.pfx -------------------------------------------------------------------------------- /MsftOpenXRGame/Config/DefaultEditor.ini: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Config/DefaultEditor.ini -------------------------------------------------------------------------------- /MsftOpenXRGame/Config/DefaultEngine.ini: -------------------------------------------------------------------------------- 1 | 2 | 3 | [/Script/HardwareTargeting.HardwareTargetingSettings] 4 | TargetedHardwareClass=Mobile 5 | AppliedTargetedHardwareClass=Mobile 6 | DefaultGraphicsPerformance=Scalable 7 | AppliedDefaultGraphicsPerformance=Scalable 8 | 9 | [/Script/Engine.Engine] 10 | 11 | [/Script/Engine.RendererSettings] 12 | r.Mobile.DisableVertexFog=True 13 | r.Shadow.CSM.MaxMobileCascades=1 14 | r.MobileMSAA=1 15 | r.Mobile.UseLegacyShadingModel=False 16 | r.Mobile.AllowDitheredLODTransition=False 17 | r.Mobile.AllowSoftwareOcclusion=False 18 | r.Mobile.VirtualTextures=False 19 | r.DiscardUnusedQuality=False 20 | r.AllowOcclusionQueries=False 21 | r.MinScreenRadiusForLights=0.030000 22 | r.MinScreenRadiusForDepthPrepass=0.030000 23 | r.MinScreenRadiusForCSMDepth=0.010000 24 | r.PrecomputedVisibilityWarning=False 25 | r.TextureStreaming=True 26 | Compat.UseDXT5NormalMaps=False 27 | r.VirtualTextures=False 28 | r.VirtualTexturedLightmaps=False 29 | r.VT.TileSize=128 30 | r.VT.TileBorderSize=4 31 | r.vt.FeedbackFactor=16 32 | r.VT.EnableCompressZlib=True 33 | r.VT.EnableCompressCrunch=False 34 | r.ClearCoatNormal=False 35 | r.AnisotropicBRDF=False 36 | r.ReflectionCaptureResolution=128 37 | r.ReflectionEnvironmentLightmapMixBasedOnRoughness=True 38 | r.ForwardShading=True 39 | r.VertexFoggingForOpaque=False 40 | r.AllowStaticLighting=True 41 | r.NormalMapsForStaticLighting=False 42 | r.GenerateMeshDistanceFields=False 43 | r.DistanceFieldBuild.EightBit=False 44 | r.GenerateLandscapeGIData=False 45 | r.DistanceFieldBuild.Compress=False 46 | r.TessellationAdaptivePixelsPerTriangle=48.000000 47 | r.SeparateTranslucency=False 48 | r.TranslucentSortPolicy=0 49 | TranslucentSortAxis=(X=0.000000,Y=-1.000000,Z=0.000000) 50 | r.CustomDepth=0 51 | r.CustomDepthTemporalAAJitter=True 52 | r.PostProcessing.PropagateAlpha=0 53 | r.DefaultFeature.Bloom=False 54 | r.DefaultFeature.AmbientOcclusion=False 55 | r.DefaultFeature.AmbientOcclusionStaticFraction=True 56 | r.DefaultFeature.AutoExposure=False 57 | r.DefaultFeature.AutoExposure.Method=0 58 | r.DefaultFeature.AutoExposure.Bias=1.000000 59 | r.DefaultFeature.AutoExposure.ExtendDefaultLuminanceRange=False 60 | r.UsePreExposure=True 61 | r.EyeAdaptation.EditorOnly=False 62 | r.DefaultFeature.MotionBlur=False 63 | r.DefaultFeature.LensFlare=False 64 | r.TemporalAA.Upsampling=False 65 | r.SSGI.Enable=False 66 | r.DefaultFeature.AntiAliasing=0 67 | r.DefaultFeature.LightUnits=1 68 | r.DefaultBackBufferPixelFormat=4 69 | r.Shadow.UnbuiltPreviewInGame=True 70 | r.StencilForLODDither=False 71 | r.EarlyZPass=3 72 | r.EarlyZPassOnlyMaterialMasking=False 73 | r.DBuffer=True 74 | r.ClearSceneMethod=1 75 | r.BasePassOutputsVelocity=False 76 | r.VertexDeformationOutputsVelocity=False 77 | r.SelectiveBasePassOutputs=False 78 | bDefaultParticleCutouts=False 79 | fx.GPUSimulationTextureSizeX=1024 80 | fx.GPUSimulationTextureSizeY=1024 81 | r.AllowGlobalClipPlane=False 82 | r.GBufferFormat=1 83 | r.MorphTarget.Mode=True 84 | r.GPUCrashDebugging=False 85 | vr.InstancedStereo=True 86 | r.MobileHDR=False 87 | vr.MobileMultiView=True 88 | r.Mobile.UseHWsRGBEncoding=False 89 | vr.RoundRobinOcclusion=False 90 | vr.ODSCapture=False 91 | r.MeshStreaming=False 92 | r.WireframeCullThreshold=5.000000 93 | r.RayTracing=False 94 | r.RayTracing.UseTextureLod=False 95 | r.SupportStationarySkylight=True 96 | r.SupportLowQualityLightmaps=True 97 | r.SupportPointLightWholeSceneShadows=True 98 | r.SupportAtmosphericFog=True 99 | r.SupportSkyAtmosphere=True 100 | r.SupportSkyAtmosphereAffectsHeightFog=False 101 | r.SkinCache.CompileShaders=False 102 | r.SkinCache.DefaultBehavior=1 103 | r.SkinCache.SceneMemoryLimitInMB=128.000000 104 | r.Mobile.EnableStaticAndCSMShadowReceivers=True 105 | r.Mobile.EnableMovableLightCSMShaderCulling=True 106 | r.Mobile.AllowDistanceFieldShadows=True 107 | r.Mobile.AllowMovableDirectionalLights=True 108 | r.MobileNumDynamicPointLights=0 109 | r.MobileDynamicPointLightsUseStaticBranch=True 110 | r.Mobile.EnableMovableSpotlights=False 111 | r.GPUSkin.Support16BitBoneIndex=False 112 | r.GPUSkin.Limit2BoneInfluences=False 113 | r.SupportDepthOnlyIndexBuffers=True 114 | r.SupportReversedIndexBuffers=True 115 | r.SupportMaterialLayers=False 116 | r.LightPropagationVolume=False 117 | bStreamSkeletalMeshLODs=(Default=False,PerPlatform=()) 118 | bDiscardSkeletalMeshOptionalLODs=(Default=False,PerPlatform=()) 119 | VisualizeCalibrationColorMaterialPath=None 120 | VisualizeCalibrationCustomMaterialPath=None 121 | VisualizeCalibrationGrayscaleMaterialPath=None 122 | 123 | [/Script/EngineSettings.GameMapsSettings] 124 | EditorStartupMap=/Game/Maps/DefaultMap.DefaultMap 125 | LocalMapOptions= 126 | TransitionMap=None 127 | bUseSplitscreen=False 128 | TwoPlayerSplitscreenLayout=Horizontal 129 | ThreePlayerSplitscreenLayout=FavorTop 130 | FourPlayerSplitscreenLayout=Grid 131 | bOffsetPlayerGamepadIds=False 132 | GameInstanceClass=/Script/Engine.GameInstance 133 | GameDefaultMap=/Game/Maps/DefaultMap.DefaultMap 134 | ServerDefaultMap=/Engine/Maps/Entry.Entry 135 | GlobalDefaultGameMode=/Script/Engine.GameModeBase 136 | GlobalDefaultServerGameMode=None 137 | 138 | [/Script/Slate.SlateSettings] 139 | bExplicitCanvasChildZOrder=True 140 | 141 | [/Game/MRPlayerPawn.MRPlayerPawn_C] 142 | ASA_AccountKey="" 143 | ASA_AccountID="" 144 | ASA_AccountDomain="" 145 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Config/DefaultGame.ini: -------------------------------------------------------------------------------- 1 | 2 | 3 | [/Script/EngineSettings.GeneralProjectSettings] 4 | ProjectID=99F0EDDB4B092C7CA6713EAA6EFCB918 5 | CompanyName=Microsoft 6 | CompanyDistinguishedName=CN=Microsoft 7 | bStartInVR=True 8 | ProjectName=MsftOpenXRGame 9 | ProjectVersion=1.0.0.0 10 | 11 | [/Script/EngineSettings.XRVisualizationSettings] 12 | HandMeshMaterial=Material'/Game/Materials/M_BasicUnlit.M_BasicUnlit' 13 | 14 | [/Script/HoloLensSettings.SceneUnderstanding] 15 | ShouldDoSceneUnderstandingMeshDetection=true 16 | SceneUnderstandingVolumeHeight=4 17 | 18 | [/Script/UnrealEd.ProjectPackagingSettings] 19 | +DirectoriesToAlwaysStageAsNonUFS=(Path="AzureObjectAnchors") 20 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Config/HoloLens/HoloLensEngine.ini: -------------------------------------------------------------------------------- 1 | 2 | 3 | [/Script/HoloLensPlatformEditor.HoloLensTargetSettings] 4 | bBuildForEmulation=False 5 | bBuildForDevice=True 6 | bUseNameForLogo=True 7 | bBuildForRetailWindowsStore=False 8 | bAutoIncrementVersion=False 9 | bShouldCreateAppInstaller=False 10 | AppInstallerInstallationURL= 11 | HoursBetweenUpdateChecks=0 12 | bEnablePIXProfiling=False 13 | TileBackgroundColor=(B=64,G=0,R=0,A=255) 14 | SplashScreenBackgroundColor=(B=64,G=0,R=0,A=255) 15 | +PerCultureResources=(CultureId="",Strings=(PackageDisplayName="Unreal OpenXR Game",PublisherDisplayName="Microsoft Corporation",PackageDescription="Demonstrates the Microsoft OpenXR plugin for Unreal",ApplicationDisplayName="Unreal OpenXR Game",ApplicationDescription=""),Images=()) 16 | TargetDeviceFamily=Windows.Holographic 17 | MinimumPlatformVersion= 18 | MaximumPlatformVersionTested=10.0.18362.0 19 | MaxTrianglesPerCubicMeter=500.000000 20 | SpatialMeshingVolumeSize=20.000000 21 | CompilerVersion=Default 22 | Windows10SDKVersion=10.0.18362.0 23 | +CapabilityList=internetClientServer 24 | +CapabilityList=privateNetworkClientServer 25 | +DeviceCapabilityList=gazeInput 26 | +DeviceCapabilityList=webcam 27 | +DeviceCapabilityList=microphone 28 | +DeviceCapabilityList=wiFiControl 29 | +DeviceCapabilityList=proximity 30 | +DeviceCapabilityList=location 31 | +DeviceCapabilityList=bluetooth 32 | +Uap2CapabilityList=spatialPerception 33 | bSetDefaultCapabilities=False 34 | SpatializationPlugin= 35 | ReverbPlugin= 36 | OcclusionPlugin= 37 | SoundCueCookQualityIndex=-1 38 | 39 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/AzureObjectAnchorActor.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/AzureObjectAnchorActor.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/AzureObjectAnchorQueryModifiers.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/AzureObjectAnchorQueryModifiers.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/AzureObjectAnchors/cube_a03ddc87-441f-4d1f-b1c8-d915245d827c.ou: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/AzureObjectAnchors/cube_a03ddc87-441f-4d1f-b1c8-d915245d827c.ou -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Blueprints/Pivot.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Blueprints/Pivot.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/CamCapture.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/CamCapture.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/EyeTrackingComponent.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/EyeTrackingComponent.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/HandTrackingComponent.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/HandTrackingComponent.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/MRGameMode.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/MRGameMode.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/MRPlayerPawn.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/MRPlayerPawn.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Maps/DefaultMap.umap: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Maps/DefaultMap.umap -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Maps/DefaultMap_BuiltData.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Maps/DefaultMap_BuiltData.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/AzureObjectAnchorMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/AzureObjectAnchorMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/CamTextureMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/CamTextureMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/DefaultTextMaterialOpaque.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/DefaultTextMaterialOpaque.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/Floor.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/Floor.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/InteractionSphere.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/InteractionSphere.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/MI_GlowPlastic.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/MI_GlowPlastic.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/M_BasicUnlit.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/M_BasicUnlit.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/M_GlowPlastic.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/M_GlowPlastic.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/M_Plastic.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/M_Plastic.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/MaterialLayers/T_ML_Aluminum01.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/MaterialLayers/T_ML_Aluminum01.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/MaterialLayers/T_ML_Aluminum01_N.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/MaterialLayers/T_ML_Aluminum01_N.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SceneUnderstanding/CeilingMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SceneUnderstanding/CeilingMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SceneUnderstanding/FloorMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SceneUnderstanding/FloorMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SceneUnderstanding/SpatialMappingMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SceneUnderstanding/SpatialMappingMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SceneUnderstanding/TableMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SceneUnderstanding/TableMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SceneUnderstanding/WallMaterial.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SceneUnderstanding/WallMaterial.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Blue.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Blue.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/ColorEnum.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/ColorEnum.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/ColorMat.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/ColorMat.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Green.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Green.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Orange.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Orange.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Pink.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Pink.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Purple.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Purple.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Red.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Red.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/White.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/White.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Materials/SpeechMaterials/Yellow.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Materials/SpeechMaterials/Yellow.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Meshes/SpawnArrow_Arrow.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Meshes/SpawnArrow_Arrow.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Meshes/SpawnArrow_Base.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Meshes/SpawnArrow_Base.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Meshes/SpawnArrow_GreenCircle.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Meshes/SpawnArrow_GreenCircle.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/Meshes/SpawnArrow_RedEx.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/Meshes/SpawnArrow_RedEx.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/QRTracker.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/QRTracker.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/SpatialMappingActor.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/SpatialMappingActor.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/SpeechActor.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/SpeechActor.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/TextActor.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/TextActor.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/TextWidget.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/TextWidget.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/Content/WMRConfig.uasset: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Content/WMRConfig.uasset -------------------------------------------------------------------------------- /MsftOpenXRGame/MsftOpenXRGame.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/MsftOpenXRGame.png -------------------------------------------------------------------------------- /MsftOpenXRGame/MsftOpenXRGame.uproject: -------------------------------------------------------------------------------- 1 | { 2 | "FileVersion": 3, 3 | "EngineAssociation": "4.27", 4 | "Category": "", 5 | "Description": "", 6 | "Modules": [ 7 | { 8 | "Name": "MsftOpenXRGame", 9 | "Type": "Runtime", 10 | "LoadingPhase": "Default" 11 | } 12 | ], 13 | "Plugins": [ 14 | { 15 | "Name": "SteamVR", 16 | "Enabled": false, 17 | "SupportedTargetPlatforms": [ 18 | "Win32", 19 | "Win64", 20 | "Linux", 21 | "Mac", 22 | "Android" 23 | ] 24 | }, 25 | { 26 | "Name": "MagicLeap", 27 | "Enabled": false, 28 | "SupportedTargetPlatforms": [ 29 | "Lumin", 30 | "Mac", 31 | "Win64" 32 | ] 33 | }, 34 | { 35 | "Name": "MagicLeapMedia", 36 | "Enabled": false, 37 | "SupportedTargetPlatforms": [ 38 | "Lumin" 39 | ] 40 | }, 41 | { 42 | "Name": "LuminPlatformFeatures", 43 | "Enabled": false, 44 | "SupportedTargetPlatforms": [ 45 | "Lumin" 46 | ] 47 | }, 48 | { 49 | "Name": "MLSDK", 50 | "Enabled": false 51 | }, 52 | { 53 | "Name": "MagicLeapPassableWorld", 54 | "Enabled": false, 55 | "SupportedTargetPlatforms": [ 56 | "Lumin", 57 | "Mac", 58 | "Win64" 59 | ] 60 | }, 61 | { 62 | "Name": "XRVisualization", 63 | "Enabled": true 64 | } 65 | ] 66 | } -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Config/FilterPlugin.ini: -------------------------------------------------------------------------------- 1 | [FilterPlugin] 2 | ; This section lists additional files which will be packaged along with your plugin. Paths should be listed relative to the root plugin directory, and 3 | ; may include "...", "*", and "?" wildcards to match directories, files, and individual characters respectively. 4 | ; 5 | ; Examples: 6 | ; /README.txt 7 | ; /Extras/... 8 | ; /Binaries/ThirdParty/*.dll 9 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/MicrosoftOpenXR.uplugin: -------------------------------------------------------------------------------- 1 | { 2 | "FileVersion": 3, 3 | "Version": 1, 4 | "VersionName": "1.1.15", 5 | "FriendlyName": "Microsoft OpenXR", 6 | "Description": "The Microsoft OpenXR plugin is a game plugin which provides additional features available on Microsoft's Mixed Reality devices like the HoloLens 2 when using OpenXR.", 7 | "Category": "Mixed Reality", 8 | "CreatedBy": "Microsoft", 9 | "CreatedByURL": "", 10 | "DocsURL": "", 11 | "MarketplaceURL": "com.epicgames.launcher://ue/marketplace/product/ef8930ca860148c498b46887da196239", 12 | "SupportURL": "https://github.com/microsoft/Microsoft-OpenXR-Unreal", 13 | "EngineVersion": "4.27.0", 14 | "CanContainContent": true, 15 | "IsBetaVersion": false, 16 | "IsExperimentalVersion": false, 17 | "Installed": false, 18 | "SupportedTargetPlatforms": [ 19 | "Win64", 20 | "HoloLens" 21 | ], 22 | "Modules": [ 23 | { 24 | "Name": "NuGetModule", 25 | "Type": "Runtime", 26 | "LoadingPhase": "PostConfigInit", 27 | "WhitelistPlatforms": [ 28 | "Win64", 29 | "HoloLens" 30 | ] 31 | }, 32 | { 33 | "Name": "MicrosoftOpenXR", 34 | "Type": "Runtime", 35 | "LoadingPhase": "PostConfigInit" 36 | }, 37 | { 38 | "Name": "MicrosoftOpenXRRuntimeSettings", 39 | "Type": "Runtime", 40 | "LoadingPhase": "PostConfigInit", 41 | "WhitelistPlatforms": [ 42 | "Win64", 43 | "HoloLens" 44 | ] 45 | }, 46 | { 47 | "Name": "MicrosoftOpenXREditor", 48 | "Type": "Editor", 49 | "LoadingPhase": "PostEngineInit", 50 | "WhitelistPlatforms": [ 51 | "Win64" 52 | ] 53 | }, 54 | { 55 | "Name": "AzureSpatialAnchorsForOpenXR", 56 | "Type": "Runtime", 57 | "WhitelistPlatforms": [ 58 | "Win64", 59 | "HoloLens" 60 | ] 61 | } 62 | ], 63 | "Plugins": [ 64 | { 65 | "Name": "OpenXR", 66 | "Enabled": true 67 | }, 68 | { 69 | "Name": "OpenXREyeTracker", 70 | "Enabled": true 71 | }, 72 | { 73 | "Name": "OpenXRHandTracking", 74 | "Enabled": true 75 | }, 76 | { 77 | "Name": "OpenXRMsftHandInteraction", 78 | "Enabled": true 79 | }, 80 | { 81 | "Name": "XRVisualization", 82 | "Enabled": true 83 | }, 84 | { 85 | "Name": "HoloLensAR", 86 | "Enabled": true, 87 | "WhitelistPlatforms": [ 88 | "Win64" 89 | ] 90 | }, 91 | { 92 | "Name": "AzureSpatialAnchors", 93 | "Enabled": true, 94 | "WhitelistPlatforms": [ 95 | "Win64", 96 | "HoloLens" 97 | ] 98 | } 99 | ] 100 | } 101 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Resources/Icon128.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/Microsoft-OpenXR-Unreal/5c2d81bf8c71fbe4d2d1b71c91f7c8526a8408a5/MsftOpenXRGame/Plugins/MicrosoftOpenXR/Resources/Icon128.png -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/AzureSpatialAnchorsForOpenXR/AzureSpatialAnchorsForOpenXR.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using System; 5 | using System.IO; 6 | using System.Linq; 7 | using UnrealBuildTool; 8 | using Tools.DotNETCommon; 9 | using System.Collections.Generic; 10 | 11 | namespace UnrealBuildTool.Rules 12 | { 13 | public class AzureSpatialAnchorsForOpenXR : ModuleRules 14 | { 15 | public AzureSpatialAnchorsForOpenXR(ReadOnlyTargetRules Target) : base(Target) 16 | { 17 | if (Target.Platform == UnrealTargetPlatform.Win64 || Target.Platform == UnrealTargetPlatform.HoloLens) 18 | { 19 | // these parameters mandatory for winrt support 20 | bEnableExceptions = true; 21 | bUseUnity = false; 22 | CppStandard = CppStandardVersion.Cpp17; 23 | PublicSystemLibraries.AddRange(new string[] { "shlwapi.lib", "runtimeobject.lib" }); 24 | } 25 | 26 | PrivateDependencyModuleNames.AddRange( 27 | new string[] 28 | { 29 | "Core", 30 | "CoreUObject", 31 | "Engine", 32 | "AugmentedReality", 33 | "HeadMountedDisplay", 34 | "OpenXRHMD", 35 | "NuGetModule", 36 | "Projects", 37 | "MicrosoftOpenXR" 38 | } 39 | ); 40 | 41 | PublicDependencyModuleNames.AddRange( 42 | new string[] 43 | { 44 | "AzureSpatialAnchors", 45 | "MicrosoftOpenXR" 46 | } 47 | ); 48 | 49 | PrivateIncludePathModuleNames.AddRange( 50 | new string[] 51 | { 52 | "HeadMountedDisplay", 53 | "NuGetModule" 54 | } 55 | ); 56 | 57 | PublicIncludePathModuleNames.AddRange( 58 | new string[] 59 | { 60 | "HeadMountedDisplay", 61 | "NuGetModule" 62 | } 63 | ); 64 | 65 | string OpenXRPluginIncludePath = Path.Combine(PluginDirectory, "Source", "MicrosoftOpenXR", "Private"); 66 | string OpenXRPrivateIncludePath = Path.Combine(PluginDirectory, "Source", "MicrosoftOpenXR", "Private", "External"); 67 | PrivateIncludePaths.Add(OpenXRPluginIncludePath); 68 | PrivateIncludePaths.Add(OpenXRPrivateIncludePath); 69 | } 70 | } 71 | } 72 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/AzureSpatialAnchorsForOpenXR/Public/AzureSpatialAnchorsForOpenXR.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 7 | #include "Interfaces/IPluginManager.h" 8 | 9 | #include "OpenXRCommon.h" 10 | 11 | #include "IAzureSpatialAnchors.h" 12 | #include "AzureSpatialAnchorsBase.h" 13 | #include "AzureCloudSpatialAnchor.h" 14 | #include "UObject/GCObject.h" 15 | 16 | #include "HeadMountedDisplayTypes.h" 17 | #include "ARBlueprintLibrary.h" 18 | 19 | #include "Windows/AllowWindowsPlatformTypes.h" 20 | #include "Windows/AllowWindowsPlatformAtomics.h" 21 | #include "Windows/PreWindowsApi.h" 22 | 23 | #include 24 | #include 25 | #include 26 | #include 27 | 28 | #include "Windows/PostWindowsApi.h" 29 | #include "Windows/HideWindowsPlatformAtomics.h" 30 | #include "Windows/HideWindowsPlatformTypes.h" 31 | 32 | class FAzureSpatialAnchorsForOpenXR : public FAzureSpatialAnchorsBase, public IModuleInterface 33 | { 34 | public: 35 | void StartupModule() override; 36 | 37 | public: 38 | void ShutdownModule() override; 39 | 40 | // IAzureSpatialAnchors Implementation 41 | 42 | bool CreateSession() override; 43 | void DestroySession() override; 44 | 45 | void GetAccessTokenWithAccountKeyAsync(const FString& AccountKey, Callback_Result_String Callback) override; 46 | void GetAccessTokenWithAuthenticationTokenAsync(const FString& AuthenticationToken, Callback_Result_String Callback) override; 47 | EAzureSpatialAnchorsResult StartSession() override; 48 | void StopSession() override; 49 | EAzureSpatialAnchorsResult ResetSession() override; 50 | void DisposeSession() override; 51 | void GetSessionStatusAsync(Callback_Result_SessionStatus Callback) override; 52 | EAzureSpatialAnchorsResult ConstructAnchor(UARPin* InARPin, CloudAnchorID& OutCloudAnchorID) override; 53 | void CreateAnchorAsync(CloudAnchorID InCloudAnchorID, Callback_Result Callback) override; // note this 'creates' the anchor in the azure cloud, aka saves it to the cloud. 54 | void DeleteAnchorAsync(CloudAnchorID InCloudAnchorID, Callback_Result Callback) override; 55 | EAzureSpatialAnchorsResult CreateWatcher(const FAzureSpatialAnchorsLocateCriteria& InLocateCriteria, float InWorldToMetersScale, WatcherID& OutWatcherID, FString& OutErrorString) override; 56 | EAzureSpatialAnchorsResult GetActiveWatchers(TArray& OutWatcherIDs) override; 57 | void GetAnchorPropertiesAsync(const FString& InCloudAnchorIdentifier, Callback_Result_CloudAnchorID Callback) override; 58 | void RefreshAnchorPropertiesAsync(CloudAnchorID InCloudAnchorID, Callback_Result Callback) override; 59 | void UpdateAnchorPropertiesAsync(CloudAnchorID InCloudAnchorID, Callback_Result Callback) override; 60 | EAzureSpatialAnchorsResult GetConfiguration(FAzureSpatialAnchorsSessionConfiguration& OutConfig) override; 61 | EAzureSpatialAnchorsResult SetConfiguration(const FAzureSpatialAnchorsSessionConfiguration& InConfig) override; 62 | EAzureSpatialAnchorsResult SetLocationProvider(const FCoarseLocalizationSettings& InConfig) override; 63 | EAzureSpatialAnchorsResult GetLogLevel(EAzureSpatialAnchorsLogVerbosity& OutLogVerbosity) override; 64 | EAzureSpatialAnchorsResult SetLogLevel(EAzureSpatialAnchorsLogVerbosity InLogVerbosity) override; 65 | EAzureSpatialAnchorsResult GetSessionId(FString& OutSessionID) override; 66 | 67 | EAzureSpatialAnchorsResult StopWatcher(WatcherID WatcherID) override; 68 | 69 | EAzureSpatialAnchorsResult GetCloudSpatialAnchorIdentifier(CloudAnchorID InCloudAnchorID, FString& OutCloudAnchorIdentifier) override; 70 | EAzureSpatialAnchorsResult SetCloudAnchorExpiration(CloudAnchorID InCloudAnchorID, float InLifetimeInSeconds) override; 71 | EAzureSpatialAnchorsResult GetCloudAnchorExpiration(CloudAnchorID InCloudAnchorID, float& OutLifetimeInSeconds) override; 72 | EAzureSpatialAnchorsResult SetCloudAnchorAppProperties(CloudAnchorID InCloudAnchorID, const TMap& InAppProperties) override; 73 | EAzureSpatialAnchorsResult GetCloudAnchorAppProperties(CloudAnchorID InCloudAnchorID, TMap& OutAppProperties) override; 74 | 75 | EAzureSpatialAnchorsResult SetDiagnosticsConfig(FAzureSpatialAnchorsDiagnosticsConfig& InConfig) override; 76 | void CreateDiagnosticsManifestAsync(const FString& Description, Callback_Result_String Callback) override; 77 | void SubmitDiagnosticsManifestAsync(const FString& ManifestPath, Callback_Result Callback) override; 78 | 79 | void CreateNamedARPinAroundAnchor(const FString& InLocalAnchorId, UARPin*& OutARPin) override; 80 | bool CreateARPinAroundAzureCloudSpatialAnchor(const FString& InPinId, UAzureCloudSpatialAnchor* InAzureCloudSpatialAnchor, UARPin*& OutARPin) override; 81 | 82 | private: 83 | bool m_sessionStarted{ false }; 84 | 85 | winrt::Microsoft::Azure::SpatialAnchors::CloudSpatialAnchorSession m_cloudSession { nullptr }; 86 | 87 | winrt::event_revoker m_anchorLocatedToken; 88 | winrt::event_revoker m_locateAnchorsCompletedToken; 89 | winrt::event_revoker m_sessionUpdatedToken; 90 | winrt::event_revoker m_errorToken; 91 | winrt::event_revoker m_onLogDebugToken; 92 | 93 | // map of cloud anchors ids to cloud anchors 94 | std::map m_cloudAnchors; 95 | mutable std::mutex m_cloudAnchorsMutex; 96 | 97 | std::map m_watcherMap; 98 | mutable std::mutex m_watcherMapMutex; 99 | 100 | winrt::Microsoft::Azure::SpatialAnchors::CloudSpatialAnchor* GetNativeCloudAnchor(CloudAnchorID cloudAnchorID); 101 | IAzureSpatialAnchors::CloudAnchorID CloudAnchorIdentifierToID(const winrt::hstring& CloudAnchorIdentifier) const; 102 | IAzureSpatialAnchors::CloudAnchorID GetNextCloudAnchorID(); 103 | 104 | void AddEventListeners(); 105 | void RemoveEventListeners(); 106 | 107 | private: 108 | bool CheckForSession(const wchar_t* context) const; 109 | 110 | winrt::Windows::Perception::Spatial::SpatialPerceptionAccessStatus spatialPerceptionAccessStatus; 111 | }; 112 | #endif 113 | 114 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/MicrosoftOpenXR.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using System.IO; 5 | using System.Linq; 6 | using System.Runtime.Remoting.Messaging; 7 | using UnrealBuildTool; 8 | using Tools.DotNETCommon; 9 | using System; 10 | using System.Collections.Generic; 11 | 12 | public class MicrosoftOpenXR : ModuleRules 13 | { 14 | public MicrosoftOpenXR(ReadOnlyTargetRules Target) : base(Target) 15 | { 16 | if (Target.Platform == UnrealTargetPlatform.Win64 || Target.Platform == UnrealTargetPlatform.HoloLens) 17 | { 18 | // these parameters mandatory for winrt support 19 | bEnableExceptions = true; 20 | bUseUnity = false; 21 | CppStandard = CppStandardVersion.Cpp17; 22 | PublicSystemLibraries.AddRange(new string[] { "shlwapi.lib", "runtimeobject.lib" }); 23 | } 24 | 25 | PCHUsage = ModuleRules.PCHUsageMode.UseExplicitOrSharedPCHs; 26 | PrivatePCHHeaderFile = @"Private\OpenXRCommon.h"; 27 | 28 | PrivateIncludePaths.AddRange( 29 | new string[] { 30 | // This private include path ensures our newer copy of the openxr headers take precedence over the engine's copy. 31 | "MicrosoftOpenXR/Private/External" 32 | } 33 | ); 34 | 35 | 36 | PrivateDependencyModuleNames.AddRange( 37 | new string[] 38 | { 39 | "Core", 40 | "CoreUObject", 41 | "ApplicationCore", 42 | "Engine", 43 | "Slate", 44 | "SlateCore", 45 | "InputCore", 46 | "OpenXRHMD", 47 | "MicrosoftOpenXRRuntimeSettings", 48 | "HeadMountedDisplay", 49 | "AugmentedReality", 50 | "OpenXRAR", 51 | "RHI", 52 | "RenderCore", 53 | "Projects", 54 | "NuGetModule" 55 | } 56 | ); 57 | 58 | PublicDependencyModuleNames.AddRange( 59 | new string[] 60 | { 61 | "NuGetModule" 62 | } 63 | ); 64 | 65 | if (Target.bBuildEditor) 66 | { 67 | PrivateDependencyModuleNames.AddRange( 68 | new string[] 69 | { 70 | "UnrealEd" 71 | } 72 | ); 73 | } 74 | 75 | PrivateIncludePathModuleNames.AddRange( 76 | new string[] 77 | { 78 | "HeadMountedDisplay", 79 | "NuGetModule" 80 | } 81 | ); 82 | 83 | PublicIncludePathModuleNames.AddRange( 84 | new string[] 85 | { 86 | "HeadMountedDisplay", 87 | "NuGetModule" 88 | } 89 | ); 90 | 91 | // DX-specific code for webcam texture processing 92 | if (Target.Platform == UnrealTargetPlatform.Win64 || Target.Platform == UnrealTargetPlatform.HoloLens) 93 | { 94 | PrivateDependencyModuleNames.AddRange(new string[] { 95 | "D3D11RHI", 96 | "D3D12RHI" 97 | }); 98 | 99 | var EngineDir = Path.GetFullPath(Target.RelativeEnginePath); 100 | PrivateIncludePaths.AddRange( 101 | new string[] { 102 | Path.Combine(EngineDir, @"Source\Runtime\Windows\D3D11RHI\Private"), 103 | Path.Combine(EngineDir, @"Source\Runtime\Windows\D3D11RHI\Private\Windows"), 104 | Path.Combine(EngineDir, @"Source\Runtime\D3D12RHI\Private"), 105 | Path.Combine(EngineDir, @"Source\Runtime\D3D12RHI\Private\Windows") 106 | }); 107 | 108 | AddEngineThirdPartyPrivateStaticDependencies(Target, "DX11"); 109 | AddEngineThirdPartyPrivateStaticDependencies(Target, "DX12"); 110 | AddEngineThirdPartyPrivateStaticDependencies(Target, "NVAPI"); 111 | AddEngineThirdPartyPrivateStaticDependencies(Target, "AMD_AGS"); 112 | AddEngineThirdPartyPrivateStaticDependencies(Target, "NVAftermath"); 113 | AddEngineThirdPartyPrivateStaticDependencies(Target, "IntelMetricsDiscovery"); 114 | AddEngineThirdPartyPrivateStaticDependencies(Target, "IntelExtensionsFramework"); 115 | } 116 | } 117 | } 118 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/AzureObjectAnchorsPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2022 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 7 | 8 | #include "OpenXRCommon.h" 9 | #include "OpenXRCore.h" 10 | #include "MicrosoftOpenXR.h" 11 | 12 | #include 13 | #include 14 | #include 15 | 16 | #include "Async/Async.h" 17 | #include "Engine/Engine.h" 18 | 19 | #include "WindowsMixedRealityInteropUtility.h" 20 | 21 | #include "IOpenXRARTrackedGeometryHolder.h" 22 | #include "IOpenXRARModule.h" 23 | #include "Misc/EngineVersionComparison.h" 24 | 25 | #include "TrackedGeometryCollision.h" 26 | #include "ARBlueprintLibrary.h" 27 | #include "HeadMountedDisplayFunctionLibrary.h" 28 | 29 | namespace OA = winrt::Microsoft::Azure::ObjectAnchors; 30 | 31 | namespace MicrosoftOpenXR 32 | { 33 | class FAzureObjectAnchorsPlugin : 34 | public IOpenXRExtensionPlugin, 35 | public IOpenXRCustomCaptureSupport, 36 | public TSharedFromThis 37 | { 38 | public: 39 | void Register(); 40 | void Unregister(); 41 | 42 | ~FAzureObjectAnchorsPlugin(); 43 | 44 | void OnStartARSession(class UARSessionConfig* SessionConfig) override; 45 | 46 | bool GetRequiredExtensions(TArray& OutExtensions) override; 47 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 48 | virtual const void* OnBeginSession(XrSession InSession, const void* InNext) override; 49 | void UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) override; 50 | 51 | bool OnToggleARCapture(const bool bOnOff) override; 52 | void InitAzureObjectAnchors(FAzureObjectAnchorSessionConfiguration AOAConfiguration); 53 | void ResetObjectSearchAreaAroundHead(); 54 | void ResetObjectSearchAreaAroundPoint(FVector Point, float Radius, bool ClearExistingSearchAreas); 55 | 56 | TArray OnLineTraceTrackedObjects(const TSharedPtr ARCompositionComponent, const FVector Start, const FVector End, const EARLineTraceChannels TraceChannels) override; 57 | 58 | private: 59 | XrSession Session; 60 | 61 | PFN_xrCreateSpatialGraphNodeSpaceMSFT xrCreateSpatialGraphNodeSpaceMSFT; 62 | PFN_xrEnumerateInstanceExtensionProperties xrEnumerateInstanceExtensionProperties; 63 | PFN_xrTryCreateSpatialGraphStaticNodeBindingMSFT xrTryCreateSpatialGraphStaticNodeBindingMSFT; 64 | PFN_xrGetSpatialGraphNodeBindingPropertiesMSFT xrGetSpatialGraphNodeBindingPropertiesMSFT; 65 | 66 | bool IsAzureObjectAnchorsStartDesired = false; 67 | 68 | bool CanUseSpatialGraphExtension = false; 69 | bool ARSessionStarted = false; 70 | bool FrameOfReferenceLocated = false; 71 | bool IsCurrentlyDetecting = false; 72 | 73 | winrt::Windows::Foundation::IAsyncOperation> DetectTask = nullptr; 74 | 75 | FAzureObjectAnchorSessionConfiguration AzureObjectAnchorConfiguration; 76 | OA::SpatialGraph::SpatialSphere SearchSphere; 77 | std::vector Queries; 78 | 79 | IOpenXRARTrackedMeshHolder* TrackedMeshHolder = nullptr; 80 | float WorldToMetersScale = 100.0f; 81 | 82 | OA::ObjectAnchorsSession ObjectAnchorSession = nullptr; 83 | OA::ObjectObserver ObjectAnchorObserver = nullptr; 84 | 85 | OA::SpatialGraph::SpatialGraphCoordinateSystem OriginCoordinateSystem; 86 | XrSpace OriginCoordinateSystemTrackingSpace = XR_NULL_HANDLE; 87 | 88 | struct AOAContext 89 | { 90 | winrt::guid SpatialGraphNodeId; 91 | XrSpace Space = XR_NULL_HANDLE; 92 | 93 | OA::ObjectModel ObjectModel = nullptr; 94 | OA::ObjectInstance ObjectInstance = nullptr; 95 | winrt::event_token ObjectInstanceChangedEventToken; 96 | 97 | bool HasChanged = false; 98 | OA::ObjectInstanceState LastKnownState; 99 | 100 | TArray Vertices; 101 | TArray Indices; 102 | 103 | TrackedGeometryCollision* CollisionInfo = nullptr; 104 | 105 | public: 106 | void ResetObjectInstance(IOpenXRARTrackedMeshHolder* TrackedMeshHolder) 107 | { 108 | if (ObjectInstance != nullptr) 109 | { 110 | if (ObjectInstanceChangedEventToken.value != 0) 111 | { 112 | ObjectInstance.Changed(ObjectInstanceChangedEventToken); 113 | } 114 | 115 | ObjectInstance.Close(); 116 | 117 | TrackedMeshHolder->StartMeshUpdates(); 118 | TrackedMeshHolder->RemoveMesh(WMRUtility::GUIDToFGuid(ObjectModel.Id())); 119 | TrackedMeshHolder->EndMeshUpdates(); 120 | } 121 | 122 | ObjectInstanceChangedEventToken.value = 0; 123 | 124 | if (Space != XR_NULL_HANDLE) 125 | { 126 | xrDestroySpace(Space); 127 | Space = XR_NULL_HANDLE; 128 | } 129 | 130 | SpatialGraphNodeId = winrt::guid(); 131 | HasChanged = false; 132 | } 133 | 134 | bool IsCurrentlyBeingTracked() 135 | { 136 | return ObjectInstanceChangedEventToken.value != 0; 137 | } 138 | 139 | // This can happen if the device loses tracking: 140 | // The underlying object instance can be closed without triggering a changed event. 141 | // Reset the object instance to find a new one. 142 | void ResetStaleObjectInstance(IOpenXRARTrackedMeshHolder* TrackedMeshHolder) 143 | { 144 | if (IsCurrentlyBeingTracked() && 145 | (ObjectInstance == nullptr || 146 | ObjectInstance.TryGetCurrentState() == nullptr)) 147 | { 148 | ResetObjectInstance(TrackedMeshHolder); 149 | } 150 | } 151 | }; 152 | 153 | FCriticalSection MeshComponentLock; 154 | TMap AOAMap; 155 | 156 | std::vector SearchAreas; 157 | 158 | void StopAzureObjectAnchors(); 159 | void LoadObjectModelsAsync(winrt::hstring Path); 160 | void UpdateSpatialGraphInfoForObjectInstance(OA::ObjectInstance Object); 161 | }; 162 | } // namespace MicrosoftOpenXR 163 | 164 | #endif //PLATFORM_WINDOWS || PLATFORM_HOLOLENS -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/External/openxr/openxr_platform_defines.h: -------------------------------------------------------------------------------- 1 | /* 2 | ** Copyright (c) 2017-2022, The Khronos Group Inc. 3 | ** 4 | ** SPDX-License-Identifier: Apache-2.0 OR MIT 5 | */ 6 | 7 | #ifndef OPENXR_PLATFORM_DEFINES_H_ 8 | #define OPENXR_PLATFORM_DEFINES_H_ 1 9 | 10 | #ifdef __cplusplus 11 | extern "C" { 12 | #endif 13 | 14 | /* Platform-specific calling convention macros. 15 | * 16 | * Platforms should define these so that OpenXR clients call OpenXR functions 17 | * with the same calling conventions that the OpenXR implementation expects. 18 | * 19 | * XRAPI_ATTR - Placed before the return type in function declarations. 20 | * Useful for C++11 and GCC/Clang-style function attribute syntax. 21 | * XRAPI_CALL - Placed after the return type in function declarations. 22 | * Useful for MSVC-style calling convention syntax. 23 | * XRAPI_PTR - Placed between the '(' and '*' in function pointer types. 24 | * 25 | * Function declaration: XRAPI_ATTR void XRAPI_CALL xrFunction(void); 26 | * Function pointer type: typedef void (XRAPI_PTR *PFN_xrFunction)(void); 27 | */ 28 | #if defined(_WIN32) 29 | #define XRAPI_ATTR 30 | // On Windows, functions use the stdcall convention 31 | #define XRAPI_CALL __stdcall 32 | #define XRAPI_PTR XRAPI_CALL 33 | #elif defined(__ANDROID__) && defined(__ARM_ARCH) && __ARM_ARCH < 7 34 | #error "API not supported for the 'armeabi' NDK ABI" 35 | #elif defined(__ANDROID__) && defined(__ARM_ARCH) && __ARM_ARCH >= 7 && defined(__ARM_32BIT_STATE) 36 | // On Android 32-bit ARM targets, functions use the "hardfloat" 37 | // calling convention, i.e. float parameters are passed in registers. This 38 | // is true even if the rest of the application passes floats on the stack, 39 | // as it does by default when compiling for the armeabi-v7a NDK ABI. 40 | #define XRAPI_ATTR __attribute__((pcs("aapcs-vfp"))) 41 | #define XRAPI_CALL 42 | #define XRAPI_PTR XRAPI_ATTR 43 | #else 44 | // On other platforms, use the default calling convention 45 | #define XRAPI_ATTR 46 | #define XRAPI_CALL 47 | #define XRAPI_PTR 48 | #endif 49 | 50 | #include 51 | 52 | #if !defined(XR_NO_STDINT_H) 53 | #if defined(_MSC_VER) && (_MSC_VER < 1600) 54 | typedef signed __int8 int8_t; 55 | typedef unsigned __int8 uint8_t; 56 | typedef signed __int16 int16_t; 57 | typedef unsigned __int16 uint16_t; 58 | typedef signed __int32 int32_t; 59 | typedef unsigned __int32 uint32_t; 60 | typedef signed __int64 int64_t; 61 | typedef unsigned __int64 uint64_t; 62 | #else 63 | #include 64 | #endif 65 | #endif // !defined( XR_NO_STDINT_H ) 66 | 67 | // XR_PTR_SIZE (in bytes) 68 | #if (defined(__LP64__) || defined(_WIN64) || (defined(__x86_64__) && !defined(__ILP32__) ) || defined(_M_X64) || defined(__ia64) || defined (_M_IA64) || defined(__aarch64__) || defined(__powerpc64__)) 69 | #define XR_PTR_SIZE 8 70 | #else 71 | #define XR_PTR_SIZE 4 72 | #endif 73 | 74 | // Needed so we can use clang __has_feature portably. 75 | #if !defined(XR_COMPILER_HAS_FEATURE) 76 | #if defined(__clang__) 77 | #define XR_COMPILER_HAS_FEATURE(x) __has_feature(x) 78 | #else 79 | #define XR_COMPILER_HAS_FEATURE(x) 0 80 | #endif 81 | #endif 82 | 83 | // Identifies if the current compiler has C++11 support enabled. 84 | // Does not by itself identify if any given C++11 feature is present. 85 | #if !defined(XR_CPP11_ENABLED) && defined(__cplusplus) 86 | #if defined(__GNUC__) && defined(__GXX_EXPERIMENTAL_CXX0X__) 87 | #define XR_CPP11_ENABLED 1 88 | #elif defined(_MSC_VER) && (_MSC_VER >= 1600) 89 | #define XR_CPP11_ENABLED 1 90 | #elif (__cplusplus >= 201103L) // 201103 is the first C++11 version. 91 | #define XR_CPP11_ENABLED 1 92 | #endif 93 | #endif 94 | 95 | // Identifies if the current compiler supports C++11 nullptr. 96 | #if !defined(XR_CPP_NULLPTR_SUPPORTED) 97 | #if defined(XR_CPP11_ENABLED) && \ 98 | ((defined(__clang__) && XR_COMPILER_HAS_FEATURE(cxx_nullptr)) || \ 99 | (defined(__GNUC__) && (((__GNUC__ * 1000) + __GNUC_MINOR__) >= 4006)) || \ 100 | (defined(_MSC_VER) && (_MSC_VER >= 1600)) || \ 101 | (defined(__EDG_VERSION__) && (__EDG_VERSION__ >= 403))) 102 | #define XR_CPP_NULLPTR_SUPPORTED 1 103 | #endif 104 | #endif 105 | 106 | #ifdef __cplusplus 107 | } 108 | #endif 109 | 110 | #endif 111 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/HandMeshPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | #include "ARTypes.h" 8 | #include "MicrosoftOpenXR.h" 9 | #include "IHandTracker.h" 10 | 11 | #include 12 | 13 | class IOpenXRARTrackedMeshHolder; 14 | 15 | namespace MicrosoftOpenXR 16 | { 17 | class FHandMeshPlugin : public IOpenXRExtensionPlugin, public IHandTracker 18 | { 19 | public: 20 | static const int HandCount = 2; 21 | enum Hand {Left = 0, Right = 1}; 22 | 23 | struct FHandState : public FNoncopyable 24 | { 25 | FHandState(); 26 | 27 | XrHandTrackerEXT HandTracker{}; 28 | XrSpace Space{}; 29 | 30 | std::vector Indices; 31 | std::vector Vertices; 32 | 33 | size_t VerticesCount = 0; 34 | size_t IndicesCount = 0; 35 | 36 | size_t VerticesMaxAmount = 0; 37 | size_t IndicesMaxAmount = 0; 38 | 39 | FGuid Guid; 40 | 41 | EARTrackingState TrackingState = EARTrackingState::Unknown; 42 | FTransform LocalToTrackingTransform; 43 | }; 44 | 45 | void Register(); 46 | void Unregister(); 47 | 48 | bool GetRequiredExtensions(TArray& OutExtensions) override; 49 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 50 | const void* OnBeginSession(XrSession InSession, const void* InNext) override; 51 | void UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) override; 52 | 53 | bool Turn(EHandMeshStatus Mode); 54 | private: 55 | FHandState HandStates[HandCount]; 56 | 57 | EHandMeshStatus HandMeshStatus = EHandMeshStatus::NotInitialised; 58 | 59 | PFN_xrCreateHandTrackerEXT xrCreateHandTrackerEXT = nullptr; 60 | 61 | PFN_xrCreateHandMeshSpaceMSFT xrCreateHandMeshSpaceMSFT = nullptr; 62 | PFN_xrUpdateHandMeshMSFT xrUpdateHandMeshMSFT = nullptr; 63 | 64 | class IXRTrackingSystem* XRTrackingSystem = nullptr; 65 | IOpenXRARTrackedMeshHolder* TrackedMeshHolder = nullptr; 66 | 67 | // Inherited via IHandTracker 68 | virtual FName GetHandTrackerDeviceTypeName() const override 69 | { 70 | return FName(TEXT("MicrosoftOpenXRHandTracking")); 71 | } 72 | 73 | virtual bool IsHandTrackingStateValid() const override; 74 | 75 | virtual bool GetKeypointState(EControllerHand Hand, EHandKeypoint Keypoint, FTransform& OutTransform, float& OutRadius) const override 76 | { 77 | return false; 78 | } 79 | 80 | virtual bool GetAllKeypointStates(EControllerHand Hand, TArray& OutPositions, TArray& OutRotations, TArray& OutRadii) const override 81 | { 82 | return false; 83 | } 84 | 85 | virtual bool HasHandMeshData() const override; 86 | 87 | virtual bool GetHandMeshData(EControllerHand Hand, TArray& OutVertices, TArray& OutNormals, TArray& OutIndices, FTransform& OutHandMeshTransform) const override; 88 | }; 89 | } // namespace MicrosoftOpenXR 90 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/HolographicRemotingPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "MicrosoftOpenXR.h" 7 | 8 | #if SUPPORTS_REMOTING 9 | 10 | #include "Interfaces/IPluginManager.h" 11 | 12 | #include "OpenXRCommon.h" 13 | #include "openxr_msft_holographic_remoting.h " 14 | 15 | #include "Windows/AllowWindowsPlatformTypes.h" 16 | #include 17 | #include "Windows/HideWindowsPlatformTypes.h" 18 | 19 | #include "OpenXRCore.h" 20 | #include "HeadMountedDisplayTypes.h" 21 | 22 | #include "Engine/World.h" 23 | 24 | #if WITH_EDITOR 25 | #include "Editor.h" 26 | #endif 27 | 28 | #include "MicrosoftOpenXRRuntimeSettings.h" 29 | 30 | #include "Misc/EngineVersionComparison.h" 31 | 32 | // Microsoft.Holographic.AppRemoting binaries only exist for Win64. 33 | #define SUPPORTS_REMOTING_IN_PACKAGED_BUILD (!WITH_EDITOR && PLATFORM_DESKTOP && PLATFORM_64BITS) 34 | 35 | namespace MicrosoftOpenXR 36 | { 37 | class FHolographicRemotingPlugin : public IOpenXRExtensionPlugin, public TSharedFromThis 38 | { 39 | public: 40 | void Register(); 41 | void Unregister(); 42 | 43 | bool GetCustomLoader(PFN_xrGetInstanceProcAddr* OutGetProcAddr) override; 44 | bool GetRequiredExtensions(TArray& OutExtensions) override; 45 | void OnEvent(XrSession InSession, const XrEventDataBaseHeader* InHeader) override; 46 | void PostGetSystem(XrInstance InInstance, XrSystemId InSystem) override; 47 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 48 | 49 | bool IsRemoting() 50 | { 51 | return remotingEnabled; 52 | } 53 | 54 | private: 55 | void* LoaderHandle; 56 | bool remotingEnabled = false; 57 | bool autoConnectRemoting = false; 58 | 59 | XrInstance Instance; 60 | XrSystemId System = XR_NULL_SYSTEM_ID; 61 | 62 | RemotingConnectionData remotingConnectionData; 63 | 64 | XrRemotingConnectionStateMSFT GetRemotingConnectionState(); 65 | void BindConnectButtons(); 66 | void ConnectToRemoteDevice(RemotingConnectionData data); 67 | void DisconnectFromRemoteDevice(); 68 | 69 | void SetRemotingStatusText(FString message, FLinearColor statusColor); 70 | void UpdateDisconnectedText(); 71 | FString GetDisconnectedReason(int index); 72 | 73 | void ParseRemotingConfig(); 74 | bool ParseRemotingCmdArgs(); 75 | }; 76 | } // namespace MicrosoftOpenXR 77 | 78 | #endif 79 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/HolographicWindowAttachmentPlugin.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "HolographicWindowAttachmentPlugin.h" 5 | 6 | #if PLATFORM_HOLOLENS 7 | #include "HoloLens/HoloLensApplication.h" 8 | 9 | #include 10 | 11 | namespace MicrosoftOpenXR 12 | { 13 | void FHolographicWindowAttachmentPlugin::Register() 14 | { 15 | IModularFeatures::Get().RegisterModularFeature(GetModularFeatureName(), this); 16 | } 17 | 18 | void FHolographicWindowAttachmentPlugin::Unregister() 19 | { 20 | IModularFeatures::Get().UnregisterModularFeature(GetModularFeatureName(), this); 21 | } 22 | 23 | bool FHolographicWindowAttachmentPlugin::GetOptionalExtensions(TArray& OutExtensions) 24 | { 25 | OutExtensions.Add(XR_MSFT_HOLOGRAPHIC_WINDOW_ATTACHMENT_EXTENSION_NAME); 26 | return true; 27 | } 28 | 29 | const void* FHolographicWindowAttachmentPlugin::OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) 30 | { 31 | // Ensure xrCreateSession is called from the game thread which is the main app view CoreWindow thread. 32 | check(IsInGameThread()); 33 | if (IOpenXRHMDPlugin::Get().IsExtensionEnabled(XR_MSFT_HOLOGRAPHIC_WINDOW_ATTACHMENT_EXTENSION_NAME)) 34 | { 35 | // If the app requests to start in VR then the XrSession should take over the main app view's CoreWindow and 36 | // HolographicSpace. This ensures keyboard input and other things will still route to the engine. Otherwise the 37 | // XrSession will create its own CoreWindow and HolographicSpace and it will take all focus. The HolographicSpace will 38 | // only be available if start in VR is enabled. 39 | Windows::Graphics::Holographic::HolographicSpace ^ holographicSpace = 40 | FHoloLensApplication::GetHoloLensApplication()->GetHolographicSpace(); 41 | if (holographicSpace) 42 | { 43 | HolographicWindowAttachment.next = InNext; 44 | HolographicWindowAttachment.holographicSpace = reinterpret_cast<::IUnknown*>(holographicSpace); 45 | HolographicWindowAttachment.coreWindow = 46 | reinterpret_cast<::IUnknown*>(Windows::UI::Core::CoreWindow::GetForCurrentThread()); 47 | return &HolographicWindowAttachment; 48 | } 49 | } 50 | 51 | return InNext; 52 | } 53 | } // namespace MicrosoftOpenXR 54 | 55 | #endif 56 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/HolographicWindowAttachmentPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_HOLOLENS 7 | 8 | #include "OpenXRCommon.h" 9 | 10 | namespace MicrosoftOpenXR 11 | { 12 | class FHolographicWindowAttachmentPlugin : public IOpenXRExtensionPlugin 13 | { 14 | public: 15 | void Register(); 16 | void Unregister(); 17 | 18 | bool GetOptionalExtensions(TArray& OutExtensions) override; 19 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 20 | 21 | private: 22 | XrHolographicWindowAttachmentMSFT HolographicWindowAttachment{XR_TYPE_HOLOGRAPHIC_WINDOW_ATTACHMENT_MSFT}; 23 | }; 24 | } // namespace MicrosoftOpenXR 25 | 26 | #endif 27 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/LocatableCamPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 6 | 7 | #include "OpenXRCommon.h" 8 | #include "ARTypes.h" 9 | #include "MicrosoftOpenXR.h" 10 | #include "ARTextures.h" 11 | 12 | #include "Windows/AllowWindowsPlatformTypes.h" 13 | #include "Windows/AllowWindowsPlatformAtomics.h" 14 | #include "Windows/PreWindowsApi.h" 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | #include "Windows/PostWindowsApi.h" 27 | #include "Windows/HideWindowsPlatformAtomics.h" 28 | #include "Windows/HideWindowsPlatformTypes.h" 29 | 30 | class UOpenXRCameraImageTexture; 31 | 32 | namespace MicrosoftOpenXR 33 | { 34 | class FLocatableCamPlugin : public IOpenXRExtensionPlugin, public IOpenXRCustomCaptureSupport 35 | { 36 | public: 37 | 38 | FLocatableCamPlugin(); 39 | ~FLocatableCamPlugin(); 40 | 41 | void Register(); 42 | void Unregister(); 43 | 44 | bool GetRequiredExtensions(TArray& OutExtensions) override; 45 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 46 | void UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) override; 47 | 48 | virtual void OnStartARSession(class UARSessionConfig* SessionConfig) override; 49 | virtual void OnStopARSession() override; 50 | 51 | virtual IOpenXRCustomCaptureSupport* GetCustomCaptureSupport(const EARCaptureType CaptureType) override; 52 | 53 | 54 | FTransform GetCameraTransform() const override; 55 | 56 | bool GetPVCameraIntrinsics(FVector2D& focalLength, int& width, int& height, FVector2D& principalPoint, FVector& radialDistortion, FVector2D& tangentialDistortion) const; 57 | 58 | FVector GetWorldSpaceRayFromCameraPoint(FVector2D pixelCoordinate) const override; 59 | 60 | virtual bool OnGetCameraIntrinsics(FARCameraIntrinsics& OutCameraIntrinsics) const override; 61 | virtual class UARTexture* OnGetARTexture(EARTextureType TextureType) const override; 62 | virtual bool OnToggleARCapture(const bool bOnOff) override; 63 | bool IsEnabled() const override; 64 | 65 | private: 66 | 67 | class FSharedTextureHolder : public FGCObject 68 | { 69 | public: 70 | virtual void AddReferencedObjects(FReferenceCollector& Collector) override; 71 | 72 | UOpenXRCameraImageTexture* CameraImage = nullptr; 73 | }; 74 | 75 | PFN_xrCreateSpatialGraphNodeSpaceMSFT xrCreateSpatialGraphNodeSpaceMSFT; 76 | 77 | void StartCameraCapture(int DesiredWidth, int DesiredHeight, int DesiredFPS); 78 | void StopCameraCapture(); 79 | 80 | void OnFrameArrived(winrt::Windows::Media::Capture::Frames::MediaFrameReader SendingFrameReader, winrt::Windows::Media::Capture::Frames::MediaFrameArrivedEventArgs FrameArrivedArgs); 81 | 82 | 83 | /** Controls access to our references */ 84 | mutable std::recursive_mutex RefsLock; 85 | /** The objects we need in order to receive frames of camera data */ 86 | winrt::agile_ref CameraCapture = nullptr; 87 | winrt::Windows::Media::Capture::Frames::MediaFrameReader CameraFrameReader = nullptr; 88 | winrt::Windows::Media::Capture::Frames::MediaFrameSource CameraFrameSource = nullptr; 89 | winrt::Windows::Media::Devices::Core::CameraIntrinsics CameraIntrinsics = nullptr; 90 | 91 | TOptional> DynamicNode; 92 | 93 | winrt::Windows::Media::Capture::Frames::MediaFrameReader::FrameArrived_revoker OnFrameArrivedEvent; 94 | 95 | winrt::Windows::Foundation::IAsyncInfo AsyncInfo; 96 | 97 | std::shared_ptr SharedDXTexture; 98 | XrSpace Space = XR_NULL_HANDLE; 99 | 100 | class IXRTrackingSystem* XRTrackingSystem = nullptr; 101 | FARVideoFormat Format; 102 | 103 | FTransform PVCameraToWorldMatrix; 104 | TSharedPtr< FSharedTextureHolder> SharedTextureHolder; 105 | 106 | bool IsCameraCaptureDesired = false; 107 | }; 108 | } // namespace MicrosoftOpenXR 109 | 110 | #endif //PLATFORM_WINDOWS || PLATFORM_HOLOLENS 111 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/MicrosoftOpenXR.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MicrosoftOpenXR.h" 5 | 6 | #include "AzureObjectAnchorsPlugin.h" 7 | #include "CoreMinimal.h" 8 | #include "HandMeshPlugin.h" 9 | #include "HolographicRemotingPlugin.h" 10 | #include "HolographicWindowAttachmentPlugin.h" 11 | #include "Interfaces/IPluginManager.h" 12 | #include "LocatableCamPlugin.h" 13 | #include "Modules/ModuleManager.h" 14 | #include "QRTrackingPlugin.h" 15 | #include "SceneUnderstandingPlugin.h" 16 | #include "SecondaryViewConfiguration.h" 17 | #include "ShaderCore.h" 18 | #include "SpatialAnchorPlugin.h" 19 | #include "SpatialMappingPlugin.h" 20 | #include "SpeechPlugin.h" 21 | 22 | DEFINE_LOG_CATEGORY(LogAOA) 23 | 24 | #define LOCTEXT_NAMESPACE "FMicrosoftOpenXRModule" 25 | 26 | namespace MicrosoftOpenXR 27 | { 28 | static class FMicrosoftOpenXRModule * g_MicrosoftOpenXRModule; 29 | 30 | class FMicrosoftOpenXRModule : public IModuleInterface 31 | { 32 | public: 33 | void StartupModule() override 34 | { 35 | SpatialAnchorPlugin.Register(); 36 | HandMeshPlugin.Register(); 37 | SecondaryViewConfigurationPlugin.Register(); 38 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 39 | AzureObjectAnchorsPlugin = MakeShared(); 40 | AzureObjectAnchorsPlugin->Register(); 41 | QRTrackingPlugin.Register(); 42 | LocatableCamPlugin.Register(); 43 | SpeechPlugin.Register(); 44 | SpatialMappingPlugin.Register(); 45 | #if !UE_VERSION_OLDER_THAN(4, 27, 1) 46 | SceneUnderstandingPlugin.Register(); 47 | #endif // !UE_VERSION_OLDER_THAN(4, 27, 1) 48 | #endif // PLATFORM_WINDOWS || PLATFORM_HOLOLENS 49 | 50 | #if SUPPORTS_REMOTING 51 | HolographicRemotingPlugin = MakeShared(); 52 | HolographicRemotingPlugin->Register(); 53 | #endif 54 | 55 | #if PLATFORM_HOLOLENS 56 | HolographicWindowAttachmentPlugin.Register(); 57 | #endif 58 | 59 | g_MicrosoftOpenXRModule = this; 60 | } 61 | 62 | void ShutdownModule() override 63 | { 64 | g_MicrosoftOpenXRModule = nullptr; 65 | 66 | SpatialAnchorPlugin.Unregister(); 67 | HandMeshPlugin.Unregister(); 68 | SecondaryViewConfigurationPlugin.Unregister(); 69 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 70 | AzureObjectAnchorsPlugin->Unregister(); 71 | QRTrackingPlugin.Unregister(); 72 | LocatableCamPlugin.Unregister(); 73 | SpeechPlugin.Unregister(); 74 | SpatialMappingPlugin.Unregister(); 75 | #if !UE_VERSION_OLDER_THAN(4, 27, 1) 76 | SceneUnderstandingPlugin.Unregister(); 77 | #endif // !UE_VERSION_OLDER_THAN(4, 27, 1) 78 | #endif // PLATFORM_WINDOWS || PLATFORM_HOLOLENS 79 | 80 | #if SUPPORTS_REMOTING 81 | HolographicRemotingPlugin->Unregister(); 82 | #endif 83 | 84 | #if PLATFORM_HOLOLENS 85 | HolographicWindowAttachmentPlugin.Unregister(); 86 | #endif 87 | } 88 | 89 | FSecondaryViewConfigurationPlugin SecondaryViewConfigurationPlugin; 90 | FHandMeshPlugin HandMeshPlugin; 91 | FSpatialAnchorPlugin SpatialAnchorPlugin; 92 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 93 | TSharedPtr AzureObjectAnchorsPlugin; 94 | FQRTrackingPlugin QRTrackingPlugin; 95 | FLocatableCamPlugin LocatableCamPlugin; 96 | FSpeechPlugin SpeechPlugin; 97 | FSpatialMappingPlugin SpatialMappingPlugin; 98 | #if !UE_VERSION_OLDER_THAN(4, 27, 1) 99 | FSceneUnderstandingPlugin SceneUnderstandingPlugin; 100 | #endif // !UE_VERSION_OLDER_THAN(4, 27, 1) 101 | #endif // PLATFORM_WINDOWS || PLATFORM_HOLOLENS 102 | 103 | #if SUPPORTS_REMOTING 104 | TSharedPtr HolographicRemotingPlugin; 105 | #endif 106 | 107 | #if PLATFORM_HOLOLENS 108 | FHolographicWindowAttachmentPlugin HolographicWindowAttachmentPlugin; 109 | #endif 110 | }; 111 | } // namespace MicrosoftOpenXR 112 | 113 | bool UMicrosoftOpenXRFunctionLibrary::SetUseHandMesh(EHandMeshStatus Mode) 114 | { 115 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->HandMeshPlugin.Turn(Mode); 116 | } 117 | 118 | bool UMicrosoftOpenXRFunctionLibrary::IsQREnabled() 119 | { 120 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 121 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->QRTrackingPlugin.IsEnabled(); 122 | #else 123 | return false; 124 | #endif 125 | } 126 | 127 | FTransform UMicrosoftOpenXRFunctionLibrary::GetPVCameraToWorldTransform() 128 | { 129 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 130 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->LocatableCamPlugin.GetCameraTransform(); 131 | #else 132 | return FTransform::Identity; 133 | #endif 134 | } 135 | 136 | bool UMicrosoftOpenXRFunctionLibrary::GetPVCameraIntrinsics(FVector2D& focalLength, int& width, int& height, 137 | FVector2D& principalPoint, FVector& radialDistortion, FVector2D& tangentialDistortion) 138 | { 139 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 140 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->LocatableCamPlugin.GetPVCameraIntrinsics( 141 | focalLength, width, height, principalPoint, radialDistortion, tangentialDistortion); 142 | #else 143 | return false; 144 | #endif 145 | } 146 | 147 | FVector UMicrosoftOpenXRFunctionLibrary::GetWorldSpaceRayFromCameraPoint(FVector2D pixelCoordinate) 148 | { 149 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 150 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->LocatableCamPlugin.GetWorldSpaceRayFromCameraPoint(pixelCoordinate); 151 | #else 152 | return FVector::ZeroVector; 153 | #endif 154 | } 155 | 156 | bool UMicrosoftOpenXRFunctionLibrary::IsSpeechRecognitionAvailable() 157 | { 158 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 159 | return true; 160 | #endif 161 | 162 | return false; 163 | } 164 | 165 | void UMicrosoftOpenXRFunctionLibrary::AddKeywords(TArray Keywords) 166 | { 167 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 168 | MicrosoftOpenXR::g_MicrosoftOpenXRModule->SpeechPlugin.AddKeywords(Keywords); 169 | #endif 170 | } 171 | 172 | void UMicrosoftOpenXRFunctionLibrary::RemoveKeywords(TArray Keywords) 173 | { 174 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 175 | MicrosoftOpenXR::g_MicrosoftOpenXRModule->SpeechPlugin.RemoveKeywords(Keywords); 176 | #endif 177 | } 178 | 179 | bool UMicrosoftOpenXRFunctionLibrary::GetPerceptionAnchorFromOpenXRAnchor(void* AnchorID, void** OutPerceptionAnchor) 180 | { 181 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 182 | if (MicrosoftOpenXR::g_MicrosoftOpenXRModule == nullptr) 183 | { 184 | return false; 185 | } 186 | 187 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->SpatialAnchorPlugin.GetPerceptionAnchorFromOpenXRAnchor( 188 | (XrSpatialAnchorMSFT) AnchorID, (::IUnknown**)OutPerceptionAnchor); 189 | #else 190 | return false; 191 | #endif 192 | } 193 | 194 | bool UMicrosoftOpenXRFunctionLibrary::StorePerceptionAnchor(const FString& InPinId, void* InPerceptionAnchor) 195 | { 196 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 197 | if (MicrosoftOpenXR::g_MicrosoftOpenXRModule == nullptr) 198 | { 199 | return false; 200 | } 201 | 202 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->SpatialAnchorPlugin.StorePerceptionAnchor(InPinId, (::IUnknown*)InPerceptionAnchor); 203 | #else 204 | return false; 205 | #endif 206 | } 207 | 208 | bool UMicrosoftOpenXRFunctionLibrary::IsRemoting() 209 | { 210 | #if SUPPORTS_REMOTING 211 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->HolographicRemotingPlugin->IsRemoting(); 212 | #endif 213 | 214 | return false; 215 | } 216 | 217 | bool UMicrosoftOpenXRFunctionLibrary::CanDetectPlanes() 218 | { 219 | #if UE_VERSION_OLDER_THAN(4, 27, 1) 220 | // SU was introduced in 4.27.1 221 | return false; 222 | #elif PLATFORM_WINDOWS || PLATFORM_HOLOLENS 223 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->SceneUnderstandingPlugin.CanDetectPlanes(); 224 | #endif 225 | 226 | return false; 227 | } 228 | 229 | bool UMicrosoftOpenXRFunctionLibrary::ToggleAzureObjectAnchors(const bool bOnOff) 230 | { 231 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 232 | return MicrosoftOpenXR::g_MicrosoftOpenXRModule->AzureObjectAnchorsPlugin->OnToggleARCapture(bOnOff); 233 | #endif 234 | 235 | return true; 236 | } 237 | 238 | void UMicrosoftOpenXRFunctionLibrary::InitAzureObjectAnchors(FAzureObjectAnchorSessionConfiguration AOAConfiguration) 239 | { 240 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 241 | MicrosoftOpenXR::g_MicrosoftOpenXRModule->AzureObjectAnchorsPlugin->InitAzureObjectAnchors(AOAConfiguration); 242 | #endif 243 | } 244 | 245 | void UMicrosoftOpenXRFunctionLibrary::ResetObjectSearchAreaAroundHead() 246 | { 247 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 248 | MicrosoftOpenXR::g_MicrosoftOpenXRModule->AzureObjectAnchorsPlugin->ResetObjectSearchAreaAroundHead(); 249 | #endif 250 | } 251 | 252 | void UMicrosoftOpenXRFunctionLibrary::ResetObjectSearchAreaAroundPoint(FVector Point, float Radius, bool ClearExistingSearchAreas) 253 | { 254 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 255 | MicrosoftOpenXR::g_MicrosoftOpenXRModule->AzureObjectAnchorsPlugin->ResetObjectSearchAreaAroundPoint(Point, Radius, ClearExistingSearchAreas); 256 | #endif 257 | } 258 | 259 | TArray UMicrosoftOpenXRFunctionLibrary::LineTraceTrackedAzureObjectAnchors(const FVector Start, const FVector End) 260 | { 261 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 262 | TArray Results = MicrosoftOpenXR::g_MicrosoftOpenXRModule->AzureObjectAnchorsPlugin->OnLineTraceTrackedObjects(nullptr, Start, End, EARLineTraceChannels::None); 263 | 264 | Results.Sort([](const FARTraceResult& A, const FARTraceResult& B) 265 | { 266 | return A.GetDistanceFromCamera() < B.GetDistanceFromCamera(); 267 | }); 268 | 269 | return Results; 270 | #endif 271 | 272 | return {}; 273 | } 274 | 275 | #undef LOCTEXT_NAMESPACE 276 | 277 | IMPLEMENT_MODULE(MicrosoftOpenXR::FMicrosoftOpenXRModule, MicrosoftOpenXR) 278 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/OpenXRCameraImageTexture.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "ARTextures.h" 7 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 8 | #include 9 | #include 10 | #endif 11 | 12 | 13 | #include "OpenXRCameraImageTexture.generated.h" 14 | 15 | /** 16 | * Provides access to the camera's image data as a texture 17 | */ 18 | UCLASS(NotBlueprintType) 19 | class UOpenXRCameraImageTexture : 20 | public UARTextureCameraImage 21 | { 22 | GENERATED_UCLASS_BODY() 23 | 24 | public: 25 | // UTexture interface implementation 26 | virtual void BeginDestroy() override; 27 | virtual FTextureResource* CreateResource() override; 28 | virtual EMaterialValueType GetMaterialType() const override { return MCT_Texture2D; } 29 | // End UTexture interface 30 | 31 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 32 | /** Forces the reconstruction of the texture data and conversion from Nv12 to RGB */ 33 | virtual void Init(std::shared_ptr handle); 34 | #endif 35 | 36 | friend class FOpenXRCameraImageResource; 37 | 38 | private: 39 | /** Used to prevent two updates of the texture in the same game frame */ 40 | uint64 LastUpdateFrame; 41 | 42 | bool IsDX11 = false; 43 | }; 44 | 45 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/OpenXRCameraImageTexture_UE426.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "OpenXRCameraImageTexture.h" 5 | #include 6 | 7 | #if UE_VERSION_OLDER_THAN(4, 27, 0) // See OpenXRCameraImageTexture.cpp for the 4.27 implementation. 8 | 9 | #include "GlobalShader.h" 10 | #include "RenderUtils.h" 11 | #include "RHIStaticStates.h" 12 | #include "PipelineStateCache.h" 13 | #include "ShaderParameterUtils.h" 14 | #include "SceneUtils.h" 15 | #include "MediaShaders.h" 16 | #include "HeadMountedDisplayTypes.h" 17 | 18 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 19 | #include "Windows/AllowWindowsPlatformTypes.h" 20 | 21 | #pragma warning(push) 22 | #pragma warning(disable:4005) // macro redefinition 23 | THIRD_PARTY_INCLUDES_START 24 | #include 25 | #include 26 | #include 27 | #include 28 | THIRD_PARTY_INCLUDES_END 29 | #pragma warning(pop) 30 | 31 | #include "Windows/COMPointer.h" 32 | #include "Windows/HideWindowsPlatformTypes.h" 33 | #endif 34 | 35 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 36 | 37 | /** Resource class to do all of the setup work on the render thread */ 38 | class FOpenXRCameraImageResource : 39 | public FTextureResource 40 | { 41 | public: 42 | FOpenXRCameraImageResource(UOpenXRCameraImageTexture* InOwner) 43 | : LastFrameNumber(0) 44 | , Owner(InOwner) 45 | { 46 | } 47 | 48 | virtual ~FOpenXRCameraImageResource() 49 | { 50 | } 51 | 52 | /** 53 | * Called when the resource is initialized. This is only called by the rendering thread. 54 | */ 55 | virtual void InitRHI() override 56 | { 57 | check(IsInRenderingThread()); 58 | 59 | FString RHIString = FApp::GetGraphicsRHI(); 60 | if (RHIString.IsEmpty()) 61 | { 62 | return; 63 | } 64 | 65 | if (RHIString != TEXT("DirectX 11")) 66 | { 67 | return; 68 | } 69 | 70 | FSamplerStateInitializerRHI SamplerStateInitializer(SF_Bilinear, AM_Clamp, AM_Clamp, AM_Clamp); 71 | SamplerStateRHI = RHICreateSamplerState(SamplerStateInitializer); 72 | 73 | bool bDidConvert = false; 74 | if (CameraImageHandle) 75 | { 76 | // Open the shared texture from the HoloLens camera on Unreal's d3d device. 77 | ID3D11Device* D3D11Device = static_cast(GDynamicRHI->RHIGetNativeDevice()); 78 | TComPtr D3D11DeviceContext = nullptr; 79 | D3D11Device->GetImmediateContext(&D3D11DeviceContext); 80 | if (D3D11DeviceContext == nullptr) 81 | { 82 | CameraImageHandle = nullptr; 83 | 84 | return; 85 | } 86 | 87 | TComPtr cameraImageTexture; 88 | TComPtr cameraImageResource(NULL); 89 | if (FAILED(((ID3D11Device1*)D3D11Device)->OpenSharedResource1(CameraImageHandle->get(), __uuidof(IDXGIResource1), (void**)&cameraImageResource))) 90 | { 91 | UE_LOG(LogHMD, Log, TEXT("ID3D11Device1::OpenSharedResource1 failed in FOpenXRCameraImageResource::InitRHI")); 92 | return; 93 | } 94 | if (FAILED(cameraImageResource->QueryInterface(__uuidof(ID3D11Texture2D), (void**)(&cameraImageTexture)))) 95 | { 96 | UE_LOG(LogHMD, Log, TEXT("IDXGIResource1::QueryInterface failed in FOpenXRCameraImageResource::InitRHI")); 97 | return; 98 | } 99 | 100 | D3D11_TEXTURE2D_DESC Desc; 101 | cameraImageTexture->GetDesc(&Desc); 102 | 103 | Size.X = Desc.Width; 104 | Size.Y = Desc.Height; 105 | 106 | // Create the copy target 107 | { 108 | FRHIResourceCreateInfo CreateInfo; 109 | CopyTextureRef = RHICreateTexture2D(Size.X, Size.Y, PF_NV12, 1, 1, TexCreate_Dynamic | TexCreate_ShaderResource, CreateInfo); 110 | } 111 | // Create the render target 112 | { 113 | FRHIResourceCreateInfo CreateInfo; 114 | TRefCountPtr DummyTexture2DRHI; 115 | // Create our render target that we'll convert to 116 | RHICreateTargetableShaderResource2D(Size.X, Size.Y, PF_B8G8R8A8, 1, TexCreate_Dynamic, TexCreate_RenderTargetable, false, CreateInfo, DecodedTextureRef, DummyTexture2DRHI); 117 | } 118 | 119 | if (PerformCopy(cameraImageTexture, D3D11DeviceContext)) 120 | { 121 | PerformConversion(); 122 | bDidConvert = true; 123 | } 124 | // Now that the conversion is done, we can get rid of our refs 125 | CameraImageHandle = nullptr; 126 | CopyTextureRef.SafeRelease(); 127 | } 128 | 129 | // Default to an empty 1x1 texture if we don't have a camera image or failed to convert 130 | if (!bDidConvert) 131 | { 132 | FRHIResourceCreateInfo CreateInfo; 133 | Size.X = Size.Y = 1; 134 | DecodedTextureRef = RHICreateTexture2D(Size.X, Size.Y, PF_B8G8R8A8, 1, 1, TexCreate_ShaderResource, CreateInfo); 135 | } 136 | 137 | TextureRHI = DecodedTextureRef; 138 | TextureRHI->SetName(Owner->GetFName()); 139 | RHIBindDebugLabelName(TextureRHI, *Owner->GetName()); 140 | RHIUpdateTextureReference(Owner->TextureReference.TextureReferenceRHI, TextureRHI); 141 | } 142 | 143 | virtual void ReleaseRHI() override 144 | { 145 | RHIUpdateTextureReference(Owner->TextureReference.TextureReferenceRHI, nullptr); 146 | CameraImageHandle = nullptr; 147 | CopyTextureRef.SafeRelease(); 148 | DecodedTextureRef.SafeRelease(); 149 | FTextureResource::ReleaseRHI(); 150 | } 151 | 152 | /** Returns the width of the texture in pixels. */ 153 | virtual uint32 GetSizeX() const override 154 | { 155 | return Size.X; 156 | } 157 | 158 | /** Returns the height of the texture in pixels. */ 159 | virtual uint32 GetSizeY() const override 160 | { 161 | return Size.Y; 162 | } 163 | 164 | /** Render thread update of the texture so we don't get 2 updates per frame on the render thread */ 165 | void Init_RenderThread(std::shared_ptr handle) 166 | { 167 | check(IsInRenderingThread()); 168 | if (LastFrameNumber != GFrameNumber) 169 | { 170 | LastFrameNumber = GFrameNumber; 171 | ReleaseRHI(); 172 | CameraImageHandle = handle; 173 | InitRHI(); 174 | } 175 | } 176 | 177 | private: 178 | /** Copy CameraImage to our CopyTextureRef using the GPU */ 179 | bool PerformCopy(const TComPtr& texture, const TComPtr& context) 180 | { 181 | // These must already be prepped 182 | if (texture == nullptr 183 | || context == nullptr 184 | || !CopyTextureRef.IsValid()) 185 | { 186 | return false; 187 | } 188 | // Get the underlying interface for the texture we are copying to 189 | TComPtr CopyTexture = reinterpret_cast(CopyTextureRef->GetNativeResource()); 190 | if (CopyTexture == nullptr) 191 | { 192 | return false; 193 | } 194 | 195 | context->CopyResource(CopyTexture.Get(), texture); 196 | 197 | return true; 198 | } 199 | 200 | /** Runs a shader to convert YUV to RGB */ 201 | void PerformConversion() 202 | { 203 | FRHICommandListImmediate& CommandList = FRHICommandListExecutor::GetImmediateCommandList(); 204 | SCOPED_DRAW_EVENT(CommandList, HoloLensCameraImageConversion); 205 | 206 | FGraphicsPipelineStateInitializer GraphicsPSOInit; 207 | FRHITexture* RenderTarget = DecodedTextureRef.GetReference(); 208 | CommandList.Transition(FRHITransitionInfo(RenderTarget, ERHIAccess::Unknown, ERHIAccess::RTV)); 209 | 210 | FIntPoint OutputDim(RenderTarget->GetSizeXYZ().X, RenderTarget->GetSizeXYZ().Y); 211 | 212 | FRHIRenderPassInfo RPInfo(RenderTarget, ERenderTargetActions::DontLoad_Store); 213 | CommandList.BeginRenderPass(RPInfo, TEXT("HoloLensCameraImageConversion")); 214 | { 215 | CommandList.ApplyCachedRenderTargets(GraphicsPSOInit); 216 | CommandList.SetViewport(0, 0, 0.0f, OutputDim.X, OutputDim.Y, 1.0f); 217 | 218 | GraphicsPSOInit.DepthStencilState = TStaticDepthStencilState::GetRHI(); 219 | GraphicsPSOInit.RasterizerState = TStaticRasterizerState<>::GetRHI(); 220 | GraphicsPSOInit.BlendState = TStaticBlendStateWriteMask::GetRHI(); 221 | GraphicsPSOInit.PrimitiveType = PT_TriangleStrip; 222 | 223 | // configure media shaders 224 | auto ShaderMap = GetGlobalShaderMap(GMaxRHIFeatureLevel); 225 | TShaderMapRef VertexShader(ShaderMap); 226 | 227 | GraphicsPSOInit.BoundShaderState.VertexDeclarationRHI = GMediaVertexDeclaration.VertexDeclarationRHI; 228 | GraphicsPSOInit.BoundShaderState.VertexShaderRHI = VertexShader.GetVertexShader(); 229 | 230 | // Use the sample format to choose the conversion path 231 | TShaderMapRef ConvertShader(ShaderMap); 232 | GraphicsPSOInit.BoundShaderState.PixelShaderRHI = ConvertShader.GetPixelShader(); 233 | SetGraphicsPipelineState(CommandList, GraphicsPSOInit); 234 | 235 | FShaderResourceViewRHIRef Y_SRV = RHICreateShaderResourceView(CopyTextureRef, 0, 1, PF_G8); 236 | FShaderResourceViewRHIRef UV_SRV = RHICreateShaderResourceView(CopyTextureRef, 0, 1, PF_R8G8); 237 | 238 | ConvertShader->SetParameters(CommandList, CopyTextureRef->GetSizeXY(), Y_SRV, UV_SRV, OutputDim, MediaShaders::YuvToSrgbDefault, MediaShaders::YUVOffset8bits, false); 239 | 240 | // draw full size quad into render target 241 | FVertexBufferRHIRef VertexBuffer = CreateTempMediaVertexBuffer(); 242 | CommandList.SetStreamSource(0, VertexBuffer, 0); 243 | // set viewport to RT size 244 | CommandList.SetViewport(0, 0, 0.0f, OutputDim.X, OutputDim.Y, 1.0f); 245 | 246 | CommandList.DrawPrimitive(0, 2, 1); 247 | } 248 | CommandList.EndRenderPass(); 249 | CommandList.Transition(FRHITransitionInfo(RenderTarget, ERHIAccess::RTV, ERHIAccess::SRVGraphics)); 250 | 251 | } 252 | 253 | /** The size we get from the incoming camera image */ 254 | FIntPoint Size; 255 | 256 | /** The raw camera image from the HoloLens which we copy to our texture to allow it to be quickly released */ 257 | std::shared_ptr CameraImageHandle; 258 | /** The nv12 texture that we copy into so we don't block the camera from being able to send frames */ 259 | FTexture2DRHIRef CopyTextureRef; 260 | /** The texture that we actually render with which is populated via a shader that converts nv12 to rgba */ 261 | FTexture2DRHIRef DecodedTextureRef; 262 | /** The last frame we were updated on */ 263 | uint32 LastFrameNumber; 264 | 265 | const UOpenXRCameraImageTexture* Owner; 266 | }; 267 | 268 | #endif 269 | 270 | 271 | UOpenXRCameraImageTexture::UOpenXRCameraImageTexture(const FObjectInitializer& ObjectInitializer) 272 | : Super(ObjectInitializer) 273 | , LastUpdateFrame(0) 274 | { 275 | FString RHIString = FApp::GetGraphicsRHI(); 276 | if (!RHIString.IsEmpty()) 277 | { 278 | IsDX11 = RHIString == TEXT("DirectX 11"); 279 | } 280 | } 281 | 282 | void UOpenXRCameraImageTexture::BeginDestroy() 283 | { 284 | Super::BeginDestroy(); 285 | } 286 | 287 | FTextureResource* UOpenXRCameraImageTexture::CreateResource() 288 | { 289 | // 4.26 does not support DX12, do not attempt to create a resource if we are a DX12 app. 290 | if (!IsDX11) 291 | { 292 | return nullptr; 293 | } 294 | 295 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 296 | return new FOpenXRCameraImageResource(this); 297 | #else 298 | return nullptr; 299 | #endif 300 | } 301 | 302 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 303 | /** Forces the reconstruction of the texture data and conversion from Nv12 to RGB */ 304 | void UOpenXRCameraImageTexture::Init(std::shared_ptr handle) 305 | { 306 | // It's possible that we get more than one queued thread update per game frame 307 | // Skip any additional frames because it will cause the recursive flush rendering commands ensure 308 | if (LastUpdateFrame != GFrameCounter) 309 | { 310 | LastUpdateFrame = GFrameCounter; 311 | if (Resource != nullptr) 312 | { 313 | FOpenXRCameraImageResource* LambdaResource = static_cast(Resource); 314 | ENQUEUE_RENDER_COMMAND(Init_RenderThread)( 315 | [LambdaResource, handle](FRHICommandListImmediate&) 316 | { 317 | LambdaResource->Init_RenderThread(handle); 318 | }); 319 | } 320 | else 321 | { 322 | // This should end up only being called once, the first time we get a texture update 323 | UpdateResource(); 324 | } 325 | } 326 | } 327 | #endif 328 | 329 | #endif // UE_VERSION_OLDER_THAN(4, 27, 0) -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/OpenXRCommon.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "OpenXRCommon.h" 5 | #include "CoreMinimal.h" 6 | #include "OpenXRCore.h" 7 | 8 | namespace MicrosoftOpenXR 9 | { 10 | XrPath GetXrPath(XrInstance Instance, const char* PathString) 11 | { 12 | XrPath Path = XR_NULL_PATH; 13 | XrResult Result = xrStringToPath(Instance, PathString, &Path); 14 | check(XR_SUCCEEDED(Result)); 15 | return Path; 16 | } 17 | } // namespace MicrosoftOpenXR 18 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/OpenXRCommon.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include 7 | 8 | #include "HAL/Platform.h" 9 | 10 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 11 | 12 | #pragma warning(disable : 5205 4265 4268 4946) 13 | 14 | #define XR_USE_PLATFORM_WIN32 1 15 | #include "Windows/AllowWindowsPlatformTypes.h" 16 | #include "Windows/AllowWindowsPlatformAtomics.h" 17 | #include "Windows/PreWindowsApi.h" 18 | 19 | #include 20 | #include 21 | #include 22 | #endif 23 | 24 | #include 25 | 26 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 27 | #include "Windows/PostWindowsApi.h" 28 | #include "Windows/HideWindowsPlatformAtomics.h" 29 | #include "Windows/HideWindowsPlatformTypes.h" 30 | #endif 31 | 32 | #include "IOpenXRHMDPlugin.h" 33 | #include "IOpenXRExtensionPlugin.h" 34 | 35 | namespace MicrosoftOpenXR 36 | { 37 | #if DO_CHECK 38 | // Modified from OpenXRCore's XR_ENSURE which uses engine OpenXR headers to get a result string. 39 | // Otherwise, this will fail to build when using newer OpenXR headers in this plugin. 40 | #define XR_ENSURE_MSFT(x) [] (XrResult Result) \ 41 | { \ 42 | return ensureMsgf(XR_SUCCEEDED(Result), TEXT("OpenXR call failed with result: %d"), Result); \ 43 | } (x) 44 | #else 45 | #define XR_ENSURE_MSFT(x) XR_SUCCEEDED(x) 46 | #endif 47 | 48 | XrPath GetXrPath(XrInstance Instance, const char* PathString); 49 | } 50 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/QRTrackingPlugin.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "QRTrackingPlugin.h" 5 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 6 | #include "OpenXRCore.h" 7 | #include "IOpenXRARTrackedGeometryHolder.h" 8 | #include "IXRTrackingSystem.h" 9 | 10 | #include "WindowsMixedRealityInteropUtility.h" 11 | 12 | #include 13 | 14 | #if WITH_EDITOR 15 | #include "Editor.h" 16 | #endif 17 | 18 | using namespace winrt::Windows::Foundation; 19 | using namespace winrt::Windows::Foundation::Collections; 20 | using namespace winrt::Windows::Perception::Spatial; 21 | using namespace winrt::Windows::Perception::Spatial::Surfaces; 22 | using namespace winrt::Microsoft::MixedReality::QR; 23 | 24 | namespace MicrosoftOpenXR 25 | { 26 | void FQRTrackingPlugin::Register() 27 | { 28 | IModularFeatures::Get().RegisterModularFeature(IOpenXRExtensionPlugin::GetModularFeatureName(), this); 29 | #if PLATFORM_WINDOWS 30 | const FString PluginBaseDir = IPluginManager::Get().FindPlugin("MicrosoftOpenXR")->GetBaseDir(); 31 | const FString DllName = "Microsoft.MixedReality.QR.dll"; 32 | const FString LibrariesDir = PluginBaseDir / THIRDPARTY_BINARY_SUBFOLDER; 33 | 34 | FPlatformProcess::PushDllDirectory(*LibrariesDir); 35 | if (!FPlatformProcess::GetDllHandle(*DllName)) 36 | { 37 | UE_LOG(LogHMD, Warning, TEXT("Dll \'%s\' can't be loaded from \'%s\'"), *DllName, *LibrariesDir); 38 | } 39 | FPlatformProcess::PopDllDirectory(*LibrariesDir); 40 | #endif 41 | 42 | #if WITH_EDITOR 43 | // When PIE stops (when remoting), ShutdownModule is not called. 44 | // This will prevent QR from working on subsequent plays. 45 | FEditorDelegates::EndPIE.AddRaw(this, &FQRTrackingPlugin::HandleEndPIE); 46 | #endif 47 | } 48 | 49 | void FQRTrackingPlugin::HandleEndPIE(const bool InIsSimulating) 50 | { 51 | Unregister(); 52 | } 53 | 54 | void FQRTrackingPlugin::Unregister() 55 | { 56 | IModularFeatures::Get().UnregisterModularFeature(IOpenXRExtensionPlugin::GetModularFeatureName(), this); 57 | StopQRCodeWatcher(); 58 | } 59 | 60 | bool FQRTrackingPlugin::GetRequiredExtensions(TArray& OutExtensions) 61 | { 62 | OutExtensions.Add(XR_MSFT_SPATIAL_GRAPH_BRIDGE_EXTENSION_NAME); 63 | return true; 64 | } 65 | 66 | 67 | const void* FQRTrackingPlugin::OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) 68 | { 69 | XR_ENSURE_MSFT(xrGetInstanceProcAddr(InInstance, "xrCreateSpatialGraphNodeSpaceMSFT", (PFN_xrVoidFunction*)&xrCreateSpatialGraphNodeSpaceMSFT)); 70 | 71 | QRCodeHolder = IModularFeatures::Get().GetModularFeatureImplementations(IOpenXRARTrackedGeometryHolder::GetModularFeatureName())[0]; 72 | 73 | ensure(QRCodeHolder != nullptr); 74 | 75 | static FName SystemName(TEXT("OpenXR")); 76 | if (GEngine->XRSystem.IsValid() && (GEngine->XRSystem->GetSystemName() == SystemName)) 77 | { 78 | XRTrackingSystem = GEngine->XRSystem.Get(); 79 | } 80 | 81 | ensure(XRTrackingSystem != nullptr); 82 | 83 | return InNext; 84 | } 85 | 86 | bool FQRTrackingPlugin::OnToggleARCapture(const bool On) 87 | { 88 | if (On) 89 | { 90 | return StartQRCodeWatcher(); 91 | } 92 | else 93 | { 94 | StopQRCodeWatcher(); 95 | } 96 | return true; 97 | } 98 | 99 | bool FQRTrackingPlugin::IsEnabled() const 100 | { 101 | std::lock_guard lock(QRCodeRefsLock); 102 | return QRTrackerInstance != nullptr; 103 | } 104 | 105 | bool FQRTrackingPlugin::StartQRCodeWatcher() 106 | { 107 | std::lock_guard lock(QRCodeRefsLock); 108 | 109 | // Create the tracker and register the callbacks 110 | if (QRTrackerInstance == nullptr) 111 | { 112 | try 113 | { 114 | if (!QRCodeWatcher::IsSupported()) 115 | { 116 | UE_LOG(LogHMD, Log, TEXT("QRCodeWatcher is not supported.")); 117 | return false; 118 | } 119 | } 120 | catch (winrt::hresult_error e) 121 | { 122 | UE_LOG(LogHMD, Error, TEXT("QRCodeWatcher::IsSupported failed with error: %d"), e.code().value); 123 | return false; 124 | } 125 | 126 | m_QRTrackerAsyncOperation = QRCodeWatcher::RequestAccessAsync(); 127 | m_QRTrackerAsyncOperation.Completed([=](auto&& asyncInfo, auto&& asyncStatus) 128 | { 129 | if (asyncStatus == winrt::Windows::Foundation::AsyncStatus::Completed) 130 | { 131 | if (asyncInfo.GetResults() == QRCodeWatcherAccessStatus::Allowed) 132 | { 133 | std::lock_guard lock(QRCodeRefsLock); 134 | 135 | QRTrackerInstance = QRCodeWatcher(); 136 | OnAddedEventToken = QRTrackerInstance.Added(winrt::auto_revoke, [=](auto&& sender, auto&& args) { OnAdded(sender, args); }); 137 | OnUpdatedEventToken = QRTrackerInstance.Updated(winrt::auto_revoke, [=](auto&& sender, auto&& args) { OnUpdated(sender, args); }); 138 | OnRemovedEventToken = QRTrackerInstance.Removed(winrt::auto_revoke, [=](auto&& sender, auto&& args) { OnRemoved(sender, args); }); 139 | OnEnumerationCompletedToken = QRTrackerInstance.EnumerationCompleted(winrt::auto_revoke, [=](auto&& sender, auto&& args) { OnEnumerationCompleted(sender, args); }); 140 | 141 | // Start the tracker 142 | QRTrackerInstance.Start(); 143 | 144 | m_QRTrackerAsyncOperation = nullptr; 145 | } 146 | else 147 | { 148 | UE_LOG(LogHMD, Log, TEXT("QRTracker access request returns error: %d"), asyncInfo.GetResults()); 149 | } 150 | } 151 | }); 152 | } 153 | return true; 154 | } 155 | 156 | void FQRTrackingPlugin::StopQRCodeWatcher() 157 | { 158 | std::lock_guard lock(QRCodeRefsLock); 159 | if (m_QRTrackerAsyncOperation != nullptr && m_QRTrackerAsyncOperation.Status() != winrt::Windows::Foundation::AsyncStatus::Completed) 160 | { 161 | m_QRTrackerAsyncOperation.Cancel(); 162 | } 163 | 164 | if (QRTrackerInstance != nullptr) 165 | { 166 | OnAddedEventToken.revoke(); 167 | OnUpdatedEventToken.revoke(); 168 | OnRemovedEventToken.revoke(); 169 | OnEnumerationCompletedToken.revoke(); 170 | 171 | // Stop the tracker 172 | QRTrackerInstance.Stop(); 173 | QRTrackerInstance = nullptr; 174 | } 175 | 176 | { 177 | FScopeLock Lock(&QRCodeContextsMutex); 178 | QRCodeContexts.Empty(); 179 | } 180 | } 181 | 182 | void FQRTrackingPlugin::OnAdded(QRCodeWatcher sender, QRCodeAddedEventArgs args) 183 | { 184 | #if !UE_VERSION_OLDER_THAN(4, 27, 0) 185 | auto QRCode = MakeShared(); 186 | #else 187 | auto QRCode = new FOpenXRQRCodeData(); 188 | #endif 189 | 190 | auto InCode = args.Code(); 191 | 192 | QRCode->Id = WMRUtility::GUIDToFGuid(InCode.Id()); 193 | QRCode->Version = (int32_t)InCode.Version(); 194 | QRCode->QRCode = InCode.Data().c_str(); 195 | QRCode->Size = FVector2D(InCode.PhysicalSideLength()) * XRTrackingSystem->GetWorldToMetersScale(); 196 | QRCode->Timestamp = FPlatformTime::Seconds(); 197 | QRCode->TrackingState = EARTrackingState::NotTracking; 198 | QRCode->LocalToTrackingTransform = FTransform::Identity; //OnAdded returns no actual pose 199 | 200 | auto Context = MakeShared(); 201 | Context->SpatialGraphNodeId = WMRUtility::GUIDToFGuid(InCode.SpatialGraphNodeId()); 202 | 203 | { 204 | FScopeLock Lock(&QRCodeContextsMutex); 205 | QRCodeContexts.FindOrAdd(QRCode->Id) = Context; 206 | } 207 | 208 | QRCodeHolder->ARTrackedGeometryAdded(QRCode); 209 | } 210 | 211 | void FQRTrackingPlugin::OnUpdated(QRCodeWatcher sender, QRCodeUpdatedEventArgs args) 212 | { 213 | // Mark the tracked QRCode as updated so it will be located in UpdateDeviceLocations. 214 | std::lock_guard lock(QRCodeRefsLock); 215 | if (QRTrackerInstance == nullptr) { return; } 216 | 217 | FScopeLock Lock(&QRCodeContextsMutex); 218 | FGuid Guid = WMRUtility::GUIDToFGuid(args.Code().Id()); 219 | if (QRCodeContexts.Contains(Guid)) 220 | { 221 | QRCodeContextPtr& Context = QRCodeContexts[Guid]; 222 | Context->HasChanged = true; 223 | } 224 | } 225 | 226 | void FQRTrackingPlugin::OnRemoved(QRCodeWatcher sender, QRCodeRemovedEventArgs args) 227 | { 228 | #if !UE_VERSION_OLDER_THAN(4, 27, 0) 229 | auto QRCode = MakeShared(); 230 | #else 231 | auto QRCode = new FOpenXRQRCodeData(); 232 | #endif 233 | 234 | auto InCode = args.Code(); 235 | 236 | QRCode->Id = WMRUtility::GUIDToFGuid(InCode.Id()); 237 | QRCode->Timestamp = FPlatformTime::Seconds(); 238 | 239 | { 240 | FScopeLock Lock(&QRCodeContextsMutex); 241 | QRCodeContexts.Remove(QRCode->Id); 242 | } 243 | 244 | 245 | QRCodeHolder->ARTrackedGeometryRemoved(QRCode); 246 | } 247 | 248 | void FQRTrackingPlugin::OnEnumerationCompleted(QRCodeWatcher sender, winrt::Windows::Foundation::IInspectable args) 249 | { 250 | 251 | } 252 | 253 | void FQRTrackingPlugin::UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) 254 | { 255 | std::lock_guard lock(QRCodeRefsLock); 256 | if (QRTrackerInstance == nullptr) { return; } 257 | 258 | for(auto&& InCode : QRTrackerInstance.GetList()) 259 | { 260 | FGuid OutGuid = WMRUtility::GUIDToFGuid(InCode.Id()); 261 | QRCodeContextPtr Context; 262 | { 263 | FScopeLock Lock(&QRCodeContextsMutex); 264 | if (auto PtrPtr = QRCodeContexts.Find(OutGuid)) 265 | { 266 | Context = *PtrPtr; 267 | } 268 | else 269 | { 270 | continue; //QRCode is being deleted 271 | } 272 | } 273 | 274 | if (!Context->HasChanged) 275 | { 276 | // Only update QR Codes that have updated. 277 | continue; 278 | } 279 | Context->HasChanged = false; 280 | 281 | #if !UE_VERSION_OLDER_THAN(4, 27, 0) 282 | auto OutCode = MakeShared(); 283 | #else 284 | auto OutCode = new FOpenXRQRCodeData(); 285 | #endif 286 | 287 | OutCode->Id = OutGuid; 288 | OutCode->Version = (int32_t)InCode.Version(); 289 | OutCode->QRCode = InCode.Data().c_str(); 290 | OutCode->Size = FVector2D(InCode.PhysicalSideLength()) * XRTrackingSystem->GetWorldToMetersScale(); 291 | OutCode->Timestamp = FPlatformTime::Seconds(); 292 | 293 | if (Context->Space == XR_NULL_HANDLE) 294 | { 295 | XrSpatialGraphNodeSpaceCreateInfoMSFT SpatialGraphNodeSpaceCreateInfo{ XR_TYPE_SPATIAL_GRAPH_NODE_SPACE_CREATE_INFO_MSFT }; 296 | SpatialGraphNodeSpaceCreateInfo.nodeType = XR_SPATIAL_GRAPH_NODE_TYPE_STATIC_MSFT; 297 | SpatialGraphNodeSpaceCreateInfo.pose = ToXrPose(FTransform::Identity, XRTrackingSystem->GetWorldToMetersScale()); 298 | 299 | check(sizeof(SpatialGraphNodeSpaceCreateInfo.nodeId) == sizeof(winrt::guid)); 300 | winrt::guid SourceGuid = WMRUtility::FGUIDToGuid(Context->SpatialGraphNodeId); 301 | FMemory::Memcpy(&SpatialGraphNodeSpaceCreateInfo.nodeId, &SourceGuid, sizeof(SpatialGraphNodeSpaceCreateInfo.nodeId)); 302 | 303 | XR_ENSURE_MSFT(xrCreateSpatialGraphNodeSpaceMSFT(InSession, &SpatialGraphNodeSpaceCreateInfo, &Context->Space)); 304 | } 305 | 306 | XrSpaceLocation SpaceLocation{ XR_TYPE_SPACE_LOCATION }; 307 | XR_ENSURE_MSFT(xrLocateSpace(Context->Space, TrackingSpace, DisplayTime, &SpaceLocation)); 308 | const XrSpaceLocationFlags ValidFlags = XR_SPACE_LOCATION_ORIENTATION_VALID_BIT | XR_SPACE_LOCATION_POSITION_VALID_BIT; 309 | if ((SpaceLocation.locationFlags & ValidFlags) == ValidFlags) 310 | { 311 | OutCode->LocalToTrackingTransform = ToFTransform(SpaceLocation.pose, XRTrackingSystem->GetWorldToMetersScale()); 312 | OutCode->TrackingState = EARTrackingState::Tracking; 313 | } 314 | else 315 | { 316 | OutCode->TrackingState = EARTrackingState::NotTracking; 317 | } 318 | 319 | QRCodeHolder->ARTrackedGeometryUpdated(OutCode); 320 | } 321 | } 322 | 323 | IOpenXRCustomCaptureSupport* FQRTrackingPlugin::GetCustomCaptureSupport(const EARCaptureType CaptureType) 324 | { 325 | if (CaptureType == EARCaptureType::QRCode) 326 | { 327 | return this; 328 | } 329 | return nullptr; 330 | } 331 | 332 | 333 | FQRTrackingPlugin::QRCodeContext::~QRCodeContext() 334 | { 335 | if (Space != XR_NULL_HANDLE) 336 | { 337 | xrDestroySpace(Space); 338 | } 339 | } 340 | 341 | } // namespace MicrosoftOpenXR 342 | 343 | #endif //PLATFORM_WINDOWS || PLATFORM_HOLOLENS 344 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/QRTrackingPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 7 | #include "Interfaces/IPluginManager.h" 8 | 9 | #include "OpenXRCommon.h" 10 | #include "ARTypes.h" 11 | 12 | #include "Windows/AllowWindowsPlatformTypes.h" 13 | #include "Windows/AllowWindowsPlatformAtomics.h" 14 | #include "Windows/PreWindowsApi.h" 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | 24 | #include "Windows/PostWindowsApi.h" 25 | #include "Windows/HideWindowsPlatformAtomics.h" 26 | #include "Windows/HideWindowsPlatformTypes.h" 27 | 28 | #include "Misc/EngineVersionComparison.h" 29 | 30 | class IOpenXRARTrackedGeometryHolder; 31 | struct FOpenXRQRCodeData; 32 | 33 | 34 | namespace MicrosoftOpenXR 35 | { 36 | class FQRTrackingPlugin : public IOpenXRExtensionPlugin, public IOpenXRCustomCaptureSupport 37 | { 38 | public: 39 | void Register(); 40 | void Unregister(); 41 | 42 | virtual bool GetRequiredExtensions(TArray& OutExtensions) override; 43 | 44 | virtual const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 45 | virtual void UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) override; 46 | 47 | virtual IOpenXRCustomCaptureSupport* GetCustomCaptureSupport(const EARCaptureType CaptureType) override; 48 | 49 | bool OnToggleARCapture(const bool bOnOff) override; 50 | bool IsEnabled() const; 51 | private: 52 | 53 | PFN_xrCreateSpatialGraphNodeSpaceMSFT xrCreateSpatialGraphNodeSpaceMSFT; 54 | 55 | bool StartQRCodeWatcher(); 56 | void StopQRCodeWatcher(); 57 | 58 | void HandleEndPIE(const bool InIsSimulating); 59 | 60 | // WinRT handlers 61 | void OnAdded(winrt::Microsoft::MixedReality::QR::QRCodeWatcher sender, winrt::Microsoft::MixedReality::QR::QRCodeAddedEventArgs args); 62 | void OnUpdated(winrt::Microsoft::MixedReality::QR::QRCodeWatcher sender, winrt::Microsoft::MixedReality::QR::QRCodeUpdatedEventArgs args); 63 | void OnRemoved(winrt::Microsoft::MixedReality::QR::QRCodeWatcher sender, winrt::Microsoft::MixedReality::QR::QRCodeRemovedEventArgs args); 64 | void OnEnumerationCompleted(winrt::Microsoft::MixedReality::QR::QRCodeWatcher sender, winrt::Windows::Foundation::IInspectable args); 65 | 66 | winrt::Microsoft::MixedReality::QR::QRCodeWatcher QRTrackerInstance = nullptr; 67 | winrt::Windows::Foundation::IAsyncOperation < winrt::Microsoft::MixedReality::QR::QRCodeWatcherAccessStatus > m_QRTrackerAsyncOperation = nullptr; 68 | 69 | winrt::Microsoft::MixedReality::QR::QRCodeWatcher::Added_revoker OnAddedEventToken; 70 | winrt::Microsoft::MixedReality::QR::QRCodeWatcher::Updated_revoker OnUpdatedEventToken; 71 | winrt::Microsoft::MixedReality::QR::QRCodeWatcher::Removed_revoker OnRemovedEventToken; 72 | winrt::Microsoft::MixedReality::QR::QRCodeWatcher::EnumerationCompleted_revoker OnEnumerationCompletedToken; 73 | 74 | mutable std::recursive_mutex QRCodeRefsLock; 75 | 76 | struct QRCodeContext 77 | { 78 | EARTrackingState TrackingState = EARTrackingState::Unknown; 79 | FGuid SpatialGraphNodeId; 80 | XrSpace Space = XR_NULL_HANDLE; 81 | bool HasChanged = false; 82 | 83 | ~QRCodeContext(); 84 | }; 85 | 86 | typedef TSharedPtr QRCodeContextPtr; 87 | 88 | IOpenXRARTrackedGeometryHolder* QRCodeHolder; 89 | 90 | class IXRTrackingSystem* XRTrackingSystem = nullptr; 91 | 92 | TMap QRCodeContexts; 93 | FCriticalSection QRCodeContextsMutex; 94 | }; 95 | } // namespace MicrosoftOpenXR 96 | 97 | #endif //PLATFORM_WINDOWS || PLATFORM_HOLOLENS 98 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SceneUnderstandingBase.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "ARBlueprintLibrary.h" 7 | #include "Engine.h" 8 | #include "IOpenXRARModule.h" 9 | #include "IOpenXRARTrackedGeometryHolder.h" 10 | #include "IXRTrackingSystem.h" 11 | #include "MicrosoftOpenXR.h" 12 | #include "Misc/EngineVersionComparison.h" 13 | #include "OpenXRCore.h" 14 | #include "SceneUnderstandingUtility.h" 15 | #include "TrackedGeometryCollision.h" 16 | #include "UniqueHandle.h" 17 | 18 | #if SUPPORTS_REMOTING 19 | #include "openxr_msft_holographic_remoting.h" 20 | #endif 21 | 22 | class IOpenXRARTrackedMeshHolder; 23 | 24 | namespace MicrosoftOpenXR 25 | { 26 | enum class EScanState 27 | { 28 | Idle, 29 | Waiting, 30 | Processing, 31 | AddMeshesToScene, 32 | Locating 33 | }; 34 | 35 | struct FPlaneData 36 | { 37 | FGuid MeshGuid; 38 | }; 39 | 40 | struct FPlaneUpdate 41 | { 42 | FGuid MeshGuid; 43 | EARObjectClassification Type; 44 | FVector Extent; 45 | TArray Vertices; 46 | TArray Indices; 47 | }; 48 | 49 | struct FSceneUpdate 50 | { 51 | FSceneHandle Scene; 52 | TMap Planes; 53 | TArray PlaneUuids; 54 | TMap PlaneCollisionInfo; 55 | TMap MeshCollisionInfo; 56 | }; 57 | 58 | class FSceneUnderstandingBase : public IOpenXRExtensionPlugin, public IOpenXRCustomCaptureSupport 59 | { 60 | public: 61 | void Register(); 62 | void Unregister(); 63 | 64 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 65 | const void* OnBeginSession(XrSession InSession, const void* InNext) override; 66 | 67 | bool GetRequiredExtensions(TArray& OutExtensions) override; 68 | bool OnToggleARCapture(const bool bOnOff) override; 69 | 70 | IOpenXRCustomCaptureSupport* GetCustomCaptureSupport(const EARCaptureType CaptureType) override = 0; 71 | void OnStartARSession(class UARSessionConfig* SessionConfig) override; 72 | 73 | void UpdateDeviceLocations(XrSession InSession, XrTime DisplayTime, XrSpace TrackingSpace) override; 74 | 75 | TArray OnLineTraceTrackedObjects( 76 | const TSharedPtr ARCompositionComponent, const FVector Start, 77 | const FVector End, const EARLineTraceChannels TraceChannels) override; 78 | 79 | bool CanDetectPlanes(); 80 | 81 | protected: 82 | virtual XrSceneComputeConsistencyMSFT GetSceneComputeConsistency() = 0; 83 | virtual TArray GetSceneComputeFeatures(class UARSessionConfig* SessionConfig) = 0; 84 | 85 | private: 86 | void UpdateObjectLocations(XrTime DisplayTime, XrSpace TrackingSpace); 87 | 88 | void ProcessSceneUpdate(FSceneUpdate&& SceneUpdate, XrTime DisplayTime, XrSpace TrackingSpace); 89 | 90 | void HandleEndPIE(const bool InIsSimulating); 91 | void Stop(); 92 | 93 | void ComputeNewScene(XrTime DisplayTime); 94 | 95 | ExtensionDispatchTable Ext{}; 96 | 97 | FSceneObserverHandle SceneObserver; 98 | FSceneHandle LocatingScene; 99 | FSpaceHandle ViewSpace; 100 | EScanState ScanState{ EScanState::Idle }; 101 | 102 | TArray ComputeFeatures; 103 | TArray PlaneAlignmentFilters; 104 | 105 | // Members for reading scene components 106 | TArray UuidsToLocate; 107 | TMap Planes; 108 | TArray Locations; 109 | TMap PreviousPlanes; 110 | int UuidToLocateThisFrame = 0; 111 | int UuidsToLocatePerFrame = 5; 112 | 113 | TMap PlaneCollisionInfo; 114 | TMap MeshCollisionInfo; 115 | 116 | TFuture> SceneUpdateFuture; 117 | 118 | class IXRTrackingSystem* XRTrackingSystem = nullptr; 119 | IOpenXRARTrackedMeshHolder* TrackedMeshHolder = nullptr; 120 | XrNewSceneComputeInfoMSFT SceneComputeInfo{ XR_TYPE_NEW_SCENE_COMPUTE_INFO_MSFT }; 121 | XrSceneOrientedBoxBoundMSFT SceneBox; 122 | XrSceneSphereBoundMSFT SceneSphere; 123 | float SphereBoundRadius = 10.0f; // meters 124 | float BoundHeight = 0.0f; // meters 125 | bool bShouldStartSceneUnderstanding = false; 126 | bool bARSessionStarted = false; 127 | bool bCanDetectPlanes = false; 128 | }; 129 | } 130 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SceneUnderstandingPlugin.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "SceneUnderstandingPlugin.h" 5 | 6 | namespace MicrosoftOpenXR 7 | { 8 | FSceneUnderstandingPlugin::FSceneUnderstandingPlugin() 9 | { 10 | } 11 | 12 | bool FSceneUnderstandingPlugin::GetRequiredExtensions(TArray& OutExtensions) 13 | { 14 | #if UE_VERSION_OLDER_THAN(4, 27, 1) 15 | // Scene Understanding requires engine fixes introduced in 4.27.1 16 | UE_LOG(LogHMD, Warning, TEXT("Scene Understanding requires at least 4.27.1.")); 17 | return false; 18 | #endif 19 | 20 | return FSceneUnderstandingBase::GetRequiredExtensions(OutExtensions); 21 | } 22 | 23 | IOpenXRCustomCaptureSupport* FSceneUnderstandingPlugin::GetCustomCaptureSupport(const EARCaptureType CaptureType) 24 | { 25 | return CaptureType == EARCaptureType::SceneUnderstanding ? this : nullptr; 26 | } 27 | 28 | bool FSceneUnderstandingPlugin::OnToggleARCapture(const bool bOnOff) 29 | { 30 | if (bOnOff && !CanDetectPlanes()) 31 | { 32 | UE_LOG(LogHMD, Warning, TEXT("Attempting to use scene understanding in a runtime that does not support plane finding. Use spatial mapping instead.")); 33 | return false; 34 | } 35 | 36 | return FSceneUnderstandingBase::OnToggleARCapture(bOnOff); 37 | } 38 | 39 | XrSceneComputeConsistencyMSFT FSceneUnderstandingPlugin::GetSceneComputeConsistency() 40 | { 41 | return XR_SCENE_COMPUTE_CONSISTENCY_SNAPSHOT_COMPLETE_MSFT; 42 | } 43 | 44 | TArray FSceneUnderstandingPlugin::GetSceneComputeFeatures(class UARSessionConfig* SessionConfig) 45 | { 46 | bool bGenerateSceneMeshData = false; 47 | GConfig->GetBool(TEXT("/Script/HoloLensSettings.SceneUnderstanding"), TEXT("ShouldDoSceneUnderstandingMeshDetection"), 48 | bGenerateSceneMeshData, GGameIni); 49 | 50 | //TODO: Restore this block when the session config exposes this flag (UE-126562). Update version when known. 51 | //#if !UE_VERSION_OLDER_THAN(4, 27, 2) 52 | //if (!bGenerateSceneMeshData) 53 | //{ 54 | // // Game ini does not have mesh detection flag set, check session config 55 | // bGenerateSceneMeshData = SessionConfig->ShouldDoSceneUnderstandingMeshDetection(); 56 | //} 57 | //#endif 58 | 59 | TArray SceneComputeFeatures; 60 | SceneComputeFeatures.AddUnique(XR_SCENE_COMPUTE_FEATURE_PLANE_MSFT); 61 | 62 | if (bGenerateSceneMeshData) 63 | { 64 | SceneComputeFeatures.AddUnique(XR_SCENE_COMPUTE_FEATURE_PLANE_MESH_MSFT); 65 | } 66 | 67 | return SceneComputeFeatures; 68 | } 69 | 70 | } // namespace MicrosoftOpenXR 71 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SceneUnderstandingPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | #include "SceneUnderstandingBase.h" 8 | 9 | namespace MicrosoftOpenXR 10 | { 11 | class FSceneUnderstandingPlugin : public FSceneUnderstandingBase 12 | { 13 | public: 14 | FSceneUnderstandingPlugin(); 15 | 16 | bool GetRequiredExtensions(TArray& OutExtensions) override; 17 | 18 | IOpenXRCustomCaptureSupport* GetCustomCaptureSupport(const EARCaptureType CaptureType) override; 19 | 20 | bool OnToggleARCapture(const bool bOnOff) override; 21 | 22 | protected: 23 | XrSceneComputeConsistencyMSFT GetSceneComputeConsistency() override; 24 | TArray GetSceneComputeFeatures(class UARSessionConfig* SessionConfig) override; 25 | }; 26 | } // namespace MicrosoftOpenXR 27 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SceneUnderstandingUtility.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | #include "UniqueHandle.h" 8 | 9 | inline bool operator==(const XrUuidMSFT& Lh, const XrUuidMSFT& Rh) noexcept 10 | { 11 | return memcmp(&Rh, &Lh, sizeof(XrUuidMSFT)) == 0; 12 | } 13 | 14 | inline bool operator!=(const XrUuidMSFT& Lh, const XrUuidMSFT& Rh) noexcept 15 | { 16 | return !(Lh == Rh); 17 | } 18 | 19 | // Allows XrUuidMSFT to be used as a key in TMap 20 | inline uint32 GetTypeHash(const XrUuidMSFT& Uuid) 21 | { 22 | static_assert(sizeof(XrUuidMSFT) == sizeof(uint64) * 2); 23 | const uint64* V = reinterpret_cast(Uuid.bytes); 24 | return HashCombine(GetTypeHash(V[0]), GetTypeHash(V[1])); 25 | } 26 | 27 | namespace MicrosoftOpenXR 28 | { 29 | struct ExtensionDispatchTable 30 | { 31 | PFN_xrEnumerateSceneComputeFeaturesMSFT xrEnumerateSceneComputeFeaturesMSFT; 32 | PFN_xrCreateSceneObserverMSFT xrCreateSceneObserverMSFT; 33 | PFN_xrDestroySceneObserverMSFT xrDestroySceneObserverMSFT; 34 | PFN_xrCreateSceneMSFT xrCreateSceneMSFT; 35 | PFN_xrDestroySceneMSFT xrDestroySceneMSFT; 36 | PFN_xrComputeNewSceneMSFT xrComputeNewSceneMSFT; 37 | PFN_xrGetSceneComputeStateMSFT xrGetSceneComputeStateMSFT; 38 | PFN_xrGetSceneComponentsMSFT xrGetSceneComponentsMSFT; 39 | PFN_xrLocateSceneComponentsMSFT xrLocateSceneComponentsMSFT; 40 | PFN_xrGetSceneMeshBuffersMSFT xrGetSceneMeshBuffersMSFT; 41 | }; 42 | 43 | // Defines a move-only smart handle for XrSceneObserverMSFT. 44 | class FSceneObserverHandle : public MicrosoftOpenXR::TUniqueExtHandle 45 | { 46 | }; 47 | // Defines a move-only smart handle for XrSceneMSFT. 48 | class FSceneHandle : public MicrosoftOpenXR::TUniqueExtHandle 49 | { 50 | }; 51 | 52 | inline constexpr bool IsPoseValid(XrSpaceLocationFlags LocationFlags) 53 | { 54 | constexpr XrSpaceLocationFlags PoseValidFlags = 55 | XR_SPACE_LOCATION_POSITION_VALID_BIT | XR_SPACE_LOCATION_ORIENTATION_VALID_BIT; 56 | return (LocationFlags & PoseValidFlags) == PoseValidFlags; 57 | } 58 | 59 | inline FSceneObserverHandle CreateSceneObserver(const ExtensionDispatchTable& Ext, XrSession SessionHandle) 60 | { 61 | FSceneObserverHandle UniqueSceneObserverHandle; 62 | XrSceneObserverCreateInfoMSFT CreateInfo{XR_TYPE_SCENE_OBSERVER_CREATE_INFO_MSFT}; 63 | XR_ENSURE(Ext.xrCreateSceneObserverMSFT( 64 | SessionHandle, &CreateInfo, UniqueSceneObserverHandle.Put(Ext.xrDestroySceneObserverMSFT))); 65 | return UniqueSceneObserverHandle; 66 | } 67 | 68 | inline FSceneHandle CreateScene(const ExtensionDispatchTable& Ext, XrSceneObserverMSFT SceneObserverHandle) 69 | { 70 | FSceneHandle UniqueSceneHandle; 71 | XrSceneCreateInfoMSFT CreateInfo{XR_TYPE_SCENE_CREATE_INFO_MSFT}; 72 | XR_ENSURE(Ext.xrCreateSceneMSFT(SceneObserverHandle, &CreateInfo, UniqueSceneHandle.Put(Ext.xrDestroySceneMSFT))); 73 | return UniqueSceneHandle; 74 | } 75 | 76 | template 77 | void InsertExtensionStruct(XrStruct& BaseStruct, XrExtension& ExtensionStruct) 78 | { 79 | ExtensionStruct.next = BaseStruct.next; 80 | BaseStruct.next = &ExtensionStruct; 81 | } 82 | 83 | inline FGuid XrUuidMSFTToFGuid(const XrUuidMSFT& Uuid) 84 | { 85 | static_assert(sizeof(FGuid) == sizeof(XrUuidMSFT)); 86 | FGuid OutGuid; 87 | FMemory::Memcpy(&OutGuid, &Uuid, sizeof(FGuid)); 88 | return OutGuid; 89 | } 90 | 91 | // Maps the OpenXR ObjectType to Unreal's object classification. 92 | inline EARObjectClassification GetObjectClassification(XrSceneObjectTypeMSFT ObjectType) 93 | { 94 | switch (ObjectType) 95 | { 96 | case XR_SCENE_OBJECT_TYPE_INFERRED_MSFT: 97 | return EARObjectClassification::Unknown; 98 | case XR_SCENE_OBJECT_TYPE_UNCATEGORIZED_MSFT: 99 | return EARObjectClassification::Unknown; 100 | case XR_SCENE_OBJECT_TYPE_BACKGROUND_MSFT: 101 | return EARObjectClassification::Unknown; 102 | case XR_SCENE_OBJECT_TYPE_WALL_MSFT: 103 | return EARObjectClassification::Wall; 104 | case XR_SCENE_OBJECT_TYPE_FLOOR_MSFT: 105 | return EARObjectClassification::Floor; 106 | case XR_SCENE_OBJECT_TYPE_CEILING_MSFT: 107 | return EARObjectClassification::Ceiling; 108 | case XR_SCENE_OBJECT_TYPE_PLATFORM_MSFT: 109 | return EARObjectClassification::Table; 110 | default: 111 | return EARObjectClassification::NotApplicable; 112 | } 113 | } 114 | 115 | inline XrSceneObjectTypeMSFT GetObjectType(const TMap& ObjectTypeMap, const XrUuidMSFT& Uuid) 116 | { 117 | const auto* Value = ObjectTypeMap.Find(Uuid); 118 | return Value != nullptr ? *Value : XR_SCENE_OBJECT_TYPE_UNCATEGORIZED_MSFT; 119 | } 120 | 121 | // Maps a SceneObject's UUID to an ObjectType. 122 | inline TMap GetObjectTypeMap(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext) 123 | { 124 | XrSceneComponentsGetInfoMSFT GetInfo{XR_TYPE_SCENE_COMPONENTS_GET_INFO_MSFT}; 125 | GetInfo.componentType = XR_SCENE_COMPONENT_TYPE_OBJECT_MSFT; 126 | 127 | XrSceneComponentsMSFT SceneComponents{XR_TYPE_SCENE_COMPONENTS_MSFT}; 128 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 129 | const uint32_t Count = SceneComponents.componentCountOutput; 130 | 131 | TArray Components; 132 | Components.SetNum(Count); 133 | SceneComponents.componentCapacityInput = Count; 134 | SceneComponents.components = Components.GetData(); 135 | 136 | TArray Objects; 137 | Objects.SetNum(Count); 138 | XrSceneObjectsMSFT sceneObjects{XR_TYPE_SCENE_OBJECTS_MSFT}; 139 | sceneObjects.sceneObjectCount = Count; 140 | sceneObjects.sceneObjects = Objects.GetData(); 141 | InsertExtensionStruct(SceneComponents, sceneObjects); 142 | 143 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 144 | 145 | TMap Map; 146 | Map.Reserve(Count); 147 | for (uint32_t Index = 0; Index < Count; Index++) 148 | { 149 | Map.Add(Components[Index].id, Objects[Index].objectType); 150 | } 151 | return Map; 152 | } 153 | 154 | inline void GetSceneVisibleMeshes(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext, 155 | const TArray& PlaneAlignmentFilters, TArray& Components, 156 | TArray& Meshes) 157 | { 158 | XrSceneComponentsGetInfoMSFT GetInfo{ XR_TYPE_SCENE_COMPONENTS_GET_INFO_MSFT }; 159 | GetInfo.componentType = XR_SCENE_COMPONENT_TYPE_VISUAL_MESH_MSFT; 160 | XrScenePlaneAlignmentFilterInfoMSFT AlignmentFilter{ XR_TYPE_SCENE_PLANE_ALIGNMENT_FILTER_INFO_MSFT }; 161 | if (PlaneAlignmentFilters.Num() > 0) 162 | { 163 | AlignmentFilter.alignmentCount = static_cast(PlaneAlignmentFilters.Num()); 164 | AlignmentFilter.alignments = PlaneAlignmentFilters.GetData(); 165 | InsertExtensionStruct(GetInfo, AlignmentFilter); 166 | } 167 | XrSceneComponentsMSFT SceneComponents{ XR_TYPE_SCENE_COMPONENTS_MSFT }; 168 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 169 | const uint32_t Count = SceneComponents.componentCountOutput; 170 | Components.SetNum(Count); 171 | SceneComponents.componentCapacityInput = Count; 172 | SceneComponents.components = Components.GetData(); 173 | Meshes.SetNum(Count); 174 | XrSceneMeshesMSFT SceneMeshes{ XR_TYPE_SCENE_MESHES_MSFT }; 175 | SceneMeshes.sceneMeshCount = Count; 176 | SceneMeshes.sceneMeshes = Meshes.GetData(); 177 | InsertExtensionStruct(SceneComponents, SceneMeshes); 178 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 179 | } 180 | 181 | inline void GetScenePlanes(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext, 182 | const TArray& PlaneAlignmentFilters, TArray& Components, 183 | TArray& Planes) 184 | { 185 | XrSceneComponentsGetInfoMSFT GetInfo{XR_TYPE_SCENE_COMPONENTS_GET_INFO_MSFT}; 186 | GetInfo.componentType = XR_SCENE_COMPONENT_TYPE_PLANE_MSFT; 187 | 188 | XrScenePlaneAlignmentFilterInfoMSFT AlignmentFilter{XR_TYPE_SCENE_PLANE_ALIGNMENT_FILTER_INFO_MSFT}; 189 | if (PlaneAlignmentFilters.Num() > 0) 190 | { 191 | AlignmentFilter.alignmentCount = static_cast(PlaneAlignmentFilters.Num()); 192 | AlignmentFilter.alignments = PlaneAlignmentFilters.GetData(); 193 | InsertExtensionStruct(GetInfo, AlignmentFilter); 194 | } 195 | 196 | XrSceneComponentsMSFT SceneComponents{XR_TYPE_SCENE_COMPONENTS_MSFT}; 197 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 198 | const uint32_t Count = SceneComponents.componentCountOutput; 199 | 200 | Components.SetNum(Count); 201 | SceneComponents.componentCapacityInput = Count; 202 | SceneComponents.components = Components.GetData(); 203 | 204 | Planes.SetNum(Count); 205 | XrScenePlanesMSFT ScenePlanes{XR_TYPE_SCENE_PLANES_MSFT}; 206 | ScenePlanes.scenePlaneCount = Count; 207 | ScenePlanes.scenePlanes = Planes.GetData(); 208 | InsertExtensionStruct(SceneComponents, ScenePlanes); 209 | 210 | XR_ENSURE(Ext.xrGetSceneComponentsMSFT(SceneHandle, &GetInfo, &SceneComponents)); 211 | } 212 | 213 | // Locate components given space and time. 214 | inline void LocateObjects(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext, XrSpace BaseSpace, XrTime Time, 215 | const TArray& Identifiers, TArray& Locations) 216 | { 217 | XrSceneComponentsLocateInfoMSFT LocateInfo{XR_TYPE_SCENE_COMPONENTS_LOCATE_INFO_MSFT}; 218 | LocateInfo.baseSpace = BaseSpace; 219 | LocateInfo.time = Time; 220 | LocateInfo.componentIdCount = static_cast(Identifiers.Num()); 221 | LocateInfo.componentIds = Identifiers.GetData(); 222 | 223 | Locations.SetNum(Identifiers.Num()); 224 | XrSceneComponentLocationsMSFT ComponentLocations{XR_TYPE_SCENE_COMPONENT_LOCATIONS_MSFT}; 225 | ComponentLocations.locationCount = static_cast(Locations.Num()); 226 | ComponentLocations.locations = Locations.GetData(); 227 | 228 | XR_ENSURE(Ext.xrLocateSceneComponentsMSFT(SceneHandle, &LocateInfo, &ComponentLocations)); 229 | } 230 | 231 | // Reads mesh vertices and 32-bit indices. 232 | inline void ReadMeshBuffers(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext, uint64_t MeshBufferId, 233 | TArray& VertexBuffer, TArray& IndexBuffer) 234 | { 235 | static_assert(sizeof(XrVector3f) == sizeof(FVector)); 236 | XrSceneMeshBuffersGetInfoMSFT MeshGetInfo{XR_TYPE_SCENE_MESH_BUFFERS_GET_INFO_MSFT}; 237 | MeshGetInfo.meshBufferId = MeshBufferId; 238 | 239 | XrSceneMeshBuffersMSFT MeshBuffers{XR_TYPE_SCENE_MESH_BUFFERS_MSFT}; 240 | XrSceneMeshVertexBufferMSFT Vertices{XR_TYPE_SCENE_MESH_VERTEX_BUFFER_MSFT}; 241 | XrSceneMeshIndicesUint32MSFT Indices{XR_TYPE_SCENE_MESH_INDICES_UINT32_MSFT}; 242 | InsertExtensionStruct(MeshBuffers, Vertices); 243 | InsertExtensionStruct(MeshBuffers, Indices); 244 | XR_ENSURE(Ext.xrGetSceneMeshBuffersMSFT(SceneHandle, &MeshGetInfo, &MeshBuffers)); 245 | 246 | VertexBuffer.SetNum(Vertices.vertexCountOutput); 247 | IndexBuffer.SetNum(Indices.indexCountOutput); 248 | Vertices.vertexCapacityInput = Vertices.vertexCountOutput; 249 | Indices.indexCapacityInput = Indices.indexCountOutput; 250 | Vertices.vertices = reinterpret_cast(VertexBuffer.GetData()); 251 | Indices.indices = IndexBuffer.GetData(); 252 | XR_ENSURE(Ext.xrGetSceneMeshBuffersMSFT(SceneHandle, &MeshGetInfo, &MeshBuffers)); 253 | VertexBuffer.SetNum(Vertices.vertexCountOutput); 254 | IndexBuffer.SetNum(Indices.indexCountOutput); 255 | } 256 | 257 | // Reads mesh vertices and 16-bit indices. 258 | inline void ReadMeshBuffers(XrSceneMSFT SceneHandle, const ExtensionDispatchTable& Ext, uint64_t MeshBufferId, 259 | TArray& VertexBuffer, TArray& IndexBuffer) 260 | { 261 | static_assert(sizeof(XrVector3f) == sizeof(FVector)); 262 | XrSceneMeshBuffersGetInfoMSFT MeshGetInfo{XR_TYPE_SCENE_MESH_BUFFERS_GET_INFO_MSFT}; 263 | MeshGetInfo.meshBufferId = MeshBufferId; 264 | 265 | XrSceneMeshBuffersMSFT MeshBuffers{XR_TYPE_SCENE_MESH_BUFFERS_MSFT}; 266 | XrSceneMeshVertexBufferMSFT Vertices{XR_TYPE_SCENE_MESH_VERTEX_BUFFER_MSFT}; 267 | XrSceneMeshIndicesUint16MSFT Indices{XR_TYPE_SCENE_MESH_INDICES_UINT16_MSFT}; 268 | InsertExtensionStruct(MeshBuffers, Vertices); 269 | InsertExtensionStruct(MeshBuffers, Indices); 270 | XR_ENSURE(Ext.xrGetSceneMeshBuffersMSFT(SceneHandle, &MeshGetInfo, &MeshBuffers)); 271 | 272 | VertexBuffer.SetNum(Vertices.vertexCountOutput); 273 | IndexBuffer.SetNum(Indices.indexCountOutput); 274 | Vertices.vertexCapacityInput = Vertices.vertexCountOutput; 275 | Indices.indexCapacityInput = Indices.indexCountOutput; 276 | Vertices.vertices = reinterpret_cast(VertexBuffer.GetData()); 277 | Indices.indices = IndexBuffer.GetData(); 278 | XR_ENSURE(Ext.xrGetSceneMeshBuffersMSFT(SceneHandle, &MeshGetInfo, &MeshBuffers)); 279 | VertexBuffer.SetNum(Vertices.vertexCountOutput); 280 | IndexBuffer.SetNum(Indices.indexCountOutput); 281 | } 282 | 283 | } // namespace MicrosoftOpenXR 284 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SecondaryViewConfiguration.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "OpenXRCommon.h" 4 | 5 | namespace MicrosoftOpenXR 6 | { 7 | class FSecondaryViewConfigurationPlugin : public IOpenXRExtensionPlugin 8 | { 9 | // This keeps all data structures and memory alive in a safe way for a single projection layer. 10 | struct SingleProjectionLayer 11 | { 12 | SingleProjectionLayer(XrViewConfigurationType InViewConfigType, XrEnvironmentBlendMode InEnvBlendMode, XrSpace InSpace, 13 | TArray InViews) 14 | : ViewConfigType(InViewConfigType), EnvBlendMode(InEnvBlendMode), Space(InSpace), Views(std::move(InViews)) 15 | { 16 | } 17 | 18 | void Finalize(XrSecondaryViewConfigurationLayerInfoMSFT* SecondaryViewInfo) 19 | { 20 | // This struct may be relocated while inside a resizing TArray. Only call this function once the TArray is complete 21 | // to ensure valid pointers. 22 | Layer = {XR_TYPE_COMPOSITION_LAYER_PROJECTION}; 23 | Layer.space = Space; 24 | Layer.viewCount = Views.Num(); 25 | Layer.views = Views.GetData(); 26 | Layer.layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT; 27 | 28 | LayerPtrs.Reset(); 29 | LayerPtrs.Add(reinterpret_cast(&Layer)); 30 | 31 | *SecondaryViewInfo = {XR_TYPE_SECONDARY_VIEW_CONFIGURATION_LAYER_INFO_MSFT, nullptr, ViewConfigType, EnvBlendMode, 32 | (uint32_t) LayerPtrs.Num(), 33 | LayerPtrs.GetData()}; 34 | } 35 | 36 | private: 37 | const XrViewConfigurationType ViewConfigType; 38 | const XrEnvironmentBlendMode EnvBlendMode; 39 | const XrSpace Space; 40 | const TArray Views; 41 | 42 | // The following are set up at 43 | XrCompositionLayerProjection Layer{XR_TYPE_COMPOSITION_LAYER_PROJECTION}; 44 | TArray LayerPtrs; 45 | }; 46 | 47 | struct PiplinedFrameState 48 | { 49 | // State of every secondary view configuration that was enabled. Updated on xrWaitFrame. 50 | TArray SecondaryViewConfigStates; 51 | // Latest XrView (Pose and Fov) returned to engine used for rendering. 52 | TArray> SecondaryViews; 53 | 54 | XrSpace ViewSpace{XR_NULL_HANDLE}; 55 | }; 56 | 57 | public: 58 | void Register(); 59 | void Unregister(); 60 | 61 | bool GetRequiredExtensions(TArray& OutExtensions) override; 62 | bool GetOptionalExtensions(TArray& OutExtensions) override; 63 | 64 | void PostGetSystem(XrInstance InInstance, XrSystemId InSystem) override; 65 | 66 | const void* OnBeginSession(XrSession InSession, const void* InNext) override; 67 | 68 | void* OnWaitFrame(XrSession InSession, void* InNext) override; 69 | const void* OnBeginFrame(XrSession InSession, XrTime DisplayTime, const void* InNext) override; 70 | 71 | void GetViewConfigurations(XrSystemId InSystem, TArray& OutViews) override; 72 | void GetViewLocations(XrSession InSession, XrTime DisplayTime, XrSpace space, TArray& OutViews) override; 73 | const void* OnEndFrame(XrSession InSession, XrTime DisplayTime, const TArray InColorImages, 74 | const TArray InDepthImages, const void* InNext) override; 75 | 76 | private: 77 | PiplinedFrameState& GetSecondaryViewStateForThread(); 78 | 79 | private: 80 | XrInstance Instance{XR_NULL_HANDLE}; 81 | XrSystemId System{XR_NULL_SYSTEM_ID}; 82 | 83 | // View configs that have been enabled on xrBeginSession and their cached properties. 84 | TArray EnabledViewConfigTypes; 85 | TArray EnabledViewConfigEnvBlendModes; 86 | TArray> EnabledViewConfigurationViews; 87 | 88 | PiplinedFrameState SecondaryViewState_GameThread; 89 | PiplinedFrameState SecondaryViewState_RenderThread; 90 | 91 | XrSecondaryViewConfigurationSessionBeginInfoMSFT SecondaryViewConfigurationSessionBeginInfo; 92 | XrSecondaryViewConfigurationFrameStateMSFT SecondaryViewConfigurationFrameState; 93 | XrSecondaryViewConfigurationFrameEndInfoMSFT SecondaryViewConfigurationEndInfo; 94 | 95 | TArray SecondaryProjectionLayers; 96 | TArray SecondaryViewLayers; 97 | }; 98 | } // namespace MicrosoftOpenXR -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SpatialAnchorPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | 8 | #if (PLATFORM_WINDOWS || PLATFORM_HOLOLENS) 9 | #include "Windows/AllowWindowsPlatformTypes.h" 10 | #include "Windows/AllowWindowsPlatformAtomics.h" 11 | #include "Windows/PreWindowsApi.h" 12 | 13 | #include 14 | #include 15 | #include 16 | #include 17 | 18 | #include "Windows/PostWindowsApi.h" 19 | #include "Windows/HideWindowsPlatformAtomics.h" 20 | #include "Windows/HideWindowsPlatformTypes.h" 21 | 22 | #include 23 | #define WINRT_ANCHOR_STORE_AVAILABLE 1 24 | #else 25 | #define WINRT_ANCHOR_STORE_AVAILABLE 0 26 | #endif 27 | 28 | namespace MicrosoftOpenXR 29 | { 30 | struct SAnchorMSFT 31 | { 32 | XrSpatialAnchorMSFT Anchor; 33 | XrSpace Space; 34 | }; 35 | 36 | class FSpatialAnchorPlugin : public IOpenXRExtensionPlugin, public IOpenXRCustomAnchorSupport 37 | { 38 | public: 39 | void Register(); 40 | void Unregister(); 41 | 42 | virtual bool GetRequiredExtensions(TArray& OutExtensions) override; 43 | 44 | virtual bool GetOptionalExtensions(TArray& OutExtensions) override; 45 | 46 | virtual const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 47 | virtual const void* OnBeginSession(XrSession InSession, const void* InNext) override; 48 | 49 | virtual IOpenXRCustomAnchorSupport* GetCustomAnchorSupport() override; 50 | 51 | virtual bool OnPinComponent(class UARPin* Pin, XrSession InSession, XrSpace TrackingSpace, XrTime DisplayTime, float worldToMeterScale) override; 52 | 53 | virtual void OnRemovePin(class UARPin* Pin) override; 54 | 55 | virtual void OnUpdatePin(class UARPin* Pin, XrSession InSession, XrSpace TrackingSpace, XrTime DisplayTime, float worldToMeterScale) override; 56 | 57 | virtual bool IsLocalPinSaveSupported() const override; 58 | 59 | bool IsAnchorStoreReady(); 60 | virtual bool ArePinsReadyToLoad() override; 61 | 62 | virtual void LoadARPins(XrSession InSession, TFunction OnCreatePin) override; 63 | 64 | virtual bool SaveARPin(XrSession InSession, FName InName, UARPin* InPin) override; 65 | 66 | virtual void RemoveSavedARPin(XrSession InSession, FName InName) override; 67 | 68 | virtual void RemoveAllSavedARPins(XrSession InSession) override; 69 | 70 | bool GetPerceptionAnchorFromOpenXRAnchor(XrSpatialAnchorMSFT AnchorId, ::IUnknown** OutPerceptionAnchor); 71 | bool StorePerceptionAnchor(const FString& InPinId, ::IUnknown* InPerceptionAnchor); 72 | 73 | private: 74 | XrSession Session; 75 | 76 | PFN_xrCreateSpatialAnchorMSFT xrCreateSpatialAnchorMSFT; 77 | PFN_xrDestroySpatialAnchorMSFT xrDestroySpatialAnchorMSFT; 78 | PFN_xrCreateSpatialAnchorSpaceMSFT xrCreateSpatialAnchorSpaceMSFT; 79 | 80 | bool bIsAnchorPersistenceExtensionSupported = false; 81 | bool bIsPerceptionAnchorInteropExtensionSupported = false; 82 | 83 | #if WINRT_ANCHOR_STORE_AVAILABLE 84 | PFN_xrCreateSpatialAnchorFromPerceptionAnchorMSFT xrCreateSpatialAnchorFromPerceptionAnchorMSFT; 85 | PFN_xrTryGetPerceptionAnchorFromSpatialAnchorMSFT xrTryGetPerceptionAnchorFromSpatialAnchorMSFT; 86 | 87 | std::mutex m_spatialAnchorStoreLock; 88 | winrt::Windows::Foundation::IAsyncOperation m_spatialAnchorStoreAsyncOperation; 89 | winrt::Windows::Perception::Spatial::SpatialAnchorStore m_spatialAnchorStore{ nullptr }; 90 | #endif 91 | 92 | PFN_xrCreateSpatialAnchorStoreConnectionMSFT xrCreateSpatialAnchorStoreConnectionMSFT; 93 | PFN_xrDestroySpatialAnchorStoreConnectionMSFT xrDestroySpatialAnchorStoreConnectionMSFT; 94 | PFN_xrPersistSpatialAnchorMSFT xrPersistSpatialAnchorMSFT; 95 | PFN_xrEnumeratePersistedSpatialAnchorNamesMSFT xrEnumeratePersistedSpatialAnchorNamesMSFT; 96 | PFN_xrCreateSpatialAnchorFromPersistedNameMSFT xrCreateSpatialAnchorFromPersistedNameMSFT; 97 | PFN_xrUnpersistSpatialAnchorMSFT xrUnpersistSpatialAnchorMSFT; 98 | PFN_xrClearSpatialAnchorStoreMSFT xrClearSpatialAnchorStoreMSFT; 99 | 100 | XrSpatialAnchorStoreConnectionMSFT SpatialAnchorStoreMSFT = XR_NULL_HANDLE; 101 | 102 | void OnEndPlay(); 103 | }; 104 | } // namespace MicrosoftOpenXR 105 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SpatialMappingPlugin.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "SpatialMappingPlugin.h" 5 | 6 | namespace MicrosoftOpenXR 7 | { 8 | FSpatialMappingPlugin::FSpatialMappingPlugin() 9 | { 10 | } 11 | 12 | IOpenXRCustomCaptureSupport* FSpatialMappingPlugin::GetCustomCaptureSupport(const EARCaptureType CaptureType) 13 | { 14 | return CaptureType == EARCaptureType::SpatialMapping ? this : nullptr; 15 | } 16 | 17 | XrSceneComputeConsistencyMSFT FSpatialMappingPlugin::GetSceneComputeConsistency() 18 | { 19 | return XR_SCENE_COMPUTE_CONSISTENCY_OCCLUSION_OPTIMIZED_MSFT; 20 | } 21 | 22 | TArray FSpatialMappingPlugin::GetSceneComputeFeatures(class UARSessionConfig* SessionConfig) 23 | { 24 | TArray SceneComputeFeatures; 25 | SceneComputeFeatures.AddUnique(XR_SCENE_COMPUTE_FEATURE_VISUAL_MESH_MSFT); 26 | 27 | return SceneComputeFeatures; 28 | } 29 | 30 | } // namespace MicrosoftOpenXR 31 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SpatialMappingPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | #include "SceneUnderstandingBase.h" 8 | 9 | namespace MicrosoftOpenXR 10 | { 11 | class FSpatialMappingPlugin : public FSceneUnderstandingBase 12 | { 13 | public: 14 | FSpatialMappingPlugin(); 15 | 16 | IOpenXRCustomCaptureSupport* GetCustomCaptureSupport(const EARCaptureType CaptureType) override; 17 | 18 | protected: 19 | XrSceneComputeConsistencyMSFT GetSceneComputeConsistency() override; 20 | TArray GetSceneComputeFeatures(class UARSessionConfig* SessionConfig) override; 21 | }; 22 | } // namespace MicrosoftOpenXR 23 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/SpeechPlugin.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 7 | 8 | #include "OpenXRCommon.h" 9 | #include "OpenXRCore.h" 10 | #include "MicrosoftOpenXR.h" 11 | #include "HeadMountedDisplayTypes.h" 12 | 13 | #include "GameFramework/PlayerInput.h" 14 | #include "GameFramework/PlayerController.h" 15 | #include "GameFramework/InputSettings.h" 16 | 17 | #include "UObject/NameTypes.h" 18 | #include "UObject/UObjectGlobals.h" 19 | #include "Async/Async.h" 20 | 21 | #include "Windows/AllowWindowsPlatformTypes.h" 22 | #include "Windows/AllowWindowsPlatformAtomics.h" 23 | #include "Windows/PreWindowsApi.h" 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include "Windows/PostWindowsApi.h" 31 | #include "Windows/HideWindowsPlatformAtomics.h" 32 | #include "Windows/HideWindowsPlatformTypes.h" 33 | 34 | #if SUPPORTS_REMOTING 35 | #include "openxr_msft_holographic_remoting.h " 36 | #include "openxr_msft_remoting_speech.h" 37 | #endif 38 | 39 | namespace MicrosoftOpenXR 40 | { 41 | class FSpeechPlugin : public IOpenXRExtensionPlugin 42 | { 43 | public: 44 | void Register(); 45 | void Unregister(); 46 | 47 | bool GetOptionalExtensions(TArray& OutExtensions) override; 48 | const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override; 49 | const void* OnBeginSession(XrSession InSession, const void* InNext) override; 50 | void OnEvent(XrSession InSession, const XrEventDataBaseHeader* InHeader) override; 51 | void OnEndPlay(); 52 | 53 | void AddKeywords(TArray KeywordsToAdd); 54 | void RemoveKeywords(TArray KeywordsToRemove); 55 | 56 | private: 57 | winrt::Windows::Media::SpeechRecognition::SpeechRecognizer SpeechRecognizer = nullptr; 58 | winrt::Windows::Foundation::IAsyncOperation CompileConstraintsAsyncOperation; 59 | winrt::Windows::Foundation::IAsyncAction SessionStartAction; 60 | winrt::event_token ResultsGeneratedToken; 61 | std::vector Keywords; 62 | TMap KeywordMap; 63 | 64 | APlayerController* GetPlayerController(); 65 | 66 | void RegisterKeyword(FKey Key, FString Keyword); 67 | void CallSpeechCallback(FKey InKey); 68 | void StartSpeechRecognizer(); 69 | void StopSpeechRecognizer(); 70 | 71 | XrSession Session; 72 | 73 | // Remoting 74 | #if SUPPORTS_REMOTING 75 | PFN_xrInitializeRemotingSpeechMSFT xrInitializeRemotingSpeechMSFT; 76 | PFN_xrRetrieveRemotingSpeechRecognizedTextMSFT xrRetrieveRemotingSpeechRecognizedTextMSFT; 77 | #endif 78 | bool bIsRemotingSpeechExtensionEnabled = false; 79 | 80 | void RegisterSpeechCommandsWithRemoting(); 81 | }; 82 | } // namespace MicrosoftOpenXR 83 | 84 | #endif //PLATFORM_WINDOWS || PLATFORM_HOLOLENS 85 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/TrackedGeometryCollision.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "TrackedGeometryCollision.h" 5 | 6 | namespace MicrosoftOpenXR 7 | { 8 | TrackedGeometryCollision::TrackedGeometryCollision(TArray InVertices, TArray InIndices) 9 | : Vertices(std::move(InVertices)), Indices(std::move(InIndices)) 10 | { 11 | // Create a bounding box from the input vertices to reduce the number of full meshes that need to be hit-tested. 12 | if (Vertices.Num() > 0) 13 | { 14 | BoundingBox = FBox(&Vertices[0], Vertices.Num()); 15 | } 16 | } 17 | 18 | bool TrackedGeometryCollision::Collides(const FVector Start, const FVector End, const FTransform MeshToWorld, 19 | FVector& OutHitPoint, FVector& OutHitNormal, float& OutHitDistance) 20 | { 21 | if (MeshToWorld.GetScale3D().IsNearlyZero()) 22 | { 23 | return false; 24 | } 25 | 26 | // Check bounding box collision first so we don't check triangles for meshes we definitely won't collide with. 27 | if (!FMath::LineBoxIntersection(BoundingBox.TransformBy(MeshToWorld), Start, End, End - Start)) 28 | { 29 | return false; 30 | } 31 | 32 | // Check for triangle collision and set the output hit position, normal, and distance. 33 | for (int i = 0; i < Indices.Num(); i += 3) 34 | { 35 | // Ignore this triangle if it has indices out of range. 36 | if ((unsigned int) Indices[i] > (unsigned int) Vertices.Num() || 37 | (unsigned int) Indices[i + 1] > (unsigned int) Vertices.Num() || 38 | (unsigned int) Indices[i + 2] > (unsigned int) Vertices.Num()) 39 | { 40 | continue; 41 | } 42 | 43 | if (FMath::SegmentTriangleIntersection(Start, End, MeshToWorld.TransformPosition(Vertices[Indices[i]]), 44 | MeshToWorld.TransformPosition(Vertices[Indices[i + 1]]), 45 | MeshToWorld.TransformPosition(Vertices[Indices[i + 2]]), OutHitPoint, OutHitNormal)) 46 | { 47 | OutHitDistance = (OutHitPoint - Start).Size(); 48 | return true; 49 | } 50 | } 51 | 52 | return false; 53 | } 54 | 55 | void TrackedGeometryCollision::CreateMeshDataForBoundingBox(FVector Center, FVector HalfExtents, TArray& OutVertices, TArray& OutIndices) 56 | { 57 | // Ensure output arrays are empty. 58 | OutVertices.Empty(); 59 | OutIndices.Empty(); 60 | 61 | // Top Vertices (+Z) 62 | OutVertices.Add(Center + HalfExtents); 63 | OutVertices.Add(Center + FVector(-HalfExtents.X, HalfExtents.Y, HalfExtents.Z)); 64 | OutVertices.Add(Center + FVector(HalfExtents.X, -HalfExtents.Y, HalfExtents.Z)); 65 | OutVertices.Add(Center + FVector(-HalfExtents.X, -HalfExtents.Y, HalfExtents.Z)); 66 | 67 | // Bottom Vertices (-Z) 68 | OutVertices.Add(Center + FVector(HalfExtents.X, HalfExtents.Y, -HalfExtents.Z)); 69 | OutVertices.Add(Center + FVector(-HalfExtents.X, HalfExtents.Y, -HalfExtents.Z)); 70 | OutVertices.Add(Center + FVector(HalfExtents.X, -HalfExtents.Y, -HalfExtents.Z)); 71 | OutVertices.Add(Center - HalfExtents); 72 | 73 | // Clockwise winding 74 | //Top 75 | OutIndices.Add(0); 76 | OutIndices.Add(1); 77 | OutIndices.Add(2); 78 | 79 | OutIndices.Add(2); 80 | OutIndices.Add(1); 81 | OutIndices.Add(3); 82 | 83 | // Front 84 | OutIndices.Add(1); 85 | OutIndices.Add(5); 86 | OutIndices.Add(3); 87 | 88 | OutIndices.Add(3); 89 | OutIndices.Add(5); 90 | OutIndices.Add(7); 91 | 92 | // Bottom 93 | OutIndices.Add(6); 94 | OutIndices.Add(5); 95 | OutIndices.Add(4); 96 | 97 | OutIndices.Add(5); 98 | OutIndices.Add(6); 99 | OutIndices.Add(7); 100 | 101 | // Back 102 | OutIndices.Add(0); 103 | OutIndices.Add(2); 104 | OutIndices.Add(4); 105 | 106 | OutIndices.Add(2); 107 | OutIndices.Add(6); 108 | OutIndices.Add(4); 109 | 110 | // Left 111 | OutIndices.Add(2); 112 | OutIndices.Add(3); 113 | OutIndices.Add(6); 114 | 115 | OutIndices.Add(3); 116 | OutIndices.Add(7); 117 | OutIndices.Add(6); 118 | 119 | // Right 120 | OutIndices.Add(1); 121 | OutIndices.Add(0); 122 | OutIndices.Add(4); 123 | 124 | OutIndices.Add(1); 125 | OutIndices.Add(4); 126 | OutIndices.Add(5); 127 | } 128 | } // namespace MicrosoftOpenXR 129 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/TrackedGeometryCollision.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | #include "OpenXRCommon.h" 6 | #include "OpenXRCore.h" 7 | #include "IOpenXRARModule.h" 8 | #include "IOpenXRARTrackedGeometryHolder.h" 9 | 10 | #include "HeadMountedDisplayTypes.h" 11 | #include "ARTypes.h" 12 | 13 | namespace MicrosoftOpenXR 14 | { 15 | class TrackedGeometryCollision 16 | { 17 | public: 18 | TrackedGeometryCollision(TArray InVertices, TArray InIndices); 19 | 20 | /// 21 | /// Hit test a ray against tracked mesh data. 22 | /// 23 | /// Start of collision ray in world space 24 | /// End of collision ray in world space 25 | /// Transform from mesh local space to world space. The mesh may not be in tracking space. 26 | /// Position of hit in world space 27 | /// Normal of hit in world space 28 | /// Distance from ray start 29 | /// True if the input ray collides with this mesh. 30 | bool Collides(const FVector Start, const FVector End, const FTransform MeshToWorld, FVector& OutHitPoint, FVector& OutHitNormal, float& OutHitDistance); 31 | 32 | static void CreateMeshDataForBoundingBox(FVector Center, FVector HalfExtents, TArray& OutVertices, TArray& OutIndices); 33 | 34 | private: 35 | TArray Vertices; 36 | TArray Indices; 37 | 38 | FBox BoundingBox; 39 | }; 40 | } // namespace MicrosoftOpenXR 41 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/UniqueHandle.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2021 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "OpenXRCommon.h" 7 | 8 | namespace MicrosoftOpenXR 9 | { 10 | template 11 | class TUniqueExtHandle 12 | { 13 | using PFN_DestroyFunction = XrResult(XRAPI_PTR*)(HandleType); 14 | 15 | public: 16 | TUniqueExtHandle() = default; 17 | TUniqueExtHandle(const TUniqueExtHandle&) = delete; 18 | TUniqueExtHandle(TUniqueExtHandle&& Other) noexcept 19 | { 20 | *this = std::move(Other); 21 | } 22 | 23 | ~TUniqueExtHandle() noexcept 24 | { 25 | Reset(); 26 | } 27 | 28 | TUniqueExtHandle& operator=(const TUniqueExtHandle&) = delete; 29 | TUniqueExtHandle& operator=(TUniqueExtHandle&& Other) noexcept 30 | { 31 | if (HandleValue != Other.HandleValue || Destroyer != Other.Destroyer) 32 | { 33 | Reset(); 34 | 35 | HandleValue = Other.HandleValue; 36 | Destroyer = Other.Destroyer; 37 | 38 | Other.HandleValue = XR_NULL_HANDLE; 39 | Other.Destroyer = nullptr; 40 | } 41 | return *this; 42 | } 43 | 44 | operator bool() const noexcept 45 | { 46 | return HandleValue != XR_NULL_HANDLE; 47 | } 48 | 49 | HandleType Handle() const noexcept 50 | { 51 | return HandleValue; 52 | } 53 | 54 | // Extension functions cannot be statically linked, so the creator must pass in the destroy function. 55 | HandleType* Put(PFN_DestroyFunction DestroyFunction) noexcept 56 | { 57 | assert(DestroyFunction != nullptr); 58 | Reset(); 59 | Destroyer = DestroyFunction; 60 | return &HandleValue; 61 | } 62 | 63 | void Reset() noexcept 64 | { 65 | if (HandleValue != XR_NULL_HANDLE) 66 | { 67 | Destroyer(HandleValue); 68 | HandleValue = XR_NULL_HANDLE; 69 | } 70 | Destroyer = nullptr; 71 | } 72 | 73 | private: 74 | HandleType HandleValue{XR_NULL_HANDLE}; 75 | PFN_DestroyFunction Destroyer{nullptr}; 76 | }; 77 | 78 | // Defines a move-only smart handle for XrSpace. 79 | class FSpaceHandle : public MicrosoftOpenXR::TUniqueExtHandle 80 | { 81 | }; 82 | 83 | inline FSpaceHandle CreateViewSpace(XrSession Session) 84 | { 85 | FSpaceHandle ViewSpace; 86 | XrReferenceSpaceCreateInfo SpaceCreateInfo{XR_TYPE_REFERENCE_SPACE_CREATE_INFO}; 87 | SpaceCreateInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW; 88 | SpaceCreateInfo.poseInReferenceSpace = {{0, 0, 0, 1}, {0, 0, 0}}; 89 | XR_ENSURE(xrCreateReferenceSpace(Session, &SpaceCreateInfo, ViewSpace.Put(xrDestroySpace))); 90 | return ViewSpace; 91 | } 92 | 93 | } // namespace MicrosoftOpenXR 94 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/WindowsMixedRealityInteropUtility.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #if PLATFORM_WINDOWS || PLATFORM_HOLOLENS 7 | #include "CoreMinimal.h" 8 | #include "HAL\UnrealMemory.h" 9 | 10 | #include "Windows/WindowsHWrapper.h" 11 | #include "HeadMountedDisplayTypes.h" 12 | 13 | #include 14 | #include 15 | #include 16 | 17 | 18 | namespace MicrosoftOpenXR 19 | { 20 | class WMRUtility 21 | { 22 | public: 23 | // Convert between DirectX XMMatrix to Unreal FMatrix. 24 | static FORCEINLINE FMatrix ToFMatrix(DirectX::XMMATRIX& M) 25 | { 26 | DirectX::XMFLOAT4X4 dst; 27 | DirectX::XMStoreFloat4x4(&dst, M); 28 | 29 | return FMatrix( 30 | FPlane(dst._11, dst._21, dst._31, dst._41), 31 | FPlane(dst._12, dst._22, dst._32, dst._42), 32 | FPlane(dst._13, dst._23, dst._33, dst._43), 33 | FPlane(dst._14, dst._24, dst._34, dst._44)); 34 | } 35 | 36 | static FORCEINLINE FMatrix ToFMatrix(DirectX::XMFLOAT4X4& M) 37 | { 38 | return FMatrix( 39 | FPlane(M._11, M._21, M._31, M._41), 40 | FPlane(M._12, M._22, M._32, M._42), 41 | FPlane(M._13, M._23, M._33, M._43), 42 | FPlane(M._14, M._24, M._34, M._44)); 43 | } 44 | 45 | static FORCEINLINE FTransform FromMixedRealityTransform(const DirectX::XMMATRIX& M, float InScale = 1.0f) 46 | { 47 | DirectX::XMVECTOR Scale; 48 | DirectX::XMVECTOR Rotation; 49 | DirectX::XMVECTOR Translation; 50 | DirectX::XMMatrixDecompose(&Scale, &Rotation, &Translation, M); 51 | 52 | return FTransform(FromXMVectorRotation(Rotation), FromXMVectorTranslation(Translation, InScale), FromXMVectorScale(Scale)); 53 | } 54 | 55 | static FORCEINLINE FVector FromMixedRealityVector(DirectX::XMFLOAT3 pos) 56 | { 57 | return FVector( 58 | -1.0f * pos.z, 59 | pos.x, 60 | pos.y); 61 | } 62 | 63 | static FORCEINLINE DirectX::XMFLOAT3 ToMixedRealityVector(FVector pos) 64 | { 65 | return DirectX::XMFLOAT3( 66 | pos.Y, 67 | pos.Z, 68 | -1.0f * pos.X); 69 | } 70 | 71 | static FORCEINLINE FVector FromMixedRealityScale(DirectX::XMFLOAT3 pos) 72 | { 73 | return FVector( 74 | pos.z, 75 | pos.x, 76 | pos.y); 77 | } 78 | 79 | static FORCEINLINE FVector FromMixedRealityScale(winrt::Windows::Foundation::Numerics::float3 pos) 80 | { 81 | return FVector( 82 | pos.z, 83 | pos.x, 84 | pos.y); 85 | } 86 | 87 | static FORCEINLINE DirectX::XMFLOAT3 ToMixedRealityScale(FVector pos) 88 | { 89 | return DirectX::XMFLOAT3( 90 | pos.Y, 91 | pos.Z, 92 | pos.X); 93 | } 94 | 95 | 96 | static FORCEINLINE FQuat FromMixedRealityQuaternion(DirectX::XMFLOAT4 rot) 97 | { 98 | FQuat quaternion( 99 | -1.0f * rot.z, 100 | rot.x, 101 | rot.y, 102 | -1.0f * rot.w); 103 | quaternion.Normalize(); 104 | 105 | return quaternion; 106 | } 107 | 108 | static FORCEINLINE DirectX::XMFLOAT4 ToMixedRealityQuaternion(FQuat rot) 109 | { 110 | // Windows api IsNormalized checks fail on a negative identity quaternion. 111 | if (rot == FQuat::Identity) 112 | { 113 | return DirectX::XMFLOAT4(0.0f, 0.0f, 0.0f, 1.0f); 114 | } 115 | 116 | DirectX::XMVECTOR v = DirectX::XMVectorSet( 117 | rot.Y, 118 | rot.Z, 119 | -1.0f * rot.X, 120 | -1.0f * rot.W); 121 | DirectX::XMQuaternionNormalize(v); 122 | 123 | DirectX::XMFLOAT4 quatf4out; 124 | DirectX::XMStoreFloat4(&quatf4out, v); 125 | return quatf4out; 126 | } 127 | 128 | static FORCEINLINE FVector FromFloat3(winrt::Windows::Foundation::Numerics::float3 pos, float scale = 1.0f) 129 | { 130 | return FVector( 131 | -1.0f * pos.z, 132 | pos.x, 133 | pos.y) * scale; 134 | } 135 | 136 | static FORCEINLINE FVector FromXMVectorTranslation(DirectX::XMVECTOR InValue, float scale = 1.0f) 137 | { 138 | InValue = DirectX::XMVectorMultiply(InValue, DirectX::XMVectorSet(scale, scale, -1 * scale, scale)); 139 | InValue = DirectX::XMVectorSwizzle(InValue, 2, 0, 1, 3); 140 | 141 | DirectX::XMFLOAT3 Dest; 142 | DirectX::XMStoreFloat3(&Dest, InValue); 143 | return FVector(Dest.x, Dest.y, Dest.z); 144 | } 145 | 146 | static FORCEINLINE FQuat FromXMVectorRotation(DirectX::XMVECTOR InValue) 147 | { 148 | DirectX::XMFLOAT4 Dest; 149 | DirectX::XMStoreFloat4(&Dest, InValue); 150 | 151 | return FromMixedRealityQuaternion(Dest); 152 | } 153 | 154 | static FORCEINLINE FVector FromXMVectorScale(DirectX::XMVECTOR InValue) 155 | { 156 | InValue = DirectX::XMVectorSwizzle(InValue, 2, 0, 1, 3); 157 | 158 | DirectX::XMFLOAT3 Dest; 159 | DirectX::XMStoreFloat3(&Dest, InValue); 160 | return FVector(Dest.x, Dest.y, Dest.z); 161 | } 162 | 163 | static FORCEINLINE FVector2D FromFloat2(winrt::Windows::Foundation::Numerics::float2 pos) 164 | { 165 | return FVector2D(pos.x, pos.y); 166 | } 167 | 168 | 169 | 170 | static FORCEINLINE FGuid GUIDToFGuid(const winrt::guid& InGuid) 171 | { 172 | // Pack the FGuid correctly from the input winrt guid data, so the value is not lost in conversion. 173 | // FGuids use 4 uint32's 174 | // winrt guids use uint32, uint16, uint16, uint8[8] 175 | // Do not simply memcpy guid to FGuid, or B, C, and D will be mismatched. 176 | 177 | uint32 A = InGuid.Data1; 178 | uint32 B = (uint32)InGuid.Data2 << 16 | InGuid.Data3; 179 | uint32 C = (uint32)InGuid.Data4[0] << 24 | 180 | (uint32)InGuid.Data4[1] << 16 | 181 | (uint16)InGuid.Data4[2] << 8 | 182 | InGuid.Data4[3]; 183 | uint32 D = (uint32)InGuid.Data4[4] << 24 | 184 | (uint32)InGuid.Data4[5] << 16 | 185 | (uint16)InGuid.Data4[6] << 8 | 186 | InGuid.Data4[7]; 187 | 188 | return FGuid(A, B, C, D); 189 | } 190 | 191 | static FORCEINLINE winrt::guid FGUIDToGuid(const FGuid& InGuid) 192 | { 193 | // Pack the winrt guid correctly from the input FGuid data, so the value is not lost in conversion. 194 | // FGuids use 4 uint32's 195 | // winrt guids use uint32, uint16, uint16, uint8[8] 196 | // Do not simply memcpy FGuid to guid, or Data2, Data3, and Data4 will be mismatched. 197 | 198 | uint32_t Data1 = InGuid.A; 199 | uint16_t Data2 = (uint16_t)((InGuid.B & 0xFFFF0000) >> 16); 200 | uint16_t Data3 = (uint16_t)(InGuid.B & 0x0000FFFF); 201 | 202 | std::array Data4; 203 | Data4[0] = (uint8_t)((InGuid.C & 0xFF000000) >> 24); 204 | Data4[1] = (uint8_t)((InGuid.C & 0x00FF0000) >> 16); 205 | Data4[2] = (uint8_t)((InGuid.C & 0x0000FF00) >> 8); 206 | Data4[3] = (uint8_t)(InGuid.C & 0x000000FF); 207 | 208 | Data4[4] = (uint8_t)((InGuid.D & 0xFF000000) >> 24); 209 | Data4[5] = (uint8_t)((InGuid.D & 0x00FF0000) >> 16); 210 | Data4[6] = (uint8_t)((InGuid.D & 0x0000FF00) >> 8); 211 | Data4[7] = (uint8_t)(InGuid.D & 0x000000FF); 212 | 213 | return winrt::guid(Data1, Data2, Data3, Data4); 214 | } 215 | }; 216 | } 217 | 218 | 219 | #endif 220 | 221 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Public/AzureObjectAnchorTypes.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2022 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "Engine.h" 7 | #include "CoreMinimal.h" 8 | #include "UObject/ObjectMacros.h" 9 | 10 | #include "AzureObjectAnchorTypes.generated.h" 11 | 12 | UENUM(BlueprintType, Category = "MicrosoftOpenXR|Azure Object Anchors") 13 | enum class EObjectInstanceTrackingMode : uint8 14 | { 15 | LowLatencyCoarsePosition, 16 | HighLatencyAccuratePosition 17 | }; 18 | 19 | /*How to render the tracked object with the MRMesh*/ 20 | UENUM(BlueprintType, Category = "MicrosoftOpenXR|Azure Object Anchors") 21 | enum class EObjectRenderMode : uint8 22 | { 23 | /*Do not render the mesh*/ 24 | None, 25 | /*Use the full model mesh data*/ 26 | Mesh, 27 | /*Render just the bounding box*/ 28 | BoundingBox 29 | }; 30 | 31 | UCLASS(BlueprintType, Blueprintable, ClassGroup = AzureObjectAnchors, Category = "MicrosoftOpenXR|Azure Object Anchors") 32 | class MICROSOFTOPENXR_API UAzureObjectAnchorQueryModifiers : public UDataAsset 33 | { 34 | GENERATED_BODY() 35 | 36 | public: 37 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (InlineEditConditionToggle), Category = "MicrosoftOpenXR|Azure Object Anchors") 38 | bool bUseExpectedMaxVerticalOrientationInDegrees = false; 39 | 40 | /*Expected maximum angle in degrees between up direction of an object instance and gravity, from 0 to 180. 41 | Small value indicates object is expected to be up-right, while large value allows more variation on the layout.*/ 42 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (ClampMin = "0.0", ClampMax = "180.0", EditCondition = "bUseExpectedMaxVerticalOrientationInDegrees"), Category = "MicrosoftOpenXR|Azure Object Anchors") 43 | float ExpectedMaxVerticalOrientationInDegrees = 0.0f; 44 | 45 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (InlineEditConditionToggle), Category = "MicrosoftOpenXR|Azure Object Anchors") 46 | bool bUseExpectedToBeStandingOnGroundPlane = false; 47 | 48 | /*If true, only look for objects on the ground.*/ 49 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (EditCondition = "bUseExpectedToBeStandingOnGroundPlane"), Category = "MicrosoftOpenXR|Azure Object Anchors") 50 | bool ExpectedToBeStandingOnGroundPlane = true; 51 | 52 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (InlineEditConditionToggle), Category = "MicrosoftOpenXR|Azure Object Anchors") 53 | bool bUseMaxScaleChange = false; 54 | 55 | /*Maximum scale change from 1*/ 56 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (EditCondition = "bUseMaxScaleChange"), Category = "MicrosoftOpenXR|Azure Object Anchors") 57 | float MaxScaleChange = 0.0f; 58 | 59 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (InlineEditConditionToggle), Category = "MicrosoftOpenXR|Azure Object Anchors") 60 | bool bUseMinSurfaceCoverage = false; 61 | 62 | /*Minimum required surface coverage ratio to consider an object instance as true positive, from 0 to 1.*/ 63 | UPROPERTY(EditAnywhere, BlueprintReadWrite, meta = (ClampMin = "0.0", ClampMax = "1.0", EditCondition = "bUseMinSurfaceCoverage"), Category = "MicrosoftOpenXR|Azure Object Anchors") 64 | float MinSurfaceCoverage = 0.0f; 65 | }; 66 | 67 | /*Configuration for Azure Object Anchors session.*/ 68 | USTRUCT(BlueprintType, Category = "MicrosoftOpenXR|Azure Object Anchors") 69 | struct FAzureObjectAnchorSessionConfiguration 70 | { 71 | GENERATED_BODY() 72 | 73 | /*Object Anchors Account ID.*/ 74 | UPROPERTY(BlueprintReadWrite, Category = "MicrosoftOpenXR|Azure Object Anchors") 75 | FString AccountID; 76 | 77 | /*Object Anchors Account Key.*/ 78 | UPROPERTY(BlueprintReadWrite, Category = "MicrosoftOpenXR|Azure Object Anchors") 79 | FString AccountKey; 80 | 81 | /*Object Anchors Account Domain.*/ 82 | UPROPERTY(BlueprintReadWrite, Category = "MicrosoftOpenXR|Azure Object Anchors") 83 | FString AccountDomain; 84 | 85 | /*Radius around head to search for models.*/ 86 | UPROPERTY(BlueprintReadWrite, AdvancedDisplay, Category = "MicrosoftOpenXR|Azure Object Anchors") 87 | float SearchRadius = 500; 88 | 89 | /*Tracking mode for tracked objects.*/ 90 | UPROPERTY(BlueprintReadWrite, AdvancedDisplay, Category = "MicrosoftOpenXR|Azure Object Anchors") 91 | EObjectInstanceTrackingMode TrackingMode = EObjectInstanceTrackingMode::LowLatencyCoarsePosition; 92 | 93 | /*How to render the model with the MRMesh. BoundingBox or None can improve performance if the mesh is large.*/ 94 | UPROPERTY(BlueprintReadWrite, AdvancedDisplay, Category = "MicrosoftOpenXR|Azure Object Anchors") 95 | EObjectRenderMode ObjectRenderMode = EObjectRenderMode::Mesh; 96 | 97 | /*Optional query modifiers to use when locating objects.*/ 98 | UPROPERTY(BlueprintReadWrite, AdvancedDisplay, Category = "MicrosoftOpenXR|Azure Object Anchors") 99 | UAzureObjectAnchorQueryModifiers* QueryModifiers = nullptr; 100 | }; 101 | 102 | DECLARE_LOG_CATEGORY_EXTERN(LogAOA, All, All); 103 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Public/MicrosoftOpenXR.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "CoreMinimal.h" 7 | #include "UObject/ObjectMacros.h" 8 | #include "Kismet/BlueprintFunctionLibrary.h" 9 | #include "Components/InputComponent.h" 10 | 11 | #include "AzureObjectAnchorTypes.h" 12 | 13 | #include "MicrosoftOpenXR.generated.h" 14 | 15 | // Currently remoting only supports x64 Windows: Editor and Packaged Exe 16 | #define SUPPORTS_REMOTING (PLATFORM_WINDOWS && PLATFORM_64BITS) 17 | 18 | USTRUCT(BlueprintType, Category = "MicrosoftOpenXR|OpenXR") 19 | struct FKeywordInput 20 | { 21 | GENERATED_USTRUCT_BODY() 22 | 23 | UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "MicrosoftOpenXR|OpenXR") 24 | FString Keyword; 25 | 26 | UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "MicrosoftOpenXR|OpenXR") 27 | FInputActionHandlerDynamicSignature Callback; 28 | }; 29 | 30 | UENUM(BlueprintType, Category = "MicrosoftOpenXR|OpenXR") 31 | enum class EHandMeshStatus : uint8 32 | { 33 | NotInitialised = 0 UMETA(Hidden), 34 | Disabled = 1, 35 | EnabledTrackingGeometry = 2, 36 | EnabledXRVisualization = 3 37 | }; 38 | 39 | UCLASS(ClassGroup = OpenXR) 40 | class MICROSOFTOPENXR_API UMicrosoftOpenXRFunctionLibrary : 41 | public UBlueprintFunctionLibrary 42 | { 43 | GENERATED_BODY() 44 | 45 | public: 46 | 47 | /** 48 | Turn Hand Mesh 49 | 50 | @param On true if enable 51 | @return true if the command successes 52 | */ 53 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|OpenXR") 54 | static bool SetUseHandMesh(EHandMeshStatus Mode); 55 | 56 | /** 57 | Is QR Tracking enabled 58 | 59 | @return true if the command successes 60 | */ 61 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 62 | static bool IsQREnabled(); 63 | 64 | /** 65 | * Get the transform from PV camera space to Unreal world space. 66 | */ 67 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 68 | static FTransform GetPVCameraToWorldTransform(); 69 | 70 | /** 71 | * Get the PV Camera intrinsics. 72 | */ 73 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 74 | static bool GetPVCameraIntrinsics(FVector2D& focalLength, int& width, int& height, FVector2D& principalPoint, FVector& radialDistortion, FVector2D& tangentialDistortion); 75 | 76 | /** 77 | * Get a ray into the scene from a camera point. 78 | * X is left/right 79 | * Y is up/down 80 | */ 81 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 82 | static FVector GetWorldSpaceRayFromCameraPoint(FVector2D pixelCoordinate); 83 | 84 | /** 85 | Check if the current platform supports speech recognition. 86 | */ 87 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 88 | static bool IsSpeechRecognitionAvailable(); 89 | 90 | /** 91 | Add new speech keywords with associated callbacks. 92 | 93 | @param Keywords list of keyword and callbacks to add. 94 | */ 95 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|OpenXR") 96 | static void AddKeywords(TArray Keywords); 97 | 98 | /** 99 | Remove speech keywords. 100 | 101 | @param Keywords list of keyword to remove. 102 | */ 103 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|OpenXR") 104 | static void RemoveKeywords(TArray Keywords); 105 | 106 | // Helper function to use OpenXR functions from the AzureSpatialAnchors module. 107 | static bool GetPerceptionAnchorFromOpenXRAnchor(void* AnchorID, void** OutPerceptionAnchor); 108 | static bool StorePerceptionAnchor(const FString& InPinId, void* InPerceptionAnchor); 109 | 110 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 111 | static bool IsRemoting(); 112 | 113 | UFUNCTION(BlueprintPure, Category = "MicrosoftOpenXR|OpenXR") 114 | static bool CanDetectPlanes(); 115 | 116 | // Azure Object Anchors 117 | /*Toggle Azure Object Anchor detection on or off. 118 | @note After toggling on, InitAzureObjectAnchors must be called with a valid session configuration. 119 | @param bOnOff Whether to turn on or off. 120 | */ 121 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|Azure Object Anchors") 122 | static bool ToggleAzureObjectAnchors(const bool bOnOff); 123 | 124 | /*Initialize Azure Object Anchors with the input session config. This will start the Azure Object detection.*/ 125 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|Azure Object Anchors") 126 | static void InitAzureObjectAnchors(FAzureObjectAnchorSessionConfiguration AOAConfiguration); 127 | 128 | /*Clear any existing search areas and look for objects around the current pose, using the search radius from the initial session config.*/ 129 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|Azure Object Anchors") 130 | static void ResetObjectSearchAreaAroundHead(); 131 | 132 | /*Add a search area around a point in space and trigger new object detection. 133 | @param Point Point in space to search around. 134 | @param Radius Distance around point to search for objects. 135 | @param ClearExistingSearchAreas If true, clear any existing search areas and only search around this point. 136 | */ 137 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|Azure Object Anchors") 138 | static void ResetObjectSearchAreaAroundPoint(FVector Point, float Radius = 100, bool ClearExistingSearchAreas = false); 139 | 140 | /*Check for collisions against any tracked Azure Object Anchors along the input ray. */ 141 | UFUNCTION(BlueprintCallable, Category = "MicrosoftOpenXR|Azure Object Anchors") 142 | static TArray LineTraceTrackedAzureObjectAnchors(const FVector Start, const FVector End); 143 | }; 144 | 145 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXREditor/MicrosoftOpenXREditor.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using UnrealBuildTool; 5 | 6 | public class MicrosoftOpenXREditor : ModuleRules 7 | { 8 | public MicrosoftOpenXREditor(ReadOnlyTargetRules Target) : base(Target) 9 | { 10 | PrivateDependencyModuleNames.AddRange( 11 | new string[] { 12 | "Core", 13 | "CoreUObject", 14 | "InputCore", 15 | "Engine", 16 | "Slate", 17 | "SlateCore", 18 | "EditorStyle", 19 | "EditorWidgets", 20 | "DesktopWidgets", 21 | "PropertyEditor", 22 | "UnrealEd", 23 | "SharedSettingsWidgets", 24 | "TargetPlatform", 25 | "RenderCore", 26 | "MicrosoftOpenXRRuntimeSettings" 27 | } 28 | ); 29 | 30 | PrivateIncludePathModuleNames.AddRange( 31 | new string[] { 32 | "Settings" 33 | } 34 | ); 35 | } 36 | } 37 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXREditor/Private/MicrosoftOpenXRDetails.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MicrosoftOpenXRDetails.h" 5 | 6 | #define LOCTEXT_NAMESPACE "FMicrosoftOpenXRDetails" 7 | 8 | TSharedRef FMicrosoftOpenXRDetails::MakeInstance() 9 | { 10 | return MakeShareable(new FMicrosoftOpenXRDetails); 11 | } 12 | 13 | void FMicrosoftOpenXRDetails::CustomizeDetails(IDetailLayoutBuilder& DetailBuilder) 14 | { 15 | statusTextWidget = SNew(STextBlock); 16 | 17 | UMicrosoftOpenXRRuntimeSettings::Get()->OnRemotingStatusChanged.BindSP(this, &FMicrosoftOpenXRDetails::SetStatusText); 18 | 19 | IDetailCategoryBuilder& remotingCategory = DetailBuilder.EditCategory(TEXT("OpenXR Holographic Remoting")); 20 | remotingCategory.AddCustomRow(LOCTEXT("Connect Button", "Connect Button")) 21 | [ 22 | SNew(SButton) 23 | .Text(LOCTEXT("Connect", "Connect")) 24 | .OnClicked_Raw(this, &FMicrosoftOpenXRDetails::OnConnectButtonClicked) 25 | .IsEnabled_Raw(this, &FMicrosoftOpenXRDetails::AreButtonsEnabled) 26 | ]; 27 | 28 | remotingCategory.AddCustomRow(LOCTEXT("Disconnect Button", "Disconnect Button")) 29 | [ 30 | SNew(SButton) 31 | .Text(LOCTEXT("Disconnect", "Disconnect")) 32 | .OnClicked_Raw(this, &FMicrosoftOpenXRDetails::OnDisconnectButtonClicked) 33 | .IsEnabled_Raw(this, &FMicrosoftOpenXRDetails::AreButtonsEnabled) 34 | ]; 35 | 36 | remotingCategory.AddCustomRow(LOCTEXT("Status Text", "Status Text"))[statusTextWidget.ToSharedRef()]; 37 | } 38 | 39 | void FMicrosoftOpenXRDetails::SetStatusText(FString message, FLinearColor statusColor) 40 | { 41 | if (statusTextWidget == nullptr) 42 | { 43 | return; 44 | } 45 | 46 | statusTextWidget->SetText(FText::FromString(message)); 47 | statusTextWidget->SetColorAndOpacity(FSlateColor(statusColor)); 48 | } 49 | 50 | FReply FMicrosoftOpenXRDetails::OnConnectButtonClicked() 51 | { 52 | MicrosoftOpenXR::RemotingConnectionData data; 53 | UMicrosoftOpenXRRuntimeSettings::ParseAddress( 54 | UMicrosoftOpenXRRuntimeSettings::Get()->RemoteHoloLensIP, 55 | data.IP, data.Port); 56 | data.Bitrate = UMicrosoftOpenXRRuntimeSettings::Get()->MaxBitrate; 57 | data.EnableAudio = UMicrosoftOpenXRRuntimeSettings::Get()->EnableAudio; 58 | data.ConnectionType = UMicrosoftOpenXRRuntimeSettings::Get()->ConnectionType; 59 | data.ConnectionCodec = UMicrosoftOpenXRRuntimeSettings::Get()->ConnectionCodec; 60 | 61 | UMicrosoftOpenXRRuntimeSettings::Get()->OnRemotingConnect.ExecuteIfBound(data); 62 | 63 | return FReply::Handled(); 64 | } 65 | 66 | FReply FMicrosoftOpenXRDetails::OnDisconnectButtonClicked() 67 | { 68 | UMicrosoftOpenXRRuntimeSettings::Get()->OnRemotingDisconnect.ExecuteIfBound(); 69 | 70 | return FReply::Handled(); 71 | } 72 | 73 | bool FMicrosoftOpenXRDetails::AreButtonsEnabled() const 74 | { 75 | UMicrosoftOpenXRRuntimeSettings* settings = UMicrosoftOpenXRRuntimeSettings::Get(); 76 | return settings->bEnableRemotingForEditor; 77 | } 78 | 79 | #undef LOCTEXT_NAMESPACE 80 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXREditor/Private/MicrosoftOpenXRDetails.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "IDetailCustomNodeBuilder.h" 7 | #include "PropertyHandle.h" 8 | #include "IDetailCustomization.h" 9 | #include "IPropertyTypeCustomization.h" 10 | #include "PropertyCustomizationHelpers.h" 11 | 12 | #include "Widgets/Text/STextBlock.h" 13 | #include "Widgets/Input/SButton.h" 14 | 15 | #include "DetailLayoutBuilder.h" 16 | #include "DetailCategoryBuilder.h" 17 | #include "IDetailPropertyRow.h" 18 | #include "DetailWidgetRow.h" 19 | #include "IDetailGroup.h" 20 | 21 | #include "MicrosoftOpenXRRuntimeSettings.h" 22 | 23 | class FMicrosoftOpenXRDetails : public IDetailCustomization 24 | { 25 | public: 26 | static TSharedRef MakeInstance(); 27 | virtual void CustomizeDetails(IDetailLayoutBuilder& DetailBuilder) override; 28 | 29 | private: 30 | FString StatusText; 31 | TSharedPtr statusTextWidget; 32 | 33 | void SetStatusText(FString message, FLinearColor statusColor); 34 | FReply OnConnectButtonClicked(); 35 | FReply OnDisconnectButtonClicked(); 36 | bool AreButtonsEnabled() const; 37 | }; 38 | 39 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXREditor/Private/MicrosoftOpenXREditorModule.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MicrosoftOpenXRRuntimeSettings.h" 5 | #include "Modules/ModuleInterface.h" 6 | #include "ISettingsModule.h" 7 | #include "Modules/ModuleManager.h" 8 | #include "PropertyEditorModule.h" 9 | #include "MicrosoftOpenXRDetails.h" 10 | 11 | #define LOCTEXT_NAMESPACE "FMicrosoftOpenXREditorModule" 12 | 13 | 14 | /** 15 | * Module for MicrosoftOpenXR platform editor utilities 16 | */ 17 | class FMicrosoftOpenXREditorModule 18 | : public IModuleInterface 19 | { 20 | virtual void StartupModule() override 21 | { 22 | FPropertyEditorModule& PropertyModule = FModuleManager::LoadModuleChecked("PropertyEditor"); 23 | PropertyModule.RegisterCustomClassLayout(FName("MicrosoftOpenXRRuntimeSettings"), FOnGetDetailCustomizationInstance::CreateStatic(&FMicrosoftOpenXRDetails::MakeInstance)); 24 | PropertyModule.NotifyCustomizationModuleChanged(); 25 | 26 | // register settings 27 | ISettingsModule* SettingsModule = FModuleManager::GetModulePtr("Settings"); 28 | 29 | if (SettingsModule != nullptr) 30 | { 31 | SettingsModule->RegisterSettings("Project", "Platforms", "MicrosoftOpenXR", 32 | // Using "Windows Mixed Reality" here to preserve the remoting location from legacy WMR in Project Settings/ Windows Mixed Reality 33 | LOCTEXT("RuntimeSettingsName", "Windows Mixed Reality"), 34 | LOCTEXT("RuntimeSettingsDescription", "Project settings for Mixed Reality Platform Extensions"), 35 | GetMutableDefault() 36 | ); 37 | } 38 | } 39 | 40 | virtual void ShutdownModule() override 41 | { 42 | ISettingsModule* SettingsModule = FModuleManager::GetModulePtr("Settings"); 43 | 44 | if (SettingsModule != nullptr) 45 | { 46 | SettingsModule->UnregisterSettings("Project", "Platforms", "MicrosoftOpenXR"); 47 | } 48 | } 49 | }; 50 | 51 | 52 | IMPLEMENT_MODULE(FMicrosoftOpenXREditorModule, MicrosoftOpenXREditor); 53 | 54 | #undef LOCTEXT_NAMESPACE 55 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXRRuntimeSettings/MicrosoftOpenXRRuntimeSettings.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using UnrealBuildTool; 5 | 6 | public class MicrosoftOpenXRRuntimeSettings : ModuleRules 7 | { 8 | public MicrosoftOpenXRRuntimeSettings(ReadOnlyTargetRules Target) : base(Target) 9 | { 10 | PrivateDependencyModuleNames.AddRange( 11 | new string[] 12 | { 13 | "Core", 14 | "CoreUObject", 15 | "Engine" 16 | } 17 | ); 18 | 19 | if (Target.Type == TargetRules.TargetType.Editor || Target.Type == TargetRules.TargetType.Program) 20 | { 21 | PrivateDependencyModuleNames.AddRange( 22 | new string[] 23 | { 24 | "TargetPlatform" 25 | } 26 | ); 27 | } 28 | } 29 | } 30 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXRRuntimeSettings/Private/MicrosoftOpenXRRuntimeSettings.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MicrosoftOpenXRRuntimeSettings.h" 5 | #include "Misc/ConfigCacheIni.h" 6 | #include "CoreGlobals.h" 7 | #include "UObject/Package.h" 8 | 9 | UMicrosoftOpenXRRuntimeSettings* UMicrosoftOpenXRRuntimeSettings::MicrosoftOpenXRSettingsSingleton = nullptr; 10 | 11 | #if WITH_EDITOR 12 | void UMicrosoftOpenXRRuntimeSettings::PostEditChangeProperty(struct FPropertyChangedEvent& PropertyChangedEvent) 13 | { 14 | GConfig->Flush(false); 15 | } 16 | #endif 17 | 18 | UMicrosoftOpenXRRuntimeSettings* UMicrosoftOpenXRRuntimeSettings::Get() 19 | { 20 | if (MicrosoftOpenXRSettingsSingleton == nullptr && GetTransientPackage() != nullptr) 21 | { 22 | static const TCHAR* SettingsContainerName = TEXT("MicrosoftOpenXRRuntimeSettingsContainer"); 23 | 24 | MicrosoftOpenXRSettingsSingleton = FindObject(GetTransientPackage(), SettingsContainerName); 25 | 26 | if (MicrosoftOpenXRSettingsSingleton == nullptr) 27 | { 28 | MicrosoftOpenXRSettingsSingleton = NewObject( 29 | GetTransientPackage(), UMicrosoftOpenXRRuntimeSettings::StaticClass(), SettingsContainerName); 30 | MicrosoftOpenXRSettingsSingleton->AddToRoot(); 31 | } 32 | } 33 | return MicrosoftOpenXRSettingsSingleton; 34 | } 35 | 36 | bool UMicrosoftOpenXRRuntimeSettings::ParseAddress(const FString& StringToParse, FString& Address, uint32& Port) 37 | { 38 | FString PortStr; 39 | 40 | if (StringToParse.Len() == 0) 41 | { 42 | return false; 43 | } 44 | 45 | if (StringToParse.Split(TEXT(":"), &Address, &PortStr)) 46 | { 47 | // Parse the input in format "IP:Port" 48 | Port = FCString::Atoi(*PortStr); 49 | } 50 | else if (!StringToParse.Contains(".")) 51 | { 52 | // If the given ip is not valid, try using it as a port for a listen connection to the remoting player. 53 | Address = TEXT("0.0.0.0"); 54 | Port = FCString::Atoi(*StringToParse); 55 | } 56 | else 57 | { 58 | // Otherwise only an IP is set. Use the default port. 59 | Address = StringToParse; 60 | Port = 8265; 61 | } 62 | 63 | return true; 64 | } 65 | 66 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXRRuntimeSettings/Private/MicrosoftOpenXRRuntimeSettingsModule.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "Modules/ModuleInterface.h" 5 | #include "Modules/ModuleManager.h" 6 | 7 | IMPLEMENT_MODULE(FDefaultModuleImpl, MicrosoftOpenXRRuntimeSettings); 8 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXRRuntimeSettings/Public/MicrosoftOpenXRRuntimeSettings.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "CoreMinimal.h" 7 | #include "UObject/ObjectMacros.h" 8 | #include "UObject/Object.h" 9 | #include "Engine/EngineTypes.h" 10 | 11 | #include "MicrosoftOpenXRRuntimeSettings.generated.h" 12 | 13 | UENUM() 14 | enum class RemotingConnectionType 15 | { 16 | Connect = 0, 17 | Listen = 1 18 | }; 19 | 20 | // This will be cast to XrRemotingVideoCodecMSFT, IDs must match 21 | UENUM() 22 | enum class RemotingCodec 23 | { 24 | Any = 0, 25 | H264 = 1, 26 | H265 = 2 27 | }; 28 | 29 | namespace MicrosoftOpenXR 30 | { 31 | struct RemotingConnectionData 32 | { 33 | FString IP; 34 | uint32 Port = 8265; 35 | int Bitrate = 20000; 36 | bool EnableAudio = false; 37 | RemotingConnectionType ConnectionType = RemotingConnectionType::Connect; 38 | RemotingCodec ConnectionCodec = RemotingCodec::Any; 39 | }; 40 | } // namespace MicrosoftOpenXR 41 | 42 | DECLARE_DELEGATE_TwoParams(FMicrosoftOpenXRRemotingStatusChanged, FString /*RemotingMessage*/, FLinearColor /*StatusColor*/); 43 | DECLARE_DELEGATE_OneParam(FMicrosoftOpenXRRemotingConnect, MicrosoftOpenXR::RemotingConnectionData); 44 | DECLARE_DELEGATE(FMicrosoftOpenXRRemotingDisconnect); 45 | 46 | /** 47 | * Implements the settings for Microsoft OpenXR runtime platforms. 48 | */ 49 | UCLASS(config=EditorPerProjectUserSettings) 50 | class MICROSOFTOPENXRRUNTIMESETTINGS_API UMicrosoftOpenXRRuntimeSettings : public UObject 51 | { 52 | public: 53 | GENERATED_BODY() 54 | 55 | FMicrosoftOpenXRRemotingStatusChanged OnRemotingStatusChanged; 56 | FMicrosoftOpenXRRemotingConnect OnRemotingConnect; 57 | FMicrosoftOpenXRRemotingDisconnect OnRemotingDisconnect; 58 | 59 | static UMicrosoftOpenXRRuntimeSettings* Get(); 60 | 61 | static bool ParseAddress(const FString& StringToParse, FString& Address, uint32& Port); 62 | 63 | #if WITH_EDITOR 64 | virtual void PostEditChangeProperty(struct FPropertyChangedEvent& PropertyChangedEvent) override; 65 | #endif // WITH_EDITOR 66 | 67 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (ConfigRestartRequired = true, DisplayName = "Enable Remoting For Editor (Requires Restart)", Tooltip = "If true, start with a valid HMD to enable connecting via remoting. Editor restart required.")) 68 | bool bEnableRemotingForEditor = false; 69 | 70 | /** The IP of the HoloLens or WMR HMD to remote to. */ 71 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Remote Device IP.")) 72 | FString RemoteHoloLensIP; 73 | 74 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Automatically connect to remote device.")) 75 | bool bAutoConnectRemoting = false; 76 | 77 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Max network transfer rate (kb/s).")) 78 | unsigned int MaxBitrate = 20000; 79 | 80 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Use audio from PC when remoting.")) 81 | bool EnableAudio = false; 82 | 83 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Connection Type.")) 84 | RemotingConnectionType ConnectionType = RemotingConnectionType::Connect; 85 | 86 | UPROPERTY(GlobalConfig, EditAnywhere, Category = "OpenXR Holographic Remoting", Meta = (EditCondition = "bEnableRemotingForEditor", DisplayName = "Connection Codec.")) 87 | RemotingCodec ConnectionCodec = RemotingCodec::Any; 88 | 89 | private: 90 | static class UMicrosoftOpenXRRuntimeSettings* MicrosoftOpenXRSettingsSingleton; 91 | }; 92 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/NuGetModule/NuGetModule.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using System.IO; 5 | using System.Linq; 6 | using System.Runtime.Remoting.Messaging; 7 | using UnrealBuildTool; 8 | using Tools.DotNETCommon; 9 | using System; 10 | using System.Collections.Generic; 11 | 12 | public class NuGetModule : ModuleRules 13 | { 14 | public NuGetModule(ReadOnlyTargetRules Target) : base(Target) 15 | { 16 | PrecompileForTargets = PrecompileTargetsType.Any; 17 | 18 | PrivateDependencyModuleNames.AddRange( 19 | new string[] 20 | { 21 | "Core", 22 | } 23 | ); 24 | 25 | // WinRT with Nuget support 26 | if (Target.Platform == UnrealTargetPlatform.Win64 || Target.Platform == UnrealTargetPlatform.HoloLens) 27 | { 28 | // these parameters mandatory for winrt support 29 | bEnableExceptions = true; 30 | bUseUnity = false; 31 | CppStandard = CppStandardVersion.Cpp17; 32 | PublicSystemLibraries.AddRange(new string [] { "shlwapi.lib", "runtimeobject.lib" }); 33 | 34 | // prepare everything for nuget 35 | string MyModuleName = GetType().Name; 36 | string NugetFolder = Path.Combine(PluginDirectory, "Intermediate", "Nuget", MyModuleName); 37 | Directory.CreateDirectory(NugetFolder); 38 | 39 | string BinariesSubFolder = Path.Combine("Binaries", "ThirdParty", Target.Type.ToString(), Target.Platform.ToString(), Target.Architecture); 40 | 41 | PublicDefinitions.Add(string.Format("THIRDPARTY_BINARY_SUBFOLDER=\"{0}\"", BinariesSubFolder.Replace(@"\", @"\\"))); 42 | 43 | string BinariesFolder = Path.Combine(PluginDirectory, BinariesSubFolder); 44 | Directory.CreateDirectory(BinariesFolder); 45 | 46 | ExternalDependencies.Add("packages.config"); 47 | 48 | // download nuget 49 | string NugetExe = Path.Combine(NugetFolder, "nuget.exe"); 50 | if (!File.Exists(NugetExe)) 51 | { 52 | using (System.Net.WebClient myWebClient = new System.Net.WebClient()) 53 | { 54 | // we aren't focusing on a specific nuget version, we can use any of them but the latest one is preferable 55 | myWebClient.DownloadFile(@"https://dist.nuget.org/win-x86-commandline/latest/nuget.exe", NugetExe); 56 | } 57 | } 58 | 59 | // run nuget to update the packages 60 | { 61 | var StartInfo = new System.Diagnostics.ProcessStartInfo(NugetExe, string.Format("install \"{0}\" -OutputDirectory \"{1}\"", Path.Combine(ModuleDirectory, "packages.config"), NugetFolder)); 62 | StartInfo.UseShellExecute = false; 63 | StartInfo.CreateNoWindow = true; 64 | var ExitCode = Utils.RunLocalProcessAndPrintfOutput(StartInfo); 65 | if (ExitCode < 0) 66 | { 67 | throw new BuildException("Failed to get nuget packages. See log for details."); 68 | } 69 | } 70 | 71 | // get list of the installed packages, that's needed because the code should get particular versions of the installed packages 72 | string[] InstalledPackages = Utils.RunLocalProcessAndReturnStdOut(NugetExe, string.Format("list -Source \"{0}\"", NugetFolder)).Split(new char[] { '\r', '\n' }); 73 | 74 | // winmd files of the packages 75 | List WinMDFiles = new List(); 76 | 77 | // WinRT lib for some job 78 | string QRPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.MixedReality.QR")); 79 | if (!string.IsNullOrEmpty(QRPackage)) 80 | { 81 | string QRFolderName = QRPackage.Replace(" ", "."); 82 | 83 | // copying dll and winmd binaries to our local binaries folder 84 | // !!!!! please make sure that you use the path of file! Unreal can't do it for you !!!!! 85 | string WinMDFile = Path.Combine(NugetFolder, QRFolderName, @"lib\uap10.0.18362\Microsoft.MixedReality.QR.winmd"); 86 | SafeCopy(WinMDFile, Path.Combine(BinariesFolder, "Microsoft.MixedReality.QR.winmd")); 87 | 88 | SafeCopy(Path.Combine(NugetFolder, QRFolderName, string.Format(@"runtimes\win10-{0}\native\Microsoft.MixedReality.QR.dll", Target.WindowsPlatform.Architecture.ToString())), 89 | Path.Combine(BinariesFolder, "Microsoft.MixedReality.QR.dll")); 90 | 91 | // also both both binaries must be in RuntimeDependencies, unless you get failures in Hololens platform 92 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.MixedReality.QR.dll")); 93 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.MixedReality.QR.winmd")); 94 | 95 | //add winmd file to the list for further processing using cppwinrt.exe 96 | WinMDFiles.Add(WinMDFile); 97 | } 98 | 99 | string ASAPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.Azure.SpatialAnchors.WinRT")); 100 | if (!string.IsNullOrEmpty(ASAPackage)) 101 | { 102 | string ASAFolderName = ASAPackage.Replace(" ", "."); 103 | 104 | // copying dll and winmd binaries to our local binaries folder 105 | // !!!!! please make sure that you use the path of file! Unreal can't do it for you !!!!! 106 | string WinMDFile = Path.Combine(NugetFolder, ASAFolderName, @"lib\uap10.0\Microsoft.Azure.SpatialAnchors.winmd"); 107 | SafeCopy(WinMDFile, Path.Combine(BinariesFolder, "Microsoft.Azure.SpatialAnchors.winmd")); 108 | 109 | SafeCopy(Path.Combine(NugetFolder, ASAFolderName, string.Format(@"runtimes\win10-{0}\native\Microsoft.Azure.SpatialAnchors.dll", Target.WindowsPlatform.Architecture.ToString())), 110 | Path.Combine(BinariesFolder, "Microsoft.Azure.SpatialAnchors.dll")); 111 | 112 | SafeCopy(Path.Combine(NugetFolder, ASAFolderName, string.Format(@"runtimes\win10-{0}\native\CoarseRelocUW.dll", Target.WindowsPlatform.Architecture.ToString())), 113 | Path.Combine(BinariesFolder, "CoarseRelocUW.dll")); 114 | 115 | // also all binaries must be in RuntimeDependencies 116 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.Azure.SpatialAnchors.dll")); 117 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.Azure.SpatialAnchors.winmd")); 118 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "CoarseRelocUW.dll")); 119 | 120 | // add winmd file to the list for further processing using cppwinrt.exe 121 | WinMDFiles.Add(WinMDFile); 122 | } 123 | 124 | string AOAPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.Azure.ObjectAnchors.Runtime.WinRT")); 125 | if (!string.IsNullOrEmpty(AOAPackage)) 126 | { 127 | string AOAFolderName = AOAPackage.Replace(" ", "."); 128 | 129 | // Copy dll and winmd binaries to our local binaries folder 130 | string WinMDFile = Path.Combine(NugetFolder, AOAFolderName, @"lib\uap10.0\Microsoft.Azure.ObjectAnchors.winmd"); 131 | SafeCopy(WinMDFile, Path.Combine(BinariesFolder, "Microsoft.Azure.ObjectAnchors.winmd")); 132 | 133 | String[] Binaries = { 134 | "Microsoft.Azure.ObjectAnchors.dll", 135 | "ObjectTrackerApi.dll", 136 | "ObjectTrackerDiagnostics.dll", 137 | "ObjectTrackerFusion.dll", 138 | "ObjectTrackerRefinement.dll", 139 | "VolumeFusionAPI.dll" 140 | }; 141 | 142 | foreach (String Binary in Binaries) 143 | { 144 | SafeCopy(Path.Combine(NugetFolder, AOAFolderName, string.Format(@"runtimes\win10-{0}\native\{1}", Target.WindowsPlatform.Architecture.ToString(), Binary)), 145 | Path.Combine(BinariesFolder, Binary)); 146 | 147 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, Binary)); 148 | } 149 | 150 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.Azure.ObjectAnchors.winmd")); 151 | 152 | // Add winmd file to the list for further processing using cppwinrt.exe 153 | WinMDFiles.Add(WinMDFile); 154 | } 155 | 156 | if (Target.Platform == UnrealTargetPlatform.Win64) 157 | { 158 | // Microsoft.VCRTForwarders.140 is needed to run WinRT dlls in Win64 platforms 159 | string VCRTForwardersPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.VCRTForwarders.140")); 160 | if (!string.IsNullOrEmpty(VCRTForwardersPackage)) 161 | { 162 | string VCRTForwardersName = VCRTForwardersPackage.Replace(" ", "."); 163 | foreach (var Dll in Directory.EnumerateFiles(Path.Combine(NugetFolder, VCRTForwardersName, "runtimes/win10-x64/native/release"), "*_app.dll")) 164 | { 165 | string newDll = Path.Combine(BinariesFolder, Path.GetFileName(Dll)); 166 | SafeCopy(Dll, newDll); 167 | RuntimeDependencies.Add(newDll); 168 | } 169 | } 170 | 171 | string RemotingPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.Holographic.Remoting.OpenXr")); 172 | if (!string.IsNullOrEmpty(RemotingPackage)) 173 | { 174 | string RemotingFolderName = RemotingPackage.Replace(" ", "."); 175 | 176 | SafeCopy(Path.Combine(NugetFolder, RemotingFolderName, @"build\native\bin\x64\Desktop\Microsoft.Holographic.AppRemoting.OpenXr.dll"), 177 | Path.Combine(BinariesFolder, "Microsoft.Holographic.AppRemoting.OpenXr.dll")); 178 | 179 | SafeCopy(Path.Combine(NugetFolder, RemotingFolderName, @"build\native\bin\x64\Desktop\Microsoft.Holographic.AppRemoting.OpenXr.SU.dll"), 180 | Path.Combine(BinariesFolder, "Microsoft.Holographic.AppRemoting.OpenXr.SU.dll")); 181 | 182 | SafeCopy(Path.Combine(NugetFolder, RemotingFolderName, @"build\native\bin\x64\Desktop\RemotingXR.json"), 183 | Path.Combine(BinariesFolder, "RemotingXR.json")); 184 | 185 | PublicIncludePaths.Add(Path.Combine(NugetFolder, RemotingFolderName, @"build\native\include\openxr")); 186 | 187 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.Holographic.AppRemoting.OpenXr.dll")); 188 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "Microsoft.Holographic.AppRemoting.OpenXr.SU.dll")); 189 | RuntimeDependencies.Add(Path.Combine(BinariesFolder, "RemotingXR.json")); 190 | } 191 | } 192 | 193 | // get WinRT package 194 | string CppWinRTPackage = InstalledPackages.FirstOrDefault(x => x.StartsWith("Microsoft.Windows.CppWinRT")); 195 | if (!string.IsNullOrEmpty(CppWinRTPackage)) 196 | { 197 | string CppWinRTName = CppWinRTPackage.Replace(" ", "."); 198 | string CppWinRTExe = Path.Combine(NugetFolder, CppWinRTName, "bin", "cppwinrt.exe"); 199 | string CppWinRTFolder = Path.Combine(PluginDirectory, "Intermediate", CppWinRTName, MyModuleName); 200 | Directory.CreateDirectory(CppWinRTFolder); 201 | 202 | // all downloaded winmd file with WinSDK to be processed by cppwinrt.exe 203 | var WinMDFilesStringbuilder = new System.Text.StringBuilder(); 204 | foreach (var winmd in WinMDFiles) 205 | { 206 | WinMDFilesStringbuilder.Append(" -input \""); 207 | WinMDFilesStringbuilder.Append(winmd); 208 | WinMDFilesStringbuilder.Append("\""); 209 | } 210 | 211 | // generate winrt headers and add them into include paths 212 | var StartInfo = new System.Diagnostics.ProcessStartInfo(CppWinRTExe, string.Format("{0} -input \"{1}\" -output \"{2}\"", WinMDFilesStringbuilder, Target.WindowsPlatform.WindowsSdkVersion, CppWinRTFolder)); 213 | StartInfo.UseShellExecute = false; 214 | StartInfo.CreateNoWindow = true; 215 | var ExitCode = Utils.RunLocalProcessAndPrintfOutput(StartInfo); 216 | if (ExitCode < 0) 217 | { 218 | throw new BuildException("Failed to get generate WinRT headers. See log for details."); 219 | } 220 | 221 | PublicIncludePaths.Add(CppWinRTFolder); 222 | } 223 | else 224 | { 225 | // fall back to default WinSDK headers if no winrt package in our list 226 | PublicIncludePaths.Add(Path.Combine(Target.WindowsPlatform.WindowsSdkDir, "Include", Target.WindowsPlatform.WindowsSdkVersion, "cppwinrt")); 227 | } 228 | } 229 | } 230 | 231 | private void SafeCopy(string source, string destination) 232 | { 233 | if(!File.Exists(source)) 234 | { 235 | Log.TraceError("Class {0} can't find {1} file for copying", this.GetType().Name, source); 236 | return; 237 | } 238 | 239 | try 240 | { 241 | File.Copy(source, destination, true); 242 | } 243 | catch(IOException ex) 244 | { 245 | Log.TraceWarning("Failed to copy {0} to {1} with exception: {2}", source, destination, ex.Message); 246 | if (!File.Exists(destination)) 247 | { 248 | Log.TraceError("Destination file {0} does not exist", destination); 249 | return; 250 | } 251 | 252 | Log.TraceWarning("Destination file {0} already existed and is probably in use. The old file will be used for the runtime dependency. This may happen when packaging a Win64 exe from the editor.", destination); 253 | } 254 | } 255 | } 256 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/NuGetModule/Private/NuGetModule.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "NuGetModule.h" 5 | 6 | IMPLEMENT_MODULE(FNuGetModule, NuGetModule) -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/NuGetModule/Private/NuGetModule.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "CoreMinimal.h" 7 | #include "Modules/ModuleInterface.h" 8 | #include "Modules/ModuleManager.h" 9 | 10 | class FNuGetModule : public IModuleInterface 11 | { 12 | public: 13 | void StartupModule() override { } 14 | void ShutdownModule() override { } 15 | }; 16 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/NuGetModule/packages.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame.Target.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using UnrealBuildTool; 5 | using System.Collections.Generic; 6 | 7 | public class MsftOpenXRGameTarget : TargetRules 8 | { 9 | public MsftOpenXRGameTarget( TargetInfo Target) : base(Target) 10 | { 11 | Type = TargetType.Game; 12 | DefaultBuildSettings = BuildSettingsVersion.V2; 13 | ExtraModuleNames.AddRange( new string[] { "MsftOpenXRGame" } ); 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame/MsftOpenXRGame.Build.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using UnrealBuildTool; 5 | 6 | public class MsftOpenXRGame : ModuleRules 7 | { 8 | public MsftOpenXRGame(ReadOnlyTargetRules Target) : base(Target) 9 | { 10 | PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs; 11 | 12 | PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore" }); 13 | 14 | PrivateDependencyModuleNames.AddRange(new string[] { }); 15 | 16 | // Uncomment if you are using Slate UI 17 | // PrivateDependencyModuleNames.AddRange(new string[] { "Slate", "SlateCore" }); 18 | 19 | // Uncomment if you are using online features 20 | // PrivateDependencyModuleNames.Add("OnlineSubsystem"); 21 | 22 | // To include OnlineSubsystemSteam, add it to the plugins section in your uproject file with the Enabled attribute set to true 23 | } 24 | } 25 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame/MsftOpenXRGame.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MsftOpenXRGame.h" 5 | #include "Modules/ModuleManager.h" 6 | 7 | IMPLEMENT_PRIMARY_GAME_MODULE( FDefaultGameModuleImpl, MsftOpenXRGame, "MsftOpenXRGame" ); 8 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame/MsftOpenXRGame.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "CoreMinimal.h" 7 | 8 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame/MsftOpenXRGameGameModeBase.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #include "MsftOpenXRGameGameModeBase.h" 5 | 6 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGame/MsftOpenXRGameGameModeBase.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | #pragma once 5 | 6 | #include "CoreMinimal.h" 7 | #include "GameFramework/GameModeBase.h" 8 | #include "MsftOpenXRGameGameModeBase.generated.h" 9 | 10 | /** 11 | * 12 | */ 13 | UCLASS() 14 | class MSFTOPENXRGAME_API AMsftOpenXRGameGameModeBase : public AGameModeBase 15 | { 16 | GENERATED_BODY() 17 | 18 | }; 19 | -------------------------------------------------------------------------------- /MsftOpenXRGame/Source/MsftOpenXRGameEditor.Target.cs: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2020 Microsoft Corporation. 2 | // Licensed under the MIT License. 3 | 4 | using UnrealBuildTool; 5 | using System.Collections.Generic; 6 | 7 | public class MsftOpenXRGameEditorTarget : TargetRules 8 | { 9 | public MsftOpenXRGameEditorTarget( TargetInfo Target) : base(Target) 10 | { 11 | Type = TargetType.Editor; 12 | DefaultBuildSettings = BuildSettingsVersion.V2; 13 | ExtraModuleNames.AddRange( new string[] { "MsftOpenXRGame" } ); 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Microsoft OpenXR for Unreal Engine](Docs/Images/Banner.png) 2 | 3 | # What is the Microsoft OpenXR plugin? 4 | The Microsoft OpenXR plugin is an Unreal Engine game plugin which provides additional features 5 | available on Microsoft's Mixed Reality devices like the HoloLens 2 when using OpenXR. 6 | OpenXR is an open royalty-free API standard from Khronos that provides engines with native access to 7 | a wide range of devices from vendors across the mixed reality spectrum. While Unreal Engine supports 8 | OpenXR natively, some additional functionality specific to Microsoft's Mixed Reality platforms is 9 | only available through this plugin. 10 | 11 | | NOTE: The Microsoft OpenXR plugin requires **Unreal 4.26.0+**. | 12 | | --- | 13 | 14 | # Feature Summary 15 | 16 | Unreal Engine 4.26 with OpenXR provides the following built-in functionality: 17 | * Eye tracking 18 | * Hand joint tracking 19 | * Hand and controller input action mapping 20 | 21 | The Microsoft OpenXR plugin provides the following additional functionality: 22 | * Keyboard input routing 23 | * Dynamic hand mesh rendering 24 | * Spatial anchoring 25 | * Holographic Remoting from the PC 26 | * Photo/Video (PV) camera access 27 | * QR code tracking 28 | * Spatial mapping providing a mesh of the physical world 29 | * Voice input 30 | * Azure Spatial Anchors 31 | * Secondary View Configuration (requires 4.26.2+) 32 | 33 | If you're new to Mixed Reality development in Unreal, visit the 34 | [Unreal development journey](https://docs.microsoft.com/windows/mixed-reality/unreal-development-overview) 35 | in the Microsoft Docs. The Unreal development journey is specifically tailored to walk new developers 36 | through the installation, core concepts, and usage of the Microsoft OpenXR plugin. 37 | 38 | # Using the Microsoft OpenXR plugin 39 | 40 | ## Use the plugin from GitHub source 41 | 42 | 1. If you have the MicrosoftOpenXR plugin installed from the Unreal Engine Marketplace, first uninstall that plugin so it will not collide with the source version. 43 | 2. Copy MsftOpenXRGame/Plugins/MicrosoftOpenXR to your game's Plugins directory. Skip this step if using the example MsftOpenXRGame project. 44 | 3. This plugin maintains compatibility with older engine versions, so using the source directly may not work as expected in the latest engine release. 45 | If you see this message prompt, you *must* retarget the plugin to match your engine version: 46 | ![The 'MicrosoftOpenXR' plugin was designed for build 4.26.0. Attempt to load it anyway?](Docs/Images/InvalidVersion.png) 47 | 48 | Modify the ["EngineVersion"](https://github.com/microsoft/Microsoft-OpenXR-Unreal/blob/fccb12a31070bab0d45e8e948f809e6dbdde5937/MsftOpenXRGame/Plugins/MicrosoftOpenXR/MicrosoftOpenXR.uplugin#L13) string in Plugins/MicrosoftOpenXR/MicrosoftOpenXR.uplugin to match the engine version you are using. 49 | For example, if you are using UE **4.27**, change the line to: **"EngineVersion": "4.27.0"** 50 | 4. Reopen the project and the plugin will load correctly. 51 | 52 | ## Install the plugin from the Unreal Marketplace 53 | 54 | 1. Install the [Microsoft OpenXR](https://www.unrealengine.com/marketplace/product/ef8930ca860148c498b46887da196239) plugin from the Unreal Engine Marketplace to your engine. 55 | 1. Open Unreal Engine, go to **Project Settings** > **Plugins** and search for "Microsoft OpenXR". Verify that the plugin has been enabled. You may need to restart the engine for changes to take effect. 56 | > :warning: The "Microsoft Windows Mixed Reality" plugin must be disabled in your project for OpenXR to work. 57 | 58 | ## Try the example project 59 | 60 | If you want to see how each of the features available in OpenXR can be used, check out the example project (/MsftOpenXRGame) contained in this repository. This project has examples of how to use each of the features listed above, as well as a copy of the Microsoft OpenXR plugin in the MsftOpenXRGame/Plugins directory. 61 | 62 | > :warning: This example project embeds the source code for the Microsoft OpenXR plugin which conflicts with the Microsoft OpenXR plugin installed from the Unreal Marketplace. The Microsoft OpenXR plugin from the Unreal Marketplace must be uninstalled before opening the example project. This can be done by clicking the "Installed Plugins" link below the engine in your library. 63 | 64 | 1. Clone this repository. 65 | 2. Navigate to the MsftOpenXRGame folder and double-click MsftOpenXRGame.uproject to open the project in Unreal Engine. 66 | 3. See the documention for instructions on how to [stream](https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-streaming?tabs=openxr) or [package and deploy](https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-deploying) to device 67 | 68 | # Feedback and contributions 69 | If you have a feature request or run into issues using the plugin, please [file an issue](https://github.com/microsoft/Microsoft-OpenXR-Unreal/issues). 70 | 71 | This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. 72 | 73 | When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. 74 | 75 | This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments. -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)) of a security vulnerability, please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://microsoft.com/msrc/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd). 40 | 41 | 42 | --------------------------------------------------------------------------------