├── .gitignore ├── README.md ├── ardiffusionmuseum.xcodeproj ├── project.pbxproj └── project.xcworkspace │ ├── contents.xcworkspacedata │ └── xcshareddata │ ├── IDEWorkspaceChecks.plist │ └── swiftpm │ └── Package.resolved ├── ardiffusionmuseum ├── App │ ├── AppConstant.swift │ └── ardiffusionmuseumApp.swift ├── Assets.xcassets │ ├── AccentColor.colorset │ │ └── Contents.json │ ├── AppIcon.appiconset │ │ ├── Contents.json │ │ └── appIcon1024.png │ ├── Contents.json │ ├── HomeBGColor.colorset │ │ └── Contents.json │ ├── sample1.imageset │ │ ├── Contents.json │ │ └── img512.png │ ├── sample2.imageset │ │ ├── Contents.json │ │ └── Sample2.jpg │ ├── sample3.imageset │ │ ├── Contents.json │ │ └── sample3.jpg │ └── sample4.imageset │ │ ├── Contents.json │ │ └── sample4.jpg ├── Assets │ ├── frame1.usdz │ └── frame2.usdz ├── ImageGenerator │ └── ImageGenerator.swift ├── Info.plist ├── Managers │ └── AssetManager.swift ├── Preview Content │ └── Preview Assets.xcassets │ │ └── Contents.json ├── Utilities │ ├── Bundle+extensions.swift │ └── Log.swift ├── Views │ ├── AR View │ │ ├── ARContainerView.swift │ │ ├── ARContentView.swift │ │ └── ARViewController.swift │ └── ImgGen View │ │ ├── ImageListView.swift │ │ ├── ImgGenView.swift │ │ ├── ParameterView.swift │ │ ├── SettingsView.swift │ │ └── StatusView.swift └── ardiffusionmuseum.entitlements └── images ├── appIcon180.png ├── gif1_640.gif ├── ipad_960.jpg ├── relatedapps.jpg ├── ss1_1280.jpg ├── ss2_1280.jpg └── ui_960.jpg /.gitignore: -------------------------------------------------------------------------------- 1 | # Xcode 2 | # 3 | # gitignore contributors: remember to update Global/Xcode.gitignore, Objective-C.gitignore & Swift.gitignore 4 | 5 | # avoid checking CoreML models 6 | CoreMLModels 7 | merges.txt 8 | vocab.json 9 | *.mlmodelc 10 | *.mlpackage 11 | 12 | .DS_Store 13 | README.md~ 14 | .gitignore~ 15 | 16 | ## User settings 17 | xcuserdata/ 18 | 19 | ## compatibility with Xcode 8 and earlier (ignoring not required starting Xcode 9) 20 | *.xcscmblueprint 21 | *.xccheckout 22 | 23 | ## compatibility with Xcode 3 and earlier (ignoring not required starting Xcode 4) 24 | build/ 25 | DerivedData/ 26 | *.moved-aside 27 | *.pbxuser 28 | !default.pbxuser 29 | *.mode1v3 30 | !default.mode1v3 31 | *.mode2v3 32 | !default.mode2v3 33 | *.perspectivev3 34 | !default.perspectivev3 35 | 36 | ## Obj-C/Swift specific 37 | *.hmap 38 | 39 | ## App packaging 40 | *.ipa 41 | *.dSYM.zip 42 | *.dSYM 43 | 44 | ## Playgrounds 45 | timeline.xctimeline 46 | playground.xcworkspace 47 | 48 | # Swift Package Manager 49 | # 50 | # Add this line if you want to avoid checking in source code from Swift Package Manager dependencies. 51 | # Packages/ 52 | # Package.pins 53 | # Package.resolved 54 | # *.xcodeproj 55 | # 56 | # Xcode automatically generates this directory with a .xcworkspacedata file and xcuserdata 57 | # hence it is not needed unless you have added a package configuration file to your project 58 | # .swiftpm 59 | 60 | .build/ 61 | 62 | # CocoaPods 63 | # 64 | # We recommend against adding the Pods directory to your .gitignore. However 65 | # you should judge for yourself, the pros and cons are mentioned at: 66 | # https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control 67 | # 68 | # Pods/ 69 | # 70 | # Add this line if you want to avoid checking in source code from the Xcode workspace 71 | # *.xcworkspace 72 | 73 | # Carthage 74 | # 75 | # Add this line if you want to avoid checking in source code from Carthage dependencies. 76 | # Carthage/Checkouts 77 | 78 | Carthage/Build/ 79 | 80 | # Accio dependency management 81 | Dependencies/ 82 | .accio/ 83 | 84 | # fastlane 85 | # 86 | # It is recommended to not store the screenshots in the git repo. 87 | # Instead, use fastlane to re-generate the screenshots whenever they are needed. 88 | # For more information about the recommended setup visit: 89 | # https://docs.fastlane.tools/best-practices/source-control/#source-control 90 | 91 | fastlane/report.xml 92 | fastlane/Preview.html 93 | fastlane/screenshots/**/*.png 94 | fastlane/test_output 95 | 96 | # Code Injection 97 | # 98 | # After new code Injection tools there's a generated folder /iOSInjectionProject 99 | # https://github.com/johnno1962/injectionforxcode 100 | 101 | iOSInjectionProject/ 102 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # iOS app - AR Diffusion Museum 2 | 3 | An iOS app that generates images using Stable Diffusion and displays them in AR. 4 | 5 | ![AppIcon](images/appIcon180.png) 6 | 7 | You can generate images specifying any prompt (text) and display them on the wall in AR. 8 | 9 | - macOS 13.1 or newer, Xcode 14.2 or newer 10 | - iPhone Pro 12+ / iOS 16.2+, iPad Pro with M1/M2 / iPadOS 16.2+ 11 | 12 | You can run the app on above mobile devices. 13 | And you can run the app on Mac, building as a Designed for iPad app. 14 | 15 | This Xcode project uses the `Apple/ml-stable-diffusion` Swift Package. 16 | 17 | - Apple/ml-stable-diffusion Swift Package/Library 18 | - SwiftUI, ARKit, RealityKit 19 | 20 | This project does not contain the CoreML models of Stable Diffusion v2 (SD2). 21 | You need to make them converting the PyTorch SD2 models using Apple converter tools. 22 | You can find the instructions of converting models in Apple's repo on GitHub. 23 | 24 | - Apple/ml-stable-diffusion repo: https://github.com/apple/ml-stable-diffusion 25 | 26 | There is a Readme in another GitHub Repository that explains how to add Stable Diffusion CoreML models 27 | to your Xcode project. Please refer to it. 28 | 29 | - Image Generator with Stable Diffusion v2: https://github.com/ynagatomo/ImgGenSD2 30 | 31 | ## Change Log 32 | - [1.3.0 (7)] - Feb 10, 2023 `[Changed]` 33 | - changed generateImages(of:enableStableDiffusion:) function to adapt to 34 | apple/ml-stable-diffusion v0.2.0 API change on generateImages(configuration:progressHandler:) 35 | - apple/ml-stable-diffusion v0.2.0 is required. 36 | - [1.2.1 (6)] - Jan 1, 2023 `[Changed]` 37 | - changed the Guidance Scale range from 0...10 to 0...50 38 | - changed the Seed range from 0...1000 to 0...UInt32.max and added a TextField for the Seed 39 | - [1.2.0 (5)] - Dec 25, 2022 `[Added]` 40 | - added the Guidance Scale. It requires the latest apple/ml-stable-diffusion Swift Package. 41 | - [1.1.0 (4)] - Dec 21, 2022 `[Added]` 42 | - added the Negative Prompt. It requires the latest apple/ml-stable-diffusion Swift Package. 43 | - [1.0.2 (3)] - Dec 12, 2022 `[Merged]` 44 | - merged the PR by Sergey Dikarev. It uses the configuration, `MLModelConfiguration.computeUnits = .cpuAndGPU`, 45 | when creating a StableDiffusionPipeline. This configuration is suitable for mobile devices. 46 | And it adds two entitlements, `Extended Virtual Addressing` and `Increased Memory Limit`. 47 | They are suitable for an execution on iPhone. 48 | - if you encounter signing and capability errors for the entitlements, 49 | make sure that you are using the App ID which are registered the capabilities, 50 | "Extended Virtual Address Space" and "Increased Memory Limit", 51 | at Developer - Identifiers site. 52 | - not use the `.cpuAndGPU` config when running on macOS, otherwise it causes errors. 53 | - disable displaying intermediate images when running on devices, to improve the performance. 54 | 55 | - [1.0.1 (2)] - 2022-12-16 `[Changed]` 56 | - use apple/ml-stable-diffusion Swift Package v0.1.0. 57 | - set `reduceMemory` option of `StableDiffusionPipeline(resource:)` to `true`. 58 | - iPhone requirements was changed to iPhone Pro 12+. 59 | 60 | ## Features 61 | 62 | 1. image generation using Stable Diffusion v2 on device 63 | 1. showing generating images step by step 64 | 1. saving generated images in Photo Library 65 | 1. displaying generated images on the wall in AR 66 | 1. automatic switching of displayed images at regular intervals 67 | 1. automatic enlargement according to viewing distance (Large projection on outdoor walls) 68 | 1. built-in sample images 69 | 70 | ![Image](images/ss1_1280.jpg) 71 | 72 | ![Image](images/ss2_1280.jpg) 73 | 74 | ![Image](images/gif1_640.gif) 75 | 76 | ## UI 77 | 78 | This project provides a minimal UI. Feel free to extend it as you like. 79 | 80 | ![Image](images/ui_960.jpg) 81 | 82 | ## Consideration 83 | 84 | ### MPS internal error 85 | 86 | - Currently, using CoreML Stable Diffusion Library and RealityKit API such as ModelEntity.load(name:) 87 | together often causes MPS internal errors. 88 | - As a workaround, the 3D model of the picture frame is replaced with a simple one now. 89 | 90 | In action on iPad: image generation => AR display 91 | ![Image](images/ipad_960.jpg) 92 | 93 | ## Related apps 94 | 95 | There are related app's repo on GitHub. 96 | 97 | - Img Gen SD2: https://github.com/ynagatomo/ImgGenSD2 98 | - AR Wall Picture: https://github.com/ynagatomo/ARWallPicture 99 | 100 | ![Image](images/relatedapps.jpg) 101 | 102 | ## References 103 | 104 | - Apple Swift Package / ml-stable-diffusion: https://github.com/apple/ml-stable-diffusion 105 | - Stable Diffusion with Core ML on Apple Silicon, Apple Dec 2022: https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon 106 | - Deploying Transformers on the Apple Neural Engine, Apple June 2022: https://machinelearning.apple.com/research/neural-engine-transformers 107 | - Hugging Face Hub - stabilityai/stable-diffusion-2: https://huggingface.co/stabilityai/stable-diffusion-2 108 | 109 | ![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat) 110 | -------------------------------------------------------------------------------- /ardiffusionmuseum.xcodeproj/project.pbxproj: -------------------------------------------------------------------------------- 1 | // !$*UTF8*$! 2 | { 3 | archiveVersion = 1; 4 | classes = { 5 | }; 6 | objectVersion = 56; 7 | objects = { 8 | 9 | /* Begin PBXBuildFile section */ 10 | BA318C95296151A500E5917B /* CoreMLModels in Resources */ = {isa = PBXBuildFile; fileRef = BA318C94296151A500E5917B /* CoreMLModels */; }; 11 | BA34A6C529966B250062CAF4 /* StableDiffusion in Frameworks */ = {isa = PBXBuildFile; productRef = BA34A6C429966B250062CAF4 /* StableDiffusion */; }; 12 | BA3FBC3B29417FA6000F5005 /* AppConstant.swift in Sources */ = {isa = PBXBuildFile; fileRef = BA3FBC3A29417FA5000F5005 /* AppConstant.swift */; }; 13 | BA3FBC3D29419E24000F5005 /* Bundle+extensions.swift in Sources */ = {isa = PBXBuildFile; fileRef = BA3FBC3C29419E24000F5005 /* Bundle+extensions.swift */; }; 14 | BA3FBC402941DA4C000F5005 /* ImageGenerator.swift in Sources */ = {isa = PBXBuildFile; fileRef = BA3FBC3F2941DA4C000F5005 /* ImageGenerator.swift */; }; 15 | BAEF6FDF294560F400320FCC /* frame1.usdz in Resources */ = {isa = PBXBuildFile; fileRef = BAEF6FDE294560F400320FCC /* frame1.usdz */; }; 16 | BAEF6FE1294560FA00320FCC /* frame2.usdz in Resources */ = {isa = PBXBuildFile; fileRef = BAEF6FE0294560FA00320FCC /* frame2.usdz */; }; 17 | BAEF6FE42945BCF700320FCC /* AssetManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAEF6FE32945BCF700320FCC /* AssetManager.swift */; }; 18 | BAEF6FEB2945F7E700320FCC /* StatusView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAEF6FEA2945F7E700320FCC /* StatusView.swift */; }; 19 | BAEF6FED2945F8BF00320FCC /* SettingsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAEF6FEC2945F8BF00320FCC /* SettingsView.swift */; }; 20 | BAEF6FEF2945FB4700320FCC /* ParameterView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAEF6FEE2945FB4700320FCC /* ParameterView.swift */; }; 21 | BAEF6FF12945FC7900320FCC /* ImageListView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAEF6FF02945FC7900320FCC /* ImageListView.swift */; }; 22 | BAFD092D293F3757003B0D9D /* ardiffusionmuseumApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD092C293F3757003B0D9D /* ardiffusionmuseumApp.swift */; }; 23 | BAFD0931293F3758003B0D9D /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = BAFD0930293F3758003B0D9D /* Assets.xcassets */; }; 24 | BAFD0934293F3758003B0D9D /* Preview Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = BAFD0933293F3758003B0D9D /* Preview Assets.xcassets */; }; 25 | BAFD0942293F42C7003B0D9D /* ImgGenView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD0941293F42C7003B0D9D /* ImgGenView.swift */; }; 26 | BAFD0944293F4394003B0D9D /* ARContentView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD0943293F4394003B0D9D /* ARContentView.swift */; }; 27 | BAFD0946293F4403003B0D9D /* ARContainerView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD0945293F4403003B0D9D /* ARContainerView.swift */; }; 28 | BAFD0948293F4428003B0D9D /* ARViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD0947293F4428003B0D9D /* ARViewController.swift */; }; 29 | BAFD094E293F44EC003B0D9D /* Log.swift in Sources */ = {isa = PBXBuildFile; fileRef = BAFD094D293F44EC003B0D9D /* Log.swift */; }; 30 | /* End PBXBuildFile section */ 31 | 32 | /* Begin PBXFileReference section */ 33 | 5D8D3C8E294DF9EF00608B2C /* ardiffusionmuseum.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = ardiffusionmuseum.entitlements; sourceTree = ""; }; 34 | BA318C94296151A500E5917B /* CoreMLModels */ = {isa = PBXFileReference; lastKnownFileType = folder; path = CoreMLModels; sourceTree = ""; }; 35 | BA3FBC3A29417FA5000F5005 /* AppConstant.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppConstant.swift; sourceTree = ""; }; 36 | BA3FBC3C29419E24000F5005 /* Bundle+extensions.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "Bundle+extensions.swift"; sourceTree = ""; }; 37 | BA3FBC3F2941DA4C000F5005 /* ImageGenerator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ImageGenerator.swift; sourceTree = ""; }; 38 | BAEF6FDE294560F400320FCC /* frame1.usdz */ = {isa = PBXFileReference; lastKnownFileType = file.usdz; path = frame1.usdz; sourceTree = ""; }; 39 | BAEF6FE0294560FA00320FCC /* frame2.usdz */ = {isa = PBXFileReference; lastKnownFileType = file.usdz; path = frame2.usdz; sourceTree = ""; }; 40 | BAEF6FE32945BCF700320FCC /* AssetManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AssetManager.swift; sourceTree = ""; }; 41 | BAEF6FEA2945F7E700320FCC /* StatusView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StatusView.swift; sourceTree = ""; }; 42 | BAEF6FEC2945F8BF00320FCC /* SettingsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsView.swift; sourceTree = ""; }; 43 | BAEF6FEE2945FB4700320FCC /* ParameterView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ParameterView.swift; sourceTree = ""; }; 44 | BAEF6FF02945FC7900320FCC /* ImageListView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ImageListView.swift; sourceTree = ""; }; 45 | BAFD0929293F3757003B0D9D /* ardiffusionmuseum.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = ardiffusionmuseum.app; sourceTree = BUILT_PRODUCTS_DIR; }; 46 | BAFD092C293F3757003B0D9D /* ardiffusionmuseumApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ardiffusionmuseumApp.swift; sourceTree = ""; }; 47 | BAFD0930293F3758003B0D9D /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = ""; }; 48 | BAFD0933293F3758003B0D9D /* Preview Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = "Preview Assets.xcassets"; sourceTree = ""; }; 49 | BAFD093B293F3A5A003B0D9D /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist; path = Info.plist; sourceTree = ""; }; 50 | BAFD093C293F3EDA003B0D9D /* README.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = README.md; sourceTree = ""; }; 51 | BAFD0941293F42C7003B0D9D /* ImgGenView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ImgGenView.swift; sourceTree = ""; }; 52 | BAFD0943293F4394003B0D9D /* ARContentView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ARContentView.swift; sourceTree = ""; }; 53 | BAFD0945293F4403003B0D9D /* ARContainerView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ARContainerView.swift; sourceTree = ""; }; 54 | BAFD0947293F4428003B0D9D /* ARViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ARViewController.swift; sourceTree = ""; }; 55 | BAFD094D293F44EC003B0D9D /* Log.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Log.swift; sourceTree = ""; }; 56 | /* End PBXFileReference section */ 57 | 58 | /* Begin PBXFrameworksBuildPhase section */ 59 | BAFD0926293F3757003B0D9D /* Frameworks */ = { 60 | isa = PBXFrameworksBuildPhase; 61 | buildActionMask = 2147483647; 62 | files = ( 63 | BA34A6C529966B250062CAF4 /* StableDiffusion in Frameworks */, 64 | ); 65 | runOnlyForDeploymentPostprocessing = 0; 66 | }; 67 | /* End PBXFrameworksBuildPhase section */ 68 | 69 | /* Begin PBXGroup section */ 70 | BA396D152940517D00FA8260 /* Assets */ = { 71 | isa = PBXGroup; 72 | children = ( 73 | BAEF6FDE294560F400320FCC /* frame1.usdz */, 74 | BAEF6FE0294560FA00320FCC /* frame2.usdz */, 75 | ); 76 | path = Assets; 77 | sourceTree = ""; 78 | }; 79 | BA3FBC3E2941DA31000F5005 /* ImageGenerator */ = { 80 | isa = PBXGroup; 81 | children = ( 82 | BA3FBC3F2941DA4C000F5005 /* ImageGenerator.swift */, 83 | ); 84 | path = ImageGenerator; 85 | sourceTree = ""; 86 | }; 87 | BAEF6FE22945BCE100320FCC /* Managers */ = { 88 | isa = PBXGroup; 89 | children = ( 90 | BAEF6FE32945BCF700320FCC /* AssetManager.swift */, 91 | ); 92 | path = Managers; 93 | sourceTree = ""; 94 | }; 95 | BAFD0920293F3757003B0D9D = { 96 | isa = PBXGroup; 97 | children = ( 98 | BAFD093C293F3EDA003B0D9D /* README.md */, 99 | BAFD092B293F3757003B0D9D /* ardiffusionmuseum */, 100 | BAFD092A293F3757003B0D9D /* Products */, 101 | ); 102 | sourceTree = ""; 103 | }; 104 | BAFD092A293F3757003B0D9D /* Products */ = { 105 | isa = PBXGroup; 106 | children = ( 107 | BAFD0929293F3757003B0D9D /* ardiffusionmuseum.app */, 108 | ); 109 | name = Products; 110 | sourceTree = ""; 111 | }; 112 | BAFD092B293F3757003B0D9D /* ardiffusionmuseum */ = { 113 | isa = PBXGroup; 114 | children = ( 115 | 5D8D3C8E294DF9EF00608B2C /* ardiffusionmuseum.entitlements */, 116 | BA318C94296151A500E5917B /* CoreMLModels */, 117 | BA396D152940517D00FA8260 /* Assets */, 118 | BAFD093E293F4248003B0D9D /* App */, 119 | BAEF6FE22945BCE100320FCC /* Managers */, 120 | BA3FBC3E2941DA31000F5005 /* ImageGenerator */, 121 | BAFD093D293F420F003B0D9D /* Views */, 122 | BAFD094C293F44C3003B0D9D /* Utilities */, 123 | BAFD093B293F3A5A003B0D9D /* Info.plist */, 124 | BAFD0930293F3758003B0D9D /* Assets.xcassets */, 125 | BAFD0932293F3758003B0D9D /* Preview Content */, 126 | ); 127 | path = ardiffusionmuseum; 128 | sourceTree = ""; 129 | }; 130 | BAFD0932293F3758003B0D9D /* Preview Content */ = { 131 | isa = PBXGroup; 132 | children = ( 133 | BAFD0933293F3758003B0D9D /* Preview Assets.xcassets */, 134 | ); 135 | path = "Preview Content"; 136 | sourceTree = ""; 137 | }; 138 | BAFD093D293F420F003B0D9D /* Views */ = { 139 | isa = PBXGroup; 140 | children = ( 141 | BAFD093F293F4280003B0D9D /* ImgGen View */, 142 | BAFD0940293F428F003B0D9D /* AR View */, 143 | ); 144 | path = Views; 145 | sourceTree = ""; 146 | }; 147 | BAFD093E293F4248003B0D9D /* App */ = { 148 | isa = PBXGroup; 149 | children = ( 150 | BAFD092C293F3757003B0D9D /* ardiffusionmuseumApp.swift */, 151 | BA3FBC3A29417FA5000F5005 /* AppConstant.swift */, 152 | ); 153 | path = App; 154 | sourceTree = ""; 155 | }; 156 | BAFD093F293F4280003B0D9D /* ImgGen View */ = { 157 | isa = PBXGroup; 158 | children = ( 159 | BAFD0941293F42C7003B0D9D /* ImgGenView.swift */, 160 | BAEF6FEA2945F7E700320FCC /* StatusView.swift */, 161 | BAEF6FF02945FC7900320FCC /* ImageListView.swift */, 162 | BAEF6FEE2945FB4700320FCC /* ParameterView.swift */, 163 | BAEF6FEC2945F8BF00320FCC /* SettingsView.swift */, 164 | ); 165 | path = "ImgGen View"; 166 | sourceTree = ""; 167 | }; 168 | BAFD0940293F428F003B0D9D /* AR View */ = { 169 | isa = PBXGroup; 170 | children = ( 171 | BAFD0943293F4394003B0D9D /* ARContentView.swift */, 172 | BAFD0945293F4403003B0D9D /* ARContainerView.swift */, 173 | BAFD0947293F4428003B0D9D /* ARViewController.swift */, 174 | ); 175 | path = "AR View"; 176 | sourceTree = ""; 177 | }; 178 | BAFD094C293F44C3003B0D9D /* Utilities */ = { 179 | isa = PBXGroup; 180 | children = ( 181 | BAFD094D293F44EC003B0D9D /* Log.swift */, 182 | BA3FBC3C29419E24000F5005 /* Bundle+extensions.swift */, 183 | ); 184 | path = Utilities; 185 | sourceTree = ""; 186 | }; 187 | /* End PBXGroup section */ 188 | 189 | /* Begin PBXNativeTarget section */ 190 | BAFD0928293F3757003B0D9D /* ardiffusionmuseum */ = { 191 | isa = PBXNativeTarget; 192 | buildConfigurationList = BAFD0937293F3758003B0D9D /* Build configuration list for PBXNativeTarget "ardiffusionmuseum" */; 193 | buildPhases = ( 194 | BAFD0925293F3757003B0D9D /* Sources */, 195 | BAFD0926293F3757003B0D9D /* Frameworks */, 196 | BAFD0927293F3757003B0D9D /* Resources */, 197 | BAFD093A293F37D5003B0D9D /* ShellScript */, 198 | ); 199 | buildRules = ( 200 | ); 201 | dependencies = ( 202 | ); 203 | name = ardiffusionmuseum; 204 | packageProductDependencies = ( 205 | BA34A6C429966B250062CAF4 /* StableDiffusion */, 206 | ); 207 | productName = ardiffusionmuseum; 208 | productReference = BAFD0929293F3757003B0D9D /* ardiffusionmuseum.app */; 209 | productType = "com.apple.product-type.application"; 210 | }; 211 | /* End PBXNativeTarget section */ 212 | 213 | /* Begin PBXProject section */ 214 | BAFD0921293F3757003B0D9D /* Project object */ = { 215 | isa = PBXProject; 216 | attributes = { 217 | BuildIndependentTargetsInParallel = 1; 218 | LastSwiftUpdateCheck = 1410; 219 | LastUpgradeCheck = 1410; 220 | TargetAttributes = { 221 | BAFD0928293F3757003B0D9D = { 222 | CreatedOnToolsVersion = 14.1; 223 | }; 224 | }; 225 | }; 226 | buildConfigurationList = BAFD0924293F3757003B0D9D /* Build configuration list for PBXProject "ardiffusionmuseum" */; 227 | compatibilityVersion = "Xcode 14.0"; 228 | developmentRegion = en; 229 | hasScannedForEncodings = 0; 230 | knownRegions = ( 231 | en, 232 | Base, 233 | ); 234 | mainGroup = BAFD0920293F3757003B0D9D; 235 | packageReferences = ( 236 | BA34A6C329966B250062CAF4 /* XCRemoteSwiftPackageReference "ml-stable-diffusion" */, 237 | ); 238 | productRefGroup = BAFD092A293F3757003B0D9D /* Products */; 239 | projectDirPath = ""; 240 | projectRoot = ""; 241 | targets = ( 242 | BAFD0928293F3757003B0D9D /* ardiffusionmuseum */, 243 | ); 244 | }; 245 | /* End PBXProject section */ 246 | 247 | /* Begin PBXResourcesBuildPhase section */ 248 | BAFD0927293F3757003B0D9D /* Resources */ = { 249 | isa = PBXResourcesBuildPhase; 250 | buildActionMask = 2147483647; 251 | files = ( 252 | BAFD0934293F3758003B0D9D /* Preview Assets.xcassets in Resources */, 253 | BA318C95296151A500E5917B /* CoreMLModels in Resources */, 254 | BAFD0931293F3758003B0D9D /* Assets.xcassets in Resources */, 255 | BAEF6FDF294560F400320FCC /* frame1.usdz in Resources */, 256 | BAEF6FE1294560FA00320FCC /* frame2.usdz in Resources */, 257 | ); 258 | runOnlyForDeploymentPostprocessing = 0; 259 | }; 260 | /* End PBXResourcesBuildPhase section */ 261 | 262 | /* Begin PBXShellScriptBuildPhase section */ 263 | BAFD093A293F37D5003B0D9D /* ShellScript */ = { 264 | isa = PBXShellScriptBuildPhase; 265 | alwaysOutOfDate = 1; 266 | buildActionMask = 2147483647; 267 | files = ( 268 | ); 269 | inputFileListPaths = ( 270 | ); 271 | inputPaths = ( 272 | ); 273 | outputFileListPaths = ( 274 | ); 275 | outputPaths = ( 276 | ); 277 | runOnlyForDeploymentPostprocessing = 0; 278 | shellPath = /bin/sh; 279 | shellScript = "# Adds support for Apple Silicon brew directory\nexport PATH=\"$PATH:/opt/homebrew/bin\"\n\nif which swiftlint >/dev/null; then\n swiftlint\nelse\n echo \"warning: SwiftLint not installed.\"\nfi\n"; 280 | }; 281 | /* End PBXShellScriptBuildPhase section */ 282 | 283 | /* Begin PBXSourcesBuildPhase section */ 284 | BAFD0925293F3757003B0D9D /* Sources */ = { 285 | isa = PBXSourcesBuildPhase; 286 | buildActionMask = 2147483647; 287 | files = ( 288 | BAFD0944293F4394003B0D9D /* ARContentView.swift in Sources */, 289 | BA3FBC402941DA4C000F5005 /* ImageGenerator.swift in Sources */, 290 | BAEF6FEF2945FB4700320FCC /* ParameterView.swift in Sources */, 291 | BAEF6FEB2945F7E700320FCC /* StatusView.swift in Sources */, 292 | BAFD094E293F44EC003B0D9D /* Log.swift in Sources */, 293 | BAFD0946293F4403003B0D9D /* ARContainerView.swift in Sources */, 294 | BAFD092D293F3757003B0D9D /* ardiffusionmuseumApp.swift in Sources */, 295 | BAFD0942293F42C7003B0D9D /* ImgGenView.swift in Sources */, 296 | BAFD0948293F4428003B0D9D /* ARViewController.swift in Sources */, 297 | BA3FBC3D29419E24000F5005 /* Bundle+extensions.swift in Sources */, 298 | BAEF6FF12945FC7900320FCC /* ImageListView.swift in Sources */, 299 | BAEF6FED2945F8BF00320FCC /* SettingsView.swift in Sources */, 300 | BAEF6FE42945BCF700320FCC /* AssetManager.swift in Sources */, 301 | BA3FBC3B29417FA6000F5005 /* AppConstant.swift in Sources */, 302 | ); 303 | runOnlyForDeploymentPostprocessing = 0; 304 | }; 305 | /* End PBXSourcesBuildPhase section */ 306 | 307 | /* Begin XCBuildConfiguration section */ 308 | BAFD0935293F3758003B0D9D /* Debug */ = { 309 | isa = XCBuildConfiguration; 310 | buildSettings = { 311 | ALWAYS_SEARCH_USER_PATHS = NO; 312 | CLANG_ANALYZER_NONNULL = YES; 313 | CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; 314 | CLANG_CXX_LANGUAGE_STANDARD = "gnu++20"; 315 | CLANG_ENABLE_MODULES = YES; 316 | CLANG_ENABLE_OBJC_ARC = YES; 317 | CLANG_ENABLE_OBJC_WEAK = YES; 318 | CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; 319 | CLANG_WARN_BOOL_CONVERSION = YES; 320 | CLANG_WARN_COMMA = YES; 321 | CLANG_WARN_CONSTANT_CONVERSION = YES; 322 | CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; 323 | CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; 324 | CLANG_WARN_DOCUMENTATION_COMMENTS = YES; 325 | CLANG_WARN_EMPTY_BODY = YES; 326 | CLANG_WARN_ENUM_CONVERSION = YES; 327 | CLANG_WARN_INFINITE_RECURSION = YES; 328 | CLANG_WARN_INT_CONVERSION = YES; 329 | CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; 330 | CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; 331 | CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; 332 | CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; 333 | CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; 334 | CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; 335 | CLANG_WARN_STRICT_PROTOTYPES = YES; 336 | CLANG_WARN_SUSPICIOUS_MOVE = YES; 337 | CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; 338 | CLANG_WARN_UNREACHABLE_CODE = YES; 339 | CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; 340 | COPY_PHASE_STRIP = NO; 341 | DEBUG_INFORMATION_FORMAT = dwarf; 342 | ENABLE_STRICT_OBJC_MSGSEND = YES; 343 | ENABLE_TESTABILITY = YES; 344 | GCC_C_LANGUAGE_STANDARD = gnu11; 345 | GCC_DYNAMIC_NO_PIC = NO; 346 | GCC_NO_COMMON_BLOCKS = YES; 347 | GCC_OPTIMIZATION_LEVEL = 0; 348 | GCC_PREPROCESSOR_DEFINITIONS = ( 349 | "DEBUG=1", 350 | "$(inherited)", 351 | ); 352 | GCC_WARN_64_TO_32_BIT_CONVERSION = YES; 353 | GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; 354 | GCC_WARN_UNDECLARED_SELECTOR = YES; 355 | GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; 356 | GCC_WARN_UNUSED_FUNCTION = YES; 357 | GCC_WARN_UNUSED_VARIABLE = YES; 358 | IPHONEOS_DEPLOYMENT_TARGET = 16.2; 359 | MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE; 360 | MTL_FAST_MATH = YES; 361 | ONLY_ACTIVE_ARCH = YES; 362 | SDKROOT = iphoneos; 363 | SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; 364 | SWIFT_OPTIMIZATION_LEVEL = "-Onone"; 365 | }; 366 | name = Debug; 367 | }; 368 | BAFD0936293F3758003B0D9D /* Release */ = { 369 | isa = XCBuildConfiguration; 370 | buildSettings = { 371 | ALWAYS_SEARCH_USER_PATHS = NO; 372 | CLANG_ANALYZER_NONNULL = YES; 373 | CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; 374 | CLANG_CXX_LANGUAGE_STANDARD = "gnu++20"; 375 | CLANG_ENABLE_MODULES = YES; 376 | CLANG_ENABLE_OBJC_ARC = YES; 377 | CLANG_ENABLE_OBJC_WEAK = YES; 378 | CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; 379 | CLANG_WARN_BOOL_CONVERSION = YES; 380 | CLANG_WARN_COMMA = YES; 381 | CLANG_WARN_CONSTANT_CONVERSION = YES; 382 | CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; 383 | CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; 384 | CLANG_WARN_DOCUMENTATION_COMMENTS = YES; 385 | CLANG_WARN_EMPTY_BODY = YES; 386 | CLANG_WARN_ENUM_CONVERSION = YES; 387 | CLANG_WARN_INFINITE_RECURSION = YES; 388 | CLANG_WARN_INT_CONVERSION = YES; 389 | CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; 390 | CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; 391 | CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; 392 | CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; 393 | CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; 394 | CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; 395 | CLANG_WARN_STRICT_PROTOTYPES = YES; 396 | CLANG_WARN_SUSPICIOUS_MOVE = YES; 397 | CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; 398 | CLANG_WARN_UNREACHABLE_CODE = YES; 399 | CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; 400 | COPY_PHASE_STRIP = NO; 401 | DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; 402 | ENABLE_NS_ASSERTIONS = NO; 403 | ENABLE_STRICT_OBJC_MSGSEND = YES; 404 | GCC_C_LANGUAGE_STANDARD = gnu11; 405 | GCC_NO_COMMON_BLOCKS = YES; 406 | GCC_WARN_64_TO_32_BIT_CONVERSION = YES; 407 | GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; 408 | GCC_WARN_UNDECLARED_SELECTOR = YES; 409 | GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; 410 | GCC_WARN_UNUSED_FUNCTION = YES; 411 | GCC_WARN_UNUSED_VARIABLE = YES; 412 | IPHONEOS_DEPLOYMENT_TARGET = 16.2; 413 | MTL_ENABLE_DEBUG_INFO = NO; 414 | MTL_FAST_MATH = YES; 415 | SDKROOT = iphoneos; 416 | SWIFT_COMPILATION_MODE = wholemodule; 417 | SWIFT_OPTIMIZATION_LEVEL = "-O"; 418 | VALIDATE_PRODUCT = YES; 419 | }; 420 | name = Release; 421 | }; 422 | BAFD0938293F3758003B0D9D /* Debug */ = { 423 | isa = XCBuildConfiguration; 424 | buildSettings = { 425 | ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; 426 | ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor; 427 | CODE_SIGN_ENTITLEMENTS = ardiffusionmuseum/ardiffusionmuseum.entitlements; 428 | CODE_SIGN_IDENTITY = "Apple Development"; 429 | CODE_SIGN_STYLE = Automatic; 430 | CURRENT_PROJECT_VERSION = 7; 431 | DEVELOPMENT_ASSET_PATHS = "\"ardiffusionmuseum/Preview Content\""; 432 | DEVELOPMENT_TEAM = J5CY9Q9UP5; 433 | ENABLE_PREVIEWS = YES; 434 | GENERATE_INFOPLIST_FILE = YES; 435 | INFOPLIST_FILE = ardiffusionmuseum/Info.plist; 436 | INFOPLIST_KEY_CFBundleDisplayName = "AR Diff Museum"; 437 | INFOPLIST_KEY_NSCameraUsageDescription = "AR requires a camera."; 438 | INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "The app will save images in the Photo Library."; 439 | INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "The app will save images in the Photo Library."; 440 | INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES; 441 | INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES; 442 | INFOPLIST_KEY_UISupportedInterfaceOrientations = "UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown"; 443 | LD_RUNPATH_SEARCH_PATHS = ( 444 | "$(inherited)", 445 | "@executable_path/Frameworks", 446 | ); 447 | MARKETING_VERSION = 1.3.0; 448 | PRODUCT_BUNDLE_IDENTIFIER = com.atarayosd.arsdmuseum2; 449 | PRODUCT_NAME = "$(TARGET_NAME)"; 450 | PROVISIONING_PROFILE_SPECIFIER = ""; 451 | SWIFT_EMIT_LOC_STRINGS = YES; 452 | SWIFT_VERSION = 5.0; 453 | TARGETED_DEVICE_FAMILY = "1,2"; 454 | }; 455 | name = Debug; 456 | }; 457 | BAFD0939293F3758003B0D9D /* Release */ = { 458 | isa = XCBuildConfiguration; 459 | buildSettings = { 460 | ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; 461 | ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor; 462 | CODE_SIGN_ENTITLEMENTS = ardiffusionmuseum/ardiffusionmuseum.entitlements; 463 | CODE_SIGN_IDENTITY = "Apple Development"; 464 | CODE_SIGN_STYLE = Automatic; 465 | CURRENT_PROJECT_VERSION = 7; 466 | DEVELOPMENT_ASSET_PATHS = "\"ardiffusionmuseum/Preview Content\""; 467 | DEVELOPMENT_TEAM = J5CY9Q9UP5; 468 | ENABLE_PREVIEWS = YES; 469 | GENERATE_INFOPLIST_FILE = YES; 470 | INFOPLIST_FILE = ardiffusionmuseum/Info.plist; 471 | INFOPLIST_KEY_CFBundleDisplayName = "AR Diff Museum"; 472 | INFOPLIST_KEY_NSCameraUsageDescription = "AR requires a camera."; 473 | INFOPLIST_KEY_NSPhotoLibraryAddUsageDescription = "The app will save images in the Photo Library."; 474 | INFOPLIST_KEY_NSPhotoLibraryUsageDescription = "The app will save images in the Photo Library."; 475 | INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES; 476 | INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES; 477 | INFOPLIST_KEY_UISupportedInterfaceOrientations = "UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown"; 478 | LD_RUNPATH_SEARCH_PATHS = ( 479 | "$(inherited)", 480 | "@executable_path/Frameworks", 481 | ); 482 | MARKETING_VERSION = 1.3.0; 483 | PRODUCT_BUNDLE_IDENTIFIER = com.atarayosd.arsdmuseum2; 484 | PRODUCT_NAME = "$(TARGET_NAME)"; 485 | PROVISIONING_PROFILE_SPECIFIER = ""; 486 | SWIFT_EMIT_LOC_STRINGS = YES; 487 | SWIFT_VERSION = 5.0; 488 | TARGETED_DEVICE_FAMILY = "1,2"; 489 | }; 490 | name = Release; 491 | }; 492 | /* End XCBuildConfiguration section */ 493 | 494 | /* Begin XCConfigurationList section */ 495 | BAFD0924293F3757003B0D9D /* Build configuration list for PBXProject "ardiffusionmuseum" */ = { 496 | isa = XCConfigurationList; 497 | buildConfigurations = ( 498 | BAFD0935293F3758003B0D9D /* Debug */, 499 | BAFD0936293F3758003B0D9D /* Release */, 500 | ); 501 | defaultConfigurationIsVisible = 0; 502 | defaultConfigurationName = Release; 503 | }; 504 | BAFD0937293F3758003B0D9D /* Build configuration list for PBXNativeTarget "ardiffusionmuseum" */ = { 505 | isa = XCConfigurationList; 506 | buildConfigurations = ( 507 | BAFD0938293F3758003B0D9D /* Debug */, 508 | BAFD0939293F3758003B0D9D /* Release */, 509 | ); 510 | defaultConfigurationIsVisible = 0; 511 | defaultConfigurationName = Release; 512 | }; 513 | /* End XCConfigurationList section */ 514 | 515 | /* Begin XCRemoteSwiftPackageReference section */ 516 | BA34A6C329966B250062CAF4 /* XCRemoteSwiftPackageReference "ml-stable-diffusion" */ = { 517 | isa = XCRemoteSwiftPackageReference; 518 | repositoryURL = "https://github.com/apple/ml-stable-diffusion#swift-requirements"; 519 | requirement = { 520 | branch = main; 521 | kind = branch; 522 | }; 523 | }; 524 | /* End XCRemoteSwiftPackageReference section */ 525 | 526 | /* Begin XCSwiftPackageProductDependency section */ 527 | BA34A6C429966B250062CAF4 /* StableDiffusion */ = { 528 | isa = XCSwiftPackageProductDependency; 529 | package = BA34A6C329966B250062CAF4 /* XCRemoteSwiftPackageReference "ml-stable-diffusion" */; 530 | productName = StableDiffusion; 531 | }; 532 | /* End XCSwiftPackageProductDependency section */ 533 | }; 534 | rootObject = BAFD0921293F3757003B0D9D /* Project object */; 535 | } 536 | -------------------------------------------------------------------------------- /ardiffusionmuseum.xcodeproj/project.xcworkspace/contents.xcworkspacedata: -------------------------------------------------------------------------------- 1 | 2 | 4 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /ardiffusionmuseum.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | IDEDidComputeMac32BitWarning 6 | 7 | 8 | 9 | -------------------------------------------------------------------------------- /ardiffusionmuseum.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved: -------------------------------------------------------------------------------- 1 | { 2 | "pins" : [ 3 | { 4 | "identity" : "ml-stable-diffusion#swift-requirements", 5 | "kind" : "remoteSourceControl", 6 | "location" : "https://github.com/apple/ml-stable-diffusion#swift-requirements", 7 | "state" : { 8 | "branch" : "main", 9 | "revision" : "00390a64182ddd87ef368f70f3a2de9acd26cb29" 10 | } 11 | }, 12 | { 13 | "identity" : "swift-argument-parser", 14 | "kind" : "remoteSourceControl", 15 | "location" : "https://github.com/apple/swift-argument-parser.git", 16 | "state" : { 17 | "revision" : "fddd1c00396eed152c45a46bea9f47b98e59301d", 18 | "version" : "1.2.0" 19 | } 20 | } 21 | ], 22 | "version" : 2 23 | } 24 | -------------------------------------------------------------------------------- /ardiffusionmuseum/App/AppConstant.swift: -------------------------------------------------------------------------------- 1 | // 2 | // AppConstant.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/08. 6 | // 7 | 8 | import Foundation 9 | 10 | struct AppConstant { 11 | // Stable Diffusion version 12 | // - This is used only to display the version string. 13 | static let stableDiffusionVersion = "2.0" 14 | 15 | // Note: Add any prompts as you like. 16 | // swiftlint:disable line_length 17 | static let registeredPrompts = [ 18 | "", // 1st prompt should be empty ("") 19 | "futuristic Tokyo with white glass domes, tress, crowded places, sunrise, great glass buildings, shops, hotels, concept city art, detailed, hq, hyperdetailed, artstation, cgsociety, 8 k", 20 | "mushrooms growing on a lost spaceship floating through the universe, 8k high definition digital art, trending on artstation", 21 | "a portrait of woman. realist abstract. key art. cyberpunk, blue and pink, intricate artwork. 8 k octane render, trending on artstation, very coherent symmetrical artwork. cinematic, hyperrealism, very detailed, iridescent accents" 22 | ] 23 | 24 | static let defaultNegativePrompt = 25 | """ 26 | lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, multiple legs, malformation 27 | """ 28 | 29 | // Layout 30 | static let imageDisplayMaxWidth: CGFloat = 600 31 | 32 | // Keys for @AppStorage (saved in UserDefaults) 33 | static let keyARDebugOptionOn = "keyARDebugOptionOn" 34 | static let keyStableDiffusionOn = "keyStableDiffusionOn" 35 | static let keyPeopleOcclusionOn = "keyPeopleOcclusionOn" 36 | static let keyObjectOcclusionOn = "keyObjectOcclusionOn" 37 | static let keyIntervalTime = "keyIntervalTime" 38 | 39 | // Sample Image Names in app bundle. 40 | static let sampleImageNames = [ 41 | "sample1", "sample2", "sample3", "sample4" 42 | ] 43 | 44 | // Picture Frame Spec 45 | struct PictureFrameSpec { 46 | let modelName: String // USDZ model name (wo ext) 47 | let enableVisualEffect: Bool 48 | } 49 | 50 | // Note: Add or replace picture frame USDZ model as you like 51 | static let pictureFrameSpecs = [ 52 | PictureFrameSpec(modelName: "frame1", // USDZ model name 53 | enableVisualEffect: false) 54 | ] 55 | 56 | // AR Render-loopb 57 | static let scaleChangeIntervalTime: Double = 1.0 // [sec] 58 | } 59 | -------------------------------------------------------------------------------- /ardiffusionmuseum/App/ardiffusionmuseumApp.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ardiffusionmuseumApp.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import SwiftUI 9 | 10 | @main 11 | struct ARDiffusionMuseumApp: App { 12 | @StateObject private var imageGenerator = ImageGenerator() 13 | 14 | var body: some Scene { 15 | WindowGroup { 16 | ImgGenView(imageGenerator: imageGenerator) 17 | } 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/AccentColor.colorset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "colors" : [ 3 | { 4 | "idiom" : "universal" 5 | } 6 | ], 7 | "info" : { 8 | "author" : "xcode", 9 | "version" : 1 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/AppIcon.appiconset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "images" : [ 3 | { 4 | "filename" : "appIcon1024.png", 5 | "idiom" : "universal", 6 | "platform" : "ios", 7 | "size" : "1024x1024" 8 | } 9 | ], 10 | "info" : { 11 | "author" : "xcode", 12 | "version" : 1 13 | } 14 | } 15 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/AppIcon.appiconset/appIcon1024.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets.xcassets/AppIcon.appiconset/appIcon1024.png -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "info" : { 3 | "author" : "xcode", 4 | "version" : 1 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/HomeBGColor.colorset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "colors" : [ 3 | { 4 | "color" : { 5 | "color-space" : "srgb", 6 | "components" : { 7 | "alpha" : "1.000", 8 | "blue" : "0.371", 9 | "green" : "0.371", 10 | "red" : "0.371" 11 | } 12 | }, 13 | "idiom" : "universal" 14 | } 15 | ], 16 | "info" : { 17 | "author" : "xcode", 18 | "version" : 1 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample1.imageset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "images" : [ 3 | { 4 | "filename" : "img512.png", 5 | "idiom" : "universal" 6 | } 7 | ], 8 | "info" : { 9 | "author" : "xcode", 10 | "version" : 1 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample1.imageset/img512.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets.xcassets/sample1.imageset/img512.png -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample2.imageset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "images" : [ 3 | { 4 | "filename" : "Sample2.jpg", 5 | "idiom" : "universal" 6 | } 7 | ], 8 | "info" : { 9 | "author" : "xcode", 10 | "version" : 1 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample2.imageset/Sample2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets.xcassets/sample2.imageset/Sample2.jpg -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample3.imageset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "images" : [ 3 | { 4 | "filename" : "sample3.jpg", 5 | "idiom" : "universal" 6 | } 7 | ], 8 | "info" : { 9 | "author" : "xcode", 10 | "version" : 1 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample3.imageset/sample3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets.xcassets/sample3.imageset/sample3.jpg -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample4.imageset/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "images" : [ 3 | { 4 | "filename" : "sample4.jpg", 5 | "idiom" : "universal" 6 | } 7 | ], 8 | "info" : { 9 | "author" : "xcode", 10 | "version" : 1 11 | } 12 | } 13 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets.xcassets/sample4.imageset/sample4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets.xcassets/sample4.imageset/sample4.jpg -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets/frame1.usdz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets/frame1.usdz -------------------------------------------------------------------------------- /ardiffusionmuseum/Assets/frame2.usdz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/ardiffusionmuseum/Assets/frame2.usdz -------------------------------------------------------------------------------- /ardiffusionmuseum/ImageGenerator/ImageGenerator.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ImageGenerator.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/08. 6 | // 7 | 8 | import UIKit 9 | import StableDiffusion 10 | import CoreML 11 | 12 | @MainActor 13 | final class ImageGenerator: NSObject, ObservableObject { 14 | struct GenerationParameter { 15 | let prompt: String 16 | let negativePrompt: String 17 | let guidanceScale: Float 18 | let seed: Int 19 | let stepCount: Int 20 | let imageCount: Int 21 | let disableSafety: Bool 22 | } 23 | 24 | struct GeneratedImages { 25 | let parameter: GenerationParameter 26 | let images: [UIImage] 27 | } 28 | 29 | enum GenerationState { 30 | case idle // idling 31 | case generating // generating images 32 | case saving // saving images into Photo Library 33 | } 34 | 35 | @Published var generationState: GenerationState = .idle 36 | @Published var generatedImages: GeneratedImages? 37 | @Published var progressStep: (step: Int, stepCount: Int) = (0, 0) // (step, stepCount) 38 | 39 | private var sdPipeline: StableDiffusionPipeline? 40 | private var savingImageCount = 0 41 | private var savedImageCount = 0 42 | 43 | // func removeSDPipeline() { 44 | // sdPipeline = nil // to reduce memory consumption :( 45 | // } 46 | 47 | private func setState(_ state: GenerationState) { // for actor isolation 48 | generationState = state 49 | } 50 | 51 | func setPipeline(_ pipeline: StableDiffusionPipeline) { // for actor isolation 52 | sdPipeline = pipeline 53 | } 54 | 55 | private func setGeneratedImages(_ images: GeneratedImages) { // for actor isolation 56 | generatedImages = images 57 | } 58 | 59 | private func setProgressStep(step: Int, stepCount: Int) { 60 | progressStep = (step, stepCount) 61 | } 62 | } 63 | 64 | // MARK: - Stable Diffusion 65 | 66 | extension ImageGenerator { 67 | // swiftlint:disable function_body_length 68 | func generateImages(of param: GenerationParameter, enableStableDiffusion: Bool) { 69 | guard generationState == .idle else { return } 70 | 71 | if enableStableDiffusion { 72 | if param.prompt == "" { return } 73 | 74 | Task.detached(priority: .high) { 75 | await self.setState(.generating) 76 | 77 | if await self.sdPipeline == nil { 78 | 79 | // Create the StableDiffusionPipeline 80 | 81 | guard let path = Bundle.main.path(forResource: "CoreMLModels", ofType: nil, inDirectory: nil) else { 82 | fatalError("IG: Fatal error: failed to find the CoreML models.") 83 | } 84 | let resourceURL = URL(fileURLWithPath: path) 85 | 86 | let config = MLModelConfiguration() 87 | if !ProcessInfo.processInfo.isiOSAppOnMac { 88 | config.computeUnits = .cpuAndGPU 89 | } 90 | debugLog("IG: creating StableDiffusionPipeline object... resosurceURL = \(resourceURL)") 91 | 92 | // reduceMemory option was added at v0.1.0 93 | // On iOS, the reduceMemory option should be set to true 94 | let reduceMemory = ProcessInfo.processInfo.isiOSAppOnMac ? false : true 95 | if let pipeline = try? StableDiffusionPipeline( resourcesAt: resourceURL, 96 | configuration: config, reduceMemory: reduceMemory) { 97 | await self.setPipeline(pipeline) 98 | } else { 99 | fatalError("IG: Fatal error: failed to create the Stable-Diffusion-Pipeline.") 100 | } 101 | } 102 | 103 | if let sdPipeline = await self.sdPipeline { 104 | 105 | // Generate images 106 | 107 | do { 108 | debugLog("IG: generating images...") 109 | await self.setProgressStep(step: 0, stepCount: param.stepCount) 110 | // [v1.3.0] 111 | // apple/ml-stable-diffusion v0.2.0 changed the generateImages() API 112 | // to generateImages(configuration:progressHandler:) 113 | // 114 | // let cgImages = try sdPipeline.generateImages(prompt: param.prompt, 115 | // negativePrompt: param.negativePrompt, 116 | // imageCount: param.imageCount, 117 | // stepCount: param.stepCount, 118 | // seed: UInt32(param.seed), 119 | // guidanceScale: param.guidanceScale, 120 | // disableSafety: param.disableSafety, 121 | // progressHandler: self.progressHandler) 122 | 123 | // [Note] Mode: textToImage or imageToImage 124 | // when startingImage != nil AND strength < 1.0, imageToImage mode is selected 125 | var configuration = StableDiffusionPipeline.Configuration(prompt: param.prompt) 126 | configuration.negativePrompt = param.negativePrompt 127 | configuration.imageCount = param.imageCount 128 | configuration.stepCount = param.stepCount 129 | configuration.seed = UInt32(param.seed) 130 | configuration.guidanceScale = param.guidanceScale 131 | configuration.disableSafety = param.disableSafety 132 | let cgImages = try sdPipeline.generateImages(configuration: configuration, 133 | progressHandler: self.progressHandler) 134 | 135 | debugLog("IG: images were generated.") 136 | let uiImages = cgImages.compactMap { image in 137 | if let cgImage = image { return UIImage(cgImage: cgImage) 138 | } else { return nil } 139 | } 140 | await self.setGeneratedImages(GeneratedImages(parameter: param, images: uiImages)) 141 | } catch { 142 | debugLog("IG: failed to generate images.") 143 | } 144 | } 145 | 146 | await self.setState(.idle) 147 | } 148 | } else { 149 | // Stable Diffusion is disable. Create sample images. 150 | let images = GeneratedImages(parameter: GenerationParameter(prompt: "", negativePrompt: "", 151 | guidanceScale: 0.0, 152 | seed: 0, stepCount: 0, 153 | imageCount: AppConstant.sampleImageNames.count, 154 | disableSafety: false), 155 | images: AppConstant.sampleImageNames.map { UIImage(named: $0)! }) 156 | setGeneratedImages(images) 157 | } 158 | } 159 | 160 | nonisolated func progressHandler(progress: StableDiffusionPipeline.Progress) -> Bool { 161 | debugLog("IG: Progress: step / stepCount = \(progress.step) / \(progress.stepCount)") 162 | 163 | if ProcessInfo.processInfo.isiOSAppOnMac { 164 | let generatedImages = GeneratedImages(parameter: GenerationParameter(prompt: progress.prompt, 165 | negativePrompt: "unknow", // progress does not provide this now 166 | guidanceScale: 0.0, // progress does not provide this now 167 | seed: 0, 168 | stepCount: progress.stepCount, 169 | imageCount: progress.currentImages.count, 170 | disableSafety: progress.isSafetyEnabled), 171 | images: progress.currentImages.compactMap { 172 | if let cgImage = $0 { 173 | return UIImage(cgImage: cgImage) 174 | } else { 175 | return nil 176 | } 177 | }) 178 | 179 | DispatchQueue.main.async { 180 | self.setGeneratedImages(generatedImages) 181 | self.setProgressStep(step: progress.step, stepCount: progress.stepCount) 182 | } 183 | } else { 184 | DispatchQueue.main.async { 185 | self.setProgressStep(step: progress.step, stepCount: progress.stepCount) 186 | } 187 | } 188 | 189 | return true // continue 190 | } 191 | } 192 | 193 | // MARK: - Save images in the Photo Library 194 | 195 | extension ImageGenerator { 196 | func saveImagesIntoPhotoLibrary(images: [UIImage]) { 197 | guard generationState == .idle else { return } 198 | 199 | savingImageCount = images.count 200 | savedImageCount = 0 201 | generationState = .saving 202 | images.forEach { image in 203 | saveImageIntoPhotoLibrary(image: image) 204 | } 205 | } 206 | 207 | private func saveImageIntoPhotoLibrary(image: UIImage) { 208 | // Adds the specified image to the user’s Camera Roll album. 209 | UIImageWriteToSavedPhotosAlbum(image, // UIImage 210 | self, // completionTarget 211 | #selector(didFinishSaving), // completionSelector 212 | nil) // contextInfo 213 | } 214 | 215 | @objc func didFinishSaving(_ image: UIImage, didFinishSavingWithError error: Error?, 216 | contextInfo: UnsafeRawPointer) { 217 | assert(generationState == .saving) 218 | 219 | savedImageCount += 1 // including errors 220 | if savedImageCount == savingImageCount { 221 | generationState = .idle 222 | } 223 | 224 | if let error { 225 | debugLog("GEN: Error: failed to save an image to Camera Roll album.") 226 | debugLog(" - \(error.localizedDescription)") 227 | } else { 228 | debugLog("GEN: completed saving an image into Photo Library.") 229 | } 230 | } 231 | } 232 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Info.plist: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | LSMinimumSystemVersion 6 | 13.1 7 | UILaunchScreen 8 | 9 | UIColorName 10 | HomeBGColor 11 | UILaunchScreen 12 | 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Managers/AssetManager.swift: -------------------------------------------------------------------------------- 1 | // 2 | // AssetManager.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/11. 6 | // 7 | 8 | import RealityKit 9 | 10 | final class AssetManager { 11 | static let share = AssetManager() 12 | // var modelEntities: [String: ModelEntity] = [:] 13 | var modelEntity: ModelEntity? 14 | 15 | private init() { } 16 | 17 | func loadModelEntity(of name: String) -> ModelEntity? { 18 | // TODO: Remove this workaround code. 19 | // This is to deal with an internal error in MPS due to the combination of 20 | // RealityKit APIs such as ModelEntity.load(named:) and CoreML Stable Diffusion. 21 | if modelEntity == nil { 22 | let meshRes = MeshResource.generatePlane(width: 0.5, height: 0.5) 23 | let material = SimpleMaterial(color: .gray, isMetallic: false) 24 | modelEntity = ModelEntity(mesh: meshRes, materials: [material]) 25 | } 26 | 27 | return modelEntity 28 | 29 | // Original code ----------------------------------------------------- 30 | // if let modelEntity = modelEntities[name] { 31 | // return modelEntity 32 | // } 33 | // 34 | // if let modelEntity = try? ModelEntity.loadModel(named: name) { 35 | // modelEntities[name] = modelEntity 36 | // return modelEntity 37 | // } 38 | // 39 | // fatalError("Failed to load a picture frame model (\(name)).") 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Preview Content/Preview Assets.xcassets/Contents.json: -------------------------------------------------------------------------------- 1 | { 2 | "info" : { 3 | "author" : "xcode", 4 | "version" : 1 5 | } 6 | } 7 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Utilities/Bundle+extensions.swift: -------------------------------------------------------------------------------- 1 | // 2 | // Bundle+extensions.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/08. 6 | // 7 | 8 | import Foundation 9 | 10 | extension Bundle { 11 | var appName: String { 12 | return (object(forInfoDictionaryKey: "CFBundleDisplayName") as? String) ?? "" 13 | } 14 | 15 | var appVersion: String { 16 | return (object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String) ?? "" 17 | } 18 | 19 | var buildNumber: String { 20 | return (object(forInfoDictionaryKey: "CFBundleVersion") as? String) ?? "0" 21 | } 22 | 23 | var buildNumberValue: Int { 24 | return Int((object(forInfoDictionaryKey: "CFBundleVersion") as? String) ?? "0") ?? 0 25 | } 26 | } 27 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Utilities/Log.swift: -------------------------------------------------------------------------------- 1 | // 2 | // Log.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import Foundation 9 | 10 | func print(_ items: Any..., separator: String = " ", terminator: String = "\n") { 11 | Swift.print("ARW : \(items.first ?? "")") 12 | } 13 | 14 | #if !DEBUG 15 | 16 | // func print(_ items: Any..., separator: String = " ", terminator: String = "\n") { } 17 | func debugPrint(_ items: Any..., separator: String = " ", terminator: String = "\n") { } 18 | 19 | @discardableResult 20 | func dump(_ value: T, name: String? = nil, indent: Int = 0, maxDepth: Int = .max, maxItems: Int = .max) -> T { 21 | return value } 22 | 23 | func debugLog(_ items: Any..., separator: String = " ", terminator: String = "\n") { } 24 | 25 | #else 26 | 27 | func debugLog(_ items: Any..., separator: String = " ", terminator: String = "\n", 28 | function: String = #function, file: String = #file, line: Int = #line) { 29 | let url = URL(fileURLWithPath: file) 30 | let filename = url.lastPathComponent 31 | let thread = Thread.isMainThread ? "(m)" : "(-)" 32 | Swift.print("[@Log] : \(filename): \(function): L\(line): T\(thread): \(items.first ?? "")", 33 | separator: separator, terminator: terminator) 34 | } 35 | 36 | #endif 37 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/AR View/ARContainerView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ARContainerView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct ARContainerView: UIViewControllerRepresentable { 11 | let images: [UIImage] 12 | let pictureFrameIndex: Int 13 | let arDebugOptionOn: Bool 14 | let peopleOcclusionOn: Bool 15 | let objectOcclusionOn: Bool 16 | let intervalTime: Double 17 | 18 | func makeUIViewController(context: Context) -> ARViewController { 19 | let arViewController = ARViewController() 20 | arViewController.setImages(images) 21 | arViewController.setOptions(arDebugOptionOn: arDebugOptionOn, 22 | peopleOcclusionOn: peopleOcclusionOn, 23 | objectOcclusionOn: objectOcclusionOn, 24 | intervalTime: intervalTime) 25 | return arViewController 26 | } 27 | 28 | func updateUIViewController(_ uiViewController: ARViewController, context: Context) { 29 | uiViewController.update(pictureFrameIndex: pictureFrameIndex) 30 | } 31 | } 32 | 33 | // struct ARContainerView_Previews: PreviewProvider { 34 | // static var previews: some View { 35 | // ARContainerView() 36 | // } 37 | // } 38 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/AR View/ARContentView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ARContentView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct ARContentView: View { 11 | @Environment(\.dismiss) private var dismiss 12 | 13 | let images: [UIImage] 14 | let arDebugOptionOn: Bool 15 | let peopleOcclusionOn: Bool 16 | let objectOcclusionOn: Bool 17 | let intervalTime: Double 18 | 19 | var body: some View { 20 | ARContainerView(images: images, 21 | pictureFrameIndex: 0, 22 | arDebugOptionOn: arDebugOptionOn, 23 | peopleOcclusionOn: peopleOcclusionOn, 24 | objectOcclusionOn: objectOcclusionOn, 25 | intervalTime: intervalTime) 26 | .ignoresSafeArea() 27 | .overlay { 28 | VStack { 29 | HStack { 30 | Text("Face the wall to display the image.") 31 | .foregroundColor(.green) 32 | Spacer() 33 | Button(action: dismiss.callAsFunction) { 34 | Image(systemName: "xmark.circle") 35 | .font(.system(size: 32)) 36 | } 37 | } 38 | .padding() 39 | Spacer() 40 | } 41 | } 42 | } 43 | } 44 | 45 | struct ARContentView_Previews: PreviewProvider { 46 | static var previews: some View { 47 | ARContentView(images: [], 48 | arDebugOptionOn: false, 49 | peopleOcclusionOn: false, 50 | objectOcclusionOn: false, 51 | intervalTime: 5.0) 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/AR View/ARViewController.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ARViewController.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import UIKit 9 | import ARKit 10 | import RealityKit 11 | import Combine 12 | 13 | // swiftlint:disable file_length 14 | 15 | final class ARViewController: UIViewController { 16 | private var arView: ARView! 17 | private var anchorEntity: AnchorEntity? 18 | private var planeTranslation: SIMD3 = .zero 19 | private var pictureFrameModelEntity: ModelEntity? 20 | private var pictureFrameTextures: [PhysicallyBasedMaterial.Texture] = [] 21 | private var displayingTextureIndex = 0 22 | private var arSessionConfig: ARWorldTrackingConfiguration! 23 | 24 | private var renderLoopSubscription: Cancellable? 25 | private var cumulativeTimeForTexture: Double = 0 26 | private var cumulativeTimeForScale: Double = 0 27 | 28 | private var pictureFrameIndex = 0 29 | private var arDebugOptionOn = false 30 | private var peopleOcclusionOn = false 31 | private var objectOcclusionOn = false 32 | private var intervalTime: Double = 0 33 | 34 | static var peopleOcclusionSupported: Bool { 35 | ARWorldTrackingConfiguration.supportsFrameSemantics(.personSegmentationWithDepth) 36 | } 37 | static var objectOcclusionSupported: Bool { 38 | ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) 39 | } 40 | 41 | override func viewDidLoad() { 42 | super.viewDidLoad() 43 | 44 | #if !targetEnvironment(simulator) 45 | // running on a real device 46 | arView = ARView(frame: .zero, 47 | cameraMode: .ar, 48 | automaticallyConfigureSession: true) 49 | #else 50 | // running on a simulator 51 | arView = ARView(frame: .zero) 52 | #endif 53 | view = arView 54 | 55 | #if DEBUG 56 | if arDebugOptionOn { 57 | arView.debugOptions = [ // .showAnchorOrigins, 58 | // .showPhysics : collision shapes 59 | .showStatistics, 60 | .showWorldOrigin 61 | // .showAnchorGeometry, 62 | // .showFeaturePoints 63 | ] 64 | } 65 | #endif 66 | } 67 | 68 | override func viewDidAppear(_ animated: Bool) { 69 | super.viewDidAppear(animated) 70 | arView.session.delegate = self 71 | startPlaneDetectionARSession() 72 | } 73 | 74 | override func viewWillDisappear(_ animated: Bool) { 75 | super.viewWillDisappear(animated) 76 | stopSession() 77 | arView.session.pause() 78 | } 79 | } 80 | 81 | // MARK: - Setup 82 | 83 | extension ARViewController { 84 | func setImages(_ images: [UIImage]) { 85 | images.forEach { 86 | if let cgImage = $0.cgImage { 87 | let options = TextureResource.CreateOptions(semantic: .color) 88 | if let textureRes = try? TextureResource.generate(from: cgImage, 89 | options: options) { 90 | let texture = PhysicallyBasedMaterial.Texture(textureRes) 91 | pictureFrameTextures.append(texture) 92 | } 93 | } 94 | } 95 | } 96 | 97 | func setOptions(arDebugOptionOn: Bool, peopleOcclusionOn: Bool, objectOcclusionOn: Bool, intervalTime: Double) { 98 | self.arDebugOptionOn = arDebugOptionOn 99 | self.peopleOcclusionOn = peopleOcclusionOn 100 | self.objectOcclusionOn = objectOcclusionOn 101 | self.intervalTime = intervalTime 102 | } 103 | } 104 | 105 | // MARK: - Render Loop 106 | 107 | extension ARViewController { 108 | private func stopSession() { 109 | renderLoopSubscription = nil 110 | } 111 | 112 | private func startSession() { 113 | renderLoopSubscription = arView.scene.subscribe(to: SceneEvents.Update.self) { event in 114 | DispatchQueue.main.async { 115 | self.updateScene(deltaTime: event.deltaTime) 116 | } 117 | } 118 | } 119 | 120 | private func updateScene(deltaTime: Double) { 121 | cumulativeTimeForTexture += deltaTime 122 | cumulativeTimeForScale += deltaTime 123 | 124 | // change the scale 125 | 126 | if cumulativeTimeForScale > AppConstant.scaleChangeIntervalTime { 127 | cumulativeTimeForScale = 0 128 | let diff = arView.cameraTransform.translation - planeTranslation 129 | let distance = sqrtf(diff.x * diff.x + diff.y * diff.y + diff.z * diff.z) 130 | // debugLog("DEBUG: distance = \(squareDistance)") 131 | if distance > 3.0 { 132 | let scale = distance - 3 + 1 133 | pictureFrameModelEntity?.scale = SIMD3(repeating: scale) 134 | } 135 | } 136 | 137 | // change the texture 138 | 139 | guard pictureFrameTextures.count >= 2 else { return } 140 | 141 | if cumulativeTimeForTexture > intervalTime { 142 | cumulativeTimeForTexture = 0 143 | displayingTextureIndex = displayingTextureIndex >= (pictureFrameTextures.count - 1) 144 | ? 0 : (displayingTextureIndex + 1) 145 | var material = UnlitMaterial() 146 | material.color.texture = pictureFrameTextures[displayingTextureIndex] 147 | pictureFrameModelEntity?.model?.materials[0] = material 148 | } 149 | } 150 | } 151 | 152 | // MARK: - Update 153 | 154 | extension ARViewController { 155 | func update(pictureFrameIndex: Int) { 156 | assert(pictureFrameIndex >= 0 157 | && pictureFrameIndex < AppConstant.pictureFrameSpecs.count) 158 | 159 | self.pictureFrameIndex = pictureFrameIndex 160 | pictureFrameModelEntity = loadPictureFrameModel(of: pictureFrameIndex) 161 | } 162 | 163 | private func loadPictureFrameModel(of frameIndex: Int) -> ModelEntity? { 164 | assert(pictureFrameIndex >= 0 165 | && pictureFrameIndex < AppConstant.pictureFrameSpecs.count) 166 | 167 | return AssetManager.share.loadModelEntity(of: 168 | AppConstant.pictureFrameSpecs[frameIndex].modelName) 169 | } 170 | 171 | private func addPictureEntity() { 172 | guard let anchorEntity else { return } 173 | 174 | // let meshRes = MeshResource.generateBox(size: 0.5) 175 | // let material = SimpleMaterial(color: .red, isMetallic: false) 176 | // let model = ModelEntity(mesh: meshRes, materials: [material]) 177 | // anchorEntity.addChild(model) 178 | 179 | if let pictureFrameModelEntity { 180 | if !pictureFrameTextures.isEmpty { 181 | var material = UnlitMaterial() 182 | material.color.texture = pictureFrameTextures[displayingTextureIndex] 183 | pictureFrameModelEntity.model?.materials[0] = material 184 | } 185 | 186 | pictureFrameModelEntity.scale = SIMD3(1, 1, 1) 187 | anchorEntity.addChild(pictureFrameModelEntity) 188 | pictureFrameModelEntity.orientation = simd_quatf(angle: Float.pi / 2, 189 | axis: SIMD3(1, 0, 0)) 190 | * simd_quatf(angle: Float.pi, axis: SIMD3(0, 1, 0)) 191 | * simd_quatf(angle: Float.pi, axis: SIMD3(0, 0, 1)) 192 | } 193 | } 194 | } 195 | 196 | // MARK: - ARSession 197 | 198 | extension ARViewController { 199 | private func startPlaneDetectionARSession() { 200 | assert(arSessionConfig == nil) 201 | #if !targetEnvironment(simulator) 202 | // running on an real devices 203 | arSessionConfig = ARWorldTrackingConfiguration() 204 | arSessionConfig.planeDetection = [.vertical] 205 | 206 | if peopleOcclusionOn { 207 | if Self.peopleOcclusionSupported { 208 | arSessionConfig.frameSemantics.insert(.personSegmentationWithDepth) 209 | debugLog("AR: People Occlusion was enabled.") 210 | } else { 211 | debugLog("AR: This device does not support People Occlusion.") 212 | } 213 | } 214 | 215 | // [Note] 216 | // When you enable scene reconstruction, ARKit provides a polygonal mesh 217 | // that estimates the shape of the physical environment. 218 | // If you enable plane detection, ARKit applies that information to the mesh. 219 | // Where the LiDAR scanner may produce a slightly uneven mesh on a real-world surface, 220 | // ARKit smooths out the mesh where it detects a plane on that surface. 221 | // If you enable people occlusion, ARKit adjusts the mesh according to any people 222 | // it detects in the camera feed. ARKit removes any part of the scene mesh that 223 | // overlaps with people 224 | if objectOcclusionOn { 225 | if Self.objectOcclusionSupported { 226 | arSessionConfig.sceneReconstruction = .mesh 227 | arView.environment.sceneUnderstanding.options.insert(.occlusion) 228 | debugLog("AR: Object Occlusion was enabled.") 229 | } else { 230 | debugLog("AR: This device does not support Object Occlusion.") 231 | } 232 | } 233 | 234 | arView.session.run(arSessionConfig) 235 | #else 236 | // running on a simulator => do nothing 237 | #endif 238 | } 239 | 240 | private func startNonPlaneDetectionARSession() { 241 | assert(arSessionConfig != nil) 242 | #if !targetEnvironment(simulator) 243 | // running on an real devices 244 | arSessionConfig.planeDetection = [] 245 | arView.session.run(arSessionConfig, options: []) 246 | #else 247 | // running on a simulator => do nothing 248 | #endif 249 | } 250 | } 251 | 252 | // MARK: - ARSessionDelegate 253 | 254 | extension ARViewController: ARSessionDelegate { 255 | #if !targetEnvironment(simulator) 256 | /// tells that ARAnchors was added cause of like a plane-detection 257 | func session(_ session: ARSession, didAdd anchors: [ARAnchor]) { 258 | // debugLog("AR: AR-DELEGATE: didAdd anchors: [ARAnchor] : \(anchors)") 259 | // can be added for environmentTexturing 260 | 261 | // [@Log] : ARViewController.swift: session(_:didAdd:): L75: T(m): AR: AR-DELEGATE: 262 | // didAdd anchors: [ARAnchor] : [ 264 | // alignment=vertical center=(-0.075000 0.000000 0.075000) extent=(0.750000 0.000000 0.950000) 265 | // classification=Wall>] 266 | 267 | for anchor in anchors { 268 | if let arPlaneAnchor = anchor as? ARPlaneAnchor { 269 | // debugLog("AR: AR-DELEGATE: didAdd an ARPlaneAnchor : \(arPlaneAnchor)") 270 | planeTranslation = SIMD3(arPlaneAnchor.transform[3].x, 271 | arPlaneAnchor.transform[3].y, 272 | arPlaneAnchor.transform[3].z) 273 | anchorEntity = AnchorEntity(anchor: arPlaneAnchor) 274 | 275 | arView.scene.addAnchor(anchorEntity!) 276 | addPictureEntity() 277 | startNonPlaneDetectionARSession() 278 | startSession() 279 | break 280 | } 281 | } 282 | } 283 | #endif 284 | 285 | // /// tells that ARAnchors were changed cause of like a progress of plane-detection 286 | // func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) { 287 | // debugLog("AR: AR-DELEGATE: ARSessionDelegate: session(_:didUpdate) was called. \(anchors) were updated.") 288 | // // can be added for environmentTexturing 289 | // 290 | // // [@Log] : ARViewController.swift: session(_:didUpdate:): L81: T(m): AR: AR-DELEGATE: 291 | // // ARSessionDelegate: session(_:didUpdate) was called. [ alignment=vertical center=(0.150000 0.000000 -0.050000) 294 | // // extent=(0.700000 0.000000 0.400000) classification=Wall> 295 | // } 296 | 297 | // /// tells that the ARAnchors were removed 298 | // func session(_ session: ARSession, didRemove anchors: [ARAnchor]) { 299 | // // debugLog("AR: AR-DELEGATE: The session(_:didRemove) was called. [ARAnchor] were removed.") 300 | // assertionFailure("The session(_:didUpdate) should not be called.") 301 | // } 302 | 303 | // /// tells that the AR session was interrupted due to app switching or something 304 | // func sessionWasInterrupted(_ session: ARSession) { 305 | // debugLog("AR: AR-DELEGATE: The sessionWasInterrupted(_:) was called.") 306 | // // Nothing to do. The system handles all. 307 | // 308 | // // DispatchQueue.main.async { 309 | // // - do something if necessary 310 | // // } 311 | // } 312 | 313 | // /// tells that the interruption was ended 314 | // func sessionInterruptionEnded(_ session: ARSession) { 315 | // debugLog("AR: AR-DELEGATE: The sessionInterruptionEnded(_:) was called.") 316 | // // Nothing to do. The system handles all. 317 | // 318 | // // DispatchQueue.main.async { 319 | // // - reset the AR tracking 320 | // // - do something if necessary 321 | // // } 322 | // } 323 | 324 | // func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) { 325 | // swiftlint:disable line_length 326 | // debugLog("AR: AR-DELEGATE: The session(_:cameraDidChangeTrackingState:) was called. cameraState = \(camera.trackingState)") 327 | // } 328 | 329 | // func session(_ session: ARSession, didUpdate frame: ARFrame) { 330 | // // You can get the camera's (device's) position in the virtual space 331 | // // from the transform property. 332 | // // The 4th column represents the position, (x, y, z, -). 333 | // let cameraTransform = frame.camera.transform 334 | // // The orientation of the camera, expressed as roll, pitch, and yaw values. 335 | // let cameraEulerAngles = frame.camera.eulerAngles // simd_float3 336 | // } 337 | 338 | // tells that an error was occurred 339 | // 340 | // - When the users don't allow to access the camera, this delegate will be called. 341 | // swiftlint:disable cyclomatic_complexity 342 | func session(_ session: ARSession, didFailWithError error: Error) { 343 | debugLog("AR: AR-DELEGATE: The didFailWithError was called.") 344 | debugLog("AR: AR-DELEGATE: error = \(error.localizedDescription)") 345 | guard let arerror = error as? ARError else { return } 346 | 347 | #if DEBUG 348 | // print the errorCase 349 | let errorCase: String 350 | switch arerror.errorCode { 351 | case ARError.Code.requestFailed.rawValue: errorCase = "requestFailed" 352 | case ARError.Code.cameraUnauthorized.rawValue: errorCase = "cameraUnauthorized" 353 | case ARError.Code.fileIOFailed.rawValue: errorCase = "fileIOFailed" 354 | case ARError.Code.insufficientFeatures.rawValue: errorCase = "insufficientFeatures" 355 | case ARError.Code.invalidConfiguration.rawValue: errorCase = "invalidConfiguration" 356 | case ARError.Code.invalidReferenceImage.rawValue: errorCase = "invalidReferenceImage" 357 | case ARError.Code.invalidReferenceObject.rawValue: errorCase = "invalidReferenceObject" 358 | case ARError.Code.invalidWorldMap.rawValue: errorCase = "invalidWorldMap" 359 | case ARError.Code.microphoneUnauthorized.rawValue: errorCase = "microphoneUnauthorized" 360 | case ARError.Code.objectMergeFailed.rawValue: errorCase = "objectMergeFailed" 361 | case ARError.Code.sensorFailed.rawValue: errorCase = "sensorFailed" 362 | case ARError.Code.sensorUnavailable.rawValue: errorCase = "sensorUnavailable" 363 | case ARError.Code.unsupportedConfiguration.rawValue: errorCase = "unsupportedConfiguration" 364 | case ARError.Code.worldTrackingFailed.rawValue: errorCase = "worldTrackingFailed" 365 | case ARError.Code.geoTrackingFailed.rawValue: errorCase = "geoTrackingFailed" 366 | case ARError.Code.geoTrackingNotAvailableAtLocation.rawValue: errorCase = "geoTrackingNotAvailableAtLocation" 367 | case ARError.Code.locationUnauthorized.rawValue: errorCase = "locationUnauthorized" 368 | case ARError.Code.invalidCollaborationData.rawValue: errorCase = "invalidCollaborationData" 369 | default: errorCase = "unknown" 370 | } 371 | debugLog("AR: AR-DELEGATE: errorCase = \(errorCase)") 372 | 373 | // print the errorWithInfo 374 | let errorWithInfo = error as NSError 375 | let messages = [ 376 | errorWithInfo.localizedDescription, 377 | errorWithInfo.localizedFailureReason, 378 | errorWithInfo.localizedRecoverySuggestion 379 | ] 380 | // remove optional error messages and connect into one string 381 | let errorMessage = messages.compactMap({ $0 }).joined(separator: "\n") 382 | debugLog("AR: AR-DELEGATE: errorWithInfo: \(errorMessage)") 383 | #endif 384 | 385 | // handle the issues 386 | if arerror.errorCode == ARError.Code.cameraUnauthorized.rawValue { 387 | // Error: The camera access is not allowed. 388 | debugLog("AR: AR-DELEGATE: The camera access is not authorized.") 389 | 390 | // Show the alert message. 391 | // "The use of the camera is not permitted.\nPlease allow it with the Settings app." 392 | } else if arerror.errorCode == ARError.Code.unsupportedConfiguration.rawValue { 393 | // Error: Unsupported Configuration 394 | // It means that now the AR session is trying to run on macOS(w/M1) or Simulator. 395 | debugLog("AR: AR-DELEGATE: unsupportedConfiguration. (running on macOS or Simulator)") 396 | assertionFailure("invalid ARSession on macOS or Simulator.") 397 | // Nothing to do in release mode. 398 | } else { 399 | // Error: Something else 400 | // Nothing to do. 401 | } 402 | } 403 | } 404 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/ImgGen View/ImageListView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ImageListView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/11. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct ImageListView: View { 11 | let images: [UIImage]? 12 | 13 | var body: some View { 14 | VStack { 15 | if let images { 16 | Color.clear 17 | .padding(.top, 50) 18 | 19 | ForEach(images, id: \.self) { 20 | Image(uiImage: $0) 21 | .resizable() 22 | .scaledToFit() 23 | .frame(maxWidth: AppConstant.imageDisplayMaxWidth) 24 | } 25 | 26 | Color.clear 27 | .padding(.top, 100) 28 | } else { 29 | Color.clear 30 | .padding(.top, 50) 31 | Text("no image") 32 | .frame(maxWidth: AppConstant.imageDisplayMaxWidth) 33 | } 34 | } // VStack 35 | } 36 | } 37 | 38 | struct ImageListView_Previews: PreviewProvider { 39 | static let images: [UIImage] = [UIImage(named: "sample1")!] 40 | 41 | static var previews: some View { 42 | ImageListView(images: images) 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/ImgGen View/ImgGenView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ImgGenView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/06. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct ImgGenView: View { 11 | @ObservedObject var imageGenerator: ImageGenerator 12 | 13 | @AppStorage(AppConstant.keyARDebugOptionOn) var arDebugOptionOn = true 14 | @AppStorage(AppConstant.keyStableDiffusionOn) var stableDiffusionOn = true 15 | @AppStorage(AppConstant.keyPeopleOcclusionOn) var peopleOcclusionOn = false 16 | @AppStorage(AppConstant.keyObjectOcclusionOn) var objectOcclusionOn = false 17 | @AppStorage(AppConstant.keyIntervalTime) var intervalTime: Double = 5.0 18 | 19 | @State private var showingSettings = false 20 | @State private var showingAR = false 21 | @State private var showingGenPanel = false 22 | 23 | struct GenParameter { 24 | var prompt: String = "" 25 | var negativePrompt: String = AppConstant.defaultNegativePrompt 26 | var guidanceScale: Float = 8.0 27 | var imageCount: Int = 1 28 | var stepCount: Double = 20 29 | var seed: Double = 100 30 | var randomSeed = true 31 | } 32 | 33 | @State private var genParameter = GenParameter() 34 | @State private var willGenerate = false 35 | 36 | private var isGeneratorIdle: Bool { 37 | imageGenerator.generationState == .idle 38 | } 39 | 40 | private var imageExists: Bool { 41 | imageGenerator.generatedImages != nil 42 | } 43 | 44 | var body: some View { 45 | ZStack { 46 | Color("HomeBGColor") 47 | 48 | VStack { 49 | ScrollView(showsIndicators: false) { 50 | VStack { 51 | ImageListView(images: imageGenerator.generatedImages?.images) 52 | 53 | Spacer() 54 | } 55 | } // ScrollView 56 | .padding(.horizontal, 20) 57 | } // VStack 58 | .overlay { 59 | VStack { 60 | HStack { 61 | Button(action: { showingSettings = true }, label: { 62 | Image(systemName: "gear") 63 | .foregroundColor(.blue) 64 | .font(.title) 65 | }) 66 | .padding(.top, 40) 67 | 68 | Spacer() 69 | } 70 | 71 | Spacer() 72 | StatusView(generating: !isGeneratorIdle, 73 | progressStep: imageGenerator.progressStep) 74 | Spacer() 75 | 76 | HStack { 77 | Button(action: saveImages) { 78 | Image(systemName: "square.and.arrow.down.on.square") 79 | .font(.title2) 80 | } 81 | .buttonStyle(.borderedProminent) 82 | .disabled(!isGeneratorIdle || !imageExists) 83 | 84 | Spacer() 85 | 86 | Button(action: showgenerationPanel, label: { 87 | Text("Generate") 88 | .font(.title2) 89 | }) 90 | .buttonStyle(.borderedProminent) 91 | .disabled(!isGeneratorIdle) 92 | 93 | Spacer() 94 | 95 | Button(action: displayAR, label: { 96 | Image(systemName: "arkit") 97 | .font(.title2) 98 | }) 99 | .buttonStyle(.borderedProminent) 100 | .disabled(ProcessInfo.processInfo.isiOSAppOnMac 101 | || !imageExists 102 | || !isGeneratorIdle) 103 | } 104 | } 105 | .padding(24) 106 | } 107 | .fullScreenCover(isPresented: $showingAR) { 108 | if let images = imageGenerator.generatedImages?.images { 109 | ARContentView(images: images, 110 | arDebugOptionOn: arDebugOptionOn, 111 | peopleOcclusionOn: peopleOcclusionOn, 112 | objectOcclusionOn: objectOcclusionOn, 113 | intervalTime: intervalTime) 114 | } else { 115 | Text("no content") 116 | } 117 | } 118 | .sheet(isPresented: $showingGenPanel, 119 | onDismiss: { 120 | if willGenerate { 121 | willGenerate = false 122 | generate() 123 | } 124 | }, content: { 125 | ParameterView(param: $genParameter, 126 | willGenerate: $willGenerate) 127 | .ignoresSafeArea() 128 | .presentationDetents([.medium, .large]) 129 | }) 130 | .sheet(isPresented: $showingSettings, 131 | onDismiss: { 132 | }, content: { 133 | SettingsView( 134 | stableDiffusionOn: $stableDiffusionOn, 135 | peopleOcclusionOn: $peopleOcclusionOn, 136 | objectOcclusionOn: $objectOcclusionOn, 137 | arDebugOptionOn: $arDebugOptionOn, 138 | intervalTime: $intervalTime 139 | ) 140 | .ignoresSafeArea() 141 | .presentationDetents([.medium, .large]) 142 | }) 143 | } // ZStack 144 | .ignoresSafeArea() 145 | .foregroundColor(.white) 146 | } 147 | 148 | private func showgenerationPanel() { 149 | showingGenPanel = true 150 | } 151 | 152 | private func saveImages() { 153 | guard let images = imageGenerator.generatedImages?.images else { return } 154 | imageGenerator.saveImagesIntoPhotoLibrary(images: images) 155 | } 156 | 157 | private func displayAR() { 158 | guard imageExists else { return } 159 | showingAR = true 160 | } 161 | 162 | private func generate() { 163 | // imageGenerator.removeSDPipeline() 164 | 165 | // Generate images 166 | let param = ImageGenerator.GenerationParameter( 167 | prompt: genParameter.prompt, 168 | negativePrompt: genParameter.negativePrompt, 169 | guidanceScale: genParameter.guidanceScale, 170 | seed: genParameter.randomSeed ? Int.random(in: 0...4_294_967_295) 171 | : Int(genParameter.seed), 172 | stepCount: Int(genParameter.stepCount), 173 | imageCount: genParameter.imageCount, 174 | disableSafety: false) 175 | imageGenerator.generateImages(of: param, 176 | enableStableDiffusion: stableDiffusionOn) 177 | } 178 | } 179 | 180 | struct ImgGenView_Previews: PreviewProvider { 181 | static let imageGenerator = ImageGenerator() 182 | static var previews: some View { 183 | ImgGenView(imageGenerator: imageGenerator) 184 | } 185 | } 186 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/ImgGen View/ParameterView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // ParameterView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/11. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct ParameterView: View { 11 | @Environment(\.dismiss) private var dismiss 12 | @Binding var param: ImgGenView.GenParameter 13 | @Binding var willGenerate: Bool 14 | @State private var selectedPrompt = 0 15 | 16 | var body: some View { 17 | ZStack { 18 | Color.gray 19 | VStack { 20 | Text("Generation Parameters").font(.title3) 21 | 22 | Stepper(value: $param.imageCount, in: 1...10) { 23 | Text("Image Count: \(param.imageCount, specifier: "%3d")") 24 | } 25 | 26 | HStack { 27 | Text("Guidance scale: \(param.guidanceScale, specifier: "%.1f")") 28 | 29 | Slider(value: $param.guidanceScale, 30 | in: 0.0...50.0, 31 | step: 0.5, 32 | minimumValueLabel: Text("0"), 33 | maximumValueLabel: Text("50")) { 34 | Text("label") 35 | } 36 | } 37 | 38 | HStack { 39 | Text("Steps: \(Int(param.stepCount), specifier: "%4d")") 40 | 41 | Slider(value: $param.stepCount, 42 | in: 1...99, 43 | step: 1, 44 | minimumValueLabel: Text("1"), 45 | maximumValueLabel: Text("99")) { 46 | Text("steps") 47 | } 48 | } 49 | 50 | Toggle("Random seed", isOn: $param.randomSeed) 51 | 52 | if !param.randomSeed { 53 | VStack { 54 | // Text("Seed: \(Int(param.seed), specifier: "%10d")") 55 | Text("Seed: \(param.seed, specifier: "%.0f")") 56 | TextField("Seed", value: $param.seed, 57 | formatter: NumberFormatter()) 58 | .onSubmit { 59 | if param.seed < 0 { 60 | param.seed = 0 61 | } else if param.seed > Double(UInt32.max) { 62 | param.seed = Double(UInt32.max) 63 | } else { 64 | // do nothing 65 | } 66 | } 67 | .textFieldStyle(.roundedBorder) 68 | .foregroundColor(.indigo) 69 | 70 | Slider(value: $param.seed, 71 | in: 0...4_294_967_295, 72 | step: 1, 73 | minimumValueLabel: Text("0"), 74 | maximumValueLabel: Text("4B") 75 | ) { 76 | Text("seed") 77 | } 78 | } 79 | } 80 | 81 | Group { 82 | Divider() 83 | Text("Prompt") 84 | TextField("Prompt", text: $param.prompt) 85 | .textFieldStyle(.roundedBorder) 86 | .foregroundColor(.indigo) 87 | .padding(.vertical, 8) 88 | 89 | Text("Registered Prompts") 90 | Picker("Prompt", selection: $selectedPrompt) { 91 | ForEach(0 ..< AppConstant.registeredPrompts.count, 92 | id: \.self) { 93 | Text(AppConstant.registeredPrompts[$0]).lineLimit(1) 94 | } 95 | } 96 | .pickerStyle(.menu) 97 | .onChange(of: selectedPrompt) { newValue in 98 | param.prompt = AppConstant.registeredPrompts[newValue] 99 | } 100 | } 101 | 102 | Group { 103 | Divider() 104 | Text("Negative Prompt") 105 | TextField("Negative Prompt", text: $param.negativePrompt) 106 | .textFieldStyle(.roundedBorder) 107 | .foregroundColor(.indigo) 108 | .padding(.vertical, 8) 109 | } 110 | 111 | HStack { 112 | Button(action: dismiss.callAsFunction) { 113 | Text("Close").font(.title2) 114 | } 115 | .buttonStyle(.bordered) 116 | .padding(8) 117 | 118 | Button(action: { 119 | willGenerate = true 120 | dismiss() 121 | }, label: { 122 | Text("Generate") 123 | .font(.title2) 124 | }) 125 | .buttonStyle(.bordered) 126 | .padding(8) 127 | } 128 | 129 | Spacer() 130 | } // VStack 131 | .tint(.orange) 132 | .padding(8) 133 | } // ZStack 134 | } 135 | } 136 | 137 | struct ParameterView_Previews: PreviewProvider { 138 | @State static var param = ImgGenView.GenParameter(prompt: "", 139 | imageCount: 1, 140 | stepCount: 20, 141 | seed: 100, 142 | randomSeed: false) 143 | @State static var willGenerate = false 144 | 145 | static var previews: some View { 146 | ParameterView(param: $param, willGenerate: $willGenerate) 147 | .foregroundColor(.white) 148 | } 149 | } 150 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/ImgGen View/SettingsView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // SettingsView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/11. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct SettingsView: View { 11 | @Environment(\.dismiss) private var dismiss 12 | @Binding var stableDiffusionOn: Bool 13 | @Binding var peopleOcclusionOn: Bool 14 | @Binding var objectOcclusionOn: Bool 15 | @Binding var arDebugOptionOn: Bool 16 | @Binding var intervalTime: Double 17 | 18 | var body: some View { 19 | ZStack { 20 | Color.gray 21 | 22 | VStack { 23 | HStack { 24 | Text("Settings") 25 | .font(.title3) 26 | .padding(.horizontal, 8) 27 | Text(Bundle.main.appName 28 | + " v" + Bundle.main.appVersion 29 | + " (" + Bundle.main.buildNumber + ")") 30 | .font(.caption) 31 | 32 | Spacer() 33 | Button(action: dismiss.callAsFunction, label: { 34 | Text("Done") 35 | }) 36 | .font(.title3) 37 | .padding(.horizontal, 8) 38 | }.padding(.top, 16) 39 | 40 | List { 41 | #if DEBUG 42 | Section(content: { 43 | Toggle("AR debug info", isOn: $arDebugOptionOn) 44 | }, 45 | header: { Text("Debug")}) 46 | #endif 47 | 48 | Section(content: { 49 | Toggle("Stable Diffusion v" 50 | + AppConstant.stableDiffusionVersion, 51 | isOn: $stableDiffusionOn) 52 | }, header: { 53 | Text("Stable Diffusion") 54 | }, footer: { 55 | // swiftlint:disable line_length 56 | Text("Turn on to generate images. It will take a few minutes for the initialization at the first image generation.") 57 | .foregroundStyle(.secondary) 58 | }) 59 | 60 | Section(content: { 61 | HStack { 62 | Text("Interval [sec]: \(Int(intervalTime), specifier: "%3d")") 63 | Slider(value: $intervalTime, 64 | in: 1...60, 65 | step: 1, 66 | minimumValueLabel: Text("1"), 67 | maximumValueLabel: Text("60")) { 68 | Text("Interval") 69 | } 70 | } 71 | Toggle("People Occlusion", isOn: $peopleOcclusionOn) 72 | .disabled(!ARViewController.peopleOcclusionSupported) 73 | Toggle("Object Occlusion", isOn: $objectOcclusionOn) 74 | .disabled(!ARViewController.objectOcclusionSupported) 75 | }, 76 | header: { Text("AR Display")}) 77 | } 78 | .foregroundColor(.primary) 79 | .tint(.orange) 80 | } 81 | } 82 | } 83 | } 84 | 85 | struct SettingsView_Previews: PreviewProvider { 86 | @State static var stableDiffusionOn = false 87 | @State static var peopleOcclusionOn = true 88 | @State static var objectOcclusionOn = true 89 | @State static var arDebugOptionOn = true 90 | @State static var intervalTime = 3.0 91 | 92 | static var previews: some View { 93 | SettingsView(stableDiffusionOn: $stableDiffusionOn, 94 | peopleOcclusionOn: $peopleOcclusionOn, 95 | objectOcclusionOn: $objectOcclusionOn, 96 | arDebugOptionOn: $arDebugOptionOn, 97 | intervalTime: $intervalTime) 98 | } 99 | } 100 | -------------------------------------------------------------------------------- /ardiffusionmuseum/Views/ImgGen View/StatusView.swift: -------------------------------------------------------------------------------- 1 | // 2 | // StatusView.swift 3 | // ardiffusionmuseum 4 | // 5 | // Created by Yasuhito Nagatomo on 2022/12/11. 6 | // 7 | 8 | import SwiftUI 9 | 10 | struct StatusView: View { 11 | let generating: Bool 12 | let progressStep: (step: Int, stepCount: Int) 13 | 14 | var body: some View { 15 | VStack { 16 | if generating { 17 | if progressStep.step == 0 { 18 | Text("Initializing... Wait for a few minutes.") 19 | .padding() 20 | .background(Color.black.opacity(0.5).cornerRadius(8)) 21 | } else { 22 | Text("Progress: \(progressStep.step) step / \(progressStep.stepCount) steps") 23 | .padding() 24 | .background(Color.black.opacity(0.5).cornerRadius(8)) 25 | } 26 | } else { 27 | Color.clear 28 | } 29 | } // VStack 30 | } 31 | } 32 | 33 | struct StatusView_Previews: PreviewProvider { 34 | static var previews: some View { 35 | StatusView(generating: true, progressStep: (10, 20)) 36 | .foregroundColor(.white) 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /ardiffusionmuseum/ardiffusionmuseum.entitlements: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | com.apple.developer.kernel.extended-virtual-addressing 6 | 7 | com.apple.developer.kernel.increased-memory-limit 8 | 9 | 10 | 11 | -------------------------------------------------------------------------------- /images/appIcon180.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/appIcon180.png -------------------------------------------------------------------------------- /images/gif1_640.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/gif1_640.gif -------------------------------------------------------------------------------- /images/ipad_960.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/ipad_960.jpg -------------------------------------------------------------------------------- /images/relatedapps.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/relatedapps.jpg -------------------------------------------------------------------------------- /images/ss1_1280.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/ss1_1280.jpg -------------------------------------------------------------------------------- /images/ss2_1280.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/ss2_1280.jpg -------------------------------------------------------------------------------- /images/ui_960.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ynagatomo/ARDiffMuseum/6b614d24b901aeb69ec687aa4f02e8233e984753/images/ui_960.jpg --------------------------------------------------------------------------------