├── .gitignore ├── LICENSE ├── Package.swift └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Xcode 2 | # 3 | # gitignore contributors: remember to update Global/Xcode.gitignore, Objective-C.gitignore & Swift.gitignore 4 | 5 | ## User settings 6 | xcuserdata/ 7 | 8 | ## compatibility with Xcode 8 and earlier (ignoring not required starting Xcode 9) 9 | *.xcscmblueprint 10 | *.xccheckout 11 | 12 | ## compatibility with Xcode 3 and earlier (ignoring not required starting Xcode 4) 13 | build/ 14 | DerivedData/ 15 | *.moved-aside 16 | *.pbxuser 17 | !default.pbxuser 18 | *.mode1v3 19 | !default.mode1v3 20 | *.mode2v3 21 | !default.mode2v3 22 | *.perspectivev3 23 | !default.perspectivev3 24 | 25 | ## Obj-C/Swift specific 26 | *.hmap 27 | 28 | ## App packaging 29 | *.ipa 30 | *.dSYM.zip 31 | *.dSYM 32 | 33 | ## Playgrounds 34 | timeline.xctimeline 35 | playground.xcworkspace 36 | 37 | # Swift Package Manager 38 | # 39 | # Add this line if you want to avoid checking in source code from Swift Package Manager dependencies. 40 | # Packages/ 41 | # Package.pins 42 | # Package.resolved 43 | # *.xcodeproj 44 | # 45 | # Xcode automatically generates this directory with a .xcworkspacedata file and xcuserdata 46 | # hence it is not needed unless you have added a package configuration file to your project 47 | # .swiftpm 48 | 49 | .build/ 50 | 51 | # CocoaPods 52 | # 53 | # We recommend against adding the Pods directory to your .gitignore. However 54 | # you should judge for yourself, the pros and cons are mentioned at: 55 | # https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control 56 | # 57 | # Pods/ 58 | # 59 | # Add this line if you want to avoid checking in source code from the Xcode workspace 60 | # *.xcworkspace 61 | 62 | # Carthage 63 | # 64 | # Add this line if you want to avoid checking in source code from Carthage dependencies. 65 | # Carthage/Checkouts 66 | 67 | Carthage/Build/ 68 | 69 | # Accio dependency management 70 | Dependencies/ 71 | .accio/ 72 | 73 | # fastlane 74 | # 75 | # It is recommended to not store the screenshots in the git repo. 76 | # Instead, use fastlane to re-generate the screenshots whenever they are needed. 77 | # For more information about the recommended setup visit: 78 | # https://docs.fastlane.tools/best-practices/source-control/#source-control 79 | 80 | fastlane/report.xml 81 | fastlane/Preview.html 82 | fastlane/screenshots/**/*.png 83 | fastlane/test_output 84 | 85 | # Code Injection 86 | # 87 | # After new code Injection tools there's a generated folder /iOSInjectionProject 88 | # https://github.com/johnno1962/injectionforxcode 89 | 90 | iOSInjectionProject/ 91 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | All source code Copyright © 2023 FieldDay Technologies, Inc. 2 | -------------------------------------------------------------------------------- /Package.swift: -------------------------------------------------------------------------------- 1 | // swift-tools-version: 5.8 2 | // The swift-tools-version declares the minimum version of Swift required to build this package. 3 | 4 | import PackageDescription 5 | 6 | let package = Package( 7 | name: "FieldDay", 8 | platforms: [ 9 | .iOS(.v15), 10 | ], 11 | products: [ 12 | // Products define the executables and libraries a package produces, and make them visible to other packages. 13 | .library( 14 | name: "FieldDay", 15 | targets: ["FieldDay"]), 16 | ], 17 | dependencies: [ 18 | // Dependencies declare other packages that this package depends on. 19 | // .package(url: /* package url */, from: "1.0.0"), 20 | ], 21 | targets: [ 22 | // Targets are the basic building blocks of a package. A target can define a module or a test suite. 23 | // Targets can depend on other targets in this package, and on products in packages this package depends on. 24 | .binaryTarget( 25 | name: "FieldDay", 26 | url: "https://github.com/newmaterialco/fieldday-ios-sdk/releases/download/1.1.1/FieldDay-1.1.1.zip", 27 | checksum: "1c169774fa5787d9d61fdb3438a319555d786ea2f30a829aa88056dfba0136b9" 28 | ) 29 | ] 30 | ) 31 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # FieldDay iOS SDK 2 | ![thumb](https://github.com/fieldday-ai/fieldday-ios-sdk/assets/58298401/e1b7fd85-7cd6-4449-ae1d-eed6eb4660c5) 3 | 4 | # Installation 5 | 6 | You can add the FieldDay iOS SDK using Swift Package Manager. 7 | 8 | - From the File menu, select Add Packages... 9 | - Enter the repo's URL 10 | 11 | ``` 12 | http://github.com/fieldday-ai/fieldday-ios-sdk.git 13 | ``` 14 | 15 | # Prepare Xcode Project 16 | 17 | - [Add the FieldDay Package](#installation) 18 | - Provide Camera Usage Description 19 | - Project > Build Settings 20 | - Scroll to Info.plist Values 21 | - Find "Privacy - Camera Usage Description" 22 | - Provide a suitable description of why you wish to access the Camera 23 | - [Optional] Lock the supported screen orientations 24 | - Target > General > Deployment Info 25 | - iPhone Orientation 26 | - Check only Portrait 27 | 28 | # Setup FieldDay Project 29 | 30 | There are two methods to use your FieldDay project with the iOS SDK. To get started, open your project in the FieldDay app. Make sure you have trained at least one model in the project. 31 | 32 | ## Method 1: Publish your project 33 | 34 | FieldDay allows you to publish your project so it can be fetched over the internet. Using this method, you can ensure your project is always kept up-to-date with any changes you make. The package caches project information, so make sure to set the appropriate cache policy as per your requirements (see [Caching](#caching)). 35 | 36 | - Tap the share button at the top right corner of the screen. 37 | - Select the `Swift` tile. 38 | - Scroll down to the section where it says "Share Code". 39 | - Copy the alphanumeric share code that appears by tapping on the box. 40 | 41 | ### Usage 42 | 43 | ```swift 44 | import SwiftUI 45 | import FieldDay 46 | 47 | struct ContentView: View { 48 | var body: some View { 49 | FDViewfinderView( 50 | /// Enter the share code that we copied from FieldDay 51 | shareCode: "________________________________" 52 | ) 53 | } 54 | } 55 | ``` 56 | 57 | ## Method 2: Use a CoreML `.mlmodel` file 58 | 59 | Alternatively, you can also package a model file with your app. This has the advantage of always working offline. But you will need to update it manually to include changes from your FieldDay project. 60 | 61 | - Tap the share button at the top right corner of the screen. 62 | - Select the `CoreML` option. 63 | - Tap "Export `.mlmodel`". This will give you two files – `Categories.swift` and `Project Name.mlmodel`. 64 | - Add the two files to the Xcode project. 65 | 66 | ### Usage 67 | 68 | ```swift 69 | import SwiftUI 70 | import FieldDay 71 | 72 | struct ContentView: View { 73 | var body: some View { 74 | FDViewfinderView( 75 | modelURL: Project_Name.urlOfModelInThisBundle, 76 | // A class like "Project_Name" is automatically generated by Xcode for your model. 77 | // Open the `.mlmodel` file in Xcode to find the class name. 78 | categories: categories 79 | // This array is defined in the Categories.swift file. 80 | // You can edit the category names or colours. 81 | ) 82 | } 83 | } 84 | ``` 85 | 86 | # Advanced Usage 87 | 88 | ## Handle prediction events 89 | 90 | At the moment, the FieldDay SDK supports handling the following events: 91 | 92 | - When the model makes a prediction 93 | - When a prediction pill is tapped 94 | 95 | _A prediction pill is the element at the bottom of the screen, showing the category name for the prediction_ 96 | 97 | These events can be handled via the `onPrediction` and `onPredictionTap` modifiers on the ViewfinderView. They can be used as follows. 98 | 99 | ```swift 100 | FDViewfinderView(...) 101 | .onPredictionTap { category in 102 | print(category.name) 103 | } 104 | .onPrediction { prediction, category in 105 | print(category.name, prediction.confidence) 106 | } 107 | ``` 108 | 109 | **`FDCategory`** 110 | 111 | ```swift 112 | struct FDCategory { 113 | var id: String 114 | var name: String 115 | var color: Color 116 | var isBackground: Bool 117 | } 118 | ``` 119 | 120 | - `id` - The unique identifier of the category 121 | - `name` - The category's name (defined in the FieldDay App) 122 | - `color` - The category's color (defined in the FieldDay App) 123 | - `isBackground` - Indicates whether the category is the default "Background" category 124 | 125 | **`FDModelPrediction`** 126 | 127 | ```swift 128 | struct FDModelPrediction { 129 | var identifier: String 130 | var confidence: Float 131 | var results: [VNClassificationObservation] 132 | } 133 | ``` 134 | 135 | - `identifier` - The identifier returned associated with the CoreML prediction 136 | - `confidence` - The confidence of the prediction, normalized from `0...1` 137 | - `results` - Contains confidence values for all categories in the model 138 | 139 | ## Modify the Viewfinder 140 | 141 | ### Viewfinder Size 142 | Use the `viewfinderSize` modifier to adjust size. This ensures the underlying camera preview is sized correctly. 143 | ```swift 144 | FDViewfinderView(...) 145 | .viewfinderSize(width: 200, height: 200) 146 | ``` 147 | 148 | ### Viewfinder Mode 149 | Use the `viewfinderMode` modifier to select between `.photo` and `.video`. This selects the appropriate aspect ratio and zoom levels for the device. The SDK defaults to `.video` mode. 150 | ```swift 151 | FDViewfinderView(...) 152 | .viewfinderMode(.photo) 153 | ``` 154 | 155 | ### Disable Model Feedback UI 156 | You can choose to hide the predictions UI with the `feedbackUIHidden` modifier. 157 | ```swift 158 | FDViewfinderView(...) 159 | .feedbackUIHidden() 160 | ``` 161 | 162 | ## Prediction Smoothing 163 | The SDK smooths the results of the FieldDay model to reduce jitter in the results. If your model doesn't require this, or the performance doesn't match the FieldDay app, you can disable the smoothing. 164 | 165 | ```swift 166 | FDViewfinderView(...) 167 | .predictionSmoothingDisabled() 168 | ``` 169 | 170 | ## Caching 171 | When using project keys, FieldDay has an option to cache network data to offer limited offline functionality. By passing in a `FDCachePolicy` in the `FDViewfinderView` intializer, you can customize how the cache is used. If no policy is passed in, the default is `.cacheThenNetwork`. 172 | ```swift 173 | FDViewfinderView( 174 | ..., 175 | cachePolicy: .ignoreCache 176 | ) 177 | ``` 178 | 179 | ### `FDCachePolicy` 180 | ```swift 181 | enum FDCachePolicy { 182 | case cacheThenNetwork 183 | case ignoreCache 184 | case ignoreCacheCompletely 185 | } 186 | ``` 187 | - `cacheThenNetwork` - Fetches from the cache, then the network, and only uses the network result if it differs from the cache. This is the default policy. 188 | - `ignoreCache` - Fetches from the network only, but still writes the result to the cache. 189 | - `ignoreCacheCompletely` - Fetches from the network only, and does not write the result to the cache. 190 | 191 | ### Manual Cache Clearing 192 | If needed, the `clearFDCache()` extension on `UserDefaults` lets you manually clear the FieldDay cache. 193 | ```swift 194 | UserDefaults.standard.clearFDCache() 195 | ``` 196 | 197 | ## Debugging 198 | 199 | To view debugging logs that expose error messages for various operations - just add the `.debug()` modifier to your `FDViewfinderView`. FieldDay logs will be prefixed with "🤖⚠️". 200 | 201 | ```swift 202 | FDViewfinderView(...) 203 | .debug() 204 | ``` 205 | --------------------------------------------------------------------------------