├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── dfxservice.go ├── docs └── tutorial.md ├── framework.go ├── go.mod ├── go.sum ├── models ├── config.go ├── endpoint.go ├── engine.go └── mapping_metadata.go ├── summary ├── summary.go └── summary_test.go ├── template.go └── utils ├── candid_serialize.go └── utils.go /.gitignore: -------------------------------------------------------------------------------- 1 | # Binaries for programs and plugins 2 | *.exe 3 | *.exe~ 4 | *.dll 5 | *.so 6 | *.dylib 7 | 8 | # Test binary, built with `go test -c` 9 | *.test 10 | 11 | # Output of the go coverage tool, specifically when used with LiteIDE 12 | *.out 13 | 14 | # Dependency directories (remove the comment below to include it) 15 | vendor/ 16 | 17 | .env -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Hypotenuse Labs 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: test 2 | test: 3 | bash -c 'echo "Testing utils..." && go test ./utils' -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Dfinity Oracles 2 | 3 | [Dfinity Oracles](https://github.com/hyplabs/dfinity-oracle-framework) is a framework for building [blockchain oracles](https://en.wikipedia.org/wiki/Blockchain_oracle) for the [Internet Computer](https://dfinity.org/). 4 | 5 | ## Table of Contents 6 | 7 | - [Table of Contents](#table-of-contents) 8 | - [Background](#background) 9 | - [Tutorial and Examples](#tutorial-and-examples) 10 | - [Framework Reference](#framework-reference) 11 | - [`oracle.Bootstrap()`](#oraclebootstrap) 12 | - [`oracle.Run()`](#oraclerun) 13 | - [The Oracle Update Lifecycle](#the-oracle-update-lifecycle) 14 | - [Acquiring data](#acquiring-data) 15 | - [Summarizing data](#summarizing-data) 16 | - [Updating the canister](#updating-the-canister) 17 | - [Testing the Framework](#testing-the-framework) 18 | - [Oracle Revocation](#oracle-revocation) 19 | 20 | ## Background 21 | 22 | Applications on the Internet Computer often require information that comes from outside sources. For example, an application might require the current temperature of various cities in the world, or it might require the current prices of various stocks and cryptocurrencies. 23 | 24 | Oracles are useful here in order to move information from traditional APIs to software running on the Internet Computer. 25 | 26 | ## Tutorial and Examples 27 | 28 | - [Tutorial](docs/tutorial.md) - step-by-step setup of a sample oracle for retrieving the temperature in different cities. 29 | - [Dfinity Weather Oracle Example](https://github.com/hyplabs/dfinity-weather-oracle) - a complete working example oracle for retrieving various weather conditions in 20 different cities. 30 | - [Dfinity Crypto Oracle Example](https://github.com/hyplabs/dfinity-crypto-oracle) - a complete working example oracle for retrieving current ETH and BTC prices. 31 | 32 | ## Framework Reference 33 | 34 | Now that we have an example app working with the library, let's take a deeper look at what the DFINITY Oracle Framework is doing behind the scenes. 35 | 36 | ### `oracle.Bootstrap()` 37 | 38 | The oracle framework bootstraps the oracle canister by calling into `dfx` for the following setup tasks: 39 | 40 | - Creating the oracle canister DFX project if it doesn't exist. 41 | - Starting the DFX network locally in the background. 42 | - Creating the oracle writer identity if it doesn't exist. 43 | - Creating the oracle canister if it doesn't exist. 44 | - Building and installing the canister (if the canister already exists, this performs an upgrade instead). 45 | - Starting the oracle canister if it isn't already running. 46 | - Claiming the oracle owner role if not already claimed, allowing it to manage canister roles. 47 | - Assigning the oracle writer identity the writer role, allowing it to update canister data. 48 | 49 | ### `oracle.Run()` 50 | 51 | The oracle framework starts the oracle service and periodically updates the mappings in the canister. Once this service is running, it will update the canister at the configured time interval. 52 | 53 | ## The Oracle Update Lifecycle 54 | 55 | The oracle framework performs updates at the interval configured via `UpdateInterval`. This update consists of several steps, which are described below. 56 | 57 | ### Acquiring data 58 | 59 | Through the information provided in `config`, we have a list of API endpoints (`Endpoints`), and for each endpoint, its URL (`Endpoint`), a collection of field names and their corresponding JSONPaths within API responses (`JSONPaths`), and an optional normalization function (`NormalizeFunc`). 60 | 61 | For more details about JSONPath syntax, see this [JSONPath reference](https://restfulapi.net/json-jsonpath). 62 | 63 | When the oracle framework performs an update, it will first make `GET` requests to every endpoint in `config`, resulting in one JSON response per endpoint. The framework then extracts the desired fields from these responses by each field's JSONPath expression, resulting in a `map[string]interface{}` (a mapping from strings to anything). 64 | 65 | In our example, we make requests to both WeatherAPI and WeatherBit, and extract the temperature, pressure, and humidity. We have two different configurations of these endpoints - one for Tokyo, and one for Delhi. 66 | 67 | This value is then passed to `NormalizeFunc`, which is responsible for turning it into a `map[string]float64`. If no `NormalizeFunc` is specified, then a default one will be used - every field's value will simply be casted as a `float64`. 68 | 69 | ### Summarizing data 70 | 71 | Oracles generally acquire redundant data from many independent sources, then combine them into one trustworthy value. 72 | 73 | From the previous step, every endpoint resulted in a `map[string]float64` result, and each of these results is supposed to represent a redundant copy of the same piece of data (e.g., the current temperature in Tokyo). We now need to turn those into a single value. 74 | 75 | This is the job of `SummarizeFunc`, which accepts a collection of `map[string]float64` values, and is responsible for converting that into a single `map[string]float64` value. In our example, we've created a custom summarization function called `summarizeWeatherData`. There are two particularly useful oracle framework utility functions that it calls: 76 | 77 | - `summary.GroupByKey([]map[string]float64) map[string][]float64`: takes a list of mappings, and converts it into a mapping where each key contains a list of all values in those mappings under those keys. 78 | - `summary.MeanWithoutOutliers([]float64) float64`: takes a list of numbers, removes outliers (outliers are values that are more than 2 standard deviations from the median, so a 95% confidence interval), and takes the mean of the remaining values. This is more stable than the median while still rejecting rogue values. 79 | 80 | ### Updating the canister 81 | 82 | As part of the bootstrap step, the oracle framework created a `writer` identity for the oracle to use - an identity that is allowed to write new values to the mappings stored in the canister. 83 | 84 | After the previous step, we now have a `map[string]float64` for Tokyo, and a `map[string]float64` for Delhi. We've written some simple Candid IDL serialization functions in Go that then turn this into a string suitable for passing into DFX. 85 | 86 | The final step is then to write this serialized string to the canister using the `writer` identity. 87 | 88 | ## Testing the Framework 89 | 90 | Although the `writer` identity is the only one capable of changing the values inside the oracle canister, reading those values can be done by anyone. To test it out, enter the following command: 91 | 92 | ```bash 93 | cd weather_oracle 94 | ``` 95 | 96 | ```bash 97 | dfx canister call weather_oracle get_map_field_value ("London", "temperature_celsius") 98 | ``` 99 | 100 | This would return the currently stored temperature in Celsius of London within the canister. 101 | 102 | Similarly, 103 | 104 | ```bash 105 | dfx canister call weather_oracle get_map_field_value ("Tokyo", "humidity_pct") 106 | ``` 107 | 108 | would return the currently stored humidity percentage of Tokyo within the canister. 109 | 110 | ## Oracle Revocation 111 | 112 | The framework assigns the `owner` identity to the user who deploys the canister, and this cannot be edited once deployed. 113 | 114 | The `owner` identity is capable of assigning the `writer` role to other identities, and also of permanently disabling the oracle if they believe it has been compromised. 115 | 116 | If the `writer` identity is compromised, but not the `owner` identity, the `owner` identity can simply assign a new `writer`, and users can continue to use the oracle as they did before. 117 | 118 | If the `owner` identity is compromised (e.g., private key leaked), they cannot take control away from the original `owner`, because the `owner` field is not editable. However, the original owner can permanently disable the oracle, preventing attackers from using it for their own gain. This disincentivizes attackers from attempting to take control of the oracle in the first place. 119 | 120 | As an oracle owner, we can permanently disable the oracle using the following command: 121 | 122 | ```bash 123 | cd weather_oracle 124 | dfx canister call weather_oracle self_destruct 125 | ``` 126 | 127 | After this, if you try to get the value through the command given in the Testing section, you will receive an error. 128 | -------------------------------------------------------------------------------- /dfxservice.go: -------------------------------------------------------------------------------- 1 | package framework 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "os" 7 | "os/exec" 8 | "path/filepath" 9 | "strings" 10 | "time" 11 | 12 | "github.com/hyplabs/dfinity-oracle-framework/models" 13 | "github.com/hyplabs/dfinity-oracle-framework/utils" 14 | "github.com/sirupsen/logrus" 15 | ) 16 | 17 | // DFXService contains various fields to be used by the DFX interface 18 | type DFXService struct { 19 | config *models.Config 20 | log *logrus.Logger 21 | } 22 | 23 | // NewDFXService creates an instance of DFX Service 24 | func NewDFXService(config *models.Config, log *logrus.Logger) *DFXService { 25 | return &DFXService{ 26 | config, 27 | log, 28 | } 29 | } 30 | 31 | func (s *DFXService) createNewDfxProject() error { 32 | s.log.Infof("Creating new project %s...", s.config.CanisterName) 33 | output, exitCode, err := dfxCall(".", []string{"new", s.config.CanisterName}, true) 34 | if err != nil { 35 | s.log.WithError(err).Errorln("Could not create new project:", output) 36 | return err 37 | } 38 | if exitCode != 0 && !strings.HasPrefix(output, "Cannot create a new project because the directory already exists.") { 39 | s.log.WithError(err).Errorln("Could not create new project:", output) 40 | return fmt.Errorf("Could not create new project: %v", output) 41 | } 42 | return nil 43 | } 44 | 45 | func (s *DFXService) updateCanisterCode() error { 46 | s.log.Infof("Updating canister code...") 47 | fileName := filepath.Join(s.config.CanisterName, "src", s.config.CanisterName, "main.mo") 48 | if err := ioutil.WriteFile(fileName, []byte(CodeTemplate), 0644); err != nil { 49 | s.log.WithError(err).Errorln("Could not write to main.mo file") 50 | return err 51 | } 52 | return nil 53 | } 54 | 55 | func (s *DFXService) stopDfxNetwork() error { 56 | s.log.Infof("Stopping existing DFX instances...") 57 | _, _, err := dfxCall(s.config.CanisterName, []string{"stop"}, false) 58 | if err != nil { 59 | s.log.WithError(err).Errorln("Could not stop DFX identity") 60 | return err 61 | } 62 | s.log.Infoln("Sleeping 5 seconds to allow local network to finish shutting down...") 63 | time.Sleep(5 * time.Second) 64 | return nil 65 | } 66 | 67 | func (s *DFXService) startDfxNetwork() error { 68 | s.log.Infof("Starting DFX in the background...") 69 | dfxExecutable, err := exec.LookPath("dfx") 70 | if err != nil { 71 | return fmt.Errorf("Could not find DFX executable: %w", err) 72 | } 73 | 74 | dfxCommand := &exec.Cmd{ 75 | Path: dfxExecutable, 76 | Dir: s.config.CanisterName, 77 | Args: []string{dfxExecutable, "start", "--background"}, 78 | Stdout: os.Stdout, 79 | Stderr: os.Stderr, 80 | } 81 | if err := dfxCommand.Run(); err != nil { 82 | s.log.WithError(err).Errorln("Could not start local network") 83 | return err 84 | } 85 | s.log.Infoln("Sleeping 5 seconds to allow local network to set up enough that we can make requests...") 86 | time.Sleep(5 * time.Second) 87 | return nil 88 | } 89 | 90 | func (s *DFXService) createWriterIdentityIfNeeded() error { 91 | s.log.Infof("Creating writer identity...") 92 | output, exitCode, err := dfxCall(s.config.CanisterName, []string{"identity", "new", "writer"}, true) 93 | if err != nil { 94 | s.log.WithError(err).Errorln("Could not create new writer identity") 95 | return err 96 | } 97 | if exitCode != 0 && !strings.HasPrefix(output, "Creating identity: \"writer\".\nIdentity already exists.") { 98 | s.log.Errorln("Could not create new writer identity:", output) 99 | return err 100 | } 101 | return nil 102 | } 103 | 104 | func (s *DFXService) doesCanisterExist() (bool, error) { 105 | s.log.Infof("Checking if canister already exists...") 106 | output, exitCode, err := dfxCall(s.config.CanisterName, []string{"canister", "id", s.config.CanisterName}, true) 107 | if err != nil { 108 | s.log.WithError(err).Errorln("Could not determine if canister exists") 109 | return false, err 110 | } 111 | if exitCode == 0 { 112 | return true, nil 113 | } 114 | if !strings.HasPrefix(output, "Cannot find canister id.") { 115 | s.log.Errorln("Could not determine if canister exists:", output) 116 | return false, err 117 | } 118 | return false, nil 119 | } 120 | 121 | func (s *DFXService) isCanisterRunning() (bool, error) { 122 | s.log.Infof("Checking if canister is running...") 123 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "status", s.config.CanisterName}, false) 124 | if err != nil { 125 | s.log.WithError(err).Errorln("Could not determine canister status:", output) 126 | return false, err 127 | } 128 | if !strings.HasPrefix(output, "Canister "+s.config.CanisterName+"'s status is ") { 129 | s.log.WithError(err).Errorln("Could not determine canister status:", output) 130 | return false, fmt.Errorf("Could not determine canister status: %v", output) 131 | } 132 | isRunning := strings.HasPrefix(output, "Canister "+s.config.CanisterName+"'s status is Running.") 133 | return isRunning, nil 134 | } 135 | 136 | func (s *DFXService) createCanister() error { 137 | s.log.Infof("Creating canister...") 138 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "create", s.config.CanisterName}, false) 139 | if err != nil { 140 | s.log.WithError(err).Errorln("Could not create canister:", output) 141 | return err 142 | } 143 | return nil 144 | } 145 | 146 | func (s *DFXService) buildCanister() error { 147 | s.log.Infof("Building canister...") 148 | output, _, err := dfxCall(s.config.CanisterName, []string{"build", s.config.CanisterName}, false) 149 | if err != nil { 150 | s.log.WithError(err).Errorln("Could not build canister:", output) 151 | return err 152 | } 153 | return nil 154 | } 155 | 156 | func (s *DFXService) installCanister(upgradeExistingCanister bool) error { 157 | s.log.Infof("Installing canister...") 158 | 159 | args := []string{"canister", "install", s.config.CanisterName} 160 | if upgradeExistingCanister { 161 | args = []string{"canister", "install", s.config.CanisterName, "--mode", "upgrade"} 162 | } 163 | 164 | output, _, err := dfxCall(s.config.CanisterName, args, false) 165 | if err != nil { 166 | s.log.WithError(err).Errorln("Could not install canister:", output) 167 | return err 168 | } 169 | return nil 170 | } 171 | 172 | func (s *DFXService) startCanister() error { 173 | s.log.Infof("Starting canister...") 174 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "start", s.config.CanisterName}, false) 175 | if err != nil { 176 | s.log.WithError(err).Errorln("Could not start canister:", output) 177 | return err 178 | } 179 | return nil 180 | } 181 | 182 | func (s *DFXService) checkIsOwner() (bool, error) { 183 | s.log.Infof("Checking if we have the owner role...") 184 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "call", s.config.CanisterName, "my_role"}, false) 185 | if err != nil { 186 | s.log.WithError(err).Errorln("Could not retrieve current role:", output) 187 | return false, err 188 | } 189 | hasOwnerRole := strings.HasPrefix(output, "(opt variant { owner })") 190 | return hasOwnerRole, nil 191 | } 192 | 193 | func (s *DFXService) assignOwnerRole() error { 194 | s.log.Infof("Assigning owner role to owner identity...") 195 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "call", s.config.CanisterName, "assign_owner_role"}, false) 196 | if err != nil { 197 | s.log.WithError(err).Errorln("Could not assign owner role to the owner identity:", output) 198 | return err 199 | } 200 | return nil 201 | } 202 | 203 | func (s *DFXService) getWriterIDPrincipal() (string, error) { 204 | s.log.Infof("Retrieving writer ID principal...") 205 | output, _, err := dfxCall(s.config.CanisterName, []string{"--identity", "writer", "identity", "get-principal"}, false) 206 | if err != nil { 207 | s.log.WithError(err).Errorln("Could not determine the writer identity's principal:", output) 208 | return "", err 209 | } 210 | return strings.TrimSpace(output), nil 211 | } 212 | 213 | func (s *DFXService) assignWriterRole(writerPrincipal string) error { 214 | s.log.Infof("Assigning writer role to writer identity %s...", writerPrincipal) 215 | callArgs := fmt.Sprintf("(%v)", utils.CandidPrincipal(writerPrincipal)) 216 | output, _, err := dfxCall(s.config.CanisterName, []string{"canister", "call", s.config.CanisterName, "assign_writer_role", callArgs}, false) 217 | if err != nil { 218 | s.log.WithError(err).Errorln("Could not assign writer role to the writer identity:", output) 219 | return err 220 | } 221 | return nil 222 | } 223 | 224 | func (s *DFXService) updateValueInCanister(key string, val map[string]float64) error { 225 | s.log.Infof("Updating value in canister...") 226 | 227 | for k, v := range val { 228 | callArgs := fmt.Sprintf("(%v,%v,%v)", utils.CandidText(key), utils.CandidText(k), utils.CandidFloat64(v)) 229 | output, _, err := dfxCall(s.config.CanisterName, []string{"--identity", "writer", "canister", "call", s.config.CanisterName, "update_map_value", callArgs}, false) 230 | if err != nil { 231 | s.log.WithError(err).Errorln("Could not update key", key, "field", k, "value", val, "in canister:", output) 232 | return err 233 | } 234 | } 235 | return nil 236 | } 237 | 238 | func dfxCall(workingDir string, args []string, allowNonzeroExitCode bool) (string, int, error) { 239 | dfxExecutable, err := exec.LookPath("dfx") 240 | if err != nil { 241 | return "", 0, fmt.Errorf("Could not find DFX executable: %w", err) 242 | } 243 | 244 | dfxCommand := &exec.Cmd{ 245 | Path: dfxExecutable, 246 | Dir: workingDir, 247 | Args: append([]string{dfxExecutable}, args...), 248 | } 249 | output, err := dfxCommand.CombinedOutput() 250 | if err != nil { 251 | if allowNonzeroExitCode { 252 | if exitError, ok := err.(*exec.ExitError); ok { 253 | return string(output), exitError.ExitCode(), nil 254 | } 255 | } 256 | return string(output), 0, fmt.Errorf("Running DFX command failed: %w", err) 257 | } 258 | return string(output), 0, nil 259 | } 260 | -------------------------------------------------------------------------------- /docs/tutorial.md: -------------------------------------------------------------------------------- 1 | # Dfinity Oracles Tutorial 2 | 3 | For this tutorial, we will create a sample oracle for fetching the current temperature in different cities, using Dfinity Oracles. To read more about the framework itself, see the [README](../README.md). 4 | 5 | This sample oracle is also available as a complete project, [Dfinity Weather Oracle Example](https://github.com/hyplabs/dfinity-weather-oracle)! 6 | 7 | ## Table of Contents 8 | 9 | - [Table of Contents](#table-of-contents) 10 | - [Step 0: Before you begin](#step-0-before-you-begin) 11 | - [Step 1: Create a new project for the sample oracle](#step-1-create-a-new-project-for-the-sample-oracle) 12 | - [Step 2: Get API endpoints for external data sources](#step-2-get-api-endpoints-for-external-data-sources) 13 | - [Step 3: Configuring the sample oracle](#step-3-configuring-the-sample-oracle) 14 | - [Step 4: Running the sample oracle](#step-4-running-the-sample-oracle) 15 | 16 | ## Step 0: Before you begin 17 | 18 | Before starting this tutorial, double-check the following: 19 | 20 | - You have downloaded and installed the [DFINITY Canister SDK](https://sdk.dfinity.org/docs/quickstart/local-quickstart.html#download-and-install). 21 | - You have downloaded and installed the [Go programming language](https://golang.org/). 22 | - You have stopped any Internet Computer network processes running on the local computer. 23 | 24 | ## Step 1: Create a new project for the sample oracle 25 | 26 | Create a new folder named `sample-oracle`, then create a new Go project in this folder: 27 | 28 | ```bash 29 | go mod init github.com/YOUR_USERNAME/sample-oracle 30 | ``` 31 | 32 | (NOTE: replace `YOUR_USERNAME` with your GitHub username) 33 | 34 | Install the framework, `dfinity-oracles`, as a Go Module dependancy: 35 | 36 | ```bash 37 | go get github.com/hyplabs/dfinity-oracle-framework 38 | ``` 39 | 40 | Note that if you are trying this tutorial before Dfinity Oracles has been publicly released, you'll want to use the following command instead: `GOPRIVATE=github.com/hyplabs/dfinity-oracle-framework GIT_TERMINAL_PROMPT=1 go get github.com/hyplabs/dfinity-oracle-framework`. 41 | 42 | ## Step 2: Get API endpoints for external data sources 43 | 44 | To fetch the current temperature in a city, we need to call various weather monitoring APIs and parse the responses. We will be using the following APIs, but you can choose your own if you prefer: 45 | 46 | - [WeatherAPI](https://weatherapi.com) - the `http://api.weatherapi.com/v1/current.json?key=WEATHERAPI_API_KEY&q=CITY` endpoint. 47 | - [Weatherbit](https://weatherbit.io) - the `https://api.weatherbit.io/v2.0/current?city=CITY&country=COUNTRY_CODE&key=WEATHERBIT_API_KEY` endpoint. 48 | 49 | Sign up for both of the above services, and obtain a developer API key for each. You will need it in the next step! 50 | 51 | ## Step 3: Configuring the sample oracle 52 | 53 | We can now configure the sample oracle. Create a new file in the `sample-oracle` folder named `main.go`, with the following contents: 54 | 55 | ```go 56 | package main 57 | 58 | import ( 59 | "time" 60 | 61 | framework "github.com/hyplabs/dfinity-oracle-framework" 62 | "github.com/hyplabs/dfinity-oracle-framework/models" 63 | ) 64 | 65 | func main() { 66 | tokyoEndpoints := []models.Endpoint{ 67 | { 68 | Endpoint: "http://api.weatherapi.com/v1/current.json?key=WEATHERAPI_API_KEY&q=Tokyo,JP", 69 | JSONPaths: map[string]string{ 70 | "temperature_celsius": "$.current.temp_c", 71 | }, 72 | }, 73 | { 74 | Endpoint: "https://api.weatherbit.io/v2.0/current?key=WEATHERBIT_API_KEY&city=Tokyo&country=JP", 75 | JSONPaths: map[string]string{ 76 | "temperature_celsius": "$.data[0].temp", 77 | }, 78 | }, 79 | } 80 | delhiEndpoints := []models.Endpoint{ 81 | { 82 | Endpoint: "http://api.weatherapi.com/v1/current.json?key=WEATHERAPI_API_KEY&q=Delhi,IN", 83 | JSONPaths: map[string]string{ 84 | "temperature_celsius": "$.current.temp_c", 85 | }, 86 | }, 87 | { 88 | Endpoint: "https://api.weatherbit.io/v2.0/current?key=WEATHERBIT_API_KEY&city=Delhi&country=IN", 89 | JSONPaths: map[string]string{ 90 | "temperature_celsius": "$.data[0].temp", 91 | }, 92 | }, 93 | } 94 | config := models.Config{ 95 | CanisterName: "sample_oracle", 96 | UpdateInterval: 5 * time.Minute, 97 | } 98 | engine := models.Engine{ 99 | Metadata: []models.MappingMetadata{ 100 | {Key: "Tokyo", Endpoints: tokyoEndpoints}, 101 | {Key: "Delhi", Endpoints: delhiEndpoints}, 102 | }, 103 | } 104 | oracle := framework.NewOracle(&config, &engine) 105 | oracle.Bootstrap() 106 | oracle.Run() 107 | } 108 | ``` 109 | 110 | (NOTE: replace `WEATHERAPI_API_KEY` with your WeatherAPI API key, and `WEATHERBIT_API_KEY` with your Weatherbit API key) 111 | 112 | Let's take a look at the key parts of this sample oracle: 113 | 114 | - `tokyoEndpoints` and `delhiEndpoints` specify the URLs where the temperature data can be found, as well as [JSONPath](https://www.baeldung.com/guide-to-jayway-jsonpath) expressions that extract just the temperature (in Celsius) out of the JSON response. 115 | - `Endpoint` is the URL. In this case, we've entered the WeatherAPI and WeatherBit API endpoint URLs here. 116 | - `JSONPaths` is a map of JSONPath expressions and the relevant info that they retrieve. In this case, the only piece of relevant info is `temperature_celsius`. 117 | - `metadata` specifies the two pieces of data that we care about - the temperature in Tokyo, and the temperature in Delhi. 118 | - In this example, since the `SummaryFunc` option of `metadata` isn't specified, a default summarization function will be applied, which simply takes the `temperature_celsius` key from the API call results, eliminates outliers (outside 2 standard deviations), and takes the average of the remaining values to obtain the final temperature. 119 | - `config` specifies the `CanisterName` and `UpdateInterval` - what the canister should be called, and how often it should update. 120 | - We create a new oracle struct by calling `NewOracle(&config)`. 121 | - We bootstrap the new oracle by calling `oracle.Bootstrap()`. 122 | - Finally, we start the oracle by calling `oracle.Run()`. 123 | 124 | For more details about what `oracle.Bootstrap()` and `oracle.Run()` do, see "Framework Reference" in the README. 125 | 126 | ## Step 4: Running the sample oracle 127 | 128 | Since we've finished configuring the sample oracle, we can now build and run it: 129 | 130 | ```bash 131 | go build 132 | ``` 133 | 134 | This should create an executable, `sample-oracle` (or `sample-oracle.exe` on Windows). Now let's run the sample oracle: 135 | 136 | ```bash 137 | ./sample-oracle 138 | ``` 139 | 140 | This will generate a new DFX project in the folder, bootstrap the project, and start the oracle service. 141 | -------------------------------------------------------------------------------- /framework.go: -------------------------------------------------------------------------------- 1 | package framework 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "time" 7 | 8 | "github.com/hyplabs/dfinity-oracle-framework/models" 9 | "github.com/hyplabs/dfinity-oracle-framework/summary" 10 | "github.com/hyplabs/dfinity-oracle-framework/utils" 11 | "github.com/sirupsen/logrus" 12 | ) 13 | 14 | // Oracle is an instance of an oracle 15 | type Oracle struct { 16 | config *models.Config 17 | dfxService *DFXService 18 | engine *models.Engine 19 | log *logrus.Logger 20 | } 21 | 22 | // NewOracle creates a new oracle instance 23 | func NewOracle(config *models.Config, engine *models.Engine) *Oracle { 24 | log := logrus.New() 25 | log.Formatter = &logrus.JSONFormatter{} 26 | 27 | dfxService := NewDFXService(config, log) 28 | 29 | return &Oracle{ 30 | config: config, 31 | dfxService: dfxService, 32 | engine: engine, 33 | log: log, 34 | } 35 | } 36 | 37 | // Bootstrap bootstraps the canister installation 38 | func (o *Oracle) Bootstrap() error { 39 | if err := o.dfxService.createNewDfxProject(); err != nil { 40 | panic(err) 41 | } 42 | if err := o.dfxService.updateCanisterCode(); err != nil { 43 | panic(err) 44 | } 45 | if err := o.dfxService.stopDfxNetwork(); err != nil { 46 | panic(err) 47 | } 48 | if err := o.dfxService.startDfxNetwork(); err != nil { 49 | panic(err) 50 | } 51 | if err := o.dfxService.createWriterIdentityIfNeeded(); err != nil { 52 | panic(err) 53 | } 54 | 55 | canisterExists, err := o.dfxService.doesCanisterExist() 56 | if err != nil { 57 | panic(err) 58 | } 59 | if !canisterExists { 60 | if err := o.dfxService.createCanister(); err != nil { 61 | return err 62 | } 63 | } 64 | if err := o.dfxService.buildCanister(); err != nil { 65 | panic(err) 66 | } 67 | if err := o.dfxService.installCanister(canisterExists); err != nil { 68 | panic(err) 69 | } 70 | canisterRunning, err := o.dfxService.isCanisterRunning() 71 | if err != nil { 72 | panic(err) 73 | } 74 | if !canisterRunning { 75 | if err := o.dfxService.startCanister(); err != nil { 76 | panic(err) 77 | } 78 | } 79 | 80 | isOwner, err := o.dfxService.checkIsOwner() 81 | if err != nil { 82 | panic(err) 83 | } 84 | if !isOwner { 85 | if err := o.dfxService.assignOwnerRole(); err != nil { 86 | panic(err) 87 | } 88 | } 89 | writerPrincipal, err := o.dfxService.getWriterIDPrincipal() 90 | if err != nil { 91 | panic(err) 92 | } 93 | if err := o.dfxService.assignWriterRole(writerPrincipal); err != nil { 94 | panic(err) 95 | } 96 | return nil 97 | } 98 | 99 | // Run starts the Oracle service 100 | func (o *Oracle) Run() { 101 | o.log.Infof("Starting %s oracle service...", o.config.CanisterName) 102 | 103 | o.updateOracle() 104 | for range time.Tick(o.config.UpdateInterval) { 105 | o.updateOracle() 106 | } 107 | } 108 | 109 | func (o *Oracle) updateOracle() { 110 | for _, meta := range o.engine.Metadata { 111 | o.updateMeta(meta) 112 | } 113 | o.log.Infof("Oracle update completed") 114 | } 115 | 116 | func (o *Oracle) updateMeta(meta models.MappingMetadata) error { 117 | type apiInfo struct { 118 | Endpoint models.Endpoint 119 | Value map[string]float64 120 | Err error 121 | } 122 | dataset := make([]map[string]float64, 0) 123 | ch := make(chan apiInfo) 124 | for _, endpoint := range meta.Endpoints { 125 | go func(endpoint models.Endpoint, ch chan<- apiInfo) { 126 | val, err := utils.GetAPIInfo(endpoint) 127 | ch <- apiInfo{Endpoint: endpoint, Value: val, Err: err} 128 | }(endpoint, ch) 129 | } 130 | for range meta.Endpoints { 131 | r := <-ch 132 | if r.Err != nil { 133 | o.log.WithError(r.Err).Errorf("Could not retrieve information from API %s", r.Endpoint.Endpoint) 134 | return r.Err 135 | } 136 | dataset = append(dataset, r.Value) 137 | valStr, err := json.Marshal(r.Value) 138 | if err != nil { 139 | o.log.WithError(err).Errorf("Retrieved non-JSON-serializable value %v from %s for %s", r.Value, r.Endpoint.Endpoint, meta.Key) 140 | return err 141 | } 142 | o.log.Infof("Retrieved value %v from %s for %s", string(valStr), r.Endpoint.Endpoint, meta.Key) 143 | } 144 | if len(dataset) == 0 { 145 | o.log.Errorf("No values from any API endpoints, skipping update for %s", meta.Key) 146 | return fmt.Errorf("No values from any API endpoints, skipping update for %s", meta.Key) 147 | } 148 | 149 | var summarizedVal map[string]float64 150 | if meta.SummaryFunc != nil { 151 | summarizedVal = meta.SummaryFunc(dataset) 152 | } else { 153 | summarizedVal = summary.MeanWithoutOutliers(dataset) 154 | } 155 | o.dfxService.updateValueInCanister(meta.Key, summarizedVal) 156 | return nil 157 | } 158 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/hyplabs/dfinity-oracle-framework 2 | 3 | go 1.14 4 | 5 | require ( 6 | github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 7 | github.com/sirupsen/logrus v1.8.1 8 | golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57 // indirect 9 | ) 10 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 2 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 3 | github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 h1:Yl0tPBa8QPjGmesFh1D0rDy+q1Twx6FyU7VWHi8wZbI= 4 | github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852/go.mod h1:eqOVx5Vwu4gd2mmMZvVZsgIqNSaW3xxRThUJ0k/TPk4= 5 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 6 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 7 | github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM= 8 | github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= 9 | github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE= 10 | github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= 11 | github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w= 12 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 13 | golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4= 14 | golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 15 | golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c h1:VwygUrnw9jn88c4u8GD3rZQbqrP/tgas88tPUbBxQrk= 16 | golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 17 | golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57 h1:F5Gozwx4I1xtr/sr/8CFbb57iKi3297KFs0QDbGN60A= 18 | golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 19 | -------------------------------------------------------------------------------- /models/config.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import "time" 4 | 5 | // Config is the configuration for the oracle to be made 6 | type Config struct { 7 | CanisterName string 8 | UpdateInterval time.Duration 9 | } 10 | -------------------------------------------------------------------------------- /models/endpoint.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | // Endpoint is an endpoint configuration for the oracle 4 | type Endpoint struct { 5 | Endpoint string 6 | JSONPaths map[string]string 7 | NormalizeFunc func(map[string]interface{}) (map[string]float64, error) 8 | } 9 | -------------------------------------------------------------------------------- /models/engine.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | // Engine is the configuration for the oracle core to work 4 | type Engine struct { 5 | Metadata []MappingMetadata 6 | } 7 | -------------------------------------------------------------------------------- /models/mapping_metadata.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | // MappingMetadata is the data required for the smart contract to store arbitrary key-values 4 | type MappingMetadata struct { 5 | Key string 6 | SummaryFunc func([]map[string]float64) map[string]float64 7 | Endpoints []Endpoint 8 | } 9 | -------------------------------------------------------------------------------- /summary/summary.go: -------------------------------------------------------------------------------- 1 | package summary 2 | 3 | import ( 4 | "math" 5 | ) 6 | 7 | // Mean: Returns the man of the dataset 8 | func Mean(dataset []map[string]float64) map[string]float64 { 9 | result := make(map[string]float64) 10 | 11 | for key, values := range groupByKey(dataset) { 12 | result[key] = meanOfArray(values) 13 | } 14 | 15 | return result 16 | } 17 | 18 | // Median: Returns the median of the dataset 19 | func Median(dataset []map[string]float64) map[string]float64 { 20 | result := make(map[string]float64) 21 | 22 | for key, values := range groupByKey(dataset) { 23 | result[key] = medianOfArray(values) 24 | } 25 | 26 | return result 27 | } 28 | 29 | // Mode: Returns the mode of the dataset 30 | func Mode(dataset []map[string]float64) map[string]float64 { 31 | result := make(map[string]float64) 32 | 33 | for key, values := range groupByKey(dataset) { 34 | result[key] = modeOfArray(values) 35 | } 36 | 37 | return result 38 | } 39 | 40 | // MeanWithoutOutliers: Returns the mean of the dataset after removing values outside 2 standard deviations from the median 41 | func MeanWithoutOutliers(dataset []map[string]float64) map[string]float64 { 42 | result := make(map[string]float64) 43 | 44 | for key, values := range groupByKey(dataset) { 45 | result[key] = meanOfArray(RemoveOutlier(values)) 46 | } 47 | 48 | return result 49 | } 50 | 51 | // MedianWithoutOutliers: Returns the median of the dataset after removing values outside 2 std deviations from the median 52 | func MedianWithoutOutliers(dataset []map[string]float64) map[string]float64 { 53 | result := make(map[string]float64) 54 | 55 | for key, values := range groupByKey(dataset) { 56 | result[key] = medianOfArray(RemoveOutlier(values)) 57 | } 58 | 59 | return result 60 | } 61 | 62 | // groupBykey: returns the dataset grouped by keys from each entry in the dataset 63 | func groupByKey(dataset []map[string]float64) map[string][]float64 { 64 | result := make(map[string][]float64) 65 | for _, entry := range dataset { 66 | for k, v := range entry { 67 | result[k] = append(result[k], v) 68 | } 69 | } 70 | return result 71 | } 72 | 73 | func RemoveOutlier(dataset []float64) []float64 { 74 | if len(dataset) <= 2 { 75 | return dataset 76 | } 77 | 78 | var sum float64 = 0.0 79 | var squaredSum float64 = 0.0 80 | for _, x := range dataset { 81 | sum += x 82 | squaredSum += x * x 83 | } 84 | var mean float64 = sum / float64(len(dataset)) 85 | var standardDeviation float64 = math.Sqrt(squaredSum/float64(len(dataset)) - (mean * mean)) 86 | 87 | slicedData := make([]float64, 0) 88 | 89 | median := medianOfArray(dataset) 90 | for _, x := range dataset { 91 | if median-(2*standardDeviation) <= x && x <= median+(2*standardDeviation) { 92 | slicedData = append(slicedData, x) 93 | } 94 | } 95 | return slicedData 96 | } 97 | 98 | func meanOfArray(dataset []float64) float64 { 99 | if len(dataset) == 1 { 100 | return dataset[0] 101 | } 102 | 103 | var sum float64 = 0 104 | for i := 0; i < len(dataset); i++ { 105 | sum += dataset[i] 106 | } 107 | 108 | return sum / float64(len(dataset)) 109 | } 110 | 111 | func medianOfArray(dataset []float64) float64 { 112 | if len(dataset)%2 == 0 { 113 | return (dataset[len(dataset)/2] + dataset[(len(dataset)/2)-1]) / 2 114 | } else { 115 | return dataset[(len(dataset)-1)/2] 116 | } 117 | } 118 | 119 | func modeOfArray(dataset []float64) float64 { 120 | counter := make(map[float64]int) 121 | for _, x := range dataset { 122 | counter[x]++ 123 | } 124 | modeX, modeCount := 0.0, 0 125 | for x, count := range counter { 126 | if count > modeCount { 127 | modeCount = count 128 | modeX = x 129 | } 130 | } 131 | return modeX 132 | } 133 | -------------------------------------------------------------------------------- /summary/summary_test.go: -------------------------------------------------------------------------------- 1 | package summary 2 | 3 | import ( 4 | "reflect" 5 | "testing" 6 | ) 7 | 8 | func TestRemoveOutlierLarge(t *testing.T) { 9 | dataset := []float64{16.6, 23.4, 13.5, 1.1, 52.2, 5.5, 17.1, 50.2, 35.5, 100000000000000} 10 | expectedDataset := []float64{1.1, 5.5, 13.5, 16.6, 17.1, 23.4, 35.5, 50.2, 52.2} 11 | 12 | result := RemoveOutlier(dataset) 13 | 14 | if !reflect.DeepEqual(result, expectedDataset) { 15 | t.Errorf("Incorrect dataset from remove outlier, expected %v, got %v", expectedDataset, result) 16 | } 17 | } 18 | 19 | func TestRemoveOutlieSmall(t *testing.T) { 20 | dataset := []float64{-16.0, -100.0, -18.0, -6.0, -2000, 16.6, 23.4, 13.5, 1.1, 52.2, 5.5, 17.1, 35.5} 21 | expectedDataset := []float64{-100, -18, -16, -6, 1.1, 5.5, 13.5, 16.6, 17.1, 23.4, 35.5, 52.2} 22 | 23 | result := RemoveOutlier(dataset) 24 | 25 | if !reflect.DeepEqual(result, expectedDataset) { 26 | t.Errorf("Incorrect dataset from remove outlier, expected %v, got %v", expectedDataset, result) 27 | } 28 | } 29 | -------------------------------------------------------------------------------- /template.go: -------------------------------------------------------------------------------- 1 | package framework 2 | 3 | const CodeTemplate = `// Import Base Modules 4 | import AssocList "mo:base/AssocList"; 5 | import Error "mo:base/Error"; 6 | import List "mo:base/List"; 7 | import Option "mo:base/Option"; 8 | import Text "mo:base/Text"; 9 | 10 | shared (msg) actor class() { 11 | // Define custom types 12 | public type Role = { 13 | #owner; 14 | #writer; 15 | }; 16 | 17 | // Application State 18 | private stable var owner: ?Principal = null; 19 | private stable var roles: AssocList.AssocList = List.nil(); 20 | private stable var map: AssocList.AssocList> = List.nil(); 21 | private stable var destructed: Bool = false; 22 | 23 | // Favorite Cities Functions 24 | public shared ({caller}) func update_map_value(k: Text, p: Text, v: Float): async() { 25 | await require_role(caller, ?#writer); 26 | await require_undestructed(); 27 | 28 | var sublist: ?AssocList.AssocList = AssocList.find>(map, k, text_eq); 29 | var newSublist: AssocList.AssocList = List.nil(); 30 | var text: Text = ""; 31 | 32 | if (Option.isSome(sublist) == true) { 33 | let flattenedSublist = Option.flatten(sublist); 34 | newSublist := AssocList.replace(flattenedSublist, p, text_eq, ?v).0; 35 | map := AssocList.replace>(map, k, text_eq, ?newSublist).0; 36 | } else if (Option.isNull(sublist) == true) { 37 | newSublist := AssocList.replace(newSublist, p, text_eq, ?v).0; 38 | map := AssocList.replace>(map, k, text_eq, ?newSublist).0; 39 | }; 40 | }; 41 | 42 | public func get_map_value(k: Text): async ?AssocList.AssocList { 43 | await require_undestructed(); 44 | return AssocList.find>(map, k, text_eq); 45 | }; 46 | 47 | public func get_map_field_value(k: Text, p: Text): async ?Float { 48 | await require_undestructed(); 49 | let sublist = AssocList.find>(map, k, text_eq); 50 | if (Option.isSome(sublist)) { 51 | let flattenedSublist = Option.flatten(sublist); 52 | return AssocList.find(flattenedSublist, p, text_eq); 53 | }; 54 | return null; 55 | }; 56 | 57 | public func get_map(): async AssocList.AssocList> { 58 | await require_undestructed(); 59 | return map; 60 | }; 61 | 62 | func text_eq(a: Text, b: Text): Bool { 63 | return a == b; 64 | }; 65 | 66 | // Identity Access Control Functions 67 | func principal_eq(a: Principal, b: Principal): Bool { 68 | return a == b; 69 | }; 70 | 71 | func get_role(p: Principal): ?Role { 72 | if (Option.isNull(owner) == true) { 73 | return null; 74 | } else if (?p == owner) { 75 | return ?#owner; 76 | } else { 77 | return AssocList.find(roles, p, principal_eq); 78 | }; 79 | }; 80 | 81 | func require_role(p: Principal, r: ?Role): async() { 82 | if(r != get_role(p)) { 83 | throw Error.reject("You do not have the required role to perform this operation.") 84 | }; 85 | }; 86 | 87 | func require_undestructed(): async() { 88 | if (destructed == true) { 89 | throw Error.reject("This oracle canister was destructed by the owner. It may have been corrupted or become malicious.") 90 | }; 91 | }; 92 | 93 | public shared ({caller}) func assign_owner_role(): async() { 94 | if (Option.isSome(owner) == true) { 95 | throw Error.reject("Cannot set owner if there is already an owner"); 96 | }; 97 | owner := ?caller; 98 | }; 99 | 100 | public shared ({caller}) func assign_writer_role(p: Principal): async() { 101 | await require_role(caller, ?#owner); 102 | await require_undestructed(); 103 | if (?p == owner) { 104 | throw Error.reject("Specified principal is the canister owner, which cannot also be the canister writer"); 105 | }; 106 | roles := AssocList.replace(roles, p, principal_eq, ?#writer).0; 107 | }; 108 | 109 | public shared ({caller}) func revoke_writer_role(p: Principal): async() { 110 | await require_role(caller, ?#owner); 111 | await require_undestructed(); 112 | if (?p == owner) { 113 | throw Error.reject("Specified principal is the canister owner, which cannot also be the canister writer"); 114 | }; 115 | if (get_role(p) != ?#writer) { 116 | throw Error.reject("Specified principal was not a writer to begin with"); 117 | }; 118 | roles := AssocList.replace(roles, p, principal_eq, null).0; 119 | }; 120 | 121 | public shared ({caller}) func my_role(): async ?Role { 122 | return get_role(caller); 123 | }; 124 | 125 | public shared ({caller}) func get_roles(): async List.List<(Principal, Role)> { 126 | await require_role(caller, ?#owner); 127 | await require_undestructed(); 128 | return roles; 129 | }; 130 | 131 | public shared ({caller}) func self_destruct(): async() { 132 | await require_role(caller, ?#owner); 133 | destructed := true; 134 | map := List.nil(); 135 | roles := List.nil(); 136 | } 137 | }` 138 | -------------------------------------------------------------------------------- /utils/candid_serialize.go: -------------------------------------------------------------------------------- 1 | package utils 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | ) 7 | 8 | // based on the syntax at https://sdk.dfinity.org/docs/candid-guide/candid-types.html 9 | 10 | // CandidText returns the given string in serialized Candid IDL format 11 | func CandidText(value string) string { 12 | var result strings.Builder 13 | result.WriteRune('"') 14 | for _, char := range value { 15 | if char == '\n' { 16 | result.WriteString("\\n") 17 | } else if char == '\r' { 18 | result.WriteString("\\r") 19 | } else if char == '\t' { 20 | result.WriteString("\\t") 21 | } else if char == '\\' { 22 | result.WriteString("\\\\") 23 | } else if char == '"' { 24 | result.WriteString("\\\"") 25 | } else if char == '\'' { 26 | result.WriteString("\\'") 27 | } else if 32 <= char && char <= 126 { // ASCII printable range 28 | result.WriteRune(char) 29 | } else { // other character, do a Unicode escape 30 | result.WriteString(fmt.Sprintf("\\u{%X}", char)) 31 | } 32 | } 33 | result.WriteRune('"') 34 | return result.String() 35 | } 36 | 37 | // CandidInt returns the given int in serialized Candid IDL format 38 | func CandidInt(value int) string { 39 | return fmt.Sprintf("%d", value) 40 | } 41 | 42 | // CandidFloat64 returns the given float64 in serialized Candid IDL format 43 | func CandidFloat64(value float64) string { 44 | return fmt.Sprintf("%f", value) 45 | } 46 | 47 | // CandidBool returns the given bool in serialized Candid IDL format 48 | func CandidBool(value bool) string { 49 | if value { 50 | return "true" 51 | } 52 | return "false" 53 | } 54 | 55 | // CandidPrincipal returns the given principal string in serialized Candid IDL format 56 | func CandidPrincipal(value string) string { 57 | return fmt.Sprintf("principal %s", CandidText(value)) 58 | } 59 | -------------------------------------------------------------------------------- /utils/utils.go: -------------------------------------------------------------------------------- 1 | package utils 2 | 3 | import ( 4 | "encoding/json" 5 | "io/ioutil" 6 | "net/http" 7 | 8 | "github.com/hyplabs/dfinity-oracle-framework/models" 9 | "github.com/oliveagle/jsonpath" 10 | ) 11 | 12 | func getEndpoint(endpoint string) ([]byte, error) { 13 | resp, err := http.Get(endpoint) 14 | if err != nil { 15 | return nil, err 16 | } 17 | 18 | // Read response body 19 | body, err := ioutil.ReadAll(resp.Body) 20 | if err != nil { 21 | return nil, err 22 | } 23 | 24 | return body, nil 25 | } 26 | 27 | // GetAPIInfo takes a given endpoint and parses the endpoint data 28 | // Currently assumes all output is in map of floats format 29 | func GetAPIInfo(e models.Endpoint) (map[string]float64, error) { 30 | responseBody, err := getEndpoint(e.Endpoint) 31 | if err != nil { 32 | return map[string]float64{}, err 33 | } 34 | 35 | var jsonData interface{} 36 | json.Unmarshal(responseBody, &jsonData) 37 | 38 | result := make(map[string]interface{}) 39 | for fieldName, jsonPath := range e.JSONPaths { 40 | resp, err := jsonpath.JsonPathLookup(jsonData, jsonPath) 41 | if err != nil { 42 | return map[string]float64{}, err 43 | } 44 | result[fieldName] = resp 45 | } 46 | 47 | if e.NormalizeFunc != nil { 48 | normalizedResult, err := e.NormalizeFunc(result) 49 | if err != nil { 50 | return map[string]float64{}, err 51 | } 52 | return normalizedResult, nil 53 | } else { 54 | normalizedResult := make(map[string]float64) 55 | for k, v := range result { 56 | normalizedResult[k] = v.(float64) 57 | } 58 | return normalizedResult, nil 59 | } 60 | } 61 | --------------------------------------------------------------------------------