├── .github └── workflows │ └── template-ci.yaml ├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── doc ├── asynccall.jpg ├── asyncdag.jpg ├── design │ ├── 3rd-party-statestore.jpg │ ├── 3rd-party-storage.jpg │ ├── adapter-pattern.jpg │ ├── aggregate-pattern.jpg │ ├── complete-faas.jpg │ └── event-driven-iteration.jpg ├── internal.jpg ├── jetbrains.png ├── overview.jpg ├── synccall.jpg └── tracing.png └── template └── faas-flow ├── .dockerignore ├── Dockerfile ├── config ├── consul_dc.go ├── consul_url.go ├── gateway_url.go ├── parse_int_or_duration_value.go ├── read_timeout.go ├── trace_server.go └── write_timeout.go ├── eventhandler ├── faas_event_handler.go ├── init_request_tracer.go └── trace_handler.go ├── function └── handler.go ├── go.mod ├── go.sum ├── log └── std_out_logger.go ├── main.go ├── openfaas ├── init_data_store.go ├── init_state_store.go ├── openfaas_executor.go ├── openfaas_runtime.go └── read_secret.go └── template.yml /.github/workflows/template-ci.yaml: -------------------------------------------------------------------------------- 1 | name: Template Docker Image CI 2 | 3 | on: [push] 4 | 5 | jobs: 6 | build: 7 | runs-on: ubuntu-latest 8 | steps: 9 | - uses: actions/checkout@v2 10 | - name: Build the Docker image 11 | run: make build-template 12 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Swarvanu Sengupta 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: build 2 | all: build-template 3 | 4 | build-template: 5 | docker build -t faas-flow:test template/faas-flow 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Faas-flow - Function Composition for [OpenFaaS](https://github.com/openfaas/faas) 2 | 3 | ![Template CI](https://github.com/s8sg/faas-flow/workflows/Template%20Docker%20Image%20CI/badge.svg) 4 | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 5 | [![OpenTracing Badge](https://img.shields.io/badge/OpenTracing-enabled-blue.svg)](http://opentracing.io) 6 | [![OpenFaaS](https://img.shields.io/badge/openfaas-serverless-blue.svg)](https://www.openfaas.com) 7 | [![GoDoc](https://godoc.org/github.com/faasflow/lib/openfaas?status.svg)](https://godoc.org/github.com/faasflow/lib/openfaas) 8 | 9 | > - [x] **Pure**              FaaS with [OpenFaaS](https://github.com/openfaas/faas) 10 | > - [x] **Fast**               Built with `Go` 11 | > - [x] **Secured**        With `HMAC` 12 | > - [x] **Stateless**      By design 13 | > - [x] **Tracing**         With `open-tracing` 14 | > - [x] **Available**      As `faas-flow` template 15 | 16 | [**Faas-flow tower**](https://github.com/s8sg/faas-flow-tower) visualizes and monitors flow functions. 17 | 18 | ## Overview 19 | 20 | Faas-flow allows you to realize OpenFaaS function composition with ease. By 21 | defining a simple pipeline, you can orchestrate multiple functions without 22 | having to worry about the internals. 23 | 24 | ```go 25 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 26 | flow.SyncNode().Apply("Func1").Apply("Func2") 27 | return nil 28 | } 29 | ``` 30 | 31 | After building and deploying, it will give you an OpenFaaS function that 32 | orchestrates calling `Func2` with the output of `Func1`. 33 | 34 | ## Use Cases 35 | 36 | Faas-flow as a function composure provides the back-bone for building complex 37 | solutions and promote automation. 38 | 39 | ### Data Processing Pipeline 40 | 41 | Faas-flow can orchestrate a pipeline with long and short running function 42 | performing ETL jobs without having to orchestrate them manually or maintaining a 43 | separate application. Faas-flow ensures the execution order of several functions 44 | running in parallel or dynamically and provides rich construct to aggregate 45 | results while maintaining the intermediate data. 46 | 47 | ### Application Orchestration Workflow 48 | 49 | Functions are great for isolating certain functionalities of an application. 50 | Although one still need to call the functions, write workflow logic, handle 51 | parallel processing and retries on failures. Using Faas-flow you can combine 52 | multiple OpenFaaS functions with little codes while your workflow will scale 53 | up/down automatically to handle the load. 54 | 55 | ### Function Reusability 56 | 57 | Fass-flow allows you to write function only focused on solving one problem 58 | without having to worry about the next. It makes function loosely coupled from 59 | the business logic promoting reusability. You can write the stateless function 60 | and use it across multiple applications, where Faas-flow maintains the execution 61 | state for individual workflow per requests. 62 | 63 | ## Pipeline Definition 64 | 65 | By supplying a number of pipeline operators, the complex composition can be 66 | achieved with little work: 67 | ![alt overview](https://github.com/s8sg/faas-flow/blob/master/doc/overview.jpg) 68 | 69 | The above pipelines can be achieved with little, but powerful code: 70 | 71 | ### Sync chain 72 | 73 | ```go 74 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 75 | flow.SyncNode() 76 | .Apply("func1") 77 | .Apply("func2") 78 | .Modify(func(data []byte) ([]byte, error) { 79 | // do something 80 | return data, nil 81 | }) 82 | return nil 83 | } 84 | ``` 85 | 86 | ### Async chain 87 | 88 | ```go 89 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 90 | dag := flow.Dag() 91 | dag.Node("n1").Apply("func1") 92 | dag.Node("n2") 93 | .Apply("func2") 94 | .Modify(func(data []byte) ([]byte, error) { 95 | // do something 96 | return data, nil 97 | }) 98 | dag.Node("n3").Apply("func4") 99 | dag.Edge("n1", "n2") 100 | dag.Edge("n2", "n3") 101 | return nil 102 | } 103 | ``` 104 | 105 | ### Parallel branching 106 | 107 | ```go 108 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 109 | dag := flow.Dag() 110 | dag.Node("n1").Modify(func(data []byte) ([]byte, error) { 111 | // do something 112 | return data, nil 113 | }) 114 | dag.Node("n2").Apply("func1") 115 | dag.Node("n3").Apply("func2").Modify(func(data []byte) ([]byte, error) { 116 | // do something 117 | return data, nil 118 | }) 119 | dag.Node("n4", faasflow.Aggregator(func(data map[string][]byte) ([]byte, error) { 120 | // aggregate branch result data["n2"] and data["n3"] 121 | return []byte(""), nil 122 | })) 123 | 124 | dag.Edge("n1", "n2") 125 | dag.Edge("n1", "n3") 126 | dag.Edge("n2", "n4") 127 | dag.Edge("n3", "n4") 128 | return nil 129 | } 130 | ``` 131 | 132 | ### Dynamic branching 133 | 134 | ```go 135 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 136 | dag := flow.Dag() 137 | dag.Node("n1").Modify(func(data []byte) ([]byte, error) { 138 | // do something 139 | return data, nil 140 | }) 141 | conditionalDags := dag.ConditionalBranch( 142 | "C", 143 | []string{"c1", "c2"}, // possible conditions 144 | func(response []byte) []string { 145 | // for each returned condition the corresponding branch will execute 146 | // this function executes in the runtime of condition C 147 | return []string{"c1", "c2"} 148 | }, 149 | faasflow.Aggregator(func(data map[string][]byte) ([]byte, error) { 150 | // aggregate all dynamic branches results 151 | return []byte(""), nil 152 | }), 153 | ) 154 | 155 | conditionalDags["c2"].Node("n1").Apply("func1").Modify(func(data []byte) ([]byte, error) { 156 | // do something 157 | return data, nil 158 | }) 159 | foreachDag := conditionalDags["c1"].ForEachBranch( 160 | "F", 161 | func(data []byte) map[string][]byte { 162 | // for each returned key in the hashmap a new branch will be executed 163 | // this function executes in the runtime of foreach F 164 | return map[string][]byte{"f1": data, "f2": data} 165 | }, 166 | faasflow.Aggregator(func(data map[string][]byte) ([]byte, error) { 167 | // aggregate all dynamic branches results 168 | return []byte(""), nil 169 | }), 170 | ) 171 | foreachDag.Node("n1").Modify(func(data []byte) ([]byte, error) { 172 | // do something 173 | return data, nil 174 | }) 175 | dag.Node("n2") 176 | dag.Edge("n1", "C") 177 | dag.Edge("C", "n2") 178 | 179 | return nil 180 | } 181 | ``` 182 | 183 | Full implementation of the above examples are available 184 | [here](https://github.com/s8sg/faasflow-example). 185 | 186 | OpenFaaS flow librray are available as [github.com/faasflow/lib/openfaas](https://godoc.org/github.com/faasflow/lib/openfaas) 187 | 188 | ## Faas-flow Design 189 | 190 | The current design consideration is made based on the below goals: 191 | 192 | 1. Leverage the OpenFaaS platform 193 | 2. Not to violate the notions of function 194 | 3. Provide flexibility, scalability, and adaptability 195 | 196 | ### Just as function as any other 197 | 198 | Faas-flow is deployed and provisioned just like any other OpenFaaS function. It 199 | allows Faas-flow to take advantage of rich functionalities available on 200 | OpenFaaS. Faas-flow provide an OpenFaaS template (`faas-flow`) and just like any 201 | other OpenFaaS function it can be deployed with `faas-cli`. 202 | 203 | ![alt its a function](https://github.com/s8sg/faas-flow/blob/master/doc/design/complete-faas.jpg) 204 | 205 | ### Adapter pattern for zero instrumentation in code 206 | 207 | Faas-flow function follows the adapter pattern. Here the adaptee is the 208 | functions and the adapter is the flow. For each node execution, Faas-flow handle 209 | the calls to the functions. Once the execution is over, it forwards an event to 210 | itself. This way the arrangement logic is separated from the functions and is 211 | implemented in the adapter. Compositions need no code instrumentations, making 212 | functions completely independent of the details of the compositions. 213 | 214 | ![alt function is independent of composition](https://github.com/s8sg/faas-flow/blob/master/doc/design/adapter-pattern.jpg) 215 | 216 | ### Aggregate pattern as chaining 217 | 218 | Aggregation of separate function calls is done as chaining. Multiple functions 219 | can be called from a single node with order maintained as per the chain. This 220 | way one execution node can be implemented as an aggregator function that invokes 221 | multiple functions collects the results, optionally applies business logic, and 222 | returns a consolidated response to the client or forward to next nodes. 223 | Faas-flow fuses the adapter pattern and aggregate pattern to support more 224 | complex use cases. 225 | 226 | ![alt aggregation](https://github.com/s8sg/faas-flow/blob/master/doc/design/aggregate-pattern.jpg) 227 | 228 | ### Event driven iteration 229 | 230 | OpenFaaS uses [Nats](https://nats.io) for event delivery and Faas-flow leverages 231 | OpenFaaS platform. Node execution in Faas-flow starts by a completion event of 232 | one or more previous nodes. A completion event denotes that all the previous 233 | dependent nodes have completed. The event carries the execution state and 234 | identifies the next node to execute. With events Faas-flow asynchronously 235 | carry-on execution of nodes by iterating itself over and over till all nodes are 236 | executed. 237 | 238 | ![alt iteration](https://github.com/s8sg/faas-flow/blob/master/doc/design/event-driven-iteration.jpg) 239 | 240 | ### 3rd party KV store for coordination 241 | 242 | When executing branches, one node is dependent on more than one predecessor 243 | nodes. In that scenario, the event for completion is generated by coordination 244 | of earlier nodes. Like any distributed system the coordination is achieved via a 245 | centralized service. Faas-flow keeps the logic of the coordination controller 246 | inside of Faas-flow implementation and lets the user use any external 247 | synchronous KV store by implementing 248 | [`StateStore`](https://godoc.org/github.com/faasflow/sdk#StateStore). 249 | 250 | ![alt coordination](https://github.com/s8sg/faas-flow/blob/master/doc/design/3rd-party-statestore.jpg) 251 | 252 | ### 3rd party Storage for intermediate data 253 | 254 | Results from function execution and intermediate data can be handled by the user 255 | manually. Faas-flow provides data-store for intermediate result storage. It 256 | automatically initializes, store, retrieve and remove data between nodes. This 257 | fits great for data processing applications. Faas-flow keeps the logic of 258 | storage controller inside of Faas-flow implementation and lets the user use any 259 | external object storage by implementing 260 | [`DataStore`](https://godoc.org/github.com/faasflow/sdk#DataStore). 261 | 262 | ![alt storage](https://github.com/s8sg/faas-flow/blob/master/doc/design/3rd-party-storage.jpg) 263 | 264 | Faas-flow design is not fixed and like any good design, it is evolving. Please 265 | contribute to make it better. 266 | 267 | ## Getting Started 268 | 269 | ### Deploy OpenFaaS 270 | 271 | FaasFlow requires the OpenFaaS to be deployed and the OpenFaaS Cli to be installed. You 272 | can either have your OpenFaaS deployed in [Kubernets](https://kubernetes.io) or 273 | in [Swarm](https://docs.docker.com/engine/swarm/). 274 | 275 | To deploy OpenFaaS and to 276 | install the OpenFaaS cli client follow this guide: 277 | [https://docs.openfaas.com/deployment/](https://docs.openfaas.com/deployment/). 278 | 279 | ### Deploy Faas-flow Components with Faas-flow Infra 280 | 281 | [Faas-Flow infra](https://github.com/s8sg/faas-flow-infra) provides the kubernetes and swarm deployment resources for faas-flow dependencies. Follow the [README](https://github.com/faasflow/faas-flow-infra#getting-started) to deploy Faas-Flow Infra 282 | in Kubernets or in Swarm 283 | 284 | ### Deploy Faas-flow Tower 285 | 286 | [Faas-Flow tower](https://github.com/faasflow/faas-flow-tower) provides the dashboard to visualise and monitor your flow. Follow the [README](https://github.com/faasflow/faas-flow-tower#deploy-faas-flow-tower) to deploy Faas-Flow tower on OpenFaaS 287 | 288 | ### Writing Flow 289 | 290 | This example implements a very simple flow to `Greet` 291 | 292 | #### Get template 293 | 294 | Pull `faas-flow` template with the `faas-cli` 295 | 296 | ```shell 297 | faas template pull https://github.com/s8sg/faas-flow 298 | ``` 299 | 300 | #### Create new flow function 301 | 302 | Create a new function using `faas-flow` template 303 | 304 | ```shell 305 | faas new greet --lang faas-flow 306 | ``` 307 | 308 | #### Edit stack.yml 309 | 310 | Edit function stack file `greet.yml` 311 | 312 | ```yaml 313 | greet: 314 | lang: faas-flow 315 | handler: ./greet 316 | image: greet:latest 317 | labels: 318 | faas-flow: 1 319 | annotations: 320 | faas-flow-desc: "test flow to greet" 321 | environment_file: 322 | - flow.yml 323 | secrets: 324 | - s3-secret-key 325 | - s3-access-key 326 | ``` 327 | 328 | #### Add configuration 329 | 330 | Add a separate configuration file `flow.yml` with faas-flow related configuration. 331 | 332 | ```yaml 333 | environment: 334 | gateway: "gateway.openfaas:8080" # The address of OpenFaaS gateway 335 | enable_tracing: true # tracing allows to monitor requests 336 | trace_server: "jaeger-agent.faasflow:5775" # The address of jaeger tracing agent 337 | consul_url: "consul.faasflow:8500" # The address of consul 338 | s3_url: "minio.faasflow:9000" # The address of minio 339 | ``` 340 | 341 | #### Edit flow definition 342 | 343 | Edit `greet/handler.go` and Update `Define()` 344 | 345 | ```go 346 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 347 | flow.SyncNode().Modify(func(data []byte) ([]byte, error) { 348 | result := "Hello " + string(data) 349 | return []byte(result), nil 350 | }) 351 | return nil 352 | } 353 | ``` 354 | 355 | #### Build and Deploy 356 | 357 | Build and deploy 358 | 359 | ```shell 360 | faas build -f greet.yml 361 | faas deploy -f greet.yml 362 | ``` 363 | 364 | This function will generate one Synchronous node 365 | 366 | ```text 367 | Modify("name") -> Hello name 368 | ``` 369 | 370 | All calls will be performed in one single execution of the flow function and 371 | result will be returned to the callee. 372 | 373 | > Note: For flow that has more than one nodes, Faas-flow doesn't return any 374 | > response. External storage or callback can be used to retrieve an async result. 375 | 376 | #### Invoke 377 | 378 | ```shell 379 | echo "Adam" | faas invoke greet 380 | ``` 381 | 382 | 383 | 384 | ## Request Tracking by ID 385 | 386 | For each new request, faas-flow generates a unique `Request Id` for the flow. 387 | The same Id is used when logging. 388 | 389 | ```shell 390 | 2018/08/13 07:51:59 [Request `bdojh7oi7u6bl8te4r0g`] Created 391 | 2018/08/13 07:52:03 [Request `bdojh7oi7u6bl8te4r0g`] Received 392 | ``` 393 | 394 | The assigned request Id is set on the response header `X-Faas-Flow-Reqid` 395 | One may provide custom request Id by setting `X-Faas-Flow-Reqid` in the request 396 | header. 397 | 398 | ## Request Tracing with [Faas-Flow-Tower](https://github.com/s8sg/faas-flow-tower) 399 | 400 | FaasFlow Tower enables the real time monitoring 401 | for each requests. Request traces are visible when `enable_tracing` is enabled. FaaSFlow is 402 | the best way to monitor flows and execution status of each node for each request. 403 | 404 | Below is an example of tracing page for a request of 405 | [faas-flow-example](https://github.com/faasflow/faas-flow-example). 406 | 407 | ![alt monitoring](https://github.com/s8sg/faas-flow-tower/blob/master/doc/monitoring.png) 408 | 409 | ## Use of Callback 410 | 411 | To receive a result of long running **FaaSFlow** request, you can specify the 412 | `X-Faas-Flow-Callback-Url`. FaaSFlow will invoked the callback URL with the 413 | final result and with the request ID set as `X-Faas-Flow-Reqid` in request 414 | Header. 415 | > Note: `X-Callback-Url` from OpenFaaS is not supported in FaaSFlow. 416 | 417 | ## Pause, Resume or Stop Request 418 | 419 | A request in faas-flow has three states: 420 | 421 | 1. Running 422 | 2. Paused 423 | 3. Stopped 424 | 425 | Faas-flow doesn't keep the state of a finished request 426 | 427 | To pause a running request: 428 | 429 | ```shell 430 | faas invoke --query pause-flow= 431 | ``` 432 | 433 | To resume a paused request 434 | 435 | ```shell 436 | faas invoke --query resume-flow= 437 | ``` 438 | 439 | To stop an active (paused/running) request 440 | 441 | ```shell 442 | faas invoke --query stop-flow= 443 | ``` 444 | 445 | ## Use of context 446 | 447 | Context can be used inside definition for different use cases. Context provide 448 | various information such as: 449 | 450 | - **HttpQuery** to retrieve original request queries 451 | - **State** to get flow state 452 | - **Node** to get current node 453 | along with that it wraps the **DataStore** to store data 454 | 455 | ### Store data in context with `DataStore` 456 | 457 | Context uses `DataStore` to store/retrieve data. User can do the same by calling 458 | `Get()`, `Set()`, and `Del()` from `context`: 459 | 460 | ```go 461 | flow.SyncNode(). 462 | Modify(func(data []byte) { 463 | // parse data and set to be used later 464 | // json.Unmarshal(&req, data) 465 | context.Set("commitsha", req.Sha) 466 | }) 467 | .Apply("myfunc") 468 | .Modify(func(data []byte) { 469 | // retrieve the data that was set in the context 470 | commitsha, _ = context.GetString("commitsha") 471 | // use the query 472 | }) 473 | ``` 474 | 475 | ### Getting Http Query to Workflow 476 | 477 | Http Query to flow can be used retrieved from context using `context.Query` 478 | 479 | ```go 480 | flow.SyncNode() 481 | .Apply("myfunc", Query("auth-token", context.Query.Get("token"))) // pass as a function query 482 | .Modify(func(data []byte) { 483 | token = context.Query.Get("token") // get query inside modifier 484 | }) 485 | ``` 486 | 487 | ### Use of request context 488 | 489 | Node, requestId, State is provided by the `context` 490 | 491 | ```go 492 | currentNode := context.GetNode() 493 | requestId := context.GetRequestId() 494 | state := context.State 495 | ``` 496 | 497 | for more details check Faas-flow 498 | [GoDoc](https://godoc.org/github.com/s8sg/faas-flow). 499 | 500 | ## External `StateStore` for coordination controller 501 | 502 | Faas-flow implements coordination controller and store the intermediate request 503 | with StateStore. By default Faas-flow uses 504 | [consul](https://github.com/faasflow/faas-flow-consul-statestore) as default 505 | state-store, although user can define custom state-store with `StateStore` 506 | interface and use any external Synchronous KV store as backend. 507 | 508 | ```go 509 | type StateStore interface { 510 | // Configure the StateStore with flow name and request ID 511 | Configure(flowName string, requestId string) 512 | // Initialize the StateStore (called only once in a request span) 513 | Init() error 514 | // Set a value (override existing, or create one) 515 | Set(key string, value string) error 516 | // Get a value 517 | Get(key string) (string, error) 518 | // Compare and Update a value 519 | Update(key string, oldValue string, newValue string) error 520 | // Cleanup all the resorces in StateStore (called only once in a request span) 521 | Cleanup() error 522 | } 523 | ``` 524 | 525 | The custom `StateStore` can be set with `OverrideStateStore()` at 526 | `function/handler.go`: 527 | 528 | ```go 529 | // OverrideStateStore provides the override of the default StateStore 530 | func OverrideStateStore() (faasflow.StateStore, error) { 531 | myss, err := myStateStore.Init() 532 | return myss, err 533 | } 534 | ``` 535 | 536 | `StateStore` is mandatory for a FaaSFlow to operate. 537 | 538 | ### Official state-stores 539 | 540 | - **[ConsulStateStore](https://github.com/faasflow/faas-flow-consul-statestore)**: 541 | statestore implementation with **consul** (default); 542 | - **[EtcdStateStore](https://github.com/s8sg/faas-flow-etcd-statestore)**: 543 | statewtore implementation with **etcd**. 544 | 545 | ## External `DataStore` for storage controller 546 | 547 | Faas-flow uses the `DataStore` to store partially completed data between nodes 548 | and request context data. By default Faas-flow uses 549 | [minio](https://github.com/faasflow/faas-flow-minio-datastore) as default data-store, 550 | although user can define custom data-store with `DataStore` interface and use 551 | any external storage as backend. 552 | 553 | ```go 554 | type DataStore interface { 555 | // Configure the DaraStore with flow name and request ID 556 | Configure(flowName string, requestId string) 557 | // Initialize the DataStore (called only once in a request span) 558 | Init() error 559 | // Set store a value for key, in failure returns error 560 | Set(key string, value string) error 561 | // Get retrives a value by key, if failure returns error 562 | Get(key string) (string, error) 563 | // Del delets a value by a key 564 | Del(key string) error 565 | // Cleanup all the resorces in DataStore 566 | Cleanup() error 567 | } 568 | ``` 569 | 570 | Data Store can be implemented and set by user at the `OverrideDataStore()` at 571 | `function/handler.go`: 572 | 573 | ```go 574 | // OverrideDataStore provides the override of the default DataStore 575 | func OverrideDataStore() (faasflow.DataStore, error) { 576 | myds, err := myDs.Init() 577 | return myds, err 578 | } 579 | ``` 580 | 581 | `DataStore` is mandatory for a FaaSFlow to operate. 582 | 583 | ### Available data-stores 584 | 585 | - **[MinioDataStore](https://github.com/faasflow/faas-flow-minio-datastore)**: 586 | allows to store data in **amazon s3** or local **minio DB** (default). 587 | 588 | ## Cleanup with `Finally()` 589 | 590 | Finally provides an efficient way to perform post-execution steps of the flow. 591 | If specified `Finally()` invokes in case of both failure and success of the 592 | flow. A Finally method can be set as: 593 | 594 | ```go 595 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 596 | // Define flow 597 | flow.SyncNode().Modify(func(data []byte) { 598 | // parse data and set to be used later 599 | // json.Unmarshal(&req, data) 600 | context.Set("commitsha", req.Sha) 601 | }). 602 | Apply("myfunc").Modify(func(data []byte) { 603 | // retrieve the data in different node from context 604 | commitsha, _ = context.GetString("commitsha") 605 | }) 606 | flow.OnFailure(func(err error) { 607 | // failure handler 608 | }) 609 | flow.Finally(func() { 610 | // delete the state resource 611 | context.Del("commitsha") 612 | }) 613 | } 614 | ``` 615 | 616 | ## Contribute 617 | 618 | - **Issue/Suggestion** Create an issue at 619 | [Faas-flow-issue](https://github.com/s8sg/faas-flow/issues). 620 | - **ReviewPR/Implement** Create Pull Request at 621 | [Faas-flow-pr](https://github.com/s8sg/faas-flow/issues). 622 | 623 | Join Faasflow [Slack](https://join.slack.com/t/faas-flow/shared_invite/enQtNzgwNDY2MjI4NTc5LWZiOGQ4M2ZlZTI0OTI0ZjU5YmUyMDgwOWJiOWU0YzIzMGQ3Y2QxMTMzMDlhZGZhYWFlZTkzMGQxMzU4NDdmOGU) 624 | for more. 625 | 626 | ## Supported By 627 | 628 | 629 | 630 | -------------------------------------------------------------------------------- /doc/asynccall.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/asynccall.jpg -------------------------------------------------------------------------------- /doc/asyncdag.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/asyncdag.jpg -------------------------------------------------------------------------------- /doc/design/3rd-party-statestore.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/3rd-party-statestore.jpg -------------------------------------------------------------------------------- /doc/design/3rd-party-storage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/3rd-party-storage.jpg -------------------------------------------------------------------------------- /doc/design/adapter-pattern.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/adapter-pattern.jpg -------------------------------------------------------------------------------- /doc/design/aggregate-pattern.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/aggregate-pattern.jpg -------------------------------------------------------------------------------- /doc/design/complete-faas.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/complete-faas.jpg -------------------------------------------------------------------------------- /doc/design/event-driven-iteration.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/design/event-driven-iteration.jpg -------------------------------------------------------------------------------- /doc/internal.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/internal.jpg -------------------------------------------------------------------------------- /doc/jetbrains.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/jetbrains.png -------------------------------------------------------------------------------- /doc/overview.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/overview.jpg -------------------------------------------------------------------------------- /doc/synccall.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/synccall.jpg -------------------------------------------------------------------------------- /doc/tracing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/s8sg/faas-flow/a53000e2bea1fa067fc17bb7de8664fed3121a46/doc/tracing.png -------------------------------------------------------------------------------- /template/faas-flow/.dockerignore: -------------------------------------------------------------------------------- 1 | Dockerfile 2 | template.yml 3 | -------------------------------------------------------------------------------- /template/faas-flow/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ghcr.io/openfaas/of-watchdog as watchdog 2 | FROM golang:1.14.3-alpine3.11 as build 3 | 4 | COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog 5 | RUN chmod +x /usr/bin/fwatchdog 6 | 7 | RUN mkdir -p /go/src/handler 8 | WORKDIR /go/src/handler 9 | 10 | RUN apk update && \ 11 | apk --no-cache add \ 12 | gcc \ 13 | musl-dev 14 | 15 | COPY ./go.mod ./go.sum ./ 16 | 17 | RUN go mod download 18 | 19 | COPY . . 20 | 21 | # Run a gofmt and exclude all vendored code. 22 | RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 1; } 23 | 24 | RUN CGO_ENABLED=0 GOOS=linux \ 25 | go build --ldflags "-s -w" -a -installsuffix cgo -o handler . && \ 26 | go test $(go list ./... | grep -v /vendor/) -cover 27 | 28 | FROM alpine:3.11.6 29 | # Add non root user and certs 30 | RUN apk --no-cache add ca-certificates \ 31 | && addgroup -S app && adduser -S -g app app \ 32 | && mkdir -p /home/app \ 33 | && chown app /home/app 34 | 35 | WORKDIR /home/app 36 | 37 | COPY --from=build /go/src/handler/handler . 38 | COPY --from=build /usr/bin/fwatchdog . 39 | COPY --from=build /go/src/handler/function/ . 40 | 41 | RUN chown -R app /home/app 42 | 43 | USER app 44 | 45 | ENV fprocess="./handler" 46 | ENV mode="http" 47 | ENV upstream_url="http://127.0.0.1:8082" 48 | 49 | ENV read_timeout: 120 50 | ENV write_timeout: 120 51 | ENV exec_timeout: 0 52 | ENV write_debug: true 53 | ENV combine_output: false 54 | 55 | CMD ["./fwatchdog"] 56 | -------------------------------------------------------------------------------- /template/faas-flow/config/consul_dc.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | ) 6 | 7 | func ConsulDC() string { 8 | val := os.Getenv("consul_dc") 9 | if len(val) == 0 { 10 | val = "dc1" 11 | } 12 | return val 13 | } 14 | -------------------------------------------------------------------------------- /template/faas-flow/config/consul_url.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | ) 6 | 7 | func ConsulURL() string { 8 | val := os.Getenv("consul_url") 9 | if len(val) == 0 { 10 | val = "consul.faasflow:8500" 11 | } 12 | return val 13 | } 14 | -------------------------------------------------------------------------------- /template/faas-flow/config/gateway_url.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | ) 6 | 7 | // GatewayURL return the gateway address from env 8 | func GatewayURL() string { 9 | gateway := os.Getenv("gateway") 10 | if gateway == "" { 11 | gateway = "gateway.openfaas:8080" 12 | } 13 | return gateway 14 | } 15 | -------------------------------------------------------------------------------- /template/faas-flow/config/parse_int_or_duration_value.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "strconv" 5 | "time" 6 | ) 7 | 8 | func parseIntOrDurationValue(val string, fallback time.Duration) time.Duration { 9 | if len(val) > 0 { 10 | parsedVal, parseErr := strconv.Atoi(val) 11 | if parseErr == nil && parsedVal >= 0 { 12 | return time.Duration(parsedVal) * time.Second 13 | } 14 | } 15 | 16 | duration, durationErr := time.ParseDuration(val) 17 | if durationErr != nil { 18 | return fallback 19 | } 20 | return duration 21 | } 22 | -------------------------------------------------------------------------------- /template/faas-flow/config/read_timeout.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | "time" 6 | ) 7 | 8 | func ReadTimeout() time.Duration { 9 | return parseIntOrDurationValue(os.Getenv("read_timeout"), 10*time.Second) 10 | } 11 | -------------------------------------------------------------------------------- /template/faas-flow/config/trace_server.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | ) 6 | 7 | // TraceServer get the traceserver address 8 | func TraceServer() string { 9 | traceServer := os.Getenv("trace_server") 10 | if traceServer == "" { 11 | traceServer = "jaeger.faasflow:5775" 12 | } 13 | return traceServer 14 | } 15 | -------------------------------------------------------------------------------- /template/faas-flow/config/write_timeout.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | import ( 4 | "os" 5 | "time" 6 | ) 7 | 8 | func WriteTimeout() time.Duration { 9 | return parseIntOrDurationValue(os.Getenv("write_timeout"), 10*time.Second) 10 | } 11 | -------------------------------------------------------------------------------- /template/faas-flow/eventhandler/faas_event_handler.go: -------------------------------------------------------------------------------- 1 | package eventhandler 2 | 3 | import ( 4 | "fmt" 5 | "net/http" 6 | ) 7 | 8 | // implements faasflow.EventHandler 9 | type FaasEventHandler struct { 10 | CurrentNodeID string // used to inject current node id in Tracer 11 | Tracer *TraceHandler // handle traces with open-tracing 12 | flowName string 13 | Header http.Header 14 | } 15 | 16 | func (eh *FaasEventHandler) Configure(flowName string, requestID string) { 17 | eh.flowName = flowName 18 | } 19 | 20 | func (eh *FaasEventHandler) Init() error { 21 | var err error 22 | 23 | // initialize trace server if tracing enabled 24 | eh.Tracer, err = initRequestTracer(eh.flowName) 25 | if err != nil { 26 | return fmt.Errorf("failed to init request Tracer, error %v", err) 27 | } 28 | return nil 29 | } 30 | 31 | func (eh *FaasEventHandler) ReportRequestStart(requestID string) { 32 | eh.Tracer.StartReqSpan(requestID) 33 | } 34 | 35 | func (eh *FaasEventHandler) ReportRequestFailure(requestID string, err error) { 36 | // TODO: add log 37 | eh.Tracer.StopReqSpan() 38 | } 39 | 40 | func (eh *FaasEventHandler) ReportExecutionForward(currentNodeID string, requestID string) { 41 | eh.CurrentNodeID = currentNodeID 42 | } 43 | 44 | func (eh *FaasEventHandler) ReportExecutionContinuation(requestID string) { 45 | eh.Tracer.ContinueReqSpan(requestID, eh.Header) 46 | } 47 | 48 | func (eh *FaasEventHandler) ReportRequestEnd(requestID string) { 49 | eh.Tracer.StopReqSpan() 50 | } 51 | 52 | func (eh *FaasEventHandler) ReportNodeStart(nodeID string, requestID string) { 53 | eh.Tracer.StartNodeSpan(nodeID, requestID) 54 | } 55 | 56 | func (eh *FaasEventHandler) ReportNodeEnd(nodeID string, requestID string) { 57 | eh.Tracer.StopNodeSpan(nodeID) 58 | } 59 | 60 | func (eh *FaasEventHandler) ReportNodeFailure(nodeID string, requestID string, err error) { 61 | // TODO: add log 62 | eh.Tracer.StopNodeSpan(nodeID) 63 | } 64 | 65 | func (eh *FaasEventHandler) ReportOperationStart(operationID string, nodeID string, requestID string) { 66 | eh.Tracer.StartOperationSpan(nodeID, requestID, operationID) 67 | } 68 | 69 | func (eh *FaasEventHandler) ReportOperationEnd(operationID string, nodeID string, requestID string) { 70 | eh.Tracer.StopOperationSpan(nodeID, operationID) 71 | } 72 | 73 | func (eh *FaasEventHandler) ReportOperationFailure(operationID string, nodeID string, requestID string, err error) { 74 | // TODO: add log 75 | eh.Tracer.StopOperationSpan(nodeID, operationID) 76 | } 77 | 78 | func (eh *FaasEventHandler) Flush() { 79 | eh.Tracer.FlushTracer() 80 | } 81 | -------------------------------------------------------------------------------- /template/faas-flow/eventhandler/init_request_tracer.go: -------------------------------------------------------------------------------- 1 | package eventhandler 2 | 3 | import ( 4 | "fmt" 5 | "time" 6 | 7 | hconfig "handler/config" 8 | 9 | "github.com/opentracing/opentracing-go" 10 | "github.com/uber/jaeger-client-go" 11 | "github.com/uber/jaeger-client-go/config" 12 | ) 13 | 14 | // initRequestTracer init global trace with configuration 15 | func initRequestTracer(flowName string) (*TraceHandler, error) { 16 | tracerObj := &TraceHandler{} 17 | 18 | agentPort := hconfig.TraceServer() 19 | 20 | cfg := config.Configuration{ 21 | ServiceName: flowName, 22 | Sampler: &config.SamplerConfig{ 23 | Type: "const", 24 | Param: 1, 25 | }, 26 | Reporter: &config.ReporterConfig{ 27 | LogSpans: true, 28 | BufferFlushInterval: 1 * time.Second, 29 | LocalAgentHostPort: agentPort, 30 | }, 31 | } 32 | 33 | opentracer, traceCloser, err := cfg.NewTracer( 34 | config.Logger(jaeger.StdLogger), 35 | ) 36 | if err != nil { 37 | return nil, fmt.Errorf("failed to init Tracer, error %v", err.Error()) 38 | } 39 | 40 | tracerObj.closer = traceCloser 41 | tracerObj.tracer = opentracer 42 | tracerObj.nodeSpans = make(map[string]opentracing.Span) 43 | tracerObj.operationSpans = make(map[string]map[string]opentracing.Span) 44 | 45 | return tracerObj, nil 46 | } 47 | -------------------------------------------------------------------------------- /template/faas-flow/eventhandler/trace_handler.go: -------------------------------------------------------------------------------- 1 | package eventhandler 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/opentracing/opentracing-go" 7 | "github.com/opentracing/opentracing-go/ext" 8 | 9 | "io" 10 | "net/http" 11 | ) 12 | 13 | type TraceHandler struct { 14 | tracer opentracing.Tracer 15 | closer io.Closer 16 | 17 | reqSpan opentracing.Span 18 | reqSpanCtx opentracing.SpanContext 19 | 20 | nodeSpans map[string]opentracing.Span 21 | operationSpans map[string]map[string]opentracing.Span 22 | } 23 | 24 | // StartReqSpan starts a request span 25 | func (tracerObj *TraceHandler) StartReqSpan(reqID string) { 26 | tracerObj.reqSpan = tracerObj.tracer.StartSpan(reqID) 27 | tracerObj.reqSpan.SetTag("request", reqID) 28 | tracerObj.reqSpanCtx = tracerObj.reqSpan.Context() 29 | } 30 | 31 | // ContinueReqSpan continue request span 32 | func (tracerObj *TraceHandler) ContinueReqSpan(reqID string, header http.Header) { 33 | var err error 34 | 35 | tracerObj.reqSpanCtx, err = tracerObj.tracer.Extract( 36 | opentracing.HTTPHeaders, 37 | opentracing.HTTPHeadersCarrier(header), 38 | ) 39 | if err != nil { 40 | fmt.Printf("[Request %s] failed to continue req span for tracing, error %v\n", reqID, err) 41 | return 42 | } 43 | 44 | tracerObj.reqSpan = nil 45 | // TODO: Its not Supported to get span from spanContext as of now 46 | // https://github.com/opentracing/specification/issues/81 47 | // it will support us to extend the request span for nodes 48 | //reqSpan = opentracing.SpanFromContext(reqSpanCtx) 49 | } 50 | 51 | // ExtendReqSpan extend req span over a request 52 | // func ExtendReqSpan(url string, req *http.Request) { 53 | func (tracerObj *TraceHandler) ExtendReqSpan(reqID string, lastNode string, url string, req *http.Request) { 54 | // TODO: as requestSpan can't be regenerated with the span context we 55 | // forward the nodes SpanContext 56 | // span := reqSpan 57 | span := tracerObj.nodeSpans[lastNode] 58 | if span == nil { 59 | return 60 | } 61 | 62 | ext.SpanKindRPCClient.Set(span) 63 | ext.HTTPUrl.Set(span, url) 64 | ext.HTTPMethod.Set(span, "POST") 65 | err := span.Tracer().Inject( 66 | span.Context(), 67 | opentracing.HTTPHeaders, 68 | opentracing.HTTPHeadersCarrier(req.Header), 69 | ) 70 | if err != nil { 71 | fmt.Printf("[Request %s] failed to extend req span for tracing, error %v\n", reqID, err) 72 | } 73 | if req.Header.Get("Uber-Trace-Id") == "" { 74 | fmt.Printf("[Request %s] failed to extend req span for tracing, error Uber-Trace-Id not set\n", 75 | reqID) 76 | } 77 | } 78 | 79 | // StopReqSpan terminate a request span 80 | func (tracerObj *TraceHandler) StopReqSpan() { 81 | if tracerObj.reqSpan == nil { 82 | return 83 | } 84 | 85 | tracerObj.reqSpan.Finish() 86 | } 87 | 88 | // StartNodeSpan starts a node span 89 | func (tracerObj *TraceHandler) StartNodeSpan(node string, reqID string) { 90 | 91 | tracerObj.nodeSpans[node] = tracerObj.tracer.StartSpan( 92 | node, ext.RPCServerOption(tracerObj.reqSpanCtx)) 93 | 94 | /* 95 | tracerObj.nodeSpans[node] = tracerObj.Tracer.StartSpan( 96 | node, opentracing.ChildOf(reqSpan.Context())) 97 | */ 98 | 99 | tracerObj.nodeSpans[node].SetTag("async", "true") 100 | tracerObj.nodeSpans[node].SetTag("request", reqID) 101 | tracerObj.nodeSpans[node].SetTag("node", node) 102 | } 103 | 104 | // StopNodeSpan terminates a node span 105 | func (tracerObj *TraceHandler) StopNodeSpan(node string) { 106 | 107 | tracerObj.nodeSpans[node].Finish() 108 | } 109 | 110 | // StartOperationSpan starts an operation span 111 | func (tracerObj *TraceHandler) StartOperationSpan(node string, reqID string, operationID string) { 112 | 113 | if tracerObj.nodeSpans[node] == nil { 114 | return 115 | } 116 | 117 | operationSpans, ok := tracerObj.operationSpans[node] 118 | if !ok { 119 | operationSpans = make(map[string]opentracing.Span) 120 | tracerObj.operationSpans[node] = operationSpans 121 | } 122 | 123 | nodeContext := tracerObj.nodeSpans[node].Context() 124 | operationSpans[operationID] = tracerObj.tracer.StartSpan( 125 | operationID, opentracing.ChildOf(nodeContext)) 126 | 127 | operationSpans[operationID].SetTag("request", reqID) 128 | operationSpans[operationID].SetTag("node", node) 129 | operationSpans[operationID].SetTag("operation", operationID) 130 | } 131 | 132 | // StopOperationSpan stops an operation span 133 | func (tracerObj *TraceHandler) StopOperationSpan(node string, operationID string) { 134 | 135 | if tracerObj.nodeSpans[node] == nil { 136 | return 137 | } 138 | 139 | operationSpans := tracerObj.operationSpans[node] 140 | operationSpans[operationID].Finish() 141 | } 142 | 143 | // FlushTracer flush all pending traces 144 | func (tracerObj *TraceHandler) FlushTracer() { 145 | tracerObj.closer.Close() 146 | } 147 | -------------------------------------------------------------------------------- /template/faas-flow/function/handler.go: -------------------------------------------------------------------------------- 1 | package function 2 | 3 | import ( 4 | "fmt" 5 | faasflow "github.com/faasflow/lib/openfaas" 6 | ) 7 | 8 | // Define provide definition of the workflow 9 | func Define(flow *faasflow.Workflow, context *faasflow.Context) (err error) { 10 | flow.SyncNode().Modify(func(data []byte) ([]byte, error) { 11 | return []byte(fmt.Sprintf("you said \"%s\"", string(data))), nil 12 | }) 13 | return 14 | } 15 | 16 | // OverrideStateStore provides the override of the default StateStore 17 | func OverrideStateStore() (faasflow.StateStore, error) { 18 | // NOTE: By default FaaS-Flow use consul as a state-store, 19 | // This can be overridden with other synchronous KV store (e.g. ETCD) 20 | return nil, nil 21 | } 22 | 23 | // OverrideDataStore provides the override of the default DataStore 24 | func OverrideDataStore() (faasflow.DataStore, error) { 25 | // NOTE: By default FaaS-Flow use minio as a data-store, 26 | // This can be overridden with other synchronous KV store 27 | return nil, nil 28 | } 29 | -------------------------------------------------------------------------------- /template/faas-flow/go.mod: -------------------------------------------------------------------------------- 1 | module handler 2 | 3 | go 1.14 4 | 5 | require ( 6 | github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd // indirect 7 | github.com/faasflow/faas-flow-consul-statestore v1.0.0 8 | github.com/faasflow/faas-flow-minio-datastore v1.0.0 9 | github.com/faasflow/lib v1.0.0 10 | github.com/faasflow/runtime v0.2.2 11 | github.com/faasflow/sdk v1.0.0 12 | github.com/opentracing/opentracing-go v1.2.0 13 | github.com/pkg/errors v0.9.1 // indirect 14 | github.com/uber/jaeger-client-go v2.24.0+incompatible 15 | github.com/uber/jaeger-lib v2.2.0+incompatible // indirect 16 | go.uber.org/atomic v1.6.0 // indirect 17 | ) 18 | -------------------------------------------------------------------------------- /template/faas-flow/go.sum: -------------------------------------------------------------------------------- 1 | github.com/alexellis/hmac v0.0.0-20180624211220-5c52ab81c0de h1:jiPEvtW8VT0KwJxRyjW2VAAvlssjj9SfecsQ3Vgv5tk= 2 | github.com/alexellis/hmac v0.0.0-20180624211220-5c52ab81c0de/go.mod h1:uAbpy8G7sjNB4qYdY6ymf5OIQ+TLDPApBYiR0Vc3lhk= 3 | github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= 4 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da h1:8GUt8eRujhVEGZFFEjBj46YV4rDjvGrNxb0KMWYkL2I= 5 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= 6 | github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= 7 | github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= 8 | github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd h1:qMd81Ts1T2OTKmB4acZcyKaMtRnY5Y44NuXGX2GFJ1w= 9 | github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI= 10 | github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= 11 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 12 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 13 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 14 | github.com/faasflow/faas-flow-consul-statestore v1.0.0 h1:VAQK6b62xwGmCpJPKumAivEblq4ENRDZFOyPrzifIF8= 15 | github.com/faasflow/faas-flow-consul-statestore v1.0.0/go.mod h1:a/WTHCMVyFKIc9cjolP0xTPt6RLY861ce9oEm6lMe1w= 16 | github.com/faasflow/faas-flow-minio-datastore v1.0.0 h1:07Wavpwt8ObcNoSPoMijhReu0Oa0+pubJaC4tyg3Nbc= 17 | github.com/faasflow/faas-flow-minio-datastore v1.0.0/go.mod h1:zNa1S2s606xsUm03rz6A16/LCZ8kfQszuTpHQAt3TkE= 18 | github.com/faasflow/lib v1.0.0 h1:zvmxeCayhB6N0nBB69R7lM5Xm6TOiUlVJYM0ZL34OxY= 19 | github.com/faasflow/lib v1.0.0/go.mod h1:548zpug3iJ+J61P0MHaHTWGaJOU5S91yKid7M0o7GZU= 20 | github.com/faasflow/runtime v0.2.2 h1:XaKJU9X9DuLuVZhc5Von7R98aw1GgtYjfQNvWQWHma0= 21 | github.com/faasflow/runtime v0.2.2/go.mod h1:fd+6ZuXgYquHpKeaWSwbTWUrJuirfqIrt/Lrm3Rr/kY= 22 | github.com/faasflow/sdk v0.0.0-20200704131052-ae6fbb9a1706/go.mod h1:cpcCvb40uzDNzTT0qxiA6QGuOu8a71LMV2w/ikAW5LU= 23 | github.com/faasflow/sdk v0.0.0-20200705012738-72f2bcdb62d1 h1:KhnO2F+FhUhNoUAOCM9ShoPynlP0VYrB7EY6UQXQ8GE= 24 | github.com/faasflow/sdk v0.0.0-20200705012738-72f2bcdb62d1/go.mod h1:cpcCvb40uzDNzTT0qxiA6QGuOu8a71LMV2w/ikAW5LU= 25 | github.com/faasflow/sdk v1.0.0 h1:ykb4xqkh/mO56Fut9i90jN2/mIlvBqx8nPqzbjON0Pc= 26 | github.com/faasflow/sdk v1.0.0/go.mod h1:cpcCvb40uzDNzTT0qxiA6QGuOu8a71LMV2w/ikAW5LU= 27 | github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= 28 | github.com/fatih/color v1.9.0 h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s= 29 | github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU= 30 | github.com/go-ini/ini v1.57.0 h1:Qwzj3wZQW+Plax5Ntj+GYe07DfGj1OH+aL1nMTMaNow= 31 | github.com/go-ini/ini v1.57.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8= 32 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c h1:964Od4U6p2jUkFxvCydnIczKteheJEzHRToSGK3Bnlw= 33 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= 34 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= 35 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= 36 | github.com/hashicorp/consul/api v1.5.0 h1:Yo2bneoGy68A7aNwmuETFnPhjyBEm7n3vzRacEVMjvI= 37 | github.com/hashicorp/consul/api v1.5.0/go.mod h1:LqwrLNW876eYSuUOo4ZLHBcdKc038txr/IMfbLPATa4= 38 | github.com/hashicorp/consul/sdk v0.5.0 h1:WC4594Wp/LkEeML/OdQKEC1yqBmEYkRp6i7X5u0zDAs= 39 | github.com/hashicorp/consul/sdk v0.5.0/go.mod h1:fY08Y9z5SvJqevyZNy6WWPXiG3KwBPAvlcdx16zZ0fM= 40 | github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= 41 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= 42 | github.com/hashicorp/go-cleanhttp v0.5.1 h1:dH3aiDG9Jvb5r5+bYHsikaOUIpcM0xvgMXVoDkXMzJM= 43 | github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= 44 | github.com/hashicorp/go-hclog v0.12.0 h1:d4QkX8FRTYaKaCZBoXYY8zJX2BXjWxurN/GA2tkrmZM= 45 | github.com/hashicorp/go-hclog v0.12.0/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ= 46 | github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0= 47 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= 48 | github.com/hashicorp/go-msgpack v0.5.3 h1:zKjpN5BK/P5lMYrLmBHdBULWbJ0XpYR+7NGzqkZzoD4= 49 | github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= 50 | github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o= 51 | github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= 52 | github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc= 53 | github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8= 54 | github.com/hashicorp/go-sockaddr v1.0.0 h1:GeH6tui99pF4NJgfnhp+L6+FfobzVW3Ah46sLo0ICXs= 55 | github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= 56 | github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= 57 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 58 | github.com/hashicorp/go-uuid v1.0.1 h1:fv1ep09latC32wFoVwnqcnKJGnMSdBanPczbHAYm1BE= 59 | github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 60 | github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo= 61 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 62 | github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= 63 | github.com/hashicorp/mdns v1.0.1/go.mod h1:4gW7WsVCke5TE7EPeYliwHlRUyBtfCwuFwuMg2DmyNY= 64 | github.com/hashicorp/memberlist v0.2.0 h1:WeeNspppWi5s1OFefTviPQueC/Bq8dONfvNjPhiEQKE= 65 | github.com/hashicorp/memberlist v0.2.0/go.mod h1:MS2lj3INKhZjWNqd3N0m3J+Jxf3DAOnAH9VT3Sh9MUE= 66 | github.com/hashicorp/serf v0.9.0 h1:+Zd/16AJ9lxk9RzfTDyv/TLhZ8UerqYS0/+JGCIDaa0= 67 | github.com/hashicorp/serf v0.9.0/go.mod h1:YL0HO+FifKOW2u1ke99DGVu1zhcpZzNwrLIqBC7vbYU= 68 | github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= 69 | github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= 70 | github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U= 71 | github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= 72 | github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs= 73 | github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= 74 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 75 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 76 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 77 | github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= 78 | github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA= 79 | github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= 80 | github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= 81 | github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= 82 | github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84= 83 | github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE= 84 | github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY= 85 | github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= 86 | github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= 87 | github.com/miekg/dns v1.1.26 h1:gPxPSwALAeHJSjarOs00QjVdV9QoBvc1D2ujQUr5BzU= 88 | github.com/miekg/dns v1.1.26/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso= 89 | github.com/minio/minio-go v6.0.14+incompatible h1:fnV+GD28LeqdN6vT2XdGKW8Qe/IfjJDswNVuni6km9o= 90 | github.com/minio/minio-go v6.0.14+incompatible/go.mod h1:7guKYtitv8dktvNUGrhzmNlA5wrAABTQXCoesZdFQO8= 91 | github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= 92 | github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= 93 | github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= 94 | github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0= 95 | github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= 96 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 97 | github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE= 98 | github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 99 | github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs= 100 | github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc= 101 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c h1:Lgl0gzECD8GnQ5QCWA8o6BtfL6mDH5rQgM4/fX3avOs= 102 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= 103 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 104 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= 105 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 106 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 107 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 108 | github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= 109 | github.com/rs/xid v1.2.1 h1:mhH9Nq+C1fY2l1XIpgxIiUOfNpRBYH1kKcr+qfKgjRc= 110 | github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ= 111 | github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= 112 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I= 113 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= 114 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= 115 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= 116 | github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s= 117 | github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= 118 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 119 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 120 | github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= 121 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= 122 | github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= 123 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 124 | github.com/uber/jaeger-client-go v2.24.0+incompatible h1:CGchgJcHsDd2jWnaL4XngByMrXoGHh3n8oCqAKx0uMo= 125 | github.com/uber/jaeger-client-go v2.24.0+incompatible/go.mod h1:WVhlPFC8FDjOFMMWRy2pZqQJSXxYSwNYOkTr/Z6d3Kk= 126 | github.com/uber/jaeger-lib v2.2.0+incompatible h1:MxZXOiR2JuoANZ3J6DE/U0kSFv/eJ/GfSYVCjK7dyaw= 127 | github.com/uber/jaeger-lib v2.2.0+incompatible/go.mod h1:ComeNDZlWwrWnDv8aPp0Ba6+uUTzImX/AauajbLI56U= 128 | go.uber.org/atomic v1.6.0 h1:Ezj3JGmsOnG1MoRWQkPBsKLe9DwWD9QeXzTRzzldNVk= 129 | go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= 130 | golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 131 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 132 | golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392 h1:ACG4HJsFiNMf47Y4PeRoebLNy/2lXT9EtprMuTFWt1M= 133 | golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY= 134 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI= 135 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 136 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs= 137 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= 138 | golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 139 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 140 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 141 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 142 | golang.org/x/net v0.0.0-20190923162816-aa69164e4478 h1:l5EDrHhldLYb3ZRHDUhXF7Om7MvYXnkV9/iQNo1lX6g= 143 | golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 144 | golang.org/x/net v0.0.0-20200625001655-4c5254603344 h1:vGXIOMxbNfDTk/aXCmfdLgkrSV+Z2tcbze+pEc3v5W4= 145 | golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= 146 | golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 147 | golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU= 148 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 149 | golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 150 | golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 151 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 152 | golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 153 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 154 | golang.org/x/sys v0.0.0-20190922100055-0a153f010e69/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 155 | golang.org/x/sys v0.0.0-20190924154521-2837fb4f24fe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 156 | golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 157 | golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 158 | golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 159 | golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9 h1:1/DFK4b7JH8DmkqhUk48onnSfrPzImPoVxuomtbT2nk= 160 | golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 161 | golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd h1:xhmwyvizuTgC2qz7ZlMluP20uW+C3Rm0FD/WLDX8884= 162 | golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 163 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 164 | golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs= 165 | golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= 166 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 167 | golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 168 | golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= 169 | golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 170 | golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c h1:IGkKhmfzcztjm6gYkykvu/NiS8kaqbCWAEWWAyf8J5U= 171 | golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 172 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 173 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 174 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= 175 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 176 | gopkg.in/ini.v1 v1.57.0 h1:9unxIsFcTt4I55uWluz+UmL95q4kdJ0buvQ1ZIqVQww= 177 | gopkg.in/ini.v1 v1.57.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= 178 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 179 | gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= 180 | gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 181 | -------------------------------------------------------------------------------- /template/faas-flow/log/std_out_logger.go: -------------------------------------------------------------------------------- 1 | package log 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | // implements faasflow.Logger 8 | type StdOutLogger struct{} 9 | 10 | func (l *StdOutLogger) Configure(flowName string, requestId string) {} 11 | 12 | func (l *StdOutLogger) Init() error { 13 | return nil 14 | } 15 | func (l *StdOutLogger) Log(str string) { 16 | fmt.Print(str) 17 | } 18 | -------------------------------------------------------------------------------- /template/faas-flow/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "github.com/faasflow/runtime/controller/http" 5 | "handler/config" 6 | "handler/openfaas" 7 | "log" 8 | ) 9 | 10 | func main() { 11 | runtime := &openfaas.OpenFaasRuntime{} 12 | port := 8082 13 | readTimeout := config.ReadTimeout() 14 | writeTimeout := config.WriteTimeout() 15 | log.Fatal(http.StartServer(runtime, port, readTimeout, writeTimeout)) 16 | } 17 | -------------------------------------------------------------------------------- /template/faas-flow/openfaas/init_data_store.go: -------------------------------------------------------------------------------- 1 | package openfaas 2 | 3 | import ( 4 | "log" 5 | 6 | "handler/function" 7 | 8 | minioDataStore "github.com/faasflow/faas-flow-minio-datastore" 9 | "github.com/faasflow/sdk" 10 | ) 11 | 12 | func initDataStore() (dataStore sdk.DataStore, err error) { 13 | dataStore, err = function.OverrideDataStore() 14 | if err != nil { 15 | return nil, err 16 | } 17 | if dataStore == nil { 18 | 19 | /* 20 | minioUrl := os.Getenv("s3_url") 21 | if len(minioUrl) == 0 { 22 | minioUrl = "minio.faasflow:9000" 23 | } 24 | 25 | minioRegion := os.Getenv("s3_region") 26 | if len(minioRegion) == 0 { 27 | minioUrl = "us-east-1" 28 | } 29 | 30 | secretKeyName := os.Getenv("s3_secret_key_name") 31 | if len(secretKeyName) == 0 { 32 | secretKeyName = "s3-secret-key" 33 | } 34 | 35 | accessKeyName := os.Getenv("s3_access_key_name") 36 | if len(accessKeyName) == 0 { 37 | accessKeyName = "s3-access-key" 38 | } 39 | 40 | tlsEnabled := false 41 | if connection := os.Getenv("s3_tls"); connection == "true" || connection == "1" { 42 | tlsEnabled = true 43 | } 44 | 45 | dataStore, err = minioDataStore.Init(minioUrl, minioRegion, secretKeyName, accessKeyName, tlsEnabled) 46 | */ 47 | dataStore, err = minioDataStore.InitFromEnv() 48 | 49 | log.Print("Using default data store (minio)") 50 | } 51 | return dataStore, err 52 | } 53 | -------------------------------------------------------------------------------- /template/faas-flow/openfaas/init_state_store.go: -------------------------------------------------------------------------------- 1 | package openfaas 2 | 3 | import ( 4 | "log" 5 | 6 | "handler/config" 7 | "handler/function" 8 | 9 | consulStateStore "github.com/faasflow/faas-flow-consul-statestore" 10 | "github.com/faasflow/sdk" 11 | ) 12 | 13 | func initStateStore() (stateStore sdk.StateStore, err error) { 14 | stateStore, err = function.OverrideStateStore() 15 | if err != nil { 16 | return nil, err 17 | } 18 | 19 | if stateStore == nil { 20 | log.Print("Using default state store (consul)") 21 | 22 | consulURL := config.ConsulURL() 23 | consulDC := config.ConsulDC() 24 | 25 | stateStore, err = consulStateStore.GetConsulStateStore(consulURL, consulDC) 26 | } 27 | 28 | return stateStore, err 29 | } 30 | -------------------------------------------------------------------------------- /template/faas-flow/openfaas/openfaas_executor.go: -------------------------------------------------------------------------------- 1 | package openfaas 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "github.com/faasflow/runtime" 7 | "github.com/faasflow/runtime/controller/util" 8 | "io/ioutil" 9 | "log" 10 | "net/http" 11 | "net/url" 12 | "os" 13 | "path" 14 | "strings" 15 | 16 | faasflow "github.com/faasflow/lib/openfaas" 17 | sdk "github.com/faasflow/sdk" 18 | "github.com/faasflow/sdk/executor" 19 | "handler/config" 20 | "handler/eventhandler" 21 | "handler/function" 22 | hlog "handler/log" 23 | ) 24 | 25 | // A signature of SHA265 equivalent of github.com/s8sg/faas-flow 26 | const defaultHmacKey = "71F1D3011F8E6160813B4997BA29856744375A7F26D427D491E1CCABD4627E7C" 27 | 28 | // implements faasflow.Executor + RequestHandler 29 | type OpenFaasExecutor struct { 30 | gateway string 31 | asyncURL string // the async URL of the flow 32 | flowName string // the name of the function 33 | reqID string // the request id 34 | CallbackURL string // the callback url 35 | partialState []byte 36 | rawRequest *executor.RawRequest 37 | StateStore sdk.StateStore 38 | DataStore sdk.DataStore 39 | EventHandler sdk.EventHandler 40 | logger hlog.StdOutLogger 41 | } 42 | 43 | func (of *OpenFaasExecutor) HandleNextNode(partial *executor.PartialState) error { 44 | 45 | state, err := partial.Encode() 46 | if err != nil { 47 | return fmt.Errorf("failed to encode partial state, error %v", err) 48 | } 49 | 50 | url, _ := url.Parse(of.asyncURL) 51 | url.Path = path.Join(url.Path, "flow", of.reqID, "forward") 52 | 53 | // build url for calling the flow in async 54 | httpReq, _ := http.NewRequest(http.MethodPost, url.String(), bytes.NewReader(state)) 55 | httpReq.Header.Add("Accept", "application/json") 56 | httpReq.Header.Add("Content-Type", "application/json") 57 | httpReq.Header.Add(util.RequestIdHeader, of.reqID) 58 | httpReq.Header.Set(util.CallbackUrlHeader, of.CallbackURL) 59 | 60 | fmt.Println(fmt.Sprint(of.asyncURL, of.reqID)) 61 | fmt.Println(httpReq) 62 | 63 | // extend req span for async call 64 | if of.MonitoringEnabled() { 65 | faasHandler := of.EventHandler.(*eventhandler.FaasEventHandler) 66 | faasHandler.Tracer.ExtendReqSpan(of.reqID, faasHandler.CurrentNodeID, url.String(), httpReq) 67 | } 68 | 69 | client := &http.Client{} 70 | res, resErr := client.Do(httpReq) 71 | if resErr != nil { 72 | return resErr 73 | } 74 | 75 | defer res.Body.Close() 76 | resData, _ := ioutil.ReadAll(res.Body) 77 | 78 | if res.StatusCode != http.StatusOK && res.StatusCode != http.StatusAccepted { 79 | return fmt.Errorf("%d: %s", res.StatusCode, string(resData)) 80 | } 81 | return nil 82 | } 83 | 84 | func (of *OpenFaasExecutor) GetExecutionOption(operation sdk.Operation) map[string]interface{} { 85 | options := make(map[string]interface{}) 86 | options["gateway"] = of.gateway 87 | options["request-id"] = of.reqID 88 | 89 | return options 90 | } 91 | 92 | func (of *OpenFaasExecutor) HandleExecutionCompletion(data []byte) error { 93 | if of.CallbackURL == "" { 94 | return nil 95 | } 96 | 97 | log.Printf("calling callback url (%s) with result", of.CallbackURL) 98 | httpreq, _ := http.NewRequest(http.MethodPost, of.CallbackURL, bytes.NewReader(data)) 99 | httpreq.Header.Add("X-Faas-Flow-ReqiD", of.reqID) 100 | client := &http.Client{} 101 | 102 | res, resErr := client.Do(httpreq) 103 | if resErr != nil { 104 | return resErr 105 | } 106 | defer res.Body.Close() 107 | resData, _ := ioutil.ReadAll(res.Body) 108 | 109 | if res.StatusCode != http.StatusOK && res.StatusCode != http.StatusAccepted { 110 | return fmt.Errorf("failed to call callback %d: %s", res.StatusCode, string(resData)) 111 | } 112 | 113 | return nil 114 | } 115 | 116 | func (of *OpenFaasExecutor) Configure(requestID string) { 117 | of.reqID = requestID 118 | } 119 | 120 | func (of *OpenFaasExecutor) GetFlowName() string { 121 | return of.flowName 122 | } 123 | 124 | func (of *OpenFaasExecutor) GetFlowDefinition(pipeline *sdk.Pipeline, context *sdk.Context) error { 125 | workflow := faasflow.GetWorkflow(pipeline) 126 | faasflowContext := (*faasflow.Context)(context) 127 | err := function.Define(workflow, faasflowContext) 128 | return err 129 | } 130 | 131 | func (of *OpenFaasExecutor) ReqValidationEnabled() bool { 132 | status := true 133 | hmacStatus := os.Getenv("validate_request") 134 | if strings.ToUpper(hmacStatus) == "FALSE" { 135 | status = false 136 | } 137 | return status 138 | } 139 | 140 | func (of *OpenFaasExecutor) GetValidationKey() (string, error) { 141 | key, keyErr := ReadSecret("faasflow-hmac-secret") 142 | if keyErr != nil { 143 | key = defaultHmacKey 144 | } 145 | return key, nil 146 | } 147 | 148 | func (of *OpenFaasExecutor) ReqAuthEnabled() bool { 149 | status := false 150 | verifyStatus := os.Getenv("authenticate_request") 151 | if strings.ToUpper(verifyStatus) == "TRUE" { 152 | status = true 153 | } 154 | return status 155 | } 156 | 157 | func (of *OpenFaasExecutor) GetReqAuthKey() (string, error) { 158 | key, keyErr := ReadSecret("faasflow-hmac-secret") 159 | return key, keyErr 160 | } 161 | 162 | func (of *OpenFaasExecutor) MonitoringEnabled() bool { 163 | tracing := os.Getenv("enable_tracing") 164 | if strings.ToUpper(tracing) == "TRUE" { 165 | return true 166 | } 167 | return false 168 | } 169 | 170 | func (of *OpenFaasExecutor) GetEventHandler() (sdk.EventHandler, error) { 171 | return of.EventHandler, nil 172 | } 173 | 174 | func (of *OpenFaasExecutor) LoggingEnabled() bool { 175 | return true 176 | } 177 | 178 | func (of *OpenFaasExecutor) GetLogger() (sdk.Logger, error) { 179 | return &of.logger, nil 180 | } 181 | 182 | func (of *OpenFaasExecutor) GetStateStore() (sdk.StateStore, error) { 183 | return of.StateStore, nil 184 | } 185 | 186 | func (of *OpenFaasExecutor) GetDataStore() (sdk.DataStore, error) { 187 | return of.DataStore, nil 188 | } 189 | 190 | func (of *OpenFaasExecutor) Init(request *runtime.Request) error { 191 | of.gateway = config.GatewayURL() 192 | of.flowName = request.FlowName 193 | of.asyncURL = buildURL("http://"+of.gateway, "async-function", of.flowName) 194 | 195 | callbackURL := request.GetHeader("X-Faas-Flow-Callback-Url") 196 | of.CallbackURL = callbackURL 197 | 198 | faasHandler := of.EventHandler.(*eventhandler.FaasEventHandler) 199 | faasHandler.Header = request.Header 200 | 201 | return nil 202 | } 203 | 204 | // buildURL builds execution url for the flow 205 | func buildURL(gateway, rPath, function string) string { 206 | u, _ := url.Parse(gateway) 207 | u.Path = path.Join(u.Path, rPath, function) 208 | return u.String() 209 | } 210 | -------------------------------------------------------------------------------- /template/faas-flow/openfaas/openfaas_runtime.go: -------------------------------------------------------------------------------- 1 | package openfaas 2 | 3 | import ( 4 | "fmt" 5 | "github.com/faasflow/runtime" 6 | sdk "github.com/faasflow/sdk" 7 | "github.com/faasflow/sdk/executor" 8 | "handler/eventhandler" 9 | ) 10 | 11 | type OpenFaasRuntime struct { 12 | stateStore sdk.StateStore 13 | dataStore sdk.DataStore 14 | eventHandler sdk.EventHandler 15 | } 16 | 17 | func (ofRuntime *OpenFaasRuntime) Init() error { 18 | var err error 19 | ofRuntime.stateStore, err = initStateStore() 20 | if err != nil { 21 | return fmt.Errorf("Failed to initialize the StateStore, %v", err) 22 | } 23 | 24 | ofRuntime.dataStore, err = initDataStore() 25 | if err != nil { 26 | return fmt.Errorf("Failed to initialize the StateStore, %v", err) 27 | } 28 | 29 | ofRuntime.eventHandler = &eventhandler.FaasEventHandler{} 30 | 31 | return nil 32 | } 33 | 34 | func (ofRuntime *OpenFaasRuntime) CreateExecutor(request *runtime.Request) (executor.Executor, error) { 35 | ex := &OpenFaasExecutor{StateStore: ofRuntime.stateStore, DataStore: ofRuntime.dataStore, EventHandler: ofRuntime.eventHandler} 36 | error := ex.Init(request) 37 | return ex, error 38 | } 39 | -------------------------------------------------------------------------------- /template/faas-flow/openfaas/read_secret.go: -------------------------------------------------------------------------------- 1 | package openfaas 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "os" 7 | "path" 8 | "strings" 9 | ) 10 | 11 | // ReadSecret reads a secret from /var/openfaas/secrets or from 12 | // env-var 'secret_mount_path' if set. 13 | func ReadSecret(key string) (string, error) { 14 | basePath := "/var/openfaas/secrets/" 15 | if len(os.Getenv("secret_mount_path")) > 0 { 16 | basePath = os.Getenv("secret_mount_path") 17 | } 18 | 19 | readPath := path.Join(basePath, key) 20 | secretBytes, readErr := ioutil.ReadFile(readPath) 21 | if readErr != nil { 22 | return "", fmt.Errorf("unable to read secret: %s, error: %s", readPath, readErr) 23 | } 24 | val := strings.TrimSpace(string(secretBytes)) 25 | return val, nil 26 | } 27 | -------------------------------------------------------------------------------- /template/faas-flow/template.yml: -------------------------------------------------------------------------------- 1 | language: faas-flow 2 | fprocess: ./handler 3 | welcome_message: | 4 | You have created a FaasFlow function. 5 | 6 | Add the following configuration for function into Stack file. 7 | 8 | labels: 9 | faas-flow: 1 10 | annotations: 11 | faas-flow-desc: "my awesome flow" 12 | environment_file: 13 | - conf.yml 14 | secrets: 15 | - s3-secret-key 16 | - s3-access-key 17 | 18 | Create a flow configuration file conf.yml with the following configuration. 19 | 20 | environment: 21 | gateway: "gateway.openfaas:8080" 22 | enable_tracing: true 23 | trace_server: "jaeger-agent.faasflow:5775" 24 | consul_url: "consul.faasflow:8500" 25 | s3_url: "minio.faasflow:9000" 26 | 27 | (You may need to change the configuration based on your faas-flow deployment). 28 | --------------------------------------------------------------------------------