├── .gitignore ├── README.adoc ├── call-function.sh ├── deploy.sh ├── pom.xml └── src ├── main └── java │ └── mcp │ ├── DemoConsumer.java │ ├── DemoFunction.java │ └── DemoSupplier.java └── test └── java └── mcp └── DemoApplicationTests.java /.gitignore: -------------------------------------------------------------------------------- 1 | **/.idea/* 2 | **/target/* 3 | **/*.iml 4 | -------------------------------------------------------------------------------- /README.adoc: -------------------------------------------------------------------------------- 1 | = Introduction to the Project Riff Functions-as-a-Service Platform 2 | Mario Gray 3 | :Author Initials: MVG 4 | :toc: 5 | :icons: 6 | :numbered: 7 | :website: https://github.com/projectRiff/Riff= 8 | 9 | === Motivation 10 | Functions as a Service are a new paradigm for application composition and deployment. 11 | Functions are a smaller deployable unit than microservices and especially 12 | traditional monoliths. https://projectRiff.io/[Project Riff] is a new Pivotal project 13 | that runs functions in a cloud environment like Pivotal Container Services (Kubernetes). 14 | Lets get started by setting up a new Riff environment and then creating some functions. 15 | 16 | [[X7]] 17 | == Preparing the environment 18 | `minikube` is the tool that we'll use to run Kubernetes locally. 19 | In this demo, we assume a configuration containing Minikube `0.25` 20 | and Helm/tiller `2.7.2`. Make sure to start Minikube: 21 | 22 | [source,script,indent=0] 23 | ---- 24 | minikube start \ 25 | --memory=4096 \ 26 | --cpus=2 27 | ---- 28 | 29 | ==== Navigating uncharted waters with Helm 30 | All of the steps to manually prepare the environment locally are 31 | outlined in the https://github.com/projectRiff/Riff[Riff Github Repo]. 32 | Helm is a package manager for Kubernetes. Riff provides helm packages 33 | ( Helm "charts" ) to ease installation and migration. For further information on installing Helm, refer to the project's 34 | https://docs.helm.sh/using_helm/#from-script[Installation guide]. 35 | 36 | [[X8]] 37 | === Installing the Riff Environment with Helm 38 | 39 | Homebrew can be used to install helm in a jiffy: 40 | [source,script,indent=0] 41 | ---- 42 | brew install kubernetes-helm 43 | ---- 44 | 45 | Linux and Windows users may use alternative methods, or your simply want 46 | to install things manually on OSX, follow https://github.com/Kubernetes/helm[the instructions for Helm]. 47 | 48 | Then, add the Helm chart for Riff: 49 | 50 | [source,script,indent=0] 51 | ---- 52 | helm repo add riffrepo https://riff-charts.storage.googleapis.com 53 | helm repo update 54 | ---- 55 | 56 | Install Riff with the release name `demo`: 57 | 58 | [source,script,indent=0] 59 | ---- 60 | helm install riffrepo/riff --name demo 61 | ---- 62 | 63 | Once this command is complete, you will have a functioning Riff 64 | environment. You can enumerate the Riff componenets using this command: 65 | 66 | [source,script,indent=0] 67 | ---- 68 | kubectl get pods --show-labels -l app=riff 69 | ---- 70 | 71 | You'll notice several pods in service. The entry point to your functions 72 | is `demo-riff-http-gateway`. Enter the following command to enumerate 73 | Kubernetes services and ensure it is accepting connections from `localhost`; 74 | 75 | [source,script,indent=0] 76 | ---- 77 | kubectl get svc -l app=riff 78 | ---- 79 | 80 | Check that `demo-riff-http-gateway` has a `TYPE` of `NodePort`. 81 | 82 | ==== Choosing the right service type for your environment. 83 | 84 | If your riff-http-gateway says that EXTERNAL-IP is Pending and fails 85 | to resolve after several minutes, then you will need to change the 86 | service type. 87 | 88 | [source,script,indent=0] 89 | ---- 90 | kubectl edit service demo-riff-http-gateway 91 | ---- 92 | Update the `type` field from `LoadBalancer` to `NodePort`, and save. 93 | This will expose your riff-http-gateway to the local network. 94 | 95 | Additional networking configuration may be necessary as you might 96 | have subnetting/firewalls in place, especailly in a controlled 97 | environtment. 98 | 99 | === Installing the Riff Client 100 | 101 | In order to use Riff, you'll need the `riff` command line client interface (CLI). 102 | Clone https://github.com/projectRiff/Riff[the `git` repository] and add it to your `PATH`. 103 | Now you can use `riff` in the shell. 104 | 105 | == Writing the Function 106 | 107 | Riff maintains a registry of functions. These functions can optionally accept payloads as requests 108 | and optionally produce replies. Riff will forward requests to your functions and deliver replies 109 | from your functions over message channels. Messages originate from a number of different sources: 110 | HTTP, Kafka, RabbitMQ, etc. As of this writing, while message payloads are routed through Kafka 111 | topics, communication to the front-end of Riff take HTTP requests only. 112 | 113 | At the moment, Riff supports the following languages and runtimes: Python, Node.js, `/bin/bash`, and Java. 114 | We will focus on Java in this example. Riff supports three standard Java 8 functional interfaces: 115 | 116 | * `java.util.function.Supplier`: returns a value but does not accept an argument 117 | * `java.util.function.Function`: returns a value and does accept an argument 118 | * `java.util.function.Consumer`: does not return a value but does accept an argument. 119 | 120 | Let's look at a simple `Function`, which both accepts an input argument and 121 | produces a reply. This is the only one that we will deploy in this article. 122 | 123 | .`DemoFunction.java` 124 | [source,java,indent=0] 125 | ---- 126 | package mcp; 127 | 128 | import java.util.function.Function; 129 | 130 | public class DemoFunction implements Function { 131 | public String apply(String s) { 132 | return new StringBuilder(s).reverse().toString(); 133 | } 134 | } 135 | ---- 136 | 137 | == Executing the function 138 | 139 | We will need to create and deploy our function in a container. We can create our own `Dockerfile`, 140 | service description (`.yaml`) configuration, and container images manually if we wanted to. 141 | There is no need to, though, as Riff 0.0.3 will do all this for us! 142 | 143 | .Deploy a function 144 | [source,script] 145 | ---- 146 | riff create --name demofn --input reverse-in \ 147 | --protocol pipes --artifact target/demofn-0.0.1.jar \ 148 | --handler mcp.DemoFunction 149 | ---- 150 | 151 | This command creates a function. You specify the input and output topics, a fully 152 | qualified classname and method, and the artifact (the `.jar`) that contains the class. 153 | Your topics are logical names for pipes that conduct requests and replies between 154 | functions. 155 | 156 | The following command will locate your `riff-http-gateway`'s IP and port, 157 | then create an HTTP request that will be sent to the `reverse-in` topic. 158 | We tell Riff to wait for a response payload from the input topic with 159 | the `--reply` parameter. 160 | 161 | .Invoke the function using the `riff` CLI. 162 | [source,script] 163 | ---- 164 | riff publish --input reverse-in --data GNIRPS --reply 165 | ---- 166 | 167 | Riff will publish 'GNIRPS' data to the 'reverse-in' topic. By 168 | specifying `--reply` in the riff command, we are requesting that riff 169 | await for retured data to print to the console. 170 | 171 | == Scaling the function 172 | 173 | Riff uses the function-controller to replicate (or scale) your 174 | functions as request rates increase, and decrease over time. 175 | The default behavior implements replication factor equal to the number 176 | of partitions of the input topic. You can edit a topic resource 177 | by using `kubectl edit topic MY_TOPIC` command and 178 | change `spec.partitions` to your needs. Alternately, the default 179 | behavior can be modified by editing the function's configuration, 180 | setting `spec.maxReplicas` to a desired value. 181 | 182 | === Next Steps 183 | 184 | We've only begun to scratch the surface in this look at Project Riff. It would be worth in another 185 | installment looking at using some of the other types of functions - `Consumer`s and `Supplier`s - 186 | and look at when and how they're used. In another installment we could 187 | look at how functions connect with each other. In another installment, we could look at how 188 | function-capable frameworks like https://cloud.spring.io/spring-cloud-function/[Spring Cloud Function] work 189 | in the Riff environment. 190 | -------------------------------------------------------------------------------- /call-function.sh: -------------------------------------------------------------------------------- 1 | riff publish --input reverse-in -d OIRAM --reply 2 | -------------------------------------------------------------------------------- /deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | MY_VERSION=$1; shift 4 | DEMO_VERSION=0.0.1 5 | 6 | [[ -z ${MY_VERSION//} ]] \ 7 | || DEMO_VERSION=${MY_VERSION} 8 | 9 | export DEMO_VERSION 10 | 11 | rm -f *.yaml 12 | rm -f Dockerfile 13 | 14 | function exitfn { 15 | MSG=$1; shift 16 | echo >&2 ${MSG} 17 | exit 1 18 | } 19 | 20 | function exists_or_bail { 21 | COMMAND=$1; shift 22 | if type $COMMAND > /dev/null 2>&1 23 | then : OK 24 | else 25 | exitfn "${COMMAND} is required." 26 | fi 27 | } 28 | 29 | function exists_or { 30 | COMMAND=$1; shift 31 | if type $COMMAND > /dev/null 2>&1 32 | then : OK 33 | else 34 | echo >&2 "${COMMAND} is required." 35 | return 1 36 | fi 37 | return 0 38 | } 39 | 40 | [[ -z ${RIFF_HOME} || ${RIFF_HOME} == .* ]] && exitfn "Set env \$RIFF_HOME to (absolute) root of local RIFF clone." 41 | 42 | exists_or_bail minikube 43 | exists_or_bail docker 44 | 45 | PARENT=${USERNAME} 46 | DELIM="" 47 | 48 | function docker_detect { 49 | [[ -z ${DOCKER_HOST//} ]] && eval $(minikube docker-env) 50 | } 51 | 52 | # Single or Double namespace environment is assumed 53 | [[ -z ${PARENT//} ]] && DELIM="/" 54 | 55 | function docker_images { 56 | set -v 57 | docker_detect 58 | 59 | echo docker images --filter=reference="${PARENT}${DELIM}'*'" 60 | } 61 | 62 | # input image container name 63 | function docker_image_by_name { 64 | 65 | IMAGE_NAME=$1; shift 66 | 67 | docker_detect 68 | 69 | docker images --filter=reference=${PARENT}${DELIM}${IMAGE_NAME} --format "{{.ID}}" 70 | } 71 | 72 | # input image container name 73 | function docker_rmi_by_name { 74 | 75 | IMAGE_NAME=$1; shift 76 | 77 | docker_detect 78 | 79 | docker rmi `docker_image_by_name ${IMAGE_NAME}` 80 | } 81 | 82 | set -a -e -v 83 | 84 | [[ ${0} == *_nb.sh ]] || mvn clean package 85 | 86 | PATH=$PATH:${RIFF_HOME} 87 | 88 | exists_or_bail riff 89 | eval $(minikube docker-env) 90 | 91 | DEMO_JAR=target/demofn-${DEMO_VERSION}.jar 92 | 93 | riff create --name demofn --input reverse-in\ 94 | --protocol pipes --artifact ${DEMO_JAR}\ 95 | --handler mcp.DemoFunction -------------------------------------------------------------------------------- /pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 4 | 4.0.0 5 | mcp 6 | demofn 7 | 0.0.1 8 | jar 9 | 10 | UTF-8 11 | UTF-8 12 | 1.8 13 | 14 | 15 | 16 | org.projectlombok 17 | lombok 18 | 1.16.18 19 | 20 | 21 | junit 22 | junit 23 | 4.12 24 | test 25 | 26 | 27 | org.assertj 28 | assertj-core 29 | 3.8.0 30 | test 31 | 32 | 33 | 34 | 35 | 36 | org.apache.maven.plugins 37 | maven-compiler-plugin 38 | 3.7.0 39 | 40 | 1.8 41 | 1.8 42 | 43 | 44 | 45 | 46 | -------------------------------------------------------------------------------- /src/main/java/mcp/DemoConsumer.java: -------------------------------------------------------------------------------- 1 | package mcp; 2 | 3 | import lombok.extern.java.Log; 4 | 5 | import java.util.function.Consumer; 6 | 7 | @Log 8 | public class DemoConsumer implements Consumer { 9 | public void accept(String s) { 10 | log.info(s); 11 | } 12 | } -------------------------------------------------------------------------------- /src/main/java/mcp/DemoFunction.java: -------------------------------------------------------------------------------- 1 | package mcp; 2 | 3 | import java.util.function.Function; 4 | 5 | public class DemoFunction implements Function { 6 | public String apply(String s) { 7 | return new StringBuilder(s).reverse().toString(); 8 | } 9 | } -------------------------------------------------------------------------------- /src/main/java/mcp/DemoSupplier.java: -------------------------------------------------------------------------------- 1 | package mcp; 2 | 3 | import java.util.Arrays; 4 | import java.util.List; 5 | import java.util.Random; 6 | import java.util.function.Supplier; 7 | 8 | public class DemoSupplier implements Supplier { 9 | 10 | private List kingsBook = Arrays.asList( 11 | "YKCOWREBBAJ", 12 | "sevot yhtils eht dna,gillirb sawT’", 13 | "ebaw eht ni elbmig dna eryg diD", 14 | ",sevogorob eht erew ysmim llA", 15 | ".ebargtuo shtar emom eht dnA"); 16 | 17 | private final Random random = new Random(); 18 | 19 | public String get() { 20 | int r = random.nextInt() * kingsBook.size(); 21 | return kingsBook.get(r); 22 | } 23 | } -------------------------------------------------------------------------------- /src/test/java/mcp/DemoApplicationTests.java: -------------------------------------------------------------------------------- 1 | package mcp; 2 | 3 | import mcp.DemoConsumer; 4 | import mcp.DemoFunction; 5 | import mcp.DemoSupplier; 6 | import org.assertj.core.api.Assertions; 7 | import org.junit.Rule; 8 | import org.junit.Test; 9 | import org.junit.rules.ExpectedException; 10 | 11 | public class DemoApplicationTests { 12 | 13 | @Rule 14 | public ExpectedException thrown = ExpectedException.none(); 15 | 16 | @Test 17 | public void testShouldExecFunction() { 18 | DemoFunction demoFn = new DemoFunction(); 19 | String test = demoFn.apply("foo"); 20 | 21 | Assertions.assertThat(test).isNotEmpty(); 22 | } 23 | 24 | @Test 25 | public void testShouldConsume() { 26 | DemoConsumer demoFn = new DemoConsumer(); 27 | demoFn.accept("foo"); 28 | } 29 | 30 | @Test 31 | public void testShouldSupplyData() { 32 | DemoSupplier demoFn = new DemoSupplier(); 33 | String test = demoFn.get(); 34 | 35 | Assertions.assertThat(test).isNotNull(); 36 | } 37 | } --------------------------------------------------------------------------------