├── LICENSE ├── README.md └── fortio ├── .gitignore ├── Makefile ├── README.md ├── cert.yaml ├── fortio.yaml ├── ingress.yaml └── stage ├── Makefile ├── cert.yaml ├── fortio.yaml └── ingress.yaml /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Fortio deployment files. 2 | -------------------------------------------------------------------------------- /fortio/.gitignore: -------------------------------------------------------------------------------- 1 | *.json 2 | istio-*.*.* 3 | tmp 4 | -------------------------------------------------------------------------------- /fortio/Makefile: -------------------------------------------------------------------------------- 1 | 2 | deploy-fortio: 3 | istioctl kube-inject --debug=false -f fortio.yaml | kubectl apply -f - 4 | 5 | all: cert-setup ingress-setup cert-issue deploy-fortio 6 | 7 | cert-setup: 8 | # TODO: more granular rbac roles 9 | -kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default 10 | helm init 11 | helm repo update 12 | helm install stable/cert-manager 13 | 14 | ingress-setup: 15 | kubectl apply -f ingress.yaml 16 | 17 | cert-issue: 18 | kubectl apply -f cert.yaml 19 | 20 | # dangerous as it deletes the cert and secret and if using the production 21 | # letsencrypt server, rate limits may mean you can't get a new cert until 22 | # the following week. uncomment if you understand the risks: 23 | #force-reissue: 24 | # kubectl delete secret -n istio-system istio-ingress-certs 25 | 26 | .PHONY: deploy-fortio cert-setup ingress-setup cert-issue all 27 | -------------------------------------------------------------------------------- /fortio/README.md: -------------------------------------------------------------------------------- 1 | # Fortio on Istio 2 | 3 | Experimental istio on istio deployment: 4 | 5 | [https://fortio.istio.io/](https://fortio.istio.io/) is running an istio deployment of the `fortio report` application used to serve performance results. 6 | 7 | Using envoy directly as the internet facing istio ingress. SSL certificates are 8 | automatically provisioned and renewed for multiple domains 9 | ([https://istio.fortio.org/](https://istio.fortio.org/) being the second one for the sake of this demonstration). 10 | 11 | The data presented is pulled from a configurable google cloud storage or aws s3 bucket. 12 | 13 | Before making any change here, you must try the changes in [stage/](stage/) first. 14 | Note that stage is also where [fortio-daily.istio.io](https://fortio-daily.istio.io/) is running. 15 | 16 | ## Initial setup: 17 | 18 | One time setup: 19 | 20 | - 3 Nodes 1 vCPU elastic cluster `istio-prod` (can be redone for `istio-stage` but must stay permanently including upgrades to be tested on stage first, for the prod one) 21 | 22 | - Make sure you connect to the corresponding cluster, ie `gcloud container clusters get-credentials fortio-stage --zone us-west1-c --project istio-io` 23 | 24 | - Istio itself 25 | ``` 26 | # Istio 'perf' mode installation: 27 | sh -c 'sed -e "s/_debug//g" install/kubernetes/istio-auth.yaml | egrep -v -e "- (-v|\"2\")" | kubectl apply -f -' 28 | ``` 29 | 30 | - Cert-manager 31 | 32 | This installs the [cert-manager](https://github.com/jetstack/cert-manager) 33 | ``` 34 | make cert-setup 35 | ``` 36 | 37 | - Ingress 38 | 39 | You can run this step separately for instance if you change ingress rules 40 | in ingress.yaml. 41 | ``` 42 | make ingress-setup 43 | ``` 44 | 45 | - Get SSL certs 46 | 47 | You can run this step separately for instance if switching from staging to 48 | prod or editing cert.yaml to add new domains for instance. 49 | 50 | The ingress ip and DNS must match. 51 | 52 | ``` 53 | make cert-issue 54 | ``` 55 | (check pod logs at each step etc) 56 | 57 | ## Fortio report app 58 | 59 | Install fortio report app: 60 | ``` 61 | make deploy-fortio # or just 'make' 62 | ``` 63 | 64 | You can also delete the fortio-report pods to upgrade to latest fortio 65 | -------------------------------------------------------------------------------- /fortio/cert.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: certmanager.k8s.io/v1alpha1 2 | kind: Issuer 3 | metadata: 4 | name: letsencrypt-staging 5 | namespace: istio-system 6 | spec: 7 | acme: 8 | # The ACME server URL 9 | server: https://acme-staging.api.letsencrypt.org/directory 10 | # Email address used for ACME registration 11 | email: ldemailly@google.com 12 | # Name of a secret used to store the ACME account private key 13 | privateKeySecretRef: 14 | name: letsencrypt-staging-secret 15 | # Enable the HTTP-01 challenge provider 16 | http01: {} 17 | --- 18 | apiVersion: certmanager.k8s.io/v1alpha1 19 | kind: Issuer 20 | metadata: 21 | name: letsencrypt-prod 22 | namespace: istio-system 23 | spec: 24 | acme: 25 | # The ACME server URL 26 | server: https://acme-v01.api.letsencrypt.org/directory 27 | # Email address used for ACME registration 28 | email: ldemailly@google.com 29 | # Name of a secret used to store the ACME account private key 30 | privateKeySecretRef: 31 | name: letsencrypt-prod-secret 32 | # Enable the HTTP-01 challenge provider 33 | http01: {} 34 | --- 35 | apiVersion: certmanager.k8s.io/v1alpha1 36 | kind: Certificate 37 | metadata: 38 | name: fortio-cert-prod 39 | namespace: istio-system 40 | spec: 41 | secretName: istio-ingress-certs 42 | issuerRef: 43 | name: letsencrypt-prod 44 | commonName: fortio.istio.io 45 | dnsNames: 46 | - fortio.istio.io 47 | - istio.fortio.org 48 | acme: 49 | config: 50 | - http01: 51 | # This class doesn't exist on purpose so we'll use the less specific 52 | # istio ingress one because we need the auth off annotation on the svc 53 | ingressClass: not-there 54 | domains: 55 | - fortio.istio.io 56 | - istio.fortio.org 57 | --- 58 | apiVersion: v1 59 | kind: Service 60 | metadata: 61 | name: cert-manager-ingress1 62 | namespace: istio-system 63 | annotations: 64 | auth.istio.io/8089: NONE 65 | spec: 66 | ports: 67 | - port: 8089 68 | name: http-certingr 69 | selector: 70 | certmanager.k8s.io/domain: fortio.istio.io 71 | --- 72 | apiVersion: v1 73 | kind: Service 74 | metadata: 75 | name: cert-manager-ingress2 76 | namespace: istio-system 77 | annotations: 78 | auth.istio.io/8089: NONE 79 | spec: 80 | ports: 81 | - port: 8089 82 | name: http-certingr 83 | selector: 84 | certmanager.k8s.io/domain: istio.fortio.org 85 | --- 86 | apiVersion: extensions/v1beta1 87 | kind: Ingress 88 | metadata: 89 | annotations: 90 | kubernetes.io/ingress.class: istio 91 | certmanager.k8s.io/acme-challenge-type: http01 92 | certmanager.k8s.io/issuer: letsencrypt-prod 93 | # not working: 94 | # kubernetes.io/ingress.global-static-ip-name: fortio-prod-ip 95 | name: istio-ingress-certs-mgr 96 | namespace: istio-system 97 | spec: 98 | rules: 99 | - http: 100 | paths: 101 | # cert-manager adds its own rules to the ingress but we need our 102 | # rule because we have to selectively disable auth for the service 103 | # and route to the service and not a nodeport 104 | - path: /.well-known/acme-challenge/.* 105 | backend: 106 | serviceName: cert-manager-ingress1 107 | servicePort: http-certingr 108 | # Unfortunately host isn't an array and there are no "*" allowed for all 109 | host: fortio.istio.io 110 | - http: 111 | paths: 112 | - path: /.well-known/acme-challenge/.* 113 | backend: 114 | serviceName: cert-manager-ingress2 115 | servicePort: http-certingr 116 | host: istio.fortio.org 117 | -------------------------------------------------------------------------------- /fortio/fortio.yaml: -------------------------------------------------------------------------------- 1 | # TODO: parametrize this yaml and/or keep in sync when 2 | # succesfully experimenting in stage/ (on -stage sites) 3 | --- 4 | # first as it's needed at startup for the deployment below 5 | apiVersion: "config.istio.io/v1alpha2" 6 | kind: EgressRule 7 | metadata: 8 | name: cloud-storage-egress-rule 9 | spec: 10 | destination: 11 | service: "storage.googleapis.com" 12 | ports: 13 | - port: 443 14 | protocol: https 15 | --- 16 | # Service definition 17 | apiVersion: v1 18 | kind: Service 19 | metadata: 20 | name: fortio-report 21 | spec: 22 | ports: 23 | - port: 8079 24 | name: grpc-ping 25 | - port: 8080 26 | name: http-report 27 | - port: 8081 28 | name: http-redir 29 | selector: 30 | app: fortio-report 31 | --- 32 | # Deployment - 2 pods for miminal HA 33 | apiVersion: apps/v1beta1 34 | kind: Deployment 35 | metadata: 36 | name: fortio-report-deployment 37 | spec: 38 | replicas: 2 # tells deployment to run 2 pods matching the template 39 | template: # create pods using pod definition in this template 40 | metadata: 41 | # a unique name is generated from the deployment name 42 | labels: 43 | app: fortio-report 44 | spec: 45 | containers: 46 | - name: fortio-report 47 | image: fortio/fortio:latest_release 48 | imagePullPolicy: Always # needed despite what is documented to really get latest 49 | ports: 50 | - containerPort: 8079 # grpc echo 51 | - containerPort: 8080 # main serving port 52 | - containerPort: 8081 # redirection to https port 53 | args: 54 | - report # report only (readonly) mode 55 | - -sync 56 | # http...443 is not a typo, this is to work with egress 57 | - http://storage.googleapis.com:443/fortio-data?prefix=fortio.istio.io/ 58 | - -sync-interval 59 | - 15m # sync every 15 minutes 60 | #- -loglevel 61 | #- verbose 62 | volumeMounts: 63 | - mountPath: /var/lib/fortio 64 | name: fortio-data 65 | volumes: 66 | - name: fortio-data 67 | emptyDir: 68 | medium: Memory 69 | --- 70 | # Service definition 71 | apiVersion: v1 72 | kind: Service 73 | metadata: 74 | name: fortio-debug 75 | spec: 76 | ports: 77 | - port: 8080 78 | name: http-debug 79 | - port: 8079 80 | name: grpc-ping 81 | selector: 82 | app: fortio-debug 83 | --- 84 | # Deployment (volume definition is in fortio-volume.yaml) 85 | apiVersion: apps/v1beta1 86 | kind: Deployment 87 | metadata: 88 | name: fortio-debug-deployment 89 | spec: 90 | replicas: 1 # tells deployment to run 1 pods matching the template 91 | template: # create pods using pod definition in this template 92 | metadata: 93 | # a unique name is generated from the deployment name 94 | labels: 95 | app: fortio-debug 96 | spec: 97 | containers: 98 | - name: fortio-debug 99 | image: fortio/fortio:latest_release 100 | imagePullPolicy: Always 101 | ports: 102 | - containerPort: 8079 # grpc echo 103 | - containerPort: 8080 # http echo/debug 104 | -------------------------------------------------------------------------------- /fortio/ingress.yaml: -------------------------------------------------------------------------------- 1 | # https version: 2 | apiVersion: extensions/v1beta1 3 | kind: Ingress 4 | metadata: 5 | annotations: 6 | kubernetes.io/ingress.class: istio 7 | # Not working: 8 | # kubernetes.io/ingress.global-static-ip-name: fortio-prod-ip 9 | # Also not working: 10 | kubernetes.io/ingress.allow-http: "false" 11 | name: istio-ingress-https 12 | spec: 13 | tls: 14 | - secretName: istio-ingress-certs # currently ignored/must be this 15 | rules: 16 | - http: 17 | paths: 18 | - path: /debug 19 | backend: 20 | serviceName: fortio-debug 21 | servicePort: http-debug 22 | - path: /fgrpc.PingServer/.* 23 | backend: 24 | serviceName: fortio-debug 25 | servicePort: grpc-ping 26 | - path: /grpc.health.v1.Health/Check 27 | backend: 28 | serviceName: fortio-debug 29 | servicePort: grpc-ping 30 | - path: /.* 31 | backend: 32 | serviceName: fortio-report 33 | servicePort: http-report 34 | # Unfortunately host isn't an array and there are no "*" allowed for all 35 | host: fortio.istio.io 36 | - http: 37 | paths: 38 | - path: /debug 39 | backend: 40 | serviceName: fortio-debug 41 | servicePort: http-debug 42 | - path: /.* 43 | backend: 44 | serviceName: fortio-report 45 | servicePort: http-report 46 | host: istio.fortio.org 47 | --- 48 | # http version (and catch all for all hosts/ips/...) 49 | apiVersion: extensions/v1beta1 50 | kind: Ingress 51 | metadata: 52 | annotations: 53 | kubernetes.io/ingress.class: istio 54 | # not working: 55 | # kubernetes.io/ingress.global-static-ip-name: fortio-prod-ip 56 | name: istio-ingress-http 57 | spec: 58 | rules: 59 | - http: 60 | paths: 61 | - path: /debug 62 | backend: 63 | serviceName: fortio-debug 64 | servicePort: http-debug 65 | - path: /.* 66 | backend: 67 | serviceName: fortio-report 68 | servicePort: http-report 69 | # duplication needed because of #2573 70 | - http: 71 | paths: 72 | - path: /debug 73 | backend: 74 | serviceName: fortio-debug 75 | servicePort: http-debug 76 | - path: /fgrpc.PingServer/.* 77 | backend: 78 | serviceName: fortio-debug 79 | servicePort: grpc-ping 80 | - path: /grpc.health.v1.Health/Check 81 | backend: 82 | serviceName: fortio-debug 83 | servicePort: grpc-ping 84 | - path: /.* 85 | backend: 86 | serviceName: fortio-report 87 | servicePort: http-redir 88 | host: fortio.istio.io 89 | - http: 90 | paths: 91 | - path: /.* 92 | backend: 93 | serviceName: fortio-report 94 | servicePort: http-redir 95 | host: istio.fortio.org 96 | -------------------------------------------------------------------------------- /fortio/stage/Makefile: -------------------------------------------------------------------------------- 1 | 2 | # TODO: parametrize the parent directory yaml etc instead of copy pasta 3 | # on the other hand it is safer to not accidentally damage the prod version. 4 | 5 | 6 | FORTIO_VERSION:=latest 7 | FORTIO_IMAGE:=fortio/fortio:$(FORTIO_VERSION) 8 | FORTIO_MAX_STREAMS:=0 9 | 10 | deploy-fortio: 11 | sed -e s%FORTIO_IMAGE%$(FORTIO_IMAGE)%g -e s%FORTIO_MAX_STREAMS%$(FORTIO_MAX_STREAMS)%g fortio.yaml | $(ISTIO_DIR)/bin/istioctl kube-inject --debug=false -f - | kubectl apply -f - 12 | 13 | all: cert-setup ingress-setup cert-issue deploy-fortio 14 | 15 | PROJECT:=istio-io 16 | MACHINE_TYPE:=n1-standard-1 17 | NUM_NODES:=3 18 | EXTRA_CLUSTER_OPTIONS:=--no-enable-legacy-authorization --enable-autoupgrade --enable-autorepair --enable-autoscaling --min-nodes "3" --max-nodes "10" 19 | 20 | create-cluster: 21 | gcloud container clusters create fortio-stage --zone us-west1-c --project $(PROJECT) --machine-type=$(MACHINE_TYPE) --num-nodes=$(NUM_NODES) $(EXTRA_CLUSTER_OPTIONS) 22 | 23 | ISTIO_VERSION:=0.7.1 24 | ISTIO_DIR:=istio-$(ISTIO_VERSION) 25 | download-istio: 26 | curl -L https://git.io/getLatestIstio | ISTIO_VERSION=$(ISTIO_VERSION) sh - 27 | 28 | install-istio: 29 | kubectl apply -f $(ISTIO_DIR)/install/kubernetes/istio-auth.yaml 30 | 31 | cert-setup: 32 | # TODO: more granular rbac roles 33 | -kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default 34 | helm init 35 | helm repo update 36 | sleep 5 # tiller not ready right away 37 | helm install stable/cert-manager 38 | 39 | ingress-setup: 40 | kubectl apply -f ingress.yaml 41 | @echo "you now need to mark the ingress ip permanent, see kubectl get ingress" 42 | sleep 5 && kubectl get ingress 43 | @echo "also check the dns entry matches:" 44 | host fortio-stage.istio.io 45 | host istio-stage.fortio.org 46 | host fortio-daily.istio.io 47 | 48 | cert-issue: 49 | kubectl apply -f cert.yaml 50 | 51 | dump-images: 52 | kubectl get pods --all-namespaces -o jsonpath="{..image}" | \ 53 | tr -s '[[:space:]]' '\n' | sort | uniq -c 54 | 55 | 56 | # dangerous as it deletes the cert and secret and if using the production 57 | # letsencrypt server, rate limits may mean you can't get a new cert until 58 | # the following week. uncomment if you understand the risks: 59 | #force-reissue: 60 | # kubectl delete secret -n istio-system istio-ingress-certs 61 | 62 | .PHONY: deploy-fortio cert-setup ingress-setup cert-issue all 63 | -------------------------------------------------------------------------------- /fortio/stage/cert.yaml: -------------------------------------------------------------------------------- 1 | # TODO: parametrize the parent directory yaml and/or keep in sync when 2 | # succesfully experimenting here / on -stage sites 3 | apiVersion: certmanager.k8s.io/v1alpha1 4 | kind: Issuer 5 | metadata: 6 | name: letsencrypt-stage 7 | namespace: istio-system 8 | spec: 9 | acme: 10 | # The ACME server URL 11 | server: https://acme-v01.api.letsencrypt.org/directory 12 | # Email address used for ACME registration 13 | email: ldemailly@google.com 14 | # Name of a secret used to store the ACME account private key 15 | privateKeySecretRef: 16 | name: letsencrypt-stage-secret # -stage because of fortio-stage but those are prod letsencrypt certs ! 17 | # Enable the HTTP-01 challenge provider 18 | http01: {} 19 | --- 20 | apiVersion: certmanager.k8s.io/v1alpha1 21 | kind: Certificate 22 | metadata: 23 | name: fortio-cert-stage 24 | namespace: istio-system 25 | spec: 26 | secretName: istio-ingress-certs 27 | issuerRef: 28 | name: letsencrypt-stage 29 | commonName: fortio-stage.istio.io 30 | dnsNames: 31 | - fortio-stage.istio.io 32 | - istio-stage.fortio.org 33 | - fortio-daily.istio.io 34 | acme: 35 | config: 36 | - http01: 37 | # This class doesn't exist on purpose so we'll use the less specific 38 | # istio ingress one because we need the auth off annotation on the svc 39 | ingressClass: not-there 40 | domains: 41 | - fortio-stage.istio.io 42 | - istio-stage.fortio.org 43 | - fortio-daily.istio.io 44 | --- 45 | apiVersion: v1 46 | kind: Service 47 | metadata: 48 | name: cert-manager-ingress1 49 | namespace: istio-system 50 | annotations: 51 | auth.istio.io/8089: NONE 52 | spec: 53 | ports: 54 | - port: 8089 55 | name: http-certingr 56 | selector: 57 | certmanager.k8s.io/domain: fortio-stage.istio.io 58 | --- 59 | apiVersion: v1 60 | kind: Service 61 | metadata: 62 | name: cert-manager-ingress2 63 | namespace: istio-system 64 | annotations: 65 | auth.istio.io/8089: NONE 66 | spec: 67 | ports: 68 | - port: 8089 69 | name: http-certingr 70 | selector: 71 | certmanager.k8s.io/domain: istio-stage.fortio.org 72 | --- 73 | apiVersion: v1 74 | kind: Service 75 | metadata: 76 | name: cert-manager-ingress3 77 | namespace: istio-system 78 | annotations: 79 | auth.istio.io/8089: NONE 80 | spec: 81 | ports: 82 | - port: 8089 83 | name: http-certingr 84 | selector: 85 | certmanager.k8s.io/domain: fortio-daily.istio.io 86 | --- 87 | apiVersion: extensions/v1beta1 88 | kind: Ingress 89 | metadata: 90 | annotations: 91 | kubernetes.io/ingress.class: istio 92 | certmanager.k8s.io/acme-challenge-type: http01 93 | certmanager.k8s.io/issuer: letsencrypt-stage 94 | # not working: 95 | # kubernetes.io/ingress.global-static-ip-name: fortio-stage-ip 96 | name: istio-ingress-certs-mgr 97 | namespace: istio-system 98 | spec: 99 | rules: 100 | - http: 101 | paths: 102 | # cert-manager adds its own rules to the ingress but we need our 103 | # rule because we have to selectively disable auth for the service 104 | # and route to the service and not a nodeport 105 | - path: /.well-known/acme-challenge/.* 106 | backend: 107 | serviceName: cert-manager-ingress1 108 | servicePort: http-certingr 109 | # Unfortunately host isn't an array and there are no "*" allowed for all 110 | host: fortio-stage.istio.io 111 | - http: 112 | paths: 113 | - path: /.well-known/acme-challenge/.* 114 | backend: 115 | serviceName: cert-manager-ingress2 116 | servicePort: http-certingr 117 | host: istio-stage.fortio.org 118 | - http: 119 | paths: 120 | - path: /.well-known/acme-challenge/.* 121 | backend: 122 | serviceName: cert-manager-ingress3 123 | servicePort: http-certingr 124 | host: fortio-daily.istio.io 125 | -------------------------------------------------------------------------------- /fortio/stage/fortio.yaml: -------------------------------------------------------------------------------- 1 | # TODO: parametrize the parent directory yaml and/or keep in sync when 2 | # succesfully experimenting here / on -stage sites 3 | --- 4 | # first as it's needed at startup for the deployment below 5 | apiVersion: "config.istio.io/v1alpha2" 6 | kind: EgressRule 7 | metadata: 8 | name: cloud-storage-egress-rule 9 | spec: 10 | destination: 11 | service: "storage.googleapis.com" 12 | ports: 13 | - port: 443 14 | protocol: https 15 | --- 16 | # Service definition 17 | apiVersion: v1 18 | kind: Service 19 | metadata: 20 | name: fortio-report 21 | spec: 22 | ports: 23 | - port: 8079 24 | name: grpc-ping 25 | - port: 8080 26 | name: http-report 27 | - port: 8081 28 | name: http-redir 29 | selector: 30 | app: fortio-report 31 | --- 32 | # Deployment - 2 pods for miminal HA 33 | apiVersion: apps/v1beta1 34 | kind: Deployment 35 | metadata: 36 | name: fortio-report-deployment 37 | spec: 38 | replicas: 2 # tells deployment to run 2 pods matching the template 39 | template: # create pods using pod definition in this template 40 | metadata: 41 | # a unique name is generated from the deployment name 42 | labels: 43 | app: fortio-report 44 | spec: 45 | containers: 46 | - name: fortio-report 47 | image: FORTIO_IMAGE 48 | imagePullPolicy: Always # needed despite what is documented to really get latest 49 | ports: 50 | - containerPort: 8079 # grpc echo 51 | - containerPort: 8080 # main serving port 52 | - containerPort: 8081 # redirection to https port 53 | args: 54 | - report # report only (readonly) mode 55 | - -sync 56 | # http...443 is not a typo, this is to work with egress 57 | - http://storage.googleapis.com:443/fortio-data?prefix=daily.releases/ 58 | - -sync-interval 59 | - 5m # sync every 5 minutes 60 | #- -loglevel 61 | #- verbose 62 | volumeMounts: 63 | - mountPath: /var/lib/fortio 64 | name: fortio-data 65 | volumes: 66 | - name: fortio-data 67 | emptyDir: 68 | medium: Memory 69 | --- 70 | # Service definition 71 | apiVersion: v1 72 | kind: Service 73 | metadata: 74 | name: fortio-debug 75 | spec: 76 | ports: 77 | - port: 8080 78 | name: http-debug 79 | - port: 8079 80 | name: grpc-ping 81 | selector: 82 | app: fortio-debug 83 | --- 84 | # Deployment (volume definition is in fortio-volume.yaml) 85 | apiVersion: apps/v1beta1 86 | kind: Deployment 87 | metadata: 88 | name: fortio-debug-deployment 89 | spec: 90 | replicas: 1 # tells deployment to run 1 pods matching the template 91 | template: # create pods using pod definition in this template 92 | metadata: 93 | # a unique name is generated from the deployment name 94 | labels: 95 | app: fortio-debug 96 | spec: 97 | containers: 98 | - name: fortio-debug 99 | image: FORTIO_IMAGE 100 | imagePullPolicy: Always 101 | ports: 102 | - containerPort: 8079 # grpc echo 103 | - containerPort: 8080 # http echo/debug 104 | args: 105 | - server 106 | - -grpc-max-streams 107 | - FORTIO_MAX_STREAMS 108 | -------------------------------------------------------------------------------- /fortio/stage/ingress.yaml: -------------------------------------------------------------------------------- 1 | # TODO: parametrize the parent directory yaml and/or keep in sync when 2 | # succesfully experimenting here / on -stage sites 3 | # https version: 4 | apiVersion: extensions/v1beta1 5 | kind: Ingress 6 | metadata: 7 | annotations: 8 | kubernetes.io/ingress.class: istio 9 | # Not working: 10 | # kubernetes.io/ingress.global-static-ip-name: fortio-prod-ip 11 | # Also not working: 12 | kubernetes.io/ingress.allow-http: "false" 13 | name: istio-ingress-https 14 | spec: 15 | tls: 16 | - secretName: istio-ingress-certs # currently ignored/must be this 17 | rules: 18 | - http: 19 | paths: 20 | - path: /debug 21 | backend: 22 | serviceName: fortio-debug 23 | servicePort: http-debug 24 | - path: /fgrpc.PingServer/.* 25 | backend: 26 | serviceName: fortio-debug 27 | servicePort: grpc-ping 28 | - path: /grpc.health.v1.Health/Check 29 | backend: 30 | serviceName: fortio-debug 31 | servicePort: grpc-ping 32 | - path: /.* 33 | backend: 34 | serviceName: fortio-report 35 | servicePort: http-report 36 | # Unfortunately host isn't an array and there are no "*" allowed for all 37 | host: fortio-stage.istio.io 38 | - http: 39 | paths: 40 | - path: /debug 41 | backend: 42 | serviceName: fortio-debug 43 | servicePort: http-debug 44 | - path: /.* 45 | backend: 46 | serviceName: fortio-report 47 | servicePort: http-report 48 | host: istio-stage.fortio.org 49 | - http: 50 | paths: 51 | - path: /debug 52 | backend: 53 | serviceName: fortio-debug 54 | servicePort: http-debug 55 | - path: /.* 56 | backend: 57 | serviceName: fortio-report 58 | servicePort: http-report 59 | host: fortio-daily.istio.io 60 | --- 61 | # http version (and catch all for all hosts/ips/...) 62 | apiVersion: extensions/v1beta1 63 | kind: Ingress 64 | metadata: 65 | annotations: 66 | kubernetes.io/ingress.class: istio 67 | # not working: 68 | # kubernetes.io/ingress.global-static-ip-name: fortio-prod-ip 69 | name: istio-ingress-http 70 | spec: 71 | rules: 72 | - http: 73 | paths: 74 | - path: /debug 75 | backend: 76 | serviceName: fortio-debug 77 | servicePort: http-debug 78 | - path: /.* 79 | backend: 80 | serviceName: fortio-report 81 | servicePort: http-report 82 | # duplication needed because of #2573 83 | - http: 84 | paths: 85 | - path: /debug 86 | backend: 87 | serviceName: fortio-debug 88 | servicePort: http-debug 89 | - path: /fgrpc.PingServer/.* 90 | backend: 91 | serviceName: fortio-debug 92 | servicePort: grpc-ping 93 | - path: /grpc.health.v1.Health/Check 94 | backend: 95 | serviceName: fortio-debug 96 | servicePort: grpc-ping 97 | - path: /.* 98 | backend: 99 | serviceName: fortio-report 100 | servicePort: http-redir 101 | host: fortio-stage.istio.io 102 | - http: 103 | paths: 104 | - path: /.* 105 | backend: 106 | serviceName: fortio-report 107 | servicePort: http-redir 108 | host: istio-stage.fortio.org 109 | - http: 110 | paths: 111 | - path: /.* 112 | backend: 113 | serviceName: fortio-report 114 | servicePort: http-redir 115 | host: fortio-daily.istio.io 116 | --------------------------------------------------------------------------------