├── LICENSE.md ├── README.adoc ├── camel-amq-fakeapp ├── README.adoc ├── pom.xml └── src │ ├── main │ ├── java │ │ ├── META-INF │ │ │ └── MANIFEST.MF │ │ └── org │ │ │ ├── apache │ │ │ └── camel │ │ │ │ └── opentracing │ │ │ │ └── propagation │ │ │ │ └── CamelHeadersExtractAdapter.java │ │ │ └── fgiloux │ │ │ └── camel │ │ │ └── amq │ │ │ └── fake │ │ │ ├── AMQPConfiguration.java │ │ │ ├── Application.java │ │ │ └── LoadSimulator.java │ ├── openshift │ │ ├── camel-amq-fakeapp-bc.yaml │ │ ├── camel-amq-fakeapp-dc.yaml │ │ ├── camel-amq-fakeapp-is.yaml │ │ ├── camel-amq-fakeapp-props-cm.yaml │ │ └── camel-amq-fakeapp-svc.yaml │ └── resources │ │ ├── amqp-certs │ │ ├── amqp.jks │ │ └── tls.crt │ │ ├── application.properties │ │ ├── logback.xml │ │ └── spring │ │ └── camel-context.xml │ └── test │ ├── data │ └── body1.txt │ └── java │ └── org │ └── fgiloux │ └── camel │ └── amq │ └── fake │ ├── ComponentTest.java │ └── RouteTest.java ├── enmasse ├── README.adoc └── enmasse-0.19.0.tgz ├── jenkins ├── README.adoc ├── openshift │ ├── perftest-jenkins-bc.yaml │ ├── perftest-jenkins-is.yaml │ ├── perftest-jenkins-persistent-tm.yaml │ └── perftest-pipeline-bc.yaml ├── pipelines │ ├── Jenkinsfile │ └── README.adoc ├── plugins.txt └── plugins │ ├── performance.hpi │ └── webhook-step.hpi ├── jmeter ├── README.adoc ├── container │ ├── Dockerfile │ ├── README.adoc │ ├── binaries │ │ └── apache-jmeter-4.0.zip │ ├── lib │ │ ├── artemis-jms-client-all-2.6.1.jar │ │ ├── gson-2.8.4.jar │ │ ├── jaeger-client-0.30.3.jar │ │ ├── jaeger-core-0.30.3.jar │ │ ├── jaeger-thrift-0.30.3.jar │ │ ├── jaeger-tracerresolver-0.30.3.jar │ │ ├── opentracing-api-0.31.0.jar │ │ ├── opentracing-jms-2-0.0.10.jar │ │ ├── opentracing-jms-common-0.0.10.jar │ │ ├── opentracing-noop-0.31.0.jar │ │ ├── opentracing-tracerresolver-0.1.4.jar │ │ ├── opentracing-util-0.31.0.jar │ │ ├── proton-j-0.16.0.jar │ │ └── qpid-jms-client-0.11.1.jar │ └── scripts │ │ └── run.sh ├── examples │ ├── README.adoc │ ├── jms-code.jmx │ ├── jms-declarative.jmx │ ├── jms-p2p-example.jmx │ ├── pain_samples.csv │ └── pain_template.xml ├── observability │ └── README.adoc └── openshift │ ├── README.adoc │ ├── apt-jmeter-bc.yaml │ ├── apt-jmeter-is.yaml │ ├── apt-jmeter-job-persistent-tm.yaml │ ├── apt-jmeter-job-tm.yaml │ ├── apt-jmeter-pvc.yaml │ ├── apt-jmx-cm.yaml │ ├── influxdb-dc.yaml │ ├── influxdb-persistent-dc.yaml │ ├── influxdb-pvc.yaml │ ├── influxdb-srv.yaml │ └── java-centos-openjdk8-jdk-is.yaml └── observability ├── README.adoc ├── grafana ├── README.adoc ├── overview.json └── overview.png ├── opentracing └── README.adoc └── prometheus ├── README.adoc └── openshift └── camel-amq-fakeapp-svc.yaml /README.adoc: -------------------------------------------------------------------------------- 1 | = Overview 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | This repository aims to demonstrate how OpenShift and other tools can be leveraged to automate performance tests. 16 | 17 | The current setup covers components that communicate asynchronously through a message broker. 18 | 19 | This was tested with OpenShift 3.9 and 3.10 but may work with other versions as well. 20 | 21 | Details and instructions on the different aspects can be found here: 22 | 23 | * <<./enmasse/README.adoc#,EnMasse broker>> 24 | * <<./camel-amq-fakeapp/README.adoc#,Fake application>> 25 | * <<./jmeter/README.adoc#,JMeter>> 26 | * <<./jenkins/README.adoc#,Automation with Jenkins>> 27 | * <<./observability/README.adoc#,Observability>> 28 | 29 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/README.adoc: -------------------------------------------------------------------------------- 1 | = Fake messaging app used for demo 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | This app consumes and sends messages through a broker. https://github.com/EnMasseProject/enmasse[EnMasse] is used for the purpose of the performance tests that are demonstrated in this project. 16 | 17 | Different response times can be simulated. The following parameters are configurable through a configMap and related to the processing of a single message. 18 | 19 | * load in fraction of CPU cores available (capped to 1) 20 | * duration in milliseconds 21 | * load deviation 22 | * duration deviation 23 | 24 | The distribution is normal/Gaussian: 25 | 26 | Random.nextGaussian() x processing.duration.deviation + processing.duration 27 | 28 | If the processing duration is set to 50 a deviation of 10 means that 70% of values will fall between duration +/- 10, in other words between 40 and 60 milliseconds. 29 | Choosing a value of 0 means no deviation and the duration or load is always the same. 30 | 31 | Failure injection may be added with a future version. 32 | 33 | == Build 34 | 35 | The container image can easily be build with the fabric8 build target, which uses the S2I strategy provided you are connected to OpenShift. Once you have cloned the git repository execute the following commands: 36 | 37 | $ cd auto-perf-test/camel-amq-fakeapp 38 | $ oc login 39 | $ mvn fabric8:build 40 | 41 | == Deployment 42 | 43 | After the build a deployment configuration has been created under target/classes/META-INF/fabric8/openshift/camel-amq-fakeapp-deploymentconfig.yml and can be used to deploy the image with: 44 | 45 | $ mvn fabric8:deploy 46 | 47 | However the fabric8 plugin does not support mounting a secret, which we will use to inject the broker certificates in the next section. The OpenShift object definitions have been stored under auto-perf-test/camel-amq-fakeapp/src/main/openshift and can be manually recreated with: 48 | 49 | $ oc create -f $filename 50 | 51 | where $filename is to be replaced with the names of the files containing the object definition. 52 | 53 | == Certificates 54 | 55 | The certificate extraction and keystore creation has been documented in the EnMasse readme. The instructions are repeated here for convenience: 56 | 57 | [source,bash] 58 | ---- 59 | $ mkdir amqp-certs 60 | $ oc extract secret/external-certs-messaging --to=amqp-certs 61 | $ keytool -v -import -file amqp-certs/tls.crt -alias cacrt -keystore amqp-certs/amqp.jks 62 | ---- 63 | 64 | The next step is to create a secret containing the java keystore and the password to access it: 65 | 66 | [source,bash] 67 | ---- 68 | $ oc create secret generic app-broker-jks --from-literal=amqp.options.transport.trustStorePassword=xxxxxx --from-file=./amqp-certs/amqp.jks 69 | ---- 70 | 71 | The secret can then be mounted in the container by updating the deployment configuration with the following command. You won't need this step if you have used the deployment configuration definition file provided in the git repository. 72 | 73 | $ oc set volume dc/camel-amq-fakeapp --add --name=broker-jks -m /opt/broker-jks -t secret --secret-name=app-broker-jks 74 | 75 | == Configuration externalisation 76 | 77 | You may have noted that the openshift directory in the git repository also contains a configMap. This configMap is used for externalising environment dependent properties that are injected into the Spring Boot application through application.properties. Defaults can be defined in the property file provided with the application archive (jar). Only properties whose values should be overwritten can be specified in the ConfigMap. This works as Spring Boot will privilege values defined externally to the ones defined in application.properties embedded in the jar. *You will have to change the "amqp.host" propery with the name of your own broker*. Here are the steps that have been used afterwards. Alternatively you can use the "oc create -f" command provided earlier. 78 | 79 | $ oc create configmap camel-amq-fakeapp-props --from-env-file=src/main/resources/application.properties 80 | 81 | application.properties is a streamlined version only including values that differ from the ones embedded in the jar. 82 | 83 | Mounting the configMap 84 | 85 | $ oc set volume dc/camel-amq-fakeapp --add --name=app-properties -m /deployments/configuration -t configmap --configmap-name=camel-amq-fakeapp-props 86 | 87 | 88 | Updating the DC to make use of the new application.properties 89 | 90 | $ oc set env dc/camel-amq-fakeapp SPRING_CONFIG_LOCATION='/deployments/configuration/application.properties' 91 | 92 | Another option instead of mounting an additional application.properties into the file system would be to inject the parameters from the configMap directly into the deployment configuration as environment variables. This would however requires a container restart to have new values made available to the application. 93 | 94 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 4.0.0 5 | 6 | org.fgiloux.sandbox 7 | camel-amq-fakeapp 8 | 1.0-SNAPSHOT 9 | jar 10 | 11 | 12 | Camel and ActiveMQ 13 | Spring Boot app running a Camel route connecting to ActiveMQ and simulating some processing 14 | 15 | 16 | UTF-8 17 | 18 | 19 | 3.0.11.fuse-000065-redhat-3 20 | 1.5.4.RELEASE 21 | 22 | 23 | 3.5.34 24 | 3.6.0 25 | 2.19.1 26 | 27 | 28 | 29 | 30 | 0.30.3 31 | 32 | 33 | 34 | 35 | 36 | io.fabric8 37 | fabric8-project-bom-camel-spring-boot 38 | ${fabric8.version} 39 | pom 40 | import 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | io.opentracing.contrib 50 | opentracing-jms-2 51 | 0.0.10 52 | 53 | 54 | 55 | 56 | org.springframework.boot 57 | spring-boot-starter-actuator 58 | 59 | 60 | org.springframework.boot 61 | spring-boot-starter-web 62 | 63 | 64 | org.springframework.boot 65 | spring-boot-starter-tomcat 66 | 67 | 68 | 69 | 70 | org.springframework.boot 71 | spring-boot-starter-undertow 72 | 73 | 74 | org.springframework.boot 75 | spring-boot-configuration-processor 76 | true 77 | 78 | 79 | 80 | 81 | org.apache.camel 82 | camel-spring-boot-starter 83 | 84 | 85 | 86 | org.apache.camel 87 | camel-amqp-starter 88 | 89 | 90 | 91 | org.apache.activemq 92 | activemq-jms-pool 93 | 94 | 95 | 96 | 97 | org.apache.camel 98 | camel-opentracing-starter 99 | 100 | 101 | io.jaegertracing 102 | jaeger-client 103 | ${jaeger.version} 104 | 105 | 106 | 107 | 108 | junit 109 | junit 110 | test 111 | 112 | 113 | org.springframework 114 | spring-test 115 | test 116 | 117 | 118 | org.springframework.boot 119 | spring-boot-starter-test 120 | test 121 | 122 | 123 | org.apache.camel 124 | camel-test-spring 125 | test 126 | 127 | 128 | org.jboss.arquillian.junit 129 | arquillian-junit-container 130 | test 131 | 132 | 133 | org.arquillian.cube 134 | arquillian-cube-openshift 135 | test 136 | 137 | 138 | io.fabric8 139 | kubernetes-assertions 140 | test 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | src/test/resources 149 | true 150 | 151 | 152 | 153 | 154 | 155 | 156 | maven-compiler-plugin 157 | ${maven-compiler-plugin.version} 158 | 159 | 1.8 160 | 1.8 161 | 162 | 163 | 164 | org.apache.maven.plugins 165 | maven-surefire-plugin 166 | ${maven-surefire-plugin.version} 167 | true 168 | 169 | -DenableImageStreamDetection=true 170 | 171 | **/*KT.java 172 | 173 | 174 | 175 | 176 | 177 | org.springframework.boot 178 | spring-boot-maven-plugin 179 | ${spring-boot.version} 180 | 181 | 182 | 183 | repackage 184 | 185 | 186 | 187 | 188 | 189 | 190 | io.fabric8 191 | fabric8-maven-plugin 192 | ${fabric8.maven.plugin.version} 193 | 194 | 195 | 196 | resource 197 | build 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/java/META-INF/MANIFEST.MF: -------------------------------------------------------------------------------- 1 | Manifest-Version: 1.0 2 | Class-Path: 3 | 4 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/java/org/apache/camel/opentracing/propagation/CamelHeadersExtractAdapter.java: -------------------------------------------------------------------------------- 1 | /** 2 | * Licensed to the Apache Software Foundation (ASF) under one or more 3 | * contributor license agreements. See the NOTICE file distributed with 4 | * this work for additional information regarding copyright ownership. 5 | * The ASF licenses this file to You under the Apache License, Version 2.0 6 | * (the "License"); you may not use this file except in compliance with 7 | * the License. You may obtain a copy of the License at 8 | * 9 | * http://www.apache.org/licenses/LICENSE-2.0 10 | * 11 | * Unless required by applicable law or agreed to in writing, software 12 | * distributed under the License is distributed on an "AS IS" BASIS, 13 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | * See the License for the specific language governing permissions and 15 | * limitations under the License. 16 | */ 17 | package org.apache.camel.opentracing.propagation; 18 | 19 | import java.util.HashMap; 20 | import java.util.Iterator; 21 | import java.util.Map; 22 | 23 | import io.opentracing.propagation.TextMap; 24 | 25 | public final class CamelHeadersExtractAdapter implements TextMap { 26 | 27 | // As per the JMS spec, header names must be valid Java identifier part characters. 28 | // This means that any header names that contain illegal characters (- for example) should be handled correctly 29 | // Opentracing java-jms does it as follows, which is not compatible with the Camel implementation. 30 | static final String JMS_DASH = "_$dash$_"; 31 | 32 | private final Map map = new HashMap<>(); 33 | 34 | public CamelHeadersExtractAdapter(final Map map) { 35 | // Extract string valued map entries 36 | map.entrySet().stream().filter(e -> e.getValue() instanceof String).forEach(e -> 37 | this.map.put(e.getKey().replace(JMS_DASH, "-"), (String) e.getValue())); 38 | } 39 | 40 | @Override 41 | public Iterator> iterator() { 42 | return map.entrySet().iterator(); 43 | } 44 | 45 | @Override 46 | public void put(String key, String value) { 47 | throw new UnsupportedOperationException("CamelHeadersExtractAdapter should only be used with Tracer.extract()"); 48 | } 49 | } 50 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/java/org/fgiloux/camel/amq/fake/AMQPConfiguration.java: -------------------------------------------------------------------------------- 1 | package org.fgiloux.camel.amq.fake; 2 | 3 | import java.util.HashMap; 4 | import java.util.Map; 5 | 6 | import org.slf4j.Logger; 7 | import org.slf4j.LoggerFactory; 8 | import org.springframework.boot.context.properties.ConfigurationProperties; 9 | import org.springframework.context.annotation.Configuration; 10 | 11 | /** 12 | * Configuration parameters filled in from application.properties and overridden using env variables on Openshift. 13 | */ 14 | @Configuration 15 | @ConfigurationProperties(prefix = "amqp") 16 | public class AMQPConfiguration { 17 | 18 | private static final Logger LOG = LoggerFactory.getLogger(AMQPConfiguration.class); 19 | 20 | private String host; 21 | private Integer port; 22 | private String username; 23 | private String password; 24 | private String truststoreLocation; 25 | private String truststorePassword; 26 | private String scheme; 27 | 28 | private Map options = new HashMap(); 29 | 30 | public AMQPConfiguration() { 31 | } 32 | 33 | public String getHost() { 34 | return host; 35 | } 36 | 37 | public void setHost(String host) { 38 | this.host = host; 39 | } 40 | 41 | public Integer getPort() { 42 | return port; 43 | } 44 | 45 | public void setPort(Integer port) { 46 | this.port = port; 47 | } 48 | 49 | public String getUsername() { 50 | return username; 51 | } 52 | 53 | public void setUsername(String username) { 54 | this.username = username; 55 | } 56 | 57 | public String getPassword() { 58 | return password; 59 | } 60 | 61 | public void setPassword(String password) { 62 | this.password = password; 63 | } 64 | 65 | public String getTruststoreLocation() { 66 | return truststoreLocation; 67 | } 68 | 69 | public void setTruststoreLocation(String truststoreLocation) { 70 | this.truststoreLocation = truststoreLocation; 71 | } 72 | 73 | public String getTruststorePassword() { 74 | return truststorePassword; 75 | } 76 | 77 | public void setTruststorePassword(String truststorePassword) { 78 | this.truststorePassword = truststorePassword; 79 | } 80 | 81 | public String getScheme() { 82 | return scheme; 83 | } 84 | 85 | public void setScheme(String scheme) { 86 | this.scheme = scheme; 87 | } 88 | 89 | public Map getOptions() { 90 | return options; 91 | } 92 | 93 | public String getFullUri() { 94 | StringBuilder optionStringBuilder = new StringBuilder(); 95 | for (Map.Entry option: options.entrySet()) 96 | optionStringBuilder.append(option.getKey()).append("=").append(option.getValue()).append("&"); 97 | optionStringBuilder.setLength(Math.max(optionStringBuilder.length() - 1, 0)); 98 | String res = new StringBuilder(scheme). 99 | append("://"). 100 | append(host). 101 | append(":"). 102 | append(port). 103 | append("?"). 104 | append(optionStringBuilder).toString(); 105 | LOG.debug("URI: {}", res); 106 | return res; 107 | } 108 | } 109 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/java/org/fgiloux/camel/amq/fake/Application.java: -------------------------------------------------------------------------------- 1 | package org.fgiloux.camel.amq.fake; 2 | 3 | import org.apache.activemq.jms.pool.PooledConnectionFactory; 4 | import org.apache.camel.component.amqp.AMQPComponent; 5 | import org.apache.camel.opentracing.starter.CamelOpenTracing; 6 | import org.apache.qpid.jms.JmsConnectionFactory; 7 | import org.springframework.boot.SpringApplication; 8 | import org.springframework.boot.autoconfigure.SpringBootApplication; 9 | import org.springframework.context.annotation.Bean; 10 | import org.springframework.context.annotation.ImportResource; 11 | 12 | /** 13 | * The Spring-boot main class. 14 | */ 15 | @SpringBootApplication 16 | @CamelOpenTracing 17 | @ImportResource({"classpath:spring/camel-context.xml"}) 18 | public class Application { 19 | 20 | public static void main(String[] args) { 21 | SpringApplication.run(Application.class, args); 22 | } 23 | 24 | @Bean(name = "amqp") 25 | AMQPComponent amqpComponent(AMQPConfiguration config) { 26 | JmsConnectionFactory qpid = new JmsConnectionFactory(); 27 | qpid.setTopicPrefix("topic://"); 28 | qpid.setUsername(config.getUsername()); 29 | qpid.setPassword(config.getPassword()); 30 | qpid.setRemoteURI(config.getFullUri()); 31 | PooledConnectionFactory factory = new PooledConnectionFactory(); 32 | factory.setConnectionFactory(qpid); 33 | return new AMQPComponent(factory); 34 | } 35 | 36 | @Bean(name = "loadSimulator") 37 | LoadSimulator loadSimulator() { 38 | return new LoadSimulator(); 39 | } 40 | 41 | } 42 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/java/org/fgiloux/camel/amq/fake/LoadSimulator.java: -------------------------------------------------------------------------------- 1 | package org.fgiloux.camel.amq.fake; 2 | 3 | import java.util.concurrent.ThreadLocalRandom; 4 | 5 | import org.apache.camel.Exchange; 6 | import org.apache.camel.Processor; 7 | import org.slf4j.Logger; 8 | import org.slf4j.LoggerFactory; 9 | import org.springframework.boot.context.properties.ConfigurationProperties; 10 | import org.springframework.context.annotation.Configuration; 11 | 12 | /** 13 | * Configuration parameters filled in from application.properties and overridden using env variables on Openshift. 14 | */ 15 | @Configuration 16 | @ConfigurationProperties(prefix = "processing") 17 | public class LoadSimulator implements Processor { 18 | 19 | protected static final Logger LOG = LoggerFactory.getLogger(LoadSimulator.class); 20 | 21 | private double load; 22 | private double duration; 23 | private double deviation4load; 24 | private double deviation4duration; 25 | 26 | public double getLoad() { 27 | return load; 28 | } 29 | public void setLoad(double load) { 30 | this.load = load; 31 | } 32 | public double getDuration() { 33 | return duration; 34 | } 35 | public void setDuration(double duration) { 36 | this.duration = duration; 37 | } 38 | public double getDeviation4load() { 39 | return deviation4load; 40 | } 41 | public void setDeviation4load(double deviation4load) { 42 | this.deviation4load = deviation4load; 43 | } 44 | public double getDeviation4duration() { 45 | return deviation4duration; 46 | } 47 | public void setDeviation4duration(int deviation4duration) { 48 | this.deviation4duration = deviation4duration; 49 | } 50 | 51 | public void process(Exchange exchange) throws Exception { 52 | // TODO: It would be possible to make the load dependent on messages. 53 | // TODO: and also to simulate IO wait. 54 | long startTime = System.currentTimeMillis(); 55 | double factualDuration = ThreadLocalRandom.current().nextGaussian() * deviation4duration + duration; 56 | double factualLoad = ThreadLocalRandom.current().nextGaussian() * deviation4load + load; 57 | try { 58 | // Loop for the given duration 59 | while (System.currentTimeMillis() - startTime < factualDuration) { 60 | // Every 100ms, sleep for the percentage of not loaded time 61 | if (System.currentTimeMillis() % 100 == 0) { 62 | Thread.sleep((long) Math.floor((1 - factualLoad) * 100)); 63 | } 64 | } 65 | } catch (InterruptedException e) { 66 | e.printStackTrace(); 67 | } 68 | } 69 | } 70 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/openshift/camel-amq-fakeapp-bc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: BuildConfig 3 | metadata: 4 | creationTimestamp: null 5 | labels: 6 | app: camel-amq-fakeapp 7 | group: org.fgiloux.sandbox 8 | provider: fabric8 9 | version: 1.0-SNAPSHOT 10 | name: camel-amq-fakeapp-s2i 11 | spec: 12 | failedBuildsHistoryLimit: 5 13 | nodeSelector: {} 14 | output: 15 | to: 16 | kind: ImageStreamTag 17 | name: camel-amq-fakeapp:latest 18 | postCommit: {} 19 | resources: {} 20 | runPolicy: Serial 21 | source: 22 | binary: {} 23 | type: Binary 24 | strategy: 25 | sourceStrategy: 26 | from: 27 | kind: DockerImage 28 | name: fabric8/s2i-java:2.1 29 | type: Source 30 | successfulBuildsHistoryLimit: 5 31 | triggers: [] 32 | status: 33 | lastVersion: 0 34 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/openshift/camel-amq-fakeapp-dc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: DeploymentConfig 3 | metadata: 4 | annotations: 5 | fabric8.io/git-branch: master 6 | fabric8.io/git-commit: 9a41048f1170bcf792cfdfa8f7059e788b018518 7 | fabric8.io/iconUrl: img/icons/camel.svg 8 | fabric8.io/metrics-path: dashboard/file/camel-routes.json/?var-project=camel-amq-fakeapp&var-version=1.0-SNAPSHOT 9 | creationTimestamp: null 10 | generation: 1 11 | labels: 12 | app: camel-amq-fakeapp 13 | group: org.fgiloux.sandbox 14 | provider: fabric8 15 | version: 1.0-SNAPSHOT 16 | name: camel-amq-fakeapp 17 | spec: 18 | replicas: 1 19 | revisionHistoryLimit: 2 20 | selector: 21 | app: camel-amq-fakeapp 22 | group: org.fgiloux.sandbox 23 | provider: fabric8 24 | strategy: 25 | activeDeadlineSeconds: 21600 26 | resources: {} 27 | rollingParams: 28 | intervalSeconds: 1 29 | maxSurge: 25% 30 | maxUnavailable: 25% 31 | timeoutSeconds: 3600 32 | updatePeriodSeconds: 1 33 | type: Rolling 34 | template: 35 | metadata: 36 | annotations: 37 | fabric8.io/git-branch: master 38 | fabric8.io/git-commit: 9a41048f1170bcf792cfdfa8f7059e788b018518 39 | fabric8.io/iconUrl: img/icons/camel.svg 40 | fabric8.io/metrics-path: dashboard/file/camel-routes.json/?var-project=camel-amq-fakeapp&var-version=1.0-SNAPSHOT 41 | creationTimestamp: null 42 | labels: 43 | app: camel-amq-fakeapp 44 | group: org.fgiloux.sandbox 45 | provider: fabric8 46 | version: 1.0-SNAPSHOT 47 | spec: 48 | containers: 49 | - env: 50 | - name: KUBERNETES_NAMESPACE 51 | valueFrom: 52 | fieldRef: 53 | apiVersion: v1 54 | fieldPath: metadata.namespace 55 | - name: SPRING_CONFIG_LOCATION 56 | value: /deployments/configuration/application.properties 57 | - name: amqp.options.transport.trustStorePassword 58 | valueFrom: 59 | secretKeyRef: 60 | name: app-broker-jks 61 | key: amqp.options.transport.trustStorePassword 62 | - name: JAEGER_SERVICE_NAME 63 | value: fakeapp 64 | - name: JAEGER_SAMPLER_PARAM 65 | value: '1.0' 66 | - name: JAEGER_PROPAGATION 67 | value: 'jaeger,b3' 68 | image: docker-registry.default.svc:5000/perftest/camel-amq-fakeapp 69 | imagePullPolicy: IfNotPresent 70 | livenessProbe: 71 | failureThreshold: 3 72 | httpGet: 73 | path: /health 74 | port: 8081 75 | scheme: HTTP 76 | initialDelaySeconds: 180 77 | periodSeconds: 10 78 | successThreshold: 1 79 | timeoutSeconds: 1 80 | name: fake-app 81 | ports: 82 | - containerPort: 8080 83 | name: http 84 | protocol: TCP 85 | - containerPort: 9779 86 | name: prometheus 87 | protocol: TCP 88 | - containerPort: 8778 89 | name: jolokia 90 | protocol: TCP 91 | readinessProbe: 92 | failureThreshold: 3 93 | httpGet: 94 | path: /health 95 | port: 8081 96 | scheme: HTTP 97 | initialDelaySeconds: 10 98 | periodSeconds: 10 99 | successThreshold: 1 100 | timeoutSeconds: 1 101 | resources: {} 102 | securityContext: 103 | privileged: false 104 | terminationMessagePath: /dev/termination-log 105 | terminationMessagePolicy: File 106 | volumeMounts: 107 | - mountPath: /deployments/configuration 108 | name: app-properties 109 | - mountPath: /opt/broker-jks 110 | name: app-broker-jks 111 | - image: jaegertracing/jaeger-agent 112 | name: jaeger-agent 113 | ports: 114 | - containerPort: 5775 115 | protocol: UDP 116 | - containerPort: 5778 117 | - containerPort: 6831 118 | protocol: UDP 119 | - containerPort: 6832 120 | protocol: UDP 121 | args: 122 | - '--collector.host-port=jaeger-collector.perftest.svc:14267' 123 | dnsPolicy: ClusterFirst 124 | restartPolicy: Always 125 | schedulerName: default-scheduler 126 | securityContext: {} 127 | terminationGracePeriodSeconds: 30 128 | volumes: 129 | - configMap: 130 | defaultMode: 420 131 | name: camel-amq-fakeapp-props 132 | name: app-properties 133 | - name: app-broker-jks 134 | secret: 135 | defaultMode: 420 136 | secretName: app-broker-jks 137 | test: false 138 | triggers: 139 | - type: ConfigChange 140 | - imageChangeParams: 141 | automatic: true 142 | containerNames: 143 | - fake-app 144 | from: 145 | kind: ImageStreamTag 146 | name: camel-amq-fakeapp:latest 147 | namespace: perftest 148 | type: ImageChange 149 | status: 150 | availableReplicas: 0 151 | latestVersion: 0 152 | observedGeneration: 0 153 | replicas: 0 154 | unavailableReplicas: 0 155 | updatedReplicas: 0 156 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/openshift/camel-amq-fakeapp-is.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ImageStream 3 | metadata: 4 | creationTimestamp: null 5 | generation: 1 6 | labels: 7 | app: camel-amq-fakeapp 8 | group: org.fgiloux.sandbox 9 | provider: fabric8 10 | version: 1.0-SNAPSHOT 11 | name: camel-amq-fakeapp 12 | spec: 13 | lookupPolicy: 14 | local: false 15 | tags: 16 | - annotations: null 17 | from: 18 | kind: DockerImage 19 | name: docker-registry.default.svc:5000/perftest/camel-amq-fakeapp:latest 20 | generation: null 21 | importPolicy: {} 22 | name: latest 23 | referencePolicy: 24 | type: "" 25 | status: 26 | dockerImageRepository: "" 27 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/openshift/camel-amq-fakeapp-props-cm.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | data: 3 | application.properties: | 4 | # Binding health checks to an internal port 5 | # management.port=8081 6 | 7 | # disable all management enpoints except health 8 | # endpoints.enabled=false 9 | # endpoints.health.enabled=true 10 | 11 | # TO BE CHANGED 12 | logging.config=classpath:logback.xml 13 | 14 | # The name of the Camel app 15 | # camel.springboot.name=fakeApp 16 | 17 | # Keeps the application alive 18 | # camel.springboot.main-run-controller=true 19 | 20 | # AMQP connection configuration (can be overridden in Openshift by injecting environment variables into the deployment configuration) 21 | # Passwords must not be populated in this file. They can be injected using secrets. The fields are here only for documentation purpose. 22 | # amqp.scheme=amqps 23 | # amqp.host=localhost 24 | 25 | # TO BE CHANGED 26 | amqp.host=messaging-perftest.apps.sandbox.com 27 | # amqp.port=443 28 | 29 | amqp.options.transport.trustStoreLocation=/opt/broker-jks/amqp.jks 30 | # amqp.options.transport.verifyHost=false 31 | amqp.options.disableReplyTo=true 32 | amqp.options.acknowledgementModeName=CLIENT_ACKNOWLEDGE 33 | 34 | # message.input=input.queue 35 | # message.output=output.queue 36 | 37 | # Processing 38 | # load in fraction of CPU cores available (capped to 1) 39 | # processing.load=0.8 40 | # duration in ms 41 | # processing.duration=200 42 | # distribution is normal/Gaussian: Random.nextGaussian() x processing.duration.deviation + processing.duration 43 | # with a deviation of 100, this means that 70% of values will fall between duration +/- 100, in other words between 400 and 600 milliseconds 44 | # if process.duration is set to 500. 45 | # choosing a value of 0 means no deviation and the duration or load is always the same. 46 | # processing.duration.deviation=0 47 | # processing.load.deviation=0 48 | kind: ConfigMap 49 | metadata: 50 | creationTimestamp: null 51 | name: camel-amq-fakeapp-props 52 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/openshift/camel-amq-fakeapp-svc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | annotations: 5 | fabric8.io/git-branch: master 6 | fabric8.io/git-commit: 9a41048f1170bcf792cfdfa8f7059e788b018518 7 | fabric8.io/iconUrl: img/icons/camel.svg 8 | prometheus.io/port: "9779" 9 | prometheus.io/scrape: "true" 10 | creationTimestamp: null 11 | labels: 12 | app: camel-amq-fakeapp 13 | expose: "true" 14 | group: org.fgiloux.sandbox 15 | provider: fabric8 16 | version: 1.0-SNAPSHOT 17 | name: camel-amq-fakeapp 18 | spec: 19 | ports: 20 | - name: http 21 | port: 8080 22 | protocol: TCP 23 | targetPort: 8080 24 | - name: prometheus 25 | port: 9779 26 | protocol: TCP 27 | targetPort: 9779 28 | selector: 29 | app: camel-amq-fakeapp 30 | group: org.fgiloux.sandbox 31 | provider: fabric8 32 | sessionAffinity: None 33 | type: ClusterIP 34 | status: 35 | loadBalancer: {} 36 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/resources/amqp-certs/amqp.jks: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/camel-amq-fakeapp/src/main/resources/amqp-certs/amqp.jks -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/resources/amqp-certs/tls.crt: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIDVzCCAj+gAwIBAgIJAKoT94Skyx2lMA0GCSqGSIb3DQEBCwUAMEIxCzAJBgNV 3 | BAYTAlhYMRUwEwYDVQQHDAxEZWZhdWx0IENpdHkxHDAaBgNVBAoME0RlZmF1bHQg 4 | Q29tcGFueSBMdGQwHhcNMTgwODEyMTMwNzM4WhcNNDgwOTIzMTMwNzM4WjBCMQsw 5 | CQYDVQQGEwJYWDEVMBMGA1UEBwwMRGVmYXVsdCBDaXR5MRwwGgYDVQQKDBNEZWZh 6 | dWx0IENvbXBhbnkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA 7 | utPSV+lv8lHo8wFoxLSCzHCGl1okpkkOGOo3Mn9/SqZpe/UdT6OIAgbLp1hU4scL 8 | Efpij6+ARyamvojkmtDR6bwlxjlvqpf8MyODsNnnn7lx6s07BsepXyD6pkq2WWg1 9 | g4eWoL9A8XI940Y0Ba/n8I8eHBpd0oJels/XhOtKKTd1yW6fJ588e8auCebBukyJ 10 | wYRUSbsuET8j02bu6S4wVkCPJhErIP7cOCA54gI3QP3mSiwwlcbdls86h2CDm/dw 11 | +6EvpSD5xT5lj5MbViX0ldnmmd/OouA0XyG3c2wExXjuH1hfoV4ajtKwX9++dYuS 12 | sx6glejXj7LuNMGADI9YEQIDAQABo1AwTjAdBgNVHQ4EFgQUjcbcup3mMkum1uwe 13 | S0+VY8dZJXAwHwYDVR0jBBgwFoAUjcbcup3mMkum1uweS0+VY8dZJXAwDAYDVR0T 14 | BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAstNamAxQqa1+lN8BBs8PYpbphWhE 15 | 2A4aCATNi6Hf5s9U1AG92pVee0TN5dpi4498E7U8VZeRN2RcQZSlI8BuWeMi94Jx 16 | 0ntYLKofdbj6YQQBvK+gJpwOJ5TqaQH6c40ZkduyRfZJU4s/Iw3R8qOHRUCFgqNR 17 | lu88GuakL0KaNdS5QKyoosDBochPBs8ddLiUy48SJpGhvMPmGqgLC28VPRxgcPqO 18 | Vf4WHvENYqPoe6Ts6ahdqzi3t5Hb9oZYvsIgQ+OIso//3Hv6LeHddQdWVLm/2k/o 19 | TOcwjcZuAxd1AzZa7iLonlCAUkNb9Q0BKyd12ODOCgAXXsTJwhYNQSIGPg== 20 | -----END CERTIFICATE----- 21 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/resources/application.properties: -------------------------------------------------------------------------------- 1 | # Binding health checks to an internal port 2 | management.port=8081 3 | 4 | # disable all management enpoints except health 5 | endpoints.enabled=false 6 | endpoints.health.enabled=true 7 | 8 | logging.config=classpath:logback.xml 9 | 10 | # The name of the Camel app 11 | camel.springboot.name=fakeApp 12 | 13 | # Keeps the application alive 14 | camel.springboot.main-run-controller=true 15 | 16 | # AMQP connection configuration (can be overridden in Openshift by injecting environment variables into the deployment configuration) 17 | # Passwords must not be populated in this file. They can be injected using secrets. The fields are here only for documentation purpose. 18 | amqp.scheme=amqps 19 | # amqp.host=localhost 20 | amqp.host=messaging-perftest.apps.sandbox.com 21 | # amqp.port=5671 22 | amqp.port=443 23 | amqp.username=theuser 24 | amqp.password=Thepassword1! 25 | amqp.options.transport.trustStoreLocation=src/main/resources/amqp-certs/amqp.jks 26 | amqp.options.transport.trustStorePassword=redhat 27 | amqp.options.transport.verifyHost=false 28 | amqp.options.acknowledgementModeName=CLIENT_ACKNOWLEDGE 29 | 30 | message.input=input.queue 31 | message.output=output.queue 32 | 33 | # Processing 34 | # load in fraction of CPU cores available (capped to 1) 35 | processing.load=0.8 36 | # duration in ms 37 | processing.duration=200 38 | # distribution is normal/Gaussian: Random.nextGaussian() x processing.duration.deviation + processing.duration 39 | # with a deviation of 100, this means that 70% of values will fall between duration +/- 100, in other words between 400 and 600 milliseconds 40 | # if process.duration is set to 500. 41 | # choosing a value of 0 means no deviation and the duration or load is always the same. 42 | processing.duration.deviation=0 43 | processing.load.deviation=0 44 | 45 | ###### TODO: nb of threads is set by max listeners 46 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/resources/logback.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 8 | 9 | %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/main/resources/spring/camel-context.xml: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 6 | 7 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/test/data/body1.txt: -------------------------------------------------------------------------------- 1 | Body of message 1: Hello everybody! -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/test/java/org/fgiloux/camel/amq/fake/ComponentTest.java: -------------------------------------------------------------------------------- 1 | package org.fgiloux.camel.amq.fake; 2 | 3 | import org.apache.camel.CamelContext; 4 | import org.apache.camel.ConsumerTemplate; 5 | import org.apache.camel.ProducerTemplate; 6 | import org.apache.commons.io.FileUtils; 7 | import org.fgiloux.camel.amq.fake.Application; 8 | import org.apache.camel.test.spring.CamelSpringBootRunner; 9 | 10 | import static org.junit.Assert.*; 11 | 12 | import java.io.File; 13 | 14 | import org.junit.Test; 15 | import org.junit.runner.RunWith; 16 | import org.slf4j.Logger; 17 | import org.slf4j.LoggerFactory; 18 | import org.springframework.beans.factory.annotation.Autowired; 19 | import org.springframework.boot.test.context.SpringBootTest; 20 | import org.springframework.test.annotation.DirtiesContext; 21 | // import org.springframework.test.context.ActiveProfiles; 22 | 23 | // Embedded Artemis should be used for component testing 24 | // as it provides isolation and repeatability. 25 | // https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-messaging.html 26 | 27 | //@TestPropertySource("classpath:test-application.properties") 28 | // test-application.properties would contain the information for the embedded broker 29 | // username and password can be generated by maven and injected from there. 30 | 31 | @RunWith(CamelSpringBootRunner.class) 32 | @SpringBootTest(classes = Application.class) 33 | //@ActiveProfiles("test") 34 | @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD) 35 | public class ComponentTest { 36 | 37 | protected static final Logger LOG = LoggerFactory.getLogger(ComponentTest.class); 38 | 39 | @Autowired 40 | CamelContext context; 41 | 42 | @Autowired 43 | ProducerTemplate producer; 44 | 45 | @Autowired 46 | ConsumerTemplate consumer; 47 | 48 | @Test 49 | public void testMainRouteE2E() throws Exception { 50 | context.start(); 51 | String body = FileUtils.readFileToString(new File("src/test/data/body1.txt")); 52 | producer.sendBody("amqp:{{message.input}}", body); 53 | // Let give up to 5s to receive the message 54 | final int WAIT_LIMIT = 5000; 55 | String message = consumer.receiveBody("amqp:{{message.output}}", WAIT_LIMIT, String.class); 56 | LOG.info("Message received: {}", message); 57 | assertNotNull(message); 58 | assertEquals(body.trim(),message.trim()); 59 | context.stop(); 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /camel-amq-fakeapp/src/test/java/org/fgiloux/camel/amq/fake/RouteTest.java: -------------------------------------------------------------------------------- 1 | package org.fgiloux.camel.amq.fake; 2 | 3 | import org.apache.camel.CamelContext; 4 | import org.apache.camel.ConsumerTemplate; 5 | import org.apache.camel.ProducerTemplate; 6 | import org.apache.camel.builder.AdviceWithRouteBuilder; 7 | import org.apache.camel.component.mock.MockEndpoint; 8 | import org.apache.camel.test.spring.MockEndpoints; 9 | import org.apache.camel.test.spring.UseAdviceWith; 10 | import org.apache.commons.io.FileUtils; 11 | import org.fgiloux.camel.amq.fake.Application; 12 | import org.apache.camel.test.spring.CamelSpringBootRunner; 13 | 14 | import static org.junit.Assert.*; 15 | 16 | import java.io.File; 17 | 18 | import org.junit.Before; 19 | import org.junit.Test; 20 | import org.junit.runner.RunWith; 21 | import org.slf4j.Logger; 22 | import org.slf4j.LoggerFactory; 23 | import org.springframework.beans.factory.annotation.Autowired; 24 | import org.springframework.boot.test.context.SpringBootTest; 25 | import org.springframework.test.annotation.DirtiesContext; 26 | // import org.springframework.test.context.ActiveProfiles; 27 | 28 | //@TestPropertySource("classpath:test-application.properties") 29 | @RunWith(CamelSpringBootRunner.class) 30 | @SpringBootTest(classes = Application.class) 31 | //@ActiveProfiles("test") 32 | @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD) 33 | @MockEndpoints 34 | @UseAdviceWith 35 | public class RouteTest { 36 | 37 | protected static final Logger LOG = LoggerFactory.getLogger(RouteTest.class); 38 | 39 | @Autowired 40 | CamelContext context; 41 | 42 | @Autowired 43 | ProducerTemplate producer; 44 | 45 | @Autowired 46 | ConsumerTemplate consumer; 47 | 48 | @Before 49 | public void configureMocks() throws Exception { 50 | context.getRouteDefinition("main") 51 | .adviceWith(context, new AdviceWithRouteBuilder() { 52 | @Override 53 | public void configure() throws Exception { 54 | // mocking endpoints is done through annotations 55 | //mockEndpoints(); 56 | replaceFromWith("direct:input"); 57 | } 58 | }); 59 | } 60 | 61 | @Test 62 | public void testRouteRunning() throws Exception { 63 | context.start(); 64 | assertTrue(context.getRouteStatus("main").isStarted()); 65 | context.stop(); 66 | } 67 | 68 | @Test 69 | public void testRouteInternal() throws Exception { 70 | context.start(); 71 | String body = FileUtils.readFileToString(new File("src/test/data/body1.txt")); 72 | producer.sendBody("direct:input", body); 73 | MockEndpoint resultEndpoint = MockEndpoint.resolve(context, "mock:amqp:{{message.output}}"); 74 | LOG.info("endpoint id:" + resultEndpoint.getId()); 75 | LOG.info("endpoint name:" + resultEndpoint.getName()); 76 | String message = resultEndpoint.getExchanges().get(0).getIn().getBody(String.class); 77 | LOG.info("Message received: {}", message); 78 | resultEndpoint.expectedMessageCount(1); 79 | resultEndpoint.assertIsSatisfied(); 80 | assertNotNull(message); 81 | assertEquals(body.trim(),message.trim()); 82 | context.stop(); 83 | } 84 | 85 | } 86 | -------------------------------------------------------------------------------- /enmasse/README.adoc: -------------------------------------------------------------------------------- 1 | = Enmasse setup 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | This page contains the instructions to set up and configure https://github.com/EnMasseProject/enmasse[EnMasse] for the purpose of the performance tests that are demonstrated in this project. Nothing prevents you to use a different messaging broker, EnMasse is however a perfect fit for test automation as it is possible to get shared or dedicated brokers provisioned on demand and running on OpenShift/Kubernetes. 16 | 17 | == Deployment 18 | 19 | The deployment is pretty straightforward. I used what was the current release at the time I set my environment up https://github.com/EnMasseProject/enmasse/releases/tag/0.19.0[0.19.0]. 20 | Once you have cloned the repository you can follow the http://enmasse.io/documentation/0.19.0/[instructions] to deploy EnMasse. In short the following will have the user admin deploying the EnMasse components in the perftest project using provided ansible playbook: 21 | [source,bash] 22 | ---- 23 | $ ./enmasse-0.19.0/deploy-openshift.sh -n perftest -u admin 24 | ---- 25 | 26 | A route then gets created for you, which provides access to the EnMasse console. 27 | 28 | By selecting the Addresses menu on the left and clicking on the Create button on the top it is possible to create a queue. For the purpose of this demo you will need to create two queues: *input.queue* and *output.queue* as pooled queues. 29 | 30 | == Certificates 31 | 32 | Extracting the certificate that will be used to connect to the broker and importing it in a java keystore can easily be done with: 33 | [source,bash] 34 | ---- 35 | $ mkdir amqp-certs 36 | $ oc extract secret/external-certs-messaging --to=amqp-certs 37 | $ keytool -v -import -file amqp-certs/tls.crt -alias cacrt -keystore amqp-certs/amqp.jks 38 | ---- 39 | -------------------------------------------------------------------------------- /enmasse/enmasse-0.19.0.tgz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/enmasse/enmasse-0.19.0.tgz -------------------------------------------------------------------------------- /jenkins/README.adoc: -------------------------------------------------------------------------------- 1 | = Jenkins overview 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | This sub-repository aims to describe the setup required for using and configuring Jenking on OpenShift for our purpose, from the extension of the Jenkins base image to its use for running automated performance tests. 16 | 17 | == Image extension 18 | 19 | Although Jenkins base image could be used as is, it makes sense to extend it with the additional plugins we will use, so that they don't have to be downloaded for each new project. 20 | 21 | The way to extend the image using S2I is in https://docs.openshift.com/container-platform/3.10/using_images/other_images/jenkins.html#jenkins-as-s2i-builder[OpenShift documentation]. 22 | 23 | The main steps for creating a new image using plugins.txt and the plugins directory consist in created an imagestream, a build configuration an launching the image build. 24 | 25 | $ oc create -f openshift/perftest-jenkins-is.yaml 26 | $ oc create -f openshift/perftest-jenkins-bc.yaml 27 | $ oc start-build perftest-jenkins 28 | 29 | == Jenkins deployment 30 | 31 | The jenkins-persistent template has been extended and has the persistent volume with JMeter results mounted under /opt/performances. It can be loaded into OpenShift with: 32 | 33 | $ oc create -f openshift/perftest-jenkins-persistent-tm.yaml 34 | 35 | Jenkins can then be deployed with 36 | 37 | $ oc process jenkins-persistent NAMESPACE=perftest JENKINS_IMAGE_STREAM_TAG=perftest-jenkins:latest | oc create -f - 38 | 39 | == Jenkins pipeline 40 | 41 | OpenShift provides nice integration with Jenkins so that a build of type JenkinsPipeline can get created and is automatically loaded into Jenkins as described in https://docs.openshift.com/container-platform/3.10/dev_guide/dev_tutorials/openshift_pipeline.html[OpenShift documentation]. The following two steps deploy and run the pipeline defined in the pipelines directory of this repository. 42 | 43 | $ oc create -f openshift/perftest-pipeline-bc.yaml 44 | $ oc start-build perftest-pipeline 45 | 46 | The details of the continuous integration pipeline are available in the following chapter: 47 | 48 | * <<./pipelines/README.adoc#,Jenkins pipeline>> 49 | -------------------------------------------------------------------------------- /jenkins/openshift/perftest-jenkins-bc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: BuildConfig 3 | metadata: 4 | name: perftest-jenkins 5 | spec: 6 | source: 7 | git: 8 | uri: https://github.com/fgiloux/auto-perf-test.git 9 | contextDir: "jenkins" 10 | type: Git 11 | strategy: 12 | sourceStrategy: 13 | from: 14 | kind: ImageStreamTag 15 | name: jenkins:latest 16 | namespace: openshift 17 | type: Source 18 | output: 19 | to: 20 | kind: ImageStreamTag 21 | name: perftest-jenkins:latest 22 | 23 | 24 | -------------------------------------------------------------------------------- /jenkins/openshift/perftest-jenkins-is.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ImageStream 3 | metadata: 4 | annotations: 5 | labels: 6 | app: perftest-jenkins 7 | name: perftest-jenkins 8 | spec: {} 9 | -------------------------------------------------------------------------------- /jenkins/openshift/perftest-jenkins-persistent-tm.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Template 3 | labels: 4 | app: jenkins-persistent 5 | template: jenkins-persistent-template 6 | message: A Jenkins service has been created in your project. Log into Jenkins with 7 | your OpenShift account. The tutorial at https://github.com/openshift/origin/blob/master/examples/jenkins/README.md 8 | contains more information about using this template. 9 | metadata: 10 | annotations: 11 | description: |- 12 | Jenkins service, with persistent storage. 13 | 14 | NOTE: You must have persistent volumes available in your cluster to use this template. 15 | iconClass: icon-jenkins 16 | openshift.io/display-name: Jenkins 17 | openshift.io/documentation-url: https://docs.openshift.org/latest/using_images/other_images/jenkins.html 18 | openshift.io/long-description: This template deploys a Jenkins server capable 19 | of managing OpenShift Pipeline builds and supporting OpenShift-based oauth login. 20 | openshift.io/provider-display-name: Red Hat, Inc. 21 | openshift.io/support-url: https://access.redhat.com 22 | tags: instant-app,jenkins 23 | creationTimestamp: null 24 | name: jenkins-persistent 25 | objects: 26 | - apiVersion: v1 27 | kind: Route 28 | metadata: 29 | annotations: 30 | haproxy.router.openshift.io/timeout: 4m 31 | template.openshift.io/expose-uri: http://{.spec.host}{.spec.path} 32 | name: ${JENKINS_SERVICE_NAME} 33 | spec: 34 | tls: 35 | insecureEdgeTerminationPolicy: Redirect 36 | termination: edge 37 | to: 38 | kind: Service 39 | name: ${JENKINS_SERVICE_NAME} 40 | - apiVersion: v1 41 | kind: PersistentVolumeClaim 42 | metadata: 43 | name: ${JENKINS_SERVICE_NAME} 44 | spec: 45 | accessModes: 46 | - ReadWriteMany 47 | resources: 48 | requests: 49 | storage: ${VOLUME_CAPACITY} 50 | - apiVersion: v1 51 | kind: DeploymentConfig 52 | metadata: 53 | annotations: 54 | template.alpha.openshift.io/wait-for-ready: "true" 55 | name: ${JENKINS_SERVICE_NAME} 56 | spec: 57 | replicas: 1 58 | selector: 59 | name: ${JENKINS_SERVICE_NAME} 60 | strategy: 61 | type: Recreate 62 | template: 63 | metadata: 64 | labels: 65 | name: ${JENKINS_SERVICE_NAME} 66 | spec: 67 | containers: 68 | - capabilities: {} 69 | env: 70 | - name: OPENSHIFT_ENABLE_OAUTH 71 | value: ${ENABLE_OAUTH} 72 | - name: OPENSHIFT_ENABLE_REDIRECT_PROMPT 73 | value: "true" 74 | - name: DISABLE_ADMINISTRATIVE_MONITORS 75 | value: ${DISABLE_ADMINISTRATIVE_MONITORS} 76 | - name: KUBERNETES_MASTER 77 | value: https://kubernetes.default:443 78 | - name: KUBERNETES_TRUST_CERTIFICATES 79 | value: "true" 80 | - name: JENKINS_SERVICE_NAME 81 | value: ${JENKINS_SERVICE_NAME} 82 | - name: JNLP_SERVICE_NAME 83 | value: ${JNLP_SERVICE_NAME} 84 | image: ' ' 85 | imagePullPolicy: IfNotPresent 86 | livenessProbe: 87 | failureThreshold: 2 88 | httpGet: 89 | path: /login 90 | port: 8080 91 | initialDelaySeconds: 420 92 | periodSeconds: 360 93 | timeoutSeconds: 240 94 | name: jenkins 95 | readinessProbe: 96 | httpGet: 97 | path: /login 98 | port: 8080 99 | initialDelaySeconds: 3 100 | timeoutSeconds: 240 101 | resources: 102 | limits: 103 | memory: ${MEMORY_LIMIT} 104 | securityContext: 105 | capabilities: {} 106 | privileged: false 107 | terminationMessagePath: /dev/termination-log 108 | volumeMounts: 109 | - mountPath: /var/lib/jenkins 110 | name: ${JENKINS_SERVICE_NAME}-data 111 | - mountPath: /opt/performances 112 | name: perftest 113 | dnsPolicy: ClusterFirst 114 | restartPolicy: Always 115 | serviceAccountName: ${JENKINS_SERVICE_NAME} 116 | volumes: 117 | - name: ${JENKINS_SERVICE_NAME}-data 118 | persistentVolumeClaim: 119 | claimName: ${JENKINS_SERVICE_NAME} 120 | - name: perftest 121 | persistentVolumeClaim: 122 | claimName: apt-jmeter 123 | triggers: 124 | - imageChangeParams: 125 | automatic: true 126 | containerNames: 127 | - jenkins 128 | from: 129 | kind: ImageStreamTag 130 | name: ${JENKINS_IMAGE_STREAM_TAG} 131 | namespace: ${NAMESPACE} 132 | lastTriggeredImage: "" 133 | type: ImageChange 134 | - type: ConfigChange 135 | - apiVersion: v1 136 | kind: ServiceAccount 137 | metadata: 138 | annotations: 139 | serviceaccounts.openshift.io/oauth-redirectreference.jenkins: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"${JENKINS_SERVICE_NAME}"}}' 140 | name: ${JENKINS_SERVICE_NAME} 141 | - apiVersion: v1 142 | groupNames: null 143 | kind: RoleBinding 144 | metadata: 145 | name: ${JENKINS_SERVICE_NAME}_edit 146 | roleRef: 147 | name: edit 148 | subjects: 149 | - kind: ServiceAccount 150 | name: ${JENKINS_SERVICE_NAME} 151 | - apiVersion: v1 152 | kind: Service 153 | metadata: 154 | name: ${JNLP_SERVICE_NAME} 155 | spec: 156 | ports: 157 | - name: agent 158 | nodePort: 0 159 | port: 50000 160 | protocol: TCP 161 | targetPort: 50000 162 | selector: 163 | name: ${JENKINS_SERVICE_NAME} 164 | sessionAffinity: None 165 | type: ClusterIP 166 | - apiVersion: v1 167 | kind: Service 168 | metadata: 169 | annotations: 170 | service.alpha.openshift.io/dependencies: '[{"name": "${JNLP_SERVICE_NAME}", 171 | "namespace": "", "kind": "Service"}]' 172 | service.openshift.io/infrastructure: "true" 173 | name: ${JENKINS_SERVICE_NAME} 174 | spec: 175 | ports: 176 | - name: web 177 | nodePort: 0 178 | port: 80 179 | protocol: TCP 180 | targetPort: 8080 181 | selector: 182 | name: ${JENKINS_SERVICE_NAME} 183 | sessionAffinity: None 184 | type: ClusterIP 185 | parameters: 186 | - description: The name of the OpenShift Service exposed for the Jenkins container. 187 | displayName: Jenkins Service Name 188 | name: JENKINS_SERVICE_NAME 189 | value: jenkins 190 | - description: The name of the service used for master/slave communication. 191 | displayName: Jenkins JNLP Service Name 192 | name: JNLP_SERVICE_NAME 193 | value: jenkins-jnlp 194 | - description: Whether to enable OAuth OpenShift integration. If false, the static 195 | account 'admin' will be initialized with the password 'password'. 196 | displayName: Enable OAuth in Jenkins 197 | name: ENABLE_OAUTH 198 | value: "true" 199 | - description: Maximum amount of memory the container can use. 200 | displayName: Memory Limit 201 | name: MEMORY_LIMIT 202 | value: 512Mi 203 | - description: Volume space available for data, e.g. 512Mi, 2Gi. 204 | displayName: Volume Capacity 205 | name: VOLUME_CAPACITY 206 | required: true 207 | value: 1Gi 208 | - description: The OpenShift Namespace where the Jenkins ImageStream resides. 209 | displayName: Jenkins ImageStream Namespace 210 | name: NAMESPACE 211 | value: openshift 212 | - description: Whether to perform memory intensive, possibly slow, synchronization 213 | with the Jenkins Update Center on start. If true, the Jenkins core update monitor 214 | and site warnings monitor are disabled. 215 | displayName: Disable memory intensive administrative monitors 216 | name: DISABLE_ADMINISTRATIVE_MONITORS 217 | value: "false" 218 | - description: Name of the ImageStreamTag to be used for the Jenkins image. 219 | displayName: Jenkins ImageStreamTag 220 | name: JENKINS_IMAGE_STREAM_TAG 221 | value: jenkins:2 222 | -------------------------------------------------------------------------------- /jenkins/openshift/perftest-pipeline-bc.yaml: -------------------------------------------------------------------------------- 1 | kind: "BuildConfig" 2 | apiVersion: "v1" 3 | metadata: 4 | name: "perftest-pipeline" 5 | spec: 6 | source: 7 | git: 8 | uri: "https://github.com/fgiloux/auto-perf-test.git" 9 | contextDir: jenkins 10 | strategy: 11 | jenkinsPipelineStrategy: 12 | jenkinsfilePath: pipelines/Jenkinsfile 13 | -------------------------------------------------------------------------------- /jenkins/pipelines/Jenkinsfile: -------------------------------------------------------------------------------- 1 | // These could also be passed as parameters 2 | def testPlanPath = 'https://raw.githubusercontent.com/fgiloux/auto-perf-test/master/jmeter/openshift/apt-jmx-cm.yaml' 3 | def testPlanName = 'apt-jmx' 4 | pipeline { 5 | agent { 6 | node { 7 | label 'master' 8 | } 9 | } 10 | environment { 11 | JENKINS_URL = "http://jenkins" 12 | } 13 | // triggers { 14 | // Execute the pipeline once a day with automated distribution. 15 | // cron('H H * * *') 16 | //} 17 | options { 18 | timeout(time: 20, unit: 'MINUTES') 19 | } 20 | stages { 21 | stage('preamble') { 22 | steps { 23 | script { 24 | openshift.withCluster() { 25 | openshift.withProject() { 26 | echo "Using project: ${openshift.project()}" 27 | } 28 | } 29 | } 30 | } 31 | } 32 | // The aim of this section is to make sure that the environment 33 | // has no left over. Delete and recreate provides confidence 34 | // and reproducibility. 35 | stage('cleanup') { 36 | steps { 37 | script { 38 | openshift.withCluster() { 39 | openshift.withProject() { 40 | // Deactivated for making demos quicker 41 | /* openshift.selector("all", [ 'app' : 'camel-amq-fakeapp' ]).delete() */ 42 | // Secrets need to be addressed separately 43 | /* if (openshift.selector("secrets", secretName).exists()) { 44 | openshift.selector("secrets", secretName).delete() 45 | } */ 46 | if (openshift.selector("all", [ "testplan" : testPlanName ]).exists()) { 47 | openshift.selector("all", [ "testplan" : testPlanName ]).delete() 48 | } 49 | if (openshift.selector("configMaps", [ "testplan" : testPlanName ]).exists()) { 50 | openshift.selector("configMaps", [ "testplan" : testPlanName ]).delete() 51 | } 52 | /* openshift.selector("all", [ job-name : apt-jmeter ]).delete() */ 53 | } 54 | } 55 | } 56 | } 57 | } 58 | // Template may be used to populate the environment 59 | // Not used here 60 | /* stage('create') { 61 | steps { 62 | script { 63 | openshift.withCluster() { 64 | openshift.withProject() { 65 | openshift.newApp(templatePath) 66 | } 67 | } 68 | } 69 | } 70 | } */ 71 | // This stage is for building the application from git repository 72 | stage('build') { 73 | steps { 74 | script { 75 | openshift.withCluster() { 76 | openshift.withProject() { 77 | // Deactivated for making demos quicker 78 | /* def builds = openshift.selector("bc", 'camel-amq-fakeapp-s2i').startBuild("--wait=true") */ 79 | } 80 | } 81 | } 82 | } 83 | } 84 | // This stage is provisioning the environment with specifics 85 | // This may include credentials and configMaps with endpoint information 86 | // A new deployment is started if no image change trigger is configured 87 | // in the deployment configuration, only wait for deployment completion otherwise 88 | // Test plans and datsets to be run are also create here. 89 | stage('deploy') { 90 | steps { 91 | script { 92 | openshift.withCluster() { 93 | openshift.withProject() { 94 | // Credentials prepopulated in environment for security reason 95 | // TODO: Create separate configMaps for 96 | // - test plan 97 | // - message template 98 | // - dataset 99 | // In real life an scm plugin would be used. 100 | // def testPlanCmSource = sh(returnStdout: true, script: "curl ${testPlanPath}") 101 | def testPlanCm = openshift.create(testPlanPath).object() 102 | testPlanCm.metadata.labels['testplan'] = testPlanName 103 | openshift.apply(testPlanCm) 104 | // Deactivated for making demos quicker 105 | // - TODO: Creation of configMap with application properties 106 | /* def dc = openshift.selector("dc", 'camel-amq-fakeapp').rollout() 107 | timeout(5) { 108 | openshift.selector("dc", 'camel-amq-fakeapp').related('pods').untilEach(1) { 109 | return (it.object().status.phase == "Running") 110 | } 111 | } */ 112 | } 113 | } 114 | } 115 | } 116 | } 117 | // This stage is for test execution 118 | stage('test') { 119 | steps { 120 | script { 121 | openshift.withCluster() { 122 | openshift.withProject() { 123 | echo "Starting tests" 124 | // Register a web hook to get notified of test completion 125 | def hook = registerWebhook() 126 | def callbackUrl = hook.getURL() 127 | // Test plan is configurable 128 | // Environment variables can be passed to run the same test plan with different conditions: 129 | // duration, number of messages sent, distribution parameters 130 | def models = openshift.process("apt-jmeter-job", "-p", "JMX_CONFIGMAP=${testPlanName}","-p","RESULT_SUB_DIR=${JOB_NAME}/${BUILD_NUMBER}","-p","CALLBACK_URL=${callbackUrl}") 131 | // It is possible to modify the objects, mounting a JKS with credentials for a specic broker for instance 132 | // Just adding a label here 133 | for ( o in models ) { 134 | o.metadata.labels['testplan'] = testPlanName 135 | } 136 | def created = openshift.create(models) 137 | // Block and wait for test completion up to 10mns 138 | timeout(10) { 139 | print "Waiting for tests to complete..." 140 | waitForWebhook hook 141 | } 142 | // The test results are available on a shared drive mounted by Jenkins 143 | perfReport "/opt/performances/${JOB_NAME}/${BUILD_NUMBER}/*.jtl" 144 | // perfReport sourceDataFiles: "/opt/performances/${JOB_NAME}/${BUILD_NUMBER}/*.jtl", compareBuildPrevious: true, modePerformancePerTestCase: true, modeOfThreshold: true, relativeFailedThresholdPositive: 50, relativeUnstableThresholdNegative: 40, relativeUnstableThresholdPositive: 40 145 | // Job delete required due to the jaeger-agent sidecar not terminating 146 | openshift.selector('job', ['testplan': testPlanName]).delete() 147 | echo "Tests completed" 148 | } 149 | } 150 | } 151 | } 152 | } 153 | // After tests have been successfully executed the image may be tagged accordingly 154 | stage('tag') { 155 | steps { 156 | script { 157 | openshift.withCluster() { 158 | openshift.withProject() { 159 | openshift.tag("camel-amq-fakeapp:latest", "camel-amq-fakeapp:staging") 160 | } 161 | } 162 | } 163 | } 164 | } 165 | } 166 | } 167 | 168 | -------------------------------------------------------------------------------- /jenkins/pipelines/README.adoc: -------------------------------------------------------------------------------- 1 | = Jenkins pipelines 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | == Creation and run 16 | 17 | Loading the pipeline defined in this repository and running it on the jenkins image created <<../README.adoc#,here>> can be done with the following two steps: 18 | 19 | $ oc create -f openshift/perftest-pipeline-bc.yaml 20 | $ oc start-build perftest-pipeline 21 | 22 | == Pipeline details 23 | 24 | === Trigger 25 | 26 | Ideally performance tests would run after ever commit to trunk. They should be part of non-regression testing! It may however not be possible due to the high resource requirement or the length of the tests. In that case on schedule runs is the next best thing. 27 | 28 | Like for any build configuration a JenkinsPipeline build can be triggered by a https://docs.openshift.com/container-platform/3.10/dev_guide/builds/triggering_builds.html#webhook-triggers[webhook]. 29 | 30 | On schedule run can be configured directly in the Jenkins pipeline 31 | 32 | environment { 33 | [...] 34 | triggers { 35 | // Execute the pipeline once a day with automated distribution. 36 | cron('H H * * *') 37 | } 38 | [...] 39 | } 40 | 41 | A good practice is also to define a timeout for the execution of the pipeline 42 | 43 | options { 44 | timeout(time: 20, unit: 'MINUTES') 45 | } 46 | 47 | === Delete & recreate 48 | 49 | The approach taken in this pipeline is to delete and recreate the test environment including the application image from the project repository each time the tests are to be run. This has several advantages. 50 | 51 | * It provides confidence in what is tested is what was intended 52 | * It ensures a clean state of the environment 53 | 54 | === Execution 55 | 56 | A webook is created before the tests are run. This allows to have the pipeline waiting till it receives a call from JMeter notifying that the tests have been run until completion 57 | 58 | def hook = registerWebhook() 59 | def callbackUrl = hook.getURL() 60 | 61 | This is using the https://wiki.jenkins.io/display/JENKINS/Webhook%2BStep%2BPlugin[webook step plugin] that has been developed by one of my US colleagues, Chris Pitman. 62 | 63 | The test plan can be made configurable (number of messages sent, distribution) and environment variables can be passed to the job template being processed to run the same test plan with different conditions. The URL to the previously created webook, the name of job and build numbers are also passed as parameters. The later allows to have the test results stored in subdirectories of the network share per run. The directory structure is as such: `job_name/build_number/testPlanFileName.jtl`. 64 | For further configuration it is possible to edit the job template from the pipeline before it is processed. This can be used for mounting a secret containing a JKS with other credentials for connecting a different broker for instance. 65 | 66 | After completion the entry point script of the JMeter container calls the webhook and the pipeline can resume. } 67 | 68 | === Test results 69 | 70 | The http://jenkinsci.github.io/performance-plugin/Reporting.html[performance plugin] allows to process among others JMeter results in Jenkins. At the pipeline level it is as simple as calling perfReport. Parameters can be provided to define thresholds that may identify whether the run was successful or not and may set the build result accordingly. Besides the display of graphs of the JMeter results the plugin also allows to compare the current results with the ones of previous runs to get an idea of trends. 71 | When that's done the pipeline deletes the job. 72 | 73 | === Tagging 74 | 75 | After successful test execution the application image can be marked as ready for the next stage. 76 | 77 | stage('tag') { 78 | steps { 79 | script { 80 | openshift.withCluster() { 81 | openshift.withProject() { 82 | openshift.tag("camel-amq-fakeapp:latest", "camel-amq-fakeapp:staging") 83 | } 84 | } 85 | } 86 | } 87 | } 88 | 89 | This could also be conditioned to the test result. 90 | -------------------------------------------------------------------------------- /jenkins/plugins.txt: -------------------------------------------------------------------------------- 1 | performance:3.12 2 | webhook-step:1.3 3 | 4 | -------------------------------------------------------------------------------- /jenkins/plugins/performance.hpi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jenkins/plugins/performance.hpi -------------------------------------------------------------------------------- /jenkins/plugins/webhook-step.hpi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jenkins/plugins/webhook-step.hpi -------------------------------------------------------------------------------- /jmeter/README.adoc: -------------------------------------------------------------------------------- 1 | = JMeter overview 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | This sub-repository aims to describe the setup required for using JMeter on OpenShift, from the creation of a JMeter image to its use for running test plans in non-GUI mode. 16 | 17 | [WARNING] 18 | ==== 19 | TODO: To add test results dashboard and analysis 20 | ==== 21 | 22 | == Design time 23 | 24 | The recommended approach is to use JMeter GUI for designing your test plan. You may have installed it on your machine or you may run it on OpenShift, although this part has not yet been covered here. Running the test plan should however be done in non-GUI mode for better performance. OpenShift is best suited for this step. 25 | // To run JMeter in GUI mode it should be started with the required parameter, a route should be created so that it is externally accessible. Extraction of the jmx file should also be documented. 26 | 27 | Using JMeter GUI for the design phase provides a quick feedback loop. The test plan can then be tweaked till it is deemed ready and can be added to the continuous integration pipeline. 28 | 29 | == Automation 30 | 31 | Although the test plan can be manually run on OpenShift the idea is to get that automated by a tool for continuous integration. This will enable running non-regression tests that also cover performance aspects every time a new version is pushed to the central repository or nightly if resources are constrained during the day. We have used Jenkins for demonstrating how it can be done. 32 | 33 | See: <<../jenkins/README.adoc#,Jenkins>> 34 | 35 | == Next 36 | 37 | See the following chapters for details: 38 | 39 | * <<./container/README.adoc#,Image creation>> 40 | * <<./openshift/README.adoc#,Deployment on OpenShift>> 41 | * <<./observability/README.adoc#,Observability>> 42 | * <<./examples/README.adoc#,Test examples>> 43 | 44 | -------------------------------------------------------------------------------- /jmeter/container/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM fabric8/java-centos-openjdk8-jdk:1.4 2 | MAINTAINER Frédéric Giloux 3 | LABEL jmeter.version="4.0" 4 | 5 | # Set JMeter base 6 | ENV JMETER_BASE /opt/jmeter 7 | 8 | # Create jmeter directory with tests and results folder and install JMeter 9 | RUN mkdir -p $JMETER_BASE/{tests,results} \ 10 | && curl -SL https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-4.0.tgz \ 11 | | tar -xzC $JMETER_BASE \ 12 | && rm -rf apache-jmeter-4.0.tgz 13 | 14 | # If you don't have internet access from the build machine use COPY instead of curl: 15 | # COPY binaries/apache-jmeter-4.0.tgz $JMETER_BASE 16 | 17 | ############# ARE PLUGINS NEEDED? 18 | # COPY binaries/JMeterPlugins-ExtrasLibs-1.4.0.zip $JMETER_BASE/apache-jmeter-4.0/ 19 | 20 | # Set JMeter home 21 | ENV JMETER_HOME $JMETER_BASE/apache-jmeter-4.0 22 | 23 | # Add JMeter to the Path 24 | ENV PATH $JMETER_HOME/bin:$PATH 25 | 26 | # Additional jars (ex. AMQP driver) can be copied into $JMETER_HOME/lib 27 | COPY lib/* $JMETER_HOME/lib/ 28 | 29 | # Copy custom user.properties file for reports dashboard to be generated 30 | # COPY user.properties $JMETER_HOME/bin/user.properties 31 | 32 | # Set working directory 33 | WORKDIR $JMETER_BASE 34 | 35 | COPY scripts/run.sh $JMETER_BASE/run.sh 36 | 37 | RUN chgrp -R 0 $JMETER_BASE 38 | RUN chmod -R g+rw $JMETER_BASE 39 | RUN chmod +x $JMETER_BASE/run.sh 40 | 41 | #1001 is an arbitrary choice 42 | USER 1001 43 | 44 | # EXPOSE 8000 45 | 46 | ENTRYPOINT $JMETER_BASE/run.sh 47 | -------------------------------------------------------------------------------- /jmeter/container/README.adoc: -------------------------------------------------------------------------------- 1 | = JMeter image creation 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | 16 | Building the JMeter image is pretty straightforward. 17 | 18 | The build configuration and image stream can be generated by launching the following command from the container directory: 19 | 20 | $ oc new-build --binary=true --name=apt-jmeter 21 | 22 | A binary build using the Dockerfile and context can then be launched with 23 | 24 | $ oc start-build apt-jmeter --from-dir=. 25 | 26 | Alternatively the object definitions for build configuration and image stream available at ../openshift can be used. 27 | -------------------------------------------------------------------------------- /jmeter/container/binaries/apache-jmeter-4.0.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/binaries/apache-jmeter-4.0.zip -------------------------------------------------------------------------------- /jmeter/container/lib/artemis-jms-client-all-2.6.1.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/artemis-jms-client-all-2.6.1.jar -------------------------------------------------------------------------------- /jmeter/container/lib/gson-2.8.4.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/gson-2.8.4.jar -------------------------------------------------------------------------------- /jmeter/container/lib/jaeger-client-0.30.3.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/jaeger-client-0.30.3.jar -------------------------------------------------------------------------------- /jmeter/container/lib/jaeger-core-0.30.3.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/jaeger-core-0.30.3.jar -------------------------------------------------------------------------------- /jmeter/container/lib/jaeger-thrift-0.30.3.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/jaeger-thrift-0.30.3.jar -------------------------------------------------------------------------------- /jmeter/container/lib/jaeger-tracerresolver-0.30.3.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/jaeger-tracerresolver-0.30.3.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-api-0.31.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-api-0.31.0.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-jms-2-0.0.10.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-jms-2-0.0.10.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-jms-common-0.0.10.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-jms-common-0.0.10.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-noop-0.31.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-noop-0.31.0.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-tracerresolver-0.1.4.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-tracerresolver-0.1.4.jar -------------------------------------------------------------------------------- /jmeter/container/lib/opentracing-util-0.31.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/opentracing-util-0.31.0.jar -------------------------------------------------------------------------------- /jmeter/container/lib/proton-j-0.16.0.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/proton-j-0.16.0.jar -------------------------------------------------------------------------------- /jmeter/container/lib/qpid-jms-client-0.11.1.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/jmeter/container/lib/qpid-jms-client-0.11.1.jar -------------------------------------------------------------------------------- /jmeter/container/scripts/run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | dir=${RESULT_SUB_DIR:-$(date +"%F_%T")} 4 | # Parse JMeter parameters, replaces the J_ in J_PARAM with -J -> -JPARAM 5 | for param in $(printenv | grep J_); do 6 | JMETER_PARAMS="$JMETER_PARAMS -J${param:2}" 7 | done 8 | 9 | # Runs tests from JMX file, creates results file and reports dashboard 10 | for TEST_FILE in $JMETER_BASE/tests/*.jmx; do 11 | RESULT_FILE=/opt/jmeter/results/${dir}/$(basename $TEST_FILE .jmx).jtl 12 | $JMETER_HOME/bin/jmeter -n -t $TEST_FILE -l $RESULT_FILE $JMETER_PARAMS 13 | # Print results file 14 | cat $RESULT_FILE 15 | done 16 | # -e -o $2-reports 17 | 18 | 19 | # Call back the webhook step in Jenkins pipeline 20 | # The name of the test suite pod is provided 21 | if [ -n "${CALLBACK_URL}" ]; then 22 | # Workaround for my not working DNS resolution 23 | CALLBACK_URL="http://jenkins/webhook-step/$(echo ${CALLBACK_URL} | sed 's/.*\///')" 24 | echo " curl -X POST -d ${HOSTNAME} ${CALLBACK_URL}" 25 | curl -s -X POST -d "${HOSTNAME}" "${CALLBACK_URL}" 26 | fi 27 | 28 | # An option would be to sleep for a few minutes to give Jenkins the chance to retrieve dashboard files through oc rsync 29 | # if persistent volumes can't be used 30 | # sleep 600 31 | -------------------------------------------------------------------------------- /jmeter/examples/README.adoc: -------------------------------------------------------------------------------- 1 | = JMeter examples 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | An exemple of a test plan is provided in this directory. The plan allows to send messages to a broker (see <<../../enmasse/README.adoc#,EnMasse>>. The messages are then consumed by an application and put into an output queue. 16 | 17 | [TIP] 18 | ==== 19 | To communicate with the AMQP broker the following libraries need to be added to the lib directory for JMeter to be able to establish a connection to the AMQP broker. 20 | 21 | * org.apache.qpid proton-j-0.16.0.jar 22 | * org.apache.qpid qpid-jms-client-0.11.1.jar 23 | * artemis-jms-client-all-2.6.1.jar 24 | ==== 25 | 26 | == Design 27 | 28 | JMeter can be used in GUI mode to create or amend the test plan. 29 | 30 | [IMPORTANT] 31 | ==== 32 | Certificate location and password may be provided in the connection string. The certificate path may differ. The jks password is injected through an environment variable. This needs to be taken in consideration in the jmx file that is mounted through the configMap. Look at the provided test plan for an example. Note that the run.sh script called when the JMeter container is started transform the J_ to a start parameter with -J as expected by JMeter. For instance when the following environment variable is set: J_JKS_PWD=XXXXX JMeter gets called with -JJKS_PWD and makes possible to get the value in the test plan with ${__P(JKS_PWD)}. 33 | 34 | With jms-code.jmx the following variables need to be provided either in user.properties for GUI run or as environment variables with J_ prefix for automated run on OpenShift. 35 | 36 | * BROKER: the hostname of the AMQP broker 37 | * PORT: the port of the AMQP broker 38 | * JKS_LOCATION: the location of the truststore on the file system 39 | * JKS_PWD: the password for the truststore 40 | * PAIN_TEMPLATE_LOCATION: the name and location of the file used as template for creating messages 41 | * PAIN_SAMPLES_CSV: the name and location of the CSV file containing values to be populated in the messages 42 | * INFLUXDB_SERVER: the hostname of the InfluxDB server 43 | * INFLUXDB_PORT: the port of the InfluxDB server 44 | 45 | Additional information on InfluxDB in the <<../observability/README.adoc#,Observability>> chapter. 46 | 47 | ==== 48 | 49 | == Run 50 | 51 | Once the test plan has been designed it can be run on OpenShift. Additional information here: <<../openshift/README.adoc#,OpenShift>> 52 | 53 | //the connection string looks now like: amqps://messaging-perftest.apps.sandbox.com:443?transport.trustStoreLocation=/opt/certificates/amqp.jks&transport.trustStorePassword=${__P(JKS_PWD)}&transport.verifyHost=false 54 | -------------------------------------------------------------------------------- /jmeter/examples/jms-code.jmx: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | false 7 | false 8 | 9 | 10 | 11 | 12 | true 13 | 14 | 15 | 16 | continue 17 | 18 | false 19 | 1 20 | 21 | 1 22 | 1 23 | false 24 | 25 | 26 | 27 | 28 | 29 | ConnectionFactory 30 | input.queue 31 | output.queue 32 | 4 33 | false 34 | false 35 | 36 | 37 | 38 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 39 | 40 | 41 | 42 | 43 | connectionFactory.ConnectionFactory 44 | ${connection_string} 45 | = 46 | 47 | 48 | queue.input.queue 49 | ${__P(input_queue,input.queue)} 50 | = 51 | 52 | 53 | queue.output.queue 54 | ${__P(output_queue,output.queue)} 55 | = 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | continue 67 | 68 | false 69 | 100 70 | 71 | 1 72 | 1 73 | 1444135913000 74 | 1444135913000 75 | true 76 | 100 77 | 5 78 | 79 | 80 | 81 | , 82 | 83 | ${__P(PAIN_SAMPLES_CSV,pain_samples.csv)} 84 | false 85 | false 86 | true 87 | shareMode.group 88 | false 89 | 90 | 91 | 92 | 93 | 2 94 | 95 | throughput 96 | 600.0 97 | 0.0 98 | 99 | 100 | 101 | 102 | true 103 | 104 | 105 | import java.time.Instant 106 | import io.opentracing.contrib.jms2.TracingMessageProducer; 107 | 108 | def session = System.getProperties().get("Session") 109 | def destination = System.getProperties().get("InputDestination") 110 | 111 | def msgId = vars.get("MSG_ID") 112 | def dbtrIban = vars.get("DBTR_IBAN") 113 | def cdtrIban = vars.get("CDTR_IBAN") 114 | 115 | def msgTemplate = System.getProperties().get("MsgTemplate") 116 | def binding = [MSG_ID: msgId, DBTR_IBAN: dbtrIban, CDTR_IBAN: cdtrIban] 117 | def engine = new groovy.text.XmlTemplateEngine() 118 | def template = engine.createTemplate(msgTemplate).make(binding) 119 | def payload = template.toString() 120 | 121 | def tracer = System.getProperties().get("Tracer") 122 | def producer = new TracingMessageProducer(session.createProducer(destination), tracer); 123 | // def producer = session.createProducer(destination) 124 | def msg = session.createTextMessage(payload) 125 | 126 | def start = Instant.now() 127 | // It would be better (less specific) to use opentracing baggage rather 128 | // than directly setting the field into the JMS header 129 | msg.setLongProperty("startTime", start.toEpochMilli()) 130 | producer.send(msg) 131 | def stop = Instant.now() 132 | 133 | producer.close() 134 | 135 | vars.put("START_TIME", Long.toString(start.toEpochMilli())) 136 | SampleResult.setDataType(org.apache.jmeter.samplers.SampleResult.TEXT) 137 | SampleResult.setLatency(stop.toEpochMilli() - start.toEpochMilli()) 138 | 139 | groovy 140 | 141 | 142 | 143 | 144 | continue 145 | 146 | false 147 | 100 148 | 149 | 1 150 | 1 151 | 1444138916000 152 | 1444138916000 153 | true 154 | 100 155 | 5 156 | 157 | 158 | 159 | groovy 160 | 161 | 162 | true 163 | import java.time.Instant 164 | import javax.jms.TextMessage 165 | import groovy.util.XmlSlurper 166 | import io.opentracing.contrib.jms.common.TracingMessageConsumer; 167 | 168 | def session = System.getProperties().get("Session") 169 | def destination = System.getProperties().get("OutputDestination") 170 | 171 | def tracer = System.getProperties().get("Tracer") 172 | def consumer = new TracingMessageConsumer(session.createConsumer(destination), tracer); 173 | // def consumer = session.createConsumer(destination) 174 | 175 | def start = Instant.now() 176 | def msg = consumer.receive(2000) 177 | def stop = Instant.now() 178 | 179 | def startTime = msg.getLongProperty("startTime") 180 | def txtMsg = msg.asType(TextMessage) 181 | def txtMsgText = txtMsg.getText() 182 | log.debug("### Incoming message:" + txtMsgText + " ###") 183 | // def propertyNames = txtMsg.getPropertyNames() 184 | // while (propertyNames.hasMoreElements()) { 185 | // log.debug("### Property:" + propertyNames.nextElement() + " ###") 186 | // } 187 | def xmlMsg = new XmlSlurper().parseText(txtMsgText) 188 | def msgId = xmlMsg.CstmrCdtTrfInitn.GrpHdr.MsgId.text().trim() 189 | log.debug("### MSG_ID: " + msgId + " ###") 190 | vars.put("MSG_ID", msgId) 191 | vars.put("START_TIME", Long.toString(startTime)) 192 | def jmsTimestamp = txtMsg.getJMSTimestamp() 193 | vars.put("JMS_TIMESTAMP", Long.toString(jmsTimestamp)) 194 | consumer.close() 195 | 196 | SampleResult.setResponseData(txtMsgText) 197 | SampleResult.setDataType(org.apache.jmeter.samplers.SampleResult.TEXT) 198 | // Latency for a synchronous call (reading a message here) 199 | // SampleResult.setLatency( stop.toEpochMilli() - start.toEpochMilli() ) 200 | // End to end latency (between send & receive) 201 | SampleResult.setLatency( jmsTimestamp - startTime ) 202 | 203 | 204 | 205 | 206 | false 207 | 208 | saveConfig 209 | 210 | 211 | true 212 | true 213 | true 214 | 215 | true 216 | true 217 | true 218 | true 219 | false 220 | true 221 | true 222 | false 223 | false 224 | false 225 | false 226 | true 227 | true 228 | false 229 | false 230 | 0 231 | true 232 | true 233 | 234 | 235 | 236 | 237 | 238 | 239 | false 240 | 241 | saveConfig 242 | 243 | 244 | true 245 | true 246 | true 247 | 248 | true 249 | true 250 | true 251 | true 252 | false 253 | true 254 | true 255 | false 256 | false 257 | false 258 | false 259 | true 260 | true 261 | false 262 | false 263 | 0 264 | true 265 | true 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | connection_string 275 | amqps://${__P(BROKER,messaging-perftest.apps.sandbox.com)}:${__P(PORT,443)}?transport.trustStoreLocation=${__P(JKS_LOCATION,/home/fgiloux/Documents/projects/sandbox/auto-perf-test/camel-amq-fakeapp/src/main/resources/amqp-certs/amqp.jks)}&transport.trustStorePassword=${__P(JKS_PWD,redhat)}&transport.verifyHost=false 276 | = 277 | 278 | 279 | setupDone 280 | "false" 281 | = 282 | 283 | 284 | teardownDone 285 | "false" 286 | = 287 | 288 | 289 | influxdb_server 290 | ${__P(INFLUXDB_SERVER,influxdb-1x-perftest.apps.sandbox.com)} 291 | = 292 | 293 | 294 | influxdb_port 295 | ${__P(INFLUXDB_PORT,80)} 296 | = 297 | 298 | 299 | 300 | 301 | 302 | false 303 | 304 | saveConfig 305 | 306 | 307 | true 308 | true 309 | true 310 | 311 | true 312 | true 313 | true 314 | true 315 | false 316 | true 317 | true 318 | false 319 | false 320 | false 321 | true 322 | false 323 | false 324 | false 325 | true 326 | 0 327 | true 328 | true 329 | true 330 | true 331 | true 332 | 333 | 334 | result.jtl 335 | 336 | 337 | 338 | 339 | 340 | 341 | influxdbMetricsSender 342 | org.apache.jmeter.visualizers.backend.influxdb.HttpMetricsSender 343 | = 344 | 345 | 346 | influxdbUrl 347 | http://${influxdb_server}:${influxdb_port}/write?db=jmeter 348 | = 349 | 350 | 351 | application 352 | fakeapp 353 | = 354 | 355 | 356 | measurement 357 | jmeter 358 | = 359 | 360 | 361 | summaryOnly 362 | false 363 | = 364 | 365 | 366 | samplersRegex 367 | .* 368 | = 369 | 370 | 371 | percentiles 372 | 90;95;99 373 | = 374 | 375 | 376 | testTitle 377 | Perftest 378 | = 379 | 380 | 381 | eventTags 382 | 383 | = 384 | 385 | 386 | 387 | org.apache.jmeter.visualizers.backend.influxdb.InfluxdbBackendListenerClient 388 | 389 | 390 | 391 | stoptestnow 392 | 393 | false 394 | 1 395 | 396 | 1 397 | 1 398 | false 399 | 400 | 401 | 402 | 403 | 404 | ${setupDone}=="false" 405 | false 406 | 407 | 408 | 409 | groovy 410 | 411 | 412 | true 413 | import javax.jms.Session 414 | import org.apache.qpid.jms.JmsConnectionFactory 415 | import io.opentracing.Tracer 416 | import io.jaegertracing.Configuration 417 | 418 | // loading the message template 419 | def painTemplateLocation = props.get("PAIN_TEMPLATE_LOCATION") 420 | def msgTemplateFile = new File(painTemplateLocation) 421 | def msgTemplate = msgTemplateFile.getText('UTF-8') 422 | log.debug(msgTemplate) 423 | System.getProperties().put("MsgTemplate", msgTemplate) 424 | 425 | // Creating a tracer 426 | Tracer tracer = Configuration.fromEnv().getTracer(); 427 | 428 | // Connections and Sessions are created once per thread and reused 429 | def factory = new JmsConnectionFactory() 430 | factory.setTopicPrefix("topic://") 431 | factory.setRemoteURI(vars.get("connection_string")) 432 | 433 | def connection = factory.createQueueConnection() 434 | def session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE) 435 | def inputDestination = session.createQueue("input.queue") 436 | def outputDestination = session.createQueue("output.queue") 437 | 438 | connection.start() 439 | 440 | log.info("### Connection established! ###") 441 | 442 | System.getProperties().put("Session", session) 443 | System.getProperties().put("Connection", connection) 444 | System.getProperties().put("InputDestination", inputDestination) 445 | System.getProperties().put("OutputDestination", outputDestination) 446 | System.getProperties().put("Tracer", tracer) 447 | 448 | vars.put("setupDone", "true") 449 | 450 | 451 | 452 | 453 | 454 | stoptestnow 455 | 456 | false 457 | 1 458 | 459 | 1 460 | 1 461 | false 462 | 463 | 464 | 465 | 466 | 467 | ${teardownDone}=="false" 468 | false 469 | 470 | 471 | 472 | groovy 473 | 474 | 475 | true 476 | System.getProperties().get("Session").close() 477 | System.getProperties().get("Connection").close() 478 | 479 | log.info("### Connection closed! ###") 480 | 481 | vars.put("teardownDone", "true") 482 | 483 | 484 | 485 | 486 | 487 | 488 | 489 | -------------------------------------------------------------------------------- /jmeter/examples/jms-declarative.jmx: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | false 7 | false 8 | 9 | 10 | 11 | 12 | true 13 | 14 | 15 | 16 | continue 17 | 18 | false 19 | 1 20 | 21 | 1 22 | 1 23 | false 24 | 25 | 26 | 27 | 28 | 29 | ConnectionFactory 30 | input.queue 31 | output.queue 32 | 4 33 | false 34 | false 35 | 36 | 37 | 38 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 39 | 40 | 41 | 42 | 43 | connectionFactory.ConnectionFactory 44 | ${connection_string} 45 | = 46 | 47 | 48 | queue.input.queue 49 | ${__P(input_queue,input.queue)} 50 | = 51 | 52 | 53 | queue.output.queue 54 | ${__P(output_queue,output.queue)} 55 | = 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | continue 67 | 68 | false 69 | 50 70 | 71 | 1 72 | 1 73 | 1444135913000 74 | 1444135913000 75 | true 76 | 20 77 | 5 78 | 79 | 80 | 81 | ConnectionFactory 82 | input.queue 83 | 84 | false 85 | false 86 | 2000 87 | <?xml version="1.0" encoding="UTF-8" standalone="yes"?> 88 | <Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.001.08"> 89 | <CstmrCdtTrfInitn> 90 | <GrpHdr> 91 | <MsgId>${MSG_ID}</MsgId> 92 | <CreDtTm>2017-03-31T18:51:44.018+03:00</CreDtTm> 93 | <NbOfTxs>39</NbOfTxs> 94 | <CtrlSum>12754007436</CtrlSum> 95 | <InitgPty> 96 | <Id> 97 | <OrgId> 98 | <AnyBIC>ONPJUR9JN26</AnyBIC> 99 | <Othr> 100 | <Id>8d0bf39a-dc8a-4e38-8cdd-28eabbd79e8</Id> 101 | </Othr> 102 | </OrgId> 103 | </Id> 104 | <CtryOfRes>MY</CtryOfRes> 105 | </InitgPty> 106 | </GrpHdr> 107 | <PmtInf> 108 | <PmtInfId>0095d7e6-c081-4d11-b5bf-c75866bab4d</PmtInfId> 109 | <PmtMtd>TRF</PmtMtd> 110 | <BtchBookg>true</BtchBookg> 111 | <CtrlSum>30</CtrlSum> 112 | <ReqdExctnDt> 113 | <DtTm>2017-03-31T18:51:44.018+03:00</DtTm> 114 | </ReqdExctnDt> 115 | <Dbtr> 116 | <Nm>liquid LLC</Nm> 117 | <Id> 118 | <OrgId> 119 | <AnyBIC>UOSLKZDPS9V</AnyBIC> 120 | <Othr> 121 | <Id>06ea1ba7-4044-45ca-bce7-d6d31bf0950</Id> 122 | </Othr> 123 | </OrgId> 124 | </Id> 125 | <CtryOfRes>SG</CtryOfRes> 126 | </Dbtr> 127 | <DbtrAcct> 128 | <Id> 129 | <IBAN>${DBTR_IBAN}</IBAN> 130 | </Id> 131 | <Ccy>GBP</Ccy> 132 | <Nm>order</Nm> 133 | </DbtrAcct> 134 | <DbtrAgt> 135 | <FinInstnId> 136 | <ClrSysMmbId> 137 | <ClrSysId> 138 | <Cd>B4D</Cd> 139 | </ClrSysId> 140 | <MmbId>glass</MmbId> 141 | </ClrSysMmbId> 142 | <Nm>protest</Nm> 143 | </FinInstnId> 144 | </DbtrAgt> 145 | <DbtrAgtAcct> 146 | <Id> 147 | <IBAN>AE25166D305NJLGDX3NQNR4</IBAN> 148 | </Id> 149 | <Tp> 150 | <Prtry>week</Prtry> 151 | </Tp> 152 | </DbtrAgtAcct> 153 | <ChrgsAcct> 154 | <Id> 155 | <IBAN>GT75LMCA0Y2LKJWKX1T4ZY8PF6KF</IBAN> 156 | </Id> 157 | <Ccy>SGD</Ccy> 158 | <Nm>mother</Nm> 159 | </ChrgsAcct> 160 | <CdtTrfTxInf> 161 | <PmtId> 162 | <InstrId>b27959d3-d5ea-4ed4-8665-7515860b61e</InstrId> 163 | <EndToEndId>68008539-e8d9-47bb-a020-dae85eb88e0</EndToEndId> 164 | </PmtId> 165 | <Amt> 166 | <EqvtAmt> 167 | <Amt Ccy="RUB">10</Amt> 168 | <CcyOfTrf>SGD</CcyOfTrf> 169 | </EqvtAmt> 170 | </Amt> 171 | <CdtrAgt> 172 | <FinInstnId> 173 | <BICFI>ZTCCMM71</BICFI> 174 | </FinInstnId> 175 | </CdtrAgt> 176 | <Cdtr> 177 | <Nm>breath Bank</Nm> 178 | <Id> 179 | <OrgId> 180 | <AnyBIC>XPORFI6THXL</AnyBIC> 181 | <Othr> 182 | <Id>3fd0322a-1d9b-484f-a4c9-d0b4f447330</Id> 183 | </Othr> 184 | </OrgId> 185 | </Id> 186 | <CtryOfRes>MY</CtryOfRes> 187 | </Cdtr> 188 | <CdtrAcct> 189 | <Id> 190 | <IBAN>${CDTR_IBAN}</IBAN> 191 | </Id> 192 | <Ccy>GBP</Ccy> 193 | </CdtrAcct> 194 | </CdtTrfTxInf> 195 | </PmtInf> 196 | </CstmrCdtTrfInitn> 197 | </Document> 198 | 199 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 200 | 201 | 202 | 203 | 204 | queue.input.queue 205 | ${__P(input_queue,input.queue)} 206 | = 207 | 208 | 209 | connectionFactory.ConnectionFactory 210 | ${connection_string} 211 | = 212 | 213 | 214 | 215 | 216 | 217 | 218 | JMSCorrelationId 219 | ${MSG_ID} 220 | java.lang.String 221 | 222 | 223 | 224 | 0 225 | 226 | 227 | 228 | , 229 | 230 | pain_samples.csv 231 | false 232 | false 233 | true 234 | shareMode.group 235 | false 236 | 237 | 238 | 239 | 240 | 2 241 | 242 | throughput 243 | 600.0 244 | 0.0 245 | 246 | 247 | 248 | 249 | 250 | continue 251 | 252 | false 253 | 50 254 | 255 | 1 256 | 1 257 | 1444138916000 258 | 1444138916000 259 | true 260 | 25 261 | 5 262 | 263 | 264 | 265 | ConnectionFactory 266 | output.queue 267 | 268 | false 269 | false 270 | 10000 271 | 272 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 273 | 274 | 275 | 276 | 277 | connectionFactory.ConnectionFactory 278 | ${connection_string} 279 | = 280 | 281 | 282 | queue.output.queue 283 | ${__P(output_queue,output.queue)} 284 | = 285 | 286 | 287 | 288 | 289 | 290 | 291 | 2 292 | 293 | 294 | 295 | 296 | false 297 | 298 | saveConfig 299 | 300 | 301 | true 302 | true 303 | true 304 | 305 | true 306 | true 307 | true 308 | true 309 | false 310 | true 311 | true 312 | false 313 | false 314 | false 315 | false 316 | true 317 | true 318 | false 319 | false 320 | 0 321 | true 322 | true 323 | 324 | 325 | 326 | 327 | 328 | 329 | false 330 | 331 | saveConfig 332 | 333 | 334 | true 335 | true 336 | true 337 | 338 | true 339 | true 340 | true 341 | true 342 | false 343 | true 344 | true 345 | false 346 | false 347 | false 348 | false 349 | true 350 | true 351 | false 352 | false 353 | 0 354 | true 355 | true 356 | 357 | 358 | 359 | 360 | 361 | 362 | 363 | 364 | connection_string 365 | amqps://${__P(BROKER,messaging-perftest.apps.sandbox.com)}:${__P(PORT,443)}?transport.trustStoreLocation=${__P(JKS_LOCATION,/home/fgiloux/Documents/projects/sandbox/auto-perf-test/camel-amq-fakeapp/src/main/resources/amqp-certs/amqp.jks)}&transport.trustStorePassword=${__P(JKS_PWD,redhat)}&transport.verifyHost=false 366 | = 367 | 368 | 369 | 370 | 371 | 372 | false 373 | 374 | saveConfig 375 | 376 | 377 | true 378 | true 379 | true 380 | 381 | true 382 | true 383 | true 384 | true 385 | false 386 | true 387 | true 388 | false 389 | false 390 | false 391 | true 392 | false 393 | false 394 | false 395 | true 396 | 0 397 | true 398 | true 399 | true 400 | true 401 | true 402 | 403 | 404 | result.jtl 405 | 406 | 407 | 408 | 409 | 410 | -------------------------------------------------------------------------------- /jmeter/examples/jms-p2p-example.jmx: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | false 7 | true 8 | 9 | 10 | 11 | 12 | true 13 | 14 | 15 | 16 | continue 17 | 18 | false 19 | 1 20 | 21 | 1 22 | 1 23 | false 24 | 25 | 26 | 27 | 28 | 29 | ConnectionFactory 30 | input.queue 31 | output.queue 32 | 4 33 | false 34 | false 35 | 36 | 37 | 38 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 39 | 40 | 41 | 42 | 43 | connectionFactory.ConnectionFactory 44 | amqps://messaging-perftest.apps.sandbox.com:443?transport.trustStoreLocation=/opt/certificates/amqp.jks&transport.trustStorePassword=${__P(JKS_PWD)}&transport.verifyHost=false 45 | = 46 | 47 | 48 | queue.input.queue 49 | input.queue 50 | = 51 | 52 | 53 | queue.output.queue 54 | output.queue 55 | = 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | continue 67 | 68 | false 69 | 10 70 | 71 | 2 72 | 1 73 | 1444135913000 74 | 1444135913000 75 | true 76 | 5 77 | 5 78 | 79 | 80 | 81 | ConnectionFactory 82 | input.queue 83 | 84 | false 85 | false 86 | 2000 87 | "Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." 88 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 89 | 90 | 91 | 92 | 93 | queue.input.queue 94 | input.queue 95 | = 96 | 97 | 98 | connectionFactory.ConnectionFactory 99 | amqps://messaging-perftest.apps.sandbox.com:443?transport.trustStoreLocation=/opt/certificates/amqp.jks&transport.trustStorePassword=${__P(JKS_PWD)}&transport.verifyHost=false 100 | = 101 | 102 | 103 | 104 | 105 | 106 | 107 | 0 108 | 109 | 110 | 111 | 112 | continue 113 | 114 | false 115 | 4 116 | 117 | 5 118 | 1 119 | 1444138916000 120 | 1444138916000 121 | true 122 | 20 123 | 0 124 | 125 | 126 | 127 | ConnectionFactory 128 | input.queue 129 | output.queue 130 | false 131 | true 132 | 10000 133 | "Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." 134 | org.apache.qpid.jms.jndi.JmsInitialContextFactory 135 | 136 | 137 | 138 | 139 | connectionFactory.ConnectionFactory 140 | amqps://messaging-perftest.apps.sandbox.com:443?transport.trustStoreLocation=/opt/certificates/amqp.jks&transport.trustStorePassword=${__P(JKS_PWD)}&transport.verifyHost=false 141 | = 142 | 143 | 144 | queue.input.queue 145 | input.queue 146 | = 147 | 148 | 149 | queue.output.queue 150 | output.queue 151 | = 152 | 153 | 154 | 155 | 156 | 157 | 158 | 2 159 | true 160 | 161 | 162 | 163 | 164 | false 165 | 166 | saveConfig 167 | 168 | 169 | true 170 | true 171 | true 172 | 173 | true 174 | true 175 | true 176 | true 177 | false 178 | true 179 | true 180 | false 181 | false 182 | false 183 | false 184 | true 185 | true 186 | false 187 | false 188 | 0 189 | true 190 | true 191 | 192 | 193 | 194 | 195 | 196 | 197 | false 198 | 199 | saveConfig 200 | 201 | 202 | true 203 | true 204 | true 205 | 206 | true 207 | true 208 | true 209 | true 210 | false 211 | true 212 | true 213 | false 214 | false 215 | false 216 | false 217 | true 218 | true 219 | false 220 | false 221 | 0 222 | true 223 | true 224 | 225 | 226 | 227 | 228 | 229 | 230 | 231 | 232 | -------------------------------------------------------------------------------- /jmeter/examples/pain_samples.csv: -------------------------------------------------------------------------------- 1 | MSG_ID,DBTR_IBAN,CDTR_IBAN 2 | 00000001,LB384689BHRRC36MJSTV6P4B01VR,GB384689BHRRC01MJSTV6P4B04VR 3 | 00000002,LB384689BHRRC36MJSTV6P4B02VR,GB384689BHRRC02MJSTV6P4B04VR 4 | 00000003,LB384689BHRRC36MJSTV6P4B03VR,GB384689BHRRC03MJSTV6P4B04VR 5 | 00000004,LB384689BHRRC36MJSTV6P4B04VR,GB384689BHRRC04MJSTV6P4B04VR 6 | 00000005,LB384689BHRRC36MJSTV6P4B05VR,GB384689BHRRC05MJSTV6P4B04VR 7 | 00000006,LB384689BHRRC36MJSTV6P4B06VR,GB384689BHRRC06MJSTV6P4B04VR 8 | 00000007,LB384689BHRRC36MJSTV6P4B07VR,GB384689BHRRC07MJSTV6P4B04VR 9 | 00000008,LB384689BHRRC36MJSTV6P4B08VR,GB384689BHRRC08MJSTV6P4B04VR 10 | 00000009,LB384689BHRRC36MJSTV6P4B09VR,GB384689BHRRC09MJSTV6P4B04VR 11 | 00000010,LB384689BHRRC36MJSTV6P4B10VR,GB384689BHRRC10MJSTV6P4B04VR 12 | -------------------------------------------------------------------------------- /jmeter/examples/pain_template.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | ${MSG_ID} 6 | 2017-03-31T18:51:44.018+03:00 7 | 39 8 | 12754007436 9 | 10 | 11 | 12 | ONPJUR9JN26 13 | 14 | 8d0bf39a-dc8a-4e38-8cdd-28eabbd79e8 15 | 16 | 17 | 18 | MY 19 | 20 | 21 | 22 | 0095d7e6-c081-4d11-b5bf-c75866bab4d 23 | TRF 24 | true 25 | 30 26 | 27 | 2017-03-31T18:51:44.018+03:00 28 | 29 | 30 | liquid LLC 31 | 32 | 33 | UOSLKZDPS9V 34 | 35 | 06ea1ba7-4044-45ca-bce7-d6d31bf0950 36 | 37 | 38 | 39 | SG 40 | 41 | 42 | 43 | ${DBTR_IBAN} 44 | 45 | GBP 46 | order 47 | 48 | 49 | 50 | 51 | 52 | B4D 53 | 54 | glass 55 | 56 | protest 57 | 58 | 59 | 60 | 61 | AE25166D305NJLGDX3NQNR4 62 | 63 | 64 | week 65 | 66 | 67 | 68 | 69 | GT75LMCA0Y2LKJWKX1T4ZY8PF6KF 70 | 71 | SGD 72 | mother 73 | 74 | 75 | 76 | b27959d3-d5ea-4ed4-8665-7515860b61e 77 | 68008539-e8d9-47bb-a020-dae85eb88e0 78 | 79 | 80 | 81 | 10 82 | SGD 83 | 84 | 85 | 86 | 87 | ZTCCMM71 88 | 89 | 90 | 91 | breath Bank 92 | 93 | 94 | XPORFI6THXL 95 | 96 | 3fd0322a-1d9b-484f-a4c9-d0b4f447330 97 | 98 | 99 | 100 | MY 101 | 102 | 103 | 104 | ${CDTR_IBAN} 105 | 106 | GBP 107 | 108 | 109 | 110 | 111 | 112 | 113 | -------------------------------------------------------------------------------- /jmeter/observability/README.adoc: -------------------------------------------------------------------------------- 1 | = JMeter observability 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | == InfluxDB 16 | 17 | Test results are stored in JTL files. It is also possible to have JMeter storing them in an InfluDB database, which can then be queried from Grafana. 18 | 19 | InfluxDB partner image can be used for deploying it on OpenShift 20 | 21 | $ ooc import-image my-influxdata/influxdb-1x --from=registry.connect.redhat.com/influxdata/influxdb-1x --confirm 22 | 23 | You may however need to first create credentials for accessing the registry if you haven't done it yet. The instructions https://docs.openshift.com/container-platform/3.10/dev_guide/managing_images.html#allowing-pods-to-reference-images-from-other-secured-registries[here] can be followed for this purpose. Copied bellow for convenience: 24 | 25 | $ docker login registry.connect.redhat.com 26 | $ oc create secret generic connect-registry \ 27 | --from-file=.dockerconfigjson=~/.docker/config.json \ 28 | --type=kubernetes.io/dockerconfigjson 29 | 30 | Use .dockercfg instead of .docker/config.json for older clients. 31 | 32 | $ oc secrets link default connect-registry --for=pull 33 | 34 | The next step is to deploy InfluxDB on OpenShift. 35 | This can be done through the web console by clicking "Add to project", choosing "Deploy Image" and then selecting the project name, influxdb-1x and latest. Alternatively `$ oc new-app influxdb-1x` can be used for the same purpose. 36 | 37 | A service influxdb-1x has been created. A route can be created to connect to it from a local JMeter 38 | 39 | $ oc expose service influxdb-1x 40 | 41 | Now that InfluxDB is available and accessible a database instance is to be created for JMeter. 42 | 43 | Use `oc rsh` to log into the container and run the following commands. 44 | 45 | $ influx 46 | $ show databases 47 | $ CREATE DATABASE jmeter 48 | $ show databases 49 | 50 | You should see your newly added jmeter database. 51 | 52 | To connect to InfluxDB from JMeter user defined variables have been configured in the JMeter test plan: 53 | 54 | influxdb_server: ${__P(INFLUXDB_SERVER,influxdb-1x-perftest.apps.sandbox.com)} 55 | influxdb_port: ${__P(INFLUXDB_PORT,80)} 56 | 57 | where influxdb-1x-perftest.apps.sandbox.com is my route, which gets used as default when the JMeter system property is not set. 58 | 59 | The JMeter pod will connect to InfluxDB directly through the service and the environment variables have been set as follows 60 | 61 | J_INFLUXDB_SERVER=influxdb-1x 62 | J_INFLUXDB_PORT=8086 63 | 64 | Finally an InfluxDB backend listener has been added to the test plan. Look at the jms-code.jmx example for further details. 65 | 66 | To check that data gets populated when the test plan is run you can log into the InfluxDB container again and run: 67 | 68 | $ use jmeter 69 | $ select * from jmeter 70 | 71 | *Alternative* 72 | 73 | There is a https://github.com/johrstrom/jmeter-prometheus-plugin[plugin] allowing the JMeter results to be exported to Prometheus instead of InfluxDB. This has not been further investigated as part of this demo as InfluxDB seems to be a good match and has broader adoption for this use case. 74 | 75 | == Grafana 76 | 77 | Now that we have data available we need something to get value out of it. Grafana is best for that. 78 | 79 | The installation of Grafana is covered <<../../observability/grafana/README.adoc#here>>. 80 | 81 | In Grafana we need first to configure a datasource pointing to our InfluxDB instance. This can be done by setting the following values. 82 | 83 | URL: http://influxdb-1x.perftest.svc:8086 84 | Access: Server/proxy (named differently depending on the Grafana version) 85 | No authentication 86 | Database: jmeter 87 | 88 | The last step is to create a dashboard. Therefore an example is available under observability/grafana. 89 | 90 | WARNING: 1. The current version of the JMeter test plan calculates the latency by substracting the timestamp set by the application sender to the time the initial message was created by the JMeter thread. This provides more meaningful information when components are chained through a messaging layer. The default JMeter calculation would just report the time required to put and read messages from the broker. The time of creation for the first message is passed directly into the JMS headers. It would be a better solution to use the baggage mechanism in OpenTracing for this mechanism as it would also work when other transports than messaging are taking part in the chain. 91 | 92 | WARNING: 2. The JMS timestamp is used as best approximation of when the message has been delivered to the broker. It is actually the time the message was handed over to the producer. This may be quite different when transactions or other mechanisms are used that batch messages. 93 | 94 | // ActiveMQ (classic) has a plugin to set broker timestamp but it is not recommended for high volume or low latency. 95 | 96 | NOTE: TODO: A JSR223 sampler could be used instead of the backend plugin to write into InfluxDB. This would provide flexibility on the information, which is stored and available for reporting. 97 | 98 | == OpenTracing 99 | 100 | OpenTracing has been deployed and configured for reporting on application performance as described here <<../../observability/opentracing/README.adoc#here>>. 101 | It makes however sense to add JMeter to the traces as it provides information on how long messages have been enqueued waiting for being picked up and processed by the application. 102 | 103 | <<../examples/README.adoc#JMeter example>> jms-code has been extended for this purpose. OpenTracing API, Jaeger and https://github.com/opentracing-contrib/java-jms[java-jms] libraries have been added to the container through the lib directory. The JMeter job template has also been extended to include a sidecar container with the Jaeger agent. It has the unfortunate side effect that the job does not terminate (as the agent does not terminate) at the end of the JMeter test. The Jenkins pipeline takes care of this by deleting the job at the end. 104 | 105 | For running the JMeter tests locally the libraries need to be added to the local JMeter installation for running JMeter. For collecting traces sent by the local JMeter instance a local Jaeger collector would need to be set up and environment variables configured. 106 | 107 | $ export JAEGER_ENDPOINT=http://jaeger-collector-perftest.apps.sandbox.com/api/traces 108 | $ export JAEGER_TAGS="scope=pertest,span.kind=jmeter" 109 | $ export JAEGER_SAMPLER_PARAM=1 110 | $ export JAEGER_SERVICE_NAME=jmeter 111 | 112 | A route may need to be created for the collector and set as JAEGER_ENDPOINT. 113 | 114 | == Observability of JMeter 115 | 116 | It is important to monitor the behavior of JMeter itself. To provide meaningful results it should not be the bottleneck. The good thing with having it running on OpenShift is that it is easy to scale it vertically or horizontally. 117 | 118 | This is not covered here but it may make sense to monitor JMeter using Prometheus to make sure that the JMeter instances were not over heated during the tests. As JMeter instances may come and go when the tests are launched and terminate (that's a good thing for freeing up resources) a Prometheus https://github.com/prometheus/pushgateway[pushgateway] may be used for metrics collection. 119 | 120 | If Jmeter were to run over capacity your tests may suffer from http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html[coordinated omission]. 121 | This demo aims at validating application (not broker) performance/behavior and with this in mind I am making the following assumptions: 122 | 123 | * JMeter is able to produce the load requested (confer the scalability statement above) 124 | * The broker is able to ingest the messages in a timely fashion (not blocking JMeter) and preventing the point above. 125 | * The latency introduced by the broker is to be considered together with the application. Tuning the way the messages are fetches by the application can impact it significantly. 126 | 127 | == Jenkins 128 | 129 | The test results can be graphed, displayed and interpreted (FAIL OR PASS) in Jenkins thanks to the performance plugin. 130 | This is described <<../../jenkins/pipelines/README.adoc#here>>. 131 | -------------------------------------------------------------------------------- /jmeter/openshift/README.adoc: -------------------------------------------------------------------------------- 1 | = JMeter deployment on OpenShift 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | 16 | After <<../container/README.adoc#,a JMeter image>> has been created you can run your test plan in a container. Therefore <<../examples/README.adoc#,an example>> has been provided, which is used for demonstrating the approach. 17 | The few generic steps are described here. The ones specific to the test plan provided as example are described there. 18 | 19 | == Test plan 20 | 21 | A configMap is used for injecting the test plan into the image. From the directory where your jmx file is located you can create the configMap with a single command. 22 | 23 | $ oc create configmap apt-jmx --from-file=jms-code.jmx --from-file=pain_samples.csv --from-file=pain_template.xml 24 | 25 | A label can also be added to the config 26 | 27 | $ oc label configMap apt-jmx job-name=apt-jmeter 28 | 29 | This configMap has been exported and stored for reuse by Jenkins (it could also be recreated each time from files) 30 | 31 | $ oc export configMap apt-jmx > ../openshift/apt-jmx-cm.yaml 32 | 33 | == JMeter properties 34 | 35 | https://jmeter.apache.org/usermanual/properties_reference.html[Prorperties] allow to customize JMeter. Interesting for our purpose is for instance the possibility to customize the result file. They are two ways of setting properties, the first one is to inject a user.properties file. The second one is to pass them as command line argument with -J, example: -Jproperty_example=property_value. The values then overwrite what has been defined in jmeter.properties or reportgenerator.properties. To facilitate this second approach the run script transforms any environment variable with the J_ prefix to a command line argument. For instance the environment variable J_myprop will be passed as the parameter -Jmyprop. 36 | 37 | == Certificates and credentials 38 | 39 | Very similar to the test plan certificates can be injected into the container. A secret may be used for this purpose rather than a configMap. A java keystore containing the broker certificates has been created in the <<../../enmasse/README.adoc#,EnMasse section>>. A password is also necessary to access the keystore and may be provided as environment variable. Both can be injected with a single secret running the following command in the same directory that was used for the creation of the java keystore. 40 | 41 | $ oc create secret generic broker-jks --from-literal=J_JKS_PWD=xxxxxx --from-file=./amqp-certs/amqp.jks 42 | 43 | == Run 44 | 45 | As we want our test plan to be run and terminate at the end a job is better suited for the purpose than a deployment configuration. A template containing the job definition may also be used to pass parameters. An example is available in this directory. The template can be processed with this command to create a new job. 46 | 47 | $ oc process -f apt-jmeter-job-tm.yaml | oc create -f - 48 | 49 | If you want to replace it with a new version you can use the follwing. 50 | 51 | $ oc process -f apt-jmeter-job-tm.yaml | oc replace --force -f - 52 | 53 | It is possible to pass the namespace and name of configMap with the JMeter test suite as parameter. 54 | 55 | It may also be useful to first create the template in OpenShift, so that it can easily be called from Jenkins for instance: 56 | 57 | $ oc create -f apt-jmeter-job-tm.yaml 58 | 59 | Another template apt-jmeter-job-persistent-tm.yaml allows to mount a persistent volume instead of emptyDir for storing the test results. An additional parameter allows the specification of the PVC name. 60 | This is useful when the test results are to be read after the job has terminated, from Jenkins for instance. 61 | -------------------------------------------------------------------------------- /jmeter/openshift/apt-jmeter-bc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: BuildConfig 3 | metadata: 4 | annotations: 5 | openshift.io/generated-by: OpenShiftNewApp 6 | creationTimestamp: null 7 | labels: 8 | app: apt-jmeter 9 | name: apt-jmeter 10 | spec: 11 | failedBuildsHistoryLimit: 5 12 | nodeSelector: null 13 | output: 14 | to: 15 | kind: ImageStreamTag 16 | name: apt-jmeter:latest 17 | postCommit: {} 18 | resources: {} 19 | runPolicy: Serial 20 | source: 21 | git: 22 | ref: master 23 | uri: https://github.com/fgiloux/auto-perf-test.git 24 | contextDir: jmeter/container 25 | type: Git 26 | strategy: 27 | dockerStrategy: 28 | from: 29 | kind: ImageStreamTag 30 | name: java-centos-openjdk8-jdk:1.4 31 | type: Docker 32 | successfulBuildsHistoryLimit: 5 33 | triggers: 34 | - github: 35 | secret: R5N6fqeAGHmIx6db7Vhx 36 | type: GitHub 37 | - generic: 38 | secret: TxtCDPVPl-R_4todBNBb 39 | type: Generic 40 | - type: ConfigChange 41 | - imageChange: {} 42 | type: ImageChange 43 | status: 44 | lastVersion: 0 45 | -------------------------------------------------------------------------------- /jmeter/openshift/apt-jmeter-is.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ImageStream 3 | metadata: 4 | annotations: 5 | openshift.io/generated-by: OpenShiftNewApp 6 | creationTimestamp: null 7 | generation: 1 8 | labels: 9 | app: apt-jmeter 10 | name: apt-jmeter 11 | spec: 12 | lookupPolicy: 13 | local: false 14 | status: 15 | dockerImageRepository: "" 16 | -------------------------------------------------------------------------------- /jmeter/openshift/apt-jmeter-job-persistent-tm.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: template.openshift.io/v1 2 | kind: Template 3 | metadata: 4 | labels: 5 | app: apt-jmeter 6 | name: apt-jmeter-job 7 | objects: 8 | - apiVersion: batch/v1 9 | kind: Job 10 | metadata: 11 | labels: 12 | app: apt-jmeter 13 | name: apt-jmeter 14 | spec: 15 | parallelism: 1 16 | completions: 1 17 | template: 18 | metadata: 19 | name: apt-jmeter 20 | labels: 21 | testplan: apt-jmx 22 | spec: 23 | containers: 24 | - name: jmeter 25 | image: docker-registry.default.svc:5000/${NAMESPACE}/apt-jmeter:latest 26 | resources: 27 | requests: 28 | cpu: '500m' 29 | memory: 1Gi 30 | env: 31 | - name: J_JKS_PWD 32 | valueFrom: 33 | secretKeyRef: 34 | name: ${JKS_SECRET} 35 | key: J_JKS_PWD 36 | - name: J_JKS_LOCATION 37 | value: ${J_JKS_LOCATION} 38 | - name: J_PAIN_TEMPLATE_LOCATION 39 | value: /opt/jmeter/tests/pain_template.xml 40 | - name: J_PAIN_CSV_LOCATION 41 | value: /opt/jmeter/tests/pain_samples.csv 42 | - name: J_sample_variables 43 | value: "MSG_ID,JMS_TIMESTAMP" 44 | - name: J_BROKER 45 | value: messaging 46 | - name: J_PORT 47 | value: 5671 48 | - name: J_INFLUXDB_SERVER 49 | value: 'influxdb-1x' 50 | - name: J_INFLUXDB_PORT 51 | value: '8086' 52 | - name: JAEGER_SERVICE_NAME 53 | value: jmeter 54 | - name: JAEGER_SAMPLER_PARAM 55 | value: '1' 56 | - name: JAEGER_PROPAGATION 57 | value: 'jaeger,b3' 58 | - name: RESULT_SUB_DIR 59 | value: ${RESULT_SUB_DIR} 60 | - name: CALLBACK_URL 61 | value: ${CALLBACK_URL} 62 | volumeMounts: 63 | - mountPath: /opt/jmeter/tests 64 | name: tests 65 | - mountPath: /opt/jmeter/results 66 | name: results 67 | - mountPath: /opt/certificates 68 | name: jks 69 | - args: 70 | - '--collector.host-port=jaeger-collector.perftest.svc:14267' 71 | image: jaegertracing/jaeger-agent 72 | imagePullPolicy: Always 73 | name: jaeger-agent 74 | ports: 75 | - containerPort: 5775 76 | protocol: UDP 77 | - containerPort: 5778 78 | protocol: TCP 79 | - containerPort: 6831 80 | protocol: UDP 81 | - containerPort: 6832 82 | protocol: UDP 83 | resources: {} 84 | terminationMessagePath: /dev/termination-log 85 | terminationMessagePolicy: File 86 | restartPolicy: OnFailure 87 | volumes: 88 | - name: "results" 89 | persistentVolumeClaim: 90 | claimName: ${JTL_PVC} 91 | - configMap: 92 | defaultMode: 420 93 | name: ${JMX_CONFIGMAP} 94 | name: tests 95 | - name: jks 96 | secret: 97 | defaultMode: 420 98 | secretName: ${JKS_SECRET} 99 | parameters: 100 | - description: Namespace/project the job is created in 101 | name: NAMESPACE 102 | value: perftest 103 | - description: Name of the ConfigMap containing the test(s) to run 104 | name: JMX_CONFIGMAP 105 | value: apt-jmx 106 | - description: Name of the secret containing the jks 107 | name: JKS_SECRET 108 | value: broker-jks 109 | - description: Jks location 110 | name: J_JKS_LOCATION 111 | value: /opt/certificates/amqp.jks 112 | - description: Name of the persistent volume claim to use for storing the test(s) results 113 | name: JTL_PVC 114 | value: apt-jmeter 115 | - description: Name of the subdirectory where test results are to be stored 116 | name: RESULT_SUB_DIR 117 | - description: Url for notification of test completion 118 | name: CALLBACK_URL 119 | -------------------------------------------------------------------------------- /jmeter/openshift/apt-jmeter-job-tm.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: template.openshift.io/v1 2 | kind: Template 3 | metadata: 4 | labels: 5 | app: apt-jmeter 6 | name: apt-jmeter-job 7 | objects: 8 | - apiVersion: batch/v1 9 | kind: Job 10 | metadata: 11 | labels: 12 | app: apt-jmeter 13 | name: apt-jmeter 14 | spec: 15 | parallelism: 1 16 | completions: 1 17 | template: 18 | metadata: 19 | name: apt-jmeter 20 | labels: 21 | testplan: apt-jmx 22 | spec: 23 | containers: 24 | - name: jmeter 25 | image: docker-registry.default.svc:5000/${NAMESPACE}/apt-jmeter:latest 26 | resources: 27 | requests: 28 | cpu: '500m' 29 | memory: 1Gi 30 | env: 31 | - name: J_JKS_PWD 32 | valueFrom: 33 | secretKeyRef: 34 | name: ${JKS_SECRET} 35 | key: J_JKS_PWD 36 | - name: J_JKS_LOCATION 37 | value: ${J_JKS_LOCATION} 38 | - name: J_PAIN_TEMPLATE_LOCATION 39 | value: /opt/jmeter/tests/pain_template.xml 40 | - name: J_PAIN_CSV_LOCATION 41 | value: /opt/jmeter/tests/pain_samples.csv 42 | - name: J_sample_variables 43 | value: "MSG_ID,JMS_TIMESTAMP" 44 | - name: J_INFLUXDB_SERVER 45 | value: 'influxdb-1x' 46 | - name: J_INFLUXDB_PORT 47 | value: '8086' 48 | - name: J_BROKER 49 | value: 'messaging' 50 | - name: J_PORT 51 | value: 5671 52 | - name: JAEGER_SERVICE_NAME 53 | value: jmeter 54 | - name: JAEGER_SAMPLER_PARAM 55 | value: '1' 56 | - name: JAEGER_PROPAGATION 57 | value: 'jaeger,b3' 58 | - name: CALLBACK_URL 59 | value: ${CALLBACK_URL} 60 | volumeMounts: 61 | - mountPath: /opt/jmeter/tests 62 | name: tests 63 | - mountPath: /opt/jmeter/results 64 | name: results 65 | - mountPath: /opt/certificates 66 | name: jks 67 | - args: 68 | - '--collector.host-port=jaeger-collector.perftest.svc:14267' 69 | image: jaegertracing/jaeger-agent 70 | imagePullPolicy: Always 71 | name: jaeger-agent 72 | ports: 73 | - containerPort: 5775 74 | protocol: UDP 75 | - containerPort: 5778 76 | protocol: TCP 77 | - containerPort: 6831 78 | protocol: UDP 79 | - containerPort: 6832 80 | protocol: UDP 81 | resources: {} 82 | terminationMessagePath: /dev/termination-log 83 | terminationMessagePolicy: File 84 | restartPolicy: OnFailure 85 | volumes: 86 | - name: results 87 | emptyDir: {} 88 | - configMap: 89 | defaultMode: 420 90 | name: ${JMX_CONFIGMAP} 91 | name: tests 92 | - name: jks 93 | secret: 94 | defaultMode: 420 95 | secretName: ${JKS_SECRET} 96 | parameters: 97 | - description: Namespace/project the job is created in 98 | name: NAMESPACE 99 | value: perftest 100 | - description: Name of the ConfigMap containing the test(s) to run 101 | name: JMX_CONFIGMAP 102 | value: apt-jmx 103 | - description: Name of the secret containing the jks 104 | name: JKS_SECRET 105 | value: broker-jks 106 | - description: JKS location 107 | name: J_JKS_LOCATION 108 | value: /opt/certificates/amqp.jks 109 | - description: Url for notification of test completion 110 | name: CALLBACK_URL 111 | -------------------------------------------------------------------------------- /jmeter/openshift/apt-jmeter-pvc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | annotations: 5 | labels: 6 | app: jmeter 7 | name: apt-jmeter 8 | name: apt-jmeter 9 | spec: 10 | accessModes: 11 | - ReadWriteMany 12 | resources: 13 | requests: 14 | storage: 10Gi 15 | 16 | -------------------------------------------------------------------------------- /jmeter/openshift/influxdb-dc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: DeploymentConfig 3 | metadata: 4 | annotations: 5 | openshift.io/generated-by: OpenShiftWebConsole 6 | creationTimestamp: null 7 | generation: 1 8 | labels: 9 | app: influxdb-1x 10 | name: influxdb-1x 11 | spec: 12 | replicas: 1 13 | selector: 14 | app: influxdb-1x 15 | deploymentconfig: influxdb-1x 16 | strategy: 17 | activeDeadlineSeconds: 21600 18 | resources: {} 19 | rollingParams: 20 | intervalSeconds: 1 21 | maxSurge: 25% 22 | maxUnavailable: 25% 23 | timeoutSeconds: 600 24 | updatePeriodSeconds: 1 25 | type: Rolling 26 | template: 27 | metadata: 28 | annotations: 29 | openshift.io/generated-by: OpenShiftWebConsole 30 | creationTimestamp: null 31 | labels: 32 | app: influxdb-1x 33 | deploymentconfig: influxdb-1x 34 | spec: 35 | containers: 36 | - image: registry.connect.redhat.com/influxdata/influxdb-1x@sha256:91b145170df87e189c2f2c9d9983fe6128bee3922dc30438173ea05c210525af 37 | imagePullPolicy: IfNotPresent 38 | name: influxdb-1x 39 | ports: 40 | - containerPort: 8086 41 | protocol: TCP 42 | resources: {} 43 | terminationMessagePath: /dev/termination-log 44 | terminationMessagePolicy: File 45 | volumeMounts: 46 | - mountPath: /var/lib/influxdb 47 | name: influxdb-1x-1 48 | dnsPolicy: ClusterFirst 49 | restartPolicy: Always 50 | schedulerName: default-scheduler 51 | securityContext: {} 52 | terminationGracePeriodSeconds: 30 53 | volumes: 54 | - emptyDir: {} 55 | name: influxdb-1x-1 56 | test: false 57 | triggers: 58 | - type: ConfigChange 59 | - imageChangeParams: 60 | automatic: true 61 | containerNames: 62 | - influxdb-1x 63 | from: 64 | kind: ImageStreamTag 65 | name: influxdb-1x:latest 66 | namespace: perftest 67 | type: ImageChange 68 | 69 | -------------------------------------------------------------------------------- /jmeter/openshift/influxdb-persistent-dc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: DeploymentConfig 3 | metadata: 4 | annotations: 5 | openshift.io/generated-by: OpenShiftWebConsole 6 | creationTimestamp: null 7 | generation: 1 8 | labels: 9 | app: influxdb-1x 10 | name: influxdb-1x 11 | spec: 12 | replicas: 1 13 | selector: 14 | app: influxdb-1x 15 | deploymentconfig: influxdb-1x 16 | strategy: 17 | activeDeadlineSeconds: 21600 18 | resources: {} 19 | rollingParams: 20 | intervalSeconds: 1 21 | maxSurge: 25% 22 | maxUnavailable: 25% 23 | timeoutSeconds: 600 24 | updatePeriodSeconds: 1 25 | type: Rolling 26 | template: 27 | metadata: 28 | annotations: 29 | openshift.io/generated-by: OpenShiftWebConsole 30 | creationTimestamp: null 31 | labels: 32 | app: influxdb-1x 33 | deploymentconfig: influxdb-1x 34 | spec: 35 | containers: 36 | - image: registry.connect.redhat.com/influxdata/influxdb-1x@sha256:91b145170df87e189c2f2c9d9983fe6128bee3922dc30438173ea05c210525af 37 | imagePullPolicy: IfNotPresent 38 | name: influxdb-1x 39 | ports: 40 | - containerPort: 8086 41 | protocol: TCP 42 | resources: {} 43 | terminationMessagePath: /dev/termination-log 44 | terminationMessagePolicy: File 45 | volumeMounts: 46 | - mountPath: /var/lib/influxdb 47 | name: influxdb-1x-1 48 | dnsPolicy: ClusterFirst 49 | restartPolicy: Always 50 | schedulerName: default-scheduler 51 | securityContext: {} 52 | terminationGracePeriodSeconds: 30 53 | volumes: 54 | - name: "influxdb-1x-1" 55 | persistentVolumeClaim: 56 | claimName: influxdb 57 | test: false 58 | triggers: 59 | - type: ConfigChange 60 | - imageChangeParams: 61 | automatic: true 62 | containerNames: 63 | - influxdb-1x 64 | from: 65 | kind: ImageStreamTag 66 | name: influxdb-1x:latest 67 | namespace: perftest 68 | type: ImageChange 69 | 70 | -------------------------------------------------------------------------------- /jmeter/openshift/influxdb-pvc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: PersistentVolumeClaim 3 | metadata: 4 | annotations: 5 | labels: 6 | app: jmeter 7 | name: influxdb 8 | name: influxdb 9 | spec: 10 | accessModes: 11 | - ReadWriteOnce 12 | resources: 13 | requests: 14 | storage: 11Gi 15 | 16 | -------------------------------------------------------------------------------- /jmeter/openshift/influxdb-srv.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | annotations: 5 | labels: 6 | app: influxdb-1x 7 | name: influxdb-1x 8 | namespace: perftest 9 | spec: 10 | ports: 11 | - name: 8086-tcp 12 | port: 8086 13 | protocol: TCP 14 | targetPort: 8086 15 | selector: 16 | deploymentconfig: influxdb-1x 17 | sessionAffinity: None 18 | type: ClusterIP 19 | 20 | -------------------------------------------------------------------------------- /jmeter/openshift/java-centos-openjdk8-jdk-is.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: ImageStream 3 | metadata: 4 | annotations: 5 | openshift.io/generated-by: OpenShiftNewApp 6 | openshift.io/image.dockerRepositoryCheck: 2018-07-20T09:40:15Z 7 | creationTimestamp: null 8 | generation: 2 9 | labels: 10 | app: apt-jmeter 11 | name: java-centos-openjdk8-jdk 12 | spec: 13 | lookupPolicy: 14 | local: false 15 | tags: 16 | - annotations: 17 | openshift.io/imported-from: fabric8/java-centos-openjdk8-jdk:1.4 18 | from: 19 | kind: DockerImage 20 | name: docker-registry.default.svc:5000/perftest/java-centos-openjdk8-jdk:1.4 21 | generation: 2 22 | importPolicy: {} 23 | name: "1.4" 24 | referencePolicy: 25 | type: Source 26 | status: 27 | dockerImageRepository: "" 28 | -------------------------------------------------------------------------------- /observability/README.adoc: -------------------------------------------------------------------------------- 1 | = Observability 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | Modern applications are designed with observability in mind. The infrastructure put in place for monitoring the application behaviour is highly valuable during non functional testing, especially performance tests and should be leveraged. 16 | The way application metrics and performance measures can be gathered is described in the following chapters. 17 | 18 | * <<./prometheus/README.adoc#,Monitoring>> 19 | * <<./opentracing/README.adoc#,Application performance management>> 20 | * <<./grafana/README.adoc#,Dashboards>> 21 | 22 | 23 | 24 | -------------------------------------------------------------------------------- /observability/grafana/README.adoc: -------------------------------------------------------------------------------- 1 | = Grafana 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | == Installation 16 | 17 | A playbook is available for deploying Grafana on the machine used for the OpenShift installation: 18 | 19 | $ ansible-playbook -vvv -i your.inventory /usr/share/ansible/openshift-ansible/playbooks/openshift-grafana/config.yml 20 | 21 | Alternatively, the ansible playbook broker can be used for installinng Grafana together with Prometheus, see https://github.com/ansibleplaybookbundle/prometheus-apb[here]. 22 | 23 | NOTE: Things are evolving quickly in this realm and an operator, configurable during the ansible installation is available with OpenShift 3.11 for installing and managing Prometheus & Grafana. 24 | 25 | == Data sources 26 | 27 | Once you have logged into Grafana the first thing is to configure the datasources. 28 | 29 | === Prometheus 30 | 31 | Select the Grafana icon menu on the top left, click on "Data Sources" and the "Add Data Source" button. A few fields need then to be filled in. 32 | 33 | .... 34 | Name: A name needs to be given to the data source I chose DS_PROMETHEUS 35 | Type: Prometheus 36 | URL: https://prometheus-openshift-metrics.apps.sandbox.com/ (change to your Prometheus route) 37 | Access: Server/proxy (named differently depending on the Grafana version) 38 | Skip TLS Verification (Insecure): If you don't have a CA configured 39 | Token (under "Prometheus settings"): set to the result of: `$ oc serviceaccounts get-token prometheus-reader -n openshift-metrics` 40 | .... 41 | 42 | === InfluxDB 43 | 44 | The configuration of the InfluxDB is covered under <<../../jmeter/observability/README.adoc#,JMeter>>. The steps are repeated here for convenience. 45 | 46 | Select the Grafana icon menu on the top left, click on "Data Sources" and the "Add Data Source" button. A few fields need then to be filled in. 47 | 48 | .... 49 | Name: A name needs to be given to the data source I chose DS_INFLUXDB 50 | Type: InfluxDB 51 | URL: http://influxdb-1x.perftest.svc:8086 (change to your InfluxDB service) 52 | Access: Server/proxy (named differently depending on the Grafana version) 53 | No authentication was configured in this demo 54 | Database: jmeter 55 | .... 56 | 57 | == Dashboards 58 | 59 | Prometheus metrics are scraped every 30s and can be used along the time series that have been recorded into InfluxDB for the JMeter runs. 60 | A sample dashboard has been created. It can be uploaded with the following steps 61 | 62 | * Click on the Grafana icon menu on the top left and select Dashboards 63 | * Click on "New dashboard" 64 | * Click on the "Import Dashboard" button 65 | * Select "Upload .json File" button 66 | * Pick the json file available in this directory 67 | 68 | Here is how it looks like 69 | 70 | image::overview.png[] 71 | -------------------------------------------------------------------------------- /observability/grafana/overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fgiloux/auto-perf-test/8827be0c8e774c193d5c7b7f92b3f107b7924ffc/observability/grafana/overview.png -------------------------------------------------------------------------------- /observability/opentracing/README.adoc: -------------------------------------------------------------------------------- 1 | = Application Performance Management 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | For the purpose of this demonstration we are using the https://opentracing.io/[OpenTracing] API with collector and agent provided by the https://www.jaegertracing.io/[Jaeger] implementation. 16 | The instrumentation is split into sevaral parts: 17 | 18 | * a collector, which acts as a data sink 19 | * an agent in charge of sending application metrics to the sink 20 | * A frontend for querying data from the sink 21 | 22 | == Collector 23 | 24 | Deploying an in-memory Jaeger collector can be done with a single command. A production installation would require the use of Cassandra or Elasticsearch for data storage. 25 | Additional information is available in https://github.com/jaegertracing/jaeger-openshift[jaeger-openshift] repository. 26 | 27 | $ oc process -f https://raw.githubusercontent.com/jaegertracing/jaeger-openshift/master/all-in-one/jaeger-all-in-one-template.yml | oc create -f - 28 | 29 | == Agent 30 | 31 | The agent can be deployed as a sidecar container. This is handy as this setup will work independent of the language the application is written in. The library deployed with the application will send traces to the localhost using UDP. 32 | Therefore the following needs to be added to the application deployment configuration under the "containers" section: 33 | 34 | ---- 35 | - image: jaegertracing/jaeger-agent 36 | name: jaeger-agent 37 | ports: 38 | - containerPort: 5775 39 | protocol: UDP 40 | - containerPort: 5778 41 | - containerPort: 6831 42 | protocol: UDP 43 | - containerPort: 6832 44 | protocol: UDP 45 | args: 46 | - "--collector.host-port=jaeger-collector.jaeger-infra.svc:14267" 47 | ---- 48 | 49 | Using jaeger-java-client library we have added the following to the application pom.xml. 50 | 51 | ---- 52 | 53 | 0.30.1 54 | 55 | 56 | 57 | org.apache.camel 58 | camel-opentracing-starter 59 | 60 | 61 | io.jaegertracing 62 | jaeger-client 63 | ${jaeger.version} 64 | 65 | ---- 66 | 67 | The next step is to tell the application to send traces by annotating the Application class with *@CamelOpenTracing* and the Camel OpenTracing implementation does the rest. Events (spans) are captured for incoming and outgoing messages being sent to/from Camel. Logs are also sent and can be seen when the consumer span is expanded. 68 | 69 | // No span is created for the components inside the Camel route.... could be a good RFE. 70 | // Creating a span for the enqueue time could be another. 71 | 72 | [NOTE] 73 | ==== 74 | CamelOpenTracing did not translate dashes when setting them into or getting them from JMS headers. This has been fixed with this https://github.com/apache/camel/pull/2540[pull request]. The code with the fix is duplicated in this repository. 75 | ==== 76 | 77 | Finally the agent can be configured through environment variables set within the deployment configuration: 78 | 79 | ---- 80 | - name: JAEGER_SERVICE_NAME 81 | value: fakeapp 82 | - name: JAEGER_SAMPLER_PARAM 83 | value: '1.0' 84 | - name: JAEGER_PROPAGATION 85 | value: 'jaeger,b3' 86 | ---- 87 | 88 | The service name identifies the service provided by the application component in the trace. Per default Jaeger will send trace information for one call out of 1000 to limit the burden put on the application. For the sake of this demonstration the environment variable tells Jaeger to do it for every single call. Different formats can be used for trace propagation. Jaeger and b3 are configured here. 89 | 90 | [NOTE] 91 | ==== 92 | For Jaeger to be used with the application running locally (outside of OpenShift) a local agent would need to be run to cover the role played by the sidecar container on OpenShift. 93 | ==== 94 | 95 | == Querying 96 | 97 | Jaeger UI has been installed with the collector and a route has been created for reaching it. It can be used to retrieve and display traces and spans. 98 | 99 | 100 | It may also be worth mentioning that enMasse / ActiveMQ arthemis are working on adding OpenTracing support: 101 | 102 | * https://issues.apache.org/jira/browse/ARTEMIS-2028 103 | * https://issues.apache.org/jira/browse/DISPATCH-1088 104 | 105 | 106 | -------------------------------------------------------------------------------- /observability/prometheus/README.adoc: -------------------------------------------------------------------------------- 1 | = Prometheus 2 | ifdef::env-github[] 3 | :tip-caption: :bulb: 4 | :note-caption: :information_source: 5 | :important-caption: :heavy_exclamation_mark: 6 | :caution-caption: :fire: 7 | :warning-caption: :warning: 8 | endif::[] 9 | ifndef::env-github[] 10 | :imagesdir: ./ 11 | endif::[] 12 | :toc: 13 | :toc-placement!: 14 | 15 | == Installation 16 | 17 | A playbook is available for deploying Prometheus on the machine used for the OpenShift installation: 18 | 19 | $ ansible-playbook -vvv -i your.inventory /usr/share/ansible/openshift-ansible/playbooks/openshift-prometheus/config.yml 20 | 21 | Alternatively, the ansible playbook broker can be used for installinng Prometheus together with Grafana, see https://github.com/ansibleplaybookbundle/prometheus-apb[here]. 22 | 23 | NOTE: Things are evolving quickly in this realm and an operator, configurable during the ansible installation is available with OpenShift 3.11 for installing and managing Prometheus & Grafana. 24 | 25 | == Instrumentation 26 | 27 | The Fabric8 S2I image (fabric8/s2i-java), which is used among others by https://access.redhat.com/documentation/en-us/red_hat_fuse/7.0/html/managing_fuse/prometheus[Fuse] handily comes with support for exporting metrics to Prometheus. Among others it has a Prometheus exporter agent, which is started with the application and expose metrics to be scrapped. The java process is started with `-javaagent:/opt/prometheus/jmx_prometheus_javaagent.jar=9779:/opt/prometheus/prometheus-config.yml` where prometheus-config.yml already includes all the Camel metrics. You can use the S2I capabilites of the image to add your own metrics. More information https://access.redhat.com/documentation/en-us/red_hat_fuse/7.0/html/managing_fuse/prometheus#configuring_prometheus[here]. 28 | 29 | A service exposing the prometheus port (9779) needs to created with the following annotations: 30 | 31 | apiVersion: v1 32 | kind: Service 33 | metadata: 34 | annotations: 35 | prometheus.io/port: '9779' 36 | prometheus.io/scheme: http 37 | prometheus.io/scrape: 'true' 38 | 39 | Finally the prometheus configmap in the openshift-metrics project may need to be amended with the following lines: 40 | 41 | # Scrape config for camel-amq-fakeapp 42 | - job_name: 'camel-amq-fakeapp' 43 | kubernetes_sd_configs: 44 | - role: endpoints 45 | namespaces: 46 | names: 47 | - perftest 48 | 49 | relabel_configs: 50 | - source_labels: [__meta_kubernetes_service_name] 51 | action: keep 52 | regex: camel-amq-fakeapp 53 | - source_labels: [__meta_kubernetes_pod_container_port_name] 54 | action: keep 55 | regex: prometheus 56 | 57 | == Dashboards 58 | 59 | See <<../grafana/README.adoc#,Dashboards>> for information on using Grafana to query and display metrics. 60 | 61 | -------------------------------------------------------------------------------- /observability/prometheus/openshift/camel-amq-fakeapp-svc.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | annotations: 5 | prometheus.io/port: "9779" 6 | prometheus.io/scheme: http 7 | prometheus.io/scrape: "true" 8 | creationTimestamp: null 9 | labels: 10 | app: camel-amq-fakeapp 11 | group: org.fgiloux.sandbox 12 | provider: fabric8 13 | version: 1.0-SNAPSHOT 14 | name: camel-amq-fakeapp 15 | spec: 16 | ports: 17 | - name: http 18 | port: 8080 19 | protocol: TCP 20 | targetPort: 8080 21 | - name: prometheus 22 | port: 9779 23 | protocol: TCP 24 | targetPort: 9779 25 | - name: jolokia 26 | port: 8778 27 | protocol: TCP 28 | targetPort: 8778 29 | selector: 30 | app: camel-amq-fakeapp 31 | group: perftest 32 | provider: fabric8 33 | sessionAffinity: None 34 | type: ClusterIP 35 | status: 36 | loadBalancer: {} 37 | --------------------------------------------------------------------------------