├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md ├── docker-compose.yml ├── pom.xml ├── readme ├── architecture.png ├── grafana.png ├── grafana_loki_log.png ├── grafana_tempo_trace.png ├── loki_derived_fields.png ├── loki_setup.png └── tempo_setup.png ├── src └── main │ ├── java │ └── com │ │ └── antongoncharov │ │ └── demo │ │ └── camel │ │ └── otel │ │ ├── Application.java │ │ ├── DispatchController.java │ │ ├── DispatcherRoute.java │ │ ├── HelloController.java │ │ ├── RequestLoggingFilterConfig.java │ │ └── ServiceProperties.java │ └── resources │ └── application.properties └── tempo-config.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled class file 2 | *.class 3 | 4 | # Log file 5 | *.log 6 | 7 | # BlueJ files 8 | *.ctxt 9 | 10 | # Mobile Tools for Java (J2ME) 11 | .mtj.tmp/ 12 | 13 | # Package Files # 14 | *.jar 15 | *.war 16 | *.nar 17 | *.ear 18 | *.zip 19 | *.tar.gz 20 | *.rar 21 | 22 | # virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml 23 | hs_err_pid* 24 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM openjdk:11-jdk-slim 2 | ADD /target/camel-opentelemetry-grafana-demo-1.0.0-SNAPSHOT.jar observable-service.jar 3 | ADD /agent/opentelemetry-javaagent-all.jar /etc/agent/opentelemetry-javaagent-all.jar 4 | EXPOSE 8080 5 | ENTRYPOINT ["java", "-javaagent:/etc/agent/opentelemetry-javaagent-all.jar" , "-jar", "observable-service.jar"] -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Anton Goncharov 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Distributed Logs and Tracing with Spring, Apache Camel, OpenTelemetry and Grafana: Example 2 | 3 | ## Introduction 4 | 5 | This project is an example app to showcase [`camel-opentelemetry`](https://camel.apache.org/components/3.7.x/others/opentelemetry.html) integration in a Spring Boot app. Apache Camel has added OpenTelemetry support in the 3.5 release. The official documentation doesn't focus on how to set everything up for logs and traces visibility, so the goal of this example is to fill this gap. 6 | 7 | [OpenTelemetry](https://opentelemetry.io/) is an open-source project resulted from merging of OpenCensus and OpenTracing. Its purpose is to enable an application's observability by providing a set of standardized data collection tools for capturing and exporting metrics, logs and traces. It's one of the most active CNCF projects these days. 8 | 9 | Using [Camel](https://camel.apache.org/) in tandem with OpenTelemetry instrumentation allows us to have distributed tracing across different routes in one or more services, and to link application logs with these traces, which makes it a great alternative to ELK and similar solutions. 10 | 11 | Important note: this example uses Apache Camel 3.10 (the last available version as of June 2021), which depends on OpenTelemetry SDK for Java version 0.15.0. OpenTelemetry specification and its tools develop rapidly, now beyond 1.x release. In the Camel tracker I see the plans to upgrade dependency but today make sure 0.15.0 is used to run this example smoothly. 12 | 13 | ## Launch 14 | 15 | I used Maven 3.8.1 + JDK 11 to create the demo, so it should work with all version from these onwards. 16 | 17 | Steps to launch and try: 18 | 19 | 1. Prepare Docker 20 | 21 | Install Loki driver to Docker (to stream logs from the standard output of a container) 22 | 23 | ```sh 24 | docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions 25 | ``` 26 | 27 | Create a network for the demo services: 28 | 29 | ```sh 30 | docker network create tempo-net 31 | ``` 32 | 33 | 2. Pack the service with Maven 34 | 35 | ```sh 36 | mvn package 37 | ``` 38 | 39 | 3. Build the container image 40 | 41 | ```sh 42 | docker build . -t hello-service:1.0.0 43 | ``` 44 | 45 | 4. Run the docker-compose script to bring up the demo 46 | 47 | ```sh 48 | docker-compose run 49 | ``` 50 | 51 | 5. Open a web browser and navigate to the Grafana dashboard: `http://localhost:3000`. Go to the Configuration->Data Sources menu item. 52 | 53 | 6. Add new Tempo data source (a storage for traces) 54 | 55 | ![image / tempo](readme/tempo_setup.png) 56 | 57 | 7. Add new Loki data source (a storage for logs) 58 | 59 | ![image / loki](readme/loki_setup.png) 60 | 61 | Configure "Derived Field" in Loki Data Source to relate logs with traces 62 | 63 | ![image / loki / derived field](readme/loki_derived_fields.png) 64 | 65 | 8. Send a sample request 66 | 67 | ```sh 68 | curl http://localhost:8080/dispatch?name=Ada 69 | ``` 70 | 71 | 9. Open the "Explore" board in Grafana to browse through the logs in Loki and the traces in Tempo 72 | 73 | ![image / dashboard](readme/grafana.png) 74 | 75 | ## How it works 76 | 77 | The project is a Spring Boot web application named 'hello-service' serving 2 endpoints. `/hello` responds with "Hello, ${name}" and `/dispatch` passes the request to a downstream service. 78 | 79 | The docker-compose config spins up 5 container instances: 80 | - **hello-service (1)** (container "dispatch") accepts requests and dispatches them down to the **hello-service (2)** 81 | - **hello-service (2)** (container "hello") handles dispatched requests 82 | - **loki** is a logs storage 83 | - **tempo** is a traces storage 84 | - **grafana** is a web dashboard with loki/tempo data visualizations 85 | 86 | Interacting **hello-service (1)** and **hello-service (2)** are here to demonstrate how OpenTelemetry traces requests across different containers. 87 | 88 | The architecture overview: 89 | 90 | ![image / architecture](readme/architecture.png) 91 | 92 | Both 'hello' services send logs to Loki via [`loki-docker-driver`](https://grafana.com/docs/loki/latest/clients/docker-driver/) and traces to Tempo via the OpenTelemetry exporter. Grafana uses Loki and Tempo as data sources. Collected logs are linked to the trace data by using traceId assigned by the OpenTelemetry instrumentation. 93 | 94 | ### Service 95 | 96 | [OpenTelemetry instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation) dynamically captures telemetry from a number of popular Java frameworks. To plug it in, add `opentelemetry-javaagent-all.jar` as a javaagent of the JAR application. This preparation is done in the Dockerfile. 97 | 98 | For the Camel integration it's important that we use `opentelemetry-javaagent-all.jar` [version 0.15.0](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/v0.15.x) as [`camel-opentelemetry`](https://camel.apache.org/components/3.7.x/others/opentelemetry.html) depends on this OTel Java SDK release. 99 | 100 | ```dockerfile 101 | ADD /agent/opentelemetry-javaagent-all.jar /etc/agent/opentelemetry-javaagent-all.jar 102 | 103 | ENTRYPOINT ["java", "-javaagent:/etc/agent/opentelemetry-javaagent-all.jar" , "-jar", "observable-service.jar"] 104 | ``` 105 | 106 | For Camel 3.5+ it's required to add the `camel-opentelemetry-starter` dependency. Then, in a `RouteBuilder` class, set up the tracer as the following: 107 | 108 | ```java 109 | OpenTelemetryTracer ott = new OpenTelemetryTracer(); 110 | ott.init(this.getContext()); 111 | ``` 112 | The docker-compose configuration has a few items to highlight. 113 | - `OTEL_EXPORTER_OTLP_ENDPOINT` is an endpoint where to send the opentelemetry data. In our case it's the Tempo's receiver. 114 | - We don't send metrics in this demo, so it's disabled to hide all related warnings. 115 | - The downstream endpoint point to the address of the second service to show how distributed tracing works. 116 | - The `logging` sections states that the container's stdout should be pipelined to the Loki storage by using the [`loki-docker-driver`](https://grafana.com/docs/loki/latest/clients/docker-driver/). 117 | 118 | ```yaml 119 | dispatch-service: 120 | image: hello-service:1.0.0 121 | container_name: dispatch 122 | networks: 123 | - "tempo-net" 124 | ports: 125 | - "8080:8080" 126 | environment: 127 | - OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:55680 128 | - OTEL_EXPORTER_OTLP_INSECURE=true 129 | - OTEL_METRICS_EXPORTER=none 130 | - OTEL_RESOURCE_ATTRIBUTES=service.name=dispatcher 131 | # sends requests to the 'hello-service' container 132 | - SRV_DOWNSTREAM_ENDPOINT=http://hello:8080/hello 133 | logging: 134 | driver: loki 135 | options: 136 | loki-url: "http://localhost:3100/loki/api/v1/push" 137 | ``` 138 | 139 | The second service instance has the similar setup except for the fact that it has no downstream service. 140 | 141 | In `application.properties`, the `logging.pattern` formats log output in a way to include **traceId** and **spanId**. It's necessary for the system to be able to index logs and link them to traces. 142 | 143 | ### Loki 144 | 145 | [Loki](https://grafana.com/oss/loki/) is a scalable log aggregation system to use with Grafana. 146 | 147 | The Loki config is pretty much default. It exposes port 3100 so all the other docker containers can send logs there. 148 | 149 | ```yaml 150 | loki: 151 | hostname: "loki" 152 | image: grafana/loki:2.2.0 153 | ports: 154 | - "3100:3100" 155 | networks: 156 | - "tempo-net" 157 | command: "-config.file=/etc/loki/local-config.yaml" 158 | ``` 159 | 160 | ### Tempo 161 | 162 | [Tempo](https://grafana.com/oss/tempo/) is a scalable trace storage. It has receivers for Jaeger, Zipkin and OpenTelemetry. 163 | 164 | The `tempo-config.yaml` is mounted from the project. It disables auth and specifies that Tempo instance should expose an OpenTelemetry receiver. The default OTel port is 55680. 165 | 166 | ```yaml 167 | tempo: 168 | hostname: "tempo" 169 | image: grafana/tempo:latest 170 | networks: 171 | - "tempo-net" 172 | volumes: 173 | - ./tempo-config.yaml:/etc/tempo-config.yaml 174 | command: "-config.file=/etc/tempo-config.yaml" 175 | ``` 176 | 177 | ### Grafana 178 | 179 | [Grafana](https://grafana.com/) is a visualization and monitoring software that can connect to a wide variety of data sources and query them. 180 | 181 | After all set and executed as in the "Launch" section above, browse to the Grafana UI at `http://localhost:3000`. Open the "Explore" view and switch to the Loki view using to dropdown on the top. 182 | 183 | Searching for `{container_name="dispatch"}` will output the recent logs for the **hello-service (1)** running in the `dispatch` container. 184 | 185 | Click on an entry that has a trace ID. It would unfold a detailed info on the log entry. Next to the "traceId" field there's a button that would carry us to the corresponding tracing. 186 | 187 | ![image / grafana / log entry](readme/grafana_loki_log.png) 188 | 189 | Voilà, we've got to the distributed tracing for this request. Each span has the corresponding log entries grouped under the "Logs" box on this panel. 190 | 191 | ![image / grafana / trace](readme/grafana_tempo_trace.png) -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.3' 2 | services: 3 | dispatch-service: 4 | image: hello-service:1.0.0 5 | container_name: dispatch 6 | networks: 7 | - "tempo-net" 8 | ports: 9 | - "8080:8080" 10 | environment: 11 | - OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:55680 12 | - OTEL_EXPORTER_OTLP_INSECURE=true 13 | - OTEL_METRICS_EXPORTER=none 14 | - OTEL_RESOURCE_ATTRIBUTES=service.name=dispatcher 15 | # sends requests to the 'hello-service' container 16 | - SRV_DOWNSTREAM_ENDPOINT=http://hello:8080/hello 17 | logging: 18 | driver: loki 19 | options: 20 | loki-url: "http://localhost:3100/loki/api/v1/push" 21 | 22 | hello-service: 23 | image: hello-service:1.0.0 24 | container_name: hello 25 | networks: 26 | - "tempo-net" 27 | ports: 28 | - "8081:8080" 29 | environment: 30 | - OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:55680 31 | - OTEL_EXPORTER_OTLP_INSECURE=true 32 | - OTEL_RESOURCE_ATTRIBUTES=service.name=hello 33 | - OTEL_METRICS_EXPORTER=none 34 | # we don't use downstream endpoint here 35 | - SRV_DOWNSTREAM_ENDPOINT=http://localhost:8081/hello 36 | logging: 37 | driver: loki 38 | options: 39 | loki-url: "http://localhost:3100/loki/api/v1/push" 40 | 41 | tempo: 42 | hostname: "tempo" 43 | image: grafana/tempo:latest 44 | networks: 45 | - "tempo-net" 46 | volumes: 47 | - ./tempo-config.yaml:/etc/tempo-config.yaml 48 | command: "-config.file=/etc/tempo-config.yaml" 49 | 50 | loki: 51 | hostname: "loki" 52 | image: grafana/loki:2.2.0 53 | ports: 54 | - "3100:3100" 55 | networks: 56 | - "tempo-net" 57 | command: "-config.file=/etc/loki/local-config.yaml" 58 | 59 | grafana: 60 | hostname: "grafana" 61 | image: grafana/grafana:7.5.2 62 | environment: 63 | - GF_AUTH_ANONYMOUS_ENABLED=true 64 | - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin 65 | - GF_AUTH_DISABLE_LOGIN_FORM=true 66 | networks: 67 | - "tempo-net" 68 | ports: 69 | - "3000:3000" 70 | networks: 71 | tempo-net: 72 | # use default driver -------------------------------------------------------------------------------- /pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 5 | 4.0.0 6 | 7 | 8 | org.springframework.boot 9 | spring-boot-starter-parent 10 | 2.4.0 11 | 12 | 13 | groupId 14 | camel-opentelemetry-grafana-demo 15 | 1.0.0-SNAPSHOT 16 | jar 17 | 18 | 19 | 11 20 | 11 21 | 3.10.0 22 | 23 | 24 | 25 | 26 | 27 | 28 | org.apache.camel.springboot 29 | camel-spring-boot-bom 30 | ${camel.version} 31 | pom 32 | import 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | org.springframework.boot 41 | spring-boot-starter-web 42 | 43 | 44 | org.apache.camel.springboot 45 | camel-spring-boot-starter 46 | 47 | 48 | org.apache.camel.springboot 49 | camel-opentelemetry-starter 50 | ${camel.version} 51 | 52 | 53 | org.apache.camel 54 | camel-undertow 55 | ${camel.version} 56 | 57 | 58 | 59 | 60 | org.springframework.boot 61 | spring-boot-starter-test 62 | test 63 | 64 | 65 | 66 | 67 | 68 | 69 | org.springframework.boot 70 | spring-boot-maven-plugin 71 | 72 | 73 | 74 | 75 | -------------------------------------------------------------------------------- /readme/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/architecture.png -------------------------------------------------------------------------------- /readme/grafana.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/grafana.png -------------------------------------------------------------------------------- /readme/grafana_loki_log.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/grafana_loki_log.png -------------------------------------------------------------------------------- /readme/grafana_tempo_trace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/grafana_tempo_trace.png -------------------------------------------------------------------------------- /readme/loki_derived_fields.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/loki_derived_fields.png -------------------------------------------------------------------------------- /readme/loki_setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/loki_setup.png -------------------------------------------------------------------------------- /readme/tempo_setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anton-goncharov/camel-opentelemetry-grafana-demo/54c622100bc4ea24107b9cfa525b84a8d7eabf72/readme/tempo_setup.png -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/Application.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.springframework.boot.SpringApplication; 4 | import org.springframework.boot.autoconfigure.SpringBootApplication; 5 | import org.springframework.boot.context.properties.EnableConfigurationProperties; 6 | 7 | /** 8 | * @author antongoncharov 9 | */ 10 | @SpringBootApplication 11 | @EnableConfigurationProperties(ServiceProperties.class) 12 | public class Application { 13 | 14 | public static void main(String[] args) { 15 | SpringApplication.run(Application.class, args); 16 | } 17 | 18 | } 19 | -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/DispatchController.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.apache.camel.ExchangePattern; 4 | import org.apache.camel.ProducerTemplate; 5 | import org.springframework.stereotype.Controller; 6 | import org.springframework.web.bind.annotation.GetMapping; 7 | import org.springframework.web.bind.annotation.RequestParam; 8 | import org.springframework.web.bind.annotation.ResponseBody; 9 | 10 | /** 11 | * @author antongoncharov 12 | */ 13 | @Controller 14 | public class DispatchController { 15 | 16 | private final ProducerTemplate template; 17 | 18 | public DispatchController( 19 | ProducerTemplate template 20 | ) { 21 | this.template = template; 22 | } 23 | 24 | @GetMapping("/dispatch") 25 | @ResponseBody 26 | public String sayHello(@RequestParam String name) { 27 | Object obj = template.sendBody("direct:dispatcher", ExchangePattern.InOut, name); 28 | return obj.toString(); 29 | } 30 | 31 | } 32 | -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/DispatcherRoute.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.apache.camel.builder.RouteBuilder; 4 | import org.apache.camel.opentelemetry.OpenTelemetryTracer; 5 | import org.springframework.beans.factory.annotation.Autowired; 6 | import org.springframework.stereotype.Component; 7 | 8 | /** 9 | * @author antongoncharov 10 | */ 11 | @Component 12 | public class DispatcherRoute extends RouteBuilder { 13 | 14 | private final ServiceProperties serviceProperties; 15 | 16 | @Autowired 17 | public DispatcherRoute(ServiceProperties serviceProperties) { 18 | this.serviceProperties = serviceProperties; 19 | } 20 | 21 | @Override 22 | public void configure() { 23 | 24 | OpenTelemetryTracer ott = new OpenTelemetryTracer(); 25 | ott.init(this.getContext()); 26 | 27 | from("direct:dispatcher").routeId("dispatcher-route") 28 | .log("incoming request, body = ${body}") 29 | .to("direct:sayHello"); 30 | 31 | from("direct:sayHello").routeId("sayHello-route") 32 | .log("dispatching to /hello endpoint") 33 | .setHeader("status", constant("dispatched")) 34 | .toD("undertow:" + serviceProperties.getDownstreamEndpoint() + "?name=${body}") 35 | .convertBodyTo(String.class); 36 | 37 | } 38 | 39 | } 40 | 41 | -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/HelloController.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.springframework.stereotype.Controller; 4 | import org.springframework.web.bind.annotation.GetMapping; 5 | import org.springframework.web.bind.annotation.RequestParam; 6 | import org.springframework.web.bind.annotation.ResponseBody; 7 | 8 | /** 9 | * @author antongoncharov 10 | */ 11 | @Controller 12 | public class HelloController { 13 | 14 | @GetMapping("/hello") 15 | @ResponseBody 16 | public String sayHello(@RequestParam String name) { 17 | return "Hi, " + name; 18 | } 19 | 20 | } 21 | -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/RequestLoggingFilterConfig.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.springframework.context.annotation.Bean; 4 | import org.springframework.context.annotation.Configuration; 5 | import org.springframework.web.filter.CommonsRequestLoggingFilter; 6 | 7 | /** 8 | * @author antongoncharov 9 | */ 10 | @Configuration 11 | public class RequestLoggingFilterConfig { 12 | 13 | @Bean 14 | public CommonsRequestLoggingFilter logFilter() { 15 | CommonsRequestLoggingFilter filter = new CommonsRequestLoggingFilter(); 16 | filter.setIncludeQueryString(true); 17 | filter.setIncludeHeaders(true); 18 | filter.setIncludeClientInfo(true); 19 | return filter; 20 | } 21 | 22 | } 23 | -------------------------------------------------------------------------------- /src/main/java/com/antongoncharov/demo/camel/otel/ServiceProperties.java: -------------------------------------------------------------------------------- 1 | package com.antongoncharov.demo.camel.otel; 2 | 3 | import org.springframework.boot.context.properties.ConfigurationProperties; 4 | import org.springframework.stereotype.Component; 5 | 6 | /** 7 | * @author antongoncharov 8 | */ 9 | @Component 10 | @ConfigurationProperties(prefix = "srv") 11 | public class ServiceProperties { 12 | 13 | private String downstreamEndpoint; 14 | 15 | public String getDownstreamEndpoint() { 16 | return downstreamEndpoint; 17 | } 18 | 19 | public void setDownstreamEndpoint(String downstreamEndpoint) { 20 | this.downstreamEndpoint = downstreamEndpoint; 21 | } 22 | 23 | } 24 | 25 | -------------------------------------------------------------------------------- /src/main/resources/application.properties: -------------------------------------------------------------------------------- 1 | camel.springboot.main-run-controller = true 2 | 3 | logging.pattern.console = %clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m traceId=%X{traceId} spanId=%X{spanId} %n%wEx 4 | 5 | logging.level.org.springframework.web.filter.CommonsRequestLoggingFilter = DEBUG 6 | logging.level.io.opentelemetry = DEBUG 7 | logging.level.root = INFO 8 | 9 | srv.downstreamEndpoint = http://localhost:8080/hello -------------------------------------------------------------------------------- /tempo-config.yaml: -------------------------------------------------------------------------------- 1 | auth_enabled: false 2 | 3 | server: 4 | http_listen_port: 3100 5 | 6 | distributor: 7 | receivers: 8 | otlp: 9 | protocols: 10 | http: 11 | grpc: 12 | 13 | storage: 14 | trace: 15 | backend: local 16 | wal: 17 | path: /tmp/tempo/wal 18 | local: 19 | path: /tmp/tempo/blocks 20 | pool: 21 | max_workers: 100 22 | queue_depth: 10000 23 | --------------------------------------------------------------------------------