├── README.md
├── fluentbit
├── cm_demo.yaml
├── example.log
├── fluentbit_cm.yaml
├── fluentbit_configmap_loki.yaml
├── fluentbit_ds.yaml
└── fluentbit_loki_ds.yaml
├── grafana
└── ingress.yaml
├── hipstershop
├── k8s-manifest.yaml
├── paymentservice-fix-the-fix.yaml
├── paymentservice-fix.yaml
├── paymentservice-new.yaml
├── paymentservice.yaml
├── setup.sh
└── teardown.sh
└── image
├── addsource.PNG
├── datasource.PNG
├── explore.png
├── fluentbit.png
├── getcm.PNG
├── k8sprom.png
├── kubernetes.png
├── log_stream_pipeline.PNG
├── logo.png
├── loki_logo.png
└── prometheus.png
/README.md:
--------------------------------------------------------------------------------
1 | # How to configure Fluent Bit to collect Logs for our K8S cluster
2 |
3 | This repository is here to guide you through the GitHub tutorial that goes hand-in-hand with a video available on YouTube and a detailed blog post on my website.
4 | Together, these resources are designed to give you a complete understanding of the topic.
5 |
6 |
7 | Here are the links to the related assets:
8 | - YouTube Video: [How to configure Fluent Bit to collect Logs for our K8S cluster](https://www.youtube.com/watch?v=KJlWV5-o8v0)
9 | - Blog Post: [How to configure Fluent Bit to collect logs for your Kubernetes cluster](https://isitobservable.io/observability/kubernetes/how-to-configure-fluent-bit-to-collect-logs-for-your-k8s-cluster)
10 |
11 |
12 | Feel free to explore the materials, star the repository, and follow along at your own pace.
13 |
14 |
15 | ## K8s and Logging with Fluentbit
16 | 
17 |
18 | This repository showcases the usage of Loki by using GKE with the HipsterShop.
19 |
20 |
21 | ## Prerequisites
22 | The following tools need to be installed on your machine :
23 | - jq
24 | - kubectl
25 | - git
26 | - gcloud (if you're using GKE)
27 | - Helm
28 |
29 | ### 1. Create a Google Cloud Platform Project
30 | ```
31 | PROJECT_ID=""
32 | gcloud services enable container.googleapis.com --project ${PROJECT_ID}
33 | gcloud services enable monitoring.googleapis.com \
34 | cloudtrace.googleapis.com \
35 | clouddebugger.googleapis.com \
36 | cloudprofiler.googleapis.com \
37 | --project ${PROJECT_ID}
38 | ```
39 | ### 2. Create a GKE cluster
40 | ```
41 | ZONE=us-central1-b
42 | gcloud containr clusters create isitobservable \
43 | --project=${PROJECT_ID} --zone=${ZONE} \
44 | --machine-type=e2-standard-2 --num-nodes=4
45 | ```
46 | ### 3.Clone the GitHub repo
47 | ```
48 | git clone https://github.com/isItObservable/Episode3--Kubernetes-Fluentbit.git
49 | cd Episode3--Kubernetes-Fluentbit
50 | ```
51 | ### 4. Deploy Prometheus
52 | #### HipsterShop
53 | ```
54 | cd hipstershop
55 | ./setup.sh
56 | ```
57 | #### Prometheus (as already done during [Episode 1](https://github.com/isItObservable/Episode1---Kubernetes-Prometheus))
58 | ```
59 | helm install prometheus stable/prometheus-operator
60 | ```
61 | #### Expose Grafana
62 | ```
63 | kubectl get svc
64 | kubectl edit svc prometheus-grafana
65 | ```
66 | change to type NodePort
67 | ```yaml
68 | apiVersion: v1
69 | kind: Service
70 | metadata:
71 | annotations:
72 | meta.helm.sh/release-name: prometheus
73 | meta.helm.sh/release-namespace: default
74 | labels:
75 | app.kubernetes.io/instance: prometheus
76 | app.kubernetes.io/managed-by: Helm
77 | app.kubernetes.io/name: grafana
78 | app.kubernetes.io/version: 7.0.3
79 | helm.sh/chart: grafana-5.3.0
80 | name: prometheus-grafana
81 | namespace: default
82 | resourceVersion: "89873265"
83 | selfLink: /api/v1/namespaces/default/services/prometheus-grafana
84 | spec:
85 | clusterIP: IPADRESSS
86 | externalTrafficPolicy: Cluster
87 | ports:
88 | - name: service
89 | nodePort: 30806
90 | port: 80
91 | protocol: TCP
92 | targetPort: 3000
93 | selector:
94 | app.kubernetes.io/instance: prometheus
95 | app.kubernetes.io/name: grafana
96 | sessionAffinity: None
97 | type: NodePort
98 | status:
99 | loadBalancer: {}
100 | ```
101 | Deploy the ingress by making sure to replace the service name of your Grafana
102 | ```
103 | cd ..\grafana
104 | kubectl apply -f ingress.yaml
105 | ```
106 | Get the login user and password of Grafana
107 | * For the password :
108 | ```
109 | kubectl get secret --namespace default prometheus-grafana -o jsonpath="{.data.admin-password}" | base64 --decode
110 | ```
111 | * For the login user:
112 | ```
113 | kubectl get secret --namespace default prometheus-grafana -o jsonpath="{.data.admin-user}" | base64 --decode
114 | ```
115 | Get the ip adress of your Grafana
116 | ```
117 | kubectl get ingress grafana-ingress -ojson | jq '.status.loadBalancer.ingress[].ip'
118 | ```
119 | #### Install Loki with Fluent Bit
120 | ```
121 | helm repo add loki https://grafana.github.io/loki/charts
122 | helm repo update
123 | helm upgrade --install loki loki/loki-stack --set fluent-bit.enabled=true,promtail.enabled=false
124 | ```
125 | #### Configure Grafana
126 | In order to build a dashboard with data stored in Loki, we first need to add a new DataSource.
127 | In Grafana, go to Configuration/Add data source.
128 | 
129 | Select the source Loki, and configure the URL to interact with it.
130 |
131 | Remember, Grafana is hosted in the same namespace as Loki.
132 | So you can simply refer to the Loki service :
133 | 
134 |
135 | #### explore the data provided by Loki in Grafana
136 | In Grafana, select Explore on the main menu
137 | Select the datasource Loki. In the drop-down menu, select the label product -> hipster-shop
138 | 
139 |
140 | #### Let's build a query
141 | Loki has a specific query language that allows you to filter, transform the data, and even plot a metric from your logs in a graph.
142 | Similar to Prometheus, you need to :
143 | * filter using labels : {app="frontend",product="hipster-shop" ,stream="stdout"}
144 | We're here only looking at the logs from hipster-shop, app frontend, and on the logs pushed in stdout.
145 | * transform using |
146 | for example :
147 | ```
148 | {namespace="hipster-shop",stream="stdout"} | json | http_resp_took_ms >10
149 | ```
150 | The first ```|``` specifies to Grafana to use the JSON parser that will extract all the JSON properties as labels.
151 | The second ```|``` will filter the logs on the new labels created by the JSON parser.
152 | In this example, we want to only get the logs where the attribute http.resp.took.ms is above 10ms ( the json parser is replace . by _)
153 |
154 | We can then extract on the field to plot it using all the various [functions available in Grafana](https://grafana.com/docs/loki/latest/logql/)
155 |
156 | If you want to plot the response time over time, you could use the function :
157 | ```
158 | rate({namespace="hipster-shop" } |="stdout" !="error" |= "debug" |="http.resp.took_ms" [30s])
159 | ```
160 |
161 | ### Let's install Fluentbit to go through the configuration
162 | Now that we have used the default configuration with Loki, let's deploy the standard Fluentbit
163 | and explore the settings.
164 |
165 | #### Installation of Fluentbit
166 | ```
167 | helm repo add fluent https://fluent.github.io/helm-charts
168 | helm install fluent-bit fluent/fluent-bit
169 | ```
170 |
171 | #### Let's jump into the Fluent Bit configuration file
172 |
173 | The configuration file is stored in a ConfigMap
174 | ```
175 | kubectl get cm
176 | ```
177 | 
178 |
179 | ```yaml
180 | [SERVICE]
181 | Flush 1
182 | Daemon Off
183 | Log_Level info
184 | Parsers_File parsers.conf
185 | HTTP_Server On
186 | HTTP_Listen 0.0.0.0
187 | HTTP_Port 2020
188 |
189 | [INPUT]
190 | Name tail
191 | Path /var/log/containers/*.log
192 | Parser docker
193 | Tag kube.*
194 | Mem_Buf_Limit 5MB
195 | Skip_Long_Lines On
196 |
197 | [INPUT]
198 | Name systemd
199 | Tag host.*
200 | Systemd_Filter _SYSTEMD_UNIT=kubelet.service
201 | Read_From_Tail On*
202 |
203 |
204 | ```
205 |
206 | Now that we have the default configuration to collect logs of our Pods
207 | Let's see how to filter and change the log stream
208 |
209 | #### Let's start by filtering Kubernetes metrics
210 | Let's add a Filter block to our current Fluent Bit pipeline
211 |
212 | ```
213 | [FILTER]
214 | Name kubernetes
215 | Match kube.*
216 | Merge_Log On
217 | Merge_Log_Trim On
218 | Labels Off
219 | Annotations Off
220 | K8S-Logging.Parser Off
221 | K8S-Logging.Exclude Off
222 | ```
223 | And an output plugin to see the transformed log in Stdout ( of our fluentbit pods)
224 | ```
225 | [OUTPUT]
226 | Name stdout
227 | Match *
228 | Format json
229 | Json_date_key timestamp
230 | Json_date_format iso8601
231 | ```
232 |
233 | #### Now let's transform our log stream to be able to send it to the Dynatrace log ingest API
234 |
235 | #### Requierements
236 | If you don't have any Dynatrace tenant, then let's start a [trial](https://dt-url.net/trial )
237 | Set up the Dynatrace K8s operator following the steps described in the [documentation](https://www.dynatrace.com/support/help/technology-support/container-platforms/kubernetes/monitor-kubernetes-environments/)
238 |
239 | In order to collect logs in Dynatrace, you'll also need to install the Active Gate.*
240 | Follow the documentation to [install the Active Gate on a seperate server](https://www.dynatrace.com/support/help/setup-and-configuration/dynatrace-activegate/)
241 |
242 | #### Configuration of Fluentbit
243 | Now we need to rename the log to content, and rename the Kubernetes information with the right fields.
244 | ```
245 | [FILTER]
246 | Name modify
247 | Match *
248 | Rename log content
249 | ```
250 |
251 | Let's use the nest filter plugin to move the kubernetes tags
252 | ```
253 | [FILTER]
254 | Name nest
255 | Match kube.*
256 | Operation lift
257 | Nested_under kubernetes
258 | Add_prefix kubernetes_
259 | ```
260 | Let's use modify plugin to rename and remove the non relevant tags
261 | ```
262 | [FILTER]
263 | Name modify
264 | Match kube.*
265 | Rename log content
266 | Rename kubernetes_pod_name k8s.pod.name
267 | Rename kubernetes_namespace_name k8s.namespace.name
268 | Remove kubernetes_container_image
269 | Remove kubernetes_docker_id
270 | Remove kubernetes_container_name
271 | Remove kubernetes_pod_id
272 | Remove kubernetes_host
273 | Remove time
274 | Remove kubernetes_container_hash
275 | Add k8s.cluster.name Onlineboutique
276 | ```
277 |
278 | The Dynatrace ingest API is limiting the number of calls per minute.
279 | We need to throttle the streams :
280 | ```
281 | [FILTER]
282 | Name throttle
283 | Match *
284 | Rate 100
285 | Window 100
286 | Interval 1m
287 | ```
288 |
289 | Last, we can now connect the Dynatrace API using the HTTP output plugin
290 | ```
291 | [OUTPUT]
292 | Name http
293 | Match *
294 | host YOURHOST
295 | port 9999
296 | URI /e//api/v2/logs/ingest
297 | header Authorization Api-Token
298 | header Content-Type application/json
299 | Format json
300 | Json_date_key timestamp
301 | Json_date_format iso8601
302 | tls On
303 | tls.verify Off
304 | ```
305 |
306 | Let's open go to [calyptia](https://cloud.calyptia.com/) to visualize our log stream pipeline:
307 | 
308 |
--------------------------------------------------------------------------------
/fluentbit/cm_demo.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: v1
2 | data:
3 | parsers.conf: |-
4 | [PARSER]
5 | Name docker
6 | Format json
7 | Time_Key time
8 | Time_Format %Y-%m-%dT%H:%M:%S.%L
9 | custom_parsers.conf: |
10 | [PARSER]
11 | Name docker_no_time
12 | Format json
13 | Time_Keep Off
14 | Time_Key time
15 | Time_Format %Y-%m-%dT%H:%M:%S.%L
16 | fluent-bit.conf: |
17 | [SERVICE]
18 | Flush 1
19 | Daemon Off
20 | Log_Level info
21 | Parsers_File parsers.conf
22 | HTTP_Server On
23 | HTTP_Listen 0.0.0.0
24 | HTTP_Port 2020
25 |
26 | [INPUT]
27 | Name tail
28 | Path /var/log/containers/*.log
29 | Parser docker
30 | Tag kube.*
31 | Mem_Buf_Limit 5MB
32 | Skip_Long_Lines On
33 |
34 | [INPUT]
35 | Name systemd
36 | Tag host.*
37 | Systemd_Filter _SYSTEMD_UNIT=kubelet.service
38 | Read_From_Tail On*
39 |
40 | [FILTER]
41 | Name modify
42 | Match *
43 | Rename message content
44 | Rename log content
45 |
46 | [FILTER]
47 | Name kubernetes
48 | Match kube.*
49 | Merge_Log On
50 | Merge_Log_Trim On
51 | Labels Off
52 | Annotations Off
53 | K8S-Logging.Parser Off
54 | K8S-Logging.Exclude Off
55 |
56 |
57 | [FILTER]
58 | Name nest
59 | Match kube.*
60 | Operation lift
61 | Nested_under kubernetes
62 | Add_prefix kubernetes_
63 |
64 | [FILTER]
65 | Name grep
66 | Match kube.*
67 | Exclude kubernetes_container_name fluent-bit
68 |
69 | [FILTER]
70 | Name modify
71 | Match kube.*
72 | Rename log content
73 | Rename kubernetes_pod_name k8s.pod.name
74 | Rename kubernetes_namespace_name k8s.namespace.name
75 | Remove kubernetes_container_image
76 | Remove kubernetes_docker_id
77 | Remove kubernetes_container_name
78 | Remove kubernetes_pod_id
79 | Remove kubernetes_host
80 | Remove time
81 | Remove kubernetes_container_hash
82 | Add k8s.cluster.name Onlineboutique
83 |
84 | [FILTER]
85 | Name throttle
86 | Match kube.*
87 | Rate 5000
88 | Window 5
89 | Print_Status true
90 | Interval 30s
91 |
92 | [OUTPUT]
93 | Name http
94 | Match kube.*
95 | host
96 | port 9999
97 | URI /e/bix24852/api/v2/logs/ingest
98 | header Authorization Api-Token
99 | header Content-Type application/json; charset=utf-8
100 | Format json
101 | allow_duplicated_headers false
102 | Json_date_key timestamp
103 | Json_date_format iso8601
104 | tls On
105 | tls.verify Off
106 | Retry_Limit false
107 |
108 | [OUTPUT]
109 | Name stdout
110 | Match kube.*
111 | Format json
112 | Json_date_key timestamp
113 | Json_date_format iso8601
114 |
115 |
116 | kind: ConfigMap
117 | metadata:
118 | annotations:
119 | meta.helm.sh/release-name: fluent-bit
120 | meta.helm.sh/release-namespace: default
121 | labels:
122 | app.kubernetes.io/instance: fluent-bit
123 | app.kubernetes.io/managed-by: Helm
124 | app.kubernetes.io/name: fluent-bit
125 | app.kubernetes.io/version: 1.7.9
126 | helm.sh/chart: fluent-bit-0.15.15
127 | name: fluent-bit
128 | namespace: default
129 |
130 |
--------------------------------------------------------------------------------
/fluentbit/example.log:
--------------------------------------------------------------------------------
1 | [1mFluent Bit v1.7.9[0m
2 | * [1m[93mCopyright (C) 2019-2021 The Fluent Bit Authors[0m
3 | * [1m[93mCopyright (C) 2015-2018 Treasure Data[0m
4 | * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
5 | * https://fluentbit.io
6 |
7 | [2021/07/08 07:46:26] [ info] [engine] started (pid=1)
8 | [2021/07/08 07:46:26] [ info] [storage] version=1.1.1, initializing...
9 | [2021/07/08 07:46:26] [ info] [storage] in-memory
10 | [2021/07/08 07:46:26] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
11 | [2021/07/08 07:46:26] [ info] [filter:kubernetes:kubernetes.1] https=1 host=kubernetes.default.svc port=443
12 | [2021/07/08 07:46:26] [ info] [filter:kubernetes:kubernetes.1] local POD info OK
13 | [2021/07/08 07:46:26] [ info] [filter:kubernetes:kubernetes.1] testing connectivity with API server...
14 | [2021/07/08 07:46:26] [ info] [filter:kubernetes:kubernetes.1] connectivity OK
15 | [2021/07/08 07:46:26] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
16 | [2021/07/08 07:46:26] [ info] [sp] stream processor started
17 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049222 watch_fd=1 name=/var/log/containers/bpf-exporter-gl999_default_bpf-exporter-e9dcc6c208ec72f3ae47f092ba64fe230580f6d2b042376b7e15d21855e25299.log
18 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049300 watch_fd=2 name=/var/log/containers/carts-577d7858c8-jxkhq_sockshop-dev_carts-8a0098e8543be7704d1727e9eb4bf33d2a931be844f1199f15240aa6803af8fb.log
19 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049347 watch_fd=3 name=/var/log/containers/carts-786dcc48b5-b6np2_sockshop-production_carts-6d6201e6b3b80329cf8c3d99bb230b6cbc729e93ab7555d1e08902e20b46f926.log
20 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049311 watch_fd=4 name=/var/log/containers/carts-786dcc48b5-b6np2_sockshop-production_istio-init-77cc0bd7694a155ebb019df526fe02e3e98db19fa2f74030df0673b23f7df66b.log
21 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049421 watch_fd=5 name=/var/log/containers/carts-786dcc48b5-b6np2_sockshop-production_istio-proxy-6c10b3a110d93aa1b12d1d4b9c0f7b160f36d7f31c12d08fadc0dce083b3a9da.log
22 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049322 watch_fd=6 name=/var/log/containers/cartservice-67dc579847-gmpvv_hipster-shop_server-6549d31224680f344cf6a3ceca40c05e4576f22940285b8a5ae4ff64fdb8ab0d.log
23 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049554 watch_fd=7 name=/var/log/containers/catalogue-78cb56f58b-h974v_sockshop-dev_catalogue-5f89f209bdc55b8c4a7fd543d7644f15d88fc45b32e35716f76aac6f27ae6b44.log
24 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049480 watch_fd=8 name=/var/log/containers/catalogue-78cb56f58b-h974v_sockshop-dev_catalogue-712b9a9c22729a98b714753926a246f209cda1428880227ccc30870814b8d86b.log
25 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049183 watch_fd=9 name=/var/log/containers/cert-manager-85db5c4c87-grkcf_cert-manager_cert-manager-197dbf72e0a316f2406e1cc6754bc187781e218fc9432c80cb2c0ed1766cb7fc.log
26 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049447 watch_fd=10 name=/var/log/containers/cert-manager-85db5c4c87-grkcf_cert-manager_cert-manager-51f45f7b7b5ffb668fe0b014318fbe2b8825de688d3995131a7f9328bcb9161d.log
27 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049238 watch_fd=11 name=/var/log/containers/cert-manager-cainjector-7959549c78-djb27_cert-manager_cert-manager-4fd5b263b7e9fab974d4d81f0ea4d7ac5b10857ded6d6e702a543a2219cd7826.log
28 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049536 watch_fd=12 name=/var/log/containers/cert-manager-cainjector-7959549c78-djb27_cert-manager_cert-manager-5e3d21dab39111e6230a66eedb501b3b65503cae746f88e5754d17117d3d145f.log
29 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049248 watch_fd=13 name=/var/log/containers/cert-manager-webhook-5c8696f555-hvrwk_cert-manager_cert-manager-a7b0a7e3e9216a82cc9ba3ba482b1d9af0d144860504951a25d2a63ae1223045.log
30 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=393709 watch_fd=14 name=/var/log/containers/dynakube-classic-sdgx9_dynatrace_dynatrace-oneagent-14d1da33dc729ec52cfb5bc927f0b87a7571525b8eddd5b8804be9131498a7fb.log
31 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=393691 watch_fd=15 name=/var/log/containers/fluentbit-gke-4d9kx_kube-system_fluentbit-8404d35ea7a54042d84adb530269348a7d46b956a6881de8c9c0ca2e97f9e5c1.log
32 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=787402 watch_fd=16 name=/var/log/containers/fluentbit-gke-4d9kx_kube-system_fluentbit-gke-a9262a8c00c99f692c50670f70bd0911543a518e54356b0dc37f2db31de89fa6.log
33 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049218 watch_fd=17 name=/var/log/containers/gke-metrics-agent-s79tc_kube-system_gke-metrics-agent-d3961f0673f128732b8950234a04916863f6383a7f016981b99a8419be262698.log
34 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=791580 watch_fd=18 name=/var/log/containers/istiod-7fd6d8d4d9-thvwx_istio-system_discovery-4f211ecd68f15a63fafcbb399ace077716542793a5c1102384d8029aa2f8d807.log
35 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=791619 watch_fd=19 name=/var/log/containers/jenkins-0_jenkins_config-reload-a95a9921705ea22e39ae59e96e1be254fa5c8e2a11e035515c784e94a5a0593e.log
36 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=791595 watch_fd=20 name=/var/log/containers/jenkins-0_jenkins_init-a316b6c6a883ebd89e9b7bf8dfeccd69b889bab9c14d4620d212ca7d855fdaf7.log
37 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=791606 watch_fd=21 name=/var/log/containers/jenkins-0_jenkins_jenkins-227a5784bad1e9c148754e39cb5417c2ca988b0f52ef92960c90b6d47965d7d2.log
38 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049157 watch_fd=22 name=/var/log/containers/kube-dns-autoscaler-58cbd4f75c-5lll2_kube-system_autoscaler-fbdebce8d893a0ebbf8d6c493a5eb886813fdb0e2bbe78854a25eeda21c7e15f.log
39 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=392152 watch_fd=23 name=/var/log/containers/kube-proxy-gke-onlineboutique-default-pool-112e98a2-j5zr_kube-system_kube-proxy-627e75b14616a7d8f8537097f5c5f8b81aad1ba29e65a26035b4e497eeb19ec7.log
40 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049395 watch_fd=24 name=/var/log/containers/l7-default-backend-5d7d4cfccb-p49hr_kube-system_default-http-backend-24f574f8dc22bc7ad00aa06d5dd15bf6fbdd1f05ad683ba4e69cf15cef2871d9.log
41 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049194 watch_fd=25 name=/var/log/containers/loadgenerator-d8bd99bbb-dpvbz_hipster-shop_main-6b2060091b8104d4589a239dd166c34abdeef6463828d44f8027736973f21503.log
42 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049491 watch_fd=26 name=/var/log/containers/loki-fluent-bit-loki-pdk9r_default_fluent-bit-loki-dc34ab33660b96613da6cf468f7c3668ac28b578c69f343a4c2018fa3125ed09.log
43 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049170 watch_fd=27 name=/var/log/containers/metrics-server-v0.3.6-7b5cdbcbb8-bph4r_kube-system_metrics-server-8f08bc7337df20b0614a37845bc009ba4476743f9c5f38c100ce864e4ace19cc.log
44 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049196 watch_fd=28 name=/var/log/containers/metrics-server-v0.3.6-7b5cdbcbb8-bph4r_kube-system_metrics-server-nanny-2ce380a67f5460e269b2b3697002fe85e05eaf74b188611db1d7f85912254588.log
45 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049327 watch_fd=29 name=/var/log/containers/payment-54c476fb7f-fq75n_sockshop-production_istio-init-7d3c45303c9f0e479d0462f25d96857aad3d752338b2d4cd7c82c20c4b196bb6.log
46 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049434 watch_fd=30 name=/var/log/containers/payment-54c476fb7f-fq75n_sockshop-production_istio-proxy-232a1b6f2894da4eab65e90cef01d9e87c9a65fe6708fdcce27ae629d10b9fe7.log
47 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049345 watch_fd=31 name=/var/log/containers/payment-54c476fb7f-fq75n_sockshop-production_payment-866fef4278ce092377e137fc3fac3c4b04d93124eaad49ca4ada78ae9b24e5bb.log
48 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=787565 watch_fd=32 name=/var/log/containers/pdcsi-node-789r7_kube-system_csi-driver-registrar-428ef51049f3b041e59fb7b58ce3af46285cd21769944a2ab4b6fbf38c53265d.log
49 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=791557 watch_fd=33 name=/var/log/containers/pdcsi-node-789r7_kube-system_gce-pd-driver-18ccb980efba47d0b533fdef9b43d3eb29daeb58de9342e8720272f1ea227975.log
50 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=792102 watch_fd=34 name=/var/log/containers/prometheus-prometheus-node-exporter-kbmwf_default_node-exporter-36653e544214ba40a73dc7d55b7c05a4a58f87f2a4e2ff3d97a3bb3b2285fb72.log
51 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049287 watch_fd=35 name=/var/log/containers/redis-cart-78fdcbd75b-z6cqr_hipster-shop_redis-b1e4626defb23264f425b41fd231978472ef656543a79c9c8de85d422a04465d.log
52 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049382 watch_fd=36 name=/var/log/containers/redis-cart-78fdcbd75b-z6cqr_hipster-shop_redis-exporter-ca3d41d4f15a68ff863cb73a8919faaa8767a4533c4c862e59de7d441d6b2b91.log
53 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049225 watch_fd=37 name=/var/log/containers/shippingservice-5fcdc568c8-xbwkm_hipster-shop_server-31182f68b823002ec719581e83825ac8287570c515ac31ef54d9fccbd2acf375.log
54 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049527 watch_fd=38 name=/var/log/containers/stackdriver-metadata-agent-cluster-level-5765499975-jfwtf_kube-system_metadata-agent-6dd7c92c90fcd2d8d20ad4b4a080c3b96490b4251123699e71c712b195de209c.log
55 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049369 watch_fd=39 name=/var/log/containers/stackdriver-metadata-agent-cluster-level-5765499975-jfwtf_kube-system_metadata-agent-nanny-164fa559e0dc91577ab3feec9d02f3a81ce0907e7da9399a5088b864b6e951a9.log
56 | [2021/07/08 07:46:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1049356 watch_fd=40 name=/var/log/containers/cartservice-67dc579847-gmpvv_hipster-shop_server-5b399fc8b16153d41ea265e2c2fabdd2ce9a4599adae9cfeb7b14a24c9f2c2ac.log
57 | [{"timestamp":"2021-07-08T07:46:26.737645Z","content":"DEMOABILITY DEBUG: AddItem method called\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.737698Z","content":"DEMOABILITY DEBUG: inside try clause\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.737706Z","content":"DEMOABILITY DEBUG: try clause finished correctly\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.737764Z","content":"DEMOABILITY DEBUG: AddItem getTracerCalled\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.737770Z","content":"AddItemAsync called with userId=5a9c308c-ff5a-4a47-8822-391808643738, productId=2ZYFJ3GM2N, quantity=4\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.798800Z","content":"DEMOABILITY DEBUG: GetCart method called\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.798852Z","content":"DEMOABILITY DEBUG: inside try clause\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.798860Z","content":"DEMOABILITY DEBUG: try clause finished correctly\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.798866Z","content":"DEMOABILITY DEBUG: GetCart getTracer called\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.799742Z","content":"GetCartAsync called with userId=2f9cc5f8-204f-49de-b537-af2f1ac8a471\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.812423Z","content":"DEMOABILITY DEBUG: GetCart method called\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.812474Z","content":"DEMOABILITY DEBUG: inside try clause\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.812482Z","content":"DEMOABILITY DEBUG: try clause finished correctly\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.812487Z","content":"DEMOABILITY DEBUG: GetCart getTracer called\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.813357Z","content":"GetCartAsync called with userId=5a9c308c-ff5a-4a47-8822-391808643738\n","stream":"stdout","k8s.pod.name":"cartservice-67dc579847-gmpvv","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"}]
58 | [{"timestamp":"2021-07-08T07:46:26.921217Z","content":"{\"message\":\"[GetQuote] received request\",\"severity\":\"info\",\"timestamp\":\"2021-07-08T07:46:26.920995363Z\"}\n","stream":"stdout","k8s.pod.name":"shippingservice-5fcdc568c8-xbwkm","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"},{"timestamp":"2021-07-08T07:46:26.921317Z","content":"{\"message\":\"[GetQuote] completed request\",\"severity\":\"info\",\"timestamp\":\"2021-07-08T07:46:26.921189875Z\"}\n","stream":"stdout","k8s.pod.name":"shippingservice-5fcdc568c8-xbwkm","k8s.namespace.name":"hipster-shop","k8s.cluster.name":"Onlineboutique"}]
59 | [2021/07/08 07:46:28] [error] [output:http:http.0] ec2-34-243-26-131.eu-west-1.compute.amazonaws.com:9999, HTTP status=400
60 |
61 |
62 | Error: 400 Bad Request
63 | Error: 400 Bad Request
Uri: /e/bix24852/api/v2/logs/ingest
64 |
65 |
66 |
67 | [2021/07/08 07:46:28] [error] [output:http:http.0] ec2-34-243-26-131.eu-west-1.compute.amazonaws.com:9999, HTTP status=400
68 |
69 |
70 | Error: 400 Bad Request
71 | Error: 400 Bad Request
Uri: /e/bix24852/api/v2/logs/ingest
72 |
73 |
74 |
75 | [2021/07/08 07:46:28] [ warn] [engine] failed to flush chunk '1-1625730386.941829702.flb', retry in 8 seconds: task_id=1, input=tail.0 > output=http.0 (out_id=0)
76 | [2021/07/08 07:46:28] [ warn] [engine] failed to flush chunk '1-1625730386.758710294.flb', retry in 10 seconds: task_id=0, input=tail.0 > output=http.0 (out_id=0)
77 | [2021/07/08 07:46:35] [error] [output:http:http.0] ec2-34-243-26-131.eu-west-1.compute.amazonaws.com:9999, HTTP status=400
78 |
79 |
80 | Error: 400 Bad Request
81 | Error: 400 Bad Request
Uri: /e/bix24852/api/v2/logs/ingest
82 |
83 |
84 |
85 | [2021/07/08 07:46:35] [ warn] [engine] chunk '1-1625730386.941829702.flb' cannot be retried: task_id=1, input=tail.0 > output=http.0
86 | [2021/07/08 07:46:37] [error] [output:http:http.0] ec2-34-243-26-131.eu-west-1.compute.amazonaws.com:9999, HTTP status=400
87 |
88 |
89 | Error: 400 Bad Request
90 | Error: 400 Bad Request
Uri: /e/bix24852/api/v2/logs/ingest
91 |
92 |
93 |
94 | [2021/07/08 07:46:37] [ warn] [engine] chunk '1-1625730386.758710294.flb' cannot be retried: task_id=0, input=tail.0 > output=http.0
95 |
--------------------------------------------------------------------------------
/fluentbit/fluentbit_cm.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: v1
2 | data:
3 | parsers.conf: |-
4 | [PARSER]
5 | Name docker
6 | Format json
7 | Time_Key time
8 | Time_Format %Y-%m-%dT%H:%M:%S.%L
9 | custom_parsers.conf: |
10 | [PARSER]
11 | Name docker_no_time
12 | Format json
13 | Time_Keep Off
14 | Time_Key time
15 | Time_Format %Y-%m-%dT%H:%M:%S.%L
16 | fluent-bit.conf: |
17 | [SERVICE]
18 | Flush 1
19 | Daemon Off
20 | Log_Level info
21 | Parsers_File parsers.conf
22 | HTTP_Server On
23 | HTTP_Listen 0.0.0.0
24 | HTTP_Port 2020
25 |
26 | [INPUT]
27 | Name tail
28 | Path /var/log/containers/*.log
29 | Parser docker
30 | Tag kube.*
31 | Mem_Buf_Limit 5MB
32 | Skip_Long_Lines On
33 |
34 | [INPUT]
35 | Name systemd
36 | Tag host.*
37 | Systemd_Filter _SYSTEMD_UNIT=kubelet.service
38 | Read_From_Tail On*
39 |
40 | [FILTER]
41 | Name modify
42 | Match *
43 | Rename message content
44 | Rename log content
45 |
46 | [FILTER]
47 | Name kubernetes
48 | Match kube.*
49 | Merge_Log On
50 | Merge_Log_Trim On
51 | Labels Off
52 | Annotations Off
53 | K8S-Logging.Parser Off
54 | K8S-Logging.Exclude Off
55 |
56 |
57 | [FILTER]
58 | Name nest
59 | Match kube.*
60 | Operation lift
61 | Nested_under kubernetes
62 | Add_prefix kubernetes_
63 |
64 | [FILTER]
65 | Name grep
66 | Match kube.*
67 | Exclude kubernetes_container_name fluent-bit
68 |
69 | [FILTER]
70 | Name modify
71 | Match kube.*
72 | Rename log content
73 | Rename kubernetes_pod_name k8s.pod.name
74 | Rename kubernetes_namespace_name k8s.namespace.name
75 | Remove kubernetes_container_image
76 | Remove kubernetes_docker_id
77 | Remove kubernetes_container_name
78 | Remove kubernetes_pod_id
79 | Remove kubernetes_host
80 | Remove time
81 | Remove kubernetes_container_hash
82 | Add k8s.cluster.name Onlineboutique
83 |
84 | [FILTER]
85 | Name throttle
86 | Match kube.*
87 | Rate 5000
88 | Window 5
89 | Print_Status true
90 | Interval 30s
91 |
92 | [OUTPUT]
93 | Name http
94 | Match kube.*
95 | host
96 | port 9999
97 | URI /e/bix24852/api/v2/logs/ingest
98 | header Authorization Api-Token
99 | header application/json; charset=utf-8
100 | Format json
101 | allow_duplicated_headers false
102 | Json_date_key timestamp
103 | Json_date_format iso8601
104 | tls On
105 | tls.verify Off
106 | Retry_Limit false
107 |
108 | [OUTPUT]
109 | Name http
110 | Match kube.*
111 | host
112 | port 8080
113 | URI /log
114 | header application/json; charset=utf-8
115 | Format json
116 | allow_duplicated_headers false
117 | Json_date_key timestamp
118 | Json_date_format iso8601
119 | Retry_Limit false
120 |
121 | [OUTPUT]
122 | Name stdout
123 | Match kube.*
124 | Format json
125 | Json_date_key timestamp
126 | Json_date_format iso8601
127 |
128 |
129 | kind: ConfigMap
130 | metadata:
131 | annotations:
132 | meta.helm.sh/release-name: fluent-bit
133 | meta.helm.sh/release-namespace: default
134 | labels:
135 | app.kubernetes.io/instance: fluent-bit
136 | app.kubernetes.io/managed-by: Helm
137 | app.kubernetes.io/name: fluent-bit
138 | app.kubernetes.io/version: 1.7.9
139 | helm.sh/chart: fluent-bit-0.15.15
140 | name: fluent-bit
141 | namespace: default
142 |
143 |
--------------------------------------------------------------------------------
/fluentbit/fluentbit_configmap_loki.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: v1
2 | data:
3 | fluent-bit.conf: |-
4 | [SERVICE]
5 | HTTP_Server On
6 | HTTP_Listen 0.0.0.0
7 | HTTP_PORT 2020
8 | Flush 1
9 | Daemon Off
10 | Log_Level warn
11 | Parsers_File parsers.conf
12 |
13 | [INPUT]
14 | Name tail
15 | Tag kube.*
16 | Path /var/log/containers/*.log
17 | Parser docker
18 | DB /run/fluent-bit/flb_kube.db
19 | Mem_Buf_Limit 5MB
20 |
21 | [FILTER]
22 | Name kubernetes
23 | Match kube.*
24 | Kube_URL https://kubernetes.default.svc:443
25 | Merge_Log On
26 | K8S-Logging.Exclude Off
27 | K8S-Logging.Parser Off
28 |
29 |
30 |
31 | [OUTPUT]
32 | Name grafana-loki
33 | Match *
34 | Url http://loki:3100/api/prom/push
35 | TenantID ""
36 | BatchWait 1
37 | BatchSize 1048576
38 | Labels {job="fluent-bit",stream=$stream }
39 | RemoveKeys kubernetes,stream
40 | AutoKubernetesLabels true
41 | LabelMapPath /fluent-bit/etc/labelmap.json
42 | LineFormat json
43 | LogLevel warn
44 |
45 |
46 |
47 | labelmap.json: |-
48 | {
49 | "kubernetes": {
50 | "container_name": "container",
51 | "host": "node",
52 | "labels": {
53 | "app": "app",
54 | "release": "release"
55 | },
56 | "namespace_name": "namespace",
57 | "pod_name": "instance"
58 | },
59 | "stream": "stream"
60 | }
61 | parsers.conf: |-
62 | [PARSER]
63 | Name docker
64 | Format json
65 | Time_Key time
66 | Time_Format %Y-%m-%dT%H:%M:%S.%L
67 | kind: ConfigMap
68 | metadata:
69 | annotations:
70 | meta.helm.sh/release-name: dynatrace
71 | meta.helm.sh/release-namespace: default
72 | labels:
73 | app: fluent-bit-loki
74 | chart: fluent-bit-2.2.0
75 | heritage: Helm
76 | name: loki-fluent-bit-loki
77 | namespace: default
78 |
79 |
--------------------------------------------------------------------------------
/fluentbit/fluentbit_ds.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: DaemonSet
3 | metadata:
4 | annotations:
5 | deprecated.daemonset.template.generation: "1"
6 | meta.helm.sh/release-name: fluent-bit
7 | meta.helm.sh/release-namespace: default
8 | generation: 1
9 | labels:
10 | app.kubernetes.io/instance: fluent-bit
11 | app.kubernetes.io/managed-by: Helm
12 | app.kubernetes.io/name: fluent-bit
13 | app.kubernetes.io/version: 1.8.0
14 | helm.sh/chart: fluent-bit-0.15.15
15 | name: fluent-bit
16 | namespace: default
17 | resourceVersion: "103328519"
18 | spec:
19 | revisionHistoryLimit: 10
20 | selector:
21 | matchLabels:
22 | app.kubernetes.io/instance: fluent-bit
23 | app.kubernetes.io/name: fluent-bit
24 | template:
25 | metadata:
26 | annotations:
27 | checksum/config: 76ef40fe3345c703b045f0d90227e9013da6df61c519d0d17c61e7d5c582a705
28 | checksum/luascripts: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
29 | creationTimestamp: null
30 | labels:
31 | app.kubernetes.io/instance: fluent-bit
32 | app.kubernetes.io/name: fluent-bit
33 | spec:
34 | containers:
35 | - image: fluent/fluent-bit:1.8.0
36 | imagePullPolicy: Always
37 | livenessProbe:
38 | failureThreshold: 3
39 | httpGet:
40 | path: /
41 | port: http
42 | scheme: HTTP
43 | periodSeconds: 10
44 | successThreshold: 1
45 | timeoutSeconds: 1
46 | name: fluent-bit
47 | ports:
48 | - containerPort: 2020
49 | name: http
50 | protocol: TCP
51 | readinessProbe:
52 | failureThreshold: 3
53 | httpGet:
54 | path: /
55 | port: http
56 | scheme: HTTP
57 | periodSeconds: 10
58 | successThreshold: 1
59 | timeoutSeconds: 1
60 | resources: {}
61 | securityContext: {}
62 | terminationMessagePath: /dev/termination-log
63 | terminationMessagePolicy: File
64 | volumeMounts:
65 | - mountPath: /fluent-bit/etc/fluent-bit.conf
66 | name: config
67 | subPath: fluent-bit.conf
68 | - mountPath: /fluent-bit/etc/custom_parsers.conf
69 | name: config
70 | subPath: custom_parsers.conf
71 | - mountPath: /var/log
72 | name: varlog
73 | - mountPath: /var/lib/docker/containers
74 | name: varlibdockercontainers
75 | readOnly: true
76 | - mountPath: /etc/machine-id
77 | name: etcmachineid
78 | readOnly: true
79 | dnsPolicy: ClusterFirst
80 | restartPolicy: Always
81 | schedulerName: default-scheduler
82 | securityContext: {}
83 | serviceAccount: fluent-bit
84 | serviceAccountName: fluent-bit
85 | terminationGracePeriodSeconds: 30
86 | volumes:
87 | - configMap:
88 | defaultMode: 420
89 | name: fluent-bit
90 | name: config
91 | - hostPath:
92 | path: /var/log
93 | type: ""
94 | name: varlog
95 | - hostPath:
96 | path: /var/lib/docker/containers
97 | type: ""
98 | name: varlibdockercontainers
99 | - hostPath:
100 | path: /etc/machine-id
101 | type: File
102 | name: etcmachineid
103 | updateStrategy:
104 | rollingUpdate:
105 | maxUnavailable: 1
106 | type: RollingUpdate
107 |
--------------------------------------------------------------------------------
/fluentbit/fluentbit_loki_ds.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: DaemonSet
3 | metadata:
4 | annotations:
5 | deprecated.daemonset.template.generation: "1"
6 | meta.helm.sh/release-name: loki
7 | meta.helm.sh/release-namespace: default
8 | generation: 1
9 | labels:
10 | app: fluent-bit-loki
11 | app.kubernetes.io/managed-by: Helm
12 | chart: fluent-bit-2.2.0
13 | heritage: Helm
14 | release: loki
15 | name: loki-fluent-bit-loki
16 | namespace: default
17 | resourceVersion: "103336341"
18 | spec:
19 | revisionHistoryLimit: 10
20 | selector:
21 | matchLabels:
22 | app: fluent-bit-loki
23 | release: loki
24 | template:
25 | metadata:
26 | annotations:
27 | checksum/config: 767253d8b1d65e24434729e4133daf08838091982746b2392ecbeef31fc96dd9
28 | prometheus.io/path: /api/v1/metrics/prometheus
29 | prometheus.io/port: "2020"
30 | prometheus.io/scrape: "true"
31 | creationTimestamp: null
32 | labels:
33 | app: fluent-bit-loki
34 | release: loki
35 | spec:
36 | affinity: {}
37 | containers:
38 | - image: grafana/fluent-bit-plugin-loki:2.1.0-amd64
39 | imagePullPolicy: IfNotPresent
40 | name: fluent-bit-loki
41 | ports:
42 | - containerPort: 2020
43 | name: http-metrics
44 | protocol: TCP
45 | resources:
46 | limits:
47 | memory: 100Mi
48 | requests:
49 | cpu: 100m
50 | memory: 100Mi
51 | terminationMessagePath: /dev/termination-log
52 | terminationMessagePolicy: File
53 | volumeMounts:
54 | - mountPath: /fluent-bit/etc
55 | name: config
56 | - mountPath: /run/fluent-bit
57 | name: run
58 | - mountPath: /var/log
59 | name: varlog
60 | - mountPath: /var/lib/docker/containers
61 | name: varlibdockercontainers
62 | readOnly: true
63 | dnsPolicy: ClusterFirst
64 | restartPolicy: Always
65 | schedulerName: default-scheduler
66 | securityContext: {}
67 | serviceAccount: loki-fluent-bit-loki
68 | serviceAccountName: loki-fluent-bit-loki
69 | terminationGracePeriodSeconds: 10
70 | tolerations:
71 | - effect: NoSchedule
72 | key: node-role.kubernetes.io/master
73 | volumes:
74 | - configMap:
75 | defaultMode: 420
76 | name: loki-fluent-bit-loki
77 | name: config
78 | - hostPath:
79 | path: /run/fluent-bit
80 | type: ""
81 | name: run
82 | - hostPath:
83 | path: /var/log
84 | type: ""
85 | name: varlog
86 | - hostPath:
87 | path: /var/lib/docker/containers
88 | type: ""
89 | name: varlibdockercontainers
90 | updateStrategy:
91 | rollingUpdate:
92 | maxUnavailable: 1
93 | type: RollingUpdate
94 |
95 |
--------------------------------------------------------------------------------
/grafana/ingress.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: networking.k8s.io/v1beta1
2 | kind: Ingress
3 | metadata:
4 | name: grafana-ingress
5 | spec:
6 | backend:
7 | serviceName: prometheus-grafana
8 | servicePort: 3000
9 |
--------------------------------------------------------------------------------
/hipstershop/k8s-manifest.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: Deployment
3 | metadata:
4 | name: emailservice
5 | spec:
6 | selector:
7 | matchLabels:
8 | app: emailservice
9 | product: hipster-shop
10 | template:
11 | metadata:
12 | labels:
13 | app: emailservice
14 | product: hipster-shop
15 | spec:
16 | terminationGracePeriodSeconds: 5
17 | containers:
18 | - name: server
19 | image: gcr.io/dynatrace-demoability/emailservice:9e830c2
20 | ports:
21 | - containerPort: 8080
22 | env:
23 | - name: PORT
24 | value: "8080"
25 | # - name: DISABLE_TRACING
26 | # value: "1"
27 | - name: DISABLE_PROFILER
28 | value: "1"
29 | readinessProbe:
30 | periodSeconds: 5
31 | exec:
32 | command: ["/bin/grpc_health_probe", "-addr=:8080"]
33 | livenessProbe:
34 | periodSeconds: 5
35 | exec:
36 | command: ["/bin/grpc_health_probe", "-addr=:8080"]
37 | resources:
38 | requests:
39 | cpu: 100m
40 | memory: 64Mi
41 | limits:
42 | cpu: 200m
43 | memory: 128Mi
44 | ---
45 | apiVersion: v1
46 | kind: Service
47 | metadata:
48 | name: emailservice
49 | spec:
50 | type: ClusterIP
51 | selector:
52 | app: emailservice
53 | ports:
54 | - name: grpc
55 | port: 5000
56 | targetPort: 8080
57 | ---
58 | apiVersion: apps/v1
59 | kind: Deployment
60 | metadata:
61 | name: checkoutservice
62 | spec:
63 | selector:
64 | matchLabels:
65 | app: checkoutservice
66 | product: hipster-shop
67 | template:
68 | metadata:
69 | labels:
70 | app: checkoutservice
71 | product: hipster-shop
72 | spec:
73 | containers:
74 | - name: server
75 | image: gcr.io/dynatrace-demoability/checkoutservice:9e830c2
76 | ports:
77 | - containerPort: 5050
78 | readinessProbe:
79 | exec:
80 | command: ["/bin/grpc_health_probe", "-addr=:5050"]
81 | livenessProbe:
82 | exec:
83 | command: ["/bin/grpc_health_probe", "-addr=:5050"]
84 | env:
85 | - name: PORT
86 | value: "5050"
87 | - name: PRODUCT_CATALOG_SERVICE_ADDR
88 | value: "productcatalogservice:3550"
89 | - name: SHIPPING_SERVICE_ADDR
90 | value: "shippingservice:50051"
91 | - name: PAYMENT_SERVICE_ADDR
92 | value: "paymentservice:50051"
93 | - name: EMAIL_SERVICE_ADDR
94 | value: "emailservice:5000"
95 | - name: CURRENCY_SERVICE_ADDR
96 | value: "currencyservice:7000"
97 | - name: CART_SERVICE_ADDR
98 | value: "cartservice:7070"
99 | # - name: DISABLE_STATS
100 | # value: "1"
101 | # - name: DISABLE_TRACING
102 | # value: "1"
103 | # - name: DISABLE_PROFILER
104 | # value: "1"
105 | # - name: JAEGER_SERVICE_ADDR
106 | # value: "jaeger-collector:14268"
107 | resources:
108 | requests:
109 | cpu: 100m
110 | memory: 64Mi
111 | limits:
112 | cpu: 200m
113 | memory: 128Mi
114 | ---
115 | apiVersion: v1
116 | kind: Service
117 | metadata:
118 | name: checkoutservice
119 | spec:
120 | type: ClusterIP
121 | selector:
122 | app: checkoutservice
123 | ports:
124 | - name: grpc
125 | port: 5050
126 | targetPort: 5050
127 | ---
128 | apiVersion: apps/v1
129 | kind: Deployment
130 | metadata:
131 | name: recommendationservice
132 | spec:
133 | selector:
134 | matchLabels:
135 | app: recommendationservice
136 | product: hipster-shop
137 | template:
138 | metadata:
139 | labels:
140 | app: recommendationservice
141 | product: hipster-shop
142 | spec:
143 | terminationGracePeriodSeconds: 5
144 | # imagePullsecrets required for perform2019????
145 | imagePullSecrets:
146 | - name: demo-registry
147 | containers:
148 | - name: server
149 | image: gcr.io/dynatrace-demoability/recommendationservice:9e830c2
150 | ports:
151 | - containerPort: 8080
152 | readinessProbe:
153 | periodSeconds: 5
154 | exec:
155 | command: ["/bin/grpc_health_probe", "-addr=:8080"]
156 | livenessProbe:
157 | periodSeconds: 5
158 | exec:
159 | command: ["/bin/grpc_health_probe", "-addr=:8080"]
160 | env:
161 | - name: PORT
162 | value: "8080"
163 | - name: PRODUCT_CATALOG_SERVICE_ADDR
164 | value: "productcatalogservice:3550"
165 | # - name: DISABLE_TRACING
166 | # value: "1"
167 | # - name: DISABLE_PROFILER
168 | # value: "1"
169 | # - name: DISABLE_DEBUGGER
170 | # value: "1"
171 | resources:
172 | requests:
173 | cpu: 100m
174 | memory: 220Mi
175 | limits:
176 | cpu: 200m
177 | memory: 450Mi
178 | ---
179 | apiVersion: v1
180 | kind: Service
181 | metadata:
182 | name: recommendationservice
183 | spec:
184 | type: ClusterIP
185 | selector:
186 | app: recommendationservice
187 | ports:
188 | - name: grpc
189 | port: 8080
190 | targetPort: 8080
191 | ---
192 | apiVersion: apps/v1
193 | kind: Deployment
194 | metadata:
195 | name: frontend
196 | spec:
197 | selector:
198 | matchLabels:
199 | app: frontend
200 | product: hipster-shop
201 | template:
202 | metadata:
203 | labels:
204 | app: frontend
205 | product: hipster-shop
206 | annotations:
207 | sidecar.istio.io/rewriteAppHTTPProbers: "true"
208 | spec:
209 | containers:
210 | - name: server
211 | # both look good
212 | #image: registry.lab.dynatrace.org/cloudplatform/hipster-shop-frontend
213 | image: gcr.io/dynatrace-demoability/frontend:9e830c2
214 | ports:
215 | - containerPort: 8080
216 | readinessProbe:
217 | initialDelaySeconds: 10
218 | httpGet:
219 | path: "/_healthz"
220 | port: 8080
221 | httpHeaders:
222 | - name: "Cookie"
223 | value: "shop_session-id=x-readiness-probe"
224 | livenessProbe:
225 | initialDelaySeconds: 10
226 | httpGet:
227 | path: "/_healthz"
228 | port: 8080
229 | httpHeaders:
230 | - name: "Cookie"
231 | value: "shop_session-id=x-liveness-probe"
232 | env:
233 | - name: PORT
234 | value: "8080"
235 | - name: PRODUCT_CATALOG_SERVICE_ADDR
236 | value: "productcatalogservice:3550"
237 | - name: CURRENCY_SERVICE_ADDR
238 | value: "currencyservice:7000"
239 | - name: CART_SERVICE_ADDR
240 | value: "cartservice:7070"
241 | - name: RECOMMENDATION_SERVICE_ADDR
242 | value: "recommendationservice:8080"
243 | - name: SHIPPING_SERVICE_ADDR
244 | value: "shippingservice:50051"
245 | - name: CHECKOUT_SERVICE_ADDR
246 | value: "checkoutservice:5050"
247 | - name: AD_SERVICE_ADDR
248 | value: "adservice:9555"
249 | - name: ENV_PLATFORM
250 | value: "gcp"
251 | # - name: DISABLE_TRACING
252 | # value: "1"
253 | # - name: DISABLE_PROFILER
254 | # value: "1"
255 | # - name: JAEGER_SERVICE_ADDR
256 | # value: "jaeger-collector:14268"
257 | resources:
258 | requests:
259 | cpu: 100m
260 | memory: 64Mi
261 | limits:
262 | cpu: 200m
263 | memory: 128Mi
264 | ---
265 | apiVersion: v1
266 | kind: Service
267 | metadata:
268 | name: frontend
269 | spec:
270 | type: ClusterIP
271 | selector:
272 | app: frontend
273 | ports:
274 | - name: http
275 | port: 80
276 | targetPort: 8080
277 | ---
278 | apiVersion: v1
279 | kind: Service
280 | metadata:
281 | name: frontend-external
282 | spec:
283 | type: LoadBalancer
284 | selector:
285 | app: frontend
286 | ports:
287 | - name: http
288 | port: 80
289 | targetPort: 8080
290 | ---
291 | apiVersion: apps/v1
292 | kind: Deployment
293 | metadata:
294 | name: paymentservice
295 | spec:
296 | selector:
297 | matchLabels:
298 | app: paymentservice
299 | product: hipster-shop
300 | template:
301 | metadata:
302 | labels:
303 | app: paymentservice
304 | product: hipster-shop
305 | spec:
306 | terminationGracePeriodSeconds: 5
307 | containers:
308 | - name: server
309 | image: gcr.io/dynatrace-demoability/paymentservice:9e830c2
310 | ports:
311 | - containerPort: 50051
312 | env:
313 | - name: PORT
314 | value: "50051"
315 | readinessProbe:
316 | exec:
317 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
318 | livenessProbe:
319 | exec:
320 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
321 | resources:
322 | requests:
323 | cpu: 100m
324 | memory: 64Mi
325 | limits:
326 | cpu: 200m
327 | memory: 128Mi
328 | ---
329 | apiVersion: v1
330 | kind: Service
331 | metadata:
332 | name: paymentservice
333 | spec:
334 | type: ClusterIP
335 | selector:
336 | app: paymentservice
337 | ports:
338 | - name: grpc
339 | port: 50051
340 | targetPort: 50051
341 | ---
342 | apiVersion: apps/v1
343 | kind: Deployment
344 | metadata:
345 | name: productcatalogservice
346 | spec:
347 | selector:
348 | matchLabels:
349 | app: productcatalogservice
350 | product: hipster-shop
351 | template:
352 | metadata:
353 | labels:
354 | app: productcatalogservice
355 | product: hipster-shop
356 | spec:
357 | terminationGracePeriodSeconds: 5
358 | containers:
359 | - name: server
360 | # image: registry.lab.dynatrace.org/cloudplatform/hipster-shop-productcatalogservice
361 | image: gcr.io/dynatrace-demoability/productcatalogservice:9e830c2
362 | ports:
363 | - containerPort: 3550
364 | env:
365 | - name: PORT
366 | value: "3550"
367 | # - name: DISABLE_STATS
368 | # value: "1"
369 | # - name: DISABLE_TRACING
370 | # value: "1"
371 | # - name: DISABLE_PROFILER
372 | # value: "1"
373 | # - name: JAEGER_SERVICE_ADDR
374 | # value: "jaeger-collector:14268"
375 | readinessProbe:
376 | exec:
377 | command: ["/bin/grpc_health_probe", "-addr=:3550"]
378 | livenessProbe:
379 | exec:
380 | command: ["/bin/grpc_health_probe", "-addr=:3550"]
381 | resources:
382 | requests:
383 | cpu: 100m
384 | memory: 64Mi
385 | limits:
386 | cpu: 200m
387 | memory: 128Mi
388 | ---
389 | apiVersion: v1
390 | kind: Service
391 | metadata:
392 | name: productcatalogservice
393 | spec:
394 | type: ClusterIP
395 | selector:
396 | app: productcatalogservice
397 | ports:
398 | - name: grpc
399 | port: 3550
400 | targetPort: 3550
401 | ---
402 | apiVersion: apps/v1
403 | kind: Deployment
404 | metadata:
405 | name: cartservice
406 | spec:
407 | selector:
408 | matchLabels:
409 | app: cartservice
410 | product: hipster-shop
411 | template:
412 | metadata:
413 | labels:
414 | app: cartservice
415 | product: hipster-shop
416 | spec:
417 | terminationGracePeriodSeconds: 5
418 | containers:
419 | - name: server
420 | image: gcr.io/dynatrace-demoability/cartservice:9e830c2
421 | ports:
422 | - containerPort: 7070
423 | env:
424 | - name: REDIS_ADDR
425 | value: "redis-cart:6379"
426 | - name: PORT
427 | value: "7070"
428 | - name: LISTEN_ADDR
429 | value: "0.0.0.0"
430 | resources:
431 | requests:
432 | cpu: 200m
433 | memory: 64Mi
434 | limits:
435 | cpu: 300m
436 | memory: 128Mi
437 | readinessProbe:
438 | initialDelaySeconds: 15
439 | exec:
440 | command: ["/bin/grpc_health_probe", "-addr=:7070", "-rpc-timeout=5s"]
441 | livenessProbe:
442 | initialDelaySeconds: 15
443 | periodSeconds: 10
444 | exec:
445 | command: ["/bin/grpc_health_probe", "-addr=:7070", "-rpc-timeout=5s"]
446 | ---
447 | apiVersion: v1
448 | kind: Service
449 | metadata:
450 | name: cartservice
451 | spec:
452 | type: ClusterIP
453 | selector:
454 | app: cartservice
455 | ports:
456 | - name: grpc
457 | port: 7070
458 | targetPort: 7070
459 | ---
460 | apiVersion: apps/v1
461 | kind: Deployment
462 | metadata:
463 | name: loadgenerator
464 | spec:
465 | selector:
466 | matchLabels:
467 | app: loadgenerator
468 | product: hipster-shop
469 | replicas: 1
470 | template:
471 | metadata:
472 | labels:
473 | app: loadgenerator
474 | product: hipster-shop
475 | annotations:
476 | sidecar.istio.io/rewriteAppHTTPProbers: "true"
477 | spec:
478 | terminationGracePeriodSeconds: 5
479 | restartPolicy: Always
480 | containers:
481 | - name: main
482 | image: gcr.io/dynatrace-demoability/loadgenerator:9e830c2
483 | env:
484 | - name: FRONTEND_ADDR
485 | value: "frontend:80"
486 | - name: USERS
487 | value: "10"
488 | resources:
489 | requests:
490 | cpu: 300m
491 | memory: 256Mi
492 | limits:
493 | cpu: 500m
494 | memory: 512Mi
495 | ---
496 | apiVersion: apps/v1
497 | kind: Deployment
498 | metadata:
499 | name: currencyservice
500 | spec:
501 | selector:
502 | matchLabels:
503 | app: currencyservice
504 | product: hipster-shop
505 | template:
506 | metadata:
507 | labels:
508 | app: currencyservice
509 | product: hipster-shop
510 | spec:
511 | terminationGracePeriodSeconds: 5
512 | containers:
513 | - name: server
514 | image: gcr.io/dynatrace-demoability/currencyservice:9e830c2
515 | ports:
516 | - name: grpc
517 | containerPort: 7000
518 | env:
519 | - name: PORT
520 | value: "7000"
521 | # - name: DISABLE_TRACING
522 | # value: "1"
523 | # - name: DISABLE_PROFILER
524 | # value: "1"
525 | # - name: DISABLE_DEBUGGER
526 | # value: "1"
527 | readinessProbe:
528 | exec:
529 | command: ["/bin/grpc_health_probe", "-addr=:7000"]
530 | livenessProbe:
531 | exec:
532 | command: ["/bin/grpc_health_probe", "-addr=:7000"]
533 | resources:
534 | requests:
535 | cpu: 100m
536 | memory: 64Mi
537 | limits:
538 | cpu: 200m
539 | memory: 128Mi
540 | ---
541 | apiVersion: v1
542 | kind: Service
543 | metadata:
544 | name: currencyservice
545 | spec:
546 | type: ClusterIP
547 | selector:
548 | app: currencyservice
549 | ports:
550 | - name: grpc
551 | port: 7000
552 | targetPort: 7000
553 | ---
554 | apiVersion: apps/v1
555 | kind: Deployment
556 | metadata:
557 | name: shippingservice
558 | spec:
559 | selector:
560 | matchLabels:
561 | app: shippingservice
562 | product: hipster-shop
563 | template:
564 | metadata:
565 | labels:
566 | app: shippingservice
567 | product: hipster-shop
568 | spec:
569 | containers:
570 | - name: server
571 | image: gcr.io/dynatrace-demoability/shippingservice:9e830c2
572 | ports:
573 | - containerPort: 50051
574 | env:
575 | - name: PORT
576 | value: "50051"
577 | # - name: DISABLE_STATS
578 | # value: "1"
579 | # - name: DISABLE_TRACING
580 | # value: "1"
581 | # - name: DISABLE_PROFILER
582 | # value: "1"
583 | # - name: JAEGER_SERVICE_ADDR
584 | # value: "jaeger-collector:14268"
585 | readinessProbe:
586 | periodSeconds: 5
587 | exec:
588 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
589 | livenessProbe:
590 | exec:
591 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
592 | resources:
593 | requests:
594 | cpu: 100m
595 | memory: 64Mi
596 | limits:
597 | cpu: 200m
598 | memory: 128Mi
599 | ---
600 | apiVersion: v1
601 | kind: Service
602 | metadata:
603 | name: shippingservice
604 | spec:
605 | type: ClusterIP
606 | selector:
607 | app: shippingservice
608 | ports:
609 | - name: grpc
610 | port: 50051
611 | targetPort: 50051
612 | ---
613 | apiVersion: apps/v1
614 | kind: Deployment
615 | metadata:
616 | name: redis-cart
617 | spec:
618 | selector:
619 | matchLabels:
620 | app: redis-cart
621 | product: hipster-shop
622 | template:
623 | metadata:
624 | labels:
625 | app: redis-cart
626 | product: hipster-shop
627 | annotations: # here we annotate the deployment to be scraped by dynatrace
628 | prometheus.io/port: '9121'
629 | prometheus.io/scrape: 'true'
630 | metrics.dynatrace.com/port: '9121'
631 | metrics.dynatrace.com/scrape: 'true'
632 | spec:
633 | containers:
634 | - name: redis
635 | image: redis:alpine
636 | ports:
637 | - containerPort: 6379
638 | readinessProbe:
639 | periodSeconds: 5
640 | tcpSocket:
641 | port: 6379
642 | livenessProbe:
643 | periodSeconds: 5
644 | tcpSocket:
645 | port: 6379
646 | volumeMounts:
647 | - mountPath: /data
648 | name: redis-data
649 | resources:
650 | limits:
651 | memory: 256Mi
652 | cpu: 125m
653 | requests:
654 | cpu: 70m
655 | memory: 200Mi
656 | - name: redis-exporter
657 | image: 'oliver006/redis_exporter:latest'
658 | resources:
659 | requests:
660 | cpu: 100m
661 | memory: 100Mi
662 | ports:
663 | - containerPort: 9121
664 | protocol: TCP
665 | volumes:
666 | - name: redis-data
667 | emptyDir: {}
668 | ---
669 | apiVersion: v1
670 | kind: Service
671 | metadata:
672 | name: redis-cart
673 | spec:
674 | type: ClusterIP
675 | selector:
676 | app: redis-cart
677 | ports:
678 | - name: redis
679 | port: 6379
680 | targetPort: 6379
681 | ---
682 | apiVersion: apps/v1
683 | kind: Deployment
684 | metadata:
685 | name: adservice
686 | spec:
687 | selector:
688 | matchLabels:
689 | app: adservice
690 | product: hipster-shop
691 | template:
692 | metadata:
693 | labels:
694 | app: adservice
695 | product: hipster-shop
696 | spec:
697 | terminationGracePeriodSeconds: 5
698 | containers:
699 | - name: server
700 | image: gcr.io/dynatrace-demoability/adservice:9e830c2
701 | ports:
702 | - containerPort: 9555
703 | env:
704 | - name: PORT
705 | value: "9555"
706 | # - name: DISABLE_STATS
707 | # value: "1"
708 | # - name: DISABLE_TRACING
709 | # value: "1"
710 | #- name: JAEGER_SERVICE_ADDR
711 | # value: "jaeger-collector:14268"
712 | resources:
713 | requests:
714 | cpu: 200m
715 | memory: 180Mi
716 | limits:
717 | cpu: 300m
718 | memory: 300Mi
719 | readinessProbe:
720 | initialDelaySeconds: 20
721 | periodSeconds: 15
722 | exec:
723 | command: ["/bin/grpc_health_probe", "-addr=:9555"]
724 | livenessProbe:
725 | initialDelaySeconds: 20
726 | periodSeconds: 15
727 | exec:
728 | command: ["/bin/grpc_health_probe", "-addr=:9555"]
729 | ---
730 | apiVersion: v1
731 | kind: Service
732 | metadata:
733 | name: adservice
734 | spec:
735 | type: ClusterIP
736 | selector:
737 | app: adservice
738 | ports:
739 | - name: grpc
740 | port: 9555
741 | targetPort: 9555
742 | ---
743 |
--------------------------------------------------------------------------------
/hipstershop/paymentservice-fix-the-fix.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: Deployment
3 | metadata:
4 | name: paymentservice
5 | namespace: hipster-shop
6 | spec:
7 | selector:
8 | matchLabels:
9 | app: paymentservice
10 | product: hipster-shop
11 | template:
12 | metadata:
13 | labels:
14 | app: paymentservice
15 | product: hipster-shop
16 | spec:
17 | terminationGracePeriodSeconds: 5
18 | containers:
19 | - name: server
20 | # image: gcr.io/google-samples/microservices-demo/paymentservice:v0.2.0
21 | image: gcr.io/dynatrace-demoability/paymentservice:9e830c2
22 | ports:
23 | - containerPort: 50051
24 | env:
25 | - name: PORT
26 | value: "50051"
27 | readinessProbe:
28 | exec:
29 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
30 | livenessProbe:
31 | exec:
32 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
33 | resources:
34 | requests:
35 | cpu: 100m
36 | memory: 64Mi
37 | limits:
38 | cpu: 200m
39 | memory: 128Mi
40 |
--------------------------------------------------------------------------------
/hipstershop/paymentservice-fix.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: Deployment
3 | metadata:
4 | name: paymentservice
5 | namespace: hipster-shop
6 | spec:
7 | selector:
8 | matchLabels:
9 | app: paymentservice
10 | product: hipster-shop
11 | template:
12 | metadata:
13 | labels:
14 | app: paymentservice
15 | product: hipster-shop
16 | spec:
17 | terminationGracePeriodSeconds: 5
18 | containers:
19 | - name: server
20 | # image: gcr.io/google-samples/microservices-demo/paymentservice:v0.2.0
21 | image: gcr.io/dynatrace-demoability/paymentservice:999999-image-tag-not-on-the-registry
22 | ports:
23 | - containerPort: 50051
24 | env:
25 | - name: PORT
26 | value: "50051"
27 | readinessProbe:
28 | exec:
29 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
30 | livenessProbe:
31 | exec:
32 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
33 | resources:
34 | requests:
35 | cpu: 100m
36 | memory: 64Mi
37 | limits:
38 | cpu: 200m
39 | memory: 128Mi
40 |
--------------------------------------------------------------------------------
/hipstershop/paymentservice-new.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: Deployment
3 | metadata:
4 | name: paymentservice
5 | namespace: hipster-shop
6 | spec:
7 | selector:
8 | matchLabels:
9 | app: paymentservice
10 | product: hipster-shop
11 | template:
12 | metadata:
13 | labels:
14 | app: paymentservice
15 | product: hipster-shop
16 | spec:
17 | terminationGracePeriodSeconds: 5
18 | containers:
19 | - name: server
20 | # image: gcr.io/google-samples/microservices-demo/paymentservice:v0.2.0
21 | image: gcr.io/dynatrace-demoability/paymentservice:9e830c2
22 | ports:
23 | - containerPort: 50051
24 | env:
25 | - name: PORT
26 | value: "50051"
27 | - name: MEMORYLEAK
28 | value: "3"
29 | readinessProbe:
30 | exec:
31 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
32 | livenessProbe:
33 | exec:
34 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
35 | resources:
36 | requests:
37 | cpu: 100m
38 | memory: 64Mi
39 | limits:
40 | cpu: 200m
41 | memory: 128Mi
42 |
--------------------------------------------------------------------------------
/hipstershop/paymentservice.yaml:
--------------------------------------------------------------------------------
1 | apiVersion: apps/v1
2 | kind: Deployment
3 | metadata:
4 | name: paymentservice
5 | namespace: hipster-shop
6 | spec:
7 | selector:
8 | matchLabels:
9 | app: paymentservice
10 | product: hipster-shop
11 | template:
12 | metadata:
13 | labels:
14 | app: paymentservice
15 | product: hipster-shop
16 | spec:
17 | terminationGracePeriodSeconds: 5
18 | containers:
19 | - name: server
20 | image: gcr.io/dynatrace-demoability/paymentservice:9e830c2
21 | ports:
22 | - containerPort: 50051
23 | env:
24 | - name: PORT
25 | value: "50051"
26 | readinessProbe:
27 | exec:
28 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
29 | livenessProbe:
30 | exec:
31 | command: ["/bin/grpc_health_probe", "-addr=:50051"]
32 | resources:
33 | requests:
34 | cpu: 100m
35 | memory: 64Mi
36 | limits:
37 | cpu: 200m
38 | memory: 128Mi
--------------------------------------------------------------------------------
/hipstershop/setup.sh:
--------------------------------------------------------------------------------
1 | echo Create namespace for sample app
2 | kubectl create ns hipster-shop
3 | kubectl -n hipster-shop create rolebinding default-view --clusterrole=view --serviceaccount=hipster-shop:default
4 | echo Deploy sample app
5 | kubectl -n hipster-shop apply -f k8s-manifest.yaml
6 |
--------------------------------------------------------------------------------
/hipstershop/teardown.sh:
--------------------------------------------------------------------------------
1 | #!/bin/sh
2 |
3 | kubectl -n hipster-shop delete -f k8s-manifest.yaml
4 | kubectl delete ns hipster-shop
5 |
--------------------------------------------------------------------------------
/image/addsource.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/addsource.PNG
--------------------------------------------------------------------------------
/image/datasource.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/datasource.PNG
--------------------------------------------------------------------------------
/image/explore.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/explore.png
--------------------------------------------------------------------------------
/image/fluentbit.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/fluentbit.png
--------------------------------------------------------------------------------
/image/getcm.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/getcm.PNG
--------------------------------------------------------------------------------
/image/k8sprom.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/k8sprom.png
--------------------------------------------------------------------------------
/image/kubernetes.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/kubernetes.png
--------------------------------------------------------------------------------
/image/log_stream_pipeline.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/log_stream_pipeline.PNG
--------------------------------------------------------------------------------
/image/logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/logo.png
--------------------------------------------------------------------------------
/image/loki_logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/loki_logo.png
--------------------------------------------------------------------------------
/image/prometheus.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/isItObservable/Episode3--Kubernetes-Fluentbit/0f20d5f7c8ae29e54dc3abecbe493fa4fa783460/image/prometheus.png
--------------------------------------------------------------------------------