├── .gitignore
├── LICENSE
├── README.md
├── assets
├── gelf_message.png
└── graylog_dashboard.png
├── graylog
├── contentpacks
│ └── grok-patterns.json
└── graylog.conf
├── pom.xml
├── sigar-libs.zip
└── src
└── main
├── java
└── com
│ └── objectpartners
│ └── plummer
│ └── graylog
│ ├── Application.java
│ ├── aspects
│ ├── ErrorAspect.java
│ └── QuoteAspect.java
│ ├── config
│ ├── SchedulingConfiguration.java
│ ├── SwaggerConfiguration.java
│ └── WebMvcConfiguration.java
│ ├── controller
│ └── QuotesController.java
│ ├── data
│ └── MongoInstance.java
│ ├── domain
│ ├── Exchange.java
│ └── QuoteResource.java
│ ├── graylog
│ ├── ConcreteDashboardList.java
│ ├── GelfMessage.java
│ ├── GraylogInstance.java
│ └── GraylogRestInterface.java
│ ├── jobs
│ └── ScheduledJobs.java
│ └── service
│ └── QuoteService.java
└── resources
├── application.yml
└── logback.xml
/.gitignore:
--------------------------------------------------------------------------------
1 | /.gradle
2 | /.idea
3 | *.iml
4 | /build
5 | /data
6 | /sigar-libs
7 | /target
8 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | The MIT License (MIT)
2 |
3 | Copyright (c) 2016 Mike Plummer
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # graylog-springboot
2 | Example of using Graylog with Spring Boot
3 |
4 | Check out my [blog post](https://objectpartners.com/2016/05/04/graylog-with-spring-boot-open-source-log-and-event-analysis/) describing the goals and process (and compromises) behind this code.
5 |
6 | ## Infrastructure
7 | Graylog uses MongoDB and ElasticSearch behind the scenes. To make this example as simple as possible I've configured the SpringBoot application to fire up embedded versions of both of these. I highly recommend not using this setup in a Production environment.
8 |
9 | ## URLs
10 | Graylog: http://localhost:9000
11 |
12 | Graylog Rest API: http://localhost:12900/api-browser
13 |
14 | App Rest API: http://localhost:8080/swagger-ui.html
15 |
16 | ## Instructions
17 | 1. Make sure you have Java and Maven installed
18 | 2. Run `mvn spring-boot:run` from project root
19 | 3. The first time you run this a LOT of stuff will download. This is a one-time occurrence.
20 | - Maven dependencies
21 | - MongoDB binaries to launch an embedded Mongo instance
22 | 4. You'll see a lot of logging - this is the Spring application initializing itself and booting up an ElasticSearch cluster, MongoDB database, and kicking off the Graylog instance
23 | 5. After about 30 seconds (depending on your machine's horsepower) you should start seeing logging like `Generated quote - VKL@59.0`
24 | 6. Open `http://localhost:9000` to open the Graylog GUI, login with username/password `admin/admin`
25 | 7. Explore! The application sets up some default GUI elements - start with the 'Stock Market' dashboard from the top menu.
26 | 8. When you're done just end the Maven process with `Ctrl-C`
27 |
28 | ## Cleanup
29 | ### MongoDB
30 | - To delete persisted Mongo data you can delete the 'data/mongodb/data' directory - the next time you launch the application a fresh database will get set up
31 | - Mongo executables get downloaded into the project directory - they can be deleted by removing the 'data/mongodb' directory. This will delete the persisted data as well. If you do this and try to re-launch the application it will re-download the executables.
32 |
33 | ### ElasticSearch
34 | If you delete the MongoDB data you'll need to fix the ElasticSearch indices or else they'll be out of sync. This can be done from the Graylog GUI or you can just delete `data/graylog/nodes` to rebuild them on next launch.
35 |
36 | ## Configuration
37 | If you're interested in tweaking the application's config you can look in three main places:
38 |
39 | 1. The code! Lots of setup and defaults are being set in the code since this is just an example and not really intended for extension/reuse.
40 | 2. application.yml: Spring Boot configs are set here (including ElasticSearch which is launched by Spring Boot)
41 | 3. graylog.conf: Config file for Graylog - lots of values in here you can mess with. Take a look at the Graylog docs for info on what they all do.
42 |
43 | ## Versions
44 | You'll need versions of Java and Maven available. I've tested with the specified versions.
45 |
46 | Java: 1.8.0_131
47 |
48 | Maven: 3.5.0
49 |
50 | Application has been verified on OSX 10.13.0. Your mileage may vary on other operating systems.
51 |
52 | ## License
53 | This code is provided under the terms of the MIT license: basically you're free to do whatever you want with it, but no guarantees are made to its validity, stability, or safety. All works referenced by or utilized by this project are the property of their respective copyright holders and retain licensing that may be more restrictive.
54 |
--------------------------------------------------------------------------------
/assets/gelf_message.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mike-plummer/graylog-springboot/28d7cc71f3413b356cacf44b71eadfd9a3d44714/assets/gelf_message.png
--------------------------------------------------------------------------------
/assets/graylog_dashboard.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mike-plummer/graylog-springboot/28d7cc71f3413b356cacf44b71eadfd9a3d44714/assets/graylog_dashboard.png
--------------------------------------------------------------------------------
/graylog/contentpacks/grok-patterns.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "Core Grok Patterns",
3 | "description": "Core grok patterns",
4 | "category": "Grok",
5 | "grok_patterns": [
6 | {
7 | "name": "USERNAME",
8 | "pattern": "[a-zA-Z0-9._-]+"
9 | },
10 | {
11 | "name": "USER",
12 | "pattern": "%{USERNAME}"
13 | },
14 | {
15 | "name": "EMAILLOCALPART",
16 | "pattern": "[a-zA-Z][a-zA-Z0-9_.+-=:]+"
17 | },
18 | {
19 | "name": "EMAILADDRESS",
20 | "pattern": "%{EMAILLOCALPART}@%{HOSTNAME}"
21 | },
22 | {
23 | "name": "HTTPDUSER",
24 | "pattern": "%{EMAILADDRESS}|%{USER}"
25 | },
26 | {
27 | "name": "INT",
28 | "pattern": "(?:[+-]?(?:[0-9]+))"
29 | },
30 | {
31 | "name": "BASE10NUM",
32 | "pattern": "(?[+-]?(?:(?:[0-9]+(?:\\.[0-9]+)?)|(?:\\.[0-9]+)))"
33 | },
34 | {
35 | "name": "NUMBER",
36 | "pattern": "(?:%{BASE10NUM})"
37 | },
38 | {
39 | "name": "BASE16NUM",
40 | "pattern": "(?(?\"(?>\\\\.|[^\\\\\"]+)+\"|\"\"|(?>'(?>\\\\.|[^\\\\']+)+')|''|(?>`(?>\\\\.|[^\\\\`]+)+`)|``))"
77 | },
78 | {
79 | "name": "UUID",
80 | "pattern": "[A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}"
81 | },
82 | {
83 | "name": "MAC",
84 | "pattern": "(?:%{CISCOMAC}|%{WINDOWSMAC}|%{COMMONMAC})"
85 | },
86 | {
87 | "name": "CISCOMAC",
88 | "pattern": "(?:(?:[A-Fa-f0-9]{4}\\.){2}[A-Fa-f0-9]{4})"
89 | },
90 | {
91 | "name": "WINDOWSMAC",
92 | "pattern": "(?:(?:[A-Fa-f0-9]{2}-){5}[A-Fa-f0-9]{2})"
93 | },
94 | {
95 | "name": "COMMONMAC",
96 | "pattern": "(?:(?:[A-Fa-f0-9]{2}:){5}[A-Fa-f0-9]{2})"
97 | },
98 | {
99 | "name": "IPV6",
100 | "pattern": "((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(\\.(25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)){3}))|:)))(%.+)?"
101 | },
102 | {
103 | "name": "IPV4",
104 | "pattern": "(?[A-Za-z]+:|\\\\)(?:\\\\[^\\\\?*]*)+"
137 | },
138 | {
139 | "name": "URIPROTO",
140 | "pattern": "[A-Za-z]+(\\+[A-Za-z+]+)?"
141 | },
142 | {
143 | "name": "URIHOST",
144 | "pattern": "%{IPORHOST}(?::%{POSINT:port})?"
145 | },
146 | {
147 | "name": "URIPATH",
148 | "pattern": "(?:/[A-Za-z0-9$.+!*'(){},~:;=@#%_\\-]*)+"
149 | },
150 | {
151 | "name": "URIPARAM",
152 | "pattern": "\\?[A-Za-z0-9$.+!*'|(){},~@#%&/=:;_?\\-\\[\\]<>]*"
153 | },
154 | {
155 | "name": "URIPATHPARAM",
156 | "pattern": "%{URIPATH}(?:%{URIPARAM})?"
157 | },
158 | {
159 | "name": "URI",
160 | "pattern": "%{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?"
161 | },
162 | {
163 | "name": "MONTH",
164 | "pattern": "\\b(?:Jan(?:uary|uar)?|Feb(?:ruary|ruar)?|M(?:a|ä)?r(?:ch|z)?|Apr(?:il)?|Ma(?:y|i)?|Jun(?:e|i)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|O(?:c|k)?t(?:ober)?|Nov(?:ember)?|De(?:c|z)(?:ember)?)\\b"
165 | },
166 | {
167 | "name": "MONTHNUM",
168 | "pattern": "(?:0?[1-9]|1[0-2])"
169 | },
170 | {
171 | "name": "MONTHNUM2",
172 | "pattern": "(?:0[1-9]|1[0-2])"
173 | },
174 | {
175 | "name": "MONTHDAY",
176 | "pattern": "(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])"
177 | },
178 | {
179 | "name": "DAY",
180 | "pattern": "(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)"
181 | },
182 | {
183 | "name": "YEAR",
184 | "pattern": "(?>\\d\\d){1,2}"
185 | },
186 | {
187 | "name": "HOUR",
188 | "pattern": "(?:2[0123]|[01]?[0-9])"
189 | },
190 | {
191 | "name": "MINUTE",
192 | "pattern": "(?:[0-5][0-9])"
193 | },
194 | {
195 | "name": "SECOND",
196 | "pattern": "(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)"
197 | },
198 | {
199 | "name": "TIME",
200 | "pattern": "(?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])"
201 | },
202 | {
203 | "name": "DATE_US",
204 | "pattern": "%{MONTHNUM}[/-]%{MONTHDAY}[/-]%{YEAR}"
205 | },
206 | {
207 | "name": "DATE_EU",
208 | "pattern": "%{MONTHDAY}[./-]%{MONTHNUM}[./-]%{YEAR}"
209 | },
210 | {
211 | "name": "ISO8601_TIMEZONE",
212 | "pattern": "(?:Z|[+-]%{HOUR}(?::?%{MINUTE}))"
213 | },
214 | {
215 | "name": "ISO8601_SECOND",
216 | "pattern": "(?:%{SECOND}|60)"
217 | },
218 | {
219 | "name": "TIMESTAMP_ISO8601",
220 | "pattern": "%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?"
221 | },
222 | {
223 | "name": "DATE",
224 | "pattern": "%{DATE_US}|%{DATE_EU}"
225 | },
226 | {
227 | "name": "DATESTAMP",
228 | "pattern": "%{DATE}[- ]%{TIME}"
229 | },
230 | {
231 | "name": "TZ",
232 | "pattern": "(?:[PMCE][SD]T|UTC)"
233 | },
234 | {
235 | "name": "DATESTAMP_RFC822",
236 | "pattern": "%{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}"
237 | },
238 | {
239 | "name": "DATESTAMP_RFC2822",
240 | "pattern": "%{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601_TIMEZONE}"
241 | },
242 | {
243 | "name": "DATESTAMP_OTHER",
244 | "pattern": "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}"
245 | },
246 | {
247 | "name": "DATESTAMP_EVENTLOG",
248 | "pattern": "%{YEAR}%{MONTHNUM2}%{MONTHDAY}%{HOUR}%{MINUTE}%{SECOND}"
249 | },
250 | {
251 | "name": "HTTPDERROR_DATE",
252 | "pattern": "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}"
253 | },
254 | {
255 | "name": "SYSLOGTIMESTAMP",
256 | "pattern": "%{MONTH} +%{MONTHDAY} %{TIME}"
257 | },
258 | {
259 | "name": "PROG",
260 | "pattern": "[\\x21-\\x5a\\x5c\\x5e-\\x7e]+"
261 | },
262 | {
263 | "name": "SYSLOGPROG",
264 | "pattern": "%{PROG:program}(?:\\[%{POSINT:pid}\\])?"
265 | },
266 | {
267 | "name": "SYSLOGHOST",
268 | "pattern": "%{IPORHOST}"
269 | },
270 | {
271 | "name": "SYSLOGFACILITY",
272 | "pattern": "<%{NONNEGINT:facility}.%{NONNEGINT:priority}>"
273 | },
274 | {
275 | "name": "HTTPDATE",
276 | "pattern": "%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}"
277 | },
278 | {
279 | "name": "QS",
280 | "pattern": "%{QUOTEDSTRING}"
281 | },
282 | {
283 | "name": "SYSLOGBASE",
284 | "pattern": "%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:"
285 | },
286 | {
287 | "name": "COMMONAPACHELOG",
288 | "pattern": "%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}\\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-)"
289 | },
290 | {
291 | "name": "COMBINEDAPACHELOG",
292 | "pattern": "%{COMMONAPACHELOG} %{QS:referrer} %{QS:agent}"
293 | },
294 | {
295 | "name": "HTTPD20_ERRORLOG",
296 | "pattern": "\\[%{HTTPDERROR_DATE:timestamp}\\] \\[%{LOGLEVEL:loglevel}\\] (?:\\[client %{IPORHOST:clientip}\\] ){0,1}%{GREEDYDATA:errormsg}"
297 | },
298 | {
299 | "name": "HTTPD24_ERRORLOG",
300 | "pattern": "\\[%{HTTPDERROR_DATE:timestamp}\\] \\[%{WORD:module}:%{LOGLEVEL:loglevel}\\] \\[pid %{POSINT:pid}:tid %{NUMBER:tid}\\]( \\(%{POSINT:proxy_errorcode}\\)%{DATA:proxy_errormessage}:)?( \\[client %{IPORHOST:client}:%{POSINT:clientport}\\])? %{DATA:errorcode}: %{GREEDYDATA:message}"
301 | },
302 | {
303 | "name": "HTTPD_ERRORLOG",
304 | "pattern": "%{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}"
305 | },
306 | {
307 | "name": "LOGLEVEL",
308 | "pattern": "([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)"
309 | }
310 | ]
311 | }
312 |
--------------------------------------------------------------------------------
/graylog/graylog.conf:
--------------------------------------------------------------------------------
1 | is_master = true
2 | node_id_file = data/node-id
3 | password_secret = replacethiswithyourownsecret!
4 | root_password_sha2 = 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
5 | plugin_dir = data/plugin
6 | rest_listen_uri = http://0.0.0.0:12900/
7 | rest_transport_uri = http://localhost:12900/
8 | web_listen_uri = http://0.0.0.0:9000/
9 | web_enable_cors = true
10 | rotation_strategy = count
11 | elasticsearch_max_docs_per_index = 20000000
12 | elasticsearch_max_number_of_indices = 20
13 | retention_strategy = delete
14 | elasticsearch_shards = 4
15 | elasticsearch_replicas = 0
16 | elasticsearch_index_prefix = graylog
17 | allow_leading_wildcard_searches = true
18 | allow_highlighting = true
19 | elasticsearch_cluster_name = graylog
20 | # elasticsearch_transport_tcp_port = 9350
21 | elasticsearch_http_enabled = false
22 | # elasticsearch_discovery_zen_ping_multicast_enabled = true
23 | elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300
24 | elasticsearch_network_host = 0.0.0.0
25 | elasticsearch_analyzer = standard
26 | output_batch_size = 500
27 | output_flush_interval = 1
28 | output_fault_count_threshold = 5
29 | output_fault_penalty_seconds = 30
30 | processbuffer_processors = 5
31 | outputbuffer_processors = 3
32 | processor_wait_strategy = blocking
33 | ring_size = 65536
34 | inputbuffer_ring_size = 65536
35 | inputbuffer_processors = 2
36 | inputbuffer_wait_strategy = blocking
37 | message_journal_enabled = true
38 | message_journal_dir = data/journal
39 | lb_recognition_period_seconds = 3
40 | mongodb_uri = mongodb://localhost/graylog
41 | mongodb_max_connections = 100
42 | mongodb_threads_allowed_to_block_multiplier = 5
43 | content_packs_loader_enabled = true
44 | content_packs_dir = graylog/contentpacks
45 | content_packs_auto_load = grok-patterns.json
--------------------------------------------------------------------------------
/pom.xml:
--------------------------------------------------------------------------------
1 |
2 |
4 | 4.0.0
5 |
6 | com.objectpartners.plummer
7 | graylog-springboot
8 | 0.0.1
9 | jar
10 |
11 | graylog-springboot
12 | Example of using GrayLog with Spring Boot
13 | https://github.com/mike-plummer/graylog-springboot
14 |
15 |
16 | 3.6
17 | 2.0.0
18 | 2.3.1
19 | 21.0
20 | 1
21 | 1.8
22 | 1.1.11
23 | 1.16.18
24 | 1.5.7.RELEASE
25 | 2.7.0
26 | 1.2
27 |
28 |
29 |
30 | org.springframework.boot
31 | spring-boot-starter-parent
32 | 1.5.7.RELEASE
33 |
34 |
35 |
36 |
37 |
38 | org.apache.maven.plugins
39 | maven-dependency-plugin
40 | 2.10
41 |
42 |
43 | org.springframework.boot
44 | spring-boot-maven-plugin
45 | 1.5.7.RELEASE
46 |
47 |
48 |
49 | repackage
50 |
51 |
52 |
53 |
54 |
55 | org.codehaus.mojo
56 | truezip-maven-plugin
57 | ${truezip.version}
58 |
59 |
60 | copy-package
61 |
62 | copy
63 |
64 | compile
65 |
66 | true
67 |
68 | sigar-libs.zip
69 | sigar-libs
70 |
71 |
72 |
73 |
74 |
75 |
76 |
77 |
78 |
79 |
80 | org.springframework.boot
81 | spring-boot-starter-aop
82 |
83 |
84 | org.springframework.boot
85 | spring-boot-starter-data-elasticsearch
86 |
87 |
88 | org.springframework.boot
89 | spring-boot-starter-web
90 |
91 |
92 | io.springfox
93 | springfox-swagger-ui
94 | ${springfox.version}
95 |
96 |
97 | io.springfox
98 | springfox-swagger2
99 | ${springfox.version}
100 |
101 |
102 | javax.inject
103 | javax.inject
104 | ${javax-inject.version}
105 |
106 |
107 | org.projectlombok
108 | lombok
109 | ${lombok.version}
110 |
111 |
112 | de.flapdoodle.embed
113 | de.flapdoodle.embed.mongo
114 | ${embedded-mongo.version}
115 |
116 |
117 | org.apache.commons
118 | commons-lang3
119 | ${commons-lang3.version}
120 |
121 |
122 | org.graylog2
123 | graylog2-server
124 | ${graylog.version}
125 |
126 |
127 |
128 | org.apache.logging.log4j
129 | log4j-slf4j-impl
130 |
131 |
132 | org.apache.logging.log4j
133 | log4j-slf4j-impl
134 |
135 |
136 | org.slf4j
137 | log4j-over-slf4j
138 |
139 |
140 |
141 |
142 | com.fasterxml.jackson.datatype
143 | jackson-datatype-joda
144 |
145 |
146 | com.github.pukkaone
147 | logback-gelf
148 | ${logback-gelf.version}
149 | runtime
150 |
151 |
152 | org.springframework.boot
153 | spring-boot-starter-test
154 | test
155 |
156 |
157 | com.google.guava
158 | guava
159 | ${guava.version}
160 |
161 |
162 |
163 |
--------------------------------------------------------------------------------
/sigar-libs.zip:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mike-plummer/graylog-springboot/28d7cc71f3413b356cacf44b71eadfd9a3d44714/sigar-libs.zip
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/Application.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog;
2 |
3 | import com.objectpartners.plummer.graylog.config.*;
4 | import org.slf4j.Logger;
5 | import org.slf4j.LoggerFactory;
6 | import org.springframework.boot.SpringApplication;
7 | import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
8 | import org.springframework.boot.autoconfigure.SpringBootApplication;
9 | import org.springframework.boot.autoconfigure.elasticsearch.jest.JestAutoConfiguration;
10 | import org.springframework.boot.autoconfigure.mongo.embedded.EmbeddedMongoAutoConfiguration;
11 | import org.springframework.context.ConfigurableApplicationContext;
12 | import org.springframework.context.annotation.ComponentScan;
13 | import org.springframework.context.annotation.Import;
14 |
15 | @SpringBootApplication
16 | @EnableAutoConfiguration(exclude = {
17 | EmbeddedMongoAutoConfiguration.class,
18 | JestAutoConfiguration.class
19 | })
20 | @Import({
21 | SchedulingConfiguration.class,
22 | SwaggerConfiguration.class,
23 | WebMvcConfiguration.class
24 | })
25 | @ComponentScan("com.objectpartners.plummer.graylog")
26 | public class Application {
27 | private static final Logger LOGGER = LoggerFactory.getLogger(Application.class);
28 |
29 | public static void main(String[] args) throws Exception {
30 | ConfigurableApplicationContext application = SpringApplication.run(Application.class, args);
31 | LOGGER.info("Application started, registering shutdown hook...");
32 | application.registerShutdownHook();
33 | }
34 | }
35 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/aspects/ErrorAspect.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.aspects;
2 |
3 | import com.objectpartners.plummer.graylog.graylog.GelfMessage;
4 | import com.objectpartners.plummer.graylog.graylog.GraylogRestInterface;
5 | import org.aspectj.lang.annotation.AfterThrowing;
6 | import org.aspectj.lang.annotation.Aspect;
7 |
8 | import javax.inject.Inject;
9 | import javax.inject.Named;
10 | import java.io.IOException;
11 | import java.io.PrintWriter;
12 | import java.io.StringWriter;
13 |
14 | @Aspect
15 | @Named
16 | public class ErrorAspect {
17 |
18 | @Inject
19 | protected GraylogRestInterface graylog;
20 |
21 | @AfterThrowing(pointcut = "execution(* com.objectpartners.plummer.graylog.jobs..*(..))", throwing = "e")
22 | public void errorThrown(Throwable e) throws IOException {
23 | GelfMessage message = new GelfMessage();
24 | message.setShortMessage("Error");
25 | message.setFullMessage(e.getMessage());
26 |
27 | try (StringWriter stringWriter = new StringWriter();
28 | PrintWriter printWriter = new PrintWriter(stringWriter)) {
29 | e.printStackTrace(printWriter);
30 | message.getAdditionalProperties().put("stacktrace", stringWriter.toString());
31 | }
32 |
33 | graylog.logEvent(message);
34 | }
35 | }
36 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/aspects/QuoteAspect.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.aspects;
2 |
3 | import com.fasterxml.jackson.databind.ObjectMapper;
4 | import com.google.common.base.Stopwatch;
5 | import com.objectpartners.plummer.graylog.domain.QuoteResource;
6 | import com.objectpartners.plummer.graylog.graylog.GelfMessage;
7 | import com.objectpartners.plummer.graylog.graylog.GraylogRestInterface;
8 | import org.aspectj.lang.ProceedingJoinPoint;
9 | import org.aspectj.lang.annotation.Around;
10 | import org.aspectj.lang.annotation.Aspect;
11 | import org.slf4j.Logger;
12 | import org.slf4j.LoggerFactory;
13 |
14 | import javax.inject.Inject;
15 | import javax.inject.Named;
16 | import java.util.concurrent.TimeUnit;
17 |
18 | @Aspect
19 | @Named
20 | public class QuoteAspect {
21 |
22 | private static final Logger LOGGER = LoggerFactory.getLogger(QuoteAspect.class);
23 |
24 | @Inject
25 | protected ObjectMapper mapper;
26 |
27 | @Inject
28 | protected GraylogRestInterface graylog;
29 |
30 | @Around("execution(* com.objectpartners.plummer.graylog.service.QuoteService.*(..))")
31 | public QuoteResource errorThrown(ProceedingJoinPoint joinPoint) throws Throwable {
32 | Stopwatch timer = Stopwatch.createStarted();
33 |
34 | QuoteResource quote = (QuoteResource) joinPoint.proceed();
35 |
36 | // Log message via Graylog UDP Logback Appender
37 | LOGGER.info("Generated quote - {}@{}", quote.getSymbol(), quote.getPrice());
38 |
39 | // Log message via Graylog HTTP Input
40 | GelfMessage message = new GelfMessage();
41 | message.setShortMessage(String.format("Quote %s@%2.2f", quote.getSymbol(), quote.getPrice()));
42 | message.setFullMessage(mapper.writeValueAsString(quote));
43 | message.getAdditionalProperties().put("elapsed_time", timer.stop().elapsed(TimeUnit.MICROSECONDS));
44 | graylog.logEvent(message);
45 |
46 | return quote;
47 | }
48 | }
49 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/config/SchedulingConfiguration.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.config;
2 |
3 | import org.springframework.context.annotation.Bean;
4 | import org.springframework.context.annotation.Configuration;
5 | import org.springframework.scheduling.annotation.EnableScheduling;
6 | import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
7 |
8 | @Configuration
9 | @EnableScheduling
10 | public class SchedulingConfiguration {
11 |
12 | @Bean
13 | public ThreadPoolTaskScheduler taskScheduler() {
14 | return new ThreadPoolTaskScheduler();
15 | }
16 | }
17 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/config/SwaggerConfiguration.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.config;
2 |
3 | import com.google.common.collect.Sets;
4 | import org.springframework.beans.factory.annotation.Value;
5 | import org.springframework.context.annotation.Bean;
6 | import org.springframework.context.annotation.Configuration;
7 | import org.springframework.http.MediaType;
8 | import springfox.documentation.builders.ApiInfoBuilder;
9 | import springfox.documentation.service.ApiInfo;
10 | import springfox.documentation.service.Contact;
11 | import springfox.documentation.spi.DocumentationType;
12 | import springfox.documentation.spring.web.plugins.Docket;
13 | import springfox.documentation.swagger2.annotations.EnableSwagger2;
14 |
15 | import java.util.Set;
16 |
17 | import static springfox.documentation.builders.RequestHandlerSelectors.basePackage;
18 |
19 | @EnableSwagger2
20 | @Configuration
21 | public class SwaggerConfiguration {
22 |
23 | @Value("${info.build.version}")
24 | private String buildVersion;
25 |
26 | @Value("${info.build.description}")
27 | private String projectDescription;
28 |
29 | @Value("${info.build.name}")
30 | private String projectName;
31 |
32 | @Bean
33 | public Docket stockmarketApi() {
34 | return new Docket(DocumentationType.SWAGGER_2)
35 | .consumes(apiContentTypes())
36 | .produces(apiContentTypes())
37 | .groupName("stockmarket")
38 | .apiInfo(apiInfo())
39 | .select()
40 | .apis(basePackage("com.objectpartners.plummer.graylog"))
41 | .build();
42 | }
43 |
44 | private ApiInfo apiInfo() {
45 | return new ApiInfoBuilder()
46 | .title(projectName)
47 | .description(projectDescription)
48 | .contact(new Contact("Mike Plummer", "https://mike-plummer.github.io", "mike.plummer@objectpartners.com"))
49 | .license("MIT")
50 | .version(buildVersion)
51 | .build();
52 | }
53 |
54 | private Set apiContentTypes() {
55 | return Sets.newHashSet(MediaType.APPLICATION_JSON_VALUE);
56 | }
57 | }
58 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/config/WebMvcConfiguration.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.config;
2 |
3 | import com.fasterxml.jackson.datatype.joda.JodaModule;
4 | import com.fasterxml.jackson.datatype.joda.cfg.JacksonJodaDateFormat;
5 | import com.fasterxml.jackson.datatype.joda.ser.DateTimeSerializer;
6 | import org.joda.time.DateTime;
7 | import org.springframework.context.annotation.Bean;
8 | import org.springframework.context.annotation.Configuration;
9 | import org.springframework.format.annotation.DateTimeFormat;
10 | import org.springframework.format.datetime.joda.DateTimeFormatterFactory;
11 | import org.springframework.http.MediaType;
12 | import org.springframework.web.servlet.config.annotation.ContentNegotiationConfigurer;
13 | import org.springframework.web.servlet.config.annotation.EnableWebMvc;
14 | import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;
15 | import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
16 |
17 | @EnableWebMvc
18 | @Configuration
19 | public class WebMvcConfiguration extends WebMvcConfigurerAdapter {
20 | @Override
21 | public void addResourceHandlers(ResourceHandlerRegistry registry) {
22 | registry.addResourceHandler("swagger-ui.html")
23 | .addResourceLocations("classpath:/META-INF/resources/");
24 |
25 | registry.addResourceHandler("/webjars/**")
26 | .addResourceLocations("classpath:/META-INF/resources/webjars/");
27 | }
28 |
29 | @Override
30 | public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
31 | configurer.defaultContentType(MediaType.APPLICATION_JSON);
32 | }
33 |
34 | @Bean
35 | public JodaModule jacksonJodaModule() {
36 | JodaModule module = new JodaModule();
37 | DateTimeFormatterFactory formatterFactory = new DateTimeFormatterFactory();
38 | formatterFactory.setIso(DateTimeFormat.ISO.DATE);
39 | module.addSerializer(DateTime.class, new DateTimeSerializer(
40 | new JacksonJodaDateFormat(formatterFactory.createDateTimeFormatter().withZoneUTC())));
41 | return module;
42 | }
43 | }
44 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/controller/QuotesController.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.controller;
2 |
3 | import com.objectpartners.plummer.graylog.domain.QuoteResource;
4 | import com.objectpartners.plummer.graylog.service.QuoteService;
5 | import io.swagger.annotations.Api;
6 | import io.swagger.annotations.ApiOperation;
7 | import io.swagger.annotations.ApiResponse;
8 | import io.swagger.annotations.ApiResponses;
9 | import org.springframework.web.bind.annotation.PathVariable;
10 | import org.springframework.web.bind.annotation.RequestMapping;
11 | import org.springframework.web.bind.annotation.RequestMethod;
12 | import org.springframework.web.bind.annotation.RestController;
13 |
14 | import javax.inject.Inject;
15 |
16 | import static com.objectpartners.plummer.graylog.controller.QuotesController.RESOURCE_ROOT_URL;
17 |
18 | @RestController
19 | @RequestMapping(RESOURCE_ROOT_URL)
20 | @Api(value = "stocks", tags = "Stocks")
21 | public class QuotesController {
22 |
23 | public static final String RESOURCE_ROOT_URL = "/quotes";
24 |
25 | @Inject
26 | protected QuoteService quoteService;
27 |
28 | @RequestMapping(method = RequestMethod.GET, value = "/{symbol}")
29 | @ApiOperation(value = "Get Quote")
30 | @ApiResponses(value = {
31 | @ApiResponse(code = 200, message = "Success"),
32 | @ApiResponse(code = 401, message = "Unauthorized"),
33 | @ApiResponse(code = 403, message = "Forbidden"),
34 | @ApiResponse(code = 404, message = "Not Found"),
35 | @ApiResponse(code = 500, message = "Failure")
36 | })
37 | public QuoteResource getQuote(@PathVariable("symbol") String symbol) {
38 | return quoteService.quote(symbol);
39 | }
40 | }
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/data/MongoInstance.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.data;
2 |
3 | import de.flapdoodle.embed.mongo.Command;
4 | import de.flapdoodle.embed.mongo.MongodExecutable;
5 | import de.flapdoodle.embed.mongo.MongodProcess;
6 | import de.flapdoodle.embed.mongo.MongodStarter;
7 | import de.flapdoodle.embed.mongo.config.*;
8 | import de.flapdoodle.embed.mongo.distribution.Version;
9 | import de.flapdoodle.embed.process.config.IRuntimeConfig;
10 | import de.flapdoodle.embed.process.config.store.IDownloadConfig;
11 | import de.flapdoodle.embed.process.io.directories.FixedPath;
12 | import de.flapdoodle.embed.process.io.directories.IDirectory;
13 | import de.flapdoodle.embed.process.runtime.Network;
14 | import org.springframework.beans.factory.DisposableBean;
15 | import org.springframework.boot.autoconfigure.mongo.MongoProperties;
16 | import org.springframework.core.Ordered;
17 | import org.springframework.core.annotation.Order;
18 |
19 | import javax.inject.Named;
20 | import javax.inject.Singleton;
21 | import java.io.IOException;
22 |
23 | @Named
24 | @Singleton
25 | @Order(Ordered.HIGHEST_PRECEDENCE)
26 | public class MongoInstance implements DisposableBean {
27 |
28 | private static final IDirectory ARTIFACT_STORE_PATH = new FixedPath("./data/mongodb");
29 | private static final IDirectory EXTRACTED_STORE_PATH = new FixedPath("./data/mongodb/extracted");
30 |
31 | private MongodExecutable mongo;
32 | private MongodProcess process;
33 |
34 | public MongoInstance() throws IOException, InterruptedException {
35 |
36 | // Download Mongo artifacts into the project directory to ease cleanup
37 | IDownloadConfig downloadConfig = new DownloadConfigBuilder()
38 | .defaultsForCommand(Command.MongoD)
39 | .artifactStorePath(ARTIFACT_STORE_PATH)
40 | .build();
41 |
42 | // Extract Mongo artifacts into the project directory to ease cleanup
43 | IRuntimeConfig runtimeConfig = new RuntimeConfigBuilder()
44 | .defaults(Command.MongoD)
45 | .artifactStore(new ExtractedArtifactStoreBuilder()
46 | .defaults(Command.MongoD)
47 | .download(downloadConfig)
48 | .extractDir(EXTRACTED_STORE_PATH)
49 | )
50 | .build();
51 |
52 | // Store Mongo data into the project directory to ease cleanup
53 | Storage replication = new Storage("./data/mongodb/data", null, 0);
54 |
55 | MongodStarter starter = MongodStarter.getInstance(runtimeConfig);
56 |
57 | IMongodConfig mongodConfig = new MongodConfigBuilder()
58 | .version(Version.Main.PRODUCTION)
59 | .cmdOptions(new MongoCmdOptionsBuilder()
60 | .useNoJournal(false)
61 | .useSmallFiles(true)
62 | .build())
63 | .net(new Net(MongoProperties.DEFAULT_PORT, Network.localhostIsIPv6()))
64 | .replication(replication)
65 | .build();
66 |
67 | mongo = starter.prepare(mongodConfig);
68 | process = mongo.start();
69 | }
70 |
71 | @Override
72 | public void destroy() throws Exception {
73 | if (process != null) {
74 | process.stop();
75 | }
76 | if (mongo != null) {
77 | mongo.stop();
78 | }
79 | }
80 | }
81 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/domain/Exchange.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.domain;
2 |
3 | public enum Exchange {
4 | NYSE,
5 | NASDAQ
6 | }
7 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/domain/QuoteResource.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.domain;
2 |
3 | import lombok.AllArgsConstructor;
4 | import lombok.Data;
5 | import lombok.EqualsAndHashCode;
6 | import lombok.NoArgsConstructor;
7 |
8 | @Data
9 | @NoArgsConstructor
10 | @AllArgsConstructor
11 | @EqualsAndHashCode(callSuper = false)
12 | public class QuoteResource {
13 | private String symbol;
14 | private Double price;
15 | private Exchange exchange;
16 | }
17 |
--------------------------------------------------------------------------------
/src/main/java/com/objectpartners/plummer/graylog/graylog/ConcreteDashboardList.java:
--------------------------------------------------------------------------------
1 | package com.objectpartners.plummer.graylog.graylog;
2 |
3 | import lombok.AllArgsConstructor;
4 | import lombok.EqualsAndHashCode;
5 | import lombok.NoArgsConstructor;
6 | import org.graylog2.rest.models.dashboards.responses.DashboardList;
7 |
8 | import java.util.ArrayList;
9 | import java.util.List;
10 | import java.util.Map;
11 |
12 | /**
13 | * Graylog rest classes don't supply a DashboardList implementation that can be easily used with Jackson deserialization.
14 | * Create one here to make it easier to pull and deserialize JSON from Graylog rest services.
15 | */
16 | @NoArgsConstructor
17 | @AllArgsConstructor
18 | @EqualsAndHashCode(callSuper = true)
19 | public class ConcreteDashboardList extends DashboardList {
20 | private int total;
21 | private List