6 |
7 | ## **About:**
8 | KafkaESK, currently in Beta, is an event-driven monitoring tool that can consume messages from Apache Kafka clusters and display the aggregated data on a dashboard for analysis and maintenance. Used in a pipeline that includes **Apache Kafka Connect** (for writing sources and sinks that either ingests the entire database and stream table updates to Kafka topics or continuously delivers data from the topics into external systems) and **ksqlDB** (for stream processing that enables executing continuous computations over an unbounded stream of events), KafkaESK can incrementally update in real-time as events arrive.
9 | This powerful tool can be used to digest live data from IoT/smart sensor technology, machine performance, and even website activity such as clickstreams.
10 |
11 |
12 | ### Stretch Features Include:
13 | - User ability to dynamically customize data charts directly on the dashboard
14 | - Integrating a terminal into the dashboard GUI that will allow interaction with the ksqlDB CLI to:
15 | - craft materialized views over streams
16 | - receive real-time push updates
17 | - pull current state on demand
18 | - transform, filter, aggregate, and join collections
19 | - push and pull queries
20 | - Caching of previously ran queries
21 | - Time machine capability
22 |
23 | ## **Prerequisites:**
24 | - Docker version 1.11 or later is [installed and running](https://docs.docker.com/engine/install/).
25 | - At the time of release, KafkaESK operates in conjunction with the Confluent Platform(version 6.0.0). Use these installation methods to quickly get a Confluent Platform development environment up and running on your local machine.
26 |
27 | - [Quick Start for Apache Kafka using Confluent Platform (Local)](https://docs.confluent.io/platform/current/quickstart/ce-quickstart.html#ce-quickstart)
28 | - [Quick Start for Apache Kafka using Confluent Platform (Docker)](https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html#ce-docker-quickstart)
29 | - [Git](https://git-scm.com/downloads)
30 | - Internet connectivity
31 |
32 | ## **Demo:**
33 | For demonstration purposes, mock data is used to simulate a live stream of HTTP requests that end in error from user clicks. The KafkaESK tool will track this activity and render the incoming messages on multiple graphs.
34 |
35 |
36 |
37 |
42 |
43 | 1. Run the following command while inside the cloned repo directory:
44 |
45 | ```bash
46 | docker run -d -p 8080:8080 -p 3333:3333 --name kafka kafkaesk
47 | ```
48 |
49 | 2. Open a terminal and launch the Confluent Platform using the Confluent CLI `confluent local services start` command. This command starts all of the Confluent Platform components, including Kafka, ZooKeeper, Schema Registry, HTTP REST Proxy for Kafka, Kafka Connect, ksqlDB, and Control Center.
50 |
51 | ```bash
52 | confluent local services start
53 | ```
54 | Your output should resemble:
55 |
56 | ```bash
57 | Starting Zookeeper
58 | Zookeeper is [UP]
59 | Starting Kafka
60 | Kafka is [UP]
61 | Starting Schema Registry
62 | Schema Registry is [UP]
63 | Starting Kafka REST
64 | Kafka REST is [UP]
65 | Starting Connect
66 | Connect is [UP]
67 | Starting KSQL Server
68 | KSQL Server is [UP]
69 | Starting Control Center
70 | Control Center is [UP]
71 | ```
72 | 3. Run the following command to start the ksqlDB CLI and connect to a ksqlDB server:
73 | ```bash
74 | ksql
75 | ```
76 | - After the ksqlDB CLI starts, your terminal should resemble the following:
77 | ```bash
78 | ===========================================
79 | = _ _ ____ ____ =
80 | = | | _____ __ _| | _ \| __ ) =
81 | = | |/ / __|/ _` | | | | | _ \ =
82 | = | <\__ \ (_| | | |_| | |_) | =
83 | = |_|\_\___/\__, |_|____/|____/ =
84 | = |_| =
85 | = Event Streaming Database purpose-built =
86 | = for stream processing apps =
87 | ===========================================
88 |
89 | Copyright 2017-2020 Confluent Inc.
90 |
91 | CLI v6.0.0, Server v6.0.0 located at http://localhost:8088
92 |
93 | Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!
94 |
95 | ksql>
96 | ```
97 | - Open the `statements.ksql` file in a code editor and walk through each section via the ksqlDB CLI.
98 | - Copy and paste each statement into ksqlDB and run. Ensure each is successful.
99 |
100 | 4. In a fresh terminal, run the producer script:
101 | - Make sure you are in the KafkaESK directory, then run the script in your terminal:
102 | ```bash
103 | ./producer.sh
104 | ```
105 | Troubleshoot: If you don't have permission to run the above command, run this command first:
106 | ```bash
107 | chmod +x ./producer.sh
108 | ```
109 | - Alternatively, you can copy and paste the contents of the `producer.sh` file into the terminal and run it:
110 | ```bash
111 | kafka-producer-perf-test \
112 | --topic CLICKSTREAM_CODES \
113 | --throughput 2 \
114 | --producer-props bootstrap.servers=localhost:9092 \
115 | --payload-file server/cmd/ksql/error_data.json \
116 | --num-records 1000 &
117 | ```
118 | 1. Run the start script:
119 | ``` bash
120 | npm start
121 | ```
122 |
123 | ## **Connect from an External Source:**
124 | Create connections from within ksqlDB by utilizing the Apache Kafka built-in framework, [Kafka Connect](https://docs.confluent.io/platform/current/connect/index.html), to integrate systems by both pulling data into Kafka and pushing it downstream.
125 |
126 |
127 | Resources:
128 |
129 | - [Why Kafka Connect?](https://confluent.buzzsprout.com/186154/1265780-why-kafka-connect-ft-robin-moffatt)
130 | - An externally hosted list of connectors is maintained by Confluent at the [Confluent Hub](https://www.confluent.io/hub/).
131 | - Checkout other integration tools within the Kafka [ecosystem](https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem).
132 |
133 | ### **How to Customize:**
134 | Then, do complex property graph analysis by cleansing and preparing the data with ksqlDB and stream it to KafkaESK. You can also calculate rolling aggregates by building tables from the streams directly in ksqlDB.
135 |
136 |
137 | Cleansing and Preparing within KafkaESK Checklist:
138 | - [ ] edit the producer script
139 | - [ ] edit the consumer script
140 | - [ ] replace `error_data.json` with any static/mock data to test
141 | - [ ] edit `statements.sql` to reflect your streams and tables
142 | - [ ] in `server.js`, replace all variable names, group IDs, and topic names within the socket connection. This way, the Kafka Consumer can consume the intended messages.
143 |
144 | ```javascript
145 | const consumer = kafka.consumer({
146 | groupId: "test-group",
147 | fromBeginning: true,
148 | });
149 | consumer_404.connect();
150 | consumer_404.subscribe({ topic: "your-topic" });
151 | consumer_404.run({
152 | eachMessage: async ({ topic, partition, message }) => {
153 | socket.broadcast.emit("your-topic", message.value.toString());
154 | },
155 | });
156 | ```
157 | - [ ] in each component that live in the `client` directory, replace within the socket event listeners the correct event name. (The event name can be referenced from `server.js`. They should be the corresponding topic names.)
158 |
159 | ```javascript
160 | socket.on('your-topic', (data) => {
161 | })
162 | ```
163 |
164 | ## **Contributors:**
165 |
166 | Ai Mi Bui | Brooke Luro | Chelsea Harris | Spencer Flohr
167 |
168 | ### **How to Contribute:**
169 | We are happy to have contributions, whether for trivial cleanups or big new features!
170 | ### Reporting an Issue:
171 | Reporting potential issues are more than welcome as a significant contribution to the project. All bugs, tasks, or enhancements are tracked as [GitHub issues](https://github.com/oslabs-beta/kafkaESK/issues). Issues that may be a good start for first-time contributors are labeled with "good first issue".
172 | If you have a question or simply are not sure if it is really an issue or not, please [contact us]() first before creating a new ticket.
173 | ### Becoming a Committer:
174 | We are always interested in adding new contributors. What we look for is a series of contributions, good taste, and an ongoing interest in the project. KafkaESK considers the following guidelines for promoting new committers:
175 | - Made significant contributions in areas such as design, code, and/or documentation. The following are some examples (list not exclusive):
176 | - Fixed critical bugs (including performance improvements).
177 | - Made major tech-debt cleanup.
178 | - Made major documentation improvements.
179 |
180 | ## License
181 | MIT License
182 |
--------------------------------------------------------------------------------
/__tests__/enzyme.js:
--------------------------------------------------------------------------------
1 | import React from 'react';
2 | import { configure, shallow } from 'enzyme';
3 | import Adapter from 'enzyme-adapter-react-16';
4 | import LineChart from '../client/components/LineChart';
5 | import DoughnutChart from '../client/components/DoughnutChart';
6 | import BarChart from '../client/components/BarChart';
7 | import PolarChart from '../client/components/PolarChart';
8 |
9 | configure({ adapter: new Adapter() });
10 | configure({ disableLifecycleMethods: true });
11 |
12 | describe('React unit tests', () => {
13 | let wrapper;
14 |
15 | describe('LineChart', () => {
16 | beforeAll(() => {
17 | wrapper = shallow();
18 | });
19 |
20 | it('Renders component within a
tag', () => {
21 | expect(wrapper.type()).toEqual('div');
22 | });
23 |
24 | it('Renders a canvas element for the line chart', () => {
25 | expect(wrapper.find('canvas'));
26 | });
27 | });
28 |
29 | describe('DoughnutChart', () => {
30 | beforeAll(() => {
31 | wrapper = shallow();
32 | });
33 |
34 | it('Renders a canvas element for the doughnut chart', () => {
35 | expect(wrapper.find('canvas'));
36 | });
37 | });
38 |
39 | describe('PolarChart', () => {
40 | beforeAll(() => {
41 | wrapper = shallow();
42 | });
43 |
44 | it('Renders a canvas element for the polar chart', () => {
45 | expect(wrapper.find('canvas'));
46 | });
47 | });
48 |
49 | describe('BarChart', () => {
50 | beforeAll(() => {
51 | wrapper = shallow();
52 | });
53 |
54 | it('Renders a canvas element for the line chart', () => {
55 | expect(wrapper.find('canvas'));
56 | });
57 | });
58 | });
59 |
--------------------------------------------------------------------------------
/client/App.jsx:
--------------------------------------------------------------------------------
1 | import React, { Component } from 'react';
2 | import Nav from './components/Nav.jsx';
3 | import LineChart from './components/LineChart.jsx';
4 | import AirtableIntegration from './components/airtable.jsx';
5 | import SubCharts from './components/subcharts.jsx';
6 | import './styles/app.scss';
7 |
8 | // eslint-disable-next-line react/prefer-stateless-function
9 | class App extends Component {
10 | constructor(props) {
11 | super(props);
12 | }
13 |
14 | render() {
15 | return (
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 | );
31 | }
32 | }
33 | export default App;
34 |
--------------------------------------------------------------------------------
/client/components/BarChart.jsx:
--------------------------------------------------------------------------------
1 | /* eslint-disable lines-between-class-members */
2 | /* eslint-disable react/destructuring-assignment */
3 | import React from 'react';
4 | import { Bar } from 'react-chartjs-2';
5 | import '../styles/mainchart.scss';
6 |
7 | class BarChart extends React.Component {
8 | constructor(props) {
9 | super(props);
10 | this.state = {
11 | // eslint-disable-next-line react/no-unused-state
12 | datasets: [
13 | {
14 | label: '404',
15 | backgroundColor: 'rgb(189,211,88)',
16 | data: [],
17 | },
18 | {
19 | label: '405',
20 | backgroundColor: 'rgb(255,255,255)',
21 | data: [],
22 | },
23 | {
24 | label: '406',
25 | backgroundColor: 'rgb(27, 176, 117)',
26 | data: [],
27 | },
28 | {
29 | label: '407',
30 | backgroundColor: 'rgb(153,151,153)',
31 | data: [],
32 | },
33 | ],
34 | };
35 | }
36 |
37 | componentDidMount() {
38 | const socket = io.connect('http://localhost:3333');
39 | // Connects 404 Error Consumer
40 | socket.on('404_ERRORS_PER_MIN', (data) => {
41 | // parse incoming data
42 | const message = JSON.parse(data);
43 |
44 | // store the current state array in a variable
45 | const currDataSets = this.state.datasets;
46 |
47 | // create a new data variable, spread current array into new array
48 | const updatedDataSets = [...currDataSets];
49 |
50 | // push incoming count to data array at index 0
51 | updatedDataSets[0].data = [message.COUNT];
52 |
53 | // set the state with the updated variable
54 | this.setState({ datasets: updatedDataSets });
55 | });
56 |
57 | // Connects 405 Error Consumer
58 | socket.on('405_ERRORS_PER_MIN', (data) => {
59 | // parse incoming data
60 | const message = JSON.parse(data);
61 |
62 | // store the current state array in a variable
63 | const currDataSets = this.state.datasets;
64 |
65 | // create a new data variable, spread current array into new array
66 | const updatedDataSets = [...currDataSets];
67 |
68 | // push incoming count to data array at index 1
69 | updatedDataSets[1].data = [message.COUNT];
70 |
71 | // set the state with the updated variable
72 | this.setState({ datasets: updatedDataSets });
73 | });
74 |
75 | // Connects 406 Error Consumer
76 | socket.on('406_ERRORS_PER_MIN', (data) => {
77 | // parse incoming data
78 | const message = JSON.parse(data);
79 |
80 | // store the current state array in a variable
81 | const currDataSets = this.state.datasets;
82 |
83 | // create a new data variable, spread current array into new array
84 | const updatedDataSets = [...currDataSets];
85 |
86 | // push incoming data to new data variable
87 | updatedDataSets[2].data = [message.COUNT];
88 |
89 | // set the state with the updated variable
90 | this.setState({ datasets: updatedDataSets });
91 | });
92 |
93 | // Connects 407 Error Consumer
94 | socket.on('407_ERRORS_PER_MIN', (data) => {
95 | // parse incoming data
96 | const message = JSON.parse(data);
97 |
98 | // store the current state array in a variable
99 | const currDataSets = this.state.datasets;
100 |
101 | // create a new data variable, spread current array into new array
102 | const updatedDataSets = [...currDataSets];
103 |
104 | // push incoming data to new data variable
105 | updatedDataSets[3].data = [message.COUNT];
106 |
107 | // set the state with the updated variable
108 | this.setState({ datasets: updatedDataSets });
109 | });
110 | }
111 | render() {
112 | return ;
113 | }
114 | }
115 |
116 | export default BarChart;
117 |
--------------------------------------------------------------------------------
/client/components/DataGenerator.jsx:
--------------------------------------------------------------------------------
1 | /* eslint-disable class-methods-use-this */
2 | import React, { Component } from 'react';
3 | import '../styles/DataGenerator.scss';
4 | // create a form
5 | // takes inputs to generate topics, producers, consumers, etc.
6 | class DataGenerator extends Component {
7 | constructor(props) {
8 | super(props);
9 | this.state = {
10 | command: '',
11 | };
12 | this.handleChange = this.handleChange.bind(this);
13 | this.handleSubmit = this.handleSubmit.bind(this);
14 | }
15 |
16 | // updates the state of command string based on form input
17 | handleChange(event) {
18 | this.setState({ [event.target.name]: event.target.value });
19 | }
20 |
21 | // should send ksql command string to server to be run in terminal
22 | handleSubmit(event) {
23 | event.preventDefault();
24 | }
25 |
26 | render() {
27 | return (
28 |
45 | );
46 | }
47 | }
48 |
49 | export default DataGenerator;
50 |
--------------------------------------------------------------------------------
/client/components/DoughnutChart.jsx:
--------------------------------------------------------------------------------
1 | import React, { Component } from 'react';
2 | import { Doughnut } from 'react-chartjs-2';
3 | import '../styles/mainchart.scss';
4 |
5 | // doughnut data array corresponds to single total for each label
6 | class DoughnutChart extends Component {
7 | constructor(props) {
8 | super(props);
9 | // state holds labels for each error type and single data point for each current count(datasets:[{data:[]}])
10 | this.state = {
11 | labels: ['404', '405', '406', '407'],
12 | datasets: [
13 | {
14 | label: 'Errors',
15 | backgroundColor: ['#BDD358', '#FFFFFF', '#1BB075', '#999799'],
16 | borderColor: 'transparent',
17 | data: [undefined, undefined, undefined, undefined],
18 | },
19 | ],
20 | };
21 | }
22 |
23 | componentDidMount() {
24 | const socket = io.connect('http://localhost:3333');
25 | // Connects 404 Error Consumer
26 | socket.on('404_ERRORS_PER_MIN', (data) => {
27 | // parse incoming data
28 | const message = JSON.parse(data);
29 |
30 | // store the current datasets array in a variable
31 | const currDataSets = this.state.datasets; // array
32 |
33 | // create a new data variable, spread current datasets into new array
34 | const updatedDataSets = [...currDataSets];
35 |
36 | updatedDataSets[0].data[0] = message.COUNT;
37 |
38 | // variable for storing updated version of state
39 | const newChartData = {
40 | ...this.state,
41 | datasets: updatedDataSets[0][data][0],
42 | };
43 |
44 | // set the state of chartData to updated version
45 | this.setState({ datasets: newChartData });
46 | });
47 | // Connects 404 Error Consumer
48 | socket.on('405_ERRORS_PER_MIN', (data) => {
49 | const message = JSON.parse(data);
50 |
51 | // store the current datasets array in a variable
52 | const currDataSets = this.state.datasets; // array
53 |
54 | // create a new data variable, spread current datasets into new array
55 | const updatedDataSets = [...currDataSets];
56 |
57 | updatedDataSets[0].data[1] = message.COUNT;
58 |
59 | // variable for storing updated version of state
60 | const newChartData = {
61 | ...this.state,
62 | datasets: updatedDataSets[0][data][1],
63 | };
64 |
65 | // set the state of chartData to updated version
66 | this.setState({ datasets: newChartData });
67 | });
68 |
69 | // Connects 406 Error Consumer
70 | socket.on('406_ERRORS_PER_MIN', (data) => {
71 | // parse incoming data
72 | const message = JSON.parse(data);
73 |
74 | // store the current datasets array in a variable
75 | const currDataSets = this.state.datasets; // array
76 |
77 | // create a new data variable, spread current datasets into new array
78 | const updatedDataSets = [...currDataSets];
79 |
80 | updatedDataSets[0].data[2] = message.COUNT;
81 |
82 | // variable for storing updated version of state
83 | const newChartData = {
84 | ...this.state,
85 | datasets: updatedDataSets[0][data][2],
86 | };
87 |
88 | // set the state of chartData to updated version
89 | this.setState({ datasets: newChartData });
90 | });
91 |
92 | // Connects 407 Error Consumer
93 | socket.on('407_ERRORS_PER_MIN', (data) => {
94 | // parse incoming data
95 | const message = JSON.parse(data);
96 |
97 | // store the current datasets array in a variable
98 | const currDataSets = this.state.datasets; // array
99 |
100 | // create a new data variable, spread current datasets into new array
101 | const updatedDataSets = [...currDataSets];
102 |
103 | updatedDataSets[0].data[3] = message.COUNT;
104 |
105 | // variable for storing updated version of state
106 | const newChartData = {
107 | ...this.state,
108 | datasets: updatedDataSets[0][data][3],
109 | };
110 |
111 | // set the state of chartData to updated version
112 | this.setState({ datasets: newChartData });
113 | });
114 | }
115 |
116 | render() {
117 | return ;
118 | }
119 | }
120 | export default DoughnutChart;
121 |
--------------------------------------------------------------------------------
/client/components/LineChart.jsx:
--------------------------------------------------------------------------------
1 | import React, { Component } from 'react';
2 | import { Line } from 'react-chartjs-2';
3 | import '../styles/lineChart.scss';
4 |
5 | class LineChart extends Component {
6 | constructor(props) {
7 | super(props);
8 | this.state = {
9 | error: null,
10 | isLoaded: false,
11 | allFilteredData: [],
12 | // eslint-disable-next-line react/no-unused-state
13 | chartData: {
14 | labels: [],
15 | datasets: [
16 | {
17 | label: '404',
18 | fill: false,
19 | lineTension: 0.1,
20 | backgroundColor: 'rgba(225,0,0,0.4)',
21 | borderColor: 'rgb(189,211,88)',
22 | borderCapStyle: 'square',
23 | borderDash: [],
24 | borderDashOffset: 0.0,
25 | borderJoinStyle: 'miter',
26 | pointBackgroundColor: 'white',
27 | pointBorderWidth: 1,
28 | pointHoverRadius: 8,
29 | pointHoverBackgroundColor: 'yellow',
30 | pointHoverBorderColor: 'brown',
31 | pointHoverBorderWidth: 2,
32 | pointRadius: 4,
33 | pointHitRadius: 10,
34 | data: [],
35 | spanGaps: true,
36 | },
37 | {
38 | label: '405',
39 | fill: false,
40 | lineTension: 0.1,
41 | borderColor: 'rgb(255,255,255)',
42 | borderCapStyle: 'butt',
43 | borderDash: [],
44 | borderDashOffset: 0.0,
45 | borderJoinStyle: 'miter',
46 | pointBackgroundColor: 'white',
47 | pointBorderWidth: 1,
48 | pointHoverRadius: 8,
49 | pointHoverBackgroundColor: 'brown',
50 | pointHoverBorderColor: 'yellow',
51 | pointHoverBorderWidth: 2,
52 | pointRadius: 4,
53 | pointHitRadius: 10,
54 | data: [],
55 | spanGaps: false,
56 | },
57 | {
58 | label: '406',
59 | fill: false,
60 | lineTension: 0.1,
61 | borderColor: 'rgb(27,176,117)',
62 | borderCapStyle: 'butt',
63 | borderDash: [],
64 | borderDashOffset: 0.0,
65 | borderJoinStyle: 'miter',
66 | pointBackgroundColor: 'white',
67 | pointBorderWidth: 1,
68 | pointHoverRadius: 8,
69 | pointHoverBackgroundColor: 'brown',
70 | pointHoverBorderColor: 'yellow',
71 | pointHoverBorderWidth: 2,
72 | pointRadius: 4,
73 | pointHitRadius: 10,
74 | data: [],
75 | spanGaps: false,
76 | },
77 | {
78 | label: '407',
79 | fill: false,
80 | lineTension: 0.1,
81 | borderColor: 'rgb(153,151,153)',
82 | borderCapStyle: 'butt',
83 | borderDash: [],
84 | borderDashOffset: 0.0,
85 | borderJoinStyle: 'miter',
86 | pointBackgroundColor: 'white',
87 | pointBorderWidth: 1,
88 | pointHoverRadius: 8,
89 | pointHoverBackgroundColor: 'brown',
90 | pointHoverBorderColor: 'yellow',
91 | pointHoverBorderWidth: 2,
92 | pointRadius: 4,
93 | pointHitRadius: 10,
94 | data: [],
95 | spanGaps: false,
96 | },
97 | ],
98 | },
99 | };
100 | }
101 |
102 | componentDidMount() {
103 | const socket = io.connect('http://localhost:3333');
104 |
105 | // Connects 404 Error Consumer
106 | socket.on('404_ERRORS_PER_MIN', (data) => {
107 | // parse incoming data
108 | const message = JSON.parse(data);
109 |
110 | // store the current datasets array in a variable
111 | const currDataSets = this.state.chartData.datasets;
112 |
113 | // create a new data variable, spread current datasets into new array
114 | const updatedDataSets = [...currDataSets];
115 |
116 | // the time stamp ("WINDOW_START") of the incoming message is already included in the chartData labels,
117 | if (this.state.chartData.labels.includes(message.WINDOW_START)
118 | // and the incoming COUNT is larger or equal to the last element in the data array,
119 | && message.COUNT >= updatedDataSets[0].data[updatedDataSets[0].data.length - 1]) {
120 | // also if the ticks on the x axis (labels) are more than 10, remove the first element(time) in the labels array
121 | if (this.state.chartData.labels.length > 10) {
122 | this.state.chartData.labels.shift();
123 | }
124 | // pop the last element(count) off the data array
125 | updatedDataSets[0].data.pop();
126 | // pop the last element(time) off the labels array
127 | this.state.chartData.labels.pop();
128 | // push the latest count from the incoming message
129 | updatedDataSets[0].data.push(message.COUNT);
130 |
131 | // variable for storing updated version of state
132 | const newChartData = {
133 | ...this.state.chartData,
134 | datasets: [...updatedDataSets],
135 | labels: this.state.chartData.labels.concat(
136 | // concat the new time from WINDOW_START in the incoming message
137 | message.WINDOW_START,
138 | ),
139 | };
140 |
141 | // set the state of chartData to updated version
142 | this.setState({ chartData: newChartData });
143 |
144 | // else the window has ended and a new window starts giving us a new time stamp(a new WINDOW_START time)
145 | } else {
146 | // check to see if the labels are currently greater than 10
147 | if (this.state.chartData.labels.length > 10) {
148 | // if true, shift the earliest label(time)
149 | this.state.chartData.labels.shift();
150 | }
151 |
152 | // push incoming count to data array at index 0
153 | updatedDataSets[0].data.push(message.COUNT);
154 |
155 | // variable for storing updated version of state
156 | const newChartData = {
157 | ...this.state.chartData,
158 | datasets: [...updatedDataSets],
159 | labels: this.state.chartData.labels.concat(
160 | // new Date().toLocaleTimeString()
161 | message.WINDOW_START,
162 | ),
163 | };
164 |
165 | // set the state with the updated variable
166 | this.setState({ chartData: newChartData });
167 | }
168 | });
169 |
170 | // Connects 405 Error Consumer
171 | socket.on('405_ERRORS_PER_MIN', (data) => {
172 | // parse incoming data
173 | const message = JSON.parse(data);
174 |
175 | // store the current datasets array in a variable
176 | const currDataSets = this.state.chartData.datasets;
177 |
178 | // create a new data variable, spread currDataSets array into new array
179 | const updatedDataSets = [...currDataSets];
180 |
181 | // the time stamp ("WINDOW_START") of the incoming message is already included in the chartData labels,
182 | if (this.state.chartData.labels.includes(message.WINDOW_START)
183 | // and the incoming COUNT is larger or equal to the last element in the data array,
184 | && message.COUNT >= updatedDataSets[1].data[updatedDataSets[1].data.length - 1]) {
185 | // pop the last element(count) off the data array
186 | updatedDataSets[1].data.pop();
187 | // pop the last element(time) off the labels array
188 | this.state.chartData.labels.pop();
189 | // push the latest count from the incoming message
190 | updatedDataSets[1].data.push(message.COUNT);
191 |
192 | // variable for storing updated version of state
193 | const newChartData = {
194 | ...this.state.chartData,
195 | datasets: [...updatedDataSets],
196 | };
197 |
198 | // set the state with the updated variable
199 | this.setState({ chartData: newChartData });
200 | } else {
201 | // push incoming count to data array at index 1
202 | updatedDataSets[1].data.push(message.COUNT);
203 |
204 | // variable for storing updated version of state
205 | const newChartData = {
206 | ...this.state.chartData, // object
207 | datasets: [...updatedDataSets], // array
208 | };
209 |
210 | // set the state with the updated variable
211 | this.setState({ chartData: newChartData });
212 | }
213 | });
214 |
215 | // Connects 406 Error Consumer
216 | socket.on('406_ERRORS_PER_MIN', (data) => {
217 | // parse incoming data
218 | const message = JSON.parse(data);
219 |
220 | // store the current datasets array in a variable
221 | const currDataSets = this.state.chartData.datasets;
222 |
223 | // create a new data variable, spread currDataSets array into new array
224 | const updatedDataSets = [...currDataSets];
225 |
226 | // the time stamp ("WINDOW_START") of the incoming message is already included in the chartData labels,
227 | if (this.state.chartData.labels.includes(message.WINDOW_START)
228 | // and the incoming COUNT is larger or equal to the last element in the data array,
229 | && message.COUNT >= updatedDataSets[2].data[updatedDataSets[2].data.length - 1]) {
230 | // pop the last element(count) off the data array
231 | updatedDataSets[2].data.pop();
232 | // pop the last element(time) off the labels array
233 | this.state.chartData.labels.pop();
234 | // push the latest count from the incoming message
235 | updatedDataSets[2].data.push(message.COUNT);
236 | console.log(updatedDataSets[2].data);
237 | const newChartData = {
238 | ...this.state.chartData, // object
239 | datasets: [...updatedDataSets],
240 | };
241 |
242 | // set the state with the updated variable
243 | this.setState({ chartData: newChartData });
244 |
245 | // else the window has ended and a new window starts giving us a new time stamp(a new WINDOW_START time)
246 | } else {
247 | // push incoming count to data array at index 2
248 | updatedDataSets[2].data.push(message.COUNT);
249 |
250 | // variable for storing updated version of state
251 | const newChartData = {
252 | ...this.state.chartData,
253 | datasets: [...updatedDataSets],
254 | };
255 |
256 | // set the state with the updated variable
257 | this.setState({ chartData: newChartData });
258 | }
259 | });
260 |
261 | // Connects 407 Error Consumer
262 | socket.on('407_ERRORS_PER_MIN', (data) => {
263 | // parse incoming data
264 | const message = JSON.parse(data);
265 |
266 | // store the current datasets array in a variable
267 | const currDataSets = this.state.chartData.datasets;
268 |
269 | // create a new data variable, spread currDataSets array into new array
270 | const updatedDataSets = [...currDataSets];
271 |
272 | // the time stamp ("WINDOW_START") of the incoming message is already included in the chartData labels,
273 | if (this.state.chartData.labels.includes(message.WINDOW_START)
274 | // and the incoming COUNT is larger or equal to the last element in the data array,
275 | && message.COUNT >= updatedDataSets[3].data[updatedDataSets[3].data.length - 1]) {
276 | // pop the last element(count) off the data array
277 | updatedDataSets[3].data.pop();
278 | // pop the last element(time) off the labels array
279 | this.state.chartData.labels.pop();
280 | // push the latest count from the incoming message
281 | updatedDataSets[3].data.push(message.COUNT);
282 |
283 | const newChartData = {
284 | ...this.state.chartData, // object
285 | datasets: [...updatedDataSets], // array
286 | };
287 |
288 | // set the state with the updated variable
289 | this.setState({ chartData: newChartData });
290 |
291 | // else the window has ended and a new window starts giving us a new time stamp(a new WINDOW_START time)
292 | } else {
293 | // push incoming count to data array at index 3
294 | updatedDataSets[3].data.push(message.COUNT);
295 |
296 | // variable for storing updated version of state
297 | const newChartData = {
298 | ...this.state.chartData,
299 | datasets: [...updatedDataSets],
300 | };
301 |
302 | // set the state with the updated variable
303 | this.setState({ chartData: newChartData });
304 | }
305 | });
306 | }
307 |
308 | render() {
309 | return (
310 |
311 |
312 |
313 |
314 |
315 |
316 | );
317 | }
318 | }
319 |
320 | // class Chart extends Component {}
321 | export default LineChart;
322 |
--------------------------------------------------------------------------------
/client/components/Nav.jsx:
--------------------------------------------------------------------------------
1 | import React, { Component } from 'react';
2 | import '../styles/nav.scss';
3 | import Logo from '../images/kafkaesk-logo.png';
4 |
5 | const Nav = (props) => (
6 |
17 | );
18 |
19 | // class Nav extends Component {
20 | // constructor(props) {
21 | // super(props);
22 | // }
23 | // render() {
24 | // return (
25 | //
35 | // );
36 | // }
37 | // }
38 |
39 | export default Nav;
40 |
--------------------------------------------------------------------------------
/client/components/PolarChart.jsx:
--------------------------------------------------------------------------------
1 | /* eslint-disable react/destructuring-assignment */
2 | import React from 'react';
3 | import { Polar } from 'react-chartjs-2';
4 | import '../styles/mainchart.scss';
5 |
6 | class PolarChart extends React.Component {
7 | constructor(props) {
8 | super(props);
9 | this.state = {
10 | data: {
11 | labels: ['404', '405', '406', '407'],
12 | datasets: [
13 | {
14 | label: '404',
15 | backgroundColor: ['#BDD358', '#FFFFFF', '#1BB075', '#999799'],
16 | borderColor: 'transparent',
17 | data: [undefined, undefined, undefined, undefined],
18 | },
19 | ],
20 | },
21 | // eslint-disable-next-line react/no-unused-state
22 | options: {
23 | title: {
24 | display: true,
25 | text: '',
26 | },
27 | },
28 | };
29 | }
30 |
31 | componentDidMount() {
32 | const socket = io.connect('http://localhost:3333');
33 |
34 | // Connects 404 Error Consumer
35 | socket.on('404_ERRORS_PER_MIN', (data) => {
36 | // parse incoming data
37 | const message = JSON.parse(data);
38 |
39 | // store the current datasets array in a variable
40 | const currDataSets = this.state.data.datasets; // array
41 |
42 | // create a new data variable, spread current datasets into new array
43 | const updatedDataSets = [...currDataSets];
44 |
45 | updatedDataSets[0].data[0] = message.COUNT;
46 |
47 | // variable for storing updated version of state
48 | const newChartData = {
49 | ...this.state.data,
50 | datasets: updatedDataSets[0][data][0],
51 | };
52 |
53 | // set the state of chartData to updated version
54 | this.setState({ data: newChartData });
55 | });
56 |
57 | // Connects 405 Error Consumer
58 | socket.on('405_ERRORS_PER_MIN', (data) => {
59 | // parse incoming data
60 | const message = JSON.parse(data);
61 |
62 | // store the current datasets array in a variable
63 | const currDataSets = this.state.data.datasets;
64 |
65 | // create a new data variable, spread current datasets into new array
66 | const updatedDataSets = [...currDataSets];
67 |
68 | updatedDataSets[0].data[1] = message.COUNT;
69 |
70 | // variable for storing updated version of state
71 | const newChartData = {
72 | ...this.state.data,
73 | datasets: updatedDataSets[0][data][1],
74 | };
75 |
76 | // set the state of chartData to updated version
77 | this.setState({ data: newChartData });
78 | });
79 |
80 | // Connects 406 Error Consumer
81 | socket.on('406_ERRORS_PER_MIN', (data) => {
82 | // parse incoming data
83 | const message = JSON.parse(data);
84 |
85 | // store the current datasets array in a variable
86 | const currDataSets = this.state.data.datasets;
87 |
88 | // create a new data variable, spread current datasets into new array
89 | const updatedDataSets = [...currDataSets];
90 |
91 | updatedDataSets[0].data[2] = message.COUNT;
92 |
93 | // variable for storing updated version of state
94 | const newChartData = {
95 | ...this.state.data,
96 | datasets: updatedDataSets[0][data][2],
97 | };
98 |
99 | // set the state of chartData to updated version
100 | this.setState({ data: newChartData });
101 | });
102 |
103 | // Connects 407 Error Consumer
104 | socket.on('407_ERRORS_PER_MIN', (data) => {
105 | // parse incoming data
106 | const message = JSON.parse(data);
107 |
108 | // store the current datasets array in a variable
109 | const currDataSets = this.state.data.datasets;
110 |
111 | // create a new data variable, spread current datasets into new array
112 | const updatedDataSets = [...currDataSets];
113 |
114 | updatedDataSets[0].data[3] = message.COUNT;
115 |
116 | // variable for storing updated version of state
117 | const newChartData = {
118 | ...this.state.data,
119 | datasets: updatedDataSets[0][data][3],
120 | };
121 |
122 | // set the state of chartData to updated version
123 | this.setState({ data: newChartData });
124 | });
125 | }
126 |
127 | render() {
128 | return ;
129 | }
130 | }
131 |
132 | export default PolarChart;
133 |
--------------------------------------------------------------------------------
/client/components/airtable.jsx:
--------------------------------------------------------------------------------
1 | import React, { Component } from 'react';
2 | import '../styles/airtable.scss';
3 | import airtablelogo from '../images/airtable-logo.png';
4 | import DataGenerator from './DataGenerator.jsx';
5 | // require and create a new instance of airtable connecting to base with an api key
6 | const Airtable = require('airtable');
7 |
8 | const base = new Airtable({ apiKey: 'keyrj2qx48S8OCScO' }).base(
9 | 'appSgoXde9kcXPb0Q',
10 | );
11 |
12 | class AirtableIntegration extends Component {
13 | constructor(props) {
14 | super(props);
15 | this.state = {
16 | Name: '',
17 | Priority: '',
18 | Status: 'Assigned',
19 | Description: '',
20 | 'Error Number': '',
21 | 'Assigned to': '',
22 | };
23 | this.handleChange = this.handleChange.bind(this);
24 | this.handleSubmit = this.handleSubmit.bind(this);
25 | }
26 |
27 | handleChange(event) {
28 | // updates state based on form input values
29 | this.setState({ [event.target.name]: event.target.value });
30 | }
31 |
32 | handleSubmit(event) {
33 | event.preventDefault();
34 | // on form submit creates a new record in airtable
35 | base('Bugs and issues').create(
36 | [
37 | {
38 | fields: this.state,
39 | },
40 | ],
41 | (err, records) => {
42 | if (err) {
43 | console.error(err);
44 | return;
45 | }
46 | records.forEach((record) => {
47 | console.log(record.getId());
48 | });
49 | },
50 | );
51 | this.setState({
52 | Name: '',
53 | Priority: '',
54 | Status: 'Assigned',
55 | Description: '',
56 | 'Error Number': '',
57 | 'Assigned to': '',
58 | });
59 | }
60 |
61 | render() {
62 | return (
63 |
67 |