├── src ├── confluent-control-center.service ├── kafka-connect.service ├── schema-registry.service ├── kafka.service └── zookeeper.service ├── LICENSE └── README.md /src/confluent-control-center.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Confluent Control Center 3 | After=network.target 4 | 5 | [Service] 6 | Type=simple 7 | User=confluent-cc 8 | Group=confluent-cc 9 | ExecStart=/usr/bin/control-center-start /etc/confluent-control-center/control-center.properties 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | 14 | -------------------------------------------------------------------------------- /src/kafka-connect.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Apache Kafka Connect 3 | After=network.target kafka.service schema-registry.service 4 | 5 | [Service] 6 | Type=forking 7 | User=kafka 8 | Group=kafka 9 | Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10040 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false" 10 | Environment="LOG_DIR=/var/log/kafka" 11 | # Uncomment the following line to enable authentication for the kafka connect 12 | # Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/kafka_connect_jaas.conf" 13 | ExecStart=/usr/bin/connect-distributed -daemon /etc/kafka/connect-distributed.properties 14 | 15 | [Install] 16 | WantedBy=multi-user.target 17 | -------------------------------------------------------------------------------- /src/schema-registry.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Confluent Schema Registry 3 | After=network.target remote-fs.target kafka.service 4 | 5 | [Service] 6 | Type=forking 7 | User=kafka 8 | Group=kafka 9 | Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10050 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false" 10 | Environment="LOG_DIR=/var/log/kafka" 11 | # Uncomment the following line to enable authentication for the schema registry 12 | # Environment="SCHEMA_REGISTRY_OPTS=-Djava.security.auth.login.config=/etc/schema-registry/schema_registry_jaas.conf" 13 | ExecStart=/usr/bin/schema-registry-start -daemon /etc/schema-registry/schema-registry.properties 14 | ExecStop=/usr/bin/schema-registry-stop 15 | 16 | [Install] 17 | WantedBy=multi-user.target 18 | -------------------------------------------------------------------------------- /src/kafka.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Confluent Kafka Broker 3 | After=network.target network-online.target remote-fs.target zookeeper.service 4 | 5 | [Service] 6 | Type=forking 7 | User=kafka 8 | Group=kafka 9 | Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10030 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false" 10 | Environment="LOG_DIR=/var/log/kafka" 11 | # Uncomment the following line to enable authentication for the broker 12 | # Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/kafka-jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf" 13 | ExecStart=/usr/bin/kafka-server-start -daemon /etc/kafka/server.properties 14 | ExecStop=/usr/bin/kafka-server-stop 15 | SuccessExitStatus=143 16 | 17 | [Install] 18 | WantedBy=multi-user.target 19 | -------------------------------------------------------------------------------- /src/zookeeper.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Confluent ZooKeeper 3 | After=network.target network-online.target remote-fs.target 4 | 5 | [Service] 6 | Type=forking 7 | User=zookeeper 8 | Group=zookeeper 9 | Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10020 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false" 10 | Environment="LOG_DIR=/var/log/zookeeper" 11 | # Uncomment the following line to enable authentication for the zookeeper 12 | # Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/zookeeper_server_jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf" 13 | ExecStart=/usr/bin/zookeeper-server-start -daemon /etc/kafka/zookeeper.properties 14 | ExecStop=/usr/bin/zookeeper-server-stop 15 | SuccessExitStatus=143 16 | 17 | [Install] 18 | WantedBy=multi-user.target 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Thomas Hamm 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # systemd services for Confluent Platform 2 | 3 | Available for: 4 | - Confluent Kafka 5 | - Confluent Schema Registry 6 | - Confluent Kafka Connect 7 | - Confluent ZooKeeper 8 | - Confluent Control Center 9 | 10 | ## Prerequisites 11 | 12 | ### Users, groups and directories 13 | - Create the users and group: 14 | - kafka 15 | - zookeeper 16 | - confluent-cc 17 | - Create directories: 18 | - /var/log/kafka 19 | - /var/log/zookeeper 20 | - /var/log/confluent-control-center 21 | - Set appropriate permissions 22 | 23 | ### Kafka Connect Log4j settings 24 | Edit /etc/kafka/connect-log4j.properties, add/change the following lines 25 | ``` 26 | log4j.rootLogger=INFO, kafkaConnectAppender 27 | 28 | log4j.appender.kafkaConnectAppender=org.apache.log4j.DailyRollingFileAppender 29 | log4j.appender.kafkaConnectAppender.DatePattern='.'yyyy-MM-dd-HH 30 | log4j.appender.kafkaConnectAppender.File=${kafka.logs.dir}/connect.log 31 | log4j.appender.kafkaConnectAppender.layout=org.apache.log4j.PatternLayout 32 | log4j.appender.kafkaConnectAppender.layout.ConversionPattern=[%d] %p %m (%c)%n 33 | ``` 34 | 35 | ### Confluent Control Center 36 | For logging into files, rename log4j-rolling.properties to log4j.properties.
37 | By default Control Center will log to /tmp. To change, set the paths in log4j.properties after renaming. 38 | 39 | # Installation 40 | Put the unit file into the location: /etc/systemd/system/
41 | Reload systemd: 42 | ``` 43 | systemctl daemon-reload 44 | ``` 45 | 46 | For auto restart of these services use: 47 | ``` 48 | systemctl enable servicename.service 49 | ``` 50 | 51 | # Additional settings 52 | Some Apache Kafka settings are set via JVM parameters and environment variables. For example security settings or log paths. 53 | 54 | To add environment parameters for systemd services "Environment" entries can be added to the "Service" section of a service. As an example the following line can be added to specify the path to the jaas configuration file needed to enable Kerberos for authentication. 55 | 56 | ``` 57 | Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/kafka-jaas.conf" 58 | ``` 59 | 60 | More information about Apache Kafka settings and environment variables can be found in the official documentation. 61 | 62 | # Summary 63 | JMX is enabled by default. To disable JXM remove the "Environment=" line. 64 | 65 | ## Confluent Kafka 66 | Kafka properties: /etc/kafka/server.properties
67 | Logs: /var/log/kafka
68 | JMX Port: 10030 69 | 70 | ## Confluent Schema Registry 71 | Schema Registry properties: /etc/schema-registry/schema-registry.properties
72 | JMX Port: 10050 73 | 74 | ## Confluent Kafka Connect 75 | Kafka Connect properties: /etc/kafka/connect-distributed.properties
76 | Log: /var/log/kafka/connect.log
77 | JMX Port: 10040 78 | 79 | ## Confluent ZooKeeper 80 | ZooKeeper properties: /etc/kafka/zookeeper.properties
81 | Logs: /var/log/zookeeper
82 | JMX Port: 10020
83 | --------------------------------------------------------------------------------