├── .gitignore ├── DANCE ├── LICENSE ├── README.md └── storm ├── Dockerfile ├── cluster.xml ├── entrypoint.sh └── worker.xml /.gitignore: -------------------------------------------------------------------------------- 1 | docker-storm.iml 2 | .idea/* -------------------------------------------------------------------------------- /DANCE: -------------------------------------------------------------------------------- 1 | (Copied from WikiHow) 2 | 3 | 1. Pinch your fingers and thumbs together in front of your chest 4 | 4 times. 5 | 2. Flap your arms four times. 6 | 3. Wiggle side to side four times while getting your back side as 7 | close to the ground as you can. 8 | 4. Clap four times. 9 | 5. Repeat steps 1-4 until you hear the swing-like music. 10 | 6. Swing around with a partner or group until the swing-like music 11 | ends. Start back at the beginning. 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Baqend 2 | All rights reserved. 3 | 4 | Chicken Dance License v0.2 5 | http://supertunaman.com/cdl/ 6 | 7 | Redistribution and use in source and binary forms, with 8 | or without modification, are permitted provided that the 9 | following conditions are met: 10 | 11 | 1. Redistributions of source code must retain the 12 | above copyright notice, this list of conditions and 13 | the following disclaimer. 14 | 2. Redistributions in binary form must reproduce the 15 | above copyright notice, this list of conditions and 16 | the following disclaimer in the documentation and/or 17 | other materials provided with the distribution. 18 | 3. Neither the name of Baqend nor the names 19 | of its contributors may be used to endorse or promote 20 | products derived from this software without specific 21 | prior written permission. 22 | 4. An entity wishing to redistribute in binary form or 23 | include this software in their product without 24 | redistribution of this software's source code with the 25 | product must also submit to these conditions where 26 | applicable: 27 | * For every thousand (1000) units distributed, at 28 | least half of the employees or persons 29 | affiliated with the product must listen to the 30 | "Der Ententanz" (AKA "The Chicken Dance") as 31 | composed by Werner Thomas for no less than two 32 | (2) minutes 33 | * For every twenty-thousand (20000) units distributed, 34 | one (1) or more persons affiliated with the entity 35 | must be recorded performing the full Chicken Dance, 36 | in an original video at the entity's own expense, 37 | and a video encoded in OGG Theora format or a format 38 | and codec specified by Baqend, at least three (3) 39 | minutes in length, must be submitted to Baqend, 40 | provided Baqend's contact information. Any and all 41 | copyrights to this video must be transfered to 42 | Baqend. The dance featured in the video 43 | must be based upon the instructions on how to perform 44 | the Chicken Dance that you should have received with 45 | this software. 46 | * Any employee or person affiliated with the product 47 | must be prohibited from saying the word "gazorninplat" in 48 | public at all times, as long as distribution of the 49 | product continues. 50 | 51 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 52 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 53 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 54 | FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 55 | COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 56 | INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 57 | BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 58 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 59 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 60 | LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 61 | ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 62 | POSSIBILITY OF SUCH DAMAGE. Baqend ACCEPTS NO LIABILITY FOR 63 | ANY INJURIES OR EXPENSES SUSTAINED IN THE ACT OF FULFILLING ANY OF 64 | THE ABOVE TERMS AND CONDITIONS, ACCIDENTAL OR OTHERWISE. 65 | 66 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # docker-storm 2 | 3 | We created this image to ease deployment of Apache Storm. More related main.resources: 4 | 5 | - Our [tutorial on how to use Storm with Docker Swarm](https://github.com/Baqend/tutorial-swarm-storm) 6 | - the [baqend/storm on Docker Hub](https://hub.docker.com/r/baqend/storm/) 7 | - the [the baqend/storm on GitHub](https://github.com/Baqend/docker-storm) 8 | 9 | 10 | The Docker image launches Apache Storm. By default, the `storm.local.hostname` property is set already, but you have to provide all other arguments that you would normally provide. 11 | 12 | ## No need for a `storm.yaml`! 13 | 14 | Storm allows you to provide any property via command line arguments like this: 15 | 16 | bin/storm ui \ 17 | -c storm.local.hostname="nimbus" 18 | You are not restricted to strings, but can also provide integer lists: 19 | 20 | bin/storm supervisor \ 21 | -c supervisor.slots.ports="[6700,6701]" 22 | Or lists of strings: 23 | 24 | bin/storm supervisor \ 25 | -c storm.zookeeper.servers="[\"zk1\",\"zk2\",\"zk3\"]" 26 | 27 | You can look up [**all the parameters in the source code**](https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/Config.java). 28 | 29 | 30 | ### Providing ZooKeeper the easy way 31 | 32 | This image also lets you specify the ZooKeeper servers with a simple list (e.g. `"zk1,zk2,zk3"`) instead of the more verbose format Storm requires (e.g. `"[\"zk1\",\"zk2\",\"zk3\"]"`). 33 | So if your ZooKeeper cluster is made up of `zk1`, `zk2` and `zk3`, you can tell Storm to use this cluster by providing the following Docker environment variable: 34 | 35 | -e STORM_ZOOKEEPER_SERVERS="zk1,zk2,zk3" 36 | 37 | Internally, this is passed to Storm as 38 | 39 | -c storm.zookeeper.servers="[\"zk1\",\"zk2\",\"zk3\"]" 40 | 41 | 42 | ### Default Values 43 | 44 | The baqend/storm Docker image uses the following default values that can be overridden by providing the corresponding arguments in the `run` statement (e.g. `-c nimbus.seeds="[\"nimbus1\",\"nimbus2\"]"`): 45 | 46 | - `-c nimbus.seeds="[\"nimbus\"]"`: is also overridden by `nimbus.host` (depreciated since 1.0.0) 47 | - `-c storm.local.hostname=`: by default, the image assigns the value `nimbus` when `nimbus` is provided as argument; the docker container ID is used as hostname else 48 | - `-c supervisor.slots.ports="[6700,6701,6702,6703]"`: only when the `supervisor` argument is provided 49 | 50 | In consequence, only the ZooKeeper servers and the role argument (either `ui `, `nimbus` or `supervisor`) have to be provided in a `docker run` statement. 51 | 52 | 53 | ### Further Reading 54 | 55 | For an example of how we use this image, also see our [tutorial on how to use Storm with Docker Swarm](https://github.com/Baqend/tutorial-swarm-storm). 56 | 57 | -------------------------------------------------------------------------------- /storm/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM baqend/java:8u171 2 | MAINTAINER wingerath wolfram@wingerath.org 3 | 4 | # The used storm version can also be supplied like this with the build command like this: 5 | # --build-arg BIN_VERSION=apache-storm-1.0.2 6 | ARG BIN_VERSION=apache-storm-1.2.2 7 | 8 | WORKDIR /usr/share/storm 9 | 10 | # supervisor: worker ports 11 | EXPOSE 6700 6701 6702 6703 12 | # logviewer 13 | EXPOSE 8000 14 | # DRPC and remote deployment 15 | EXPOSE 6627 3772 3773 16 | 17 | # Install and set everything up 18 | RUN \ 19 | alias python=python3 \ 20 | && apt-get update -y \ 21 | && apt-get install -y \ 22 | python \ 23 | && wget -q -N http://mirrors.gigenet.com/apache/storm/${BIN_VERSION}/${BIN_VERSION}.tar.gz \ 24 | && tar --strip-components=1 -C /usr/share/storm -xvf ${BIN_VERSION}.tar.gz \ 25 | && rm ${BIN_VERSION}.tar.gz \ 26 | && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 27 | 28 | # add startup script 29 | ADD entrypoint.sh entrypoint.sh 30 | ADD cluster.xml log4j2/cluster.xml 31 | ADD worker.xml log4j2/worker.xml 32 | RUN chmod +x entrypoint.sh 33 | 34 | ENTRYPOINT ["/usr/share/storm/entrypoint.sh"] 35 | -------------------------------------------------------------------------------- /storm/cluster.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | -------------------------------------------------------------------------------- /storm/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ########################### 4 | # storm.zookeeper.servers # 5 | ########################### 6 | ZOOKEEPER_SERVERS_ESCAPED= 7 | if ! [ -z "$STORM_ZOOKEEPER_SERVERS" ]; then 8 | # All ZooKeeper server IPs in an array 9 | IFS=', ' read -r -a ZOOKEEPER_SERVERS_ARRAY <<< "$STORM_ZOOKEEPER_SERVERS" 10 | for index in "${!ZOOKEEPER_SERVERS_ARRAY[@]}" 11 | do 12 | ZOOKEEPER_SERVERS_ESCAPED=$ZOOKEEPER_SERVERS_ESCAPED,"\\\"${ZOOKEEPER_SERVERS_ARRAY[index]}\\\"" 13 | done 14 | ZOOKEEPER_SERVERS_ESCAPED=[${ZOOKEEPER_SERVERS_ESCAPED:1}] 15 | ZOOKEEPER_SERVERS_ESCAPED=" -c storm.zookeeper.servers=\"$ZOOKEEPER_SERVERS_ESCAPED\"" 16 | fi 17 | 18 | ######################## 19 | # storm.local.hostname # 20 | ######################## 21 | HOST=" -c storm.local.hostname=$(hostname -i | awk '{print $1;}')" 22 | # For the nimbus, apply "nimbus" as default hostname 23 | for arg in "$@" 24 | do 25 | if [[ $arg == "nimbus" ]] ; then 26 | HOST=" -c storm.local.hostname=\"nimbus\"" 27 | fi 28 | done 29 | 30 | ########################## 31 | # supervisor.slots.ports # 32 | ########################## 33 | SUPERVISOR_SLOTS= 34 | # For a supervisor, set worker slots 35 | for arg in "$@" 36 | do 37 | if [[ $arg == "supervisor" ]] ; then 38 | SUPERVISOR_SLOTS=" -c supervisor.slots.ports=\"[6700,6701,6702,6703]\"" 39 | fi 40 | done 41 | 42 | ################ 43 | # nimbus.seeds # 44 | ################ 45 | NIMBUS_SEEDS=" -c nimbus.seeds=\"[\\\"nimbus\\\"]\"" 46 | 47 | ################################################### 48 | # Make sure provided arguments are not overridden # 49 | ################################################### 50 | for arg in "$@" 51 | do 52 | if [[ $arg == *"storm.zookeeper.servers"* ]] ; then 53 | ZOOKEEPER_SERVERS_ESCAPED= 54 | fi 55 | if [[ $arg == *"storm.local.hostname"* ]] ; then 56 | HOST= 57 | fi 58 | if [[ $arg == *"supervisor.slots.ports"* ]] ; then 59 | SUPERVISOR_SLOTS= 60 | fi 61 | if [[ $arg == *"nimbus.seeds"* ]] ; then 62 | NIMBUS_SEEDS= 63 | fi 64 | if [[ $arg == *"nimbus.host"* ]] ; then 65 | NIMBUS_SEEDS= 66 | fi 67 | done 68 | 69 | CMD="exec bin/storm $@$NIMBUS_SEEDS$SUPERVISOR_SLOTS$HOST$ZOOKEEPER_SERVERS_ESCAPED" 70 | 71 | echo "$CMD" 72 | eval "$CMD" 73 | -------------------------------------------------------------------------------- /storm/worker.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | --------------------------------------------------------------------------------