├── VERSION ├── .gitignore ├── requirements.txt ├── TODO ├── Dockerfile.monitor ├── NOTES ├── THANKS ├── fail2ban ├── f2bcluster.jail.txt └── fail2bancluster.conf ├── Changelog ├── util.py ├── configparsing.py ├── FILES ├── fail2ban-monitor.py ├── fail2ban-subscriber.py ├── fail2ban-publisher.py ├── fail2ban-cluster.conf ├── README.md ├── subscriber.py ├── daemon.py ├── publisher.py ├── monitor.py └── LICENSE /VERSION: -------------------------------------------------------------------------------- 1 | 0.2 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | *pyc 3 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pyzmq==17.0.0b3 2 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | * add better authentication 2 | * evaluate need for workers 3 | * fail2ban-subscriber: add NOBLOCK, to fix thread-stop issue 4 | -------------------------------------------------------------------------------- /Dockerfile.monitor: -------------------------------------------------------------------------------- 1 | # you will need to use bindmount to get fail2ban.log to be 2 | # accesible by this container on /var/log/fail2ban.log 3 | FROM python:3.5 4 | COPY requirements.txt 5 | COPY fail2ban-cluster.conf /code 6 | COPY fail2ban-monitor.py /code 7 | COPY daemon.py /code 8 | COPY monitor.py /code 9 | WORKDIR /code 10 | RUN pip install -r requirements.txt 11 | RUN chmod +x /code/fail2ban-monitor.py 12 | CMD ["/code/fail2ban-monitor.py"] 13 | -------------------------------------------------------------------------------- /NOTES: -------------------------------------------------------------------------------- 1 | Welcome to the first release of fail2ban-zmq-tools, also known as 2 | fail2ban-cluster. 3 | 4 | I, buanzo, am running the fail2ban-publisher on f2bcluster.buanzo.org 5 | You will need to mail me (buanzo@buanzo.com.ar) to obtain a TOKEN (see 6 | fail2ban-cluster.conf for token configuration for [monitor]). 7 | 8 | If you rather run your own Publisher instance, just create a token, and 9 | configure it on your fail2ban-cluster.conf [publisher] section, then use 10 | that token in every Monitor configuration. 11 | -------------------------------------------------------------------------------- /THANKS: -------------------------------------------------------------------------------- 1 | I would like to thank Cyril Jacquier, original author of fail2ban, for 2 | accepting me into the team, back when fail2ban stopped working because of 3 | Python version bumps everywhere. 4 | 5 | Then to Yaroslav Halchenko, who has an amazing love for fail2ban and the 6 | people around it. He always has a :) in his replies. You are awesome, man. 7 | 8 | And finally, to Harrison Johnson, for jumping on the fail2ban-cluster wagon 9 | and being the first contributor and first tester. 10 | 11 | THANK YOU ALL. 12 | 13 | Live long and prosper. 14 | 15 | 16 | Buanzo. 17 | -------------------------------------------------------------------------------- /fail2ban/f2bcluster.jail.txt: -------------------------------------------------------------------------------- 1 | # You have to add this jail to your fail2ban's jail.conf 2 | # as well as the filter file (located in this same directory) to your 3 | # fail2ban's filter.d dir. 4 | # Tweak to your desires. 5 | # Remember, this jail and the filter were designed for the first release of 6 | # fail2ban-zmq-tools aka fail2ban-cluster. 7 | # So, feel free to make lots of changes. 8 | 9 | [fail2bancluster] 10 | enabled = true 11 | port = 22,443,80,25,110,143,3128,11443 12 | protocol = tcp 13 | filter = fail2bancluster 14 | logpath = /var/log/auth.log 15 | maxretry = 1 16 | -------------------------------------------------------------------------------- /fail2ban/fail2bancluster.conf: -------------------------------------------------------------------------------- 1 | # Fail2Ban filter to be used with fail2ban-zmq-tools "Subscriber" module 2 | # ask Buanzo about it 3 | # 4 | # Example from /var/log/auth.log (by default Subscriber logs via syslog to LOG_AUTH) 5 | # 6 | # Aug 10 11:40:45 mx2 /fail2ban-subscriber.py[15572]: fail2ban-zmq-tools Subscriber: Got broadcast message: mx5.mailfighter.net|ssh|Unban|1.2.3.4 7 | 8 | [INCLUDES] 9 | 10 | before = common.conf 11 | 12 | [Definition] 13 | 14 | _daemon = fail2ban-subscriber.py 15 | 16 | failregex = ^%(__prefix_line)s%(__hostname)s.*Subscriber\: Got broadcast message\: .*\|.*\|Ban\|$ 17 | ignoreregex = 18 | 19 | -------------------------------------------------------------------------------- /Changelog: -------------------------------------------------------------------------------- 1 | ChangeLog 2 | 3 | 2015-08-11 - Added logfile truncation/rotation to fail2ban-monitor 4 | Added incoming message validation to subscriber and publisher 5 | Added equal-hostname checking for subscriber (to avoid acting 6 | on own messages) 7 | 2015-08-10 - Package is released to test-group, f2bmqc.buanzo.org access is 8 | granted per-request. Credentials and details arranged via email 9 | to buanzo@buanzo.com.ar. 10 | 2015-08-07 - fail2ban-mq-cluster Client gets its own configuration section, 11 | allowing to specify address of f2b-mq-cluster Server, auth 12 | creds, and working model (fail2ban-client ban/unban, or log 13 | writing for f2b processing) 14 | 2015-07-31 - Added /simple/ authentication layer to fail2ban-mq-cluster Server 15 | and fail2ban-mq-cluster Monitor. 16 | 2015-07-30 - Started preparing package for closed-testing with interested 17 | parties from fail2ban-users and other members of infosec 18 | community. configuration parsing added. 19 | 2015-07-29 - Working PoC - Sent announce to fail2ban-users 20 | 2015-07-28 - Started coding fail2ban-mq-cluster PoC 21 | -------------------------------------------------------------------------------- /util.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """Some general purpose utilities.""" 3 | 4 | import socket 5 | import re 6 | 7 | 8 | class f2bcUtils: 9 | """Utility functions for fail2ban-cluster.""" 10 | # valid IPv4 IP address? 11 | def valid_ipv4(address): 12 | try: 13 | socket.inet_aton(address) 14 | return True 15 | except Exception: 16 | return False 17 | 18 | # valid jailname? must only contain a-z,A-Z,0-9 and _- 19 | # TODO: verify fail2ban jailname constraints, including length 20 | def valid_jailname(jailname): 21 | match = re.match("^[a-zA-Z0-9_-]*$", jailname) 22 | return match is not None 23 | 24 | def is_valid_hostname(hostname): 25 | if len(hostname) > 255: 26 | return False 27 | if hostname[-1] == ".": 28 | # strip exactly one dot from the right, if present 29 | hostname = hostname[:-1] 30 | allowed = re.compile("(?!-)[A-Z\d-]{1,63}(? 37 | # This method checks which fail2ban jails are enabled, and only processes matching messages 38 | # 39 | # log: just log, no action. this is useful if you want to use fail2ban to parse and execute actions 40 | # for this to do anything useful, you have to use the fail2ban jail and filter.d file that comes 41 | # with fail2ban-zmq-tools, in the fail2ban directory, called "fail2bancluster.conf". Copy it to fail2ban/filter.d. 42 | # Add the example jail, cointained in "f2bcluster.jail.txt", to your fail2ban's jail.conf 43 | # 44 | # NOTE: fail2ban-zmq-tools (or fail2ban-cluster, however you want to call this set of tools) 45 | # logs via syslog LOG_AUTH facility. The included jail file targets /var/log/auth.log 46 | # 47 | # If you can implement new subscriberactions that would be awesome 48 | subscriberaction = log 49 | 50 | # 51 | # Configuration for fail2ban-publisher, the central message processing and distribution hub 52 | # for fail2ban-cluster aka fail2ban-zmq-tools 53 | # 54 | 55 | [publisher] 56 | pidfile = /var/run/fail2ban-publisher.pid 57 | broadcasterbindurl = tcp://*:5680 # address to which subscribers will connect to to receive broadcasts 58 | replybindurl = tcp://*:5678 # address to which Monitors will send messages (usually the same host, different port) 59 | 60 | # Should Publisher demand auth from Monitors? (see [monitor], too) 61 | auth = true 62 | authtoken = IF_YOU_RUN_YOUR_OWN_PUBLISHER_JUST_TYPE_A_SECRET_STRING_HERE_AND_GIVE_IT_TO_YOUR_MONITORS 63 | 64 | #TODO: add support for multiple tokens 65 | 66 | 67 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Fail2ban-zmq-tools 2 | ------------------ 3 | 4 | Author: Arturo 'Buanzo' Busleiman 5 | 6 | This is a very simple fail2ban clustering solution that works ad-hoc to 7 | a running fail2ban instance. It consists of three modules: publisher, 8 | subscriber, monitor. 9 | 10 | The monitor needs access to fail2ban.log to, ehem, monitor fail2ban.log for 11 | Ban/UnBan messages, which it will forward to Publisher. 12 | 13 | Publisher gets messages, and re-distributes it to the Subscribers. 14 | 15 | Hence, for your cluster, you need n monitors, n subscribers, but ONE 16 | publisher. 17 | 18 | Docker support is barely getting into the project as of 201801. 19 | 20 | This is a set of python scripts to be run in parallel to an existing 21 | fail2ban installation to allow multiple instances of fail2ban running in 22 | different servers to 'cluster up', and share blocked IP addresses, for 23 | proactive protection, by means of zeromq message queuing system. 24 | 25 | DESIGN: 26 | ======= 27 | 28 | There are three individual Processes: Monitor, Publisher and Subscriber. 29 | There is ONE central configuration file, with appropiate sections for each 30 | process. You do not need to run all processes, just the ones you want. Keep 31 | reading. 32 | 33 | ------------------------------------------- 34 | fail2ban-zmq-tools Cluster Subscriber 35 | (started by ./fail2ban-subscriber.py start) 36 | ------------------------------------------- 37 | 38 | The Subscriber receives messages from the Publisher, and depending on the 39 | subscriberaction configuration option, does one thing or another with the 40 | message. The idea is that the fail2ban instance that is local to the subscriber 41 | will take this message and ban/unban accordingly. 42 | 43 | NOTE: You *do not* need to run a Publisher instance (or have access tokens for another 44 | Publisher, just like mine, which is the default one) if you only want to 45 | subscribe to a Publisher. 46 | 47 | ---------------------------------------- 48 | fail2ban-zmq-tools Cluster Monitor 49 | (started by ./fail2ban-monitor.py start) 50 | ---------------------------------------- 51 | 52 | This script monitors /var/log/fail2ban.log (you can change the fail2ban log 53 | location by editing the fail2ban-cluster.conf file). When it detects a new 54 | ban or unban, it transfers this information to the Publisher processes, by 55 | means of a zeromq (www.zeromq.org) REQUEST/REPLY socket. 56 | 57 | 58 | ------------------------------------------ 59 | fail2ban-zmq-tools Cluster Publisher 60 | (started by ./fail2ban-publisher.py start) 61 | ------------------------------------------ 62 | 63 | The Publisher receives messages using the aforementioned protocol. When it 64 | gets a message, it replies back with the same message (as a simple ACK), 65 | then broadcasts it to all the connected Subscribers. 66 | 67 | 68 | 69 | 70 | 71 | NOTE: This software is an adaptation of fail2ban-cluster, a piece of 72 | software I never fully released publicly. The fail2ban-cluster.conf general 73 | configuration file is named in memoriam of that project :) 74 | 75 | The "banip" fail2ban-client command was part of that original idea. 76 | 77 | -------------------------------------------------------------------------------- /subscriber.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import time 3 | import sys 4 | import os 5 | import re 6 | import threading 7 | import queue 8 | import zmq 9 | import socket 10 | import syslog 11 | 12 | from util import f2bcUtils 13 | 14 | syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_AUTH) 15 | 16 | 17 | class Subscriber(threading.Thread): 18 | def __init__(self, subscriberconfig=None, num_worker_threads=1): 19 | threading.Thread.__init__(self) 20 | self._stopevent = threading.Event() 21 | self.subscriberconfig = subscriberconfig 22 | self.zmqPublisher = self.subscriberconfig['zmqpublisherurl'] 23 | self.publisheraction = self.subscriberconfig['subscriberaction'] 24 | # we want to ignore our own messages. 25 | self.hostname = socket.gethostname() 26 | 27 | def run(self): 28 | self.zmqSubscriberContext = zmq.Context() 29 | self.zmqSubscriberSock = self.zmqSubscriberContext.socket(zmq.SUB) 30 | # TODO: fix prefix handling 31 | self.zmqSubscriberSock.setsockopt_string(zmq.SUBSCRIBE, "") 32 | self.zmqSubscriberSock.connect(self.zmqPublisher) 33 | # Wait for messages, when one is received, process it 34 | while not self._stopevent.isSet(): 35 | message = self.zmqSubscriberSock.recv_string() 36 | # TODO: INPUT CHECK HERE - apply regex and such against message 37 | # parts (jail, ip, etc) TODO: act according to publisheraction 38 | # [see fail2ban-cluster.conf] 39 | msg = message.split('|') 40 | Hostname = msg[0] 41 | Jail = msg[1] 42 | Action = msg[2] 43 | Attacker = msg[3] 44 | # Run a series of tests on incoming messages 45 | if not f2bcUtils.is_valid_hostname(Hostname): 46 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 47 | Invalid hostname in incoming message.") 48 | continue 49 | # If hostname matches our hostname, output warning, using 50 | # different syntax to avoid triggering the fail2bancluster jail 51 | # filter. 52 | if Hostname == self.hostname: 53 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 54 | Got equal hostname broadcast. \ 55 | Our hostname is %s" % self.hostname) 56 | continue 57 | # Only accepted ban or unban actions 58 | if not f2bcUtils.is_valid_action(Action): 59 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 60 | Unknown action received.") 61 | continue 62 | # Only accept valid IPv4 IP addresses for attacker 63 | if not f2bcUtils.valid_ipv4(Attacker): 64 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 65 | Invalid attacker IP received.") 66 | continue 67 | # Jailnames must only contain chars a-z,A-Z,-_ 68 | # TODO: verify fail2ban jailname constraints 69 | if not f2bcUtils.valid_jailname(Jail): 70 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 71 | Invalid jail name received.") 72 | continue 73 | # TODO add debug level output for an invalid message 74 | syslog.syslog("fail2ban-zmq-tools Subscriber: \ 75 | Got broadcast message: %s" % message) 76 | syslog.syslog("fail2ban-zmq-tools Subscriber: thread exiting...") 77 | 78 | def join(self, timeout=None): 79 | self._stopevent.set() 80 | threading.Thread.join(self, timeout) 81 | 82 | 83 | if __name__ == "__main__": 84 | subscribing = Subscriber() 85 | -------------------------------------------------------------------------------- /daemon.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """Generic linux daemon base class for python 3.x.""" 3 | 4 | 5 | import os 6 | import sys 7 | import time 8 | import atexit 9 | import signal 10 | 11 | 12 | class daemon: 13 | """A generic daemon class. 14 | Usage: subclass the daemon class and override the run() method.""" 15 | 16 | def __init__(self, pidfile): 17 | self.pidfile = pidfile 18 | 19 | def daemonize(self): 20 | """Deamonize class. UNIX double fork mechanism.""" 21 | try: 22 | pid = os.fork() 23 | if pid > 0: 24 | # exit first parent 25 | sys.exit(0) 26 | 27 | except OSError as err: 28 | sys.stderr.write('fork #1 failed: {0}\n'.format(err)) 29 | sys.exit(1) 30 | 31 | # decouple from parent environment 32 | os.chdir('/') 33 | os.setsid() 34 | os.umask(0) 35 | 36 | # do second fork 37 | try: 38 | pid = os.fork() 39 | if pid > 0: 40 | # exit from second parent 41 | sys.exit(0) 42 | except OSError as err: 43 | sys.stderr.write('fork #2 failed: {0}\n'.format(err)) 44 | sys.exit(1) 45 | 46 | # redirect standard file descriptors 47 | sys.stdout.flush() 48 | sys.stderr.flush() 49 | si = open(os.devnull, 'r') 50 | so = open(os.devnull, 'a+') 51 | se = open(os.devnull, 'a+') 52 | 53 | # write pidfile 54 | atexit.register(self.delpid) 55 | pid = str(os.getpid()) 56 | with open(self.pidfile, 'w+') as f: 57 | f.write(pid + '\n') 58 | 59 | def delpid(self): 60 | os.remove(self.pidfile) 61 | 62 | def start(self): 63 | """Start the daemon.""" 64 | # Check for a pidfile to see if the daemon already runs 65 | try: 66 | with open(self.pidfile, 'r') as pf: 67 | pid = int(pf.read().strip()) 68 | except IOError: 69 | pid = None 70 | 71 | if pid: 72 | message = "pidfile {0} already exist. " + \ 73 | "Daemon already running?\n" 74 | sys.stderr.write(message.format(self.pidfile)) 75 | sys.exit(1) 76 | 77 | # Start the daemon 78 | self.daemonize() 79 | self.run() 80 | 81 | def stop(self): 82 | """Stop the daemon.""" 83 | # Get the pid from the pidfile 84 | try: 85 | with open(self.pidfile, 'r') as pf: 86 | pid = int(pf.read().strip()) 87 | except IOError: 88 | pid = None 89 | if not pid: 90 | message = "pidfile {0} does not exist. " + \ 91 | "Daemon not running?\n" 92 | sys.stderr.write(message.format(self.pidfile)) 93 | return # not an error in a restart 94 | 95 | # Try TERMinatting the daemon process 96 | # If that does not work, KILL IT 97 | # TODO: analyze why it doesn't work sometimes. 98 | # must be threading related... 99 | try: 100 | termcounter = 0 101 | while 1: 102 | os.kill(pid, signal.SIGTERM) 103 | time.sleep(0.1) 104 | termcounter += 1 105 | if termcounter == 100: 106 | print("Not response to SIGMTERM. Killing. (FIXME)") 107 | os.kill(pid, signal.SIGKILL) 108 | except OSError as err: 109 | e = str(err.args) 110 | if e.find("No such process") > 0: 111 | if os.path.exists(self.pidfile): 112 | os.remove(self.pidfile) 113 | else: 114 | print(str(err.args)) 115 | sys.exit(1) 116 | 117 | def restart(self): 118 | """Restart the daemon.""" 119 | self.stop() 120 | self.start() 121 | 122 | def run(self): 123 | """You should override this method when you subclass Daemon. 124 | It will be called after the process has been daemonized by 125 | start() or restart().""" 126 | -------------------------------------------------------------------------------- /publisher.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import time 3 | import sys 4 | import os 5 | import re 6 | import threading 7 | import queue 8 | import zmq 9 | import socket 10 | import errno 11 | import syslog 12 | from pprint import pprint 13 | from util import f2bcUtils 14 | 15 | syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_AUTH) 16 | 17 | 18 | class Publisher(threading.Thread): 19 | def __init__(self, publisherconfig=None, num_worker_threads=1): 20 | threading.Thread.__init__(self) 21 | self._stopevent = threading.Event() 22 | self.publisherconfig = publisherconfig 23 | self.zmqBroadcasterBindUrl = self.publisherconfig['broadcasterbindurl'] 24 | self.zmqReplyBindUrl = self.publisherconfig['replybindurl'] 25 | self.authenticate = self.publisherconfig['auth'] 26 | self.authtoken = self.publisherconfig['authtoken'] 27 | syslog.syslog("fail2ban-zmq-tools Publisher: initialization complete") 28 | 29 | def run(self): 30 | self.zmqPublisherContext = zmq.Context() 31 | self.zmqPublisherSock = self.zmqPublisherContext.socket(zmq.PUB) 32 | self.zmqPublisherSock.bind(self.zmqBroadcasterBindUrl) 33 | 34 | self.zmqReplyContext = zmq.Context() 35 | self.zmqReplySock = self.zmqReplyContext.socket(zmq.REP) 36 | self.zmqReplySock.bind(self.zmqReplyBindUrl) 37 | 38 | # http://api.zeromq.org/3-2:zmq-setsockopt 39 | self.zmqReplySock.setsockopt(zmq.MAXMSGSIZE, 64) 40 | # 1s timeout for recv() 41 | self.zmqReplySock.setsockopt(zmq.RCVTIMEO, 1000) 42 | # Wait for messages, when one is received, process it 43 | while not self._stopevent.isSet(): 44 | message = None 45 | try: 46 | message = self.zmqReplySock.recv_string() 47 | except zmq.error.ZMQError as e: 48 | if e == errno.EAGAIN: 49 | pass # Nothing to see, move along 50 | if not message: 51 | continue 52 | 53 | # Send it back to Requester (monitor instance), but first run 54 | # some tests. Failed tests trigger a NAK response, and then a 55 | # while().continue Check if splitted message has less than 4 or 56 | # more than 5 slices 57 | 58 | if len(message.split('|')) < 4 or len(message.split('|')) > 5: 59 | self.zmqReplySock.send_string("NAK") 60 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 61 | invalid message. Replying NAK.") 62 | continue 63 | # and if incoming token matches our defined token 64 | if self.authenticate == "true" and message.split('|')[0] != self.authtoken: 65 | self.zmqReplySock.send_string("NAK") 66 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 67 | invalid token. Replying NAK.") 68 | continue 69 | 70 | # remove authentication data from to-be-propagated message 71 | if self.authenticate == "true": 72 | newmsg = message.split('|') 73 | message = '|'.join(newmsg[1:]) 74 | 75 | # Now test hostname,jail,action and attacker 76 | newmsg = message.split('|') 77 | Hostname = newmsg[0] 78 | Jail = newmsg[1] 79 | Action = newmsg[2] 80 | Attacker = newmsg[3] 81 | 82 | if not f2bcUtils.is_valid_hostname(Hostname): 83 | self.zmqReplySock.send_string("NAK") 84 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 85 | invalid hostname in incoming message. \ 86 | Replying NAK.") 87 | continue 88 | if not f2bcUtils.is_valid_action(Action): 89 | self.zmqReplySock.send_string("NAK") 90 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 91 | Unknown action received in message. \ 92 | Replying NAK.") 93 | continue 94 | if not f2bcUtils.valid_ipv4(Attacker): 95 | self.zmqReplySock.send_string("NAK") 96 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 97 | Invalid attacker IP received in message.\ 98 | Replying NAK.") 99 | continue 100 | if not f2bcUtils.valid_jailname(Jail): 101 | self.zmqReplySock.send_string("NAK") 102 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 103 | Invalid jailname received in message. \ 104 | Replying NAK.") 105 | continue 106 | 107 | # If we got here, all tests were positive. we can make an OK 108 | # reply and then we can propagate the message, which now lacks 109 | # authentication information 110 | self.zmqReplySock.send_string(message) 111 | syslog.syslog("fail2ban-zmq-tools Publisher: \ 112 | Propagating %s for %s/%s from %s" % (Action, 113 | Attacker, 114 | Jail, 115 | Hostname)) 116 | self.zmqPublisherSock.send_string(message) 117 | # TODO: add loglevels 118 | syslog.syslog("fail2ban-zmq-tools Publisher: thread exiting...") 119 | sys.stdout.flush() 120 | 121 | def join(self, timeout=None): 122 | # Stop the thread 123 | self._stopevent.set() 124 | threading.Thread.join(self, timeout) 125 | 126 | 127 | if __name__ == "__main__": 128 | publishing = Publisher() 129 | -------------------------------------------------------------------------------- /monitor.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import time 3 | import sys 4 | import os 5 | import re 6 | import threading 7 | import queue 8 | import zmq 9 | import socket 10 | import errno 11 | import syslog 12 | from stat import ST_SIZE 13 | 14 | syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_AUTH) 15 | 16 | 17 | # Example BA 18 | # 2009-04-02 18:15:53,693 fail2ban.actions: WARNING [php-url-fopen] Ban 1.2.3.4 19 | class Monitor(threading.Thread): 20 | def zmqRequester(self, flag, jail, action, attacker): 21 | if jail == 'fail2bancluster': 22 | return True # ignore fail2bancluster bans 23 | syslog.syslog("Propagating: %s for %s in %s" % (action, 24 | attacker, 25 | jail)) 26 | zmqReplyContext = zmq.Context() 27 | zmqReplySock = zmqReplyContext.socket(zmq.REQ) 28 | zmqReplySock.connect(self.zmqReplyServer) 29 | # TODO: sanitize jail names, action (Ban/Unban), 30 | # TODO: and attacker (IP address regex) 31 | # TODO: also sanitize hostname according to RFC 1123 32 | outmsg = "" 33 | if self.authenticate == "true": 34 | outmsg = '{}|'.format(self.authtoken) 35 | outmsg += self.hostname + "|" + jail + "|" + action + "|" + attacker 36 | zmqReplySock.send_string(outmsg) 37 | try: 38 | inmsg = zmqReplySock.recv_string() 39 | except zmq.error.ZMQError as e: 40 | if e.errno == errno.EINTR: 41 | inmsg = outmsg 42 | pass 43 | else: 44 | syslog.syslog("Unhandled or unknown exception") 45 | raise 46 | except Exception: 47 | inmsg = outmsg # we are being shutdown, fake a good answer 48 | pass 49 | if outmsg == inmsg: 50 | return True 51 | return False 52 | 53 | def notifier(self): 54 | sys.stdout.flush() 55 | flag = 'ok' 56 | while flag != 'stop': 57 | try: 58 | flag, jail, action, attacker = self.dq.get() 59 | except Exception: 60 | pass # TODO: fix 61 | sys.stdout.flush() 62 | if flag == 'stop': 63 | # self.zmqRequester(flag,"BYEBYE","BYEBYE","BYEBYE") 64 | break 65 | self.zmqRequester(flag, jail, action, attacker) 66 | self.dq.task_done() 67 | syslog.syslog("Notifier exiting loop") 68 | sys.stdout.flush() 69 | 70 | def __init__(self, monitorconfig=None, num_worker_threads=1): 71 | threading.Thread.__init__(self) 72 | self._stopevent = threading.Event() 73 | self.monitorconfig = monitorconfig 74 | self.hostname = socket.gethostname() 75 | # I call it "ReplyServer" because it is a 76 | # zeromq REQUEST/REPLY type of socket 77 | self.zmqReplyServer = self.monitorconfig['zmqreplyserver'] 78 | self.logfilename = self.monitorconfig['fail2banlogpath'] 79 | self.authenticate = self.monitorconfig['auth'] 80 | self.authtoken = self.monitorconfig['authtoken'] 81 | # TODO: re-implement HELLOHELLO message to Publisher 82 | # self.zmqRequester('ok','HELLOHELLO','HELLOHELLO','HELLOHELLO') 83 | self.logfile = open(self.logfilename, 'r') 84 | # Prepare regex 85 | self.regex = re.compile(".*\[(.*)\]\ (Ban|Un[bB]an)\ (.*)") 86 | # Create queue for notifier 87 | self.dq = queue.Queue() 88 | self.ntPool = [] 89 | for i in range(num_worker_threads): 90 | # http://code.activestate.com/recipes/302746/ 91 | t = threading.Thread(target=self.notifier) 92 | t.setDaemon(True) 93 | t.start() 94 | self.ntPool.append(t) 95 | 96 | def run(self): 97 | # Find the size of the file and move to the end 98 | st_results = os.stat(self.logfilename) 99 | st_size = st_results[6] 100 | self.logfile.seek(st_size) 101 | 102 | while not self._stopevent.isSet(): 103 | where = self.logfile.tell() 104 | line = self.logfile.readline() 105 | if not line: 106 | # logfile truncated or rotated. we got to reset. 107 | if os.stat(self.logfilename)[ST_SIZE] < where: 108 | syslog.syslog("fail2ban-zmq-cluster Monitor: fail2ban \ 109 | logfile rotation detected.") 110 | self.logfile.close() 111 | self.logfile = open(self.logfilename, 'r') 112 | where = self.logfile.tell() 113 | else: 114 | time.sleep(1) 115 | self.logfile.seek(where) 116 | else: 117 | # match the line. attempt to extract jail\ 118 | # (postfix, apache-badbots, etc), action (Ban/UnBan) and IP 119 | logdata = self.regex.match(line) 120 | if logdata is not None: 121 | jail = logdata.group(1) 122 | action = logdata.group(2) 123 | attacker = logdata.group(3) 124 | self.dq.put(["ok", jail, action, attacker]) 125 | sys.stdout.flush() 126 | # self.zmqRequester('BYEBYE','BYEBYE','BYEBYE','BYEBYE') 127 | sys.stdout.flush() 128 | 129 | def join(self, timeout=None): 130 | """ Stop the thread 131 | """ 132 | sys.stdout.flush() 133 | for i in range(len(self.ntPool)): 134 | self.dq.put(["stop", 0, 0, 0]) 135 | sys.stdout.flush() 136 | sys.stdout.flush() 137 | while self.ntPool: 138 | time.sleep(1) 139 | sys.stdout.flush() 140 | for index, the_thread in enumerate(self.ntPool): 141 | if the_thread.isAlive(): 142 | continue 143 | else: 144 | del self.ntPool[index] 145 | break 146 | self._stopevent.set() 147 | threading.Thread.join(self, timeout) 148 | 149 | 150 | if __name__ == "__main__": 151 | from configparsing import ConfigParsing 152 | monitorconfig = ConfigParsing().Section(section='monitor') 153 | monitoreo = Monitor(monitorconfig=monitorconfig) 154 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | {description} 294 | Copyright (C) {year} {fullname} 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | {signature of Ty Coon}, 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | 341 | --------------------------------------------------------------------------------