├── .gitignore ├── LICENSE ├── README.md ├── iptables_xt_recent_parser ├── __init__.py └── ipt_recents └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | build 2 | dist 3 | *.egg-* 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) The iptables_xt_recent_parser project 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without modification, 5 | are permitted provided that the following conditions are met: 6 | 7 | 1. Redistributions of source code must retain the above copyright notice, 8 | this list of conditions and the following disclaimer. 9 | 10 | 2. Redistributions in binary form must reproduce the above copyright 11 | notice, this list of conditions and the following disclaimer in the 12 | documentation and/or other materials provided with the distribution. 13 | 14 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 15 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 16 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 17 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR 18 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 19 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 20 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 21 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 22 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 23 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 24 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # iptables_xt_recent_parser 2 | Used for converting jiffies from iptables xt_recent into timestamps. 3 | 4 | An example of xt_recent log can be like this, where only 2 syn connections in 20 seconds are allowed: 5 | 6 | ```` 7 | export IPT=iptables 8 | export SSH_PORT=22 9 | export HITCOUNT=3 # 2 syn connection (<3) 10 | export SECONDS=20 # in 20 seconds are allowed 11 | 12 | 13 | # --rcheck: Check if the source address of the packet is currently in the list. 14 | # --update: Like --rcheck, except it will update the "last seen" timestamp if it matches. 15 | 16 | $IPT -A INPUT -p tcp -m tcp --dport $SSH_PORT -m state --state NEW -m recent --set --name sshguys --rsource 17 | $IPT -A INPUT -p tcp -m tcp --dport $SSH_PORT -m state --state NEW -m recent --rcheck --seconds $SECONDS --hitcount $HITCOUNT --rttl --name sshguys --rsource -j LOG --log-prefix "BLOCKED SSH (brute force)" --log-level 4 -m limit --limit 1/minute --limit-burst 5 18 | $IPT -A INPUT -p tcp -m tcp --dport $SSH_PORT -m recent --rcheck --seconds $SECONDS --hitcount $HITCOUNT --rttl --name sshguys --rsource -j REJECT --reject-with tcp-reset 19 | $IPT -A INPUT -p tcp -m tcp --dport $SSH_PORT -m recent --update --seconds $SECONDS --hitcount $HITCOUNT --rttl --name sshguys --rsource -j REJECT --reject-with tcp-reset 20 | $IPT -A INPUT -p tcp -m tcp --dport $SSH_PORT -m state --state NEW,ESTABLISHED -j ACCEPT 21 | ```` 22 | 23 | In syslog we can see blocked connections : 24 | 25 | ```` 26 | Mar 26 14:06:41 cloudone-cla kernel: [5339977.637052] BLOCKED SSH (brute force)IN=eth0 OUT= MAC=00:50:56:92:00:04:00:14:c2:61:09:be:08:00 SRC=95.142.177.153 DST=160.97.104.18 LEN=60 TOS=0x00 PREC=0x00 TTL=50 ID=42489 DF PROTO=TCP SPT=44636 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0 27 | ```` 28 | 29 | ### Usage 30 | ```` 31 | XT_RECENT python parser 32 | 33 | 34 | usage: ipt_recents [-h] [-f F] [-txt] [-csv] 35 | 36 | optional arguments: 37 | -h, --help show this help message and exit 38 | -f F custom xt_recent path, default if omitted is: 39 | /proc/net/xt_recent/DEFAULT 40 | -txt print it in human readable format 41 | -csv print it in CSV format 42 | ```` 43 | 44 | ### Output 45 | ```` 46 | Standard readable view: 47 | 190.102.72.44, last seen: 2017-03-26 13:31:55 after 1 connections 48 | 187.112.185.153, last seen: 2017-03-26 13:28:07 after 2 connections 49 | 95.142.177.153, last seen: 2017-03-26 13:27:31 after 12 connections 50 | 51 | CSV view: 52 | ip_src;last_seen;connections;deltas_mean;delta_seconds 53 | 190.102.72.44;2017-03-26 13:31:55.462201;1;0; 54 | 187.112.185.153;2017-03-26 13:28:07.168819;2;0.0;0 55 | 95.142.177.153;2017-03-26 13:27:31.976049;12;1.7272727272727273;1,1,1,1,1,1,2,3,3,1,4 56 | 57 | ```` 58 | 59 | In CSV format there are time delta mean and time delta in seconds, for every attempts. 60 | 61 | ### Requirements 62 | 63 | - Python3 64 | -------------------------------------------------------------------------------- /iptables_xt_recent_parser/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peppelinux/iptables_xt_recent_parser/a7263dbe0cd6298c5a8cb677b8fa01684be05ab8/iptables_xt_recent_parser/__init__.py -------------------------------------------------------------------------------- /iptables_xt_recent_parser/ipt_recents: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | #~ Copyright 2017 Giuseppe De Marco 4 | #~ 5 | #~ Permission is hereby granted, free of charge, to any person obtaining a 6 | #~ copy of this software and associated documentation files (the "Software"), 7 | #~ to deal in the Software without restriction, including without limitation 8 | #~ the rights to use, copy, modify, merge, publish, distribute, sublicense, 9 | #~ and/or sell copies of the Software, and to permit persons to whom the Software 10 | #~ is furnished to do so, subject to the following conditions: 11 | #~ 12 | #~ The above copyright notice and this permission notice shall be included 13 | #~ in all copies or substantial portions of the Software. 14 | #~ 15 | #~ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS 16 | #~ OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | #~ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL 18 | #~ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | #~ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 20 | #~ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 21 | #~ DEALINGS IN THE SOFTWARE. 22 | 23 | import re 24 | import sys 25 | import datetime 26 | from copy import copy 27 | import os 28 | import subprocess 29 | 30 | _rpath = '/proc/net/xt_recent/' 31 | _fpath = _rpath + 'DEFAULT' 32 | _kernel_config_path = '/boot/config-' + subprocess.getoutput(['uname -r']) 33 | _datetime_format = '%Y-%m-%d %H:%M:%S' 34 | 35 | 36 | class JiffyTimeConverter(object): 37 | def __init__(self, kernel_config_path=_kernel_config_path): 38 | 39 | self.hz = JiffyTimeConverter.system_hz(kernel_config_path=_kernel_config_path) 40 | self.jiffies = JiffyTimeConverter.system_jiffies() 41 | 42 | def seconds_ago(self, jiffies_timestamp): 43 | return ((JiffyTimeConverter.system_jiffies() - int(jiffies_timestamp) ) / self.hz ) 44 | 45 | def minutes_ago(self, jiffies_timestamp): 46 | return self.seconds_ago / 60 47 | 48 | def datetime(self, jiffies_timestamp): 49 | now = datetime.datetime.now() 50 | td = datetime.timedelta(seconds=self.seconds_ago(jiffies_timestamp)) 51 | return now - td 52 | 53 | def convert_to_format(self, jiffy_timestamp, strftime=_datetime_format): 54 | return self.datetime(jiffy_timestamp).strftime(strftime) 55 | 56 | @staticmethod 57 | def check_system_jiffies(): 58 | """ 59 | It only prints 12 times how many jiffies runs in a second 60 | If kernel's CONFIG_HZ is 250 there will be 250 jiffies in a second 61 | It's funny to see that sometimes this value gets some oscillations (251,250,250,251...) 62 | """ 63 | last_jiffies = 0 64 | hz = 0 65 | cnt = 0 66 | while cnt < 12: 67 | new_jiffies = JiffyTimeConverter.system_jiffies() 68 | hz = new_jiffies - last_jiffies 69 | last_jiffies = new_jiffies 70 | time.sleep(1) 71 | print(hz) 72 | print(new_jiffies) 73 | print('') 74 | cnt += 1 75 | return hz 76 | 77 | @staticmethod 78 | def system_uptime(): 79 | """ 80 | returns system uptime in seconds 81 | """ 82 | from datetime import timedelta 83 | 84 | with open('/proc/uptime', 'r') as f: 85 | uptime_seconds = float(f.readline().split()[0]) 86 | uptime_string = str(timedelta(seconds = uptime_seconds)) 87 | 88 | return uptime_seconds 89 | 90 | @staticmethod 91 | def system_jiffies(): 92 | """ 93 | returns current system jiffies 94 | """ 95 | _jiffies_pattern = r'(?:jiffies[ =:]*?)([0-9]+)' 96 | 97 | with open('/proc/timer_list') as f: 98 | q = re.search(_jiffies_pattern, f.read()) 99 | if not q: 100 | sys.exit('Cannot determine jiffies in /proc/timer_list.\n\ 101 | Please check _jiffies_pattern\n\n') 102 | else: 103 | _jiffies = q.groups()[0] 104 | return float(_jiffies) 105 | 106 | @staticmethod 107 | def system_btime(): 108 | """ 109 | The "btime" line gives the time at which the system booted, in seconds since 110 | the Unix epoch. 111 | """ 112 | _pattern = r'(?:btime[ =:]*?)([0-9]+)' 113 | 114 | with open('/proc/stat') as f: 115 | q = re.search(_pattern, f.read()) 116 | if not q: 117 | sys.exit('Cannot determine btime in /proc/stat.\n\ 118 | Please check _jiffies_pattern\n\n') 119 | else: 120 | _btime = q.groups()[0] 121 | return float(_btime) 122 | 123 | @staticmethod 124 | def system_hz(kernel_config_path=_kernel_config_path): 125 | # HZ defined how many ticks the internal timer interrupt in 126 | # 1sec, which means the number of jiffies count in 1 sec. 127 | _HZ_pattern = r'(?:CONFIG_HZ[ =:]*?)([0-9]+)' 128 | 129 | with open(kernel_config_path) as f: 130 | q = re.search(_HZ_pattern, f.read()) 131 | if not q: 132 | sys.exit('Cannot determine kernel HZ freq\n\n') 133 | else: 134 | _hz = q.groups()[0] 135 | return float(_hz) 136 | 137 | 138 | class XtRecentRow(object): 139 | def __init__(self, row, debug=False): 140 | """ 141 | where row is: 142 | src=151.54.175.212 ttl: 49 last_seen: 5610057758 143 | oldest_pkt: 11 5610048214, 5610048235, 5610048281, [...] 144 | """ 145 | # regexp 146 | _src_pattern = r'(?:src\=)(?P[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})' 147 | _ttl_pattern = r'(?:ttl\:\ )(?P[0-9]+)' 148 | _last_seen_pattern = r'(?:last_seen\:\ )(?P[0-9]+)' 149 | _hitcount_pattern = r'(?:hitcount\:\ )(?P[0-9]+)' 150 | _oldest_pkt_pattern = r'(?:oldest_pkt\:\ )(?P[0-9]+)' 151 | _timestamps_pattern = r'(?:oldest_pkt\:\ [0-9]*)(?P[0-9 ,]+)' 152 | # 153 | d = {} 154 | d.update(re.search( _src_pattern, row ).groupdict()) 155 | #~ self.hitcount = d.update(re.search( _hitcount_pattern, row ).groupdict()) 156 | d.update(re.search( _ttl_pattern, row ).groupdict()) 157 | d.update(re.search( _last_seen_pattern, row ).groupdict()) 158 | d.update(re.search( _oldest_pkt_pattern, row ).groupdict()) 159 | 160 | for i in d: 161 | setattr(self, i, d[i]) 162 | self.raw_history = re.search( _timestamps_pattern, row ).groups()[0] 163 | self.history = [i.strip() for i in self.raw_history.split(',')] 164 | 165 | self.debug = debug 166 | if debug: 167 | print(d) 168 | print(self.history) 169 | print('') 170 | 171 | def convert_jiffies(self): 172 | """ 173 | converts jiffies value in datetime object 174 | then returns a copy of self with all the jiffies converted 175 | """ 176 | d = copy(self) 177 | jt = JiffyTimeConverter() 178 | d.last_seen = jt.datetime(d.last_seen) 179 | d.oldest_pkt = jt.datetime(d.oldest_pkt) 180 | 181 | d.history = [jt.datetime(i) for i in self.history] 182 | return d 183 | 184 | def format_jiffies(self, strftime_format=_datetime_format): 185 | """ 186 | displays datetime values in a preferred datetime string format 187 | returns a copy of the object 188 | """ 189 | d = self.convert_jiffies() 190 | jt = JiffyTimeConverter() 191 | 192 | d.last_seen = jt.convert_to_format(d.last_seen, strftime_format) 193 | d.oldest_pkt = jt.convert_to_format(d.oldest_pkt, strftime_format) 194 | 195 | d.history = [jt.convert_to_format(i, strftime_format) 196 | for i in self.history] 197 | return d 198 | 199 | def __repr__(self): 200 | _msg = '{}, last seen: {} after {} connections' 201 | return _msg.format(self.src, 202 | self.last_seen.strftime(_datetime_format), 203 | len(self.history)) 204 | 205 | 206 | class XtRecentTable(object): 207 | def __init__(self, fpath=None, debug=None): 208 | if fpath: 209 | self.fpath = fpath 210 | else: 211 | self.fpath = _fpath 212 | 213 | self.xt_recent = [] 214 | self.rows = [] 215 | self.debug = debug 216 | 217 | def parse(self): 218 | """ 219 | do parse of xt_recent file 220 | for every row it create a XtRecentRow object 221 | """ 222 | # flush it first 223 | self.rows = [] 224 | self.xt_recent = [] 225 | with open(self.fpath) as f: 226 | self.rows = f.readlines() 227 | for i in self.rows: 228 | if i.strip(): 229 | if self.debug: 230 | print('Parsing: {}'.format(i.replace('\n', ''), 231 | file=sys.stderr)) 232 | row = XtRecentRow(i, debug=self.debug) 233 | row_dt = row.convert_jiffies() 234 | # raw datetime in jiffies format! 235 | # self.xt_recent.append( row ) 236 | # datetime format 237 | self.xt_recent.append(row_dt) 238 | if self.debug: 239 | print(row_dt) 240 | for e in row_dt.history: 241 | print(r) 242 | 243 | def csv(self): 244 | self.parse() 245 | print(';'.join(('ip_src','last_seen','connections', 246 | 'deltas_mean', 'delta_seconds'))) 247 | for row in self.xt_recent: 248 | deltas = [] 249 | dt_cnt = 1 250 | if len(row.history) > 1: 251 | for hi in row.history: 252 | try: 253 | #~ print(row.history[dt_cnt], hi, ) 254 | dt = row.history[dt_cnt] - hi 255 | #~ print(dt) 256 | deltas.append(dt) 257 | except Exception as e: 258 | pass 259 | dt_cnt += 1 260 | 261 | if len(deltas): 262 | d_mean = sum([ d.seconds for d in deltas]) / len(deltas) 263 | else: 264 | d_mean = 0 265 | 266 | prow = (row.src, 267 | str(row.last_seen), 268 | str(len(row.history)), 269 | str(d_mean), 270 | ','.join([ str(d.seconds) for d in deltas])) 271 | print( ';'.join(prow)) 272 | 273 | def view(self): 274 | """ 275 | prints in stdout the XtRecentRow object's representation 276 | for all the rows in xt_recent 277 | """ 278 | self.parse() 279 | for row in self.xt_recent: 280 | print(row) 281 | 282 | 283 | if __name__ == '__main__': 284 | import argparse 285 | print('XT_RECENT python parser\n\n', 286 | file=sys.stderr) 287 | parser = argparse.ArgumentParser() 288 | 289 | # An int is an explicit number of arguments to accept. 290 | parser.add_argument('-f', required=False, 291 | default=_fpath, 292 | help=("custom xt_recent path, default if " 293 | "omitted is: /proc/net/xt_recent/DEFAULT")) 294 | parser.add_argument('-txt', action="store_true", default=True, 295 | help="print it in human readable format") 296 | parser.add_argument('-csv', action="store_true", 297 | help="print it in CSV format") 298 | args = parser.parse_args() 299 | 300 | if len(sys.argv)==1: 301 | parser.print_help() 302 | sys.exit(1) 303 | 304 | if args.f: 305 | _fpath = args.f 306 | if '/' not in _fpath: 307 | _fpath = _rpath + _fpath 308 | 309 | print('Parsing file: {}'.format(_fpath)) 310 | xt = XtRecentTable(fpath=_fpath) 311 | 312 | if args.csv: 313 | xt.csv() 314 | elif args.txt: 315 | xt.view() 316 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | def readme(): 4 | with open('README.md') as f: 5 | return f.read() 6 | 7 | setup(name='iptables_xt_recent_parser', 8 | version='0.6.0-1', 9 | description=("Tool used for converting jiffies from iptables " 10 | "xt_recent into timestamps."), 11 | long_description=readme(), 12 | long_description_content_type='text/markdown', 13 | classifiers=['Development Status :: 5 - Production/Stable', 14 | 'License :: OSI Approved :: BSD License', 15 | 'Programming Language :: Python :: 3'], 16 | url='https://github.com/peppelinux/iptables_xt_recent_parser', 17 | author='Giuseppe De Marco', 18 | author_email='giuseppe.demarco@unical.it', 19 | license='BSD', 20 | scripts=['iptables_xt_recent_parser/ipt_recents'], 21 | #packages=['iptables_xt_recent_parser'], 22 | install_requires=[], 23 | ) 24 | --------------------------------------------------------------------------------