├── LICENSE
├── README.md
└── bind-query-log-stats.py
/LICENSE:
--------------------------------------------------------------------------------
1 | The GNU General Public License, Version 2, June 1991 (GPLv2)
2 | ============================================================
3 |
4 | > Copyright (C) 1989, 1991 Free Software Foundation, Inc.
5 | > 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
6 |
7 | Everyone is permitted to copy and distribute verbatim copies of this license
8 | document, but changing it is not allowed.
9 |
10 |
11 | Preamble
12 | --------
13 |
14 | The licenses for most software are designed to take away your freedom to share
15 | and change it. By contrast, the GNU General Public License is intended to
16 | guarantee your freedom to share and change free software--to make sure the
17 | software is free for all its users. This General Public License applies to most
18 | of the Free Software Foundation's software and to any other program whose
19 | authors commit to using it. (Some other Free Software Foundation software is
20 | covered by the GNU Lesser General Public License instead.) You can apply it to
21 | your programs, too.
22 |
23 | When we speak of free software, we are referring to freedom, not price. Our
24 | General Public Licenses are designed to make sure that you have the freedom to
25 | distribute copies of free software (and charge for this service if you wish),
26 | that you receive source code or can get it if you want it, that you can change
27 | the software or use pieces of it in new free programs; and that you know you can
28 | do these things.
29 |
30 | To protect your rights, we need to make restrictions that forbid anyone to deny
31 | you these rights or to ask you to surrender the rights. These restrictions
32 | translate to certain responsibilities for you if you distribute copies of the
33 | software, or if you modify it.
34 |
35 | For example, if you distribute copies of such a program, whether gratis or for a
36 | fee, you must give the recipients all the rights that you have. You must make
37 | sure that they, too, receive or can get the source code. And you must show them
38 | these terms so they know their rights.
39 |
40 | We protect your rights with two steps: (1) copyright the software, and (2) offer
41 | you this license which gives you legal permission to copy, distribute and/or
42 | modify the software.
43 |
44 | Also, for each author's protection and ours, we want to make certain that
45 | everyone understands that there is no warranty for this free software. If the
46 | software is modified by someone else and passed on, we want its recipients to
47 | know that what they have is not the original, so that any problems introduced by
48 | others will not reflect on the original authors' reputations.
49 |
50 | Finally, any free program is threatened constantly by software patents. We wish
51 | to avoid the danger that redistributors of a free program will individually
52 | obtain patent licenses, in effect making the program proprietary. To prevent
53 | this, we have made it clear that any patent must be licensed for everyone's free
54 | use or not licensed at all.
55 |
56 | The precise terms and conditions for copying, distribution and modification
57 | follow.
58 |
59 |
60 | Terms And Conditions For Copying, Distribution And Modification
61 | ---------------------------------------------------------------
62 |
63 | **0.** This License applies to any program or other work which contains a notice
64 | placed by the copyright holder saying it may be distributed under the terms of
65 | this General Public License. The "Program", below, refers to any such program or
66 | work, and a "work based on the Program" means either the Program or any
67 | derivative work under copyright law: that is to say, a work containing the
68 | Program or a portion of it, either verbatim or with modifications and/or
69 | translated into another language. (Hereinafter, translation is included without
70 | limitation in the term "modification".) Each licensee is addressed as "you".
71 |
72 | Activities other than copying, distribution and modification are not covered by
73 | this License; they are outside its scope. The act of running the Program is not
74 | restricted, and the output from the Program is covered only if its contents
75 | constitute a work based on the Program (independent of having been made by
76 | running the Program). Whether that is true depends on what the Program does.
77 |
78 | **1.** You may copy and distribute verbatim copies of the Program's source code
79 | as you receive it, in any medium, provided that you conspicuously and
80 | appropriately publish on each copy an appropriate copyright notice and
81 | disclaimer of warranty; keep intact all the notices that refer to this License
82 | and to the absence of any warranty; and give any other recipients of the Program
83 | a copy of this License along with the Program.
84 |
85 | You may charge a fee for the physical act of transferring a copy, and you may at
86 | your option offer warranty protection in exchange for a fee.
87 |
88 | **2.** You may modify your copy or copies of the Program or any portion of it,
89 | thus forming a work based on the Program, and copy and distribute such
90 | modifications or work under the terms of Section 1 above, provided that you also
91 | meet all of these conditions:
92 |
93 | * **a)** You must cause the modified files to carry prominent notices stating
94 | that you changed the files and the date of any change.
95 |
96 | * **b)** You must cause any work that you distribute or publish, that in whole
97 | or in part contains or is derived from the Program or any part thereof, to
98 | be licensed as a whole at no charge to all third parties under the terms of
99 | this License.
100 |
101 | * **c)** If the modified program normally reads commands interactively when
102 | run, you must cause it, when started running for such interactive use in the
103 | most ordinary way, to print or display an announcement including an
104 | appropriate copyright notice and a notice that there is no warranty (or
105 | else, saying that you provide a warranty) and that users may redistribute
106 | the program under these conditions, and telling the user how to view a copy
107 | of this License. (Exception: if the Program itself is interactive but does
108 | not normally print such an announcement, your work based on the Program is
109 | not required to print an announcement.)
110 |
111 | These requirements apply to the modified work as a whole. If identifiable
112 | sections of that work are not derived from the Program, and can be reasonably
113 | considered independent and separate works in themselves, then this License, and
114 | its terms, do not apply to those sections when you distribute them as separate
115 | works. But when you distribute the same sections as part of a whole which is a
116 | work based on the Program, the distribution of the whole must be on the terms of
117 | this License, whose permissions for other licensees extend to the entire whole,
118 | and thus to each and every part regardless of who wrote it.
119 |
120 | Thus, it is not the intent of this section to claim rights or contest your
121 | rights to work written entirely by you; rather, the intent is to exercise the
122 | right to control the distribution of derivative or collective works based on the
123 | Program.
124 |
125 | In addition, mere aggregation of another work not based on the Program with the
126 | Program (or with a work based on the Program) on a volume of a storage or
127 | distribution medium does not bring the other work under the scope of this
128 | License.
129 |
130 | **3.** You may copy and distribute the Program (or a work based on it, under
131 | Section 2) in object code or executable form under the terms of Sections 1 and 2
132 | above provided that you also do one of the following:
133 |
134 | * **a)** Accompany it with the complete corresponding machine-readable source
135 | code, which must be distributed under the terms of Sections 1 and 2 above on
136 | a medium customarily used for software interchange; or,
137 |
138 | * **b)** Accompany it with a written offer, valid for at least three years, to
139 | give any third party, for a charge no more than your cost of physically
140 | performing source distribution, a complete machine-readable copy of the
141 | corresponding source code, to be distributed under the terms of Sections 1
142 | and 2 above on a medium customarily used for software interchange; or,
143 |
144 | * **c)** Accompany it with the information you received as to the offer to
145 | distribute corresponding source code. (This alternative is allowed only for
146 | noncommercial distribution and only if you received the program in object
147 | code or executable form with such an offer, in accord with Subsection b
148 | above.)
149 |
150 | The source code for a work means the preferred form of the work for making
151 | modifications to it. For an executable work, complete source code means all the
152 | source code for all modules it contains, plus any associated interface
153 | definition files, plus the scripts used to control compilation and installation
154 | of the executable. However, as a special exception, the source code distributed
155 | need not include anything that is normally distributed (in either source or
156 | binary form) with the major components (compiler, kernel, and so on) of the
157 | operating system on which the executable runs, unless that component itself
158 | accompanies the executable.
159 |
160 | If distribution of executable or object code is made by offering access to copy
161 | from a designated place, then offering equivalent access to copy the source code
162 | from the same place counts as distribution of the source code, even though third
163 | parties are not compelled to copy the source along with the object code.
164 |
165 | **4.** You may not copy, modify, sublicense, or distribute the Program except as
166 | expressly provided under this License. Any attempt otherwise to copy, modify,
167 | sublicense or distribute the Program is void, and will automatically terminate
168 | your rights under this License. However, parties who have received copies, or
169 | rights, from you under this License will not have their licenses terminated so
170 | long as such parties remain in full compliance.
171 |
172 | **5.** You are not required to accept this License, since you have not signed
173 | it. However, nothing else grants you permission to modify or distribute the
174 | Program or its derivative works. These actions are prohibited by law if you do
175 | not accept this License. Therefore, by modifying or distributing the Program (or
176 | any work based on the Program), you indicate your acceptance of this License to
177 | do so, and all its terms and conditions for copying, distributing or modifying
178 | the Program or works based on it.
179 |
180 | **6.** Each time you redistribute the Program (or any work based on the
181 | Program), the recipient automatically receives a license from the original
182 | licensor to copy, distribute or modify the Program subject to these terms and
183 | conditions. You may not impose any further restrictions on the recipients'
184 | exercise of the rights granted herein. You are not responsible for enforcing
185 | compliance by third parties to this License.
186 |
187 | **7.** If, as a consequence of a court judgment or allegation of patent
188 | infringement or for any other reason (not limited to patent issues), conditions
189 | are imposed on you (whether by court order, agreement or otherwise) that
190 | contradict the conditions of this License, they do not excuse you from the
191 | conditions of this License. If you cannot distribute so as to satisfy
192 | simultaneously your obligations under this License and any other pertinent
193 | obligations, then as a consequence you may not distribute the Program at all.
194 | For example, if a patent license would not permit royalty-free redistribution of
195 | the Program by all those who receive copies directly or indirectly through you,
196 | then the only way you could satisfy both it and this License would be to refrain
197 | entirely from distribution of the Program.
198 |
199 | If any portion of this section is held invalid or unenforceable under any
200 | particular circumstance, the balance of the section is intended to apply and the
201 | section as a whole is intended to apply in other circumstances.
202 |
203 | It is not the purpose of this section to induce you to infringe any patents or
204 | other property right claims or to contest validity of any such claims; this
205 | section has the sole purpose of protecting the integrity of the free software
206 | distribution system, which is implemented by public license practices. Many
207 | people have made generous contributions to the wide range of software
208 | distributed through that system in reliance on consistent application of that
209 | system; it is up to the author/donor to decide if he or she is willing to
210 | distribute software through any other system and a licensee cannot impose that
211 | choice.
212 |
213 | This section is intended to make thoroughly clear what is believed to be a
214 | consequence of the rest of this License.
215 |
216 | **8.** If the distribution and/or use of the Program is restricted in certain
217 | countries either by patents or by copyrighted interfaces, the original copyright
218 | holder who places the Program under this License may add an explicit
219 | geographical distribution limitation excluding those countries, so that
220 | distribution is permitted only in or among countries not thus excluded. In such
221 | case, this License incorporates the limitation as if written in the body of this
222 | License.
223 |
224 | **9.** The Free Software Foundation may publish revised and/or new versions of
225 | the General Public License from time to time. Such new versions will be similar
226 | in spirit to the present version, but may differ in detail to address new
227 | problems or concerns.
228 |
229 | Each version is given a distinguishing version number. If the Program specifies
230 | a version number of this License which applies to it and "any later version",
231 | you have the option of following the terms and conditions either of that version
232 | or of any later version published by the Free Software Foundation. If the
233 | Program does not specify a version number of this License, you may choose any
234 | version ever published by the Free Software Foundation.
235 |
236 | **10.** If you wish to incorporate parts of the Program into other free programs
237 | whose distribution conditions are different, write to the author to ask for
238 | permission. For software which is copyrighted by the Free Software Foundation,
239 | write to the Free Software Foundation; we sometimes make exceptions for this.
240 | Our decision will be guided by the two goals of preserving the free status of
241 | all derivatives of our free software and of promoting the sharing and reuse of
242 | software generally.
243 |
244 |
245 | No Warranty
246 | -----------
247 |
248 | **11.** BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR
249 | THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE
250 | STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM
251 | "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
252 | BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
253 | PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
254 | PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
255 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
256 |
257 | **12.** IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
258 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE
259 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
260 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR
261 | INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA
262 | BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
263 | FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER
264 | OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
265 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Bind query log statistics generator
2 |
3 | Produces a variety of statistics from one or more query logs passed on the command line.
4 |
5 | # Usage:
6 |
7 | $ ./dnsrecon.py -h
8 | usage: dnsrecon.py [-h] [--matrix] [--histogram] [--count Count]
9 | [--excludeip [IP Address [IP Address ...]]]
10 | [--excludenet [Network [Network ...]]]
11 | [--domains [Domain [Domain ...]]] [--starttime Start Time]
12 | [--endtime End Time]
13 | [Logfile [Logfile ...]]
14 |
15 | DNS Statistics Processor
16 |
17 | positional arguments:
18 | Logfile List of Bind query logs to process
19 |
20 | optional arguments:
21 | -h, --help show this help message and exit
22 | --matrix Print client to domain resolution info
23 | --histogram Print histogram of queries
24 | --count Count Number of entries to display
25 | --excludeip [IP Address [IP Address ...]]
26 | IPs to exclude from resolution matrix
27 | --excludenet [Network [Network ...]]
28 | Networks to exclude from resolution matrix
29 | --domains [Domain [Domain ...]]
30 | Create statistics for specific domains
31 | --starttime Start Time
32 | Create statistics from this time forward
33 | --endtime End Time Process statistics until this period of time
34 |
35 |
36 | # Sample Output:
37 |
38 | $ dnsrecon.py /var/log/query.log* --domains prefetch.net foo.com --exclude 1.1.1.1 2.2.2.2 --histogram --matrix
39 |
40 | Processing logfile /var/log/query.log.1
41 | Processing logfile /var/log/query.log.2
42 | Processing logfile /var/log/query.log.3
43 | Processing logfile /var/log/query.log.4
44 |
45 | Summary for 21-Sep-2016 00:00:00.001 - 21-Sep-2016 23:59:59.991
46 | Total queries processed : 1328499
47 | A records requested : 683201
48 | AAAA records requested : 570441
49 | SRV records requested : 66922
50 | SOA records requested : 3094
51 | MX records requested : 3069
52 | ANY records requested : 1060
53 | IXFR records requested : 434
54 | TXT records requested : 257
55 | PTR records requested : 15
56 | NS records requested : 6
57 |
58 | Top 100 DNS names requested:
59 | prefetch.net : 81379
60 | sheldon.prefetch.net : 75244
61 | penny.prefetch.net : 54637
62 | .....
63 |
64 | Top 100 DNS clients:
65 | leonard : 103680
66 | raj : 92486
67 | howard : 32456
68 | bernadette : 12324
69 | .....
70 |
71 | Queries per minute:
72 | 00: ******************* (149807)
73 | 01: ******************* (149894)
74 | 02: ******************************* (239495)
75 | 03: *********************************************** (356239)
76 | 04: ********************************************** (351916)
77 | 05: ********************************************* (346121)
78 | 06: ************************************************ (362635)
79 | 07: ************************************************** (377293)
80 | 08: ********************************************* (343376)
81 | 09: ********************** (169213)
82 | 10: ****************************** (229027)
83 | 11: ****************** (140477)
84 | 12: ****************** (139954)
85 | 13: ****************** (135871)
86 | 14: ****************** (137822)
87 | 15: ****************** (138635)
88 | 16: ****************** (137310)
89 | 17: ****************** (137466)
90 | 18: ***************** (134564)
91 | 19: ****************** (138332)
92 | 20: ***************** (135392)
93 | 21: ****************** (137407)
94 | 22: ****************** (137424)
95 | 23: ****************** (140056)
96 | 24: ****************** (139497)
97 | 25: ****************** (137237)
98 | 26: ****************** (137737)
99 | 27: ****************** (139208)
100 | 28: ***************** (135199)
101 | 29: ****************** (135959)
102 | 30: ****************** (140801)
103 | 31: ****************** (142486)
104 | 32: ****************** (143127)
105 | 33: ****************** (141277)
106 | 34: ****************** (142003)
107 | 35: ****************** (143132)
108 | 36: ******************* (144369)
109 | 37: ****************** (140977)
110 | 38: ****************** (139053)
111 | 39: ****************** (139480)
112 | 40: ****************** (138179)
113 | 41: ****************** (138187)
114 | 42: ****************** (139191)
115 | 43: ****************** (138635)
116 | 44: ****************** (141588)
117 | 45: ****************** (139573)
118 | 46: ****************** (140661)
119 | 47: ******************* (149312)
120 | 48: ****************** (141697)
121 | 49: ******************* (149304)
122 | 50: ****************** (142139)
123 | 51: ****************** (142245)
124 | 52: ****************** (139075)
125 | 53: ******************* (148803)
126 | 54: ******************* (144888)
127 | 55: ****************** (136482)
128 | 56: ****************** (139917)
129 | 57: ****************** (141297)
130 | 58: ****************** (137870)
131 | 59: ****************** (141526)
132 |
133 | Queries per hour:
134 | 00: ********* (325710)
135 | 01: ********** (363579)
136 | 02: ******** (304630)
137 | 03: ******** (302274)
138 | 04: ******** (296872)
139 | 05: ******** (295430)
140 | 06: ******** (309823)
141 | 07: ********* (347762)
142 | 08: ********* (350258)
143 | 09: ********** (371690)
144 | 10: *********** (397320)
145 | 11: ************ (444637)
146 | 12: ************ (448091)
147 | 13: *********** (419678)
148 | 14: ************************************************** (1765424)
149 | 15: ************ (449459)
150 | 16: ********* (338963)
151 | 17: ********* (338588)
152 | 18: ********* (340240)
153 | 19: ********* (342471)
154 | 20: ********* (332453)
155 | 21: ********* (340441)
156 | 22: ********** (377252)
157 | 23: ********* (334792)
158 |
159 | Domain to client resolution matrix:
160 |
161 | prefetch.net
162 | |-- leonard 87656
163 | |-- howard 23456
164 | |-- bernadette 3425
165 | .....
166 |
167 |
--------------------------------------------------------------------------------
/bind-query-log-stats.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | # Program: bind-query-log-statistics.py
3 | # Author: Matty < matty91 at gmail dot com >
4 | # Current Version: 1.1
5 | # Last Updated: 01-18-2018
6 | # Version history:
7 | # 1.1 First attempt to normalize query log formats
8 | # 1.0 Initial Release
9 | # Purpose: Analyzes Bind query logs and produces a variety of query statistics.
10 | # License:
11 | # This program is free software; you can redistribute it and/or modify
12 | # it under the terms of the GNU General Public License as published by
13 | # the Free Software Foundation; either version 2 of the License, or
14 | # (at your option) any later version.
15 | #
16 | # This program is distributed in the hope that it will be useful,
17 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
18 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19 | # GNU General Public License for more details.
20 |
21 |
22 | import sys
23 | import argparse
24 | import re
25 | import time
26 | import socket
27 | from collections import Counter
28 | from collections import defaultdict
29 |
30 | # Counters
31 | COUNT = 0
32 | TOTAL_QUERIES = 0
33 |
34 | # Timestsmp helpers
35 | FIRST_QUERY = time.strftime("%d-%b-%Y %H:%M:%S.000")
36 | LAST_QUERY = ""
37 |
38 | # Dictionaries to store query data
39 | DOMAINS = []
40 | LOGFILES = []
41 | NETS_TO_EXCLUDE = []
42 | IPS_TO_EXCLUDE = []
43 | DNS_QUERIES = defaultdict(int)
44 | DNS_CLIENTS = defaultdict(int)
45 | DNS_QUERY_TYPES = defaultdict(int)
46 | DNS_RESOLUTION_MATRIX = defaultdict(lambda: defaultdict(int))
47 |
48 | # Don't print a resolution matrix by default
49 | ENABLE_RESOLUTION_MATRIX = ""
50 | ENABLE_HISTOGRAMS = ""
51 |
52 | # Place to store query time period breakdown
53 | HISTOGRAM_HOUR = defaultdict(int)
54 | HISTOGRAM_MINUTE = defaultdict(int)
55 |
56 |
57 | def process_time(timestamp):
58 | """
59 | Log the first and last query times so we can
60 | display them on the final report
61 | """
62 | global FIRST_QUERY, LAST_QUERY
63 |
64 | if timestamp < FIRST_QUERY:
65 | FIRST_QUERY = timestamp
66 | elif timestamp > LAST_QUERY:
67 | LAST_QUERY = timestamp
68 |
69 |
70 | def generate_statistics(dns_question, rr_type, client_ip):
71 | """ Increment statistics for the dns_question """
72 | global TOTAL_QUERIES
73 |
74 | # If/Else added to avoid using the regex module unless absolutely necessary
75 | if DOMAINS != ".":
76 | if any(re.search(d, dns_question) for d in DOMAINS):
77 | TOTAL_QUERIES += 1
78 | DNS_QUERIES[dns_question] += 1
79 | DNS_QUERY_TYPES[rr_type] += 1
80 | DNS_CLIENTS[client_ip] += 1
81 | client_net = '.'.join(client_ip.split('.', 3)[:3])
82 | else:
83 | TOTAL_QUERIES += 1
84 | DNS_QUERIES[dns_question] += 1
85 | DNS_QUERY_TYPES[rr_type] += 1
86 | DNS_CLIENTS[client_ip] += 1
87 |
88 | if ENABLE_RESOLUTION_MATRIX:
89 | if (not (client_ip in IPS_TO_EXCLUDE or
90 | any(re.search(client_net, net) for net in NETS_TO_EXCLUDE))):
91 | DNS_RESOLUTION_MATRIX[dns_question][client_ip] += 1
92 |
93 |
94 | def print_top_dns_requests(num_print):
95 | """ Print the top DNS questions that were asked """
96 | print "\nTop ", num_print, " DNS names requested:"
97 | for query, _ in Counter(DNS_QUERIES).most_common(num_print):
98 | print " " + query + " : " + str(DNS_QUERIES[query])
99 |
100 |
101 | def print_top_dns_clients(num_print):
102 | """ Print the top DNS_CLIENTS who asked the most questions """
103 | print "\nTop ", num_print, " DNS clients:"
104 | for client_ip, num_queries in Counter(DNS_CLIENTS).most_common(num_print):
105 | try:
106 | client_name = socket.gethostbyaddr(client_ip)[0]
107 | except socket.herror:
108 | client_name = client_ip
109 | print " " + client_name + " : ", num_queries
110 |
111 |
112 | def print_dns_resolution_matrix():
113 | """ Print a list of DNS requests and the
114 | DNS_CLIENTS who asked those questions
115 | """
116 | print "\nDomain to client resolution matrix:"
117 | for domain in DNS_RESOLUTION_MATRIX:
118 | print "\n " + domain
119 | for client_ip in DNS_RESOLUTION_MATRIX[domain]:
120 | try:
121 | client_name = socket.gethostbyaddr(client_ip)[0]
122 | except socket.herror:
123 | client_name = client_ip
124 | print " |-- " + client_name + " ", DNS_RESOLUTION_MATRIX[domain][client_ip]
125 |
126 | def print_dns_summary():
127 | """ Print a number of summary statistics """
128 | print "\nSummary for %s - %s\n" % (FIRST_QUERY, LAST_QUERY)
129 | print "%-25s : %d" % ("Total DNS_QUERIES processed", TOTAL_QUERIES)
130 |
131 | for rr_type, query_count in sorted(DNS_QUERY_TYPES.items(), key=lambda a: a[1], reverse=True):
132 | print " %-6s records requested : %d" % (rr_type, query_count)
133 |
134 |
135 | def process_query(query):
136 | """ Takes a Bind query log entry and splits it to see how many
137 | entries it contains. The query log format can change between
138 | releases and can grow or shrink if views are used.
139 |
140 | Known query log formats:
141 |
142 | Bind 9.3 query log format:
143 | 20-Sep-2016 11:26:15.510 query: info: client 1.2.3.4#60010: \
144 | view standard: query: blip.prefetch.net IN AAAA +
145 |
146 | Bind 9.9 query log format:
147 | 20-Sep-2016 11:24:30.025 query: info: client 1.2.3.4#61687 \
148 | (blip.prefetch.net): view standard: query: blip.prefetch.net
149 | IN A + (10.1.1.1)
150 |
151 | # Bind 9.3 w/o views
152 | 08-Nov-2016 14:05:59.996 query: info: client 1.2.3.4#7619: \
153 | query: 10.10.10.10.in-addr.arpa IN PTR -E
154 |
155 | # Bind 9.10
156 | 18-Jan-2018 13:13:07.889 client 1.2.3.4#42872 (prefetch.net): \
157 | query: prefetch.net IN ANY + (1.2.3.4)
158 | """
159 | words_to_strip = [ "query:", "info:", "client", "view", "standard:", "queries:" ]
160 | chopped = ' '.join(i for i in query.split() if i not in words_to_strip).split()
161 |
162 | if len(chopped) == 7:
163 | timestamp = chopped[0] + " " + chopped[1]
164 | client_ip = chopped[2].split("#")[0]
165 | rr_type = chopped[5]
166 | dns_question = chopped[3]
167 |
168 | elif len(chopped) == 9:
169 | timestamp = chopped[0] + " " + chopped[1]
170 | client_ip = chopped[2].split("#")[0]
171 | rr_type = chopped[6]
172 | dns_question = chopped[4]
173 |
174 | else:
175 | print "Unknown query log format"
176 | print "Offending line -> %s" % query
177 | sys.exit(1)
178 |
179 | return timestamp, dns_question, rr_type, client_ip
180 |
181 |
182 | def processcli():
183 | """ parses the CLI arguments and returns a list
184 | of domains and logfiles to process
185 | """
186 |
187 | parser = argparse.ArgumentParser(description='DNS Statistics Processor')
188 |
189 | parser.add_argument('--matrix', help="Print client to domain resolution info",
190 | action="store_true")
191 | parser.add_argument('--histogram', help="Print histogram of queries",
192 | action="store_true")
193 | parser.add_argument('--count', nargs=1, help="Number of entries to display",
194 | default=100, metavar="Count")
195 | parser.add_argument('--excludeip', nargs='*', help="IPs to exclude from resolution matrix",
196 | default="", metavar="IP Address")
197 | parser.add_argument('--excludenet', nargs='*', help="Networks to exclude from resolution matrix",
198 | default="", metavar="Network")
199 | parser.add_argument('--domains', nargs='*', help="Create statistics for specific domains",
200 | default=".", metavar="Domain")
201 | parser.add_argument('logfiles', nargs='*',
202 | help="List of Bind query logs to process",
203 | metavar="Logfile")
204 | parser.add_argument('--starttime', nargs=1, help="Create statistics from this time forward",
205 | default="", metavar="Start Time")
206 | parser.add_argument('--endtime', nargs=1, help="Process statistics until this period of time",
207 | default="", metavar="End Time")
208 | args = parser.parse_args()
209 |
210 | return args.count, args.domains, args.logfiles, args.excludeip, args.excludenet, args.matrix, args.histogram
211 |
212 |
213 | def process_logs(logs):
214 | """ Takes a list of logfiles and calls a DNSStats
215 | method to process each logfile entry
216 | """
217 |
218 | for log in logs:
219 | print "Processing logfile " + log
220 | try:
221 | with open(log, 'r') as query_log:
222 | for query in query_log:
223 | timestamp, dns_question, rr_type, client_ip = process_query(query)
224 | process_time(timestamp)
225 | generate_statistics(dns_question, rr_type, client_ip)
226 | if ENABLE_HISTOGRAMS:
227 | populate_histograms(timestamp)
228 | except IOError:
229 | print "Could not open file " + log + " for processing"
230 | sys.exit(1)
231 |
232 | def populate_histograms(timestamp):
233 | """
234 | Create stats about when queries occurred
235 | 06-Nov-2016 11:44:43.946
236 | """
237 | day, month, year, hour, minute, second, _ = re.split(r'[-:.\s]', timestamp)
238 | HISTOGRAM_HOUR[hour] += 1
239 | HISTOGRAM_MINUTE[minute] += 1
240 |
241 |
242 | def create_histogram(label, dns_dict):
243 | """
244 | Create a histogram for queries per minute, hour, day, etc.
245 | """
246 | LARGEST_VALUE = max(dns_dict.values())
247 | SCALE = 50.00 / LARGEST_VALUE
248 |
249 | print "\nQueries per %s:" % label
250 | for interval, queries in sorted(dns_dict.iteritems()):
251 | hist_size = "*" * int(queries * SCALE)
252 | print " %2s: %s (%d)" % (interval, hist_size, queries)
253 |
254 | if __name__ == "__main__":
255 |
256 | (COUNT, DOMAINS, LOGFILES, IPS_TO_EXCLUDE, NETS_TO_EXCLUDE,
257 | ENABLE_RESOLUTION_MATRIX, ENABLE_HISTOGRAMS) = processcli()
258 |
259 | if not LOGFILES:
260 | print "No log files specified on command line"
261 | sys.exit(1)
262 |
263 | process_logs(LOGFILES)
264 | print_dns_summary()
265 | print_top_dns_requests(COUNT)
266 | print_top_dns_clients(COUNT)
267 |
268 | if ENABLE_HISTOGRAMS:
269 | create_histogram("minute", HISTOGRAM_MINUTE)
270 | create_histogram("hour", HISTOGRAM_HOUR)
271 |
272 | if ENABLE_RESOLUTION_MATRIX:
273 | print_dns_resolution_matrix()
274 |
--------------------------------------------------------------------------------