├── .gitignore ├── LICENSE ├── README.md ├── blacklists ├── README.md ├── blacklists ├── blacklists.conf.sample ├── blacklists.service ├── blacklists.timer ├── build_package ├── copyright └── postinst ├── camera_poe_ctl ├── .gitignore ├── README.md ├── build_package ├── camera_poe_ctl ├── camera_poe_ctl.conf.sample ├── camera_poe_ctl.service ├── camera_poe_ctl.timer ├── postinst └── udm.py ├── deprecated ├── away-recording │ ├── README.md │ └── away-recording ├── ntopng-udm-fedora │ ├── Docker.com-README.md │ ├── Dockerfile │ ├── LICENSE │ ├── README.md │ ├── TESTING.md │ ├── build-ntopng-docker-image │ ├── data │ │ └── ntopng │ │ │ ├── GeoIP.conf │ │ │ ├── ntopng.conf │ │ │ └── redis.conf │ ├── entrypoint.sh │ └── rpmbuild │ │ ├── README.md │ │ ├── lua-makefile-add-fPIC.patch │ │ ├── ntopng.conf │ │ ├── ntopng.service │ │ ├── ntopng.spec │ │ └── ntopng.sysconfig └── on_boot.d │ ├── 05-profile.sh │ ├── 15-add-root-ssh-key.sh │ ├── 15-nameservers.sh │ ├── 15-start-ntopng.sh │ └── 50-cronjobs.sh ├── gen-debian-repo-key ├── ppa ├── blacklists.html ├── camera_poe_ctl.html ├── index.html ├── markdown.css ├── ntopng.html ├── pgp-key.public ├── udm-hacks-stable.list └── udm-hacks.list ├── restore_dlk_apps └── update_ppa /.gitignore: -------------------------------------------------------------------------------- 1 | .gnupg 2 | ppa/pool 3 | ppa/dists 4 | 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Hacks in this repository: 2 | 3 | * **blacklists** - Configure iptables on the UDM to block the ip addresses contained in the ntopng blacklists 4 | * **camera_poe_ctl** - Turning UDM Protect cameras' POE power on and off based on SmartThings' location status 5 | 6 | Things that used to be here but which have moved: 7 | 8 | * [Running ntopng Natively On The Unbiquiti Dream Machine Pro (UDM-Pro)](https://github.com/dlk3/ntopng-on-udm-pro) - Running the ntopng network traffic analysis tool on the UDM Pro 9 | -------------------------------------------------------------------------------- /blacklists/README.md: -------------------------------------------------------------------------------- 1 | # Block IP Addresses in the Active ntopng Blacklists 2 | 3 | The script in this package configures iptables on the UDM Pro to block all traffic from the IP addresses that are listed in the ntopng blacklists. The script is executed through systemd once a day to refresh the UDM blacklist. Only the active ntopng blacklists are used. See the Settings -> Blacklists menu in ntopng to configure the active lists. 4 | 5 | This project contains the files that I used to build a Debian package that I host in my PPA repository that i9nstalls this script. Please go to [this page](https://daveking.com/udm-hacks/blacklists.html) for more details on using that package. 6 | -------------------------------------------------------------------------------- /blacklists/blacklists: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # This script configures iptables on the UDM Pro to block all traffic from 4 | # the IP addresses that are listed in the ntopng blacklists. 5 | # 6 | # Copyright (C) 2024 David King 7 | # 8 | # This program is free software; you can redistribute it and/or 9 | # modify it under the terms of the GNU General Public License, 10 | # version 2, as published by the Free Software Foundation. 11 | # 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, If not, see 19 | # . 20 | 21 | # A list of the directories where ntopng keeps the updated copies of blacklists it 22 | # downloads and the custom lists the user has defined 23 | ntop_lists_paths = ['/usr/share/ntopng/httpdocs/misc/lists/builtin', '/usr/share/ntopng/httpdocs/misc/lists/custom'] 24 | 25 | # Maximum number of elements that can be put into an ipset, default = 65536 26 | maxelem = 524288 27 | 28 | # Configure logging 29 | import logging 30 | logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.INFO) 31 | 32 | import subprocess 33 | import ipaddress 34 | import os 35 | import json 36 | import socket 37 | import sys 38 | from optparse import OptionParser 39 | 40 | # Make sure that iptables has the rules we need to define a blacklists chain called NTOPNG 41 | # that logs and drops the packets coming in from blacklisted addresses listed in an ipset 42 | # named ntopng 43 | def mk_iptables_rules(): 44 | 45 | # Create the NTOPNG chain that filters packets based on the blacklist sets 46 | try: 47 | proc = subprocess.run('iptables -nL NTOPNG', shell=True, check=True, capture_output=True) 48 | logging.debug('NTOPNG target already exists in iptables') 49 | except: 50 | logging.info('defining new NTOPNG target in iptables') 51 | subprocess.run('iptables -N NTOPNG', shell=True, check=True) 52 | 53 | # Define the NTOPNGLOGNDROP log and drop target 54 | try: 55 | proc = subprocess.run('iptables -nL NTOPNGLOGNDROP', shell=True, check=True, capture_output=True) 56 | logging.debug('NTOPNGLOGNDROP target already exists in iptables') 57 | except: 58 | logging.info('defining new NTOPNGLOGNDROP target in iptables') 59 | subprocess.run('iptables -N NTOPNGLOGNDROP', shell=True, check=True) 60 | subprocess.run('iptables -A NTOPNGLOGNDROP -m limit --limit 5/min --limit-burst 10 -j LOG --log-level warning --log-prefix "NTOPNG BLOCK: "', shell=True, check=True) 61 | subprocess.run('iptables -A NTOPNGLOGNDROP -j DROP', shell=True, check=True) 62 | 63 | # Add the INPUT rule that routes all traffic to the NTOPNG target 64 | found = False 65 | proc = subprocess.run('iptables -nL INPUT', shell=True, check=True, capture_output=True) 66 | for line in proc.stdout.decode('utf-=8').split('\n'): 67 | if 'NTOPNG' in line: 68 | found = True 69 | logging.debug('NTOPNG target already in the INPUT chain in iptables') 70 | break 71 | if not found: 72 | logging.info('adding NTOPNG target to the INPUT chain in iptables') 73 | try: 74 | index = len(proc.stdout.decode('utf-=8').split('\n')) - 3 75 | subprocess.run('iptables -I INPUT {} -j NTOPNG'.format(index), shell=True, check=True, capture_output=True) 76 | except: 77 | pass 78 | 79 | # Add the FORWARD rule that routes all traffic to the NTOPNG target 80 | found = False 81 | proc = subprocess.run('iptables -nL FORWARD', shell=True, check=True, capture_output=True) 82 | for line in proc.stdout.decode('utf-=8').split('\n'): 83 | if 'NTOPNG' in line: 84 | found = True 85 | logging.debug('NTOPNG target is already in the FORWARD chain in iptables') 86 | break 87 | if not found: 88 | logging.info('adding NTOPNG target to the FORWARD chain in iptables') 89 | try: 90 | index = len(proc.stdout.decode('utf-=8').split('\n')) - 3 91 | subprocess.run('iptables -I FORWARD {} -j NTOPNG'.format(index), shell=True, check=True, capture_output=True) 92 | except: 93 | pass 94 | 95 | # Create the ntopng ipset, if it doesn't already exist 96 | try: 97 | subprocess.run('ipset create ntopng "hash:net" maxelem {}'.format(maxelem), shell=True, check=True, capture_output=True) 98 | except subprocess.CalledProcessError: 99 | pass 100 | 101 | # Add the two ntopng ipset filtering rules to the NTOPNG chain 102 | for ipset_rule in ['ntopng src', 'ntopng dst']: 103 | found = False 104 | proc = subprocess.run('iptables -nL NTOPNG', shell=True, check=True, capture_output=True) 105 | for line in proc.stdout.decode('utf-=8').split('\n'): 106 | if ipset_rule in line: 107 | found = True 108 | logging.debug('"{}" ipset rule already exists in NTOPNG chain'.format(ipset_rule)) 109 | break 110 | if not found: 111 | logging.info('adding "{}" ipset rule to the NTOPNG chain'.format(ipset_rule)) 112 | try: 113 | subprocess.run('iptables -A NTOPNG -m set --match-set {} -j NTOPNGLOGNDROP'.format(ipset_rule), shell=True, check=True, capture_output=True) 114 | except: 115 | pass 116 | 117 | # Back out the changes made by mk_iptables_rules() 118 | # (This is not used, it's included here just for reference.) 119 | def rm_iptables_rules(): 120 | cmds = [ 121 | 'iptables -D INPUT -j NTOPNG', 122 | 'iptables -D FORWARD -j NTOPNG', 123 | 'iptables -D NTOPNGLOGNDROP -m limit --limit 5/min --limit-burst 10 -j LOG --log-level warning --log-prefix "NTOPNG BLOCK: "', 124 | 'iptables -D NTOPNGLOGNDROP -j DROP', 125 | 'iptables -D NTOPNG -m set --match-set ntopng src -j NTOPNGLOGNDROP', 126 | 'iptables -D NTOPNG -m set --match-set ntopng dst -j NTOPNGLOGNDROP', 127 | 'iptables -X NTOPNGLOGNDROP', 128 | 'iptables -X NTOPNG', 129 | 'ipset destroy ntopng' 130 | ] 131 | for cmd in cmds: 132 | try: 133 | subprocess.run(cmd, shell=True, check=True, capture_output=True) 134 | except: 135 | pass 136 | 137 | # Find ntopng's "category_lists" directory on the UDM data volumes /volume1 or /data 138 | def find_category_lists_path(): 139 | for ntop_data_path in ['/volume1', '/data']: 140 | for root, paths, files in os.walk(ntop_data_path): 141 | for path in paths: 142 | if path == 'category_lists': 143 | return os.path.join(root, path) 144 | return False 145 | 146 | # Resolve a hostname to an IP address 147 | def resolve_hostname(hostname): 148 | try: 149 | for line in socket.getaddrinfo(hostname, 0, family=socket.AF_INET, type=socket.SOCK_STREAM): 150 | return line[4][0] 151 | except socket.gaierror: 152 | pass 153 | except: 154 | logging.exception('Exception while resolving hostname: {}'.format(hostname), exc_info=True) 155 | 156 | logging.debug('Unable to resolve hostname: {}'.format(hostname)) 157 | return 158 | 159 | # Check to see if an ip address is in the excluded_address list defined above 160 | def check_if_excluded(ip): 161 | try: 162 | ip = ip.split('/')[0] 163 | def convert_ipv4(ip): 164 | return tuple(int(n) for n in ip.split('.')) 165 | for item in excluded_addresses: 166 | if '-' in item: 167 | start, end = item.split('-') 168 | else: 169 | start = end = item 170 | if convert_ipv4(start) <= convert_ipv4(ip) <= convert_ipv4(end): 171 | return True 172 | except: 173 | logging.warning('Unexpected exception in check_if_excluded(), "{}" is not excluded'.format(ip), exc_info=True) 174 | return False 175 | 176 | def load_list(list_name, ipset_name): 177 | # Read this list's data file from ntopng's data directory into a list 178 | category_lists_path = find_category_lists_path() 179 | if not category_lists_path: 180 | logging.critical('Unable to locate ntopng\'s category_lists directory') 181 | return False 182 | file_name = os.path.join(category_lists_path, '{}.txt'.format(list_name)) 183 | try: 184 | with open(file_name) as f: 185 | lines = f.read().splitlines() 186 | except: 187 | logging.exception('Unable to read "{}"'.format(file_name), exc_info=True) 188 | return False 189 | 190 | # Remove comment and blank lines 191 | temp_lines = [] 192 | for line in lines: 193 | if not line.strip().startswith('#') and line.strip() != '': 194 | temp_lines.append(line) 195 | del lines 196 | 197 | # Parse each line and add the ip/netmask value to the hash:net ipset 198 | for line in temp_lines: 199 | ip = None 200 | if ',' in line: 201 | ip = line.split(',')[0] 202 | else: 203 | line = line.split() 204 | if line[0] == '127.0.0.1' or line[0] == '0.0.0.0': 205 | ip = resolve_hostname(line[1]) 206 | else: 207 | ip = line[0] 208 | 209 | if ip: 210 | # Add the netmask to IP addresses that don't have them 211 | if not '/' in ip: 212 | ip = '{}/32'.format(ip) 213 | 214 | # Validate the ip address format 215 | try: 216 | ipaddress.ip_address(ip.split('/')[0]) 217 | except: 218 | logging.debug('"{}" is not a valid ip address'.format(ip)) 219 | continue 220 | 221 | # Loopback and private ip addresses should never be blacklisted 222 | if check_if_excluded(ip): 223 | logging.debug('{} cannot be blacklisted'.format(ip)) 224 | continue 225 | 226 | # Add the address to the ipset 227 | try: 228 | logging.debug('adding "{}" to the {} ipset'.format(ip, ipset_name)) 229 | subprocess.run('ipset -exist add {} {}'.format(ipset_name, ip), shell=True, check=True) 230 | except: 231 | logging.exception('ERROR: ipset -exist add {} {}'.format(ipset_name, ip), exc_info=True) 232 | 233 | if __name__ == "__main__": 234 | 235 | # Define command-line options for the config file and debugging 236 | config_fn = '/data/blacklists/blacklists.conf' 237 | cmdline = OptionParser(usage="usage: %prog [-d | --debug]", description='This script configures the local system\'s iptables firewall to block traffic from the IP addresses listed in the ntopng blacklists. It is designed to be run daily to apply the changes made to the blacklists as they are updated by ntopng. For more details see https://daveking.com/udm-hacks/blacklists.html') 238 | cmdline.add_option('-c', '--config', dest='conffile', default=config_fn, help='Full path to configuration file. Default: {}'.format(config_fn)) 239 | cmdline.add_option('-d', '--debug', action='store_true', dest='debug', default=False, help='Write debugging messages') 240 | opts, args = cmdline.parse_args() 241 | log = logging.getLogger() 242 | if opts.debug: 243 | log.setLevel(logging.DEBUG) 244 | 245 | # Execute the configuration file 246 | if not os.path.exists(opts.conffile): 247 | logging.critical('{} configuration file does not exist'.format(opts.conffile)) 248 | exit(1) 249 | logging.debug('Executing the configuration file: {}'.format(opts.conffile)) 250 | try: 251 | with open(opts.conffile) as f: 252 | code = compile(f.read(), opts.conffile, 'exec') 253 | exec(code) 254 | except: 255 | logging.exception('Unable to execute config file "{}"'.format(opts.conffile), exc_info=True) 256 | exit(1) 257 | 258 | # The excluded_addresses list should have come from the config file, but if 259 | # it didn't then we'll use this one. 260 | try: 261 | excluded_addresses 262 | except NameError: 263 | excluded_addresses = [ 264 | '0.0.0.0', 265 | '255.255.255.255', 266 | '127.0.0.0-127.255.255.255', 267 | '192.168.0.0-192.168.255.255', 268 | '10.0.0.0-10.255.255.255', 269 | '100.64.0.0-100.127.255.255', 270 | '172.16.0.0-172.31.255.255' 271 | ] 272 | 273 | # Make sure that the iptables rules we need to make this all work are in place 274 | mk_iptables_rules() 275 | 276 | logging.debug('Creating temporary ipset: new_ntopng') 277 | subprocess.run('ipset -exist create new_ntopng "hash:net" maxelem {}'.format(maxelem), shell=True) 278 | subprocess.run('ipset flush new_ntopng', shell=True) 279 | 280 | # Load the temporary ipset with the addresses from ntopng's active blacklists 281 | for ntop_path in ntop_lists_paths: 282 | # Inspect all the list definitions found in ntop_list_paths directories 283 | for root, paths, files in os.walk(ntop_path): 284 | # If the list is enabled in ntopng, process it into the ipset 285 | for file in files: 286 | if os.path.splitext(file)[1] == '.list': 287 | with open(os.path.join(root, file), 'r') as f: 288 | list_def = json.load(f) 289 | if list_def['enabled'] == True: 290 | logging.info('Processing ntopng blacklist file: "{}"'.format(file)) 291 | load_list(list_def['name'], 'new_ntopng') 292 | else: 293 | logging.debug('{} is not enabled, skipping'.format(file)) 294 | 295 | logging.debug('Swaping temporary ipset in place of the old one: ntopng') 296 | subprocess.run('ipset swap new_ntopng ntopng', shell=True) 297 | subprocess.run('ipset destroy new_ntopng', shell=True) 298 | 299 | -------------------------------------------------------------------------------- /blacklists/blacklists.conf.sample: -------------------------------------------------------------------------------- 1 | # Configuration file for the blacklists script 2 | # 3 | # The blacklist script executes this file as Python code 4 | 5 | # List of individual IP addresses or address ranges that should never be blacklisted 6 | excluded_addresses = [ 7 | '0.0.0.0', # Do not remove 8 | '255.255.255.255', # Do not remove 9 | '127.0.0.0-127.255.255.255', # Do not remove 10 | '192.168.0.0-192.168.255.255', # Do not remove 11 | '10.0.0.0-10.255.255.255', # Do not remove 12 | '100.64.0.0-100.127.255.255', # Do not remove 13 | '172.16.0.0-172.31.255.255', # Do not remove 14 | '1.0.0.1', # Cloudflare nameserver 15 | '1.1.1.1', # Cloudflare nameserver 16 | '8.8.4.4', # Google nameserver 17 | '8.8.8.8' # Google nameserver 18 | ] 19 | -------------------------------------------------------------------------------- /blacklists/blacklists.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=updating ntopng ipset from ntopng blacklists 3 | 4 | [Service] 5 | Type=oneshot 6 | ExecStart=/usr/bin/blacklists 7 | 8 | -------------------------------------------------------------------------------- /blacklists/blacklists.timer: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=updating ntopng ipset from ntopng blacklists 3 | 4 | [Timer] 5 | OnStartupSec=120 6 | OnCalendar=*-*-* 04:00:00 7 | Persistent=true 8 | 9 | [Install] 10 | WantedBy=timers.target 11 | -------------------------------------------------------------------------------- /blacklists/build_package: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | set -e 4 | 5 | PKGNAME="blacklists" 6 | PKGVER="2.4.3" 7 | PKGREL="0" 8 | PKGARCH="arm64" 9 | PKGMAINT="Dave King " 10 | PKGDESC="Configure iptables on the UDM Pro to block traffic from the IP addresses listed in the active ntopng blacklists. The list of blocked addresses is updated once a day from the source." 11 | 12 | BUILDDIR=${PKGNAME}_${PKGVER}-${PKGREL}_${PKGARCH} 13 | 14 | rm -fr ${BUILDDIR} &>/dev/null 15 | mkdir -p ${BUILDDIR}/DEBIAN 16 | mkdir -p ${BUILDDIR}/etc/systemd/system 17 | mkdir -p ${BUILDDIR}/usr/bin 18 | mkdir -p ${BUILDDIR}/data/blacklists 19 | mkdir -p ${BUILDDIR}/usr/share/doc/blacklists 20 | 21 | cp -v postinst ${BUILDDIR}/DEBIAN/ 22 | cp -v blacklists.timer blacklists.service ${BUILDDIR}/etc/systemd/system/ 23 | cp -v blacklists ${BUILDDIR}/usr/bin/ 24 | cp -v blacklists.conf.sample ${BUILDDIR}/data/blacklists/ 25 | cp -v copyright ${BUILDDIR}/usr/share/doc/blacklists/ 26 | chmod 755 ${BUILDDIR}/usr/bin/blacklists 27 | 28 | cat <<_EOF_ >${BUILDDIR}/DEBIAN/control 29 | Package: ${PKGNAME} 30 | Version: ${PKGVER} 31 | Architecture: ${PKGARCH} 32 | Maintainer: ${PKGMAINT} 33 | Description: ${PKGDESC} 34 | Depends: udmpro-al324-base-files (>=3), ntopng, ntopng-data 35 | _EOF_ 36 | 37 | dpkg-deb --build --root-owner-group ${BUILDDIR} 38 | 39 | mv -v ${BUILDDIR}.deb ${HOME}/src/udm-hacks/ppa/pool/main/b/blacklists/ 40 | 41 | ${HOME}/src/udm-hacks/update_ppa 42 | 43 | rm -fr ${BUILDDIR} 44 | -------------------------------------------------------------------------------- /blacklists/copyright: -------------------------------------------------------------------------------- 1 | Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ 2 | Source: https://github.com/dlk3/udm-hacks/tree/master/blacklists 3 | Upstream-Name: blacklists 4 | 5 | Files: 6 | * 7 | Copyright: 2023 David King 8 | License: GPL-2 9 | 10 | GNU GENERAL PUBLIC LICENSE 11 | Version 2, June 1991 12 | 13 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 14 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 15 | Everyone is permitted to copy and distribute verbatim copies 16 | of this license document, but changing it is not allowed. 17 | 18 | Preamble 19 | 20 | The licenses for most software are designed to take away your 21 | freedom to share and change it. By contrast, the GNU General Public 22 | License is intended to guarantee your freedom to share and change free 23 | software--to make sure the software is free for all its users. This 24 | General Public License applies to most of the Free Software 25 | Foundation's software and to any other program whose authors commit to 26 | using it. (Some other Free Software Foundation software is covered by 27 | the GNU Lesser General Public License instead.) You can apply it to 28 | your programs, too. 29 | 30 | When we speak of free software, we are referring to freedom, not 31 | price. Our General Public Licenses are designed to make sure that you 32 | have the freedom to distribute copies of free software (and charge for 33 | this service if you wish), that you receive source code or can get it 34 | if you want it, that you can change the software or use pieces of it 35 | in new free programs; and that you know you can do these things. 36 | 37 | To protect your rights, we need to make restrictions that forbid 38 | anyone to deny you these rights or to ask you to surrender the rights. 39 | These restrictions translate to certain responsibilities for you if you 40 | distribute copies of the software, or if you modify it. 41 | 42 | For example, if you distribute copies of such a program, whether 43 | gratis or for a fee, you must give the recipients all the rights that 44 | you have. You must make sure that they, too, receive or can get the 45 | source code. And you must show them these terms so they know their 46 | rights. 47 | 48 | We protect your rights with two steps: (1) copyright the software, and 49 | (2) offer you this license which gives you legal permission to copy, 50 | distribute and/or modify the software. 51 | 52 | Also, for each author's protection and ours, we want to make certain 53 | that everyone understands that there is no warranty for this free 54 | software. If the software is modified by someone else and passed on, we 55 | want its recipients to know that what they have is not the original, so 56 | that any problems introduced by others will not reflect on the original 57 | authors' reputations. 58 | 59 | Finally, any free program is threatened constantly by software 60 | patents. We wish to avoid the danger that redistributors of a free 61 | program will individually obtain patent licenses, in effect making the 62 | program proprietary. To prevent this, we have made it clear that any 63 | patent must be licensed for everyone's free use or not licensed at all. 64 | 65 | The precise terms and conditions for copying, distribution and 66 | modification follow. 67 | 68 | GNU GENERAL PUBLIC LICENSE 69 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 70 | 71 | 0. This License applies to any program or other work which contains 72 | a notice placed by the copyright holder saying it may be distributed 73 | under the terms of this General Public License. The "Program", below, 74 | refers to any such program or work, and a "work based on the Program" 75 | means either the Program or any derivative work under copyright law: 76 | that is to say, a work containing the Program or a portion of it, 77 | either verbatim or with modifications and/or translated into another 78 | language. (Hereinafter, translation is included without limitation in 79 | the term "modification".) Each licensee is addressed as "you". 80 | 81 | Activities other than copying, distribution and modification are not 82 | covered by this License; they are outside its scope. The act of 83 | running the Program is not restricted, and the output from the Program 84 | is covered only if its contents constitute a work based on the 85 | Program (independent of having been made by running the Program). 86 | Whether that is true depends on what the Program does. 87 | 88 | 1. You may copy and distribute verbatim copies of the Program's 89 | source code as you receive it, in any medium, provided that you 90 | conspicuously and appropriately publish on each copy an appropriate 91 | copyright notice and disclaimer of warranty; keep intact all the 92 | notices that refer to this License and to the absence of any warranty; 93 | and give any other recipients of the Program a copy of this License 94 | along with the Program. 95 | 96 | You may charge a fee for the physical act of transferring a copy, and 97 | you may at your option offer warranty protection in exchange for a fee. 98 | 99 | 2. You may modify your copy or copies of the Program or any portion 100 | of it, thus forming a work based on the Program, and copy and 101 | distribute such modifications or work under the terms of Section 1 102 | above, provided that you also meet all of these conditions: 103 | 104 | a) You must cause the modified files to carry prominent notices 105 | stating that you changed the files and the date of any change. 106 | 107 | b) You must cause any work that you distribute or publish, that in 108 | whole or in part contains or is derived from the Program or any 109 | part thereof, to be licensed as a whole at no charge to all third 110 | parties under the terms of this License. 111 | 112 | c) If the modified program normally reads commands interactively 113 | when run, you must cause it, when started running for such 114 | interactive use in the most ordinary way, to print or display an 115 | announcement including an appropriate copyright notice and a 116 | notice that there is no warranty (or else, saying that you provide 117 | a warranty) and that users may redistribute the program under 118 | these conditions, and telling the user how to view a copy of this 119 | License. (Exception: if the Program itself is interactive but 120 | does not normally print such an announcement, your work based on 121 | the Program is not required to print an announcement.) 122 | 123 | These requirements apply to the modified work as a whole. If 124 | identifiable sections of that work are not derived from the Program, 125 | and can be reasonably considered independent and separate works in 126 | themselves, then this License, and its terms, do not apply to those 127 | sections when you distribute them as separate works. But when you 128 | distribute the same sections as part of a whole which is a work based 129 | on the Program, the distribution of the whole must be on the terms of 130 | this License, whose permissions for other licensees extend to the 131 | entire whole, and thus to each and every part regardless of who wrote it. 132 | 133 | Thus, it is not the intent of this section to claim rights or contest 134 | your rights to work written entirely by you; rather, the intent is to 135 | exercise the right to control the distribution of derivative or 136 | collective works based on the Program. 137 | 138 | In addition, mere aggregation of another work not based on the Program 139 | with the Program (or with a work based on the Program) on a volume of 140 | a storage or distribution medium does not bring the other work under 141 | the scope of this License. 142 | 143 | 3. You may copy and distribute the Program (or a work based on it, 144 | under Section 2) in object code or executable form under the terms of 145 | Sections 1 and 2 above provided that you also do one of the following: 146 | 147 | a) Accompany it with the complete corresponding machine-readable 148 | source code, which must be distributed under the terms of Sections 149 | 1 and 2 above on a medium customarily used for software interchange; or, 150 | 151 | b) Accompany it with a written offer, valid for at least three 152 | years, to give any third party, for a charge no more than your 153 | cost of physically performing source distribution, a complete 154 | machine-readable copy of the corresponding source code, to be 155 | distributed under the terms of Sections 1 and 2 above on a medium 156 | customarily used for software interchange; or, 157 | 158 | c) Accompany it with the information you received as to the offer 159 | to distribute corresponding source code. (This alternative is 160 | allowed only for noncommercial distribution and only if you 161 | received the program in object code or executable form with such 162 | an offer, in accord with Subsection b above.) 163 | 164 | The source code for a work means the preferred form of the work for 165 | making modifications to it. For an executable work, complete source 166 | code means all the source code for all modules it contains, plus any 167 | associated interface definition files, plus the scripts used to 168 | control compilation and installation of the executable. However, as a 169 | special exception, the source code distributed need not include 170 | anything that is normally distributed (in either source or binary 171 | form) with the major components (compiler, kernel, and so on) of the 172 | operating system on which the executable runs, unless that component 173 | itself accompanies the executable. 174 | 175 | If distribution of executable or object code is made by offering 176 | access to copy from a designated place, then offering equivalent 177 | access to copy the source code from the same place counts as 178 | distribution of the source code, even though third parties are not 179 | compelled to copy the source along with the object code. 180 | 181 | 4. You may not copy, modify, sublicense, or distribute the Program 182 | except as expressly provided under this License. Any attempt 183 | otherwise to copy, modify, sublicense or distribute the Program is 184 | void, and will automatically terminate your rights under this License. 185 | However, parties who have received copies, or rights, from you under 186 | this License will not have their licenses terminated so long as such 187 | parties remain in full compliance. 188 | 189 | 5. You are not required to accept this License, since you have not 190 | signed it. However, nothing else grants you permission to modify or 191 | distribute the Program or its derivative works. These actions are 192 | prohibited by law if you do not accept this License. Therefore, by 193 | modifying or distributing the Program (or any work based on the 194 | Program), you indicate your acceptance of this License to do so, and 195 | all its terms and conditions for copying, distributing or modifying 196 | the Program or works based on it. 197 | 198 | 6. Each time you redistribute the Program (or any work based on the 199 | Program), the recipient automatically receives a license from the 200 | original licensor to copy, distribute or modify the Program subject to 201 | these terms and conditions. You may not impose any further 202 | restrictions on the recipients' exercise of the rights granted herein. 203 | You are not responsible for enforcing compliance by third parties to 204 | this License. 205 | 206 | 7. If, as a consequence of a court judgment or allegation of patent 207 | infringement or for any other reason (not limited to patent issues), 208 | conditions are imposed on you (whether by court order, agreement or 209 | otherwise) that contradict the conditions of this License, they do not 210 | excuse you from the conditions of this License. If you cannot 211 | distribute so as to satisfy simultaneously your obligations under this 212 | License and any other pertinent obligations, then as a consequence you 213 | may not distribute the Program at all. For example, if a patent 214 | license would not permit royalty-free redistribution of the Program by 215 | all those who receive copies directly or indirectly through you, then 216 | the only way you could satisfy both it and this License would be to 217 | refrain entirely from distribution of the Program. 218 | 219 | If any portion of this section is held invalid or unenforceable under 220 | any particular circumstance, the balance of the section is intended to 221 | apply and the section as a whole is intended to apply in other 222 | circumstances. 223 | 224 | It is not the purpose of this section to induce you to infringe any 225 | patents or other property right claims or to contest validity of any 226 | such claims; this section has the sole purpose of protecting the 227 | integrity of the free software distribution system, which is 228 | implemented by public license practices. Many people have made 229 | generous contributions to the wide range of software distributed 230 | through that system in reliance on consistent application of that 231 | system; it is up to the author/donor to decide if he or she is willing 232 | to distribute software through any other system and a licensee cannot 233 | impose that choice. 234 | 235 | This section is intended to make thoroughly clear what is believed to 236 | be a consequence of the rest of this License. 237 | 238 | 8. If the distribution and/or use of the Program is restricted in 239 | certain countries either by patents or by copyrighted interfaces, the 240 | original copyright holder who places the Program under this License 241 | may add an explicit geographical distribution limitation excluding 242 | those countries, so that distribution is permitted only in or among 243 | countries not thus excluded. In such case, this License incorporates 244 | the limitation as if written in the body of this License. 245 | 246 | 9. The Free Software Foundation may publish revised and/or new versions 247 | of the General Public License from time to time. Such new versions will 248 | be similar in spirit to the present version, but may differ in detail to 249 | address new problems or concerns. 250 | 251 | Each version is given a distinguishing version number. If the Program 252 | specifies a version number of this License which applies to it and "any 253 | later version", you have the option of following the terms and conditions 254 | either of that version or of any later version published by the Free 255 | Software Foundation. If the Program does not specify a version number of 256 | this License, you may choose any version ever published by the Free Software 257 | Foundation. 258 | 259 | 10. If you wish to incorporate parts of the Program into other free 260 | programs whose distribution conditions are different, write to the author 261 | to ask for permission. For software which is copyrighted by the Free 262 | Software Foundation, write to the Free Software Foundation; we sometimes 263 | make exceptions for this. Our decision will be guided by the two goals 264 | of preserving the free status of all derivatives of our free software and 265 | of promoting the sharing and reuse of software generally. 266 | 267 | NO WARRANTY 268 | 269 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 270 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 271 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 272 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 273 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 274 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 275 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 276 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 277 | REPAIR OR CORRECTION. 278 | 279 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 280 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 281 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 282 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 283 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 284 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 285 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 286 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 287 | POSSIBILITY OF SUCH DAMAGES. 288 | 289 | END OF TERMS AND CONDITIONS 290 | 291 | How to Apply These Terms to Your New Programs 292 | 293 | If you develop a new program, and you want it to be of the greatest 294 | possible use to the public, the best way to achieve this is to make it 295 | free software which everyone can redistribute and change under these terms. 296 | 297 | To do so, attach the following notices to the program. It is safest 298 | to attach them to the start of each source file to most effectively 299 | convey the exclusion of warranty; and each file should have at least 300 | the "copyright" line and a pointer to where the full notice is found. 301 | 302 | 303 | Copyright (C) 304 | 305 | This program is free software; you can redistribute it and/or modify 306 | it under the terms of the GNU General Public License as published by 307 | the Free Software Foundation; either version 2 of the License, or 308 | (at your option) any later version. 309 | 310 | This program is distributed in the hope that it will be useful, 311 | but WITHOUT ANY WARRANTY; without even the implied warranty of 312 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 313 | GNU General Public License for more details. 314 | 315 | You should have received a copy of the GNU General Public License along 316 | with this program; if not, write to the Free Software Foundation, Inc., 317 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 318 | 319 | Also add information on how to contact you by electronic and paper mail. 320 | 321 | If the program is interactive, make it output a short notice like this 322 | when it starts in an interactive mode: 323 | 324 | Gnomovision version 69, Copyright (C) year name of author 325 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 326 | This is free software, and you are welcome to redistribute it 327 | under certain conditions; type `show c' for details. 328 | 329 | The hypothetical commands `show w' and `show c' should show the appropriate 330 | parts of the General Public License. Of course, the commands you use may 331 | be called something other than `show w' and `show c'; they could even be 332 | mouse-clicks or menu items--whatever suits your program. 333 | 334 | You should also get your employer (if you work as a programmer) or your 335 | school, if any, to sign a "copyright disclaimer" for the program, if 336 | necessary. Here is a sample; alter the names: 337 | 338 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 339 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 340 | 341 | , 1 April 1989 342 | Ty Coon, President of Vice 343 | 344 | This General Public License does not permit incorporating your program into 345 | proprietary programs. If your program is a subroutine library, you may 346 | consider it more useful to permit linking proprietary applications with the 347 | library. If this is what you want to do, use the GNU Lesser General 348 | Public License instead of this License. 349 | -------------------------------------------------------------------------------- /blacklists/postinst: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | 3 | # Previous package version on upgrade 4 | PREV_VERSION=$2 5 | 6 | case "$1" in 7 | configure) 8 | # continue below 9 | ;; 10 | 11 | abort-upgrade|abort-remove|abort-deconfigure) 12 | exit 0 13 | ;; 14 | 15 | *) 16 | echo "postinst called with unknown argument \`$1'" >&2 17 | exit 0 18 | ;; 19 | esac 20 | 21 | umask 022 22 | 23 | # If there isn't already an existing configuration file, create one 24 | if [ ! -f /data/blacklists/blacklists.conf ]; then 25 | cp /data/blacklists/blacklists.conf.sample /data/blacklists/blacklists.conf 26 | fi 27 | 28 | # Enable and start the blacklists.timer 29 | systemctl daemon-reload 30 | systemctl reset-failed 31 | if [ -z "$PREV_VERSION" ]; then 32 | systemctl enable blacklists.timer 33 | fi 34 | systemctl -q is-enabled blacklists.timer && systemctl restart blacklists.timer 35 | 36 | exit 0 37 | -------------------------------------------------------------------------------- /camera_poe_ctl/.gitignore: -------------------------------------------------------------------------------- 1 | *.old 2 | *.pyc 3 | __pycache__ 4 | camera_poe_ctl.camera_list.json 5 | test.conf 6 | camera_poe_ctl.conf 7 | -------------------------------------------------------------------------------- /camera_poe_ctl/README.md: -------------------------------------------------------------------------------- 1 | # Turning UDM Protect cameras on and off based on SmartThings' location status 2 | 3 | For privacy reasons we want to keep our indoor Protect cameras turned off unless nobody is home and the house is empty. The script in this package turns on our cameras when we are away and turns them back off when we arrive back home. 4 | 5 | This project contains the files that I used to build a Debian package that I host in my PPA repository that i9nstalls this script. Please go to [this page](https://daveking.com/udm-hacks/camera_poe_ctl.html) for more details on using that package. 6 | -------------------------------------------------------------------------------- /camera_poe_ctl/build_package: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | set -e 4 | 5 | PKGNAME="camera-poe-ctl" 6 | PKGVER="1.2" 7 | PKGREL="0" 8 | PKGARCH="arm64" 9 | PKGMAINT="Dave King " 10 | PKGDESC="Controlling POE for selected cameras on the Ubiquiti Dream Machine Pro based on SmartThings location status 11 | This package contains a script that turns off power to selected cameras when we are at home and want privacy" 12 | 13 | BUILDDIR=${PKGNAME}_${PKGVER}-${PKGREL}_${PKGARCH} 14 | 15 | rm -fr ${BUILDDIR} &>/dev/null 16 | mkdir -p ${BUILDDIR}/data/camera_poe_ctl 17 | mkdir ${BUILDDIR}/DEBIAN 18 | mkdir -p ${BUILDDIR}/etc/systemd/system 19 | mkdir -p ${BUILDDIR}/usr/bin 20 | 21 | cp -v camera_poe_ctl.conf.sample ${BUILDDIR}/data/camera_poe_ctl/ 22 | cp -v postinst ${BUILDDIR}/DEBIAN/ 23 | cp -v camera_poe_ctl.timer camera_poe_ctl.service ${BUILDDIR}/etc/systemd/system/ 24 | cp -v camera_poe_ctl udm.py ${BUILDDIR}/usr/bin/ 25 | chmod 755 ${BUILDDIR}/usr/bin/camera_poe_ctl 26 | 27 | cat <<_EOF_ >${BUILDDIR}/DEBIAN/control 28 | Package: ${PKGNAME} 29 | Version: ${PKGVER} 30 | Architecture: ${PKGARCH} 31 | Maintainer: ${PKGMAINT} 32 | Description: ${PKGDESC} 33 | Depends: udmpro-al324-base-files (>=2.4) 34 | _EOF_ 35 | 36 | dpkg-deb --build --root-owner-group ${BUILDDIR} 37 | 38 | mv -v ${BUILDDIR}.deb ${HOME}/src/udm-hacks/ppa/pool/main/c/camera-poe-ctl/ 39 | 40 | ${HOME}/src/udm-hacks/update_ppa 41 | 42 | rm -fr ${BUILDDIR} 43 | -------------------------------------------------------------------------------- /camera_poe_ctl/camera_poe_ctl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Copyright (C) 2023 David King 4 | # 5 | # This Source Code Form is subject to the terms of the Mozilla Public License, 6 | # v. 2.0. If a copy of the MPL was not distbuted with this file, You can 7 | # obtain one at https://mozilla.org/MPL/2.0/. 8 | 9 | ''' 10 | Turn on the POE power to a list of cameras that are being controlled by a Ubiquiti 11 | Dream Machine when SmartThings says that the location mode is "Away." Turn it off 12 | when the location mode is something other than "Away." 13 | ''' 14 | 15 | # Configure logging 16 | import logging 17 | log_datefmt = '%d-%b-%y %H:%M:%S' 18 | logging.basicConfig(format='%(asctime)s: %(levelname)s: %(message)s', datefmt=log_datefmt, level=logging.INFO) 19 | 20 | # Import dependencies 21 | import os 22 | import sys 23 | from optparse import OptionParser 24 | import json 25 | import smtplib 26 | 27 | # Handle error that occurs if the requests module is not installed 28 | try: 29 | import requests 30 | 31 | # Turn off warnings about the use of self-signed SSL certificates. This code may 32 | # need to be modified if you are using a later version of the requests module. 33 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 34 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 35 | except: 36 | logging.error('Unable to import the Python "requests" module. Is it installed?') 37 | exit(1) 38 | 39 | # Handle error that occurs if my custom udm API module is not present 40 | try: 41 | import udm 42 | except: 43 | logging.error('Unable to import the Python "udm" module that ships with this script.') 44 | exit(1) 45 | 46 | # Return true if the SmartThings location mode is "Away". 47 | def smartthingsAway(url, location_name, api_token): 48 | 49 | headers = { 50 | 'Authorization': 'Bearer ' + api_token 51 | } 52 | 53 | # Get the locationId for the location with the specified name 54 | logging.debug('Getting locationId from SmartThings for "{}" location'.format(location_name)) 55 | locationId = '' 56 | r = requests.get(url + 'locations', headers=headers) 57 | if r.status_code != 200: 58 | logging.error(r.text) 59 | return False 60 | for item in r.json()['items']: 61 | if item['name'] == location_name: 62 | locationId = item['locationId'] 63 | logging.debug('SmartThings locationId: {}'.format(locationId)) 64 | if locationId == '': 65 | logging.error('Unable to get the locationId for the location named "{}"'.format(location_name)) 66 | logging.error('HTTP status code = {}'.format(r.status_code)) 67 | logging.error('Request response text: {}'.format(r.text)) 68 | return False 69 | 70 | # Get the "mode" at that location: Home, Away, Night 71 | logging.debug('Getting current location mode for "{}" location'.format(location_name)) 72 | location_mode = '' 73 | r = requests.get(url + 'locations/' + locationId + '/modes/current', headers=headers) 74 | if r.status_code != 200: 75 | logging.error(r.text) 76 | return False 77 | location_mode = r.json()['name'] 78 | logging.info('SmartThings "{}" location is in "{}" mode.'.format(location_name, location_mode)) 79 | if location_mode == '': 80 | logging.error('Unable to get the curreent mode for the location named "{}"'.format(location_name)) 81 | logging.error('HTTP status code = {}'.format(r.status_code)) 82 | logging.error('Request response text: {}'.format(r.text)) 83 | return False 84 | 85 | if location_mode == 'Away': 86 | return True 87 | return False 88 | 89 | # Send an email message to a list of addresses 90 | def sendMail(address_list=[], message=''): 91 | logging.debug('Sending email message:\n{}'.format(message)) 92 | try: 93 | if len(address_list) > 0: 94 | with smtplib.SMTP(smtpHost, port=smtpPort) as smtp: 95 | if smtpStartTLS: 96 | smtp.starttls() 97 | smtp.login(smtpUserid, smtpPassword) 98 | for address in address_list: 99 | logging.debug('Sending email message to {}'.format(address)) 100 | smtp.sendmail(emailFromAddress, address, 'From: {}\nSubject: {}'.format(emailFromAddress, message)) 101 | except: 102 | logging.exception('Unexpected exception while sending email:\nTo: {}\nMessage: {}'.format(address_list, message), exc_info=True) 103 | 104 | if __name__ == "__main__": 105 | 106 | # Define command-line options for the config file and debugging 107 | config_fn = os.path.abspath(sys.argv[0]) + '.conf' 108 | cmdline = OptionParser(usage="usage: %prog [-d | --debug]", description='Only power on certain security cameras attached to a Ubiquity Dream Machine when SmartThings says that a location is in "Away" mode.') 109 | cmdline.add_option('-c', '--config', dest='conffile', default=config_fn, help='Full path to configuration file. Default: {}'.format(config_fn)) 110 | cmdline.add_option('-d', '--debug', action='store_true', dest='debug', default=False, help='Write debugging messages to the log file specified in the config file') 111 | opts, args = cmdline.parse_args() 112 | log = logging.getLogger() 113 | if opts.debug: 114 | log.setLevel(logging.DEBUG) 115 | 116 | # Default configuration settings. Can be overridden in the configuration file. 117 | udmHostName = 'localhost.localdomain' 118 | udmUsername = '' 119 | udmPassword = '' 120 | smartthingsURL = 'https://api.smartthings.com/v1/' 121 | smartthingsToken = '' 122 | smartthingsLocationName = 'Home' 123 | cameraList = [] 124 | emailAddressList = [] 125 | emailFromAddress = 'camera_poe_ctl@localhost.localdomain' 126 | smtpHost = '' 127 | smtpPort = 25 128 | smtpStartTLS = False 129 | smtpUserid = '' 130 | smtpPassword = '' 131 | cameraDetailsFile = '~/camera_poe_ctl.camera_list.json' 132 | logFileName = '' 133 | 134 | # Execute the configuration file 135 | logging.debug('Executing the configuration file: {}'.format(opts.conffile)) 136 | try: 137 | with open(opts.conffile) as f: 138 | code = compile(f.read(), opts.conffile, 'exec') 139 | exec(code) 140 | except FileNotFoundError: 141 | logging.error('Config file "{}" not found'.format(opts.conffile)) 142 | exit(1) 143 | except: 144 | logging.exception('Unable to execute config file "{}"'.format(opts.conffile), exc_info=True) 145 | exit(1) 146 | 147 | # If config file specified a log file then reconfigure logging 148 | if logFileName != '': 149 | try: 150 | if log.hasHandlers(): 151 | log.handlers.clear() 152 | handler = logging.FileHandler(logFileName) 153 | formatter = logging.Formatter(fmt='%(asctime)s: %(levelname)s: %(message)s', datefmt=log_datefmt) 154 | handler.setFormatter(formatter) 155 | log.addHandler(handler) 156 | except: 157 | logging.eeception('Unexpected exception while switching logging to file: "{}"'.format(config('logFileName')), exc_info=True) 158 | exit(1) 159 | 160 | # Connect to the UDM 161 | udm = udm.UDM(udmHostName, udmUsername, udmPassword, cameraDetailsFile) 162 | 163 | # If SmartThings says location mode is "Away," turn on the cameras. Otherwise, turn 164 | # them off. Send an email if the change succeeded. 165 | away = smartthingsAway(smartthingsURL, smartthingsLocationName, smartthingsToken) 166 | for camera_name in cameraList: 167 | for udm_camera in udm.cameras: 168 | if camera_name == udm_camera.name: 169 | if away: 170 | if udm_camera.setPOE('ON'): 171 | sendMail(emailAddressList, '{} camera powered on'.format(camera_name)) 172 | else: 173 | if udm_camera.setPOE('OFF'): 174 | sendMail(emailAddressList, '{} camera powered off'.format(camera_name)) 175 | break 176 | -------------------------------------------------------------------------------- /camera_poe_ctl/camera_poe_ctl.conf.sample: -------------------------------------------------------------------------------- 1 | # Configuration file for the camera_poe_ctl script 2 | 3 | # The host name for the Ubiquiti Dream Machine. If you are running this script 4 | # on the UDM itself, set this to "localhost" 5 | udmHostName = 'localhost' 6 | 7 | # The admin logon id and password 8 | udmUsername = 'ADMIN_USERID' 9 | udmPassword = 'ADMIN_PASSWORD' 10 | 11 | # The SmartThings API URL 12 | smartthingsURL = 'https://api.smartthings.com/v1/' 13 | # Your SmartThings API token (see https://account.smartthings.com/tokens) 14 | smartthingsToken = 'API_TOKEN' 15 | # The name of the SmartThings location that will be checked for "Away" status 16 | smartthingsLocationName = 'Home' 17 | 18 | # The list of cameras that should be turned on only when SmartThings says that 19 | # the location is in "Away" status. Use Protect app camera names. 20 | cameraList = ['CAMERA_1', 'CAMERA_2'] 21 | 22 | # The list of email addresses to send an email to when cameras are powered on or 23 | # off by this script. I use this to send text messages by using email-to-SMS 24 | # gateway addresses here, like this: 25 | # 26 | # ['5555551212@txt.att.net', '6666661212@msg.fi.google.com'] 27 | # 28 | # For information on the major carriers email-to-sms gateway addresses see 29 | # https://www.makeuseof.com/tag/email-to-sms/ 30 | # 31 | # If emailAddressList is empty, no email messages will be sent 32 | emailAddressList = [] 33 | 34 | # The "From:" email address. Some SMTP servers will require that this be 35 | # a valid, existing email address, others don't care. 36 | emailFromAddress = 'camera_poe_ctl@localhost.localdomain' 37 | 38 | # Parameters needed to connect to the SMTP gateway through which emails will be sent 39 | smtpHost = 'smtp.gmail.com' 40 | smtpPort = 465 41 | smtpStartTLS = True 42 | smtpUserid = 'USERID' 43 | smtpPassword = 'PASSWORD' 44 | 45 | # The file we use to keep track of UDM cameras. The UDM Protect application 46 | # forgets about cameras that have been powered off. We need to remember this 47 | # information so that we can turn cameras back on when necessary, so we save 48 | # the camera information in this file. 49 | cameraDetailsFile = '/data/camera_poe_ctl/camera_poe_ctl.camera_list.json' 50 | 51 | # A log file where this script can write error messages. If this is set to 52 | # an empty string then messages will be written to STDERR. 53 | logFileName = '' 54 | -------------------------------------------------------------------------------- /camera_poe_ctl/camera_poe_ctl.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=controlling POE for selected cameras based on SmartThings location status 3 | 4 | [Service] 5 | Type=oneshot 6 | ExecStart=/usr/bin/camera_poe_ctl --config=/data/camera_poe_ctl/camera_poe_ctl.conf 7 | 8 | -------------------------------------------------------------------------------- /camera_poe_ctl/camera_poe_ctl.timer: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=controlling POE for selected cameras based on SmartThings location status 3 | 4 | [Timer] 5 | OnCalendar=*:0/10 6 | 7 | [Install] 8 | WantedBy=timers.target 9 | -------------------------------------------------------------------------------- /camera_poe_ctl/postinst: -------------------------------------------------------------------------------- 1 | #!/bin/sh -e 2 | 3 | # Previous package version on upgrade 4 | PREV_VERSION=$2 5 | 6 | case "$1" in 7 | configure) 8 | # continue below 9 | ;; 10 | 11 | abort-upgrade|abort-remove|abort-deconfigure) 12 | exit 0 13 | ;; 14 | 15 | *) 16 | echo "postinst called with unknown argument \`$1'" >&2 17 | exit 0 18 | ;; 19 | esac 20 | 21 | umask 022 22 | 23 | # Use /volume1 as the persistent store if it exists 24 | DATADIR="/data/camera_poe_ctl" 25 | if [ -d /volume1/.srv ]; then 26 | DATADIR="/volume1/camera_poe_ctl" 27 | if [ ! -d $DATADIR ]; then 28 | mkdir $DATADIR 29 | mv /data/camera_poe_ctl/* ${DATADIR}/ 30 | fi 31 | rm -fr /data/camera_poe_ctl 32 | sed -i "s|--config=/data/camera_poe_ctl|--config=${DATADIR}|" /etc/systemd/system/camera_poe_ctl.service 33 | sed -i "s|cameraDetailsFile = '/data/camera_poe_ctl|cameraDetailsFile = '${DATADIR}|" ${DATADIR}/camera_poe_ctl.conf.sample 34 | fi 35 | 36 | # Enable and start the openvpn-server.service 37 | systemctl daemon-reload 38 | systemctl reset-failed 39 | if [ -z "$PREV_VERSION" ]; then 40 | systemctl enable camera_poe_ctl.timer 41 | fi 42 | systemctl -q is-enabled camera_poe_ctl.timer && systemctl restart camera_poe_ctl.timer 43 | 44 | # Message to the user 45 | if [ ! -f $/DATADIR}/camera_poe_ctl.conf ]; then 46 | echo "" 47 | echo "You must create a ${DATADIR}/camera_poe_ctl.conf file to make this package work." 48 | echo "See the sample file: ${DATADIR}/camera_poe_ctl.conf.sample" 49 | fi 50 | exit 0 51 | -------------------------------------------------------------------------------- /camera_poe_ctl/udm.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Copyright (C) 2023 David King 4 | # 5 | # This Source Code Form is subject to the terms of the Mozilla Public License, 6 | # v. 2.0. If a copy of the MPL was not distbuted with this file, You can 7 | # obtain one at https://mozilla.org/MPL/2.0/. 8 | 9 | ''' 10 | An API for the Ubiquity Dream Machine 11 | 12 | This is not an official, supported API. It was constructed by introspecting the 13 | request/response exchanges between the UDM and my web browser. As a result, this API may 14 | cease to function at any time, without warning. 15 | 16 | To establish a connection with a UDM: 17 | 18 | import udm 19 | my_udm = udm.UDM(udm_url, admin_userid, admin_password) 20 | 21 | Object attributes: 22 | 23 | my_udm.devices - contains a list of network Device objects known to the UDM. 24 | (/proxy/network/api/s/default/stat/sta) 25 | my_udm.cameras - contains a list of Camera objects known to the UDM. 26 | (/proxy/protect/api/bootstrap) 27 | 28 | A "Device" object represents one network device attached to the UDM Network Console application. 29 | Cameras are one type of device that can be attached, as are the network switches they connect to. 30 | POE status is controlled through the switch device, not the camera device. 31 | 32 | Device Attributes: 33 | session = The requests session that contains the request headers needed to interact with 34 | the UDM that controls this device. 35 | url = The url of the UDM that controls this device. 36 | mac = This device's MAC address. 37 | sw_mac = The MAC address of the switch this device is plugged into. 38 | sw_port = The port number this device is plugged into. 39 | 40 | A "Camera" object represents one camera attached to the UDM Protect application. Some attributes 41 | of the corresponding network device are copied over into this object. 42 | 43 | Camera Attributes: 44 | session = The requests session that contains the request headers needed to interact with 45 | the UDM that controls this device. 46 | url = The url of the UDM that controls this device. 47 | mac = This device's MAC address. 48 | name = The name given to this camera in the Protect application. 49 | lastseen = When this camera was last seen by the Protect application, as a UNIX timestamp. 50 | switch_mac = The MAC address of the switch this camera is plugged into. 51 | switch_port = The port number this camera is plugged into. 52 | 53 | Camera Methods: 54 | getPOE() - returns True or False, indicating the current POE state of the camera, 55 | "ON" or "OFF." 56 | setPOE("ON"|"OFF") - turns camera power on or off. Returns True if no errors ocurred. If 57 | the camera POE is already in the desired state, nothing is done. 58 | ''' 59 | 60 | import logging 61 | import base64 62 | import json 63 | import os 64 | import datetime 65 | 66 | # Handle error that occurs if the requests module is not installed 67 | try: 68 | import requests 69 | 70 | # Turn off warnings about the use of self-signed SSL certificates. This code may 71 | # need to be modified if you are using a later version of the requests module. 72 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 73 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 74 | except: 75 | logging.error('ERROR: Unable to import the Python "requests" module. Is it installed?') 76 | exit(1) 77 | 78 | # Define an object class for a UDM device 79 | class UDM: 80 | 81 | # Login to UDM, creating a requests session object containing the authentication headers 82 | # needed for subsequent requests. Also get the list of known network client devices and 83 | # cameras. 84 | def __init__(self, hostname, userid, password, camera_json_filename=os.path.expanduser('~/.config/udmcameralist.json')): 85 | self.url = 'https://' + hostname 86 | 87 | logging.debug('Logging into UDM') 88 | 89 | # Create session for UDM connection 90 | self.session = requests.Session() 91 | 92 | # Login to UDM 93 | _request_data = { 94 | 'username': userid, 95 | 'password': password, 96 | 'rememberMe': False 97 | } 98 | try: 99 | _r = self.session.post(self.url + '/api/auth/login', json=_request_data, verify=False) 100 | except Exception as e: 101 | logging.exception('Unexpected exception while logging in to {}'.format(url), exc_info=True) 102 | exit(1) 103 | if _r.status_code != 200: 104 | logging.error('The login failed: ' + _r.text) 105 | exit(_r.status_code) 106 | 107 | # Set authorization cookies for all subsequent requests 108 | self.session.headers.update({'Content-Type': 'application/json; charset=utf-8'}) 109 | for _cookie in _r.headers['Set-Cookie'].split(';'): 110 | if _cookie.startswith('TOKEN='): 111 | self.session.headers.update({'Cookie': _cookie + ';'}) 112 | # If the token cookie contains a x-csrf-token value, this must be decoded 113 | # and passed in the request headers otherwise any attempt to change any 114 | # settings will fail with a "404 Not Found" error. 115 | _decoded_token = base64.b64decode(_cookie.split('=')[1].split('.')[1] + '===') 116 | _decoded_token = json.loads(_decoded_token.decode('utf-8')) 117 | if 'csrfToken' in _decoded_token: 118 | self.session.headers.update({'x-csrf-token': _decoded_token['csrfToken']}) 119 | 120 | logging.debug('Authentication headers:\n{}'.format(json.dumps(dict(self.session.headers), indent=4))) 121 | 122 | # Add lists of devices and cameras as UDM object attributes 123 | self.devices = self._getDevices() 124 | self.cameras = self._getCameras(camera_json_filename) 125 | 126 | # Return a list of Device objects, one for each network device that the UDM manages 127 | def _getDevices(self): 128 | 129 | # Get the list of network devices known to this UDM 130 | logging.debug('Getting network device list from UDM Network Console app') 131 | _device_list = [] 132 | _r = self.session.get(self.url + '/proxy/network/api/s/default/stat/sta', verify=False) 133 | if _r.status_code == 200: 134 | for device in _r.json()['data']: 135 | _device_list.append(Device(self.session, self.url, device)) 136 | logging.debug('Network client devices:\n{}'.format(json.dumps(_r.json()['data'], indent=4))) 137 | else: 138 | logging.error('The request for network client information failed with status code = {}'.format(_r.status_code)) 139 | if _r.text != '': 140 | logging.error('Response text: {}'.format(_r.text)) 141 | 142 | return _device_list 143 | 144 | # Return a list of Camera objects, one for each camera that the UDM manages 145 | def _getCameras(self, camera_json_filename): 146 | 147 | # When a camera is powered off for a period of time, the UDM Protect application 148 | # forgets it. In order to remember cameras that are powered off, we save the camera 149 | # info in a file. Cameras in the file are matched with those that Protect currently 150 | # knows about using the camera's MAC addresses. If a camera is permanently removed 151 | # from the network, it is possible that it remains in our camera file. We don't 152 | # think this will cause any problems but who knows? 153 | 154 | # Read in the camera list file, if it exists 155 | _camera_json = [] 156 | try: 157 | if os.path.exists(camera_json_filename): 158 | logging.debug('Reading existing file {}'.format(camera_json_filename)) 159 | with open(camera_json_filename, 'r') as _f: 160 | _camera_json = json.load(_f) 161 | except: 162 | logging.exception('Unexpected exception', exc_info=True) 163 | 164 | # Get the information for all of the cameras Protect currently knows about 165 | logging.debug('Getting list of cameras from UDM Protect app') 166 | _r = self.session.get(self.url + '/proxy/protect/api/bootstrap', verify=False) 167 | if _r.status_code != 200: 168 | logging.error('The request for camera information failed with status code = {}'.format(_r.status_code)) 169 | if _r.text != '': 170 | logging.error('Response text: {}'.format(_r.text)) 171 | return 172 | for _camera in _r.json()['cameras']: 173 | _found = False 174 | _camera['switch_mac'] = '' 175 | _camera['switch_port'] = '' 176 | for _c in _camera_json: 177 | if _c['name'].upper() == _camera['name'].upper(): 178 | _found = True 179 | logging.debug('Updating existing camera in the list: {}'.format(_camera['name'])) 180 | _c = _camera 181 | _c['mac'] = _camera['mac'].upper() 182 | break 183 | if not _found: 184 | logging.debug('Adding new camera to the list: {}'.format(_camera['name'])) 185 | _camera['mac'] = _camera['mac'].upper() 186 | _camera_json.append(_camera) 187 | 188 | # Update the camera list with the MAC address of the switches and the 189 | # ports on those switches that the cameras are connected to. 190 | for _device in self.devices: 191 | for _c in _camera_json: 192 | if _device.mac == _c['mac'] and _device.sw_mac and _device.sw_port: 193 | try: 194 | _c['switch_mac'] = _device.sw_mac 195 | except: 196 | pass 197 | try: 198 | _c['switch_port'] = _device.sw_port 199 | except: 200 | pass 201 | logging.debug('Updating network switch MAC and port number for {} camera\nMAC: {} Port: {}'.format(_c['name'], _c['switch_mac'], _c['switch_port'])) 202 | break 203 | 204 | # Overwrite the camera list file 205 | logging.debug('Writing camera details file {}'.format(camera_json_filename)) 206 | try: 207 | with open(camera_json_filename, 'w') as _f: 208 | json.dump(_camera_json, _f) 209 | except: 210 | logging.exception('Unexpected exception', exc_info=True) 211 | logging.debug('Camera details:\n{}'.format(json.dumps(_camera_json, indent=4))) 212 | 213 | # Return the camera list as a list of camera objects 214 | _camera_list = [] 215 | for camera in _camera_json: 216 | _camera_list.append(Camera(self.session, self.url, camera)) 217 | return _camera_list 218 | 219 | # Define object classes for network devices and cameras 220 | class Device: 221 | def __init__(self, session, url, properties): 222 | self.session = session 223 | self.url = url 224 | self._properties = properties 225 | self.mac = properties['mac'].replace(":","").upper() 226 | try: 227 | self.sw_mac = properties['sw_mac'].replace(":","").upper() 228 | except: 229 | logging.debug('Device with MAC = {} does not have a "sw_mac" attribute.'.format(self.mac)) 230 | self.sw_mac = None 231 | try: 232 | self.sw_port = properties['sw_port'] 233 | except: 234 | logging.debug('Device with MAC = {} does not have a "sw_port" attribute.'.format(self.mac)) 235 | self.sw_port = None 236 | 237 | class Camera: 238 | def __init__(self, session, url, properties): 239 | self.session = session 240 | self.url = url 241 | self._properties = properties 242 | self.name = properties['name'] 243 | self.mac = properties['mac'].replace(":","").upper() 244 | self.lastseen = properties['lastSeen'] 245 | self.switch_mac = properties['switch_mac'].replace(":","").upper() 246 | self.switch_port = properties['switch_port'] 247 | 248 | # Return the current POE status of the switch port this camera instance is 249 | # plugged into. Returns True for "ON" and False for "OFF" 250 | def getPOE(self): 251 | logging.debug('Searching for the switch this camera is connected to, MAC = {}'.format(self.switch_mac)) 252 | # Get the switch configuration details 253 | _r = self.session.get(self.url + '/proxy/network/api/s/default/stat/device', verify=False) 254 | if _r.status_code != 200: 255 | logging.error('The request for switch configuration information failed with status code = {}'.format(_r.status_code)) 256 | if _r.text != '': 257 | logging.error('Response text: {}'.format(_r.text)) 258 | return 259 | # Find the switch this camera is connected to 260 | for _device in _r.json()['data']: 261 | _mac = _device['mac'].replace(":","").upper() 262 | if _mac == self.switch_mac: 263 | # Find the port on this switch the camera is connected to 264 | logging.debug('Found the switch') 265 | logging.debug('Searching for the port this camera is plugged into, port #: {}'.format(self.switch_port)) 266 | for _port in _device['port_table']: 267 | if _port['port_idx'] == self.switch_port: 268 | logging.debug('Found the port for {} camera. Port details:\n{}'.format(self.name, json.dumps(_port, indent=4))) 269 | try: 270 | # Add the POE status and device_id as attributes 271 | self.switch_id = _device['_id'] 272 | self.poe_status = _port['poe_enable'] 273 | 274 | # Prepopulate the POST data structure we'll need to 275 | # include if we send a PUT request to change the POE 276 | # mode on this camera's switch port 277 | self.port_override = { 278 | 'autoneg': _port['autoneg'], 279 | 'egress_rate_limit_kbps_enabled': _port['egress_rate_limit_kbps_enabled'], 280 | 'forward': _port['forward'], 281 | 'full_duplex': _port['full_duplex'], 282 | 'isolation': _port['isolation'], 283 | 'name': _port['name'], 284 | 'native_networkconf_id': _port['native_networkconf_id'], 285 | 'op_mode': _port['op_mode'], 286 | 'port_idx': _port['port_idx'], 287 | 'port_poe': _port['port_poe'], 288 | 'port_security_enabled': False, 289 | 'port_security_mac_address': _port['port_security_mac_address'], 290 | 'setting_preference': _port['setting_preference'], 291 | 'show_traffic_restriction_as_allowlist': False, 292 | 'speed': _port['speed'] 293 | } 294 | 295 | # Return the current POE status 296 | return self.poe_status 297 | except KeyError as e: 298 | logging.exception('Missing port attribute: {}'.format(e), exc_info=True) 299 | return 300 | except Exception as e: 301 | logging.exception('Unexpected exception: {}'.format(e), exc_info=True) 302 | return 303 | 304 | # Set the POE "ON" or "OFF" on the switch port this camera instance is plugged into. 305 | # Will check to see what state the port is currently in and will not send a 306 | # request if the port is already in the desired state. 307 | def setPOE(self, desired_state): 308 | 309 | # Flag that indicates whether or not a PUT request needs to be sent 310 | _request_pending = False 311 | 312 | # Get the current POE state from an object attribute, or by querying 313 | # the switch. True = "ON", False = "OFF" 314 | logging.debug('Getting current state of {} camera'.format(self.name)) 315 | try: 316 | _current_state = self.poe_status 317 | logging.debug('Current state: {}, from object property'.format(_current_state)) 318 | except: 319 | _current_state = self.getPOE() 320 | logging.debug('Current state: {}, from getPOE method'.format(_current_state)) 321 | 322 | # If the POE state needs to be changed, prepare the PUT data 323 | if _current_state and desired_state == 'OFF': 324 | logging.debug('{} camera is currently on, turning it off'.format(self.name)) 325 | self.port_override['poe_mode'] = 'off' 326 | _request_pending = True 327 | elif not _current_state and desired_state == 'ON': 328 | logging.debug('{} camera is currently off, turning it on'.format(self.name)) 329 | self.port_override['poe_mode'] = 'auto' 330 | _request_pending = True 331 | else: 332 | logging.debug('{} camera is already in the desired state, doing nothing'.format(self.name)) 333 | 334 | # Send a PUT request to change the POE state 335 | if _request_pending: 336 | _data = { 337 | 'port_overrides': [self.port_override] 338 | } 339 | logging.debug('Sending POE request for {} camera:\n{}'.format(self.name, json.dumps(_data, indent=4))) 340 | _r = self.session.put(self.url + '/proxy/network/api/s/default/rest/device/' + self.switch_id, json=_data, verify=False) 341 | if _r.status_code != 200: 342 | logging.error('POE {} request failed with status code = {}'.format(desired_state, _r.status_code)) 343 | logging.error('Response text: {}'.format(_r.text)) 344 | return False 345 | else: 346 | logging.info('{} camera powered {}'.format(self.name, desired_state.lower())) 347 | return True 348 | 349 | return False 350 | -------------------------------------------------------------------------------- /deprecated/away-recording/README.md: -------------------------------------------------------------------------------- 1 | # Cameras Only Record When I'm Away 2 | 3 | This script integrates Unifi Protect with my SmartThings service so that I only record camera events when I'm away from home or asleep. SmartThings uses my cell phone as a sensor to determine whether or not I'm at home or away. The ```away-recording``` script in this repository uses the UDM API to turn off event recording on some of my cameras during the day when SmartThings says I'm at home. At night, or whenever I am away, recording is turned on. 4 | 5 | ## Installation 6 | 7 | I wanted to run this script on the UDM itself so it is written for the version of Python available in the UDM's unifi-os container, Python 2.7. It can be run elsewhere, but it may need to be modified if elsewhere uses a later version of Python. 8 | 9 | Here is the process I went through to install the script on the UDM: 10 | 11 | 1. I already have John D (boostchicken)'s [on-boot-script](https://github.com/boostchicken/udm-utilities/tree/master/on-boot-script) utility installed. If you don't, I highly recommend it. You'll need it, or something very much like it, down below. 12 | 2. Modify the ```away-recording``` script to include your UDM login credentials, your SmartThings API token and the list of cameras you want to control. 13 | 3. Install the ```away-recording``` script into the ```/mnt/data/unifi-os/dlk-udm-hacks/``` directory on your UDM. Make sure that the file has execute permissions for the root id. (If you install the script in a different subdirectory underneath /mnt/data/unifi-os, be sure to make the corresponding change in the ```20-away-recording.sh``` boot script that is mentioned below.) 14 | 4. Access the unifi-os container's shell with the command: ```unifi-os shell``` 15 | 5. Install the Python requests module: ```apt install python-requests```
This install will persist through system reboots but will need to be repeated whenever Ubiquiti updates the network controller, i.e., installs a new version of the unifi-os container, on your UDM. 16 | 6. Press Ctrl-D to exit the unifi-os container's shell and return to the base operating system's shell prompt. 17 | 7. Install the ```50-cronjobs.sh``` script from this repository in your /mnt/data/on-boot.d/ directory. Make sure that the script has execute file permissions for the root id. Every time your UDM boots, this script will create a crontab configuration file that will run the away-recording script every 10 minutes to check your status and that of your cameras. If you want to change this frequency, you can edit this script to make that modification. Run the script manually once after you modify it to put the changes you've made into effect. 18 | 8. If you want to test the script, run the command
```podman exec -it unifi-os /data/dlk-udm-hacks/away-recording``` 19 | -------------------------------------------------------------------------------- /deprecated/away-recording/away-recording: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Copyright (C) 2021 David King 4 | # 5 | # This Source Code Form is subject to the terms of the Mozilla Public License, 6 | # v. 2.0. If a copy of the MPL was not distbuted with this file, You can 7 | # obtain one at https://mozilla.org/MPL/2.0/. 8 | 9 | # I don't want to record on my indoor cameras during the day when I'm at home. 10 | # SmartThings uses my cell phone as a presence sensor and can tell when I am away 11 | # from home. 12 | 13 | ############################################################################################################# 14 | # C o n f i g u r a t i o n S e c t i o n 15 | ############################################################################################################# 16 | 17 | # I want recording turned off between the hours of 7am and 11pm when I'm at home. At night, and whan 18 | # I am away, recording will be turned on. 19 | # Set these variables to 0 and 24 to ignore the time element and only record when you are away. 20 | start_off_hour = 7 21 | end_off_hour = 23 22 | 23 | # The URL of the Unifi Dream Machine that manages the cameras. Use "https://localhost" 24 | # when running this script on the UDM itself. 25 | udm_url = 'https://localhost' 26 | 27 | # My login credentials for the Unifi Dream Machine 28 | udm_username = '' 29 | udm_password = '' 30 | 31 | # My SmartThings authetication token, from https://account.smartthings.com/tokens 32 | smartthings_token = '' 33 | 34 | # The name of the SmartThings location to check for away/home status, from the SmartThings app 35 | smartthings_location_name = 'Home' 36 | 37 | # The List of Unifi cameras to be contolled. 38 | # The names are the names they've been given in the Unifi Protect app. 39 | # The recording mode is the mode that they should use when they are set to record. 40 | # Possible choices are: "always" and "detections". The "never" mode is used to turn off recording. 41 | camera_list = [ 42 | { 43 | 'name': 'Living Room', 44 | 'recording_mode': 'detections' 45 | }, 46 | { 47 | 'name': 'Downstairs', 48 | 'recording_mode': 'detections' 49 | } 50 | ] 51 | 52 | ############################################################################################################# 53 | ############################################################################################################# 54 | 55 | import datetime 56 | from optparse import OptionParser 57 | import json 58 | import base64 59 | import datetime 60 | 61 | # Write messages to the console and log file 62 | # Later Python versions may require that the print statement used in this function be modified. 63 | def log(message): 64 | global opts 65 | timestamp = datetime.datetime.now().strftime('%Y%m%d %H:%M:%S') 66 | if opts.logfile: 67 | with open(opts.logfile, 'a') as f: 68 | f.write('{} {}\n'.format(timestamp, message)) 69 | if not opts.quiet: 70 | print('{} {}'.format(timestamp, message)) 71 | 72 | # Return true if the SmatThings location mode is "away". Requires the location 73 | # name and a SmartThings API token. 74 | def smartthingsAway(location_name, api_token): 75 | 76 | smartthings_url = 'https://api.smartthings.com/v1/' 77 | 78 | headers = { 79 | 'Authorization': 'Bearer ' + api_token 80 | } 81 | 82 | # Get the locationId for the location with the specified name 83 | locationId = '' 84 | r = requests.get(smartthings_url + 'locations', headers=headers) 85 | if r.status_code != 200: 86 | log(r.text) 87 | exit(r.status_code) 88 | for item in r.json()['items']: 89 | if item['name'] == location_name: 90 | locationId = item['locationId'] 91 | if locationId == '': 92 | log('Unable to get the locationId for the location named "{}"'.format(location_name)) 93 | exit(1) 94 | 95 | # Get the "mode" at that location: Home, Away, Night 96 | location_mode = '' 97 | r = requests.get(smartthings_url + 'locations/' + locationId + '/modes/current', headers=headers) 98 | if r.status_code != 200: 99 | log(r.text) 100 | exit(r.status_code) 101 | location_mode = r.json()['name'] 102 | if location_mode == '': 103 | log('Unable to get the curreent mode for the location named "{}"'.format(location_name)) 104 | exit(1) 105 | 106 | log('SmartThings reports that the "{}" location is in "{}" mode'.format(location_name, location_mode)) 107 | 108 | if location_mode == 'Away': 109 | return True 110 | return False 111 | 112 | # Set recording "on" or "off". Requires UDM access details, the list of cameras being controlled 113 | # and the action to be taken, i.e., either "off" or "on" 114 | def setRecording(udm_url, udm_username, udm_password, camera_list, action): 115 | 116 | # Login to UDM 117 | data = { 118 | 'username': udm_username, 119 | 'password': udm_password, 120 | 'rememberMe': False 121 | } 122 | r = requests.post(udm_url + '/api/auth/login', json=data, verify=False) 123 | if r.status_code != 200: 124 | log('ABORT: The login failed: ' + r.text) 125 | exit(r.status_code) 126 | 127 | # Set authorization cookies for all subsequent requests 128 | for cookie in r.headers['Set-Cookie'].split(';'): 129 | if cookie.startswith('TOKEN='): 130 | headers = { 131 | 'Content-Type': 'application/json; charset=utf-8', 132 | 'Cookie': cookie + ';', 133 | } 134 | # If the token cookie contains a x-csrf-token value, this must be decoded 135 | # and passed in the request headers otherwise any attempt to change any 136 | # settings will fail with a "404 Not Found" error. 137 | decoded_token = json.loads(base64.b64decode(cookie.split('=')[1].split('.')[1] + '===')) 138 | if 'csrfToken' in decoded_token: 139 | headers['x-csrf-token'] = decoded_token['csrfToken'] 140 | 141 | # Get the information for all of the cameras this UDM knows about 142 | r = requests.get(udm_url + '/proxy/protect/api/bootstrap', headers=headers, verify=False) 143 | if r.status_code != 200: 144 | log('ABORT: The request for camera information failed: ' + r.text) 145 | exit(r.status_code) 146 | 147 | # Find the cameras we want to control 148 | for camera in camera_list: 149 | 150 | log('Request is to turn {} recording for the "{}" camera'.format(action, camera['name'])) 151 | 152 | if action == 'off': 153 | desired_mode = 'never' 154 | else: 155 | desired_mode = camera['recording_mode'] 156 | 157 | found = False 158 | for unifi_camera in r.json()['cameras']: 159 | if unifi_camera['name'] == camera['name']: 160 | found = True 161 | # If the camera is not already in the desired mode, then make it so 162 | if unifi_camera['recordingSettings']['mode'] == desired_mode: 163 | log('"{}" camera is already in "{}" mode, nothing to be done'.format(camera['name'], unifi_camera['recordingSettings']['mode'])) 164 | else: 165 | log('"{}" camera is in "{}" mode, attempting to change to "{}" mode'.format(camera['name'], unifi_camera['recordingSettings']['mode'], desired_mode)) 166 | # Populate the request payload with the current settings for this camera 167 | # except for the recording mode. which we'll be changing. 168 | data = { 169 | 'channels': unifi_camera['channels'], 170 | 'recordingSettings': { 171 | "mode": desired_mode, 172 | "prePaddingSecs": unifi_camera['recordingSettings']['prePaddingSecs'], 173 | "postPaddingSecs": unifi_camera['recordingSettings']['postPaddingSecs'], 174 | "minMotionEventTrigger": unifi_camera['recordingSettings']['minMotionEventTrigger'], 175 | "enablePirTimelapse": unifi_camera['recordingSettings']['enablePirTimelapse'], 176 | "motionAlgorithm": unifi_camera['recordingSettings']['motionAlgorithm'] 177 | }, 178 | 'smartDetectSettings': unifi_camera['smartDetectSettings'] 179 | } 180 | result = requests.patch(udm_url + '/proxy/protect/api/cameras/' + unifi_camera['id'], json=data, headers=headers, verify=False) 181 | if result.status_code != 200: 182 | log('ABORT: The request to change "{}" camera\'s recording mode failed: {}'.format(camera['name'], result.text)) 183 | exit(result.status_code) 184 | 185 | if not found: 186 | log('Unable to find a camera named "{}"'.format(camera['name'])) 187 | 188 | # Define command-line options to turn off console output and to log progress into a file 189 | cmdline = OptionParser(usage="usage: %prog [options]", description='Set security camera recording mode depending on SmartThings location mode and time of day') 190 | cmdline.add_option('-q', '--quiet', action='store_true', dest='quiet', default=False, help='Do not print any log messages on the console') 191 | cmdline.add_option('-l', '--log', dest='logfile', metavar='FILE', help='Write log messages into this file') 192 | opts, args = cmdline.parse_args() 193 | 194 | # Handle error that occurs if the requests module is not installed 195 | try: 196 | import requests 197 | 198 | # Turn off warnings about the use of self-signed SSL certificates. This code may 199 | # need to be modified if you are using a later version of the requests module. 200 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 201 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 202 | except: 203 | log('ABORT: Unable to import the Python "requests" module. Is it installed?') 204 | exit(1) 205 | 206 | # Figure out whether or not we're in the time window during which recording should be 207 | # turned off. 208 | hour = datetime.datetime.now().hour 209 | turn_off_window = False 210 | if not start_off_hour == end_off_hour: 211 | if start_off_hour < end_off_hour: 212 | if hour >= start_off_hour and hour < end_off_hour: 213 | turn_off_window = True 214 | else: 215 | if hour >= start_off_hour or hour < end_off_hour: 216 | turn_off_window = True 217 | 218 | # If we're not in the time window, or I am away, turn on recording. Otherwise turn it off. 219 | if not turn_off_window or smartthingsAway(smartthings_location_name, smartthings_token): 220 | setRecording(udm_url, udm_username, udm_password, camera_list, 'on') 221 | else: 222 | setRecording(udm_url, udm_username, udm_password, camera_list, 'off') 223 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/Docker.com-README.md: -------------------------------------------------------------------------------- 1 | # Running ntopng On The Unifi Dream Machine (UDM) 2 | 3 | This container image can be used to run the ntopng network traffic analysis tool on a Unifi Dream Machine router/switch. 4 | 5 | This project was inspired by Carlos Talbot's [tusc/ntopng-udm project](https://github.com/tusc/ntopng-udm). This container differs in the following ways: 6 | 7 | 1. A Fedora container is used, because I like being on the semi-bleeding edge. 8 | 2. The unstable development version of ntopng from GitHub is used in this container. This container is rebuilt weekly, on Monday night. 9 | 3. Network Discovery is enabled, i.e., the libcap library is compiled in. 10 | 4. An up-to-date version of the geoipupdate tool is used. 11 | 5. The start-up process for the container has been modified so that startup will not loop endlessly, blocking startup, when the GeoIP.conf file has errors. 12 | 6. An `ethtool` command was added to the startup process to turn off oversized packet handling per the ntopng log. 13 | 7. The run command in this documentation has been fixed so that the ntopng configuration data stored in the redis DB is actually persisted through UDM updates and restarts. 14 | 8. The run command in this documentation has been updated to include the `--privileged` option. This gives the commands running inside the container the same access to system devices and files they would have if they were running natively on the UDM, under the root userid. This fixes a couple of errors seen in the logs. 15 | 16 | # Running The Container On The UDM 17 | 18 | ## Creating the ntopng persistent data directories 19 | 20 | Access the UDM as the root user via ssh. You will be placed into the system's root directory. The runtime environment for the container can be established by entering the following commands. 21 | ``` 22 | mkdir -p /mnt/data/ntopng/redis 23 | mkdir -p /mnt/data/ntopng/lib 24 | curl -Lo /mnt/data/ntopng/GeoIP.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/GeoIP.conf 25 | curl -Lo /mnt/data/ntopng/ntopng.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/ntopng.conf 26 | curl -Lo /mnt/data/ntopng/redis.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/redis.conf 27 | ``` 28 | If the secondary disk is installed in the UDM then `/mnt/data_ext` can be used as the root for these directories to make it so that the application data is stored on the secondary disk. 29 | 30 | ## Enabling GeoIP support 31 | 32 | If you want GeoIP support, i.e., you want geo location information for the hosts that are sending and receiving traffic, then follow [these instructions](https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md) on the ntopng web site to create a free account at MaxMind and to register for an API key. The account and API key values must be placed into the /mnt/data/ntopng/GeoIP.conf file. 33 | 34 | ## Installing The Container 35 | 36 | Run this command to copy the container from the Docker repository onto the UDM: 37 | 38 | `podman pull dlk3/ntopng-udm:latest` 39 | 40 | ## Running the container 41 | 42 | Run this command to start the container: 43 | ``` 44 | podman run -d --net=host --privileged --restart always --name ntopng \ 45 | -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \ 46 | -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \ 47 | -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \ 48 | -v /mnt/data/ntopng/lib:/var/lib/ntopng \ 49 | -v /mnt/data/ntopng/redis:/var/lib/redis \ 50 | -v /etc/localtime:/etc/localtime:ro \ 51 | ntopng-udm:latest 52 | ``` 53 | Remember to use `/mnt/data_ext` as the root for the directories if you are keeping the application data on the UDM's secondary disk. 54 | 55 | To temporarily stop the ntopng container: 56 | 57 | `podman stop ntopng` 58 | 59 | To restart the ntopng container: 60 | 61 | `podman start ntopng` 62 | 63 | You will need to do the start command each time the UDM is rebooted or upgraded. (The ntopng-udm container and its data will survive a UDM firmward update.) To perform this task automatically at boot time take a look at John D's [on-boot-script project](https://github.com/boostchicken/udm-utilities/tree/master/on-boot-script) on GitHub for a tool that can help you do that. 64 | 65 | ## Accessing ntopng 66 | 67 | Point your browser to https://YOUR.UDM.IP.ADDRESS:3001, for example: https://192.168.1.1:3001 68 | 69 | In this container ntopng is configured with a self-signed SSL certificate for HTTPS security. Your browser may ask you to accept this certificate when you access ntopng. 70 | 71 | The ntopng documentation can be found [here](https://www.ntop.org/guides/ntopng/). Note that this container runs the Community Edition of ntopng. 72 | 73 | ## Viewing the ntopng log 74 | 75 | ``` 76 | podman logs ntopng 77 | ``` 78 | 79 | This command, when entered at the UDM's command prompt, will display the contents of the podman container's console log. This will include any messages from the ntopng startup process and can be useful when troublshooting startup problems. 80 | 81 | ## Upgrading the container 82 | 83 | Use these commands to upgrade the ntopng-udm to the latest release: 84 | ``` 85 | podman stop ntopng 86 | podman rm ntopng 87 | podman rmi ntopng-udm 88 | podman pull dlk3/ntopng-udm:latest 89 | podman run -d --net=host --privileged --restart always --name ntopng \ 90 | -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \ 91 | -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \ 92 | -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \ 93 | -v /mnt/data/ntopng/lib:/var/lib/ntopng \ 94 | -v /mnt/data/ntopng/redis:/var/lib/redis \ 95 | -v /etc/localtime:/etc/localtime:ro \ 96 | ntopng-udm:latest 97 | ``` 98 | Remember to use `/mnt/data_ext` as the root for the directories if you are keeping the application data on the UDM's secondary disk. 99 | 100 | ## Uninstalling the container and all of its data 101 | 102 | Use these commands to completely uninstall the ntopng-udm container and all of its data: 103 | ``` 104 | podman stop ntopng 105 | podman rm ntopng 106 | podman rmi ntopng-udm 107 | rm -fr /mnt/data/ntopng 108 | ``` 109 | If you have been keeping the application data on the UDM's secondary disk, then use the command `rm -fr /mnt/data_ext/ntopng` as the final step in this process. 110 | 111 | ## What to do when you forget your ntopng password 112 | 113 | 1. Access the command shell inside the ntopng container with this command:

`podman exec -it ntopng /bin/sh` 114 | 115 | 2. Issue this command to delete the admin password from the redis db:

`redis-cli del ntopng.user.admin.password` 116 | 117 | 3. Exit the ntopng container shell with the `exit` command or by pressing Ctrl-D. 118 | 119 | 4. Restart the container with this command:

`podman restart ntopng` 120 | 121 | 5. When you reconnect your browser you will be able to login using the default `admin` userid and `admin` password, which you will promptly be prompted to change. 122 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM docker.io/arm64v8/fedora:latest 2 | 3 | COPY ntopng-*.aarch64.rpm /tmp/ 4 | RUN dnf -y update && \ 5 | dnf -y install dnf-plugins-core ethtool geoipupdate iproute && \ 6 | dnf -y install /tmp/ntopng* && \ 7 | dnf -y clean all && \ 8 | rm /tmp/ntopng* 9 | 10 | EXPOSE 3000/tcp 11 | EXPOSE 3001/tcp 12 | 13 | COPY entrypoint.sh /usr/local/bin/entrypoint.sh 14 | 15 | ENTRYPOINT entrypoint.sh 16 | 17 | LABEL maintainer="David King " 18 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2021 by David King 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. 20 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/README.md: -------------------------------------------------------------------------------- 1 | # Building A ntopng Docker Container For The Unifi Dream Machine (UDM) 2 | 3 | This repository contains the source material I use to build a container image that can be used to run the ntopng network traffic analysis tool on a Unifi Dream Machine router/switch. 4 | 5 | Please see the [Docker container registry](https://hub.docker.com/repository/docker/dlk3/ntopng-udm) for information on installing and using the ntopng-udm container. 6 | 7 | This project was inspired by Carlos Talbot's [tusc/ntopng-udm project](https://github.com/tusc/ntopng-udm). This container differs in the following ways: 8 | 9 | 1. A Fedora container is used, because I like being on the semi-bleeding edge. 10 | 2. The unstable development version of ntopng from GitHub is used in this container. This container is rebuilt weekly, on Monday night. 11 | 3. Network Discovery is enabled, i.e., the libcap library is compiled in. 12 | 4. An up-to-date version of the geoipupdate tool is used. 13 | 5. The start-up process for the container has been modified so that startup will not loop endlessly, blocking startup, when the GeoIP.conf file has errors. 14 | 6. An `ethtool` command was added to the startup process to turn off oversized packet handling per the ntopng log. 15 | 7. The run command in this documentation has been fixed so that the ntopng configuration data stored in the redis DB is actually persisted through UDM updates and restarts. 16 | 8. The run command in this documentation has been updated to include the `--privileged` option. This gives the commands running inside the container the same access to system devices and files they would have if they were running natively on the UDM, under the root userid. This fixes a couple of errors seen in the logs. 17 | 18 | # Automated Build Process 19 | 20 | I run the build-ntopng-docker-image script that's in this project weekly on my local workstation to automate the build of a new RPM with a new unstable release of ntopng from [GitHub](https://github.com/ntop/ntopng), followed by the build and push of a new version of the ARM64 container for the UDM. 21 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/TESTING.md: -------------------------------------------------------------------------------- 1 | # Building And Testing The Container Locally 2 | 3 | This section outlines the process I use to build and test the container on a x86_64 workstation, before I build it for the UDM which is an arm64 system. 4 | 5 | ## Building the ntopng RPM package 6 | 7 | Modify the build-ntopng-docker-image script that's in this project and set TESTBUILD to "true". Run the script. The ntopng RPM will be created and store in the ${HOME}/rpmbuild/RPMS/x86_64 directory. Move this RPM to the directory within this project that contains the Dockerfile. 8 | 9 | ## Building the container 10 | 11 | A Dockerfile is provided in this project that builds the container once the RPM is available. It can be built on a x86_64 workstation for testing purposes using one of the follwing commands, from within the folder which contains the Dockerfile: 12 | 13 | ``` 14 | sudo podman build -t ntopng-udm:latest . 15 | sudo docker build -t ntopng-udm:latest . 16 | ``` 17 | 18 | ## Creating the ntopng persistent data directories for local testing 19 | 20 | The directories and files needed for local testing of the container can be created with the following commands: 21 | 22 | ``` 23 | mkdir -p /tmp/ntopng/redis 24 | mkdir -p /tmp/ntopng/lib 25 | curl -Lo /tmp/ntopng/GeoIP.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/GeoIP.conf 26 | curl -Lo /tmp/ntopng/ntopng.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/ntopng.conf 27 | curl -Lo /tmp/ntopng/redis.conf https://github.com/dlk3/udm-hacks/raw/master/ntopng-udm-fedora/data/ntopng/redis.conf 28 | ``` 29 | 30 | You will need to change the network interface specified in /tmp/data/ntopng.conf to match the local system. 31 | 32 | ## Enabling GeoIP support 33 | 34 | If you want GeoIP support, i.e., you want geo location data for the hosts sending and receiving traffic, then follow [these instructions](https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md) on the ntopng web site to create a free account at MaxMind and to register for an API key. The account and API key values must be placed into the /tmp/ntopng/GeoIP.conf file. 35 | 36 | ## Running the container 37 | 38 | This command starts ntopng running within its container: 39 | ``` 40 | sudo podman run -d --net=host --privileged --restart always --name ntopng \ 41 | -v /tmp/ntopng/GeoIP.conf:/etc/GeoIP.conf \ 42 | -v /tmp/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \ 43 | -v /tmp/ntopng/redis.conf:/etc/redis/redis.conf \ 44 | -v /tmp/ntopng/lib:/var/lib/ntopng \ 45 | -v /tmp/ntopng/redis:/var/lib/redis \ 46 | -v /etc/localtime:/etc/localtime:ro \ 47 | ntopng-udm:latest 48 | ``` 49 | 50 | ## Accessing ntopng 51 | 52 | Point your browser to https://localhost:3001 53 | 54 | I have configured ntopng with a self-signed SSL certificate for HTTPS access. Your browser will ask you to accept this certificate the first time you access a new ntopng container. 55 | 56 | The ntopng documentation can be found [here](https://www.ntop.org/guides/ntopng/). Note that this container runs the Community Edition of ntopng. 57 | 58 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/build-ntopng-docker-image: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | # This Source Code Form is subject to the terms of the Mozilla Public 4 | # License, v. 2.0. If a copy of the MPL was not distributed with this 5 | # file, You can obtain one at https://mozilla.org/MPL/2.0/. 6 | 7 | # Run in interactive containers with a /bin/sh trap on failure 8 | DEBUG=false 9 | 10 | # Build a stable branch. Must also modify the version and release variables 11 | # and the CHANGELOG in the SPEC file, then push it to github. 12 | #STABLE='5.6' 13 | 14 | # Name of the Fedora container we use as the base for our container build 15 | FEDORA_CONTAINER="arm64v8/fedora:latest" 16 | 17 | # Keep the container running and give me a shell prompt when something bad happens 18 | if $DEBUG; then 19 | trap /bin/sh ERR 20 | fi 21 | 22 | # Abend the script and execute the trap when something bad happens 23 | set -e 24 | 25 | if [ $(cat /etc/hostname) == "fang.localdomain" ]; then 26 | 27 | # This section of the script does not run in the podman container, it only 28 | # runs on the container's host and drives the overall build process 29 | 30 | # Set to "true" to test the RPM package build process in an interactive 31 | # VM. The package will not be submitted to the docker repository. 32 | # 33 | # Set to false to build the RPM and submit it to docker.com. 34 | # 35 | # Need to have installed the QEMU emulation package: qemu-user-static 36 | TESTBUILD=false 37 | 38 | find ${HOME}/rpmbuild/SRPMS -iname ntopng-\* -delete 39 | find ${HOME}/rpmbuild/RPMS -iname ntopng-\* -delete 40 | cd $(dirname $(realpath "$0")) 41 | 42 | sudo systemctl start docker 43 | if $TESTBUILD; then 44 | # Build the RPMs 45 | echo "Starting RPM build at $(date +%H:%m:%S)" 46 | OPT="" 47 | if $DEBUG; then 48 | OPT="-it" 49 | fi 50 | /usr/bin/docker run $OPT --platform=linux/arm64 --name=rpmbuild --rm --volume=${HOME}:${HOME} docker.io/${FEDORA_CONTAINER} "$(realpath "$0")" ${HOME} 51 | echo "RPM build complete at $(date +%H:%m:%S)" 52 | echo "That took $(elapsed $STARTTIME)" 53 | else 54 | 55 | # Make sure we know the docker password 56 | if [ -z "$DOCKER_PASSWORD" ]; then 57 | echo "Please set the DOCKER_PASSWORD environment variable before running this script:" 58 | echo " $ DOCKER_PASSWORD='whatever' $0" 59 | exit 1 60 | fi 61 | 62 | # Build the RPMs 63 | STARTTIME=$(date +%s) 64 | echo "Starting RPM build at $(date +%H:%M:%S)" 65 | OPT="" 66 | if $DEBUG; then 67 | OPT="-it" 68 | fi 69 | /usr/bin/docker run $OPT --platform=linux/arm64 --name=rpmbuild --rm --volume=${HOME}:${HOME} docker.io/${FEDORA_CONTAINER} "$(realpath "$0")" ${HOME} 70 | echo "RPM build complete at $(date +%H:%M:%S)" 71 | echo "That took $(/usr/local/bin/elapsed $STARTTIME)" 72 | 73 | # Just for the hell of it, send the SRPM to COPR to be built there 74 | echo "Submitting source RPM for building on COPR" 75 | /usr/bin/copr build --nowait dlk/rpms ${HOME}/rpmbuild/SRPMS/ntopng* || true 76 | 77 | # Build the container 78 | echo "Starting container build at $(date +%H:%M:%S)" 79 | find -iname ntopng-\*.rpm -delete 80 | cp ${HOME}/rpmbuild/SRPMS/ntopng* . 81 | cp ${HOME}/rpmbuild/RPMS/aarch64/ntopng*.aarch64.rpm . 82 | VERSION=$( ls ntopng*.aarch64.rpm | sed 's/"//g;s/ntopng-\([^-]*\)-.*$/\1/' ) 83 | /usr/bin/docker buildx build $OPT --no-cache --platform=linux/arm64 -t docker.io/dlk3/ntopng-udm:$VERSION -t docker.io/dlk3/ntopng-udm:latest . 84 | if [ ! -z "$STABLE" ]; then 85 | /usr/bin/docker tag dlk3/ntopng-udm:latest docker.io/dlk3/ntopng-udm:stable 86 | fi 87 | /usr/bin/docker rmi ${FEDORA_CONTAINER} 88 | 89 | # Push the images 90 | echo "$DOCKER_PASSWORD" | /usr/bin/docker login --username dlk3 --password-stdin docker.io 91 | /usr/bin/docker push docker.io/dlk3/ntopng-udm:latest 92 | /usr/bin/docker push docker.io/dlk3/ntopng-udm:$VERSION 93 | if [ ! -z "$STABLE" ]; then 94 | /usr/bin/docker push docker.io/dlk3/ntopng-udm:stable 95 | fi 96 | /usr/bin/docker logout docker.io 97 | 98 | # Clean up 99 | /usr/bin/docker image prune --force 100 | echo "Container build complete at $(date +%H:%M:%S)" 101 | echo "The entire process took $(/usr/local/bin/elapsed $STARTTIME)" 102 | fi 103 | sudo systemctl stop docker 104 | 105 | else 106 | 107 | # This section of the script runs in the docker container to build the ntopng 108 | # RPM packages. 109 | 110 | # The resulting packages will be copied to the $HOME directory of the user 111 | # running this script on the host computer. That user needs to have the 112 | # ~/rpmbuild directory tree set up for this to work. 113 | TARGETDIR=$1 114 | 115 | dnf groupinstall -y "Development Tools" "RPM Development Tools" 116 | dnf install -y rpm-build wget git 117 | dnf upgrade -y 118 | 119 | # Create the rpmbuild directory tree inside the container 120 | mkdir -p ${HOME}/rpmbuild/BUILD 121 | mkdir -p ${HOME}/rpmbuild/BUILDROOT 122 | mkdir -p ${HOME}/rpmbuild/RPMS 123 | mkdir -p ${HOME}/rpmbuild/SOURCES 124 | mkdir -p ${HOME}/rpmbuild/SPECS 125 | mkdir -p ${HOME}/rpmbuild/SRPMS 126 | 127 | # Get the spec and other custom files that are part of the build process 128 | # and put them into the appropriate places in the rpmbuild tree 129 | git clone https://github.com/dlk3/udm-hacks.git 130 | mv udm-hacks/ntopng-udm-fedora/rpmbuild/ntopng.spec ${HOME}/rpmbuild/SPECS/ 131 | mv udm-hacks/ntopng-udm-fedora/rpmbuild/ntopng.conf ${HOME}/rpmbuild/SOURCES/ 132 | mv udm-hacks/ntopng-udm-fedora/rpmbuild/ntopng.service ${HOME}/rpmbuild/SOURCES/ 133 | mv udm-hacks/ntopng-udm-fedora/rpmbuild/ntopng.sysconfig ${HOME}/rpmbuild/SOURCES/ 134 | mv udm-hacks/ntopng-udm-fedora/rpmbuild/lua-makefile-add-fPIC.patch ${HOME}/rpmbuild/SOURCES/ 135 | rm -fr udm-hacks 136 | 137 | # Put tarballs of the program source into the container's rpmbuild/SOURCES directory 138 | git clone https://github.com/ntop/nDPI.git 139 | tar -czf ${HOME}/rpmbuild/SOURCES/nDPI.tar.gz nDPI 140 | rm -fr nDPI 141 | NAME=$(sed -n 's/^Name:[[:space:]]*//p' ${HOME}/rpmbuild/SPECS/ntopng.spec) 142 | BRANCH="" 143 | if [ ! -z "$STABLE" ]; then 144 | BRANCH="--branch=$STABLE" 145 | fi 146 | git clone https://github.com/ntop/${NAME}.git $BRANCH 147 | tar -czf ${HOME}/rpmbuild/SOURCES/${NAME}.tar.gz ${NAME} 148 | rm -fr ${NAME} 149 | 150 | # Install the BuildRequires packages listed in the spec file 151 | PACKAGES='' 152 | oIFS=$IFS 153 | IFS=$'\n' 154 | for LINE in `grep ^BuildRequires: "${HOME}/rpmbuild/SPECS/ntopng.spec"`; do 155 | PACKAGES+="$(echo $LINE | cut -f2 -d':' | sed 's/,/ /g') " 156 | done 157 | IFS=$oIFS 158 | [ "$PACKAGES" != "" ] && dnf install -y $PACKAGES 159 | 160 | # Build the package 161 | rpmbuild -ba "${HOME}/rpmbuild/SPECS/ntopng.spec" 162 | 163 | # Copy the package files out of the container and on to the host 164 | cp -vr ${HOME}/rpmbuild/RPMS/* ${TARGETDIR}/rpmbuild/RPMS/ 165 | cp -vr ${HOME}/rpmbuild/SRPMS/* ${TARGETDIR}/rpmbuild/SRPMS/ 166 | 167 | fi 168 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/data/ntopng/GeoIP.conf: -------------------------------------------------------------------------------- 1 | # GeoIP.conf file for the `geoipupdate` program that is included 2 | # in the ntopng-udm container image. 3 | # 4 | # Used to update GeoIP databases from https://www.maxmind.com. 5 | # For more information about this config file, visit the docs at 6 | # https://dev.maxmind.com/geoip/geoipupdate/. 7 | 8 | # `AccountID` is from your MaxMind account. 9 | AccountID YOUR_ACCOUNT_ID_HERE 10 | 11 | # `LicenseKey` is from your MaxMind account. 12 | LicenseKey YOUR_LICENSE_KEY_HERE 13 | 14 | # `EditionIDs` are those required by ntopng. 15 | EditionIDs GeoLite2-ASN GeoLite2-City GeoLite2-Country 16 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/data/ntopng/ntopng.conf: -------------------------------------------------------------------------------- 1 | # The configuration file is similar to the command line, with the exception that an equal 2 | # sign '=' must be used between key and value. Example: -i=p1p2 or --interface=p1p2 For 3 | # options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used. 4 | # 5 | # 6 | # -G|--pid-path 7 | # Specifies the path where the PID (process ID) is saved. This option is ignored when 8 | # ntopng is controlled with systemd (e.g., service ntopng start). 9 | # 10 | -G=/var/run/ntopng.pid 11 | # 12 | # -e|--daemon 13 | # This parameter causes ntop to become a daemon, i.e. a task which runs in the background 14 | # without connection to a specific terminal. To use ntop other than as a casual monitoring 15 | # tool, you probably will want to use this option. This option is ignored when ntopng is 16 | # controlled with systemd (e.g., service ntopng start) 17 | # 18 | # -e= 19 | # 20 | # -i|--interface 21 | # Specifies the network interface or collector endpoint to be used by ntopng for network 22 | # monitoring. On Unix you can specify both the interface name (e.g. lo) or the numeric 23 | # interface id as shown by ntopng -h. On Windows you must use the interface number instead. 24 | # Note that you can specify -i multiple times in order to instruct ntopng to create multi- 25 | # ple interfaces. 26 | # 27 | # -i=eth1 28 | # -i=eth2 29 | -i=br0 30 | -i=eth8 31 | # 32 | # -w|--http-port 33 | # Sets the HTTP port of the embedded web server. 34 | # 35 | # -w=3000 36 | # 37 | # -W|--https-port 38 | # Sets the HTTPS port of the embedded web server. 39 | -W=3001 40 | # 41 | # -m|--local-networks 42 | # ntopng determines the ip addresses and netmasks for each active interface. Any traffic on 43 | # those networks is considered local. This parameter allows the user to define additional 44 | # networks and subnetworks whose traffic is also considered local in ntopng reports. All 45 | # other hosts are considered remote. If not specified the default is set to 192.168.1.0/24. 46 | # 47 | # Commas separate multiple network values. Both netmask and CIDR notation may be used, 48 | # even mixed together, for instance "131.114.21.0/24,10.0.0.0/255.0.0.0". 49 | # 50 | # -m=10.10.123.0/24 51 | # -m=10.10.124.0/24 52 | # 53 | # -n|--dns-mode 54 | # Sets the DNS address resolution mode: 0 - Decode DNS responses and resolve only local 55 | # (-m) numeric IPs 1 - Decode DNS responses and resolve all numeric IPs 2 - Decode DNS 56 | # responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't resolve 57 | # 58 | # -n=1 59 | -n=1 60 | # 61 | # -S|--sticky-hosts 62 | # ntopng periodically purges idle hosts. With this option you can modify this behaviour by 63 | # telling ntopng not to purge the hosts specified by -S. This parameter requires an argu- 64 | # ment that can be "all" (Keep all hosts in memory), "local" (Keep only local hosts), 65 | # "remote" (Keep only remote hosts), "none" (Flush hosts when idle). 66 | # 67 | # -S= 68 | # 69 | # -d|--data-dir 70 | # Specifies the data directory (it must be writable by the user that is executing ntopng). 71 | # 72 | # -d=/var/lib/ntopng 73 | # 74 | # -q|--disable-autologout 75 | # Disable web interface logout for inactivity. 76 | # 77 | # -q=a 78 | # 79 | # -F|--dump-flows 80 | # Dump expired flows. Mode: 81 | # es Dump in ElasticSearch database 82 | # Format: es;;;; 83 | # Example: es;ntopng;ntopng-%Y.%m.%d;http://localhost:9200/_bulk; 84 | # Notes: 85 | # The accepts the strftime() format. 86 | # s have been removed starting at ElasticSearch version 6. 87 | # values whill therefore be ignored when using versions 88 | # greater than or equal to 6. 89 | # 90 | # syslog Dump in syslog 91 | # 92 | # mysql Dump in MySQL database 93 | # Format: mysql;;;;; 94 | # Exanple: mysql;localhost;ntopng;flows;root; 95 | # 96 | # mysql-nprobe Read from an nProbe-generated MySQL database 97 | # Format: mysql-nprobe;;;;; 98 | # Example: mysql-nprobe;localhost;ntopng;nf;root; 99 | # Notes: 100 | # The must be the same as used in nProbe. 101 | # Only one ntopng -i is allowed. 102 | # Flows are only read. Dump is assumed to be done by nProbe. 103 | # Example: 104 | # ./nprobe ... --mysql="localhost:ntopng:nf:root:root" 105 | # ./ntopng ... --dump-flows="mysql-nprobe;localhost;ntopng;nf;root;root" 106 | # 107 | # -F mysql;;;
;; 108 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/data/ntopng/redis.conf: -------------------------------------------------------------------------------- 1 | # Redis configuration file example. 2 | # 3 | # Note that in order to read the configuration file, Redis must be 4 | # started with the file path as first argument: 5 | # 6 | # ./redis-server /path/to/redis.conf 7 | 8 | # Note on units: when memory size is needed, it is possible to specify 9 | # it in the usual form of 1k 5GB 4M and so forth: 10 | # 11 | # 1k => 1000 bytes 12 | # 1kb => 1024 bytes 13 | # 1m => 1000000 bytes 14 | # 1mb => 1024*1024 bytes 15 | # 1g => 1000000000 bytes 16 | # 1gb => 1024*1024*1024 bytes 17 | # 18 | # units are case insensitive so 1GB 1Gb 1gB are all the same. 19 | 20 | ################################## INCLUDES ################################### 21 | 22 | # Include one or more other config files here. This is useful if you 23 | # have a standard template that goes to all Redis servers but also need 24 | # to customize a few per-server settings. Include files can include 25 | # other files, so use this wisely. 26 | # 27 | # Notice option "include" won't be rewritten by command "CONFIG REWRITE" 28 | # from admin or Redis Sentinel. Since Redis always uses the last processed 29 | # line as value of a configuration directive, you'd better put includes 30 | # at the beginning of this file to avoid overwriting config change at runtime. 31 | # 32 | # If instead you are interested in using includes to override configuration 33 | # options, it is better to use include as the last line. 34 | # 35 | # include /path/to/local.conf 36 | # include /path/to/other.conf 37 | 38 | ################################## NETWORK ##################################### 39 | 40 | # By default, if no "bind" configuration directive is specified, Redis listens 41 | # for connections from all the network interfaces available on the server. 42 | # It is possible to listen to just one or multiple selected interfaces using 43 | # the "bind" configuration directive, followed by one or more IP addresses. 44 | # 45 | # Examples: 46 | # 47 | # bind 192.168.1.100 10.0.0.1 48 | # bind 127.0.0.1 ::1 49 | # 50 | # ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the 51 | # internet, binding to all the interfaces is dangerous and will expose the 52 | # instance to everybody on the internet. So by default we uncomment the 53 | # following bind directive, that will force Redis to listen only into 54 | # the IPv4 lookback interface address (this means Redis will be able to 55 | # accept connections only from clients running into the same computer it 56 | # is running). 57 | # 58 | # IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES 59 | # JUST COMMENT THE FOLLOWING LINE. 60 | # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 61 | bind 127.0.0.1 62 | 63 | # Protected mode is a layer of security protection, in order to avoid that 64 | # Redis instances left open on the internet are accessed and exploited. 65 | # 66 | # When protected mode is on and if: 67 | # 68 | # 1) The server is not binding explicitly to a set of addresses using the 69 | # "bind" directive. 70 | # 2) No password is configured. 71 | # 72 | # The server only accepts connections from clients connecting from the 73 | # IPv4 and IPv6 loopback addresses 127.0.0.1 and ::1, and from Unix domain 74 | # sockets. 75 | # 76 | # By default protected mode is enabled. You should disable it only if 77 | # you are sure you want clients from other hosts to connect to Redis 78 | # even if no authentication is configured, nor a specific set of interfaces 79 | # are explicitly listed using the "bind" directive. 80 | protected-mode yes 81 | 82 | # Accept connections on the specified port, default is 6379 (IANA #815344). 83 | # If port 0 is specified Redis will not listen on a TCP socket. 84 | port 6379 85 | 86 | # TCP listen() backlog. 87 | # 88 | # In high requests-per-second environments you need an high backlog in order 89 | # to avoid slow clients connections issues. Note that the Linux kernel 90 | # will silently truncate it to the value of /proc/sys/net/core/somaxconn so 91 | # make sure to raise both the value of somaxconn and tcp_max_syn_backlog 92 | # in order to get the desired effect. 93 | tcp-backlog 511 94 | 95 | # Unix socket. 96 | # 97 | # Specify the path for the Unix socket that will be used to listen for 98 | # incoming connections. There is no default, so Redis will not listen 99 | # on a unix socket when not specified. 100 | # 101 | # unixsocket /var/run/redis/redis.sock 102 | # unixsocketperm 700 103 | 104 | # Close the connection after a client is idle for N seconds (0 to disable) 105 | timeout 0 106 | 107 | # TCP keepalive. 108 | # 109 | # If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence 110 | # of communication. This is useful for two reasons: 111 | # 112 | # 1) Detect dead peers. 113 | # 2) Take the connection alive from the point of view of network 114 | # equipment in the middle. 115 | # 116 | # On Linux, the specified value (in seconds) is the period used to send ACKs. 117 | # Note that to close the connection the double of the time is needed. 118 | # On other kernels the period depends on the kernel configuration. 119 | # 120 | # A reasonable value for this option is 300 seconds, which is the new 121 | # Redis default starting with Redis 3.2.1. 122 | tcp-keepalive 300 123 | 124 | ################################# GENERAL ##################################### 125 | 126 | # By default Redis does not run as a daemon. Use 'yes' if you need it. 127 | # Note that Redis will write a pid file in /var/run/redis.pid when daemonized. 128 | daemonize yes 129 | 130 | # If you run Redis from upstart or systemd, Redis can interact with your 131 | # supervision tree. Options: 132 | # supervised no - no supervision interaction 133 | # supervised upstart - signal upstart by putting Redis into SIGSTOP mode 134 | # supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET 135 | # supervised auto - detect upstart or systemd method based on 136 | # UPSTART_JOB or NOTIFY_SOCKET environment variables 137 | # Note: these supervision methods only signal "process is ready." 138 | # They do not enable continuous liveness pings back to your supervisor. 139 | supervised no 140 | 141 | # If a pid file is specified, Redis writes it where specified at startup 142 | # and removes it at exit. 143 | # 144 | # When the server runs non daemonized, no pid file is created if none is 145 | # specified in the configuration. When the server is daemonized, the pid file 146 | # is used even if not specified, defaulting to "/var/run/redis.pid". 147 | # 148 | # Creating a pid file is best effort: if Redis is not able to create it 149 | # nothing bad happens, the server will start and run normally. 150 | pidfile /var/run/redis/redis-server.pid 151 | 152 | # Specify the server verbosity level. 153 | # This can be one of: 154 | # debug (a lot of information, useful for development/testing) 155 | # verbose (many rarely useful info, but not a mess like the debug level) 156 | # notice (moderately verbose, what you want in production probably) 157 | # warning (only very important / critical messages are logged) 158 | loglevel notice 159 | 160 | # Specify the log file name. Also the empty string can be used to force 161 | # Redis to log on the standard output. Note that if you use standard 162 | # output for logging but daemonize, logs will be sent to /dev/null 163 | logfile /var/log/redis/redis-server.log 164 | 165 | # To enable logging to the system logger, just set 'syslog-enabled' to yes, 166 | # and optionally update the other syslog parameters to suit your needs. 167 | # syslog-enabled no 168 | 169 | # Specify the syslog identity. 170 | # syslog-ident redis 171 | 172 | # Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7. 173 | # syslog-facility local0 174 | 175 | # Set the number of databases. The default database is DB 0, you can select 176 | # a different one on a per-connection basis using SELECT where 177 | # dbid is a number between 0 and 'databases'-1 178 | databases 16 179 | 180 | ################################ SNAPSHOTTING ################################ 181 | # 182 | # Save the DB on disk: 183 | # 184 | # save 185 | # 186 | # Will save the DB if both the given number of seconds and the given 187 | # number of write operations against the DB occurred. 188 | # 189 | # In the example below the behaviour will be to save: 190 | # after 900 sec (15 min) if at least 1 key changed 191 | # after 300 sec (5 min) if at least 10 keys changed 192 | # after 60 sec if at least 10000 keys changed 193 | # 194 | # Note: you can disable saving completely by commenting out all "save" lines. 195 | # 196 | # It is also possible to remove all the previously configured save 197 | # points by adding a save directive with a single empty string argument 198 | # like in the following example: 199 | # 200 | # save "" 201 | 202 | save 900 1 203 | save 300 10 204 | save 60 10000 205 | 206 | # By default Redis will stop accepting writes if RDB snapshots are enabled 207 | # (at least one save point) and the latest background save failed. 208 | # This will make the user aware (in a hard way) that data is not persisting 209 | # on disk properly, otherwise chances are that no one will notice and some 210 | # disaster will happen. 211 | # 212 | # If the background saving process will start working again Redis will 213 | # automatically allow writes again. 214 | # 215 | # However if you have setup your proper monitoring of the Redis server 216 | # and persistence, you may want to disable this feature so that Redis will 217 | # continue to work as usual even if there are problems with disk, 218 | # permissions, and so forth. 219 | stop-writes-on-bgsave-error yes 220 | 221 | # Compress string objects using LZF when dump .rdb databases? 222 | # For default that's set to 'yes' as it's almost always a win. 223 | # If you want to save some CPU in the saving child set it to 'no' but 224 | # the dataset will likely be bigger if you have compressible values or keys. 225 | rdbcompression yes 226 | 227 | # Since version 5 of RDB a CRC64 checksum is placed at the end of the file. 228 | # This makes the format more resistant to corruption but there is a performance 229 | # hit to pay (around 10%) when saving and loading RDB files, so you can disable it 230 | # for maximum performances. 231 | # 232 | # RDB files created with checksum disabled have a checksum of zero that will 233 | # tell the loading code to skip the check. 234 | rdbchecksum yes 235 | 236 | # The filename where to dump the DB 237 | dbfilename dump.rdb 238 | 239 | # The working directory. 240 | # 241 | # The DB will be written inside this directory, with the filename specified 242 | # above using the 'dbfilename' configuration directive. 243 | # 244 | # The Append Only File will also be created inside this directory. 245 | # 246 | # Note that you must specify a directory here, not a file name. 247 | dir /var/lib/redis 248 | 249 | ################################# REPLICATION ################################# 250 | 251 | # Master-Slave replication. Use slaveof to make a Redis instance a copy of 252 | # another Redis server. A few things to understand ASAP about Redis replication. 253 | # 254 | # 1) Redis replication is asynchronous, but you can configure a master to 255 | # stop accepting writes if it appears to be not connected with at least 256 | # a given number of slaves. 257 | # 2) Redis slaves are able to perform a partial resynchronization with the 258 | # master if the replication link is lost for a relatively small amount of 259 | # time. You may want to configure the replication backlog size (see the next 260 | # sections of this file) with a sensible value depending on your needs. 261 | # 3) Replication is automatic and does not need user intervention. After a 262 | # network partition slaves automatically try to reconnect to masters 263 | # and resynchronize with them. 264 | # 265 | # slaveof 266 | 267 | # If the master is password protected (using the "requirepass" configuration 268 | # directive below) it is possible to tell the slave to authenticate before 269 | # starting the replication synchronization process, otherwise the master will 270 | # refuse the slave request. 271 | # 272 | # masterauth 273 | 274 | # When a slave loses its connection with the master, or when the replication 275 | # is still in progress, the slave can act in two different ways: 276 | # 277 | # 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will 278 | # still reply to client requests, possibly with out of date data, or the 279 | # data set may just be empty if this is the first synchronization. 280 | # 281 | # 2) if slave-serve-stale-data is set to 'no' the slave will reply with 282 | # an error "SYNC with master in progress" to all the kind of commands 283 | # but to INFO and SLAVEOF. 284 | # 285 | slave-serve-stale-data yes 286 | 287 | # You can configure a slave instance to accept writes or not. Writing against 288 | # a slave instance may be useful to store some ephemeral data (because data 289 | # written on a slave will be easily deleted after resync with the master) but 290 | # may also cause problems if clients are writing to it because of a 291 | # misconfiguration. 292 | # 293 | # Since Redis 2.6 by default slaves are read-only. 294 | # 295 | # Note: read only slaves are not designed to be exposed to untrusted clients 296 | # on the internet. It's just a protection layer against misuse of the instance. 297 | # Still a read only slave exports by default all the administrative commands 298 | # such as CONFIG, DEBUG, and so forth. To a limited extent you can improve 299 | # security of read only slaves using 'rename-command' to shadow all the 300 | # administrative / dangerous commands. 301 | slave-read-only yes 302 | 303 | # Replication SYNC strategy: disk or socket. 304 | # 305 | # ------------------------------------------------------- 306 | # WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY 307 | # ------------------------------------------------------- 308 | # 309 | # New slaves and reconnecting slaves that are not able to continue the replication 310 | # process just receiving differences, need to do what is called a "full 311 | # synchronization". An RDB file is transmitted from the master to the slaves. 312 | # The transmission can happen in two different ways: 313 | # 314 | # 1) Disk-backed: The Redis master creates a new process that writes the RDB 315 | # file on disk. Later the file is transferred by the parent 316 | # process to the slaves incrementally. 317 | # 2) Diskless: The Redis master creates a new process that directly writes the 318 | # RDB file to slave sockets, without touching the disk at all. 319 | # 320 | # With disk-backed replication, while the RDB file is generated, more slaves 321 | # can be queued and served with the RDB file as soon as the current child producing 322 | # the RDB file finishes its work. With diskless replication instead once 323 | # the transfer starts, new slaves arriving will be queued and a new transfer 324 | # will start when the current one terminates. 325 | # 326 | # When diskless replication is used, the master waits a configurable amount of 327 | # time (in seconds) before starting the transfer in the hope that multiple slaves 328 | # will arrive and the transfer can be parallelized. 329 | # 330 | # With slow disks and fast (large bandwidth) networks, diskless replication 331 | # works better. 332 | repl-diskless-sync no 333 | 334 | # When diskless replication is enabled, it is possible to configure the delay 335 | # the server waits in order to spawn the child that transfers the RDB via socket 336 | # to the slaves. 337 | # 338 | # This is important since once the transfer starts, it is not possible to serve 339 | # new slaves arriving, that will be queued for the next RDB transfer, so the server 340 | # waits a delay in order to let more slaves arrive. 341 | # 342 | # The delay is specified in seconds, and by default is 5 seconds. To disable 343 | # it entirely just set it to 0 seconds and the transfer will start ASAP. 344 | repl-diskless-sync-delay 5 345 | 346 | # Slaves send PINGs to server in a predefined interval. It's possible to change 347 | # this interval with the repl_ping_slave_period option. The default value is 10 348 | # seconds. 349 | # 350 | # repl-ping-slave-period 10 351 | 352 | # The following option sets the replication timeout for: 353 | # 354 | # 1) Bulk transfer I/O during SYNC, from the point of view of slave. 355 | # 2) Master timeout from the point of view of slaves (data, pings). 356 | # 3) Slave timeout from the point of view of masters (REPLCONF ACK pings). 357 | # 358 | # It is important to make sure that this value is greater than the value 359 | # specified for repl-ping-slave-period otherwise a timeout will be detected 360 | # every time there is low traffic between the master and the slave. 361 | # 362 | # repl-timeout 60 363 | 364 | # Disable TCP_NODELAY on the slave socket after SYNC? 365 | # 366 | # If you select "yes" Redis will use a smaller number of TCP packets and 367 | # less bandwidth to send data to slaves. But this can add a delay for 368 | # the data to appear on the slave side, up to 40 milliseconds with 369 | # Linux kernels using a default configuration. 370 | # 371 | # If you select "no" the delay for data to appear on the slave side will 372 | # be reduced but more bandwidth will be used for replication. 373 | # 374 | # By default we optimize for low latency, but in very high traffic conditions 375 | # or when the master and slaves are many hops away, turning this to "yes" may 376 | # be a good idea. 377 | repl-disable-tcp-nodelay no 378 | 379 | # Set the replication backlog size. The backlog is a buffer that accumulates 380 | # slave data when slaves are disconnected for some time, so that when a slave 381 | # wants to reconnect again, often a full resync is not needed, but a partial 382 | # resync is enough, just passing the portion of data the slave missed while 383 | # disconnected. 384 | # 385 | # The bigger the replication backlog, the longer the time the slave can be 386 | # disconnected and later be able to perform a partial resynchronization. 387 | # 388 | # The backlog is only allocated once there is at least a slave connected. 389 | # 390 | # repl-backlog-size 1mb 391 | 392 | # After a master has no longer connected slaves for some time, the backlog 393 | # will be freed. The following option configures the amount of seconds that 394 | # need to elapse, starting from the time the last slave disconnected, for 395 | # the backlog buffer to be freed. 396 | # 397 | # A value of 0 means to never release the backlog. 398 | # 399 | # repl-backlog-ttl 3600 400 | 401 | # The slave priority is an integer number published by Redis in the INFO output. 402 | # It is used by Redis Sentinel in order to select a slave to promote into a 403 | # master if the master is no longer working correctly. 404 | # 405 | # A slave with a low priority number is considered better for promotion, so 406 | # for instance if there are three slaves with priority 10, 100, 25 Sentinel will 407 | # pick the one with priority 10, that is the lowest. 408 | # 409 | # However a special priority of 0 marks the slave as not able to perform the 410 | # role of master, so a slave with priority of 0 will never be selected by 411 | # Redis Sentinel for promotion. 412 | # 413 | # By default the priority is 100. 414 | slave-priority 100 415 | 416 | # It is possible for a master to stop accepting writes if there are less than 417 | # N slaves connected, having a lag less or equal than M seconds. 418 | # 419 | # The N slaves need to be in "online" state. 420 | # 421 | # The lag in seconds, that must be <= the specified value, is calculated from 422 | # the last ping received from the slave, that is usually sent every second. 423 | # 424 | # This option does not GUARANTEE that N replicas will accept the write, but 425 | # will limit the window of exposure for lost writes in case not enough slaves 426 | # are available, to the specified number of seconds. 427 | # 428 | # For example to require at least 3 slaves with a lag <= 10 seconds use: 429 | # 430 | # min-slaves-to-write 3 431 | # min-slaves-max-lag 10 432 | # 433 | # Setting one or the other to 0 disables the feature. 434 | # 435 | # By default min-slaves-to-write is set to 0 (feature disabled) and 436 | # min-slaves-max-lag is set to 10. 437 | 438 | # A Redis master is able to list the address and port of the attached 439 | # slaves in different ways. For example the "INFO replication" section 440 | # offers this information, which is used, among other tools, by 441 | # Redis Sentinel in order to discover slave instances. 442 | # Another place where this info is available is in the output of the 443 | # "ROLE" command of a masteer. 444 | # 445 | # The listed IP and address normally reported by a slave is obtained 446 | # in the following way: 447 | # 448 | # IP: The address is auto detected by checking the peer address 449 | # of the socket used by the slave to connect with the master. 450 | # 451 | # Port: The port is communicated by the slave during the replication 452 | # handshake, and is normally the port that the slave is using to 453 | # list for connections. 454 | # 455 | # However when port forwarding or Network Address Translation (NAT) is 456 | # used, the slave may be actually reachable via different IP and port 457 | # pairs. The following two options can be used by a slave in order to 458 | # report to its master a specific set of IP and port, so that both INFO 459 | # and ROLE will report those values. 460 | # 461 | # There is no need to use both the options if you need to override just 462 | # the port or the IP address. 463 | # 464 | # slave-announce-ip 5.5.5.5 465 | # slave-announce-port 1234 466 | 467 | ################################## SECURITY ################################### 468 | 469 | # Require clients to issue AUTH before processing any other 470 | # commands. This might be useful in environments in which you do not trust 471 | # others with access to the host running redis-server. 472 | # 473 | # This should stay commented out for backward compatibility and because most 474 | # people do not need auth (e.g. they run their own servers). 475 | # 476 | # Warning: since Redis is pretty fast an outside user can try up to 477 | # 150k passwords per second against a good box. This means that you should 478 | # use a very strong password otherwise it will be very easy to break. 479 | # 480 | # requirepass foobared 481 | 482 | # Command renaming. 483 | # 484 | # It is possible to change the name of dangerous commands in a shared 485 | # environment. For instance the CONFIG command may be renamed into something 486 | # hard to guess so that it will still be available for internal-use tools 487 | # but not available for general clients. 488 | # 489 | # Example: 490 | # 491 | # rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52 492 | # 493 | # It is also possible to completely kill a command by renaming it into 494 | # an empty string: 495 | # 496 | # rename-command CONFIG "" 497 | # 498 | # Please note that changing the name of commands that are logged into the 499 | # AOF file or transmitted to slaves may cause problems. 500 | 501 | ################################### LIMITS #################################### 502 | 503 | # Set the max number of connected clients at the same time. By default 504 | # this limit is set to 10000 clients, however if the Redis server is not 505 | # able to configure the process file limit to allow for the specified limit 506 | # the max number of allowed clients is set to the current file limit 507 | # minus 32 (as Redis reserves a few file descriptors for internal uses). 508 | # 509 | # Once the limit is reached Redis will close all the new connections sending 510 | # an error 'max number of clients reached'. 511 | # 512 | # maxclients 10000 513 | 514 | # Don't use more memory than the specified amount of bytes. 515 | # When the memory limit is reached Redis will try to remove keys 516 | # according to the eviction policy selected (see maxmemory-policy). 517 | # 518 | # If Redis can't remove keys according to the policy, or if the policy is 519 | # set to 'noeviction', Redis will start to reply with errors to commands 520 | # that would use more memory, like SET, LPUSH, and so on, and will continue 521 | # to reply to read-only commands like GET. 522 | # 523 | # This option is usually useful when using Redis as an LRU cache, or to set 524 | # a hard memory limit for an instance (using the 'noeviction' policy). 525 | # 526 | # WARNING: If you have slaves attached to an instance with maxmemory on, 527 | # the size of the output buffers needed to feed the slaves are subtracted 528 | # from the used memory count, so that network problems / resyncs will 529 | # not trigger a loop where keys are evicted, and in turn the output 530 | # buffer of slaves is full with DELs of keys evicted triggering the deletion 531 | # of more keys, and so forth until the database is completely emptied. 532 | # 533 | # In short... if you have slaves attached it is suggested that you set a lower 534 | # limit for maxmemory so that there is some free RAM on the system for slave 535 | # output buffers (but this is not needed if the policy is 'noeviction'). 536 | # 537 | # maxmemory 538 | 539 | # MAXMEMORY POLICY: how Redis will select what to remove when maxmemory 540 | # is reached. You can select among five behaviors: 541 | # 542 | # volatile-lru -> remove the key with an expire set using an LRU algorithm 543 | # allkeys-lru -> remove any key according to the LRU algorithm 544 | # volatile-random -> remove a random key with an expire set 545 | # allkeys-random -> remove a random key, any key 546 | # volatile-ttl -> remove the key with the nearest expire time (minor TTL) 547 | # noeviction -> don't expire at all, just return an error on write operations 548 | # 549 | # Note: with any of the above policies, Redis will return an error on write 550 | # operations, when there are no suitable keys for eviction. 551 | # 552 | # At the date of writing these commands are: set setnx setex append 553 | # incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd 554 | # sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby 555 | # zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby 556 | # getset mset msetnx exec sort 557 | # 558 | # The default is: 559 | # 560 | # maxmemory-policy noeviction 561 | 562 | # LRU and minimal TTL algorithms are not precise algorithms but approximated 563 | # algorithms (in order to save memory), so you can tune it for speed or 564 | # accuracy. For default Redis will check five keys and pick the one that was 565 | # used less recently, you can change the sample size using the following 566 | # configuration directive. 567 | # 568 | # The default of 5 produces good enough results. 10 Approximates very closely 569 | # true LRU but costs a bit more CPU. 3 is very fast but not very accurate. 570 | # 571 | # maxmemory-samples 5 572 | 573 | ############################## APPEND ONLY MODE ############################### 574 | 575 | # By default Redis asynchronously dumps the dataset on disk. This mode is 576 | # good enough in many applications, but an issue with the Redis process or 577 | # a power outage may result into a few minutes of writes lost (depending on 578 | # the configured save points). 579 | # 580 | # The Append Only File is an alternative persistence mode that provides 581 | # much better durability. For instance using the default data fsync policy 582 | # (see later in the config file) Redis can lose just one second of writes in a 583 | # dramatic event like a server power outage, or a single write if something 584 | # wrong with the Redis process itself happens, but the operating system is 585 | # still running correctly. 586 | # 587 | # AOF and RDB persistence can be enabled at the same time without problems. 588 | # If the AOF is enabled on startup Redis will load the AOF, that is the file 589 | # with the better durability guarantees. 590 | # 591 | # Please check http://redis.io/topics/persistence for more information. 592 | 593 | appendonly no 594 | 595 | # The name of the append only file (default: "appendonly.aof") 596 | 597 | appendfilename "appendonly.aof" 598 | 599 | # The fsync() call tells the Operating System to actually write data on disk 600 | # instead of waiting for more data in the output buffer. Some OS will really flush 601 | # data on disk, some other OS will just try to do it ASAP. 602 | # 603 | # Redis supports three different modes: 604 | # 605 | # no: don't fsync, just let the OS flush the data when it wants. Faster. 606 | # always: fsync after every write to the append only log. Slow, Safest. 607 | # everysec: fsync only one time every second. Compromise. 608 | # 609 | # The default is "everysec", as that's usually the right compromise between 610 | # speed and data safety. It's up to you to understand if you can relax this to 611 | # "no" that will let the operating system flush the output buffer when 612 | # it wants, for better performances (but if you can live with the idea of 613 | # some data loss consider the default persistence mode that's snapshotting), 614 | # or on the contrary, use "always" that's very slow but a bit safer than 615 | # everysec. 616 | # 617 | # More details please check the following article: 618 | # http://antirez.com/post/redis-persistence-demystified.html 619 | # 620 | # If unsure, use "everysec". 621 | 622 | # appendfsync always 623 | appendfsync everysec 624 | # appendfsync no 625 | 626 | # When the AOF fsync policy is set to always or everysec, and a background 627 | # saving process (a background save or AOF log background rewriting) is 628 | # performing a lot of I/O against the disk, in some Linux configurations 629 | # Redis may block too long on the fsync() call. Note that there is no fix for 630 | # this currently, as even performing fsync in a different thread will block 631 | # our synchronous write(2) call. 632 | # 633 | # In order to mitigate this problem it's possible to use the following option 634 | # that will prevent fsync() from being called in the main process while a 635 | # BGSAVE or BGREWRITEAOF is in progress. 636 | # 637 | # This means that while another child is saving, the durability of Redis is 638 | # the same as "appendfsync none". In practical terms, this means that it is 639 | # possible to lose up to 30 seconds of log in the worst scenario (with the 640 | # default Linux settings). 641 | # 642 | # If you have latency problems turn this to "yes". Otherwise leave it as 643 | # "no" that is the safest pick from the point of view of durability. 644 | 645 | no-appendfsync-on-rewrite no 646 | 647 | # Automatic rewrite of the append only file. 648 | # Redis is able to automatically rewrite the log file implicitly calling 649 | # BGREWRITEAOF when the AOF log size grows by the specified percentage. 650 | # 651 | # This is how it works: Redis remembers the size of the AOF file after the 652 | # latest rewrite (if no rewrite has happened since the restart, the size of 653 | # the AOF at startup is used). 654 | # 655 | # This base size is compared to the current size. If the current size is 656 | # bigger than the specified percentage, the rewrite is triggered. Also 657 | # you need to specify a minimal size for the AOF file to be rewritten, this 658 | # is useful to avoid rewriting the AOF file even if the percentage increase 659 | # is reached but it is still pretty small. 660 | # 661 | # Specify a percentage of zero in order to disable the automatic AOF 662 | # rewrite feature. 663 | 664 | auto-aof-rewrite-percentage 100 665 | auto-aof-rewrite-min-size 64mb 666 | 667 | # An AOF file may be found to be truncated at the end during the Redis 668 | # startup process, when the AOF data gets loaded back into memory. 669 | # This may happen when the system where Redis is running 670 | # crashes, especially when an ext4 filesystem is mounted without the 671 | # data=ordered option (however this can't happen when Redis itself 672 | # crashes or aborts but the operating system still works correctly). 673 | # 674 | # Redis can either exit with an error when this happens, or load as much 675 | # data as possible (the default now) and start if the AOF file is found 676 | # to be truncated at the end. The following option controls this behavior. 677 | # 678 | # If aof-load-truncated is set to yes, a truncated AOF file is loaded and 679 | # the Redis server starts emitting a log to inform the user of the event. 680 | # Otherwise if the option is set to no, the server aborts with an error 681 | # and refuses to start. When the option is set to no, the user requires 682 | # to fix the AOF file using the "redis-check-aof" utility before to restart 683 | # the server. 684 | # 685 | # Note that if the AOF file will be found to be corrupted in the middle 686 | # the server will still exit with an error. This option only applies when 687 | # Redis will try to read more data from the AOF file but not enough bytes 688 | # will be found. 689 | aof-load-truncated yes 690 | 691 | ################################ LUA SCRIPTING ############################### 692 | 693 | # Max execution time of a Lua script in milliseconds. 694 | # 695 | # If the maximum execution time is reached Redis will log that a script is 696 | # still in execution after the maximum allowed time and will start to 697 | # reply to queries with an error. 698 | # 699 | # When a long running script exceeds the maximum execution time only the 700 | # SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be 701 | # used to stop a script that did not yet called write commands. The second 702 | # is the only way to shut down the server in the case a write command was 703 | # already issued by the script but the user doesn't want to wait for the natural 704 | # termination of the script. 705 | # 706 | # Set it to 0 or a negative value for unlimited execution without warnings. 707 | lua-time-limit 5000 708 | 709 | ################################ REDIS CLUSTER ############################### 710 | # 711 | # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 712 | # WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however 713 | # in order to mark it as "mature" we need to wait for a non trivial percentage 714 | # of users to deploy it in production. 715 | # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 716 | # 717 | # Normal Redis instances can't be part of a Redis Cluster; only nodes that are 718 | # started as cluster nodes can. In order to start a Redis instance as a 719 | # cluster node enable the cluster support uncommenting the following: 720 | # 721 | # cluster-enabled yes 722 | 723 | # Every cluster node has a cluster configuration file. This file is not 724 | # intended to be edited by hand. It is created and updated by Redis nodes. 725 | # Every Redis Cluster node requires a different cluster configuration file. 726 | # Make sure that instances running in the same system do not have 727 | # overlapping cluster configuration file names. 728 | # 729 | # cluster-config-file nodes-6379.conf 730 | 731 | # Cluster node timeout is the amount of milliseconds a node must be unreachable 732 | # for it to be considered in failure state. 733 | # Most other internal time limits are multiple of the node timeout. 734 | # 735 | # cluster-node-timeout 15000 736 | 737 | # A slave of a failing master will avoid to start a failover if its data 738 | # looks too old. 739 | # 740 | # There is no simple way for a slave to actually have a exact measure of 741 | # its "data age", so the following two checks are performed: 742 | # 743 | # 1) If there are multiple slaves able to failover, they exchange messages 744 | # in order to try to give an advantage to the slave with the best 745 | # replication offset (more data from the master processed). 746 | # Slaves will try to get their rank by offset, and apply to the start 747 | # of the failover a delay proportional to their rank. 748 | # 749 | # 2) Every single slave computes the time of the last interaction with 750 | # its master. This can be the last ping or command received (if the master 751 | # is still in the "connected" state), or the time that elapsed since the 752 | # disconnection with the master (if the replication link is currently down). 753 | # If the last interaction is too old, the slave will not try to failover 754 | # at all. 755 | # 756 | # The point "2" can be tuned by user. Specifically a slave will not perform 757 | # the failover if, since the last interaction with the master, the time 758 | # elapsed is greater than: 759 | # 760 | # (node-timeout * slave-validity-factor) + repl-ping-slave-period 761 | # 762 | # So for example if node-timeout is 30 seconds, and the slave-validity-factor 763 | # is 10, and assuming a default repl-ping-slave-period of 10 seconds, the 764 | # slave will not try to failover if it was not able to talk with the master 765 | # for longer than 310 seconds. 766 | # 767 | # A large slave-validity-factor may allow slaves with too old data to failover 768 | # a master, while a too small value may prevent the cluster from being able to 769 | # elect a slave at all. 770 | # 771 | # For maximum availability, it is possible to set the slave-validity-factor 772 | # to a value of 0, which means, that slaves will always try to failover the 773 | # master regardless of the last time they interacted with the master. 774 | # (However they'll always try to apply a delay proportional to their 775 | # offset rank). 776 | # 777 | # Zero is the only value able to guarantee that when all the partitions heal 778 | # the cluster will always be able to continue. 779 | # 780 | # cluster-slave-validity-factor 10 781 | 782 | # Cluster slaves are able to migrate to orphaned masters, that are masters 783 | # that are left without working slaves. This improves the cluster ability 784 | # to resist to failures as otherwise an orphaned master can't be failed over 785 | # in case of failure if it has no working slaves. 786 | # 787 | # Slaves migrate to orphaned masters only if there are still at least a 788 | # given number of other working slaves for their old master. This number 789 | # is the "migration barrier". A migration barrier of 1 means that a slave 790 | # will migrate only if there is at least 1 other working slave for its master 791 | # and so forth. It usually reflects the number of slaves you want for every 792 | # master in your cluster. 793 | # 794 | # Default is 1 (slaves migrate only if their masters remain with at least 795 | # one slave). To disable migration just set it to a very large value. 796 | # A value of 0 can be set but is useful only for debugging and dangerous 797 | # in production. 798 | # 799 | # cluster-migration-barrier 1 800 | 801 | # By default Redis Cluster nodes stop accepting queries if they detect there 802 | # is at least an hash slot uncovered (no available node is serving it). 803 | # This way if the cluster is partially down (for example a range of hash slots 804 | # are no longer covered) all the cluster becomes, eventually, unavailable. 805 | # It automatically returns available as soon as all the slots are covered again. 806 | # 807 | # However sometimes you want the subset of the cluster which is working, 808 | # to continue to accept queries for the part of the key space that is still 809 | # covered. In order to do so, just set the cluster-require-full-coverage 810 | # option to no. 811 | # 812 | # cluster-require-full-coverage yes 813 | 814 | # In order to setup your cluster make sure to read the documentation 815 | # available at http://redis.io web site. 816 | 817 | ################################## SLOW LOG ################################### 818 | 819 | # The Redis Slow Log is a system to log queries that exceeded a specified 820 | # execution time. The execution time does not include the I/O operations 821 | # like talking with the client, sending the reply and so forth, 822 | # but just the time needed to actually execute the command (this is the only 823 | # stage of command execution where the thread is blocked and can not serve 824 | # other requests in the meantime). 825 | # 826 | # You can configure the slow log with two parameters: one tells Redis 827 | # what is the execution time, in microseconds, to exceed in order for the 828 | # command to get logged, and the other parameter is the length of the 829 | # slow log. When a new command is logged the oldest one is removed from the 830 | # queue of logged commands. 831 | 832 | # The following time is expressed in microseconds, so 1000000 is equivalent 833 | # to one second. Note that a negative number disables the slow log, while 834 | # a value of zero forces the logging of every command. 835 | slowlog-log-slower-than 10000 836 | 837 | # There is no limit to this length. Just be aware that it will consume memory. 838 | # You can reclaim memory used by the slow log with SLOWLOG RESET. 839 | slowlog-max-len 128 840 | 841 | ################################ LATENCY MONITOR ############################## 842 | 843 | # The Redis latency monitoring subsystem samples different operations 844 | # at runtime in order to collect data related to possible sources of 845 | # latency of a Redis instance. 846 | # 847 | # Via the LATENCY command this information is available to the user that can 848 | # print graphs and obtain reports. 849 | # 850 | # The system only logs operations that were performed in a time equal or 851 | # greater than the amount of milliseconds specified via the 852 | # latency-monitor-threshold configuration directive. When its value is set 853 | # to zero, the latency monitor is turned off. 854 | # 855 | # By default latency monitoring is disabled since it is mostly not needed 856 | # if you don't have latency issues, and collecting data has a performance 857 | # impact, that while very small, can be measured under big load. Latency 858 | # monitoring can easily be enabled at runtime using the command 859 | # "CONFIG SET latency-monitor-threshold " if needed. 860 | latency-monitor-threshold 0 861 | 862 | ############################# EVENT NOTIFICATION ############################## 863 | 864 | # Redis can notify Pub/Sub clients about events happening in the key space. 865 | # This feature is documented at http://redis.io/topics/notifications 866 | # 867 | # For instance if keyspace events notification is enabled, and a client 868 | # performs a DEL operation on key "foo" stored in the Database 0, two 869 | # messages will be published via Pub/Sub: 870 | # 871 | # PUBLISH __keyspace@0__:foo del 872 | # PUBLISH __keyevent@0__:del foo 873 | # 874 | # It is possible to select the events that Redis will notify among a set 875 | # of classes. Every class is identified by a single character: 876 | # 877 | # K Keyspace events, published with __keyspace@__ prefix. 878 | # E Keyevent events, published with __keyevent@__ prefix. 879 | # g Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ... 880 | # $ String commands 881 | # l List commands 882 | # s Set commands 883 | # h Hash commands 884 | # z Sorted set commands 885 | # x Expired events (events generated every time a key expires) 886 | # e Evicted events (events generated when a key is evicted for maxmemory) 887 | # A Alias for g$lshzxe, so that the "AKE" string means all the events. 888 | # 889 | # The "notify-keyspace-events" takes as argument a string that is composed 890 | # of zero or multiple characters. The empty string means that notifications 891 | # are disabled. 892 | # 893 | # Example: to enable list and generic events, from the point of view of the 894 | # event name, use: 895 | # 896 | # notify-keyspace-events Elg 897 | # 898 | # Example 2: to get the stream of the expired keys subscribing to channel 899 | # name __keyevent@0__:expired use: 900 | # 901 | # notify-keyspace-events Ex 902 | # 903 | # By default all notifications are disabled because most users don't need 904 | # this feature and the feature has some overhead. Note that if you don't 905 | # specify at least one of K or E, no events will be delivered. 906 | notify-keyspace-events "" 907 | 908 | ############################### ADVANCED CONFIG ############################### 909 | 910 | # Hashes are encoded using a memory efficient data structure when they have a 911 | # small number of entries, and the biggest entry does not exceed a given 912 | # threshold. These thresholds can be configured using the following directives. 913 | hash-max-ziplist-entries 512 914 | hash-max-ziplist-value 64 915 | 916 | # Lists are also encoded in a special way to save a lot of space. 917 | # The number of entries allowed per internal list node can be specified 918 | # as a fixed maximum size or a maximum number of elements. 919 | # For a fixed maximum size, use -5 through -1, meaning: 920 | # -5: max size: 64 Kb <-- not recommended for normal workloads 921 | # -4: max size: 32 Kb <-- not recommended 922 | # -3: max size: 16 Kb <-- probably not recommended 923 | # -2: max size: 8 Kb <-- good 924 | # -1: max size: 4 Kb <-- good 925 | # Positive numbers mean store up to _exactly_ that number of elements 926 | # per list node. 927 | # The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size), 928 | # but if your use case is unique, adjust the settings as necessary. 929 | list-max-ziplist-size -2 930 | 931 | # Lists may also be compressed. 932 | # Compress depth is the number of quicklist ziplist nodes from *each* side of 933 | # the list to *exclude* from compression. The head and tail of the list 934 | # are always uncompressed for fast push/pop operations. Settings are: 935 | # 0: disable all list compression 936 | # 1: depth 1 means "don't start compressing until after 1 node into the list, 937 | # going from either the head or tail" 938 | # So: [head]->node->node->...->node->[tail] 939 | # [head], [tail] will always be uncompressed; inner nodes will compress. 940 | # 2: [head]->[next]->node->node->...->node->[prev]->[tail] 941 | # 2 here means: don't compress head or head->next or tail->prev or tail, 942 | # but compress all nodes between them. 943 | # 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail] 944 | # etc. 945 | list-compress-depth 0 946 | 947 | # Sets have a special encoding in just one case: when a set is composed 948 | # of just strings that happen to be integers in radix 10 in the range 949 | # of 64 bit signed integers. 950 | # The following configuration setting sets the limit in the size of the 951 | # set in order to use this special memory saving encoding. 952 | set-max-intset-entries 512 953 | 954 | # Similarly to hashes and lists, sorted sets are also specially encoded in 955 | # order to save a lot of space. This encoding is only used when the length and 956 | # elements of a sorted set are below the following limits: 957 | zset-max-ziplist-entries 128 958 | zset-max-ziplist-value 64 959 | 960 | # HyperLogLog sparse representation bytes limit. The limit includes the 961 | # 16 bytes header. When an HyperLogLog using the sparse representation crosses 962 | # this limit, it is converted into the dense representation. 963 | # 964 | # A value greater than 16000 is totally useless, since at that point the 965 | # dense representation is more memory efficient. 966 | # 967 | # The suggested value is ~ 3000 in order to have the benefits of 968 | # the space efficient encoding without slowing down too much PFADD, 969 | # which is O(N) with the sparse encoding. The value can be raised to 970 | # ~ 10000 when CPU is not a concern, but space is, and the data set is 971 | # composed of many HyperLogLogs with cardinality in the 0 - 15000 range. 972 | hll-sparse-max-bytes 3000 973 | 974 | # Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in 975 | # order to help rehashing the main Redis hash table (the one mapping top-level 976 | # keys to values). The hash table implementation Redis uses (see dict.c) 977 | # performs a lazy rehashing: the more operation you run into a hash table 978 | # that is rehashing, the more rehashing "steps" are performed, so if the 979 | # server is idle the rehashing is never complete and some more memory is used 980 | # by the hash table. 981 | # 982 | # The default is to use this millisecond 10 times every second in order to 983 | # actively rehash the main dictionaries, freeing memory when possible. 984 | # 985 | # If unsure: 986 | # use "activerehashing no" if you have hard latency requirements and it is 987 | # not a good thing in your environment that Redis can reply from time to time 988 | # to queries with 2 milliseconds delay. 989 | # 990 | # use "activerehashing yes" if you don't have such hard requirements but 991 | # want to free memory asap when possible. 992 | activerehashing yes 993 | 994 | # The client output buffer limits can be used to force disconnection of clients 995 | # that are not reading data from the server fast enough for some reason (a 996 | # common reason is that a Pub/Sub client can't consume messages as fast as the 997 | # publisher can produce them). 998 | # 999 | # The limit can be set differently for the three different classes of clients: 1000 | # 1001 | # normal -> normal clients including MONITOR clients 1002 | # slave -> slave clients 1003 | # pubsub -> clients subscribed to at least one pubsub channel or pattern 1004 | # 1005 | # The syntax of every client-output-buffer-limit directive is the following: 1006 | # 1007 | # client-output-buffer-limit 1008 | # 1009 | # A client is immediately disconnected once the hard limit is reached, or if 1010 | # the soft limit is reached and remains reached for the specified number of 1011 | # seconds (continuously). 1012 | # So for instance if the hard limit is 32 megabytes and the soft limit is 1013 | # 16 megabytes / 10 seconds, the client will get disconnected immediately 1014 | # if the size of the output buffers reach 32 megabytes, but will also get 1015 | # disconnected if the client reaches 16 megabytes and continuously overcomes 1016 | # the limit for 10 seconds. 1017 | # 1018 | # By default normal clients are not limited because they don't receive data 1019 | # without asking (in a push way), but just after a request, so only 1020 | # asynchronous clients may create a scenario where data is requested faster 1021 | # than it can read. 1022 | # 1023 | # Instead there is a default limit for pubsub and slave clients, since 1024 | # subscribers and slaves receive data in a push fashion. 1025 | # 1026 | # Both the hard or the soft limit can be disabled by setting them to zero. 1027 | client-output-buffer-limit normal 0 0 0 1028 | client-output-buffer-limit slave 256mb 64mb 60 1029 | client-output-buffer-limit pubsub 32mb 8mb 60 1030 | 1031 | # Redis calls an internal function to perform many background tasks, like 1032 | # closing connections of clients in timeout, purging expired keys that are 1033 | # never requested, and so forth. 1034 | # 1035 | # Not all tasks are performed with the same frequency, but Redis checks for 1036 | # tasks to perform according to the specified "hz" value. 1037 | # 1038 | # By default "hz" is set to 10. Raising the value will use more CPU when 1039 | # Redis is idle, but at the same time will make Redis more responsive when 1040 | # there are many keys expiring at the same time, and timeouts may be 1041 | # handled with more precision. 1042 | # 1043 | # The range is between 1 and 500, however a value over 100 is usually not 1044 | # a good idea. Most users should use the default of 10 and raise this up to 1045 | # 100 only in environments where very low latency is required. 1046 | hz 10 1047 | 1048 | # When a child rewrites the AOF file, if the following option is enabled 1049 | # the file will be fsync-ed every 32 MB of data generated. This is useful 1050 | # in order to commit the file to the disk more incrementally and avoid 1051 | # big latency spikes. 1052 | aof-rewrite-incremental-fsync yes 1053 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo 'Starting redis-server ...' 4 | /usr/bin/redis-server /etc/redis/redis.conf 5 | 6 | echo 'Updating GeoIP data using geoipupdate ...' 7 | /usr/bin/geoipupdate 8 | 9 | # FIXME: Even though these settings are set here, it doesn't prevent 10 | # oversize packet segementation problems, i.e., "invald packet received" 11 | # messages in the ntopng log like it's supposed to. 12 | # 13 | # Parse the interface name(s) out of the ntopng config file and 14 | # turn off oversize packet handling on each. 15 | LIST=$(grep -e "^\s*-i=" /etc/ntopng/ntopng.conf | cut -d"=" -f2) 16 | for IF in $LIST; do 17 | ethtool -K $IF gro off gso off tso off 18 | done 19 | 20 | echo 'Starting ntopng server ...' 21 | /usr/bin/ntopng /etc/ntopng/ntopng.conf 22 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/README.md: -------------------------------------------------------------------------------- 1 | These files provide the necessary support to build RPM packages for ntopng. Packages may be built for both the x86_64 and aarch64 (arm64) architectures. 2 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/lua-makefile-add-fPIC.patch: -------------------------------------------------------------------------------- 1 | --- ntopng/third-party/lua-5.4.3/src/Makefile.orig 2023-02-04 08:33:09.462114280 -0500 2 | +++ ntopng/third-party/lua-5.4.3/src/Makefile 2023-02-04 08:30:16.085688336 -0500 3 | @@ -20,7 +20,7 @@ 4 | SYSLDFLAGS= 5 | SYSLIBS= 6 | 7 | -MYCFLAGS= 8 | +MYCFLAGS= -fPIC 9 | MYLDFLAGS= 10 | MYLIBS= 11 | MYOBJS= 12 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/ntopng.conf: -------------------------------------------------------------------------------- 1 | # The configuration file is similar to the command line, with the exception that an equal 2 | # sign '=' must be used between key and value. Example: -i=p1p2 or --interface=p1p2 For 3 | # options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used. 4 | # 5 | # 6 | # -G|--pid-path 7 | # Specifies the path where the PID (process ID) is saved. This option is ignored when 8 | # ntopng is controlled with systemd (e.g., service ntopng start). 9 | # 10 | # -G=/var/run/ntopng.pid 11 | # 12 | # -e|--daemon 13 | # This parameter causes ntop to become a daemon, i.e. a task which runs in the background 14 | # without connection to a specific terminal. To use ntop other than as a casual monitoring 15 | # tool, you probably will want to use this option. This option is ignored when ntopng is 16 | # controlled with systemd (e.g., service ntopng start) 17 | # 18 | # -e= 19 | # 20 | # -i|--interface 21 | # Specifies the network interface or collector endpoint to be used by ntopng for network 22 | # monitoring. On Unix you can specify both the interface name (e.g. lo) or the numeric 23 | # interface id as shown by ntopng -h. On Windows you must use the interface number instead. 24 | # Note that you can specify -i multiple times in order to instruct ntopng to create multi- 25 | # ple interfaces. 26 | # 27 | # -i=eth1 28 | # -i=eth2 29 | # 30 | # -w|--http-port 31 | # Sets the HTTP port of the embedded web server. 32 | # 33 | # -w=3000 34 | # 35 | # -m|--local-networks 36 | # ntopng determines the ip addresses and netmasks for each active interface. Any traffic on 37 | # those networks is considered local. This parameter allows the user to define additional 38 | # networks and subnetworks whose traffic is also considered local in ntopng reports. All 39 | # other hosts are considered remote. If not specified the default is set to 192.168.1.0/24. 40 | # 41 | # Commas separate multiple network values. Both netmask and CIDR notation may be used, 42 | # even mixed together, for instance "131.114.21.0/24,10.0.0.0/255.0.0.0". 43 | # 44 | # -m=10.10.123.0/24 45 | # -m=10.10.124.0/24 46 | # 47 | # -n|--dns-mode 48 | # Sets the DNS address resolution mode: 0 - Decode DNS responses and resolve only local 49 | # (-m) numeric IPs 1 - Decode DNS responses and resolve all numeric IPs 2 - Decode DNS 50 | # responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't resolve 51 | # 52 | # -n=1 53 | # 54 | # -S|--sticky-hosts 55 | # ntopng periodically purges idle hosts. With this option you can modify this behaviour by 56 | # telling ntopng not to purge the hosts specified by -S. This parameter requires an argu- 57 | # ment that can be "all" (Keep all hosts in memory), "local" (Keep only local hosts), 58 | # "remote" (Keep only remote hosts), "none" (Flush hosts when idle). 59 | # 60 | # -S= 61 | # 62 | # -d|--data-dir 63 | # Specifies the data directory (it must be writable by the user that is executing ntopng). 64 | # 65 | # -d=/var/lib/ntopng 66 | # 67 | # -q|--disable-autologout 68 | # Disable web interface logout for inactivity. 69 | # 70 | # -q= 71 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/ntopng.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=ntopng Network Traffic Analyser 3 | Requires=redis.service 4 | After=syslog.target network.target redis.service 5 | 6 | [Service] 7 | EnvironmentFile=/etc/sysconfig/ntopng 8 | ExecStart=/usr/bin/ntopng $OPTIONS 9 | Restart=always 10 | RestartSec=30 11 | TimeoutStopSecs=10s 12 | 13 | [Install] 14 | WantedBy=multi-user.target 15 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/ntopng.spec: -------------------------------------------------------------------------------- 1 | # This Source Code Form is subject to the terms of the Mozilla Public 2 | # License, v. 2.0. If a copy of the MPL was not distributed with this 3 | # file, You can obtain one at https://mozilla.org/MPL/2.0/. 4 | 5 | %define debug_package %{nil} 6 | 7 | Name: ntopng 8 | #Version: 5.6 9 | Version: 5.7.%(date +%y%m%d) 10 | Release: 1%{?dist} 11 | Summary: A next generation network packet traffic probe used for high-speed web-based traffic analysis and flow collection. 12 | 13 | License: GPLv3.0 14 | URL: https://ntop.org 15 | Source0: %{name}.tar.gz 16 | Source1: nDPI.tar.gz 17 | Source2: %{name}.conf 18 | Source3: %{name}.service 19 | Source4: %{name}.sysconfig 20 | Patch0: lua-makefile-add-fPIC.patch 21 | 22 | Requires: geoipupdate, glib2, hiredis, libgcc, libpcap, libxml2, net-tools, openssl, redis, sqlite, zlib 23 | 24 | BuildRequires: autoconf, automake, expat-devel, gcc-c++, git, json-c-devel, hiredis-devel, jsoncpp-devel, kernel-devel, libcap-devel, libcurl-devel, libmaxminddb-devel, libpcap-devel, libsqlite3x-devel, libtool, libxml2-devel, make, mariadb-devel, openssl-devel, pkg-config, readline-devel, rrdtool-devel, zeromq-devel 25 | 26 | %description 27 | ntopng is a passive network monitoring tool focused on flows and statistics that 28 | can be obtained from the traffic captured by the server. 29 | 30 | %prep 31 | tar -zxvf %{SOURCE0} 32 | tar -zxvf %{SOURCE1} 33 | 34 | %patch0 35 | 36 | %build 37 | cd nDPI 38 | ./autogen.sh 39 | make -j 40 | cd ../%{name} 41 | ./autogen.sh 42 | ./configure 43 | make -j 44 | 45 | %install 46 | mkdir -p %{buildroot}%{_bindir} 47 | install -m 755 -t %{buildroot}%{_bindir} ntopng/ntopng 48 | mkdir -p %{buildroot}/usr/man/man8 49 | install -m 644 -t %{buildroot}/usr/man/man8 ntopng/ntopng.8 50 | mkdir -p %{buildroot}/etc/ntopng 51 | install -m 644 -t %{buildroot}/etc/ntopng %{SOURCE2} 52 | mkdir -p %{buildroot}/usr/lib/systemd/system 53 | install -m 644 -t %{buildroot}/usr/lib/systemd/system %{SOURCE3} 54 | mkdir -p %{buildroot}/etc/sysconfig/ 55 | install -m 644 %{SOURCE4} %{buildroot}/etc/sysconfig/%{NAME} 56 | mkdir -p %{buildroot}/usr/share/ntopng 57 | cp -r ntopng/httpdocs %{buildroot}/usr/share/ntopng 58 | cp -LR ntopng/scripts %{buildroot}/usr/share/ntopng 59 | find %{buildroot}/usr/share/ntopng -name "*~" | xargs /bin/rm -f 60 | find %{buildroot}/usr/share/ntopng -name ".git" | xargs /bin/rm -rf 61 | 62 | %files 63 | /etc/ntopng/ntopng.conf 64 | /etc/sysconfig/ntopng 65 | /usr/bin/ntopng 66 | /usr/lib/systemd/system/ntopng.service 67 | /usr/man/man8/ntopng.8.gz 68 | /usr/share/ntopng/* 69 | 70 | %post 71 | # Create self-signed SSL certificate 72 | cd /usr/share/ntopng/httpdocs/ssl 73 | openssl req -new -x509 -sha1 -extensions v3_ca -nodes -days 3650 -out cert.pem -subj "/O=ntopng container" &>/dev/null 74 | cat privkey.pem cert.pem > ntopng-cert.pem 75 | 76 | %preun 77 | # Delete the files that the %post scriptlet created 78 | cd /usr/share/ntopng/httpdocs/ssl 79 | rm privkey.pem cert.pem ntopng-cert.pem 80 | 81 | %changelog 82 | * Sat Feb 04 2023 David King 83 | Update version number to 5.7.x (unstable) 84 | * Sat Feb 04 2023 David King 85 | Update version number to 5.6.0 (stable) 86 | Patch ntopng to allow compiling on Fedora 37 87 | * Tue Aug 09 2022 David King 88 | Update version number to 5.5.x (unstable) 89 | * Tue Apr 26 2022 David King 90 | Add hiredis package dependency 91 | * Sat Feb 05 2022 David King 92 | Update version number to 5.3.x (unstable) 93 | * Thu Feb 03 2022 David King 94 | Update version number to 5.2.1 (stable) 95 | * Tue Sep 28 2021 David King 96 | Update version number to 5.1.x 97 | * Mon Mar 22 2021 David King 98 | Remove @BIN_DIR@ patch 99 | * Sun Mar 07 2021 David King 100 | Add @BIN_DIR@ patch 101 | * Sat Jan 30 2021 David King 102 | Initial Version 103 | -------------------------------------------------------------------------------- /deprecated/ntopng-udm-fedora/rpmbuild/ntopng.sysconfig: -------------------------------------------------------------------------------- 1 | # Command line options for ntopng... 2 | OPTIONS="/etc/ntopng/ntopng.conf" 3 | -------------------------------------------------------------------------------- /deprecated/on_boot.d/05-profile.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | echo >>/etc/profile.d/aliases.sh 4 | echo "# Added by /mnt/data/on_boot.d/05-profile.sh" >>/etc/profile.d/aliases.sh 5 | echo "alias ll='ls -l'" >>/etc/profile.d/aliases.sh 6 | -------------------------------------------------------------------------------- /deprecated/on_boot.d/15-add-root-ssh-key.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | MY_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YrKZDC2p65DHSwhah/FbhMEv6LcIOZAoeeygNMEwgfw3DtGYDgtwo8A0QhnhQLhcFRRpGGW66v2uAlJFibhJ2GRXsH6h/vW+ewkUq5sC+ngWGOFMHWZZD0Bol75ZEKzzygIQvyVN+o59Qxut40khy58WqOu5X6NGl2C7tEtp4RMFV72ap0W/uTvEi25MvaouhMnLz+2tbLH+ycNiswZI0qBZQx1oSDzVdfbiAv0AqceyY+2MLsrl8GAt17b4fq8FFwOJcV+S3P9EkjTqeXjrLGzYMB0rr/jokVhPJwnciLYD9QOovGBwqQfDNvuTL8G2VWJGWZHkuXJOIRdho7vRQ== dlk@fang.localdomain" 4 | KEYS_FILE="/root/.ssh/authorized_keys" 5 | 6 | # Places public key in ~/.ssh/authorized_keys if not present 7 | if ! grep -Fxq "$MY_SSH_KEY" "$KEYS_FILE"; then 8 | echo "$MY_SSH_KEY" >> "$KEYS_FILE" 9 | fi 10 | -------------------------------------------------------------------------------- /deprecated/on_boot.d/15-nameservers.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | echo "search localdomain" >/etc/resolv.conf 4 | echo "nameserver 192.168.1.40" >>/etc/resolv.conf 5 | echo "nameserver 192.168.1.21" >>/etc/resolv.conf 6 | -------------------------------------------------------------------------------- /deprecated/on_boot.d/15-start-ntopng.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | DATADIR="/mnt/data/ntopng" 4 | 5 | # Allow this script to be run as the "ntopng" command 6 | ln -s /mnt/data/on_boot.d/15-start-ntopng.sh /usr/bin/ntopng &>/dev/null 7 | 8 | # Start existing container 9 | echo "Trying to start existing container instance ..." 10 | podman start ntopng 11 | 12 | # If that failed, try to start a new container 13 | if [ $? -eq 125 ]; then 14 | echo "Trying to start a new container instance ..." 15 | podman run -d --net=host --privileged --restart always --name ntopng \ 16 | -v ${DATADIR}/GeoIP.conf:/etc/GeoIP.conf \ 17 | -v ${DATADIR}/ntopng.conf:/etc/ntopng/ntopng.conf \ 18 | -v ${DATADIR}/redis.conf:/etc/redis/redis.conf \ 19 | -v ${DATADIR}/lib:/var/lib/ntopng \ 20 | -v ${DATADIR}/redis:/var/lib/redis \ 21 | -v /etc/localtime:/etc/localtime:ro \ 22 | -v ${DATADIR}/ssl/ntopng-cert.pem:/usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem \ 23 | ntopng-udm:latest 24 | fi 25 | -------------------------------------------------------------------------------- /deprecated/on_boot.d/50-cronjobs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Add custom cron jobs 4 | 5 | DUCKDNS_HOSTNAME="" 6 | DUCKDNS_TOKEN="" 7 | 8 | # Run cron job commands at boot time 9 | curl -o /dev/null -s "https://www.duckdns.org/update?domains=$DUCKDNS_HOSTNAME&token=$DUCKDNS_TOKEN" 10 | /usr/bin/podman exec -it unifi-os /data/dlk-udm-hacks/away-recording -q 11 | 12 | # Create my custom cron.d file 13 | cat </etc/cron.d/udm-hacks-cronjobs 14 | 0 * * * * curl -o /dev/null -s "https://www.duckdns.org/update?domains=$DUCKDNS_HOSTNAME&token=$DUCKDNS_TOKEN" 15 | */10 * * * * /usr/bin/podman exec -it unifi-os /data/dlk-udm-hacks/away-recording -q 16 | EOF 17 | 18 | # Restart the cron daemon to pick up my changes 19 | /etc/init.d/crond restart 20 | -------------------------------------------------------------------------------- /gen-debian-repo-key: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | if [ $# != 1 ]; then 4 | echo "One parameter required = the name of the PPA repository" 5 | exit 1 6 | fi 7 | 8 | EMAIL="${1}@daveking.com" 9 | 10 | cat << __EOF__ > /tmp/pgp-key.batch 11 | %echo Generating a PGP key for ${1} 12 | Key-Type: RSA 13 | Key-Length: 4096 14 | Name-Real: $1 15 | Name-Email: $EMAIL 16 | Expire-Date: 0 17 | %no-ask-passphrase 18 | %no-protection 19 | %commit 20 | __EOF__ 21 | 22 | export GNUPGHOME="$(pwd)/.gnupg" 23 | rm -fr $GNUPGHOME 24 | if [ ! -d "$GNUPGHOME" ]; then 25 | mkdir -p "$GNUPGHOME" 26 | chmod 700 "$GNUPGHOME" 27 | fi 28 | gpg --no-tty --batch --gen-key /tmp/pgp-key.batch 29 | echo "default-key $EMAIL" >"${GNUPGHOME}/gpg.conf" 30 | gpg --armor --export "$EMAIL" >pgp-key.public 31 | rm /tmp/pgp-key.batch 32 | -------------------------------------------------------------------------------- /ppa/blacklists.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Block Internet Traffic From The IP Addresses In The ntopng Blacklists 6 | 7 | 8 |

Block Internet Traffic From The IP Addresses In The ntopng Blacklists

9 | 10 |

The script in this package configures iptables on the UDM Pro to block all traffic from the IP addresses that are listed in the active ntopng blacklists. The script is executed through systemd once a day to refresh the UDM blacklist. Only the active ntopng blacklists are used. See the Settings -> Blacklists menu in ntopng to configure the active lists.

11 | 12 |

Installing the blacklists Debian package

13 | 14 |

To enable my PPA repository on the UDM, use ssh to connect to the UDM as the root user and issue these commands:

15 | 16 | 
curl -s --compressed "https://daveking.com/udm-hacks/pgp-key.public" | gpg --dearmor | tee /etc/apt/trusted.gpg.d/udm-hacks.gpg >/dev/null
17 | curl -s --compressed -o /etc/apt/sources.list.d/udm-hacks.list "https://daveking.com/udm-hacks/udm-hacks.list"
18 | apt update
19 | 20 |

Use this command to install this package on the UDM:

21 | 
apt install blacklists
22 | 23 |

Running the Script

24 | 25 |

Systemd controls the execution of the script. As installed, the script runs every day at 4 AM. To adjust this timer, adjust the OnCalendar setting in the /etc/systemd/system/blacklists.timer file.

26 |

Systemd will also run the script 2 minutes after the UDM is rebooted to regenerate the blacklist in iptables.

27 | 28 |

Useful Commands

29 | 30 |

After logging in to the UDM Pro via ssh:

31 | 32 |
    33 |
  • journalctl -e -g "NTOPNG BLOCK" - lists the system log entries for the packets that iptables has blocked as a result of using this script. 34 |

    Limits are set for logging. If the UDM is flooded with packets coming from blacklisted addresses a maximum of 5 packets per minute will be logged. The rest will be blocked silently, without a log entry.

    • 35 | 36 |
  • ipset list ntopng - lists all the addresses and networks that are blacklisted by iptables as a result of using this script. 37 |

    This "ntopng" ipset gets rebuilt evey night by the blacklists script, based on the contents of the ntopng blacklists, which ntopng updates every day.

    • 38 | 39 |
  • ipset test ntopng ipaddress - test to see if a given IP address is in the "ntopng" ipset and will therefore be blocked by iptables.
    • 40 | 41 |
  • ipset del ntopng ipaddress[/netmask] - temporarily unblocks an IP address or network by removing it from the "ntopng" ipset. 42 |

    To prevent the address being blocked again when the blacklists script rebuilds the ipset, add the address to the list of excluded IP addresses called excluded_addresses in the /data/blacklists/blacklists.conf file.

    • 43 | 44 |
  • iptables -nvL | less - shows the iptables rules configuration. Searching for the string "NTOPNG" will locate all the rules inserted by this script.
    • 45 |
46 | -------------------------------------------------------------------------------- /ppa/camera_poe_ctl.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Turn Off Selected Ubiquiti Dream Machine Pro (UDM Pro) Cameras Based on SmartThings Location Status 6 | 7 | 8 |

Turn Off Selected Ubiquiti Dream Machine Pro (UDM Pro) Cameras Based on SmartThings Location Status

9 | 10 |

For privacy reasons we want to keep our indoor cameras turned off unless nobody is home and the house is empty. We have already set up SmartThings to use our mobile devices as sensors so that it knows when we are away from home. I have defined a SmartThings routine that switches our "Home" location to "Away" mode when it senses that we are away. The script in this package queries the SmartThings location mode and turns on the cameras that are listed in its configuration file when we are away and turns them back off when we arrive back home.

11 | 12 |

Installing the camera-poe-ctl Debian package

13 | 14 |

To enable my PPA repository on the UDM, use ssh to connect to the UDM as the root user and issue these commands:

15 | 16 | 
curl -s --compressed "https://daveking.com/udm-hacks/pgp-key.public" | gpg --dearmor | tee /etc/apt/trusted.gpg.d/udm-hacks.gpg >/dev/null
17 | curl -s --compressed -o /etc/apt/sources.list.d/udm-hacks.list "https://daveking.com/udm-hacks/udm-hacks.list"
18 | apt update
19 | 20 |

Use this command to install this package on the UDM:

21 | 
apt install camera-poe-ctl
22 | 23 |

IMPORTANT: This script uses a configuration file that must be customized with your information before the script will work properly. If your UDM has the secondary disk drive installed, then the configuration file should be in /volume1/camera_poe_ctl/camera_poe_ctl.conf. If not, it should be in /data/camera_poe_ctl/camera_poe_ctl.conf. A sample configuration file can be found in those same directories. See the comments in the sample file for guidance on the information that is needed.

24 | 25 |

Running the Script

26 | 27 |

Systemd controls the execution of the script. As installed, the script runs every 10 minutes. To adjust this interval, adjust the OnCalendar setting in the /etc/systemd/system/camera_poe_ctl.timer file.

28 | -------------------------------------------------------------------------------- /ppa/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Ubiquiti Dream Machine Pro Hacks 8 | 9 | 10 |

Ubiquiti Dream Machine Pro Hacks

11 |

This is a Debian PPA repository containing packages for my hacks for the UDM Pro. My hacks include:

12 | 16 |

Using this repository on the UDM Pro

17 |

To enable this repository on the UDM Pro, use ssh to connect to the UDM as the root user and issue these commands:

18 | 
curl -s --compressed "https://daveking.com/udm-hacks/pgp-key.public" | gpg --dearmor | tee /etc/apt/trusted.gpg.d/udm-hacks.gpg >/dev/null
19 | curl -s --compressed -o /etc/apt/sources.list.d/udm-hacks.list "https://daveking.com/udm-hacks-repo/udm-hacks.list"
20 | apt update
21 | 22 | 23 | -------------------------------------------------------------------------------- /ppa/markdown.css: -------------------------------------------------------------------------------- 1 | h1 { 2 | margin: 5px 0 20px 0; 3 | } 4 | h2, h3 { 5 | margin: 10px 0 15px 0; 6 | } 7 | img { 8 | max-width: 100%; 9 | } 10 | pre { 11 | background-color: #f1f1f1; 12 | border-radius: 5px; 13 | padding: 10px; 14 | } 15 | pre > code { 16 | padding: 0; 17 | } 18 | table { 19 | border-spacing: 0; 20 | border-style: solid; 21 | border-width: 1px; 22 | border-collapse: collapse; 23 | margin-top: 0.5em; 24 | } 25 | th, td { 26 | padding: 2px 5px; 27 | } 28 | a { 29 | color: #FF9137; 30 | text-decoration: none; 31 | } 32 | pre, code { 33 | font-family: "Source Code Pro"; 34 | font-weight: normal; 35 | } 36 | pre { 37 | display: block; 38 | background-color: #f1f1f1; 39 | white-space: no-wrap; 40 | overflow-x: auto; 41 | } 42 | code { 43 | padding: 3px; 44 | overflow: auto; 45 | line-height: 1.45em; 46 | background-color: #f1f1f1; 47 | border-radius: 5px; color: #000000; 48 | } 49 | .code-comment { 50 | color: #75715E; 51 | font-style: italic; 52 | } 53 | .code-string { 54 | color: #E6DB74; 55 | } 56 | .code-literal { 57 | color: #AE81FF; 58 | } 59 | .code-type { 60 | color: #66D9EF; 61 | } 62 | .code-builtin { 63 | color: #A6E22E; 64 | } 65 | .code-keyword { 66 | color: #F92672; 67 | } 68 | .code-other { 69 | color: #F92672; 70 | } 71 | body { 72 | font-family: Cantarell; 73 | font-weight: normal; 74 | font-size: 11pt; 75 | max-width: 1080px; 76 | margin: 0 auto; 77 | } 78 | -------------------------------------------------------------------------------- /ppa/ntopng.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Running ntopng On The Unifi Dream Machine Pro (UDM-Pro) 6 | 7 | 8 |

Running ntopng On The Unifi Dream Machine Pro (UDM-Pro)

9 | 10 |

This repository no longer contains any ntopng package files. Ntop now provides stable and unstable builds 11 | that will run on the UDM. See their installation instructions for the Raspian/rPi OS build. 12 | The "buster_pi (arm32)" version there is what will install in the Debian (Buster) 11.7 OS on the UDM. (As of 2024-01-16)

13 | 14 | -------------------------------------------------------------------------------- /ppa/pgp-key.public: -------------------------------------------------------------------------------- 1 | -----BEGIN PGP PUBLIC KEY BLOCK----- 2 | 3 | mQINBGP+JTIBEADgfRYWr3GrFdy6q0ih4RTkNBX48mcA7+oECJppB/m2cViZjd3C 4 | kg7bnn91oBG3ajQF1EnySEw3XSW5hHrx1AJ9nXBwhf8v28+eken+u0rPFxE3aPeI 5 | c2KDBNk905j8lGhd/VwdKuzZ8XkKh6Bhsl/x7peIvrGhZC1mco0OMw6mbPdTnsmS 6 | bavHjBmby0tUGf8rFo6IWs8TBRocVoSBrNWN1RQqsmeOOEQmJa9AMDBbRzDCJGXW 7 | HtM7kZydCQqc/bZbh+jI/WA+AUmGObcQaUzWRdmp37QiOCQe+3oHmBhcHsOTwpYm 8 | bKhdQ7Ylb1c7y2uihSjN5qzUmkjk97ReGkQvqSalCimQjalppDG/MaX9JaVMXHJU 9 | 8V4Hb+cO6HWncfWlnEHvR4JWNaA4QgcQhL/+1gIJytXR/i7Dor0lETNxkBng53RW 10 | uS7xi1MqVV6ao8nJ+u4I7nnQ/77RNQT9iPGjwHclXPG+aweQ9jcwlDVD5rB6uG9b 11 | Aa9nQuO5zZEAVOr/EL4y1OKLsPxsRcBPUuSwcveGwvSEqOTTIZhqa9KJH8WowSDq 12 | 52ZwCZNvGG6iupS5A8/auSN4xM8ia5PGqTgjZJtop6wPGOu4G2uVpyxh8PYjEcx7 13 | 7tgy3hXMVmgnQzhCLRtfleZAoduBXjILO+v87H/cUbGqLb7DoG4PfWA2DwARAQAB 14 | tCx1ZG0taGFja3MtcmVwbyA8dWRtLWhhY2tzLXJlcG9AZGF2ZWtpbmcuY29tPokC 15 | UQQTAQgAOxYhBMgVHx0wDji86QcLSzySDVhUVnFNBQJj/iUyAhsvBQsJCAcCAiIC 16 | BhUKCQgLAgQWAgMBAh4HAheAAAoJEDySDVhUVnFNw4IP/jtu4fDYQjOJLMSr+S2B 17 | dndLHi6UYYssi6JarfMXfmhhPQXHQov2A/o8XB1rYw9nGvgEAN/+dv3GJFpAYHKH 18 | iAUyjgZFXDDC+phmQBXtJ8Ko49+qZ+WejybnxbfdngjpEbiYt5JxDrD38v7siSnh 19 | wuep1Iq7zWaV2LUgIbhO32EqeRlpEOr+q7a+EAxU8hxeiOv3k2OM6VcrS4ElaUTD 20 | gdAraYg83ysdGqWM4CIUJiyNO8yu1eVhWOfVtGXoQPPljIpqCelmJ2AJS4foot+F 21 | mV2FcFDsXHdVi6xk6KdndRIwsvGzrQty1gU4DioZYozJHocWhCJx3JIDF4+yrONb 22 | x0oPNLyTYg/oZrv3QMzMXp7auATSXQGqOO7SvBafxYHSpQYsxCPHYPEou9evafax 23 | BvYo7cjzTt8cQlwnEuOsapvbz6T+eceFnzch31FMe1f//qVogpGyYUVA5xR1voce 24 | FL2aXlb7qPY24D+UZEFlFzCzmiMJXooIZMrBzWFD4Vzu9Dmax3tw7qfKN3SUp6no 25 | KdujvysUcD2118WyFteP22XcMGjNAnDMp3dNAq5TV0+26/togdAM/PTXRuhXq5cs 26 | yF/N7caH7C1HHmSl1ntAiUqk7fJJEtIWf7WnT2n9OZO1SO46UHozXy8Y1d4DC32t 27 | hLGDcYe5T4CgeTByfoy5pUV6 28 | =Ef6N 29 | -----END PGP PUBLIC KEY BLOCK----- 30 | -------------------------------------------------------------------------------- /ppa/udm-hacks-stable.list: -------------------------------------------------------------------------------- 1 | deb [signed-by=/etc/apt/trusted.gpg.d/udm-hacks.gpg] https://daveking.com/udm-hacks stable main 2 | -------------------------------------------------------------------------------- /ppa/udm-hacks.list: -------------------------------------------------------------------------------- 1 | deb [signed-by=/etc/apt/trusted.gpg.d/udm-hacks.gpg] https://daveking.com/udm-hacks unstable main 2 | -------------------------------------------------------------------------------- /restore_dlk_apps: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Recover my custom apps setup on the UDM Pro 4 | 5 | set -e 6 | 7 | # Add repos for my hacks 8 | curl -s --compressed "https://daveking.com/udm-hacks/pgp-key.public" | gpg --dearmor | tee /etc/apt/trusted.gpg.d/udm-hacks.gpg >/dev/null 9 | curl -s --compressed -o /etc/apt/sources.list.d/udm-hacks.list "https://daveking.com/udm-hacks/udm-hacks.list" 10 | 11 | # Install ntopng 12 | rm apt-ntop.deb* | true 13 | wget https://packages.ntop.org/RaspberryPI/apt-ntop.deb 14 | dpkg -i apt-ntop.deb 15 | rm apt-ntop.deb 16 | 17 | # Install my hacks 18 | apt update 19 | apt install jq tree camera-poe-ctl ntopng 20 | 21 | # Install ntopng custom script 22 | cp /volume1/ntopng-extras/process_alerts.py /usr/share/ntopng/scripts/shell/ 23 | chmod 755 /usr/share/ntopng/scripts/shell/process_alerts.py 24 | 25 | # Start services 26 | systemctl restart redis 27 | sleep 5 28 | systemctl restart ntopng 29 | systemctl enable --now camera_poe_ctl.timer 30 | 31 | # Restore commands to path 32 | ln -s /volume1/suricata-util /usr/local/bin/suricata-util 33 | -------------------------------------------------------------------------------- /update_ppa: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | # Update the PPA repository 4 | 5 | # Local copy of PPA contents 6 | PPADIR=~/src/udm-hacks/ppa 7 | 8 | # Terminate immediately on error 9 | set -e 10 | 11 | # Date/time format for console messages 12 | DATEFMT="+%Y-%m-%d %H:%M:%S" 13 | 14 | # Fix any file ownership issues 15 | sudo chown -R dlk:dlk "$PPADIR" 16 | 17 | # Keep only one version of each package 18 | echo "$(date "$DATEFMT"): Deleting old packages" 19 | prune() { 20 | cd "$1" 21 | PREV_NAME="" 22 | PREV_VERSION="" 23 | for f in $(ls -r *.deb); do 24 | NAME=$(echo "$f" | cut -f1 -d_) 25 | VERSION=$(echo "$f" | cut -f2 -d_) 26 | VERSION=${VERSION%.*} 27 | if [ "$NAME" == "$PREV_NAME" ] && [ "$VERSION" == "$PREV_VERSION" ]; then 28 | echo "$(date "$DATEFMT"): Deleting $f" 29 | rm "$f" &>/dev/null || true 30 | else 31 | PREV_NAME="$NAME" 32 | PREV_VERSION="$VERSION" 33 | fi 34 | done 35 | } 36 | prune "${PPADIR%%/}/pool/main/b/blacklists" 37 | prune "${PPADIR%%/}/pool/main/c/camera-poe-ctl" 38 | 39 | # Use the repository's signing key 40 | export GNUPGHOME=${PPADIR}/../.gnupg 41 | 42 | for SUBDIR in unstable stable; do 43 | # Create the Packages file in the repo 44 | echo "$(date "$DATEFMT"): Creating Packages file in ${PPADIR%%/dists/$SUBDIR}" 45 | cd "${PPADIR}" 46 | dpkg-scanpackages --arch arm64 pool/main/b/blacklists >dists/${SUBDIR}/main/binary-arm64/Packages 47 | dpkg-scanpackages --arch arm64 pool/main/c/camera-poe-ctl >>dists/${SUBDIR}/main/binary-arm64/Packages 48 | cat dists/${SUBDIR}/main/binary-arm64/Packages | gzip -9 >dists/${SUBDIR}/main/binary-arm64/Packages.gz 49 | 50 | # Create the Release file in the repo 51 | echo "$(date "$DATEFMT"): Creating Release file in ${PPADIR%%/}/dists/$SUBDIR" 52 | cd "${PPADIR%%/}/dists/$SUBDIR" 53 | do_hash() { 54 | HASH_NAME=$1 55 | HASH_CMD=$2 56 | echo "${HASH_NAME}:" 57 | for f in $(find -type f); do 58 | f=$(echo $f | cut -c3-) # remove ./ prefix 59 | if [[ $f == Release* ]] || [[ $f == InRelease* ]]; then 60 | continue 61 | fi 62 | echo " $(${HASH_CMD} ${f} | cut -d" " -f1) $(wc -c $f)" 63 | done 64 | } 65 | 66 | cat << EOF > Release 67 | Origin: udm-hacks Repository 68 | Label: udm-hacks 69 | Suite: $SUBDIR 70 | Codename: $SUBDIR 71 | Version: 1.0 72 | Architectures: arm64 73 | Components: main 74 | Description: My hacks for the Ubiquiti Dream Machine 75 | Date: $(date -Ru) 76 | EOF 77 | do_hash "MD5Sum" "md5sum" >> Release 78 | do_hash "SHA1" "sha1sum" >> Release 79 | do_hash "SHA256" "sha256sum" >> Release 80 | 81 | # Sign the Release file 82 | echo "$(date "$DATEFMT"): Signing the Release file" 83 | cd "${PPADIR%%/}/dists/$SUBDIR" 84 | cat Release | gpg -abs >Release.gpg 85 | cat Release | gpg -abs --clearsign >InRelease 86 | done 87 | 88 | echo "$(date "$DATEFMT"): Sync local repo with remote web site" 89 | rsync --archive --partial --progress --delete ${PPADIR%%/}/ daveking.com:/opt/udm-hacks/ 90 | echo "$(date "$DATEFMT"): Sync complete" 91 | --------------------------------------------------------------------------------