├── .gitattributes ├── LICENSE ├── README.md └── WebHistory_Report.sh /.gitattributes: -------------------------------------------------------------------------------- 1 | # Ensure all SH/MD/TXT files use LF. 2 | 3 | *.yml eol=lf 4 | 5 | *.sh eol=lf 6 | 7 | *.md eol=lf 8 | 9 | *.txt eol=lf 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # WebHistory-Report 2 | ASUS TrendMicro Web History Database Reporting 3 | -------------------------------------------------------------------------------- /WebHistory_Report.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | VER="v1.17" 3 | #======================================================================================= © 2016-2023 Martineau v1.17 4 | # 5 | # Scan Web History database 6 | # 7 | # WebHistory_Report [help | -h] ['ip='{[ip_address[,...] | hostname[...]]} ['flush']] ['url='{url_string[,...]}] ['nofilter'] ['email'] ['mode=or'] ['noscript'] 8 | # ['date='[yyyy/mm/dd[,...]]] ['time='[hh:mm:ss[,...]]] ['sqldb='{database}] ['backup[=directory]'] ['purgeallreset'] ['count'] ['sortby='column] 9 | # WebHistory_Report ['mac='mac_address[,...]] ['report='{file_name}] ['nodisplay'] ['showsql'] 10 | # 11 | # WebHistory_Report 12 | # Will list 'Todays' URL entries in the Web History database containing strings 'facebook' OR 'youtube' 13 | # WebHistory_Report count 14 | # Will count 'Todays' URL entries in the Web History database containing strings 'facebook' OR 'youtube' 15 | # and will only display the result count. No records are displayed on screen. 16 | # WebHistory_Report nofilter 17 | # Will list ALL entries in the Web History database. 18 | # WebHistory_Report nofilter sortby=url 19 | # Will list ALL entries in the Web History database sorted by column URL 20 | # WebHistory_Report nofilter email 21 | # Will list ALL entries in the Web History database and will send an email with the results 22 | # WebHistory_Report nofilter report=WebReport.csv nodisplay 23 | # Will list ALL entries in the Web History database and will write them in .CSV format to file 'WebReport.csv' 24 | # and no records will be displayed on screen. 25 | # WebHistory_Report url=amazon,netflix 26 | # Will list URL entries in the Web History database containing strings either 'amazon' OR 'netflix' 27 | # WebHistory_Report ip=192.168.1.1 url=amazon,netflix time=18:,19: mode=or 28 | # Will list URL entries in the Web History database for 192.168.1.1 or between 18:00-19:59 or URLs as above 29 | # Without 'mode=or' then the databse records must match ALL three criteria 30 | # WebHistory_Report date=2017/02/30 31 | # Will list entries in the Web History database created on '30th Feb 2017' 32 | # NOTE: The date specification can be an abbreviation e.g. '2017/02' for records created in 'Feb 2017' 33 | # WebHistory_Report ip=10.88.8.123, 192.168.1.120-192.168.1.123, CAMERAS 34 | # Will list database entries for five devices, plus all IPs for 'CAMERAS' entry in '/jffs/configs/IPGroups' 35 | # NOTE: Only MAC addresses are stored in the database so if the devices are not 'reserved/static' 36 | # then the report could be inaccurate. 37 | # WebHistory_Report mac=de:ad:de:ad:de:ad 38 | # Will list entries in Web History database for MAC address 'de:ad:de:ad:de:ad' 39 | # WebHistory_Report ip=10.88.8.123 flush url="www.veryexpensiveshoes.com" 40 | # Will delete all URL 'www.veryexpensiveshoes.com' history for '10.88.8.123' Ha ha - wife mode eh? ;-) 41 | # WebHistory_Report ip=10.88.8.123 flush 42 | # Will delete all history for '10.88.8.123' 43 | # WebHistory_Report time=09: 44 | # Will list entries in the Web History database created between '09:00' to '09:59' 45 | # NOTE: A full time specification can be used e.g. '12:05:30' but the report may never find a match! 46 | # WebHistory_Report backup 47 | # The current Web History database will be backed up to Entware disk '/opt/var/WebHistory/' 48 | # WebHistory_Report backup=/mnt/MyDisk 49 | # The current Web History database will be backed up to '/mnt/Mydisk/WebHistory/' 50 | # WebHistory_Report sqldb=/opt/var/WebHistory/WebHistory.db-Backup-20180401-060000 51 | # The report/queries will be extracted from the archive/backup database '/opt/var/WebHistory/WebHistory.db-Backup-20180401-060000' 52 | # WebHistory_Report purgeallreset 53 | # The current Web History database is PURGED of ALL history!!!!! (NOTE: a backup is taken first ;-) 54 | # WebHistory_Report showsql 55 | # Debug the resulting filter(s) by showing the actual SQL query 56 | 57 | # To filter by additional criteria just use grep/awk etc. to apply additional filters 58 | # 59 | # e.g. ONLY show the Echo devices 'youtube' activity where the rule contains 'cats' and 'adopt' 60 | # [CODE]./WebHistory_Report.sh ip=10.88.8.18,10.88.8.17,10.88.8.16 | grep -F "youtube" | grep -E "cats" | grep "adopt"[CODE] 61 | 62 | # [URL="https://www.snbforums.com/threads/web-history-reporting-and-management-traffic-analyzer-aiprotection-monitor.49888/"]Web History Reporting and Management (Traffic Analyzer/Aiprotection Monitor)[/URL] 63 | 64 | Say(){ 65 | echo -e $$ $@ | logger -st "($(basename $0))" 66 | } 67 | SayT(){ 68 | echo -e $$ $@ | logger -t "($(basename $0))" 69 | } 70 | # 71 | # Print between line beginning with'#==' to first blank line inclusive 72 | ShowHelp() { 73 | /usr/bin/awk '/^#==/{f=1} f{print; if (!NF) exit}' $0 74 | } 75 | # shellcheck disable=SC2034 76 | ANSIColours() { 77 | cRESET="\e[0m";cBLA="\e[30m";cRED="\e[31m";cGRE="\e[32m";cYEL="\e[33m";cBLU="\e[34m";cMAG="\e[35m";cCYA="\e[36m";cGRA="\e[37m";cFGRESET="\e[39m" 78 | cBGRA="\e[90m";cBRED="\e[91m";cBGRE="\e[92m";cBYEL="\e[93m";cBBLU="\e[94m";cBMAG="\e[95m";cBCYA="\e[96m";cBWHT="\e[97m" 79 | aBOLD="\e[1m";aDIM="\e[2m";aUNDER="\e[4m";aBLINK="\e[5m";aREVERSE="\e[7m" 80 | aBOLDr="\e[21m";aDIMr="\e[22m";aUNDERr="\e[24m";aBLINKr="\e[25m";aREVERSEr="\e[27m" 81 | cWRED="\e[41m";cWGRE="\e[42m";cWYEL="\e[43m";cWBLU="\e[44m";cWMAG="\e[45m";cWCYA="\e[46m";cWGRA="\e[47m" 82 | cYBLU="\e[93;48;5;21m" 83 | xHOME="\e[H";xERASE="\e[K";xCSRPOS="\e[s";xPOSCSR="\e[u" 84 | } 85 | StatusLine() { 86 | 87 | local ACTION=$1 88 | local FLASH="$aBLINK" 89 | 90 | if [ "${ACTION:0:7}" != "NoANSII" ];then 91 | 92 | [ "${ACTION:0:7}" == "NoFLASH" ] && local FLASH= 93 | 94 | local TEXT=$2 95 | 96 | echo -en $xCSRPOS # Save current cursor position 97 | 98 | case $ACTION in 99 | *Clear*) echo -en ${xHOME}${cRESET}$xERASE;; 100 | *) echo -en ${xHOME}${aBOLD}${FLASH}${xERASE}$TEXT;; 101 | esac 102 | 103 | echo -en $xPOSCSR # Restore previous cursor position 104 | fi 105 | 106 | } 107 | # Function Parse(String delimiter(s) variable_names) 108 | Parse() { 109 | # 110 | # Parse "Word1,Word2|Word3" ",|" VAR1 VAR2 REST 111 | # (Effectivley executes VAR1="Word1";VAR2="Word2";REST="Word3") 112 | 113 | local string IFS 114 | 115 | TEXT="$1" 116 | IFS="$2" 117 | shift 2 118 | read -r -- "$@" < Insert favorite routine here 181 | #=================================> Insert favorite routine here 182 | #=================================> Insert favorite routine here 183 | 184 | Say "You need to edit this script and add the Sendmail function first!" 185 | 186 | return 0 187 | 188 | } 189 | ExpandIPRange() { 190 | 191 | # '192.168.1.30 192.168.1.50-192.168.1.54' -> '192.168.1.30 192.168.1.50 192.168.1.51 192.168.1.52 192.168.1.53 192.168.1.54' 192 | 193 | local HOST_NAME=0 # Hostname found/returned 194 | local IP_LIST= 195 | local START_RANGE= 196 | local END_RANGE= 197 | local NUM= 198 | local MAX= 199 | 200 | local LANIPADDR=`nvram get lan_ipaddr` 201 | local LAN_PREFIX=${LANIPADDR%.*} # 1.2.3.99 -> 1.2.3 202 | 203 | for THIS in $@ 204 | do 205 | 206 | if [ -n "$(echo "$THIS" | grep -E "^#")" ];then 207 | break # Ignore comment 208 | fi 209 | 210 | # If any alphabetic characters then assume it is a name e.g. LIFX-Table_light 211 | if [ -z "$(echo $THIS | grep "[A-Za-z]")" ];then 212 | 213 | if [ -n "$(echo $THIS | grep "-")" ];then 214 | 215 | Parse $THIS "-" START_RANGE END_RANGE # 1.2.3.90-1.2.3.99 -> 1.2.3.90 1.2.3.99 216 | local START_PREFIX=${START_RANGE%.*} # 1.2.3.90 -> 1.2.3 217 | local END_PREFIX=${END_RANGE%.*} # 1.2.3.99 -> 1.2.3 218 | 219 | if [ "$START_PREFIX" != "$END_PREFIX" ];then # Restrict range of devices to 254 220 | Say "***ERROR*** invalid IP range" $THIS 221 | echo "" 222 | return 100 223 | fi 224 | 225 | NUM=${START_RANGE##*.} # Extract 4th octet 1.2.3.90 -> 90 226 | MAX=${END_RANGE##*.} # Extract 4th octet 1.2.3.99 -> 99 227 | while [ $NUM -le $MAX ] 228 | do 229 | IP_LIST=$IP_LIST" "$START_PREFIX"."$NUM 230 | NUM=$(($NUM+1)) 231 | done 232 | else 233 | local THIS_PREFIX=${THIS%.*} 234 | if [ "$THIS_PREFIX" != "$LAN_PREFIX" ];then 235 | Say "***ERROR '"$THIS"' is not on this LAN '"$LAN_PREFIX".0/24'" 236 | echo "" 237 | return 200 238 | else 239 | IP_LIST=$IP_LIST" "$THIS # Add to list 240 | fi 241 | fi 242 | else 243 | # Let the caller ultimately decide if non-IP is valid!!! 244 | #Say "**Warning non-IP" $THIS 245 | IP_LIST=$IP_LIST" "$THIS # Add to list 246 | HOST_NAME=1 247 | fi 248 | 249 | shift 1 250 | done 251 | 252 | echo $IP_LIST 253 | 254 | if [ $HOST_NAME -eq 1 ];then 255 | return 300 256 | else 257 | return 0 258 | fi 259 | } 260 | Convert_TO_IP() { 261 | 262 | # Perform a lookup if a hostname (or I/P address) is supplied and is not known to PING 263 | # NOTE: etc/host.dnsmasq is in format 264 | # 265 | # I/P address hostname 266 | # 267 | # v386+ 268 | # Use /etc/dnsmasq.conf 269 | # NOTE: etc/host.dnsmasq is in format 270 | # 271 | # dhcp-host=48:45:20:D7:A6:22,set:48:45:20:D7:A6:22,HP-Envy13,192.168.1.38 272 | # 273 | # grep "^dhcp-host" /etc/dnsmasq.conf | grep -E "192.168.1.38$" 274 | # or 275 | # grep "^dhcp-host" /etc/dnsmasq.conf | grep -iE "HP-Envy13" 276 | # 277 | 278 | 279 | 280 | local USEPATH="/jffs/configs" 281 | 282 | if [ -n "$1" ];then 283 | 284 | if [ -z $2 ];then # Name to IP Address 285 | local IP_NAME=$(echo $1 | tr '[a-z]' '[A-Z]') 286 | 287 | local IP_RANGE=$(ping -c1 -t1 -w1 $IP_NAME 2>&1 | tr -d '():' | awk '/^PING/{print $3}') 288 | 289 | # 127.0.53.53 for ANDROID? https://github.com/laravel/valet/issues/115 290 | if [ -n "$(echo $IP_RANGE | grep -E "^127")" ];then 291 | local IP_RANGE= 292 | fi 293 | 294 | if [ -z "$IP_RANGE" ];then # Not PINGable so lookup static 295 | 296 | [ -f /etc/hosts.dnsmasq ] && IP_RANGE=$(grep -i "$IP_NAME" /etc/hosts.dnsmasq | awk '{print $1}') # v1.17 297 | 298 | if [ -z "$IP_RANGE" ] && [ -f /jffs/addons/YazDHCP.d/.hostnames ];then # v1.17 299 | IP_RANGE=$(grep -i "$IP_NAME" /jffs/addons/YazDHCP.d/.hostnames | awk '{print $1}') # v1.17 300 | fi 301 | 302 | #logger -s -t "($(basename $0))" $$ "Lookup '$IP_NAME' in DNSMASQ returned:>$IP_RANGE<" 303 | 304 | # If entry not matched in /etc /hosts.dnsmasq see if it exists in our IPGroups lookup file 305 | # 306 | # KEY I/P address[ {,|-} I/P address] 307 | # 308 | if [ -z "$IP_RANGE" ] && [ -f $USEPATH/IPGroups ];then 309 | #IP_RANGE=$(grep -i "^$IP_NAME" $USEPATH/IPGroups | awk '{print $2}') 310 | IP_RANGE=$(grep -i "^$IP_NAME" $USEPATH/IPGroups | awk '{$1=""; print $0}') # All columns except 1st to allow '#comments' an # spaces and ',' between IPs v1.07 311 | #logger -s -t "($(basename $0))" $$ "Lookup '$IP_NAME' in '$USEPATH/IPGroups' returned:>$IP_RANGE<" 312 | fi 313 | fi 314 | else # IP Address to name 315 | IP_RANGE=$(nslookup $1 | grep "Address" | grep -v localhost | cut -d" " -f4) 316 | fi 317 | else 318 | local IP_RANGE= # Return a default WiFi Client???? 319 | #logger -s -t "($(basename $0))" $$ "DEFAULT '$IP_NAME' lookup returned:>$IP_RANGE<" 320 | fi 321 | 322 | echo $IP_RANGE 323 | } 324 | Hostname_from_IP() { 325 | 326 | local HOSTNAMES= 327 | 328 | for IP in $@ 329 | do 330 | local HOSTNAME=$(Convert_TO_IP "$IP" "Reverse") 331 | HOSTNAMES=$HOSTNAMES" "$HOSTNAME 332 | done 333 | echo $HOSTNAMES 334 | } 335 | Is_Private_IPv4() { 336 | # 127. 0.0.0 – 127.255.255.255 127.0.0.0 /8 337 | # 10. 0.0.0 – 10.255.255.255 10.0.0.0 /8 338 | # 172. 16.0.0 – 172. 31.255.255 172.16.0.0 /12 339 | # 192.168.0.0 – 192.168.255.255 192.168.0.0 /16 340 | #grep -oE "(^192\.168\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])$)|(^172\.([1][6-9]|[2][0-9]|[3][0-1])\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])$)|(^10\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])\.([0-9]|[0-9][0-9]|[0-2][0-5][0-5])$)" 341 | grep -oE "(^127\.)|(^(0)?10\.)|(^172\.(0)?1[6-9]\.)|(^172\.(0)?2[0-9]\.)|(^172\.(0)?3[0-1]\.)|(^169\.254\.)|(^192\.168\.)" 342 | } 343 | Is_MAC_Address() { 344 | grep -oE "([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}" 345 | } 346 | Filter_This(){ 347 | grep -E "$1" 348 | } 349 | MAC_to_IP() { 350 | 351 | # Convert MAC into IP address 352 | local FN="/etc/ethers" 353 | 354 | local RESULT= 355 | 356 | if [ $FIRMWARE -gt 38201 ];then 357 | 358 | # Check if YazDHCP installed 359 | FN="/jffs/addons/YazDHCP.d/.staticlist" # v1.17 360 | if [ -f "$FN" ];then # v1.17 361 | local IP_ADDR=$(grep -iE "$MAC" "$FN" | cut -d',' -f3 ) # v1.17 362 | if [ -n "$IP_ADDR" ];then # v1.17 363 | FN="/jffs/addons/YazDHCP.d/.hostnames" # v1.17 364 | HOST_NAME=$(grep "^$IP_ADDR" "$FN" | cut -d' ' -f2) # v1.17 365 | RESULT=$HOST_NAME" "$IP_ADDR # v1.17 366 | fi # v1.17 367 | else # v1.17 368 | # etc/ethers no longer exists/used 369 | # Instead /etc/dnsmasq.conf contains 370 | # dhcp-host=00:22:B0:B5:BB:1A,10.88.8.254 371 | # v386+ 372 | # dhcp-host=48:45:20:D7:A6:22,set:48:45:20:D7:A6:22,HP-Envy13,192.168.1.38 373 | FN="/etc/dnsmasq.conf" 374 | #local ADDR_LIST=$(grep -i "$MAC" "$FN" | awk 'BEGIN {FS=","} {print $2}') 375 | [ -z "ADDR_LIST" ] && local ADDR_LIST=$(grep -i "$MAC" "$FN" | awk 'BEGIN {FS=","} {print $4}') # v1.17 v1.15 376 | fi 377 | else 378 | local ADDR_LIST=$(grep -i "$MAC" "$FN" | awk '{print $2}') 379 | fi 380 | 381 | if [ -z "$RESULT" ];then # v1.17 382 | if [ -n "$ADDR_LIST" ];then 383 | IP_RANGE=$ADDR_LIST 384 | IP_ADDR=$(grep -iE "$IP_RANGE" $FN | awk 'BEGIN {FS=","} {print $4}') # v1.15 385 | HOST_NAME=$(grep -iE "$IP_RANGE" $FN | awk 'BEGIN {FS=","} {print $3}') # v1.15 386 | RESULT=$HOST_NAME" "$IP_ADDR 387 | else 388 | ADDR_LIST="$(arp -a | awk '{print $2","$4","$1}' | tr -d '()' | grep -iF "$MAC")" # v1.15 389 | if [ -n "$ADDR_LIST" ];then # v1.15 390 | IP_ADDR=$(echo "$ADDR_LIST" | awk 'BEGIN {FS=","} {print $1}') # v1.15 391 | HOST_NAME=$(echo "$ADDR_LIST" | awk 'BEGIN {FS=","} {print $3}') # v1.15 392 | RESULT=$HOST_NAME" "$IP_ADDR 393 | else 394 | RESULT="***ERROR MAC Address not on LAN ("$FN"): '"$2"'" 395 | fi 396 | fi 397 | fi 398 | 399 | echo "$RESULT" 400 | } 401 | Backup_DB() { 402 | 403 | local DB=$1 404 | 405 | local DBNAME=$(basename "$DB") 406 | 407 | local DB_DIR=${DBNAME%.*} 408 | 409 | local NOW=$(date +"%Y%m%d-%H%M%S") # current date and time 410 | 411 | echo -en $cBRED >&2 412 | 413 | mkdir -p $BACKUP_DIR/$DB_DIR 414 | cp -p $DB $BACKUP_DIR/$DB_DIR/$DBNAME-Backup-$NOW 415 | RC=$? 416 | if [ $RC -eq 0 ];then 417 | echo -en $cBGRE >&2 418 | Say "'"$DB"' backup completed successfully to '"$BACKUP_DIR/$DB_DIR/$DBNAME-Backup-$NOW"'" 419 | else 420 | echo -e "\a" 421 | Say "***ERROR '"$DB"' backup FAILED!" 422 | fi 423 | 424 | return $RC 425 | 426 | echo -en $cRESET >&2 427 | 428 | } 429 | #########################################################Main############################################# 430 | Main() { true; } # Syntax that is Atom Shellchecker compatible! 431 | 432 | ANSIColours 433 | 434 | # v384.13+ NVRAM variable 'lan_hostname' supersedes 'computer_name' 435 | [ -n "$(nvram get computer_name)" ] && MYROUTER=$(nvram get computer_name) || MYROUTER=$(nvram get lan_hostname) 436 | 437 | 438 | FIRMWARE=$(echo $(nvram get buildno) | awk 'BEGIN { FS = "." } {printf("%03d%02d",$1,$2)}') 439 | 440 | # Need assistance ? 441 | if [ "$1" == "-h" ] || [ "$1" == "help" ];then 442 | clear # v1.08 443 | echo -e $cBWHT 444 | ShowHelp 445 | echo -e $cRESET 446 | exit 0 447 | fi 448 | 449 | # "dpi: TrendMicro function can't use under load-balance mode" 450 | 451 | # v384.11 now includes '/usr/sbin/sqlite3' # v1.11 452 | if [ -z "$(which sqlite3)" ];then 453 | Chk_Entware 'sqlite3' || { echo -e $cBRED"\a\n\t\t***ERROR*** Entware" $ENTWARE_UTILITY "not available\n"$cRESET;exit 99; } 454 | fi 455 | 456 | SQL_DB_DESC="Web History" 457 | SQL_TABLE="history" 458 | SQL_DATABASE= 459 | SQL_ADDONS_DATABASE="/jffs/.sys/WebHistory/Addons.db" # v1.16 460 | TITLE=$SQL_DB_DESC" starting....." 461 | 462 | FILTER_INUSE= 463 | 464 | CMDNOFILTER= # Use the default URL list 465 | MODE="AND" # v1.03 Default selection criteria 'AND' between filters 466 | WHERE= # v1.03 SQL WHERE clause 467 | SEND_EMAIL=0 # Don't send report via email 468 | CMDNOSCRIPT= # v1.03 Execute this script after SQL SELECT 469 | CMDIPFLUSH= # v1.06 Purge all history for selected IPs 470 | IP_CNT=0 471 | SORTBY="time" # Default sort column 472 | SORTBY_DESC= # Implied! 473 | COLORTIME=$cBGRE # Highlight Default sort column 'time' 474 | COLORMAC="$cBCYA" 475 | COLORIP="$cBCYA" 476 | COLORURL="$cBCYA" 477 | BACKUP_DIR="/opt/var" # v1.12 Default backup directory - i.e. Entware or can be overidded by commandline 478 | 479 | USE_TODAYS_DATE=1 # v1.08 480 | USE_CURRENT_HOUR=1 # v1.08 481 | SHOWSQL=0 # v1.13 Debug i.e. show SQL query 482 | 483 | # Check options 484 | DUMMY="=================================================================== Options" 485 | while [ $# -gt 0 ]; do # Until you run out of parameters . . . # v1.07 486 | case "$1" in 487 | mode=*) 488 | OPT=$(echo "$1" | sed -n "s/^.*mode=//p" | awk '{print $1}') 489 | case $OPT in 490 | "") MODE=OR;; # Override the default; 'mode=' is a shortcut! 491 | or|OR) MODE=OR;; 492 | and|AND) MODE=AND;; 493 | *) echo -e $cBRED"\a\n\t\t***ERROR INVALID mode '$1'\n"$cRESET 494 | exit 99 495 | ;; 496 | esac 497 | echo $WHERE 498 | [ -n "$FILTER_INUSE" ] && { echo -e $cBRED"\a\n\t\t***ERROR '$1' MUST precede filter specification '$FILTER_INUSE'\n"$cRESET; exit 99;} 499 | ;; 500 | showsql) # v1.13 501 | CMDSHOWSQL="ShowSQL" 502 | SHOWSQL=1 503 | ;; 504 | noscript) 505 | CMDNOSCRIPT="NoScript" 506 | ;; 507 | count) 508 | CMDCOUNT="CountONLY" 509 | CMDCOUNT_DESC=$cBYEL"***Summary only;"$cRESET 510 | ;; 511 | nodisplay) # v1.12 512 | CMDNODISPLAY="NoDISPLAY" 513 | CMDNODISPLAY_DESC=$cBYEL"***No Display;"$cRESET 514 | ;; 515 | sqldb=*) # Override default database 516 | SQL_DATABASE=$(echo "$1" | sed -n "s/^.*sqldb=//p" | awk '{print $1}') 517 | ;; 518 | email) 519 | SEND_EMAIL=1 520 | MAILFILE="/tmp/WebHistory.txt" 521 | EMAILACTION=" > "$MAILFILE 522 | EMAIL_DESC="E-mailing results," 523 | echo -e > $MAILFILE 524 | ;; 525 | date=*) 526 | USE_TODAYS_DATE=0 # v1.08 527 | 528 | DATE_LIST="$(echo "$1" | sed -n "s/^.*date=//p" | awk '{print $1}' | tr ',' ' ')" 529 | 530 | if [ -n "$DATE_LIST" ];then # v1.08 531 | DATE_FILTER= # Used for Display info 532 | DATE_CNT=0 533 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by Date" || FILTER_DESC=$FILTER_DESC", "$MODE" by Date" 534 | 535 | DATE_SQL= # v1.04 SQL statement for multiple 'DATE match' 536 | for DATE in $DATE_LIST 537 | do 538 | # SQL format is YYYY-MM-DD so change YYYY/MM/DD ->YYYY-MM-DD 539 | DATE=$(echo "$DATE" | tr '/' '-') 540 | [ $DATE_CNT -eq 0 ] && DATE_FILTER=$DATE_FILTER""$DATE || DATE_FILTER=$DATE_FILTER"|"$DATE 541 | DATE_CNT=$((DATE_CNT+1)) 542 | [ -z "$DATE_SQL" ] && DATE_SQL=$DATE_SQL"(time LIKE '"$DATE"%'" || DATE_SQL=$DATE_SQL" OR time LIKE '"$DATE"%'" 543 | done 544 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$DATE_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$DATE_FILTER 545 | 546 | [ -z "$WHERE" ] && WHERE="WHERE ("$DATE_SQL")" || WHERE=$WHERE" "$MODE" "$DATE_SQL")" # v1.04 547 | fi 548 | CMDDATE="CMDDATE" 549 | ;; 550 | time=*) 551 | USE_CURRENT_HOUR=0 # v1.08 552 | 553 | TIME_LIST="$(echo "$1" | sed -n "s/^.*time=//p" | awk '{print $1}' | tr ',' ' ')" 554 | 555 | if [ -n "$TIME_LIST" ];then # v1.08 556 | TIME_FILTER= # Used for Display info 557 | TIME_CNT=0 558 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by Time" || FILTER_DESC=$FILTER_DESC", "$MODE" by Time" 559 | 560 | TIME_SQL= # v1.04 SQL statement for multiple 'TIME match' 561 | for TIME in $TIME_LIST 562 | do 563 | # Minimum must be 'nn' or 'HH:' or 'HH:MM' format # v1.07 564 | # NOTE 'time=10' will match anywhere e.g. '10:01:02' (HH:) as expected but also '03:10:59' (MM:) 565 | case "${#TIME}" in 566 | 2) [ $(echo "$TIME" | grep -oE "^([0-1][0-9])|^(2[0-3])$") ] || { echo -e $cBRED"\a\n\t\t***ERROR time='$TIME' (HH format) invalid\n"$cRESET; exit 55; } ;; 567 | 3) [ $(echo "$TIME" | grep -oE "^([0-1][0-9])|^(2[0-3])(:)?") ] || { echo -e $cBRED"\a\n\t\t***ERROR time='$TIME' (HH: format) invalid\n"$cRESET; exit 66; } ;; 568 | 5) [ $(echo "$TIME" | grep -oE "^([0-1][0-9])|^(2[0-3]):[0-5][0-9]$") ] || { echo -e $cBRED"\a\n\t\t***ERROR time='$TIME' (HH:MM format) invalid\n"$cRESET;exit 77; } ;; 569 | *) { echo -e $cBRED"\a\n\t\tSQL time='$TIME' invalid format (HH:MM:SS is deemed illogical for SQL requests)\n"$cRESET;exit 99; };; 570 | esac 571 | 572 | [ $TIME_CNT -eq 0 ] && TIME_FILTER=$TIME_FILTER""$TIME || TIME_FILTER=$TIME_FILTER"|"$TIME 573 | TIME_CNT=$((TIME_CNT+1)) 574 | [ -z "$TIME_SQL" ] && TIME_SQL=$TIME_SQL"(time LIKE '% "$TIME"%'" || TIME_SQL=$TIME_SQL" OR time LIKE '% "$TIME"%'" #v1.07 Fix 575 | done 576 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$TIME_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$TIME_FILTER 577 | 578 | if [ -z "$CMDNOSCRIPT" ] && [ $(echo $@ | grep -cw "noscript") -eq 0 ];then 579 | [ -z "$WHERE" ] && WHERE="WHERE ("$TIME_SQL")" || WHERE=$WHERE" "$MODE" "$TIME_SQL")" # v1.04 580 | fi 581 | fi 582 | CMDTIME="CMDTIME" 583 | ;; 584 | flush*) 585 | # *** Unique to IP processing*** 586 | if [ $IP_CNT -eq 0 ];then 587 | echo -e $cBRED"\a\n\t\t***ERROR Missing 'ip=' arg as 'flush' is only valid in this context 'ip= flush'\n"$cRESET 588 | exit 99 589 | fi 590 | CMDIPFLUSH="IPFlush" 591 | ;; 592 | mac=*) # v1.12 593 | # Whilst easier to filter on Hostname/IP, explicitly filter on MAC address 594 | CMDMAC=$(echo "$1" | sed -n "s/^.*mac=//p" | awk '{print $1}' | tr ',' ' ') 595 | MAC_LIST=$CMDMAC 596 | MAC_FILTER= # Used for Display info 597 | MAC_CNT=0 598 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by MAC" || FILTER_DESC=$FILTER_DESC", "$MODE" by MAC" 599 | 600 | MAC_SQL= # v1.04 SQL statement for multiple 'MAC match' 601 | 602 | 603 | 604 | LAN_MACS=$(echo "$LAN_MACS" | sed 's/^ //p') 605 | LAN_MACS=$(echo "$LAN_MACS" | awk '{for (i=1;i<=NF;i++) if (!a[$i]++) printf("%s%s",$i,FS)}') # Remove duplicates 606 | 607 | for MAC in $MAC_LIST 608 | do 609 | if [ -n "$(echo "$MAC" | Is_MAC_Address )" ];then 610 | [ $MAC_CNT -eq 0 ] && MAC_FILTER=$MAC_FILTER""$MAC || MAC_FILTER=$MAC_FILTER"|"$MAC 611 | MAC_CNT=$((MAC_CNT+1)) 612 | [ -z "$MAC_SQL" ] && MAC_SQL=$MAC_SQL"(mac LIKE '"$MAC"%'" || MAC_SQL=$MAC_SQL" OR mac LIKE '"$MAC"%'" 613 | else 614 | echo -e $cBRED"\a\n\t\t***ERROR Invalid MAC address '"$MAC"' in 'mac="$CMDMAC"' filter\n"$cRESET 615 | exit 99 616 | fi 617 | done 618 | 619 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$MAC_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$MAC_FILTER 620 | 621 | [ -z "$WHERE" ] && WHERE="WHERE ("$MAC_SQL")" || WHERE=$WHERE" "$MODE" "$MAC_SQL")" 622 | ;; 623 | ip=*) 624 | # If Hostname/IP then filter on MAC address 625 | CMDIP=$(echo "$1" | sed -n "s/^.*ip=//p" | awk '{print $1}' | tr ',' ' ') 626 | 627 | GROUP_FOUND=0 628 | IP_GROUP_LIST=$CMDIP 629 | while true;do # Iterate to expand any Groups within a Group 630 | for ITEM in $IP_GROUP_LIST 631 | do 632 | if [ -z "$(echo "$ITEM" | Is_Private_IPv4 )" ];then 633 | # Check for group names, and expand as necessary 634 | # e.g. '192.168.1.30,192.168.1.50-192.168.1.54' -> '192.168.1.30 192.168.1.50 192.168.1.51 192.168.1.52 192.168.1.53 192.168.1.54' 635 | if [ -f "/jffs/configs/IPGroups" ];then # '/jffs/configs/IPGroups' two columns 636 | # ID xxx.xxx.xxx.xxx[[,xxx.xxx.xxx.xxx][-xxx.xxx.xxx.xxx] 637 | GROUP_IP=$(grep -iwE -m 1 "^$ITEM" /jffs/configs/IPGroups | awk '{$1=""; print $0}') 638 | if [ -n "$GROUP_IP" ];then 639 | GROUP_FOUND=1 640 | # Expand the list of IPs as necessary 641 | # e.g. '192.168.1.30,192.168.1.50-192.168.1.54' -> '192.168.1.30 192.168.1.50 192.168.1.51 192.168.1.52 192.168.1.53 192.168.1.54' 642 | GROUP_IP=$(echo $GROUP_IP | tr ',' ' ') # CSVs ? 643 | GROUP_IP=$(echo $GROUP_IP | tr ':' '-') # Alternative range spec xxx.xxx.xxx.xxx:xxx.xxx.xxx.xxx 644 | else 645 | # Perform lookup 646 | GROUP_IP=$(nslookup "$ITEM" | grep -woE '([0-9]{1,3}\.){3}[0-9]{1,3}' | awk 'NR>2') 647 | if [ -z "$GROUP_IP" ];then 648 | echo -e $cBRED"\a\n\t\t***ERROR Hostname '$1' INVALID\n"$cRESET 649 | exit 99 650 | fi 651 | fi 652 | else 653 | GROUP_IP=$ITEM 654 | fi 655 | 656 | # Expand any ranges - allow Hostnames e.g. LIFX-Table_light to pass through 657 | if [ -n "$(echo "$GROUP_IP" | grep "-")" ];then # xxx-yyy range ? 658 | GROUP_IP="$(ExpandIPRange "$GROUP_IP")" 659 | RC=$? # Should really check 660 | fi 661 | [ -n "$GROUP_IP" ] && LAN_IPS=$LAN_IPS" "$GROUP_IP 662 | else 663 | LAN_IPS=$LAN_IPS" "$ITEM 664 | fi 665 | done 666 | 667 | if [ $GROUP_FOUND -eq 0 ];then 668 | break 669 | fi 670 | 671 | IP_GROUP_LIST=$LAN_IPS # Keep expanding 672 | LAN_IPS= 673 | GROUP_FOUND=0 674 | done 675 | 676 | LAN_IPS=$(echo "$LAN_IPS" | sed 's/^ //p') 677 | LAN_IPS=$(echo "$LAN_IPS" | awk '{for (i=1;i<=NF;i++) if (!a[$i]++) printf("%s%s",$i,FS)}') # Remove duplicates 678 | 679 | IP_FILTER= # Used for Display info 680 | IP_CNT=0 681 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by IP" || FILTER_DESC=$FILTER_DESC", "$MODE" by IP" 682 | 683 | MAC_SQL= # v1.04 SQL statement for multiple 'MAC match' 684 | for IP in $LAN_IPS 685 | do 686 | # Convert IP to MAC 687 | XIP=$(echo "$IP" | sed 's/\./\\\./g') 688 | MAC=$(grep -i "${XIP}$" /etc/dnsmasq.conf | awk 'BEGIN {FS=","} {print $1}' | sed -n "s/^dhcp-host=//p") 689 | [ $IP_CNT -eq 0 ] && IP_FILTER=$IP_FILTER""$IP || IP_FILTER=$IP_FILTER"|"$IP 690 | IP_CNT=$((IP_CNT+1)) 691 | [ -z "$MAC_SQL" ] && MAC_SQL=$MAC_SQL"(mac LIKE '"$MAC"%'" || MAC_SQL=$MAC_SQL" OR mac LIKE '"$MAC"%'" 692 | done 693 | 694 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$IP_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$IP_FILTER 695 | 696 | [ -z "$WHERE" ] && WHERE="WHERE ("$MAC_SQL")" || WHERE=$WHERE" "$MODE" "$MAC_SQL")" # v1.04 697 | 698 | # *** Unique to IP processing*** 699 | if [ -n "$CMDIPFLUSH" ];then # Should we 'flush' the history for these IPs ? 700 | FILTER_DESC=$FILTER_DESC", and they will have their history FLUSHED!" 701 | fi 702 | ;; 703 | url=*) 704 | URL_LIST="$(echo "$1" | sed -n "s/^.*url=//p" | awk '{print $1}' | tr ',' ' ')" 705 | 706 | URL_FILTER= # Used for Display info 707 | URL_CNT=0 708 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by URL" || FILTER_DESC=$FILTER_DESC", "$MODE" by URL" 709 | 710 | URL_SQL= # v1.04 SQL statement for multiple 'URL match' 711 | for URL in $URL_LIST 712 | do 713 | [ $URL_CNT -eq 0 ] && URL_FILTER=$URL_FILTER""$URL || URL_FILTER=$URL_FILTER"|"$URL 714 | URL_CNT=$((URL_CNT+1)) 715 | [ -z "$URL_SQL" ] && URL_SQL=$URL_SQL"(url LIKE '%"$URL"%'" || URL_SQL=$URL_SQL" OR url LIKE '%"$URL"%'" 716 | done 717 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$URL_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$URL_FILTER 718 | 719 | 720 | [ -z "$WHERE" ] && WHERE="WHERE ("$URL_SQL")" || WHERE=$WHERE" "$MODE" "$URL_SQL")" # v1.04 721 | CMDURL="CMDURL" 722 | ;; 723 | nofilter) 724 | CMDNOFILTER="NoFilter" 725 | ;; 726 | backup|backup=*) # v1.12 727 | if [ "$1" = "backup" ];then 728 | CMDBACKUP="Backup" # Use default '/opt/var/' Entware 729 | else 730 | CMDBACKUP="$(echo "$1" | sed -n "s/^.*backup=//p" | awk '{print $1}')" 731 | if [ "$CMDBACKUP" = "/tmp" ] || [ ! -d "$CMDBACKUP" ];then 732 | echo -e $cBRED"\a\n\t***ERROR Backup location '"$1"' INVALID. e.g. use a permanent disk e.g. '/mnt/xxxx' but NOT simply '/tmp' or '/tmp/'\n"$cRESET 733 | exit 99 734 | else 735 | BACKUP_DIR=$CMDBACKUP 736 | CMDBACKUP="Backup" 737 | fi 738 | fi 739 | ;; 740 | purgeallreset) 741 | CMDPURGEALLRESET="PurgeAllReset" 742 | ;; 743 | noansii) 744 | CMDNOANSII="NoANSII" 745 | ;; 746 | sortby=*) 747 | CMDSORTBY="$(echo "$1" | sed -n "s/^.*sortby=//p" | awk '{print $1}' | tr ',' ' ')" 748 | case $CMDSORTBY in 749 | time) SORTBY="time";; 750 | mac) SORTBY="mac";SORTBY_DESC="${cBGRE}Sorted by 'mac';";COLORTIME=$cBCYA;COLORMAC=$cBGRE;; 751 | url) SORTBY="url";SORTBY_DESC="${cBGRE}Sorted by 'url';";COLORTIME=$cBCYA;COLORURL=$cBGRE;; 752 | ip) UNIXSORT="| sort -k 7";COLORTIME=$cBCYA;COLORIP=$cBGRE;; 753 | *) 754 | echo -e $cBRED"\a\n\t***ERROR Sort column '"$1" INVALID '(time, mac, url, ip)'\n"$cRESET 755 | exit 99 756 | ;; 757 | esac 758 | ;; 759 | report=*) # v1.12 760 | REPORT_CSV=$(echo "$1" | sed -n "s/^.*report=//p" | awk '{print $1}') 761 | CMDREPORT="CreateCSV" 762 | ;; 763 | *) 764 | echo -e $cBRED"\a\n\t***ERROR unrecognised directive '"$1"'\n"$cRESET 765 | exit 99 766 | ;; 767 | esac 768 | shift # Check next set of parameters. 769 | done 770 | 771 | # Create the lookup table to perform the translation from MAC address to Hostname and IP Address and seed it with current online devices 772 | if [ ! -f $SQL_ADDONS_DATABASE ];then # v1.16 773 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_ADDONS_DATABASE CREATE TABLE IF NOT EXISTS hosts(mac varchar(15),hostname TEXT,ip varchar(19),PRIMARY KEY(mac,hostname,ip));\n" # v1.16 774 | sqlite3 $SQL_ADDONS_DATABASE "CREATE TABLE IF NOT EXISTS hosts(mac varchar(15),hostname TEXT,ip varchar(19),PRIMARY KEY(mac,hostname,ip));" # v1.16 775 | arp -a | tr -d '()' | awk '{print toupper($4)","$2","$1}' | sqlite3 -csv $SQL_ADDONS_DATABASE ".import '|cat -' hosts" 776 | fi 777 | 778 | # Use Today's date and current hour? 779 | if [ $USE_TODAYS_DATE -eq 1 ];then # v1.08 Default is Todays's date 780 | DATE_FILTER=$(date "+%F") 781 | DATE_SQL="(time LIKE '"$DATE_FILTER"%'" 782 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by Today" || FILTER_DESC=$FILTER_DESC", "$MODE" by Today" 783 | [ -z "$WHERE" ] && WHERE="WHERE ("$DATE_SQL")" || WHERE=$WHERE" "$MODE" "$DATE_SQL")" 784 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$DATE_FILTER || FILTER_INUSE=$FILTER_INUSE"|"$DATE_FILTER 785 | fi 786 | if [ $USE_CURRENT_HOUR -eq 1 ];then # v1.08 Default is current hour 787 | TIME_FILTER=$(date "+%H")":" 788 | TIME_SQL="(time LIKE '% "$TIME_FILTER"%'" 789 | 790 | [ -z "$FILTER_INUSE" ] && FILTER_DESC="by current hour" || FILTER_DESC=$FILTER_DESC", "$MODE" by current hour" 791 | [ -z "$WHERE" ] && WHERE="WHERE ("$TIME_SQL")" || WHERE=$WHERE" "$MODE" "$TIME_SQL")" 792 | [ -z "$FILTER_INUSE" ] && FILTER_INUSE=$TIME_FILTER|| FILTER_INUSE=$FILTER_INUSE"|"$TIME_FILTER 793 | fi 794 | 795 | # Remember to terminate the SQL 'WHERE' clause! 796 | [ -n "$WHERE" ] && WHERE=$WHERE")" 797 | 798 | if [ -z "$CMDNOFILTER" ];then 799 | # Default filter 800 | if [ -z "$FILTER_INUSE" ];then 801 | DATE_FILTER=$(date "+%F") # v1.05 802 | URL_FILTER="facebook|youtube" 803 | FILTER_INUSE=$DATE_FILTER"|"$URL_FILTER 804 | IP_FILTER="¬";TIME_FILTER="¬" # v1.07 unused filters cannot be NULL 805 | if [ "$MODE" == "AND" ];then # v1.07 806 | FILTER_DESC="by today's Facebook OR Youtube URL activity" 807 | WHERE="WHERE ((time LIKE '"$(date "+%F")"%' AND url LIKE '%youtube%') OR (time LIKE '"$(date "+%F")"%' AND url LIKE '%facebook%'))" 808 | else 809 | FILTER_DESC="by today's activity AND any previous Facebook OR Youtube URL activity" 810 | WHERE="WHERE (time LIKE '"$(date "+%F")"%' OR url LIKE '%youtube%' OR url LIKE '%facebook%')" 811 | fi 812 | fi 813 | else 814 | FILTER_DESC="ALL i.e. no filter" 815 | WHERE= 816 | fi 817 | 818 | # Find appropriate database '/jffs/.sys/WebHistory/WebHistory.db' 819 | if [ -z "$SQL_DATABASE" ];then 820 | SQL_DATABASE="$(find /jffs/.sys/ -name WebHistory.db)" 821 | if [ $(find /jffs/.sys -name WebHistory.db | wc -l) -ne 1 ];then 822 | if [ $(find /jffs/.sys -name WebHistory.db | wc -l) -eq 0 ];then 823 | echo -e $cBRED"\a\n\t\t***ERROR $SQL_DB_DESC database '"$SQL_DATABASE"'NOT found??!!\n"$cRESET # v1.16 824 | exit 99 825 | fi 826 | if [ $(find /jffs/.sys -name WebHistory.db | wc -l) -gt 1 ];then 827 | echo -e $cBRED"\a\n\t\t***ERROR Multiple $SQL_DB_DESC databases '"$SQL_DATABASE"'found??!!\n"$cRESET 828 | exit 98 829 | fi 830 | fi 831 | fi 832 | 833 | # Validate Web History database 834 | if [ -z "$SQL_DATABASE" ] || [ ! -f $SQL_DATABASE ];then # v1.16 835 | echo -e $cBRED"\a\n\t\t***ERROR $SQL_DB_DESC database '"$SQL_DATABASE"' NOT found!\n"$cRESET 836 | exit 97 837 | fi 838 | 839 | [ -n "$CMDNOANSII" ] && SQLDB_TITLE="'"$SQL_DATABASE"'" 840 | 841 | # Should the backup be performed? 842 | if [ -n "$CMDBACKUP" ];then # v1.06 843 | echo -e 844 | Backup_DB "$SQL_DATABASE" 845 | echo -e $cRESET 846 | exit 0 847 | fi 848 | 849 | if [ -n "$CMDPURGEALLRESET" ];then 850 | echo -e 851 | 852 | echo -en ${cBRED}$aBLINK"\a\n\t\t\t****** WARNING are you sure? ******\n\n\t\t\t"${cRESET}$cBYEL"Enter "$cBWHT"ContinueOK!"$cBYEL" or press "$cBWHT"ENTER"$CBYEL" key to"$cBYEL" ABORT\n\t\t\t >>"$cRESET 853 | read OPT 854 | if [ -n "$(echo "$OPT" | grep -oF "ContinueOK!")" ];then 855 | echo -e 856 | Backup_DB "$SQL_DATABASE" 857 | 858 | /usr/sbin/WebHistory -z 859 | /usr/sbin/WebHistory -e 860 | Say $VER "'"$SQL_DATABASE"' PURGED and RESET." 861 | else 862 | echo -e $cBWHT"\n\t\t\tRequest cancelled!" 863 | fi 864 | echo -e $cRESET 865 | exit 0 866 | fi 867 | 868 | ##################################################################Display##################################################### 869 | clear 870 | 871 | echo -e $cBWHT 872 | Say $VER "$TITLE"$SQLDB_TITLE 873 | 874 | # Hyperlink support is native under Xshell5/MobaXterm. (Xshell5 visually shows which text is URL clickable ;-) 875 | # MobaXterm: CTRL+Click the URL (must be prefixed with 'http') 876 | # PuTTY: https://www.chiark.greenend.org.uk/~sgtatham/putty/wishlist/url-launching.html 877 | # 878 | # Prevent double spacing between report lines by changing font size 879 | # MobaXTerm: CTRL+MouseScrollWheel 880 | # PuTTY: ClearType Andale Mono 9pt 881 | # 882 | echo -e $cBYEL"\tNOTE: Columns in "$cBWHT"white"${cRESET}$cBYEL" are eligible for filters; "$cBRED"red text"${cRESET}$cBYEL" indicates a match on the filters requested; (URLs are Xshell5/MobaXterm hyperlinks)" 883 | [ -n "$CMDNOFILTER" ] && FILTER_INUSE= # v1.12 884 | echo -e "\n\t"${CMDCOUNT_DESC}${SORTBY_DESC}${EMAIL_DESC}$cBMAG"Filter" $FILTER_DESC "==> '"$FILTER_INUSE"'" 885 | [ -n "$REPORT_CSV" ] && echo -e "\n\t"$cBMAG"Report file (.csv format): '"$REPORT_CSV"'" $cRESET 886 | 887 | # Execute this script i.e. create report/email ? 888 | if [ -z "$CMDNOSCRIPT" ];then 889 | 890 | printf '\n\t\t%b%b %-12s %-10s %b%-20s %-18s %-16s %b%-44s\n\n' "$cBCYA" "$COLORTIME" "YYYY/MM/DD" " HH:MM:SS" "$cBCYA" " MAC address" " Host Name" " IP address" "$COLORURL" " URL" 891 | [ -n "$CMDNODISPLAY" ] && echo -e $cRED"\t\t***No Display of records on screen requested***\n" # v1.12 892 | echo -en $cRESET 893 | 894 | # v1.07 unused filters cannot be NULL 895 | [ -z "$DATE_FILTER" ] && DATE_FILTER="¬" 896 | [ -z "$TIME_FILTER" ] && TIME_FILTER="¬" 897 | [ -z "$IP_FILTER" ] && IP_FILTER="¬" 898 | [ -z "$URL_FILTER" ] && URL_FILTER="¬" 899 | [ -z "$MAC_FILTER" ] && MAC_FILTER="¬" 900 | 901 | StatusLine $CMDNOANSII"Update" ${cYBLU}$aBOLD"Processing '$SQL_DATABASE' database....please wait!" 902 | 903 | RESULT_PAGECNT=0 # v1.08 No. records shown on screen 904 | RESULT_CNT=0 # v1.08 Total number of matching records 905 | 906 | echo -en $cBRED # Just in case SQL error e.g. 'Error: database is locked' 907 | 908 | [ "$CMDREPORT" = "CreateCSV" ] && rm $REPORT_CSV 2>/dev/null # v1.12 Erase .csv report file 909 | 910 | # Display Summary count of matches ONLY? 911 | if [ -n "$CMDCOUNT" ];then 912 | RESULT_CNT=$(sqlite3 $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, count(*) FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;" | cut -d'|' -f2) 913 | #echo -e $CMDCOUNT_DESC 914 | else 915 | # Rather than loop thru' each record to create .csv, simply allow SQL to create the .csv - much faster! 916 | if [ "$CMDREPORT" = "CreateCSV" ] && [ "$CMDNODISPLAY" = "NoDISPLAY" ];then # v1.12 917 | # [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 -csv $SQL_DATABASE SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, mac, url FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;\n" # v1.13 918 | # sqlite3 -csv $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, mac, url FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;" > $REPORT_CSV 919 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_DATABASE ATTACH '$SQL_DATABASE' as asus;ATTACH '$SQL_ADDONS_DATABASE' as martineau;SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, x.hostname, x.ip, h.mac, h.url FROM asus.history h JOIN martineau.hosts x on h.mac = x.mac;" # v1.16 920 | sqlite3 $SQL_DATABASE "ATTACH '$SQL_DATABASE' as asus;ATTACH '$SQL_ADDONS_DATABASE' as martineau;SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, x.hostname, x.ip, h.mac, h.url from asus.history h JOIN martineau.hosts x on h.mac = x.mac;" > $REPORT_CSV # v1.16 921 | RESULT_CNT=$(sqlite3 $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, mac, url, count(*) FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;" | cut -d'|' -f4) 922 | nvram set tmp_WH_TOTAL=$RESULT_CNT 923 | StatusLine $CMDNOANSII"Clear" 924 | StatusLine $CMDNOANSII"NoFLASH" ${IND}$aREVERSE"Summary: Result count = "$RESULT_CNT", $REPORT_CSV created" 925 | 926 | echo -e $cBGRE"\n\t\t[✔]${cBMAG} $REPORT_CSV ${cBGRE}created $cRESET" 927 | else 928 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_DATABASE SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, mac, url FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;\n" # v1.13 929 | sqlite3 $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, mac, url FROM $SQL_TABLE $WHERE ORDER BY $SORTBY;" | while IFS= read -r LINE 930 | 931 | do 932 | 933 | if [ -n "$(echo $LINE | grep -F "SQLite version")" ];then # Webhistory -z ??????? 934 | break 935 | fi 936 | 937 | if [ -n "$(echo $LINE | grep -F "Error: database is locked")" ];then 938 | break 939 | fi 940 | 941 | [ -z "$RECORD_CNT" ] && RECORD_CNT=0 942 | 943 | DATE=${LINE:0:10} 944 | 945 | TIME=${LINE:11:8} 946 | 947 | MAC=${LINE:20:17} 948 | 949 | DESC=$(MAC_to_IP "$MAC") 950 | HOSTNAME=${DESC% *} # First word (' ' delimiter) 951 | HOSTNAME=$(echo "$HOSTNAME" | sed s'/\.$//') # v1.16 'HP-Envy13.' ==> 'HP-Envy13' 952 | IP=${DESC##* } # Last word (' ' delimiter) 953 | if [ "${HOSTNAME:0:3}" == "***" ];then 954 | HOSTNAME="n/a" 955 | IP="n/a" 956 | else 957 | # Update Lookup table 958 | [ -n "$(sqlite3 $SQL_ADDONS_DATABASE "SELECT mac FROM hosts WHERE mac='$MAC';")" ] && sqlite3 $SQL_ADDONS_DATABASE "DELETE FROM hosts WHERE mac='$MAC';" # v1.16 959 | sqlite3 $SQL_ADDONS_DATABASE "INSERT INTO hosts values('$MAC','$IP','$HOSTNAME');" # v1.16 960 | fi 961 | 962 | URL=${LINE##*|} # Last word ('|' delimiter) 963 | 964 | 965 | # DEBUG_LINE=">"$LINE"<" 966 | # DEBUG_FILTER_INUSE=">"$FILTER_INUSE"<" 967 | # DEBUG_DATE=">"$DATE"<" 968 | # DEBUG_FILTER_DATE=">"$DATE_FILTER"<" 969 | # DEBUG_TIME=$TIME 970 | # DEBUG_FILTER_TIME=">"$TIME_FILTER"<" 971 | # DEBUG_MAC=$MAC 972 | # DEBUG_FILTER_MAC=">"$MAC_FILTER"<" 973 | # DEBUG_DESC=$DESC 974 | # DEBUG_HOSTNAME=$HOSTNAME 975 | # DEBUG_IP=$IP 976 | # DEBUG_FILTER_IP=">"$IP_FILTER"<" 977 | # DEBUG_CAT=$CAT 978 | # DEBUG_FILTER_CAT=">"$CAT_FILTER"<" 979 | # DEBUG_APP=$APP 980 | # DEBUG_FILTER_APP=">"$APP_FILTER"<" 981 | 982 | # Cosmetic highlighting! ;-) 983 | if echo "$DATE" | grep -qE "$DATE_FILTER" ;then # Date filter match? # YYYY-MM-DD 984 | COLOUR_DATE=$cBRED 985 | else 986 | COLOUR_DATE=$cRESET 987 | fi 988 | 989 | if echo "$TIME" | grep -qE "$TIME_FILTER" ;then # Time filter match? # HH:MM:SS 990 | COLOUR_TIME=$cBRED 991 | else 992 | COLOUR_TIME=$cRESET 993 | fi 994 | 995 | if echo "$MAC" | grep -qE "$MAC_FILTER" ;then # v1.12 996 | COLOUR_MAC=$cBRED 997 | else 998 | COLOUR_MAC=$cRESET 999 | fi 1000 | 1001 | if echo "$MAC" | grep -qE "$IP_FILTER" ;then # v1.06 fix MAC filter match? 1002 | COLOUR_IP=$cBRED 1003 | else 1004 | COLOUR_IP=$cRESET 1005 | fi 1006 | 1007 | if [ -n "$CMDIPFLUSH" ];then 1008 | COLOUR_URL=$cBRED 1009 | URL=$URL"**FLUSHED" 1010 | else 1011 | if echo "$URL" | grep -qE "$URL_FILTER" ;then # URL filter match? 1012 | COLOUR_URL=$cBRED 1013 | else 1014 | COLOUR_URL=$cRESET 1015 | fi 1016 | fi 1017 | 1018 | # 1019 | # SQL format is YYYY-MM-DD so convert to EU ->YYYY/MM/DD 1020 | DATE=$(echo "$DATE" | tr '-' '/') 1021 | 1022 | if [ "$CMDNODISPLAY" != "NoDISPLAY" ];then # v1.12 1023 | printf '\t\t%b %-12s %b %-10s %b %-20s %b %-18s %b %-16s %b http://%s\n' "$COLOUR_DATE" "$DATE" "$COLOUR_TIME" "$TIME" "$COLOUR_MAC" "$MAC" "$cBBLU" "$HOSTNAME" "$COLOUR_IP" "$IP" "$COLOUR_URL" "$URL" # v1.12 1024 | fi 1025 | if [ $SEND_EMAIL -eq 1 ];then 1026 | printf '%-12s %-10s %-20s %-18s %-16s http://%s\n' "$(echo $LINE | awk '{print $1" "$2}')" "$MAC" ""$(echo $DESC | awk '{print $1}')"" ""$(echo $DESC | awk '{print $NF}')"" ""$(echo $LINE | awk '{print $NF}')"" >>$MAILFILE # v1.08 1027 | fi 1028 | 1029 | # Slow...compared to 'sqlite3 -csv' invocation 1030 | if [ "$CMDREPORT" = "CreateCSV" ];then # v1.12 1031 | echo -e "\"$(echo $LINE | sed 's/|/ /' | awk '{print $1" "$2}')\",\""$MAC"\",\""$(echo $DESC | awk '{print $1}')""\",\"""$(echo $DESC | awk '{print $NF}')""\",\"""$(echo $LINE | awk 'BEGIN { FS = "|" } {print "http://"$3}')"\"" >>$REPORT_CSV 1032 | fi 1033 | echo -en $cRESET 1034 | 1035 | RECORD_CNT=$((RECORD_CNT+1)) 1036 | nvram set tmp_WH_TOTAL=$RECORD_CNT # Damn subshells VERY UGLY HACK :-( 1037 | 1038 | 1039 | done 1040 | fi 1041 | 1042 | # Delete specified URL history for device if 'url=' specified otherwise 1043 | # if 'ip=10.88.8.1 flush' then mass delete ALL history for the IP (aka MAC address) 1044 | if [ "$CMDIPFLUSH" == "IPFlush" ];then 1045 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_DATABASE DELETE from $SQL_TABLE WHERE $MAC_SQL) AND $URL_SQL)" # v1.13 1046 | sqlite3 $SQL_DATABASE "DELETE from $SQL_TABLE WHERE $MAC_SQL) AND $URL_SQL)" 1047 | RC=$? 1048 | fi 1049 | 1050 | if [ $SEND_EMAIL -eq 1 ];then 1051 | echo -e $cBYEL 1052 | StatusLine $CMDNOANSII"Update" ${cYBLU}$aBOLD"Preparing e-mail....please wait!" 1053 | sleep 1 1054 | SendMail $MAILFILE 1055 | StatusLine $CMDNOANSII"Clear" 1056 | #echo -e $cBGRE"\n\tEmail sent..."$MAILFILE 1057 | fi 1058 | 1059 | 1060 | RESULT_CNT=$(nvram get tmp_WH_TOTAL);nvram unset tmp_WH_TOTAL # Damn subshells VERY UGLY HACK :-( 1061 | [ -z "$RESULT_CNT" ] && RESULT_CNT=0 1062 | 1063 | fi 1064 | 1065 | # Summarise 1066 | [ $RESULT_CNT -eq 0 ] && IND=$cBRED || IND=$cBGRE 1067 | 1068 | if [ -z "$CMDNOANSII" ];then 1069 | if [ -n "$CMDCOUNT" ] || [ $RESULT_CNT -le 20 ];then # v1.09 1070 | StatusLine $CMDNOANSII"NoFLASH" ${IND}$aREVERSE"Summary: Result count = "$RESULT_CNT" " 1071 | else 1072 | echo -e "\n"${cRESET}${cIND}$aREVERSE"Summary: Result count = "${RESULT_CNT}" "$aREVERSEr 1073 | fi 1074 | else 1075 | echo -e "\n"${cRESET}${cIND}$aREVERSE"Summary: Result count = "${RESULT_CNT}" "$aREVERSEr 1076 | fi 1077 | 1078 | else 1079 | echo -e $cBYEL 1080 | if [ -z "$CMDCOUNT" ];then # v1.10 1081 | if [ "$CMDREPORT" = "CreateCSV" ];then # v1.12 1082 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 -header -csv $SQL_DATABASE SELECT * FROM $SQL_TABLE;\n" # v1.13 1083 | sqlite3 -header -csv $SQL_DATABASE "SELECT * FROM $SQL_TABLE;" > $REPORT_CSV # Use '*' for raw table 1084 | else 1085 | # NOTE: Display/create the additional human-friendly timestamp! 1086 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_DATABASE SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, timestamp, mac, url FROM $SQL_TABLE $WHERE;\n" # v1.13 1087 | sqlite3 $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, timestamp, mac, url FROM $SQL_TABLE $WHERE;" 1088 | fi 1089 | fi 1090 | [ $SHOWSQL -eq 1 ] && echo -e "sqlite3 $SQL_DATABASE SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, count(*) FROM $SQL_TABLE $WHERE;\n" # v1.13 1091 | SQL_TOTAL=$(sqlite3 $SQL_DATABASE "SELECT datetime(timestamp, 'unixepoch', 'localtime') AS time, count(*) FROM $SQL_TABLE $WHERE;" | cut -d'|' -f2) 1092 | echo -e $cBGRE"\nTotal Records = "$SQL_TOTAL 1093 | fi 1094 | 1095 | # "dpi: TrendMicro function can't use under load-balance mode" 1096 | # v1.10 Moved to after summary report 1097 | if [ $(nvram get bwdpi_wh_enable) -eq 0 ];then 1098 | echo -e $cBRED"\a\n**Warning $SQL_DB_DESC logging NOT currently enabled"$cRESET 1099 | #exit 97 1100 | fi 1101 | 1102 | echo -e $cRESET 1103 | 1104 | exit 0 1105 | 1106 | 1107 | --------------------------------------------------------------------------------