├── .gitignore ├── LICENSE ├── README.md ├── library ├── graylog_collector_configurations.py ├── graylog_index_sets.py ├── graylog_input.py ├── graylog_input_gelf.py ├── graylog_input_rsyslog.py ├── graylog_ldap.py ├── graylog_ldap_groups.py ├── graylog_pipelines.py ├── graylog_roles.py ├── graylog_streams.py └── graylog_users.py └── main.yml /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | test* 3 | inventory 4 | tmp-* -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ansible-graylog-modules 2 | Ansible modules for the [Graylog2/graylog2-server](https://github.com/graylog2/graylog2-server) API 3 | 4 | A full example playbook can be found in `main.yml`. 5 | 6 | ### In Progress 7 | 8 | * Indices 9 | * Inputs 10 | 11 | ### Modules 12 | 13 | The following modules are available with the corresponding actions: 14 | 15 | * graylog_users 16 | * create 17 | * update 18 | * delete 19 | * list 20 | * graylog_roles 21 | * create 22 | * update 23 | * delete 24 | * list 25 | * graylog_streams 26 | * create 27 | * create_rule 28 | * update 29 | * update_rule 30 | * delete 31 | * delete_rule 32 | * list 33 | * query_streams - query by stream name (ie: to get stream ID) 34 | * graylog_pipelines 35 | * create 36 | * create_rule 37 | * create_connection 38 | * update 39 | * update_rule 40 | * update_connection 41 | * parse_rule 42 | * delete 43 | * delete_rule 44 | * list 45 | * query_pipelines - query by pipeline name (ie: to get pipeline ID) 46 | * query_rules - query by rule name (ie: to get rule ID) 47 | * graylog_index_sets 48 | * create 49 | * update 50 | * delete 51 | * list 52 | * query_index_sets - query by index set name (ie: to get index set ID) 53 | * graylog_collector_configurations 54 | * list_configurations 55 | * query_collector_configurations 56 | * update_snippet 57 | * graylog_ldap 58 | * get 59 | * update 60 | * delete 61 | * test 62 | * graylog_input 63 | * list 64 | * delete 65 | * graylog_input_rsyslog 66 | * create 67 | * update 68 | * graylog_input_gelf 69 | * create 70 | * update 71 | 72 | 73 | ### Examples 74 | 75 | #### Users 76 | 77 | ``` 78 | - name: Create Graylog user 79 | graylog_users: 80 | action: create 81 | endpoint: "{{ endpoint }}" 82 | graylog_user: "{{ graylog_user }}" 83 | graylog_password: "{{ graylog_password }}" 84 | username: "{{ username }}" 85 | full_name: "Ansible User" 86 | password: "{{ password }}" 87 | email: "test-email@aol.com" 88 | roles: 89 | - "ansible_role" 90 | 91 | - name: Get Graylog users 92 | graylog_users: 93 | endpoint: "{{ endpoint }}" 94 | graylog_user: "{{ graylog_user }}" 95 | graylog_password: "{{ graylog_password }}" 96 | register: graylog_users 97 | 98 | - name: List users 99 | debug: 100 | msg: "{{ graylog_users.json }}" 101 | ``` 102 | 103 | #### Roles 104 | 105 | ``` 106 | - name: Create Graylog role 107 | graylog_roles: 108 | action: create 109 | endpoint: "{{ endpoint }}" 110 | graylog_user: "admin" 111 | graylog_password: "{{ graylog_password }}" 112 | name: "ansible_role" 113 | description: "Ansible test role" 114 | permissions: 115 | - "dashboards:read" 116 | read_only: "true" 117 | 118 | - name: Get Graylog roles 119 | graylog_roles: 120 | endpoint: "{{ endpoint }}" 121 | graylog_user: "{{ graylog_user }}" 122 | graylog_password: "{{ graylog_password }}" 123 | register: graylog_roles 124 | 125 | - name: List roles 126 | debug: 127 | msg: "{{ graylog_roles.json }}" 128 | ``` 129 | 130 | #### Streams and Stream Rules 131 | 132 | ``` 133 | - name: Create stream 134 | graylog_streams: 135 | action: create 136 | endpoint: "{{ endpoint }}" 137 | graylog_user: "{{ graylog_user }}" 138 | graylog_password: "{{ graylog_password }}" 139 | title: "test_stream" 140 | description: "Windows and IIS logs" 141 | matching_type: "AND" 142 | remove_matches_from_default_stream: False 143 | rules: 144 | - {"field":"message","type":1,"value":"test_stream rule","inverted": false,"description":"test_stream rule"} 145 | 146 | - name: Get Graylog streams 147 | graylog_streams: 148 | endpoint: "{{ endpoint }}" 149 | graylog_user: "{{ graylog_user }}" 150 | graylog_password: "{{ graylog_password }}" 151 | register: graylog_streams 152 | 153 | - name: Get stream from stream name query 154 | graylog_streams: 155 | action: query_streams 156 | endpoint: "{{ endpoint }}" 157 | graylog_user: "{{ graylog_user }}" 158 | graylog_password: "{{ graylog_password }}" 159 | stream_name: "test_stream" 160 | register: stream 161 | 162 | - name: List single stream by ID 163 | graylog_streams: 164 | endpoint: "{{ endpoint }}" 165 | graylog_user: "{{ graylog_user }}" 166 | graylog_password: "{{ graylog_password }}" 167 | stream_id: "{{ stream.json.id }}" 168 | 169 | - name: Create stream rule 170 | graylog_streams: 171 | action: create_rule 172 | endpoint: "{{ endpoint }}" 173 | graylog_user: "{{ graylog_user }}" 174 | graylog_password: "{{ graylog_password }}" 175 | stream_id: "{{ stream.json.id }}" 176 | description: "Windows Security Logs" 177 | field: "winlogbeat_log_name" 178 | type: "1" 179 | value: "Security" 180 | inverted: False 181 | ``` 182 | 183 | #### Pipelines, Pipeline Rules, Stream connections 184 | 185 | ``` 186 | - name: Create pipeline rule 187 | graylog_pipelines: 188 | action: create_rule 189 | endpoint: "{{ endpoint }}" 190 | graylog_user: "{{ graylog_user }}" 191 | graylog_password: "{{ graylog_password }}" 192 | title: "test_rule" 193 | description: "test" 194 | source: | 195 | rule "test_rule_domain_threat_intel" 196 | when 197 | has_field("dns_query") 198 | then 199 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 200 | set_fields(dns_query_intel); 201 | end 202 | 203 | - name: Create pipeline 204 | graylog_pipelines: 205 | action: create 206 | endpoint: "{{ endpoint }}" 207 | graylog_user: "{{ graylog_user }}" 208 | graylog_password: "{{ graylog_password }}" 209 | title: "test_pipeline" 210 | source: | 211 | pipeline "test_pipeline" 212 | stage 0 match either 213 | end 214 | description: "test_pipeline description" 215 | 216 | - name: Get pipeline from pipeline name 217 | graylog_pipelines: 218 | action: query_pipelines 219 | endpoint: "{{ endpoint }}" 220 | graylog_user: "{{ graylog_user }}" 221 | graylog_password: "{{ graylog_password }}" 222 | pipeline_name: "test_pipeline" 223 | register: pipeline 224 | 225 | - name: Update pipeline with new rule 226 | graylog_pipelines: 227 | action: update 228 | endpoint: "{{ endpoint }}" 229 | graylog_user: "{{ graylog_user }}" 230 | graylog_password: "{{ graylog_password }}" 231 | pipeline_id: "{{ pipeline.json.id }}" 232 | description: "test description update" 233 | source: | 234 | pipeline "test_pipeline" 235 | stage 0 match either 236 | rule "test_rule_domain_threat_intel" 237 | end 238 | 239 | - name: Create Stream connection to processing pipeline 240 | graylog_pipelines: 241 | action: create_connection 242 | endpoint: "{{ endpoint }}" 243 | graylog_user: "{{ graylog_user }}" 244 | graylog_password: "{{ graylog_password }}" 245 | pipeline_id: "{{ pipeline.json.id }}" 246 | stream_ids: 247 | - "{{ stream.json.id }}" 248 | ``` 249 | 250 | #### Index Sets and attach Streams 251 | 252 | ``` 253 | - name: Create index set 254 | graylog_index_sets: 255 | action: create 256 | endpoint: "{{ endpoint }}" 257 | graylog_user: "{{ graylog_user }}" 258 | graylog_password: "{{ graylog_password }}" 259 | title: "test_index_set" 260 | index_prefix: "test_index_" 261 | description: "test index set" 262 | 263 | - name: Get index set by name 264 | graylog_index_sets: 265 | action: query_index_sets 266 | endpoint: "{{ endpoint }}" 267 | graylog_user: "{{ graylog_user }}" 268 | graylog_password: "{{ graylog_password }}" 269 | title: "test_index_set" 270 | register: index_set 271 | 272 | - name: Update stream to use new index set 273 | graylog_streams: 274 | action: update 275 | endpoint: "{{ endpoint }}" 276 | graylog_user: "{{ graylog_user }}" 277 | graylog_password: "{{ graylog_password }}" 278 | stream_id: "{{ stream.json.id }}" 279 | index_set_id: "{{ index_set.json.id }}" 280 | ``` 281 | 282 | #### LDAP configuration 283 | ``` 284 | - name: Setup Active Directory authentication without SSL and set "Reader" as default role 285 | graylog_ldap: 286 | endpoint: "graylog.mydomain.com" 287 | graylog_user: "username" 288 | graylog_password: "password" 289 | enabled: "true" 290 | active_directory: "true" 291 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 292 | system_password_set: "true" 293 | system_username: "ldapbind@mydomain.com" 294 | system_password: "bindPassw0rd" 295 | search_base: "cn=users,dc=mydomain,dc=com" 296 | search_pattern: "(&(objectClass=user)(sAMAccountName={0}))" 297 | display_name_attribute: "displayName" 298 | group_search_base: "cn=groups,dc=mydomain,dc=com" 299 | group_search_pattern: "(&(objectClass=group)(cn=graylog*))" 300 | group_id_attribute: "cn" 301 | 302 | - name: Remove current LDAP authentication configuration 303 | graylog_ldap: 304 | endpoint: "graylog.mydomain.com" 305 | graylog_user: "username" 306 | graylog_password: "password" 307 | action: "delete" 308 | 309 | -name: Get current LDAP authentication configuration 310 | graylog_ldap: 311 | endpoint: "graylog.mydomain.com" 312 | graylog_user: "username" 313 | graylog_password: "password" 314 | action: "get" 315 | register: currentConfiguration 316 | 317 | - name: Print current LDAP authentication configuration 318 | debug: 319 | msg: "{{ currentConfiguration }}" 320 | 321 | - name: Test LDAP bind 322 | graylog_ldap: 323 | endpoint: "graylog.mydomain.com" 324 | graylog_user: "username" 325 | graylog_password: "password" 326 | action: "test" 327 | active_directory: "true" 328 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 329 | system_password_set: "true" 330 | system_username: "ldapbind@mydomain.com" 331 | system_password: "bindPassw0rd" 332 | ``` 333 | 334 | #### LDAP group mapping 335 | ``` 336 | - name: Get all LDAP groups 337 | graylog_ldap_groups: 338 | endpoint: "graylog.mydomain.com" 339 | graylog_user: "username" 340 | graylog_password: "password" 341 | action: "list" 342 | 343 | - name: Get the LDAP group to Graylog role mapping 344 | graylog_ldap_groups: 345 | endpoint: "graylog.mydomain.com" 346 | graylog_user: "username" 347 | graylog_password: "password" 348 | action: "list_mapping" 349 | 350 | - name: Update the LDAP group to Graylog role mapping 351 | graylog_ldap_groups: 352 | endpoint: "graylog.mydomain.com" 353 | graylog_user: "username" 354 | graylog_password: "password" 355 | action: "update" 356 | group: "{{ item.group }}" 357 | role: "{{ item.role }}" 358 | with_items: 359 | - { group : "ldap-group-admins", role : "Admin" } 360 | - { group : "ldap-group-read", role : "Reader" } 361 | ``` 362 | 363 | #### Input managment 364 | ``` 365 | - name: Display all inputs 366 | graylog_input: 367 | endpoint: "{{ graylog_endpoint }}" 368 | graylog_user: "{{ graylog_user }}" 369 | graylog_password: "{{ graylog_password }}" 370 | allow_http: "true" 371 | validate_certs: "false" 372 | action: "list" 373 | 374 | - name: Remove input with ID 1df0f1234abcd0000d0adf20 375 | graylog_input: 376 | endpoint: "{{ graylog_endpoint }}" 377 | graylog_user: "{{ graylog_user }}" 378 | graylog_password: "{{ graylog_password }}" 379 | allow_http: "true" 380 | validate_certs: "false" 381 | action: "delete" 382 | input_id: "1df0f1234abcd0000d0adf20" 383 | 384 | - name: Create Rsyslog TCP input 385 | graylog_input_syslog: 386 | endpoint: "{{ graylog_endpoint }}" 387 | graylog_user: "{{ graylog_user }}" 388 | graylog_password: "{{ graylog_password }}" 389 | allow_http: "true" 390 | validate_certs: "false" 391 | action: "create" 392 | input_type: "TCP" 393 | title: "Rsyslog TCP" 394 | global_input: "true" 395 | allow_override_date: "true" 396 | bind_address: "0.0.0.0" 397 | expand_structured_data: "false" 398 | force_rdns: "false" 399 | number_worker_threads: "2" 400 | port: "514" 401 | recv_buffer_size: "1048576" 402 | store_full_message: "true" 403 | 404 | - name: Create GELF HTTP input 405 | graylog_input_gelf: 406 | endpoint: "{{ graylog_endpoint }}" 407 | graylog_user: "{{ graylog_user }}" 408 | graylog_password: "{{ graylog_password }}" 409 | allow_http: "true" 410 | validate_certs: "false" 411 | action: "create" 412 | input_type: "HTTP" 413 | title: "Test input GELF HTTP" 414 | global_input: "true" 415 | bind_address: "0.0.0.0" 416 | 417 | - name: Update existing RSyslog input 418 | graylog_input_rsyslog: 419 | endpoint: "{{ graylog_endpoint }}" 420 | graylog_user: "{{ graylog_user }}" 421 | graylog_password: "{{ graylog_password }}" 422 | allow_http: "true" 423 | validate_certs: "false" 424 | action: "update" 425 | input_type: "TCP" 426 | title: "Rsyslog TCP" 427 | global_input: "true" 428 | allow_override_date: "true" 429 | expand_structured_data: "false" 430 | force_rdns: "true" 431 | port: "1514" 432 | store_full_message: "true" 433 | input_id: "1df0f1234abcd0000d0adf20" 434 | ``` -------------------------------------------------------------------------------- /library/graylog_collector_configurations.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_collector_configurations 14 | short_description: Communicate with the Graylog API to manage collector configurations 15 | description: 16 | - The Graylog collector_configurations module manages Graylog collector configurations. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endoint. (i.e. graylog.mydomain.com). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against collector configuration API. 50 | required: false 51 | default: list_configurations 52 | choices: [ list_configurations, query_collector_configurations ] 53 | type: str 54 | configuration_id: 55 | description: 56 | - Configuration id. 57 | required: false 58 | type: str 59 | configuration_name: 60 | description: 61 | - Configuration name. 62 | required: false 63 | type: str 64 | configuration_tags: 65 | description: 66 | - Configuration tags. 67 | required: false 68 | type: str 69 | snippet_name: 70 | description: 71 | - Snippet name. 72 | required: false 73 | type: str 74 | snippet_source: 75 | description: 76 | - Snippet source code. 77 | required: false 78 | type: str 79 | backend: 80 | description: 81 | - Snippet backend, ex: winlogbeat, filebeat, nxlog 82 | required: false 83 | type: str 84 | ''' 85 | 86 | EXAMPLES = ''' 87 | # List collector configurations 88 | - graylog_collector_configurations: 89 | endpoint: "graylog.mydomain.com" 90 | graylog_user: "username" 91 | graylog_password: "password" 92 | 93 | # Get collector configuration from configuration name query_collector_configurations 94 | - graylog_collector_configurations: 95 | action: query_collector_configurations 96 | endpoint: "graylog.mydomain.com" 97 | graylog_user: "username" 98 | graylog_password: "password" 99 | configuration_name: "windows-collector-confiuration" 100 | register: configuration 101 | 102 | # Update snippet for a configuration 103 | - graylog_collector_configurations: 104 | action: update_snippet 105 | endpoint: "graylog.mydomain.com" 106 | graylog_user: "username" 107 | graylog_password: "password" 108 | configuration_name: "windows-collector-configuration" 109 | snippet_name: "client-x" 110 | snippet_source: | 111 | # filebeat or winlog beat source here 112 | register: configuration 113 | ''' 114 | 115 | RETURN = ''' 116 | json: 117 | description: The JSON response from the Graylog API 118 | returned: always 119 | type: str 120 | status: 121 | description: The HTTP status code from the request 122 | returned: always 123 | type: int 124 | sample: 200 125 | url: 126 | description: The actual URL used for the request 127 | returned: always 128 | type: str 129 | sample: https://www.ansible.com/ 130 | ''' 131 | 132 | 133 | # import module snippets 134 | import json 135 | import base64 136 | from ansible.module_utils.basic import AnsibleModule 137 | from ansible.module_utils.urls import fetch_url, to_text 138 | 139 | 140 | def list_configurations(module, configuration_url, headers, configuration_id, query): 141 | 142 | if configuration_id is not None and configuration_id != "": 143 | url = "/".join([configuration_url, configuration_id]) 144 | elif query == "yes" and configuration_id == "": 145 | url = "/".join([configuration_url, "0"]) 146 | else: 147 | url = configuration_url 148 | 149 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 150 | 151 | if info['status'] != 200: 152 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 153 | 154 | try: 155 | content = to_text(response.read(), errors='surrogate_or_strict') 156 | except AttributeError: 157 | content = info.pop('body', '') 158 | 159 | return info['status'], info['msg'], content, url 160 | 161 | 162 | def query_collector_configurations(module, configuration_url, headers, configuration_name): 163 | 164 | url = configuration_url 165 | 166 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 167 | 168 | if info['status'] != 200: 169 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 170 | 171 | try: 172 | content = to_text(response.read(), errors='surrogate_or_strict') 173 | collector_configurations = json.loads(content) 174 | except AttributeError: 175 | content = info.pop('body', '') 176 | 177 | configuration_id = "" 178 | if collector_configurations is not None: 179 | 180 | i = 0 181 | while i < len(collector_configurations['configurations']): 182 | configuration = collector_configurations['configurations'][i] 183 | if configuration_name == configuration['name']: 184 | configuration_id = configuration['id'] 185 | break 186 | i += 1 187 | 188 | return configuration_id 189 | 190 | 191 | def query_snippets(module, configuration_url, headers, configuration_id, snippet_name): 192 | 193 | url = "/".join([configuration_url, configuration_id]) 194 | 195 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 196 | 197 | if info['status'] != 200: 198 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 199 | 200 | try: 201 | content = to_text(response.read(), errors='surrogate_or_strict') 202 | configuration = json.loads(content) 203 | snippets = configuration['snippets'] 204 | except AttributeError: 205 | content = info.pop('body', '') 206 | 207 | snippet_id = "" 208 | if snippets is not None: 209 | 210 | i = 0 211 | while i < len(snippets): 212 | snippet = snippets[i] 213 | if snippet_name == snippet['name']: 214 | snippet_id = snippet['snippet_id'] 215 | break 216 | i += 1 217 | 218 | return snippet_id 219 | 220 | 221 | def update_snippet(module, configuration_url, headers, configuration_id, snippet_id): 222 | 223 | url = "/".join([configuration_url, configuration_id, "snippets", snippet_id]) 224 | 225 | payload = {} 226 | 227 | for key in ['backend', 'snippet_name', 'snippet_source']: 228 | if module.params[key] is not None: 229 | payload[key] = module.params[key] 230 | 231 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 232 | 233 | if info['status'] != 202: 234 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 235 | 236 | try: 237 | content = to_text(response.read(), errors='surrogate_or_strict') 238 | except AttributeError: 239 | content = info.pop('body', '') 240 | 241 | return info['status'], info['msg'], content, url 242 | 243 | 244 | def get_token(module, endpoint, username, password): 245 | 246 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 247 | 248 | url = endpoint + "/api/system/sessions" 249 | 250 | payload = { 251 | 'username': username, 252 | 'password': password, 253 | 'host': endpoint 254 | } 255 | 256 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 257 | 258 | if info['status'] != 200: 259 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 260 | 261 | try: 262 | content = to_text(response.read(), errors='surrogate_or_strict') 263 | session = json.loads(content) 264 | except AttributeError: 265 | content = info.pop('body', '') 266 | 267 | session_string = session['session_id'] + ":session" 268 | session_bytes = session_string.encode('utf-8') 269 | session_token = base64.b64encode(session_bytes) 270 | 271 | return session_token 272 | 273 | 274 | def main(): 275 | module = AnsibleModule( 276 | argument_spec=dict( 277 | endpoint=dict(type='str'), 278 | graylog_user=dict(type='str'), 279 | graylog_password=dict(type='str', no_log=True), 280 | allow_http=dict(type='bool', required=False, default=False), 281 | validate_certs=dict(type='bool', required=False, default=True), 282 | action=dict(type='str', required=False, default='list_configurations', 283 | choices=['list_configurations', 'query_collector_configurations', 'update_snippet']), 284 | configuration_id=dict(type='str'), 285 | configuration_name=dict(type='str'), 286 | configuration_tags=dict(type='list'), 287 | snippet_name=dict(type='str'), 288 | snippet_source=dict(type='str'), 289 | backend=dict(type='str') 290 | ) 291 | ) 292 | 293 | endpoint = module.params['endpoint'] 294 | graylog_user = module.params['graylog_user'] 295 | graylog_password = module.params['graylog_password'] 296 | action = module.params['action'] 297 | configuration_id = module.params['configuration_id'] 298 | configuration_name = module.params['configuration_name'] 299 | snippet_name = module.params['snippet_name'] 300 | allow_http = module.params['allow_http'] 301 | 302 | if allow_http == True: 303 | endpoint = "http://" + endpoint 304 | else: 305 | endpoint = "https://" + endpoint 306 | 307 | configuration_url = endpoint + "/api/plugins/org.graylog.plugins.collector/configurations" 308 | 309 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 310 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 311 | "Authorization": "Basic ' + api_token.decode() + '" }' 312 | 313 | if action == "list_configurations": 314 | query = "no" 315 | status, message, content, url = list_configurations(module, configuration_url, headers, configuration_id, query) 316 | elif action == "update_snippet": 317 | configuration_id = query_collector_configurations(module, configuration_url, headers, configuration_name) 318 | snippet_id = query_snippets(module, configuration_url, headers, configuration_id, snippet_name) 319 | status, message, content, url = update_snippet(module, configuration_url, headers, configuration_id, snippet_id) 320 | elif action == "query_collector_configurations": 321 | configuration_id = query_collector_configurations(module, configuration_url, headers, configuration_name) 322 | query = "yes" 323 | status, message, content, url = list_configurations(module, configuration_url, headers, configuration_id, query) 324 | 325 | uresp = {} 326 | content = to_text(content, encoding='UTF-8') 327 | 328 | try: 329 | js = json.loads(content) 330 | except ValueError: 331 | js = "" 332 | 333 | uresp['json'] = js 334 | uresp['status'] = status 335 | uresp['msg'] = message 336 | uresp['url'] = url 337 | 338 | module.exit_json(**uresp) 339 | 340 | 341 | if __name__ == '__main__': 342 | main() 343 | -------------------------------------------------------------------------------- /library/graylog_index_sets.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_index_sets 14 | short_description: Communicate with the Graylog API to manage index sets 15 | description: 16 | - The Graylog index sets module manages Graylog index sets. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endoint. (i.e. graylog.mydomain.com). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against index API. 50 | required: false 51 | default: list 52 | choices: [ create, update, list, delete, query_index_sets ] 53 | type: str 54 | id: 55 | description: 56 | - Index set id. 57 | required: false 58 | type: str 59 | title: 60 | description: 61 | - Title. 62 | required: false 63 | type: str 64 | description: 65 | description: 66 | - Description. 67 | required: false 68 | type: str 69 | creation_date: 70 | description: 71 | - Index set creation date. 72 | required: false 73 | type: str 74 | index_prefix: 75 | description: 76 | - A unique prefix used in Elasticsearch indices belonging to this index set. The prefix must start 77 | with a letter or number, and can only contain letters, numbers, '_', '-' and '+'. 78 | required: false 79 | type: str 80 | field_type_refresh_interval: 81 | description: 82 | - How often the field type information for the active write index will be updated. 83 | required: false 84 | type: int 85 | default: 5000 86 | index_analyzer: 87 | description: 88 | - Elasticsearch analyzer for this index set. 89 | required: false 90 | default: "standard" 91 | type: str 92 | shards: 93 | description: 94 | - Number of Elasticsearch shards used per index in this index set. 95 | required: false 96 | default: 4 97 | type: int 98 | replicas: 99 | description: 100 | - Number of Elasticsearch replicas used per index in this index set. 101 | required: false 102 | default: 1 103 | type: int 104 | rotation_strategy_class: 105 | description: 106 | - Rotation strategy class, ex. org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy 107 | required: false 108 | default: "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy" 109 | type: str 110 | retention_strategy_class: 111 | description: 112 | - Retention strategy class, ex. org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy 113 | required: false 114 | default: "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy" 115 | type: str 116 | rotation_strategy: 117 | description: 118 | - Graylog uses multiple indices to store documents in. You can configure the strategy it uses to determine 119 | when to rotate the currently active write index. 120 | required: false 121 | default: {'type': 'org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig', 'rotation_period': 'P1D'} 122 | type: dict 123 | retention_strategy: 124 | description: 125 | - Graylog uses a retention strategy to clean up old indices. 126 | required: false 127 | default: {'type': 'org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig', 'max_number_of_indices': 14} 128 | type: dict 129 | index_optimization_max_num_segments: 130 | description: 131 | - Maximum number of segments per Elasticsearch index after optimization (force merge). 132 | required: false 133 | default: 1 134 | type: int 135 | index_optimization_disabled: 136 | description: 137 | - Disable Elasticsearch index optimization (force merge) after rotation. 138 | required: false 139 | default: False 140 | type: bool 141 | writable: 142 | description: 143 | - Writable, true or false. 144 | required: false 145 | default: True 146 | type: bool 147 | default: 148 | description: 149 | - Default index set, true or false. 150 | required: false 151 | default: False 152 | type: bool 153 | ''' 154 | 155 | EXAMPLES = ''' 156 | # List index sets 157 | - graylog_index_sets: 158 | endpoint: "graylog.mydomain.com" 159 | graylog_user: "username" 160 | graylog_password: "password" 161 | 162 | # Create index rule 163 | - graylog_index_sets: 164 | action: create_rule 165 | endpoint: "graylog.mydomain.com" 166 | graylog_user: "username" 167 | graylog_password: "password" 168 | title: "test_rule" 169 | description: "test" 170 | source: | 171 | rule "test_rule_domain_threat_intel" 172 | when 173 | has_field("dns_query") 174 | then 175 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 176 | set_fields(dns_query_intel); 177 | end 178 | ''' 179 | 180 | RETURN = ''' 181 | json: 182 | description: The JSON response from the Graylog API 183 | returned: always 184 | type: complex 185 | contains: 186 | title: 187 | description: Title. 188 | returned: success 189 | type: str 190 | sample: 'Graylog' 191 | creation_date: 192 | description: Index set creation date. 193 | returned: success 194 | type: str 195 | sample: '2019-01-21T19:45:09.098Z' 196 | default: 197 | description: Whether or not it is the default index set. 198 | returned: success 199 | type: bool 200 | sample: false 201 | description: 202 | description: Index set description. 203 | returned: success 204 | type: str 205 | sample: 'Client X index set' 206 | field_type_refresh_interval: 207 | description: How often the field type information for the active write index will be updated. 208 | returned: success 209 | type: int 210 | sample: 5000 211 | id: 212 | description: Index set ID. 213 | returned: success 214 | type: str 215 | sample: '4a362233815c349e7e2b945c' 216 | index_analyzer: 217 | description: Index set analyzer. 218 | returned: success 219 | type: str 220 | sample: 'standard' 221 | index_optimization_disabled: 222 | description: Whether index set optimization is enabled or disabled. 223 | returned: success 224 | type: bool 225 | sample: false 226 | index_optimization_max_num_segments: 227 | description: Index optimization segments. 228 | returned: success 229 | type: int 230 | sample: 1 231 | index_prefix: 232 | description: Index set prefix. 233 | returned: success 234 | type: str 235 | sample: 'graylog' 236 | replicas: 237 | description: Number of replicas. 238 | returned: success 239 | type: int 240 | sample: 1 241 | retention_strategy: 242 | description: Index set retention strategy. 243 | returned: success 244 | type: dict 245 | sample: { "max_number_of_indices": 720, "type": "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig" } 246 | rotation_strategy: 247 | description: Index set rotation strategy. 248 | returned: success 249 | type: dict 250 | sample: { "rotation_period": "PT6H", "type": "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig" } 251 | retention_strategy_class: 252 | description: Retention strategy class. 253 | returned: success 254 | type: str 255 | sample: 'org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy' 256 | rotation_strategy_class: 257 | description: Rotation strategy class. 258 | returned: success 259 | type: str 260 | sample: 'org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy' 261 | shards: 262 | description: Number of shards. 263 | returned: success 264 | type: int 265 | sample: 4 266 | writable: 267 | description: Whether or not index set is writable. 268 | returned: success 269 | type: bool 270 | sample: true 271 | status: 272 | description: The HTTP status code from the request 273 | returned: always 274 | type: int 275 | sample: 200 276 | url: 277 | description: The actual URL used for the request 278 | returned: always 279 | type: str 280 | sample: https://www.ansible.com/ 281 | ''' 282 | 283 | 284 | # import module snippets 285 | import json 286 | import datetime 287 | import base64 288 | from ansible.module_utils.basic import AnsibleModule 289 | from ansible.module_utils.urls import fetch_url, to_text 290 | 291 | 292 | def create(module, base_url, headers, creation_date): 293 | 294 | url = base_url 295 | 296 | payload = {} 297 | 298 | for key in ['title', 'description', 'index_prefix', 'field_type_refresh_interval', 'writable', 'default', 299 | 'index_analyzer', 'shards', 'replicas', 'rotation_strategy_class', 'retention_strategy_class', 300 | 'rotation_strategy', 'retention_strategy', 'index_optimization_max_num_segments', 301 | 'index_optimization_disabled']: 302 | if module.params[key] is not None: 303 | payload[key] = module.params[key] 304 | 305 | payload['creation_date'] = creation_date 306 | 307 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 308 | 309 | if info['status'] != 200: 310 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 311 | 312 | try: 313 | content = to_text(response.read(), errors='surrogate_or_strict') 314 | except AttributeError: 315 | content = info.pop('body', '') 316 | 317 | return info['status'], info['msg'], content, url 318 | 319 | 320 | def update(module, base_url, headers): 321 | 322 | url = "/".join([base_url, module.params['id']]) 323 | 324 | payload = {} 325 | 326 | for key in ['title', 'description', 'index_prefix', 'field_type_refresh_interval', 'writable', 'default', 327 | 'index_analyzer', 'shards', 'replicas', 'rotation_strategy_class', 'retention_strategy_class', 328 | 'rotation_strategy', 'retention_strategy', 'index_optimization_max_num_segments', 329 | 'index_optimization_disabled']: 330 | if module.params[key] is not None: 331 | payload[key] = module.params[key] 332 | 333 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 334 | 335 | if info['status'] != 200: 336 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 337 | 338 | try: 339 | content = to_text(response.read(), errors='surrogate_or_strict') 340 | except AttributeError: 341 | content = info.pop('body', '') 342 | 343 | return info['status'], info['msg'], content, url 344 | 345 | 346 | def delete(module, base_url, headers, id): 347 | 348 | url = base_url + "/%s" % (id) 349 | 350 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 351 | 352 | if info['status'] != 204: 353 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 354 | 355 | try: 356 | content = to_text(response.read(), errors='surrogate_or_strict') 357 | except AttributeError: 358 | content = info.pop('body', '') 359 | 360 | return info['status'], info['msg'], content, url 361 | 362 | 363 | def list(module, base_url, headers, id): 364 | 365 | if id is not None: 366 | url = base_url + "/%s" % (id) 367 | else: 368 | url = base_url 369 | 370 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 371 | 372 | if info['status'] != 200: 373 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 374 | 375 | try: 376 | content = to_text(response.read(), errors='surrogate_or_strict') 377 | except AttributeError: 378 | content = info.pop('body', '') 379 | 380 | return info['status'], info['msg'], content, url 381 | 382 | 383 | def query_index_sets(module, base_url, headers, title): 384 | 385 | url = base_url 386 | 387 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 388 | 389 | if info['status'] != 200: 390 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 391 | 392 | try: 393 | content = to_text(response.read(), errors='surrogate_or_strict') 394 | index_sets = json.loads(content) 395 | except AttributeError: 396 | content = info.pop('body', '') 397 | 398 | id = "" 399 | if index_sets is not None: 400 | 401 | i = 0 402 | while i < len(index_sets['index_sets']): 403 | index_set = index_sets['index_sets'][i] 404 | if title == index_set['title']: 405 | id = index_set['id'] 406 | break 407 | i += 1 408 | 409 | return id 410 | 411 | 412 | def get_token(module, endpoint, username, password): 413 | 414 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 415 | 416 | url = endpoint + "/api/system/sessions" 417 | 418 | payload = { 419 | 'username': username, 420 | 'password': password, 421 | 'host': endpoint 422 | } 423 | 424 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 425 | 426 | if info['status'] != 200: 427 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 428 | 429 | try: 430 | content = to_text(response.read(), errors='surrogate_or_strict') 431 | session = json.loads(content) 432 | except AttributeError: 433 | content = info.pop('body', '') 434 | 435 | session_string = session['session_id'] + ":session" 436 | session_bytes = session_string.encode('utf-8') 437 | session_token = base64.b64encode(session_bytes) 438 | 439 | return session_token 440 | 441 | 442 | def main(): 443 | module = AnsibleModule( 444 | argument_spec=dict( 445 | endpoint=dict(type='str'), 446 | graylog_user=dict(type='str'), 447 | graylog_password=dict(type='str', no_log=True), 448 | allow_http=dict(type='bool', required=False, default=False), 449 | validate_certs=dict(type='bool', required=False, default=True), 450 | action=dict(type='str', required=False, default='list', choices=['create', 'update', 'delete', 'list', 'query_index_sets']), 451 | title=dict(type='str'), 452 | description=dict(type='str'), 453 | creation_date=dict(type='str', required=False), 454 | id=dict(type='str'), 455 | index_prefix=dict(type='str'), 456 | index_analyzer=dict(type='str', default="standard"), 457 | shards=dict(type='int', default=4), 458 | replicas=dict(type='int', default=1), 459 | field_type_refresh_interval=dict(type='int', default=5000), 460 | rotation_strategy_class=dict(type='str', 461 | default='org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy'), 462 | retention_strategy_class=dict(type='str', default='org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy'), 463 | rotation_strategy=dict(type='dict', default=dict(type='org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig', 464 | rotation_period='P1D')), 465 | retention_strategy=dict(type='dict', default=dict(type='org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig', 466 | max_number_of_indices=14)), 467 | index_optimization_max_num_segments=dict(type='int', default=1), 468 | index_optimization_disabled=dict(type='bool', default=False), 469 | writable=dict(type='bool', default=True), 470 | default=dict(type='bool', default=False) 471 | ) 472 | ) 473 | 474 | endpoint = module.params['endpoint'] 475 | graylog_user = module.params['graylog_user'] 476 | graylog_password = module.params['graylog_password'] 477 | action = module.params['action'] 478 | allow_http = module.params['allow_http'] 479 | title = module.params['title'] 480 | id = module.params['id'] 481 | creation_date = module.params['creation_date'] or datetime.datetime.utcnow().isoformat() + 'Z' 482 | 483 | if allow_http == True: 484 | endpoint = "http://" + endpoint 485 | else: 486 | endpoint = "https://" + endpoint 487 | 488 | base_url = endpoint + "/api/system/indices/index_sets" 489 | 490 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 491 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 492 | "Authorization": "Basic ' + api_token.decode() + '" }' 493 | 494 | if action == "create": 495 | status, message, content, url = create(module, base_url, headers, creation_date) 496 | elif action == "update": 497 | status, message, content, url = update(module, base_url, headers) 498 | elif action == "delete": 499 | status, message, content, url = delete(module, base_url, headers, id) 500 | elif action == "list": 501 | status, message, content, url = list(module, base_url, headers, id) 502 | elif action == "query_index_sets": 503 | id = query_index_sets(module, base_url, headers, title) 504 | status, message, content, url = list(module, base_url, headers, id) 505 | 506 | uresp = {} 507 | content = to_text(content, encoding='UTF-8') 508 | 509 | try: 510 | js = json.loads(content) 511 | except ValueError: 512 | js = "" 513 | 514 | uresp['json'] = js 515 | uresp['status'] = status 516 | uresp['msg'] = message 517 | uresp['url'] = url 518 | 519 | module.exit_json(**uresp) 520 | 521 | 522 | if __name__ == '__main__': 523 | main() 524 | -------------------------------------------------------------------------------- /library/graylog_input.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # Copyright: (c) 2019, Matthieu SIMON 4 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 5 | 6 | ANSIBLE_METADATA = {'metadata_version': '1.1', 7 | 'status': ['preview'], 8 | 'supported_by': 'community'} 9 | 10 | DOCUMENTATION = ''' 11 | module: graylog_input 12 | short_description: Manage Graylog inputs 13 | description: 14 | - The Graylog inputs module allows configuration of inputs nodes. 15 | version_added: "2.9" 16 | author: "Matthieu SIMON" 17 | options: 18 | endpoint: 19 | description: 20 | - Graylog endoint. (i.e. graylog.mydomain.com). 21 | required: false 22 | type: str 23 | graylog_user: 24 | description: 25 | - Graylog privileged user username. 26 | required: false 27 | type: str 28 | graylog_password: 29 | description: 30 | - Graylog privileged user password. 31 | required: false 32 | type: str 33 | allow_http: 34 | description: 35 | - Allow non HTTPS connexion 36 | required: false 37 | default: false 38 | type: bool 39 | validate_certs: 40 | description: 41 | - Allow untrusted certificate 42 | required: false 43 | default: false 44 | type: bool 45 | action: 46 | description: 47 | - Action to take against LDAP API. 48 | required: true 49 | default: list 50 | choices: [ list, delete ] 51 | type: str 52 | input_id: 53 | description: 54 | - ID of input to remove 55 | required: false 56 | type: str 57 | ''' 58 | 59 | EXAMPLES = ''' 60 | - name: Display all inputs 61 | graylog_input: 62 | endpoint: "{{ graylog_endpoint }}" 63 | graylog_user: "{{ graylog_user }}" 64 | graylog_password: "{{ graylog_password }}" 65 | allow_http: "true" 66 | validate_certs: "false" 67 | action: "list" 68 | 69 | - name: Remove input with ID 1df0f1234abcd0000d0adf20 70 | graylog_input: 71 | endpoint: "{{ graylog_endpoint }}" 72 | graylog_user: "{{ graylog_user }}" 73 | graylog_password: "{{ graylog_password }}" 74 | allow_http: "true" 75 | validate_certs: "false" 76 | action: "delete" 77 | input_id: "1df0f1234abcd0000d0adf20" 78 | ''' 79 | 80 | # import module snippets 81 | import json 82 | import base64 83 | from ansible.module_utils.basic import AnsibleModule 84 | from ansible.module_utils.urls import fetch_url, to_text 85 | 86 | def delete(module, base_url, headers): 87 | 88 | url = base_url + "/" + module.params['input_id'] 89 | 90 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 91 | 92 | if info['status'] != 204: 93 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 94 | 95 | try: 96 | content = to_text(response.read(), errors='surrogate_or_strict') 97 | except AttributeError: 98 | content = info.pop('body', '') 99 | 100 | return info['status'], info['msg'], content, url 101 | 102 | def list(module, base_url, headers): 103 | 104 | url = base_url 105 | 106 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 107 | 108 | if info['status'] != 200: 109 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 110 | 111 | try: 112 | content = to_text(response.read(), errors='surrogate_or_strict') 113 | except AttributeError: 114 | content = info.pop('body', '') 115 | 116 | return info['status'], info['msg'], content, url 117 | 118 | def get_token(module, endpoint, username, password, allow_http): 119 | 120 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 121 | 122 | url = endpoint + "/api/system/sessions" 123 | 124 | payload = {} 125 | payload['username'] = username 126 | payload['password'] = password 127 | payload['host'] = endpoint 128 | 129 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 130 | 131 | if info['status'] != 200: 132 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 133 | 134 | try: 135 | content = to_text(response.read(), errors='surrogate_or_strict') 136 | session = json.loads(content) 137 | except AttributeError: 138 | content = info.pop('body', '') 139 | 140 | session_string = session['session_id'] + ":session" 141 | session_bytes = session_string.encode('utf-8') 142 | session_token = base64.b64encode(session_bytes) 143 | 144 | return session_token 145 | 146 | def main(): 147 | module = AnsibleModule( 148 | argument_spec=dict( 149 | endpoint=dict(type='str'), 150 | graylog_user=dict(type='str'), 151 | graylog_password=dict(type='str', no_log=True), 152 | validate_certs=dict(type='bool', required=False, default=True), 153 | allow_http=dict(type='bool', required=False, default=False), 154 | action=dict(type='str', required=False, default='list', 155 | choices=[ 'list' , 'delete' ]), 156 | input_id=dict(type='str', required=False ), 157 | ) 158 | ) 159 | 160 | endpoint = module.params['endpoint'] 161 | graylog_user = module.params['graylog_user'] 162 | graylog_password = module.params['graylog_password'] 163 | action = module.params['action'] 164 | allow_http = module.params['allow_http'] 165 | 166 | if allow_http == True: 167 | endpoint = "http://" + endpoint 168 | else: 169 | endpoint = "https://" + endpoint 170 | 171 | base_url = endpoint + "/api/system/inputs" 172 | 173 | api_token = get_token(module, endpoint, graylog_user, graylog_password, allow_http) 174 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 175 | "Authorization": "Basic ' + api_token.decode() + '" }' 176 | 177 | if action == "list": 178 | status, message, content, url = list(module, base_url, headers) 179 | elif action == "delete": 180 | status, message, content, url = delete(module, base_url, headers) 181 | 182 | uresp = {} 183 | content = to_text(content, encoding='UTF-8') 184 | 185 | try: 186 | js = json.loads(content) 187 | except ValueError: 188 | js = "" 189 | 190 | uresp['json'] = js 191 | uresp['status'] = status 192 | uresp['msg'] = message 193 | uresp['url'] = url 194 | 195 | module.exit_json(**uresp) 196 | 197 | 198 | if __name__ == '__main__': 199 | main() 200 | -------------------------------------------------------------------------------- /library/graylog_input_gelf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # Copyright: (c) 2019, Matthieu SIMON 4 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 5 | 6 | ANSIBLE_METADATA = {'metadata_version': '1.1', 7 | 'status': ['preview'], 8 | 'supported_by': 'community'} 9 | 10 | DOCUMENTATION = ''' 11 | module: graylog_inputs 12 | short_description: Manage Graylog inputs 13 | description: 14 | - The Graylog inputs module allows configuration of inputs nodes. 15 | version_added: "2.9" 16 | author: "Matthieu SIMON" 17 | options: 18 | endpoint: 19 | description: 20 | - Graylog endoint. (i.e. graylog.mydomain.com). 21 | required: false 22 | type: str 23 | graylog_user: 24 | description: 25 | - Graylog privileged user username. 26 | required: false 27 | type: str 28 | graylog_password: 29 | description: 30 | - Graylog privileged user password. 31 | required: false 32 | type: str 33 | allow_http: 34 | description: 35 | - Allow non HTTPS connexion 36 | required: false 37 | default: false 38 | type: bool 39 | validate_certs: 40 | description: 41 | - Allow untrusted certificate 42 | required: false 43 | default: false 44 | type: bool 45 | action: 46 | description: 47 | - Action to take against system/input API. 48 | - Warning : when update, all settings with default value set in this Ansible module (like bind_address, port ...) will replace existing values 49 | You must explicitly set these values if they differ from those by default 50 | required: true 51 | default: create 52 | choices: [ create, update ] 53 | type: str 54 | input_type: 55 | description: 56 | - Input type (not all are implemented at this time) 57 | required: true 58 | default: UDP 59 | choices: [ 'UDP', 'TCP', 'HTTP' ] 60 | type: str 61 | title: 62 | description: 63 | - Entitled of the input 64 | - Required with actions create, update and delete 65 | required: true 66 | type: str 67 | input_id: 68 | description: 69 | - ID of input to update 70 | required: false 71 | type: str 72 | global_input: 73 | description: 74 | - Input is present on all Graylog nodes 75 | required: false 76 | default: true 77 | type: bool 78 | node: 79 | description: 80 | - Node name if input is not global 81 | required: false 82 | type: str 83 | bind_address: 84 | description: 85 | - Address to listen on 86 | required: false 87 | default: "0.0.0.0" 88 | type: str 89 | port: 90 | description: 91 | - Port to listen on 92 | required: true 93 | default: 12201 94 | type: int 95 | number_worker_threads: 96 | description: 97 | - Number of worker threads processing network connections for this input. 98 | required: false 99 | default: 2 100 | type: int 101 | override_source: 102 | description: 103 | - The source is a hostname derived from the received packet by default. Set this if you want to override it with a custom string. 104 | required: false 105 | type: str 106 | recv_buffer_size: 107 | description: 108 | - The size in bytes of the recvBufferSize for network connections to this input. 109 | required: false 110 | default: 1048576 111 | type: int 112 | tcp_keepalive: 113 | description: 114 | - Enable TCP keepalive packets (TCP & HTTP only) 115 | required: false 116 | default: false 117 | type: bool 118 | tls_enable: 119 | description: 120 | - Accept TLS connections (TCP & HTTP only) 121 | required: false 122 | default: false 123 | type: bool 124 | tls_cert_file: 125 | description: 126 | - Path to the TLS certificate file (TCP & HTTP only) 127 | required: false 128 | type: str 129 | tls_key_file: 130 | description: 131 | - Path to the TLS private key file (TCP & HTTP only) 132 | required: false 133 | type: str 134 | tls_key_password: 135 | description: 136 | - The password for the encrypted key file. (TCP & HTTP only) 137 | required: false 138 | type: str 139 | tls_client_auth: 140 | description: 141 | - Whether clients need to authenticate themselves in a TLS connection (TCP & HTTP only) 142 | required: false 143 | default: disabled 144 | choices: [ 'disabled', 'optional', 'required' ] 145 | tls_client_auth_cert_file: 146 | description: 147 | - TLS Client Auth Trusted Certs (File or Directory) (TCP & HTTP only) 148 | required: false 149 | type: str 150 | use_null_delimiter: 151 | description: 152 | - Use null byte as frame delimiter ? Otherwise newline delimiter is used. (TCP Only) 153 | required: false 154 | default: false 155 | type: bool 156 | decompress_size_limit: 157 | description: 158 | - The maximum number of bytes after decompression. 159 | required: false 160 | default: 8388608 161 | type: int 162 | enable_cors: 163 | description: 164 | - Input sends CORS headers to satisfy browser security policies (HTTP Only) 165 | required: false 166 | default: true 167 | type: bool 168 | idle_writer_timeout: 169 | description: 170 | - The server closes the connection after the given time in seconds after the last client write request. (use 0 to disable) (HTTP Only) 171 | required: false 172 | default: 60 173 | type: int 174 | max_chunk_size: 175 | description: 176 | - The maximum HTTP chunk size in bytes (e. g. length of HTTP request body) (HTTP Only) 177 | required: false 178 | default: 65536 179 | type: int 180 | max_message_size: 181 | description: 182 | - The maximum length of a message. (TCP Only) 183 | required: false 184 | default: 2097152 185 | type: int 186 | ''' 187 | 188 | EXAMPLES = ''' 189 | - name: Create GELF HTTP input 190 | graylog_input_gelf: 191 | endpoint: "{{ graylog_endpoint }}" 192 | graylog_user: "{{ graylog_user }}" 193 | graylog_password: "{{ graylog_password }}" 194 | allow_http: "true" 195 | validate_certs: "false" 196 | action: "create" 197 | input_type: "HTTP" 198 | title: "Test input GELF HTTP" 199 | global_input: "true" 200 | bind_address: "0.0.0.0" 201 | ''' 202 | 203 | # import module snippets 204 | import json 205 | import base64 206 | from ansible.module_utils.basic import AnsibleModule 207 | from ansible.module_utils.urls import fetch_url, to_text 208 | import re 209 | 210 | def search_by_name(module, base_url, headers, title): 211 | 212 | url = base_url 213 | inputExist = False 214 | 215 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 216 | 217 | if info['status'] != 200: 218 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 219 | 220 | try: 221 | content = to_text(response.read(), errors='surrogate_or_strict') 222 | data = json.loads(content) 223 | except AttributeError: 224 | content = info.pop('body', '') 225 | 226 | regex = r"^" + re.escape(title) + r"$" 227 | 228 | for graylogInputs in data['inputs']: 229 | if re.match(regex, graylogInputs['title']) is not None: 230 | inputExist = True 231 | 232 | return inputExist 233 | 234 | 235 | def action(module, base_url, headers): 236 | 237 | url = base_url 238 | 239 | if module.params['action'] == "create": 240 | inputExist = search_by_name(module, base_url, headers, module.params['title']) 241 | if inputExist == True: 242 | module.exit_json(changed=False) 243 | httpMethod = "POST" 244 | else: 245 | httpMethod = "PUT" 246 | url = base_url + "/" + module.params['input_id'] 247 | 248 | configuration = {} 249 | for key in [ 'bind_address', 'port', 'number_worker_threads', 'override_source', 'recv_buffer_size', \ 250 | 'tcp_keepalive', 'tls_enable', 'tls_cert_file', 'tls_key_file', 'tls_key_password', \ 251 | 'tls_client_auth', 'tls_client_auth_cert_file', 'use_null_delimiter', 'decompress_size_limit', \ 252 | 'enable_cors', 'idle_writer_timeout', 'max_chunk_size', 'max_message_size' ]: 253 | if module.params[key] is not None: 254 | configuration[key] = module.params[key] 255 | 256 | payload = {} 257 | 258 | payload['type'] = module.params['input_type'] 259 | payload['title'] = module.params['title'] 260 | payload['global'] = module.params['global_input'] 261 | payload['node'] = module.params['node'] 262 | payload['configuration'] = configuration 263 | 264 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method=httpMethod, data=module.jsonify(payload)) 265 | 266 | if info['status'] != 201: 267 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 268 | 269 | try: 270 | content = to_text(response.read(), errors='surrogate_or_strict') 271 | except AttributeError: 272 | content = info.pop('body', '') 273 | 274 | return info['status'], info['msg'], content, base_url 275 | 276 | def get_token(module, endpoint, username, password, allow_http): 277 | 278 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 279 | 280 | url = endpoint + "/api/system/sessions" 281 | 282 | payload = {} 283 | payload['username'] = username 284 | payload['password'] = password 285 | payload['host'] = endpoint 286 | 287 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 288 | 289 | if info['status'] != 200: 290 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 291 | 292 | try: 293 | content = to_text(response.read(), errors='surrogate_or_strict') 294 | session = json.loads(content) 295 | except AttributeError: 296 | content = info.pop('body', '') 297 | 298 | session_string = session['session_id'] + ":session" 299 | session_bytes = session_string.encode('utf-8') 300 | session_token = base64.b64encode(session_bytes) 301 | 302 | return session_token 303 | 304 | def main(): 305 | module = AnsibleModule( 306 | argument_spec=dict( 307 | endpoint=dict(type='str'), 308 | graylog_user=dict(type='str'), 309 | graylog_password=dict(type='str', no_log=True), 310 | validate_certs=dict(type='bool', required=False, default=True), 311 | allow_http=dict(type='bool', required=False, default=False), 312 | action=dict(type='str', required=False, default='create', 313 | choices=[ 'create', 'update' ]), 314 | input_type=dict(type='str', required=False, default='UDP', 315 | choices=[ 'UDP', 'TCP', 'HTTP' ]), 316 | title=dict(type='str', required=True ), 317 | global_input=dict(type='bool', required=False, default=True), 318 | node=dict(type='str', required=False), 319 | bind_address=dict(type='str', required=False, default='0.0.0.0'), 320 | port=dict(type='int', required=False, default=12201), 321 | number_worker_threads=dict(type='int', required=False, default=2), 322 | override_source=dict(type='str', required=False), 323 | recv_buffer_size=dict(type='int', required=False, default=1048576), 324 | tcp_keepalive=dict(type='bool', required=False, default=False), 325 | tls_enable=dict(type='bool', required=False, default=False), 326 | tls_cert_file=dict(type='str', required=False), 327 | tls_key_file=dict(type='str', required=False), 328 | tls_key_password=dict(type='str', required=False, no_log=True), 329 | tls_client_auth=dict(type='str', required=False, default='disabled', 330 | choices=[ 'disabled', 'optional', 'required' ]), 331 | tls_client_auth_cert_file=dict(type='str', required=False), 332 | use_null_delimiter=dict(type='bool', required=False, default=False), 333 | decompress_size_limit=dict(type='int', required=False, default=8388608), 334 | enable_cors=dict(type='bool', required=False, default=True), 335 | idle_writer_timeout=dict(type='int', required=False, default=60), 336 | max_chunk_size=dict(type='int', required=False, default=65536), 337 | max_message_size=dict(type='int', required=False, default=2097152) 338 | ) 339 | ) 340 | 341 | endpoint = module.params['endpoint'] 342 | graylog_user = module.params['graylog_user'] 343 | graylog_password = module.params['graylog_password'] 344 | allow_http = module.params['allow_http'] 345 | 346 | if allow_http == True: 347 | endpoint = "http://" + endpoint 348 | else: 349 | endpoint = "https://" + endpoint 350 | 351 | # Build full name of input type 352 | if module.params['input_type'] == "TCP": 353 | module.params['input_type'] = "org.graylog2.inputs.gelf.tcp.GELFTCPInput" 354 | elif module.params['input_type'] == "UDP": 355 | module.params['input_type'] = "org.graylog2.inputs.gelf.udp.GELFUDPInput" 356 | else: 357 | module.params['input_type'] = "org.graylog2.inputs.gelf.http.GELFHttpInput" 358 | 359 | base_url = endpoint + "/api/system/inputs" 360 | 361 | api_token = get_token(module, endpoint, graylog_user, graylog_password, allow_http) 362 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 363 | "Authorization": "Basic ' + api_token.decode() + '" }' 364 | 365 | status, message, content, url = action(module, base_url, headers) 366 | 367 | uresp = {} 368 | content = to_text(content, encoding='UTF-8') 369 | 370 | try: 371 | js = json.loads(content) 372 | except ValueError: 373 | js = "" 374 | 375 | uresp['json'] = js 376 | uresp['status'] = status 377 | uresp['msg'] = message 378 | uresp['url'] = url 379 | 380 | module.exit_json(**uresp) 381 | 382 | 383 | if __name__ == '__main__': 384 | main() 385 | -------------------------------------------------------------------------------- /library/graylog_input_rsyslog.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # Copyright: (c) 2019, Matthieu SIMON 4 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 5 | 6 | ANSIBLE_METADATA = {'metadata_version': '1.1', 7 | 'status': ['preview'], 8 | 'supported_by': 'community'} 9 | 10 | DOCUMENTATION = ''' 11 | module: graylog_input_rsyslog 12 | short_description: Manage Graylog input type Syslog 13 | description: 14 | - The Graylog inputs module allows configuration of input type Syslog on nodes. 15 | version_added: "2.9" 16 | author: "Matthieu SIMON" 17 | options: 18 | endpoint: 19 | description: 20 | - Graylog endoint. (i.e. graylog.mydomain.com). 21 | required: false 22 | type: str 23 | graylog_user: 24 | description: 25 | - Graylog privileged user username. 26 | required: false 27 | type: str 28 | graylog_password: 29 | description: 30 | - Graylog privileged user password. 31 | required: false 32 | type: str 33 | allow_http: 34 | description: 35 | - Allow non HTTPS connexion 36 | required: false 37 | default: false 38 | type: bool 39 | validate_certs: 40 | description: 41 | - Allow untrusted certificate 42 | required: false 43 | default: false 44 | type: bool 45 | action: 46 | description: 47 | - Action to take against system/input API. 48 | - Warning : when update, all settings with default value set in this Ansible module (like bind_address, port ...) will replace existing values 49 | You must explicitly set these values if they differ from those by default 50 | required: true 51 | default: create 52 | choices: [ create, update ] 53 | type: str 54 | force: 55 | description: 56 | - Create input if RSyslog input with the same name exist 57 | required: false 58 | default: False 59 | type: bool 60 | input_type: 61 | description: 62 | - Input type 63 | required: false 64 | default: UDP 65 | choices: [ 'UDP', 'TCP' ] 66 | type: str 67 | title: 68 | description: 69 | - Entitled of the input 70 | required: true 71 | type: str 72 | input_id: 73 | description: 74 | - ID of input to update 75 | required: false 76 | type: str 77 | global_input: 78 | description: 79 | - Should this input start on all nodes 80 | required: false 81 | default: true 82 | type: bool 83 | node: 84 | description: 85 | - Node name if input is not global 86 | required: false 87 | type: str 88 | bind_address: 89 | description: 90 | - Address to listen on 91 | required: false 92 | default: "0.0.0.0" 93 | type: str 94 | port: 95 | description: 96 | - Port to listen on 97 | required: true 98 | default: 514 99 | type: int 100 | allow_override_date: 101 | description: 102 | - Allow to override with current date if date could not be parsed 103 | required: false 104 | default: false 105 | type: bool 106 | expand_structured_data: 107 | description: 108 | - Expand structured data elements by prefixing attributes with their SD-ID 109 | required: false 110 | default: false 111 | type: bool 112 | force_rdns: 113 | description: 114 | - Force rDNS resolution of hostname. Use if hostname cannot be parsed. (Be careful if you are sending DNS logs into this input because it can cause a feedback loop.) 115 | required: false 116 | default: false 117 | type: bool 118 | number_worker_threads: 119 | description: 120 | - Number of worker threads processing network connections for this input. 121 | required: false 122 | default: 2 123 | type: int 124 | override_source: 125 | description: 126 | - The source is a hostname derived from the received packet by default. Set this if you want to override it with a custom string. 127 | required: false 128 | type: str 129 | recv_buffer_size: 130 | description: 131 | - The size in bytes of the recvBufferSize for network connections to this input. 132 | required: false 133 | default: 1048576 134 | type: int 135 | store_full_message: 136 | description: 137 | - Store the full original syslog message as full_message 138 | required: false 139 | default: false 140 | type: bool 141 | tcp_keepalive: 142 | description: 143 | - Enable TCP keepalive packets (TCP only) 144 | required: false 145 | default: false 146 | type: bool 147 | tls_enable: 148 | description: 149 | - Accept TLS connections (TCP only) 150 | required: false 151 | default: false 152 | type: bool 153 | tls_cert_file: 154 | description: 155 | - Path to the TLS certificate file (TCP only) 156 | required: false 157 | type: str 158 | tls_key_file: 159 | description: 160 | - Path to the TLS private key file (TCP only) 161 | required: false 162 | type: str 163 | tls_key_password: 164 | description: 165 | - The password for the encrypted key file. (TCP only) 166 | required: false 167 | type: str 168 | tls_client_auth: 169 | description: 170 | - Whether clients need to authenticate themselves in a TLS connection (TCP only) 171 | required: false 172 | default: disabled 173 | choices: [ 'disabled', 'optional', 'required' ] 174 | tls_client_auth_cert_file: 175 | description: 176 | - TLS Client Auth Trusted Certs (File or Directory) (TCP only) 177 | required: false 178 | type: str 179 | use_null_delimiter: 180 | description: 181 | - Use null byte as frame delimiter ? Otherwise newline delimiter is used. (TCP only) 182 | required: false 183 | default: false 184 | type: bool 185 | ''' 186 | 187 | EXAMPLES = ''' 188 | - name: Create Rsyslog TCP input 189 | graylog_input_rsyslog: 190 | endpoint: "{{ graylog_endpoint }}" 191 | graylog_user: "{{ graylog_user }}" 192 | graylog_password: "{{ graylog_password }}" 193 | allow_http: "true" 194 | validate_certs: "false" 195 | action: "create" 196 | input_type: "TCP" 197 | title: "Rsyslog TCP" 198 | global_input: "true" 199 | allow_override_date: "true" 200 | bind_address: "0.0.0.0" 201 | expand_structured_data: "false" 202 | force_rdns: "false" 203 | number_worker_threads: "2" 204 | port: "514" 205 | recv_buffer_size: "1048576" 206 | store_full_message: "true" 207 | 208 | 209 | - name: Update existing input 210 | graylog_input_rsyslog: 211 | endpoint: "{{ graylog_endpoint }}" 212 | graylog_user: "{{ graylog_user }}" 213 | graylog_password: "{{ graylog_password }}" 214 | allow_http: "true" 215 | validate_certs: "false" 216 | action: "update" 217 | input_type: "TCP" 218 | title: "Rsyslog TCP" 219 | global_input: "true" 220 | allow_override_date: "true" 221 | expand_structured_data: "false" 222 | force_rdns: "true" 223 | port: "1514" 224 | store_full_message: "true" 225 | input_id: "1df0f1234abcd0000d0adf20" 226 | ''' 227 | 228 | # import module snippets 229 | import json 230 | import base64 231 | from ansible.module_utils.basic import AnsibleModule 232 | from ansible.module_utils.urls import fetch_url, to_text 233 | import re 234 | 235 | def search_by_name(module, base_url, headers, title): 236 | 237 | url = base_url 238 | inputExist = False 239 | 240 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 241 | 242 | if info['status'] != 200: 243 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 244 | 245 | 246 | try: 247 | content = to_text(response.read(), errors='surrogate_or_strict') 248 | data = json.loads(content) 249 | except AttributeError: 250 | content = info.pop('body', '') 251 | 252 | 253 | regex = r"^" + re.escape(title) + r"$" 254 | 255 | for graylogInputs in data['inputs']: 256 | if re.match(regex, graylogInputs['title']) is not None: 257 | inputExist = True 258 | 259 | return inputExist 260 | 261 | def action(module, base_url, headers): 262 | 263 | url = base_url 264 | 265 | if module.params['action'] == "create": 266 | if module.params['force'] == False: 267 | inputExist = search_by_name(module, base_url, headers, module.params['title']) 268 | if inputExist == True: 269 | module.exit_json(changed=False) 270 | httpMethod = "POST" 271 | else: 272 | httpMethod = "PUT" 273 | url = base_url + "/" + module.params['input_id'] 274 | 275 | configuration = {} 276 | for key in [ 'bind_address', 'port', 'allow_override_date', 'expand_structured_data', 'force_rdns', \ 277 | 'number_worker_threads', 'override_source', 'recv_buffer_size', 'store_full_message', \ 278 | 'tcp_keepalive', 'tls_enable', 'tls_cert_file', 'tls_key_file', 'tls_key_password', \ 279 | 'tls_client_auth', 'tls_client_auth_cert_file', 'use_null_delimiter' ]: 280 | if module.params[key] is not None: 281 | configuration[key] = module.params[key] 282 | 283 | payload = {} 284 | 285 | payload['type'] = module.params['input_type'] 286 | payload['title'] = module.params['title'] 287 | payload['global'] = module.params['global_input'] 288 | payload['node'] = module.params['node'] 289 | payload['configuration'] = configuration 290 | 291 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method=httpMethod, data=module.jsonify(payload)) 292 | 293 | if info['status'] != 201: 294 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 295 | 296 | try: 297 | content = to_text(response.read(), errors='surrogate_or_strict') 298 | except AttributeError: 299 | content = info.pop('body', '') 300 | 301 | return info['status'], info['msg'], content, url 302 | 303 | 304 | def get_token(module, endpoint, username, password, allow_http): 305 | 306 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 307 | 308 | url = endpoint + "/api/system/sessions" 309 | 310 | payload = {} 311 | payload['username'] = username 312 | payload['password'] = password 313 | payload['host'] = endpoint 314 | 315 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 316 | 317 | if info['status'] != 200: 318 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 319 | 320 | try: 321 | content = to_text(response.read(), errors='surrogate_or_strict') 322 | session = json.loads(content) 323 | except AttributeError: 324 | content = info.pop('body', '') 325 | 326 | session_string = session['session_id'] + ":session" 327 | session_bytes = session_string.encode('utf-8') 328 | session_token = base64.b64encode(session_bytes) 329 | 330 | return session_token 331 | 332 | def main(): 333 | module = AnsibleModule( 334 | argument_spec=dict( 335 | endpoint=dict(type='str'), 336 | graylog_user=dict(type='str'), 337 | graylog_password=dict(type='str', no_log=True), 338 | validate_certs=dict(type='bool', required=False, default=True), 339 | allow_http=dict(type='bool', required=False, default=False), 340 | action=dict(type='str', required=False, default='create', 341 | choices=[ 'create', 'update' ]), 342 | force=dict(type='bool', required=False, default=False), 343 | input_type=dict(type='str', required=False, default='UDP', 344 | choices=[ 'UDP', 'TCP' ]), 345 | title=dict(type='str', required=True), 346 | input_id=dict(type='str', required=False), 347 | global_input=dict(type='bool', required=False, default=True), 348 | node=dict(type='str', required=False), 349 | bind_address=dict(type='str', required=False, default='0.0.0.0'), 350 | port=dict(type='int', required=False, default=514), 351 | allow_override_date=dict(type='bool', required=False, default=False), 352 | expand_structured_data=dict(type='bool', required=False, default=False), 353 | force_rdns=dict(type='bool', required=False, default=False), 354 | number_worker_threads=dict(type='int', required=False, default=2), 355 | override_source=dict(type='str', required=False), 356 | recv_buffer_size=dict(type='int', required=False, default=1048576), 357 | store_full_message=dict(type='bool', required=False, default=False), 358 | tcp_keepalive=dict(type='bool', required=False, default=False), 359 | tls_enable=dict(type='bool', required=False, default=False), 360 | tls_cert_file=dict(type='str', required=False), 361 | tls_key_file=dict(type='str', required=False), 362 | tls_key_password=dict(type='str', required=False, no_log=True), 363 | tls_client_auth=dict(type='str', required=False, default='disabled', 364 | choices=[ 'disabled', 'optional', 'required' ]), 365 | tls_client_auth_cert_file=dict(type='str', required=False), 366 | use_null_delimiter=dict(type='bool', required=False, default=False) 367 | ) 368 | ) 369 | 370 | endpoint = module.params['endpoint'] 371 | graylog_user = module.params['graylog_user'] 372 | graylog_password = module.params['graylog_password'] 373 | allow_http = module.params['allow_http'] 374 | 375 | 376 | if allow_http == True: 377 | endpoint = "http://" + endpoint 378 | else: 379 | endpoint = "https://" + endpoint 380 | 381 | # Build full name of input type 382 | if module.params['input_type'] == "UDP": 383 | module.params['input_type'] = "org.graylog2.inputs.syslog.udp.SyslogUDPInput" 384 | else: 385 | module.params['input_type'] = "org.graylog2.inputs.syslog.tcp.SyslogTCPInput" 386 | 387 | base_url = endpoint + "/api/system/inputs" 388 | 389 | api_token = get_token(module, endpoint, graylog_user, graylog_password, allow_http) 390 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 391 | "Authorization": "Basic ' + api_token.decode() + '" }' 392 | 393 | status, message, content, url = action(module, base_url, headers) 394 | 395 | uresp = {} 396 | content = to_text(content, encoding='UTF-8') 397 | 398 | try: 399 | js = json.loads(content) 400 | except ValueError: 401 | js = "" 402 | 403 | uresp['json'] = js 404 | uresp['status'] = status 405 | uresp['msg'] = message 406 | uresp['url'] = url 407 | 408 | module.exit_json(**uresp) 409 | 410 | 411 | if __name__ == '__main__': 412 | main() 413 | -------------------------------------------------------------------------------- /library/graylog_ldap.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # Copyright: (c) 2019, Matthieu SIMON 4 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 5 | 6 | ANSIBLE_METADATA = {'metadata_version': '1.1', 7 | 'status': ['preview'], 8 | 'supported_by': 'community'} 9 | 10 | DOCUMENTATION = ''' 11 | module: graylog_ldap 12 | short_description: Communicate with the Graylog API to configure LDAP authentication 13 | description: 14 | - The Graylog ldap module allows configuration LDAP authentication parameters. 15 | version_added: "2.9" 16 | author: "Matthieu SIMON" 17 | options: 18 | endpoint: 19 | description: 20 | - Graylog endpoint. (i.e. graylog.mydomain.com:9000). 21 | required: false 22 | type: str 23 | graylog_user: 24 | description: 25 | - Graylog privileged user username, used to auth with Graylog API. 26 | required: false 27 | type: str 28 | graylog_password: 29 | description: 30 | - Graylog privileged user password, used to auth with Graylog API. 31 | required: false 32 | type: str 33 | allow_http: 34 | description: 35 | - Allow non HTTPS connexion 36 | required: false 37 | default: false 38 | type: bool 39 | validate_certs: 40 | description: 41 | - Allow untrusted certificate 42 | required: false 43 | default: false 44 | type: bool 45 | action: 46 | description: 47 | - Action to take against LDAP API. 48 | required: true 49 | default: get 50 | choices: [ get, update, delete, test ] 51 | type: str 52 | enabled: 53 | description: 54 | - Enable / disable LDAP authentication 55 | required: false 56 | type: bool 57 | active_directory: 58 | description: 59 | - Define LDAP flavour as Active Directory 60 | required: false 61 | default: false 62 | type: bool 63 | ldap_uri: 64 | desctiption: 65 | - LDAP URI (ex ldap://myldapserver.mydomain.com:389) 66 | required: false 67 | type: str 68 | use_start_tls 69 | description: 70 | - Enable start TLS 71 | required: false 72 | default: false 73 | type: bool 74 | trust_all_certificates: 75 | description: 76 | - Allow LDAP self-signed certificates 77 | required: false 78 | default: false 79 | type: bool 80 | system_password_set: 81 | description: 82 | - Enable binding authentication 83 | required: false 84 | default: false 85 | type: str 86 | system_username: 87 | description: 88 | - The username for LDAP binding, e.g. ldapbind@some.domain. 89 | required: false 90 | type: str 91 | system_password: 92 | description: 93 | - Bind username password 94 | required: false 95 | type: str 96 | search_base: 97 | description: 98 | - The base tree to limit the user search query to, e.g. cn=users,dc=example,dc=com 99 | required: false 100 | type: str 101 | search_pattern: 102 | description: 103 | - User search filter. For example (&(objectClass=user)(sAMAccountName={0})) 104 | required: false 105 | type: str 106 | display_name_attribute 107 | description: 108 | - Attribute to use for the full name of the user in Graylog 109 | required: false 110 | type: str 111 | group_search_base: 112 | description: 113 | - The base tree to limit the LDAP group search query to 114 | required: false 115 | type: str 116 | group_search_pattern: 117 | description: 118 | - The search pattern used to find groups in LDAP for mapping to Graylog roles 119 | required: false 120 | type: str 121 | group_id_attribute: 122 | description: 123 | - Attribute to use for the full name of the group, usually cn 124 | required: false 125 | type: str 126 | default_group: 127 | description: 128 | - Default role assigned to LDAP group 129 | required: false 130 | default: Reader 131 | type: str 132 | group_mapping: 133 | description: 134 | - Additional roles assigned to LDAP group 135 | required: false 136 | type: list 137 | ''' 138 | 139 | EXAMPLES = ''' 140 | # Setup Active Directory authentication without SSL and set "Reader" as default role 141 | - graylog_ldap: 142 | endpoint: "graylog.mydomain.com" 143 | graylog_user: "username" 144 | graylog_password: "password" 145 | enabled: "true" 146 | action: "update" 147 | active_directory: "true" 148 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 149 | system_password_set: "true" 150 | system_username: "ldapbind@mydomain.com" 151 | system_password: "bindPassw0rd" 152 | search_base: "cn=users,dc=mydomain,dc=com" 153 | search_pattern: "(&(objectClass=user)(sAMAccountName={0}))" 154 | display_name_attribute: "displayName" 155 | group_search_base: "cn=groups,dc=mydomain,dc=com" 156 | group_search_pattern: "(&(objectClass=group)(cn=graylog*))" 157 | group_id_attribute: "cn" 158 | 159 | # Remove current LDAP authentication configuration 160 | - graylog_ldap: 161 | endpoint: "graylog.mydomain.com" 162 | graylog_user: "username" 163 | graylog_password: "password" 164 | action: "delete" 165 | 166 | # Get current LDAP authentication configuration 167 | - graylog_ldap: 168 | endpoint: "graylog.mydomain.com" 169 | graylog_user: "username" 170 | graylog_password: "password" 171 | action: "get" 172 | register: currentConfiguration 173 | 174 | - name: Print 175 | debug: 176 | msg: "{{ currentConfiguration }}" 177 | 178 | # Test LDAP bind 179 | - graylog_ldap: 180 | endpoint: "graylog.mydomain.com" 181 | graylog_user: "username" 182 | graylog_password: "password" 183 | action: "test" 184 | active_directory: "true" 185 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 186 | system_password_set: "true" 187 | system_username: "ldapbind@mydomain.com" 188 | system_password: "bindPassw0rd" 189 | ''' 190 | 191 | # import module snippets 192 | import json 193 | import base64 194 | from ansible.module_utils.basic import AnsibleModule 195 | from ansible.module_utils.urls import fetch_url, to_text 196 | 197 | 198 | def get(module, base_url, headers): 199 | 200 | url = base_url + "/settings" 201 | 202 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 203 | 204 | if info['status'] != 200: 205 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 206 | 207 | try: 208 | content = to_text(response.read(), errors='surrogate_or_strict') 209 | except AttributeError: 210 | content = info.pop('body', '') 211 | 212 | return info['status'], info['msg'], content, url 213 | 214 | 215 | def delete(module, base_url, headers): 216 | 217 | url = base_url + "/settings" 218 | 219 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 220 | 221 | if info['status'] != 204: 222 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 223 | 224 | try: 225 | content = to_text(response.read(), errors='surrogate_or_strict') 226 | except AttributeError: 227 | content = info.pop('body', '') 228 | 229 | return info['status'], info['msg'], content, url 230 | 231 | 232 | def test(module, base_url, headers): 233 | 234 | url = base_url + "/test" 235 | 236 | payload = {} 237 | 238 | for key in [ 'system_username', 'system_password', 'ldap_uri', 'use_start_tls', 'trust_all_certificates', \ 239 | 'active_directory', 'search_base', 'search_pattern', 'group_search_base', 'group_id_attribute', \ 240 | 'group_search_pattern' ]: 241 | if module.params[key] is not None: 242 | payload[key] = module.params[key] 243 | 244 | payload['test_connect_only'] = "true" 245 | 246 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 247 | 248 | if info['status'] != 200: 249 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 250 | 251 | try: 252 | content = to_text(response.read(), errors='surrogate_or_strict') 253 | except AttributeError: 254 | content = info.pop('body', '') 255 | 256 | jsonContent = json.loads(content) 257 | 258 | if jsonContent['connected'] == False: 259 | module.fail_json(msg="Fail: LDAP bind fail !") 260 | 261 | return info['status'], info['msg'], content, url 262 | 263 | 264 | def update(module, base_url, headers): 265 | 266 | url = base_url + "/settings" 267 | 268 | payload = {} 269 | 270 | for key in ['enabled', 'active_directory', 'ldap_uri', 'use_start_tls', 'trust_all_certificates', \ 271 | 'system_password_set', 'system_username', 'system_password', 'search_base', 'search_pattern', \ 272 | 'display_name_attribute', 'group_search_base', 'group_search_pattern', 'group_id_attribute', \ 273 | 'default_group', 'group_mapping', 'enabled', 'active_directory' ]: 274 | if module.params[key] is not None: 275 | payload[key] = module.params[key] 276 | 277 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 278 | 279 | if info['status'] != 204: 280 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 281 | 282 | try: 283 | content = to_text(response.read(), errors='surrogate_or_strict') 284 | except AttributeError: 285 | content = info.pop('body', '') 286 | 287 | return info['status'], info['msg'], content, url 288 | 289 | def get_token(module, endpoint, username, password, allow_http): 290 | 291 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 292 | 293 | url = endpoint + "/api/system/sessions" 294 | 295 | payload = {} 296 | payload['username'] = username 297 | payload['password'] = password 298 | payload['host'] = endpoint 299 | 300 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 301 | 302 | if info['status'] != 200: 303 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 304 | 305 | try: 306 | content = to_text(response.read(), errors='surrogate_or_strict') 307 | session = json.loads(content) 308 | except AttributeError: 309 | content = info.pop('body', '') 310 | 311 | session_string = session['session_id'] + ":session" 312 | session_bytes = session_string.encode('utf-8') 313 | session_token = base64.b64encode(session_bytes) 314 | 315 | return session_token 316 | 317 | 318 | def main(): 319 | module = AnsibleModule( 320 | argument_spec=dict( 321 | endpoint=dict(type='str'), 322 | graylog_user=dict(type='str'), 323 | graylog_password=dict(type='str', no_log=True), 324 | action=dict(type='str', required=False, default='get', 325 | choices=['get', 'update', 'delete', 'test']), 326 | allow_http=dict(type='bool', required=False, default=False), 327 | validate_certs=dict(type='bool', required=False, default=True), 328 | enabled=dict(type='bool', required=False, default=False), 329 | active_directory=dict(type='bool', required=False, default=False), 330 | ldap_uri=dict(type='str', required=False), 331 | use_start_tls=dict(type='bool', required=False, default=False), 332 | trust_all_certificates=dict(type='bool', required=False, default=False), 333 | system_password_set=dict(type='bool', required=False, default=False), 334 | system_username=dict(type='str', required=False), 335 | system_password=dict(type='str', required=False, no_log=True), 336 | search_base=dict(type='str', required=False), 337 | search_pattern=dict(type='str', required=False), 338 | display_name_attribute=dict(type='str', required=False), 339 | group_search_base=dict(type='str', required=False), 340 | group_search_pattern=dict(type='str', required=False), 341 | group_id_attribute=dict(type='str', required=False), 342 | default_group=dict(type='str', required=False, default='Reader'), 343 | group_mapping=dict(type='list', required=False) 344 | ) 345 | ) 346 | 347 | endpoint = module.params['endpoint'] 348 | graylog_user = module.params['graylog_user'] 349 | graylog_password = module.params['graylog_password'] 350 | action = module.params['action'] 351 | allow_http = module.params['allow_http'] 352 | 353 | if allow_http == True: 354 | endpoint = "http://" + endpoint 355 | else: 356 | endpoint = "https://" + endpoint 357 | 358 | base_url = endpoint + "/api/system/ldap" 359 | 360 | api_token = get_token(module, endpoint, graylog_user, graylog_password, allow_http) 361 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 362 | "Authorization": "Basic ' + api_token.decode() + '" }' 363 | 364 | if action == "get": 365 | status, message, content, url = get(module, base_url, headers) 366 | elif action == "update": 367 | status, message, content, url = update(module, base_url, headers) 368 | elif action == "delete": 369 | status, message, content, url = delete(module, base_url, headers) 370 | elif action == "test": 371 | status, message, content, url = test(module, base_url, headers) 372 | 373 | uresp = {} 374 | content = to_text(content, encoding='UTF-8') 375 | 376 | try: 377 | js = json.loads(content) 378 | except ValueError: 379 | js = "" 380 | 381 | uresp['json'] = js 382 | uresp['status'] = status 383 | uresp['msg'] = message 384 | uresp['url'] = url 385 | 386 | module.exit_json(**uresp) 387 | 388 | 389 | if __name__ == '__main__': 390 | main() 391 | -------------------------------------------------------------------------------- /library/graylog_ldap_groups.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # Copyright: (c) 2019, Matthieu SIMON 4 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 5 | 6 | ANSIBLE_METADATA = {'metadata_version': '1.1', 7 | 'status': ['preview'], 8 | 'supported_by': 'community'} 9 | 10 | DOCUMENTATION = ''' 11 | module: graylog_ldap 12 | short_description: Communicate with the Graylog API to configure LDAP authentication 13 | description: 14 | - The Graylog ldap module allows configuration LDAP authentication parameters. 15 | version_added: "2.9" 16 | author: "Matthieu SIMON" 17 | options: 18 | endpoint: 19 | description: 20 | - Graylog endpoint. (i.e. graylog.mydomain.com:9000). 21 | required: false 22 | type: str 23 | graylog_user: 24 | description: 25 | - Graylog privileged user username, used to auth with Graylog API. 26 | required: false 27 | type: str 28 | graylog_password: 29 | description: 30 | - Graylog privileged user password, used to auth with Graylog API. 31 | required: false 32 | type: str 33 | allow_http: 34 | description: 35 | - Allow non HTTPS connexion 36 | required: false 37 | default: false 38 | type: bool 39 | validate_certs: 40 | description: 41 | - Allow untrusted certificate 42 | required: false 43 | default: false 44 | type: bool 45 | action: 46 | description: 47 | - Action to take against LDAP API. 48 | required: true 49 | default: list 50 | choices: [ list, list_mapping, update ] 51 | type: str 52 | group: 53 | description: 54 | - LDAP group whose role is to update 55 | required: false 56 | type: str 57 | role: 58 | description: 59 | - Graylog role to assign to the LDAP group 60 | require: false 61 | type: str 62 | ''' 63 | 64 | EXAMPLES = ''' 65 | # Get all LDAP groups 66 | - graylog_ldap_groups: 67 | endpoint: "graylog.mydomain.com" 68 | graylog_user: "username" 69 | graylog_password: "password" 70 | action: "list" 71 | 72 | # Get the LDAP group to Graylog role mapping 73 | - graylog_ldap_groups: 74 | endpoint: "graylog.mydomain.com" 75 | graylog_user: "username" 76 | graylog_password: "password" 77 | action: "list_mapping" 78 | 79 | # Update the LDAP group to Graylog role mapping 80 | - graylog_ldap_groups: 81 | endpoint: "graylog.mydomain.com" 82 | graylog_user: "username" 83 | graylog_password: "password" 84 | action: "update" 85 | group: "{{ item.group }}" 86 | role: "{{ item.role }}" 87 | with_items: 88 | - { group : "ldap-group-admins", role : "Admin" } 89 | - { group : "ldap-group-read", role : "Reader" } 90 | 91 | # Remove Graylog role mapping 92 | - graylog_ldap_groups: 93 | endpoint: "graylog.mydomain.com" 94 | graylog_user: "username" 95 | graylog_password: "password" 96 | action: "update" 97 | group: "ldap-group-foobar" 98 | role: "None" 99 | ''' 100 | 101 | # import module snippets 102 | import json 103 | import base64 104 | from ansible.module_utils.basic import AnsibleModule 105 | from ansible.module_utils.urls import fetch_url, to_text 106 | 107 | def list(module, base_url, headers): 108 | 109 | url = base_url + "/groups" 110 | 111 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 112 | 113 | if info['status'] != 200: 114 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 115 | 116 | try: 117 | content = to_text(response.read(), errors='surrogate_or_strict') 118 | except AttributeError: 119 | content = info.pop('body', '') 120 | 121 | return info['status'], info['msg'], content, url 122 | 123 | def list_mapping(module, base_url, headers): 124 | 125 | url = base_url + "/settings/groups" 126 | 127 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 128 | 129 | if info['status'] != 200: 130 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 131 | 132 | try: 133 | content = to_text(response.read(), errors='surrogate_or_strict') 134 | except AttributeError: 135 | content = info.pop('body', '') 136 | 137 | return info['status'], info['msg'], content, url 138 | 139 | 140 | def update(module, base_url, headers): 141 | 142 | url = base_url + "/settings/groups" 143 | 144 | # Get current mapping 145 | (currentMapping) = list_mapping(module, base_url, headers) 146 | payload = json.loads(currentMapping[2]) 147 | 148 | # Update value 149 | group = module.params['group'] 150 | payload[group] = module.params['role'] 151 | 152 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 153 | 154 | if info['status'] != 204: 155 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 156 | 157 | try: 158 | content = to_text(response.read(), errors='surrogate_or_strict') 159 | except AttributeError: 160 | content = info.pop('body', '') 161 | 162 | return info['status'], info['msg'], content, url 163 | 164 | def get_token(module, endpoint, username, password, allow_http): 165 | 166 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 167 | 168 | url = endpoint + "/api/system/sessions" 169 | 170 | payload = {} 171 | payload['username'] = username 172 | payload['password'] = password 173 | payload['host'] = endpoint 174 | 175 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 176 | 177 | if info['status'] != 200: 178 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 179 | 180 | try: 181 | content = to_text(response.read(), errors='surrogate_or_strict') 182 | session = json.loads(content) 183 | except AttributeError: 184 | content = info.pop('body', '') 185 | 186 | session_string = session['session_id'] + ":session" 187 | session_bytes = session_string.encode('utf-8') 188 | session_token = base64.b64encode(session_bytes) 189 | 190 | return session_token 191 | 192 | def main(): 193 | module = AnsibleModule( 194 | argument_spec=dict( 195 | endpoint=dict(type='str'), 196 | graylog_user=dict(type='str'), 197 | graylog_password=dict(type='str', no_log=True), 198 | action=dict(type='str', required=False, default='list', 199 | choices=[ 'list', 'list_mapping', 'update' ]), 200 | allow_http=dict(type='bool', required=False, default=False), 201 | validate_certs=dict(type='bool', required=False, default=True), 202 | group=dict(type='str'), 203 | role=dict(type='str') 204 | ) 205 | ) 206 | 207 | endpoint = module.params['endpoint'] 208 | graylog_user = module.params['graylog_user'] 209 | graylog_password = module.params['graylog_password'] 210 | action = module.params['action'] 211 | allow_http = module.params['allow_http'] 212 | 213 | if allow_http == True: 214 | endpoint = "http://" + endpoint 215 | else: 216 | endpoint = "https://" + endpoint 217 | 218 | base_url = endpoint + "/api/system/ldap" 219 | 220 | api_token = get_token(module, endpoint, graylog_user, graylog_password, allow_http) 221 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 222 | "Authorization": "Basic ' + api_token.decode() + '" }' 223 | 224 | if action == "list": 225 | status, message, content, url = list(module, base_url, headers) 226 | elif action == "list_mapping": 227 | status, message, content, url = list_mapping(module, base_url, headers) 228 | elif action == "update": 229 | status, message, content, url = update(module, base_url, headers) 230 | 231 | uresp = {} 232 | content = to_text(content, encoding='UTF-8') 233 | 234 | try: 235 | js = json.loads(content) 236 | except ValueError: 237 | js = "" 238 | 239 | uresp['json'] = js 240 | uresp['status'] = status 241 | uresp['msg'] = message 242 | uresp['url'] = url 243 | 244 | module.exit_json(**uresp) 245 | 246 | 247 | if __name__ == '__main__': 248 | main() 249 | -------------------------------------------------------------------------------- /library/graylog_pipelines.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_pipelines 14 | short_description: Communicate with the Graylog API to manage pipelines 15 | description: 16 | - The Graylog pipelines module manages Graylog pipelines. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endoint. (i.e. graylog.mydomain.com). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against pipeline API. 50 | required: false 51 | default: list 52 | choices: [ create, create_connection, parse_pipeline, parse_rule, create_rule, update, 53 | update_connection, update_rule, delete, delete_rule, list, list_rules, query_rules, query_pipelines ] 54 | type: str 55 | pipeline_id: 56 | description: 57 | - Pipeline ID. 58 | required: false 59 | type: str 60 | pipeline_name: 61 | description: 62 | - Pipeline name. 63 | required: false 64 | type: str 65 | stream_ids: 66 | description: 67 | - Stream IDs. 68 | required: false 69 | type: list 70 | rule_id: 71 | description: 72 | - Rule ID. 73 | required: false 74 | type: str 75 | rule_name: 76 | description: 77 | - Rule name. 78 | required: false 79 | type: str 80 | title: 81 | description: 82 | - Title. 83 | required: false 84 | type: str 85 | description: 86 | description: 87 | - Description. 88 | required: false 89 | type: str 90 | source: 91 | description: 92 | - Rule source. 93 | required: false 94 | type: str 95 | ''' 96 | 97 | EXAMPLES = ''' 98 | # List pipelines 99 | - graylog_pipelines: 100 | endpoint: "graylog.mydomain.com" 101 | graylog_user: "username" 102 | graylog_password: "password" 103 | 104 | # Validate/parse pipeline rule 105 | - graylog_pipelines: 106 | action: parse_rule 107 | endpoint: "graylog.mydomain.com" 108 | graylog_user: "username" 109 | graylog_password: "password" 110 | source: | 111 | rule "test_rule_domain_threat_intel" 112 | when 113 | has_field("dns_query") 114 | then 115 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 116 | set_fields(dns_query_intel); 117 | end 118 | 119 | # Create pipeline rule 120 | - graylog_pipelines: 121 | action: create_rule 122 | endpoint: "graylog.mydomain.com" 123 | graylog_user: "username" 124 | graylog_password: "password" 125 | title: "test_rule" 126 | description: "test" 127 | source: | 128 | rule "test_rule_domain_threat_intel" 129 | when 130 | has_field("dns_query") 131 | then 132 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 133 | set_fields(dns_query_intel); 134 | end 135 | 136 | # Validate/parse pipeline 137 | - graylog_pipelines: 138 | action: parse_pipeline 139 | endpoint: "graylog.mydomain.com" 140 | graylog_user: "username" 141 | graylog_password: "password" 142 | source: | 143 | pipeline "test_pipeline" 144 | stage 1 match either 145 | rule "test_rule_domain_threat_intel 146 | end 147 | 148 | # Create pipeline with new rule 149 | - graylog_pipelines: 150 | action: create 151 | endpoint: "graylog.mydomain.com" 152 | graylog_user: "username" 153 | graylog_password: "password" 154 | title: "test_pipeline" 155 | description: "test" 156 | source: | 157 | pipeline "test_pipeline" 158 | stage 1 match either 159 | rule "test_rule_domain_threat_intel 160 | end 161 | 162 | # Get pipeline from pipeline name query_pipelines 163 | - graylog_pipelines: 164 | action: query_pipelines 165 | endpoint: "graylog.mydomain.com" 166 | graylog_user: "username" 167 | graylog_password: "password" 168 | pipeline_name: "test_pipeline" 169 | register: pipeline 170 | 171 | # Create Stream connection to processing pipeline 172 | - graylog_pipelines: 173 | action: create_connection 174 | endpoint: "graylog.mydomain.com" 175 | graylog_user: "username" 176 | graylog_password: "password" 177 | pipeline_id: "{{ pipeline.json.id }}" 178 | stream_ids: 179 | - "{{ stream.json.id }}" 180 | 181 | # Update Stream connection to processing pipeline 182 | - graylog_pipelines: 183 | action: update_connection 184 | endpoint: "graylog.mydomain.com" 185 | graylog_user: "username" 186 | graylog_password: "password" 187 | pipeline_id: "{{ pipeline.json.id }}" 188 | stream_ids: 189 | - "{{ stream.json.id }}" 190 | 191 | # Remove all Streams from a pipeline 192 | - graylog_pipelines: 193 | action: update_connection 194 | endpoint: "graylog.mydomain.com" 195 | graylog_user: "username" 196 | graylog_password: "password" 197 | pipeline_id: "{{ pipeline.json.id }}" 198 | stream_ids: [] 199 | ''' 200 | 201 | RETURN = ''' 202 | json: 203 | description: The JSON response from the Graylog API 204 | returned: always 205 | type: complex 206 | contains: 207 | title: 208 | description: Pipeline title. 209 | returned: success 210 | type: str 211 | sample: 'Threat Detection' 212 | created_at: 213 | description: Pipeline creation time. 214 | returned: success 215 | type: str 216 | sample: '2018-10-17T17:53:42.675Z' 217 | description: 218 | description: Pipeline description. 219 | returned: success 220 | type: str 221 | sample: 'Threat Detection pipeline' 222 | id: 223 | description: Pipeline ID. 224 | returned: success 225 | type: str 226 | sample: '4a362233815c349e7e2b945c' 227 | modified_at: 228 | description: Pipeline modified time. 229 | returned: success 230 | type: str 231 | sample: '2018-10-17T18:22:42.599Z' 232 | source: 233 | description: Pipeline source. 234 | returned: success 235 | type: str 236 | sample: 'pipeline \"Threat Detection\"\nstage 0 match either\nrule \"threat_rules\"\nend' 237 | stages: 238 | description: Pipeline title. 239 | returned: success 240 | type: dict 241 | sample: '{ "match_all": false, "rules": [ "threat_rules" ], "stage": 0 }' 242 | status: 243 | description: The HTTP status code from the request 244 | returned: always 245 | type: int 246 | sample: 200 247 | url: 248 | description: The actual URL used for the request 249 | returned: always 250 | type: str 251 | sample: https://www.ansible.com/ 252 | ''' 253 | 254 | 255 | # import module snippets 256 | import json 257 | import base64 258 | from ansible.module_utils.basic import AnsibleModule 259 | from ansible.module_utils.urls import fetch_url, to_text 260 | 261 | 262 | def create(module, pipeline_url, headers): 263 | 264 | url = pipeline_url 265 | 266 | payload = {} 267 | 268 | for key in ['title', 'description', 'source']: 269 | if module.params[key] is not None: 270 | payload[key] = module.params[key] 271 | 272 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='POST', data=module.jsonify(payload)) 273 | 274 | if info['status'] != 200: 275 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 276 | 277 | try: 278 | content = to_text(response.read(), errors='surrogate_or_strict') 279 | except AttributeError: 280 | content = info.pop('body', '') 281 | 282 | return info['status'], info['msg'], content, url 283 | 284 | 285 | def create_connection(module, connection_url, headers): 286 | 287 | url = "/".join([connection_url, "to_pipeline"]) 288 | 289 | payload = {} 290 | 291 | for key in ['pipeline_id', 'stream_ids']: 292 | if module.params[key] is not None: 293 | payload[key] = module.params[key] 294 | 295 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 296 | 297 | if info['status'] != 200: 298 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 299 | 300 | try: 301 | content = to_text(response.read(), errors='surrogate_or_strict') 302 | except AttributeError: 303 | content = info.pop('body', '') 304 | 305 | return info['status'], info['msg'], content, url 306 | 307 | 308 | def parse_rule(module, rule_url, headers): 309 | 310 | url = "/".join([rule_url, "parse"]) 311 | 312 | payload = {} 313 | 314 | for key in ['source']: 315 | if module.params[key] is not None: 316 | payload[key] = module.params[key] 317 | 318 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='POST', data=module.jsonify(payload)) 319 | 320 | if info['status'] != 200: 321 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 322 | 323 | try: 324 | content = to_text(response.read(), errors='surrogate_or_strict') 325 | except AttributeError: 326 | content = info.pop('body', '') 327 | 328 | return info['status'], info['msg'], content, url 329 | 330 | 331 | def parse_pipeline(module, pipeline_url, headers): 332 | 333 | url = "/".join([pipeline_url, "parse"]) 334 | 335 | payload = {} 336 | 337 | for key in ['source']: 338 | if module.params[key] is not None: 339 | payload[key] = module.params[key] 340 | 341 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='POST', data=module.jsonify(payload)) 342 | 343 | if info['status'] != 200: 344 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 345 | 346 | try: 347 | content = to_text(response.read(), errors='surrogate_or_strict') 348 | except AttributeError: 349 | content = info.pop('body', '') 350 | 351 | return info['status'], info['msg'], content, url 352 | 353 | 354 | def create_rule(module, rule_url, headers): 355 | 356 | url = rule_url 357 | 358 | payload = {} 359 | 360 | for key in ['title', 'description', 'source']: 361 | if module.params[key] is not None: 362 | payload[key] = module.params[key] 363 | 364 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='POST', data=module.jsonify(payload)) 365 | 366 | if info['status'] != 200: 367 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 368 | 369 | try: 370 | content = to_text(response.read(), errors='surrogate_or_strict') 371 | except AttributeError: 372 | content = info.pop('body', '') 373 | 374 | return info['status'], info['msg'], content, url 375 | 376 | 377 | def update(module, pipeline_url, headers): 378 | 379 | payload = {} 380 | 381 | if module.params['pipeline_id'] is not None: 382 | url = "/".join([pipeline_url, module.params['pipeline_id']]) 383 | else: 384 | url = pipeline_url 385 | 386 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 387 | 388 | if info['status'] != 200: 389 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 390 | 391 | try: 392 | content = to_text(response.read(), errors='surrogate_or_strict') 393 | payload_current = json.loads(content) 394 | except AttributeError: 395 | content = info.pop('body', '') 396 | 397 | url = "/".join([pipeline_url, module.params['pipeline_id']]) 398 | 399 | for key in ['title', 'description', 'source']: 400 | if module.params[key] is not None: 401 | payload[key] = module.params[key] 402 | 403 | if module.params['source'] is None: 404 | payload['source'] = payload_current['source'] 405 | 406 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='PUT', data=module.jsonify(payload)) 407 | 408 | if info['status'] != 200: 409 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 410 | 411 | try: 412 | content = to_text(response.read(), errors='surrogate_or_strict') 413 | except AttributeError: 414 | content = info.pop('body', '') 415 | 416 | return info['status'], info['msg'], content, url 417 | 418 | 419 | def update_connection(module, connection_url, headers): 420 | 421 | url = "/".join([connection_url, "to_pipeline"]) 422 | 423 | payload = {} 424 | 425 | for key in ['pipeline_id', 'stream_ids']: 426 | if module.params[key] is not None: 427 | payload[key] = module.params[key] 428 | 429 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 430 | 431 | if info['status'] != 200: 432 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 433 | 434 | try: 435 | content = to_text(response.read(), errors='surrogate_or_strict') 436 | except AttributeError: 437 | content = info.pop('body', '') 438 | 439 | return info['status'], info['msg'], content, url 440 | 441 | 442 | def update_rule(module, rule_url, headers): 443 | 444 | url = "/".join([rule_url, module.params['rule_id']]) 445 | 446 | payload = {} 447 | 448 | for key in ['title', 'description', 'source']: 449 | if module.params[key] is not None: 450 | payload[key] = module.params[key] 451 | 452 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', timeout=20, data=module.jsonify(payload)) 453 | 454 | if info['status'] != 200: 455 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 456 | 457 | try: 458 | content = to_text(response.read(), errors='surrogate_or_strict') 459 | except AttributeError: 460 | content = info.pop('body', '') 461 | 462 | return info['status'], info['msg'], content, url 463 | 464 | 465 | def delete(module, pipeline_url, headers, pipeline_id): 466 | 467 | url = "/".join([pipeline_url, pipeline_id]) 468 | 469 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 470 | 471 | if info['status'] != 204: 472 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 473 | 474 | try: 475 | content = to_text(response.read(), errors='surrogate_or_strict') 476 | except AttributeError: 477 | content = info.pop('body', '') 478 | 479 | return info['status'], info['msg'], content, url 480 | 481 | 482 | def delete_rule(module, rule_url, headers, rule_id): 483 | 484 | url = "/".join([rule_url, rule_id]) 485 | 486 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 487 | 488 | if info['status'] != 204: 489 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 490 | 491 | try: 492 | content = to_text(response.read(), errors='surrogate_or_strict') 493 | except AttributeError: 494 | content = info.pop('body', '') 495 | 496 | return info['status'], info['msg'], content, url 497 | 498 | 499 | def list(module, pipeline_url, headers, pipeline_id, query): 500 | 501 | if pipeline_id is not None and pipeline_id != "": 502 | url = "/".join([pipeline_url, pipeline_id]) 503 | elif query == "yes" and pipeline_id == "": 504 | url = "/".join([pipeline_url, "0"]) 505 | else: 506 | url = pipeline_url 507 | 508 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='GET') 509 | 510 | if info['status'] != 200: 511 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 512 | 513 | try: 514 | content = to_text(response.read(), errors='surrogate_or_strict') 515 | except AttributeError: 516 | content = info.pop('body', '') 517 | 518 | return info['status'], info['msg'], content, url 519 | 520 | 521 | def list_rules(module, rule_url, headers, rule_id, query): 522 | 523 | if rule_id is not None and rule_id != "": 524 | url = "/".join([rule_url, rule_id]) 525 | elif query == "yes" and rule_id == "": 526 | url = "/".join([rule_url, "0"]) 527 | else: 528 | url = rule_url 529 | 530 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='GET') 531 | 532 | if info['status'] != 200: 533 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 534 | 535 | try: 536 | content = to_text(response.read(), errors='surrogate_or_strict') 537 | except AttributeError: 538 | content = info.pop('body', '') 539 | 540 | return info['status'], info['msg'], content, url 541 | 542 | 543 | def query_pipelines(module, pipeline_url, headers, pipeline_name): 544 | 545 | url = pipeline_url 546 | 547 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 548 | 549 | if info['status'] != 200: 550 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 551 | 552 | try: 553 | content = to_text(response.read(), errors='surrogate_or_strict') 554 | pipelines = json.loads(content) 555 | except AttributeError: 556 | content = info.pop('body', '') 557 | 558 | pipeline_id = "" 559 | if pipelines is not None: 560 | 561 | i = 0 562 | while i < len(pipelines): 563 | pipeline = pipelines[i] 564 | if pipeline_name == pipeline['title']: 565 | pipeline_id = pipeline['id'] 566 | break 567 | i += 1 568 | 569 | return pipeline_id 570 | 571 | 572 | def query_rules(module, rule_url, headers, rule_name): 573 | 574 | url = rule_url 575 | 576 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), timeout=20, method='GET') 577 | 578 | if info['status'] != 200: 579 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 580 | 581 | try: 582 | content = to_text(response.read(), errors='surrogate_or_strict') 583 | rules = json.loads(content) 584 | except AttributeError: 585 | content = info.pop('body', '') 586 | 587 | rule_id = "" 588 | if rules is not None: 589 | 590 | i = 0 591 | while i < len(rules): 592 | rule = rules[i] 593 | if rule_name == rule['title']: 594 | rule_id = rule['id'] 595 | break 596 | i += 1 597 | 598 | return rule_id 599 | 600 | 601 | def get_token(module, endpoint, username, password): 602 | 603 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 604 | 605 | url = endpoint + "/api/system/sessions" 606 | 607 | payload = { 608 | 'username': username, 609 | 'password': password, 610 | 'host': endpoint 611 | } 612 | 613 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 614 | 615 | if info['status'] != 200: 616 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 617 | 618 | try: 619 | content = to_text(response.read(), errors='surrogate_or_strict') 620 | session = json.loads(content) 621 | except AttributeError: 622 | content = info.pop('body', '') 623 | 624 | session_string = session['session_id'] + ":session" 625 | session_bytes = session_string.encode('utf-8') 626 | session_token = base64.b64encode(session_bytes) 627 | 628 | return session_token 629 | 630 | 631 | def main(): 632 | module = AnsibleModule( 633 | argument_spec=dict( 634 | endpoint=dict(type='str'), 635 | graylog_user=dict(type='str'), 636 | graylog_password=dict(type='str', no_log=True), 637 | allow_http=dict(type='bool', required=False, default=False), 638 | validate_certs=dict(type='bool', required=False, default=True), 639 | action=dict(type='str', required=False, default='list', 640 | choices=['create', 'create_connection', 'parse_pipeline', 'parse_rule', 'create_rule', 'update', 'update_connection', 641 | 'update_rule', 'delete', 'delete_rule', 'list', 'list_rules', 'query_rules', 'query_pipelines']), 642 | pipeline_id=dict(type='str'), 643 | pipeline_name=dict(type='str'), 644 | rule_id=dict(type='str'), 645 | rule_name=dict(type='str'), 646 | stream_ids=dict(type='list'), 647 | title=dict(type='str'), 648 | description=dict(type='str'), 649 | source=dict(type='str') 650 | ) 651 | ) 652 | 653 | endpoint = module.params['endpoint'] 654 | graylog_user = module.params['graylog_user'] 655 | graylog_password = module.params['graylog_password'] 656 | action = module.params['action'] 657 | pipeline_id = module.params['pipeline_id'] 658 | pipeline_name = module.params['pipeline_name'] 659 | rule_id = module.params['rule_id'] 660 | rule_name = module.params['rule_name'] 661 | allow_http = module.params['allow_http'] 662 | 663 | if allow_http == True: 664 | endpoint = "http://" + endpoint 665 | else: 666 | endpoint = "https://" + endpoint 667 | 668 | pipeline_url = endpoint + "/api/system/pipelines/pipeline" 669 | rule_url = endpoint + "/api/system/pipelines/rule" 670 | connection_url = endpoint + "/api/system/pipelines/connections" 671 | 672 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 673 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 674 | "Authorization": "Basic ' + api_token.decode() + '" }' 675 | 676 | if action == "create": 677 | status, message, content, url = create(module, pipeline_url, headers) 678 | elif action == "parse_pipeline": 679 | status, message, content, url = parse_pipeline(module, pipeline_url, headers) 680 | elif action == "parse_rule": 681 | status, message, content, url = parse_rule(module, rule_url, headers) 682 | elif action == "create_rule": 683 | status, message, content, url = create_rule(module, rule_url, headers) 684 | elif action == "create_connection": 685 | status, message, content, url = create_connection(module, connection_url, headers) 686 | elif action == "update": 687 | status, message, content, url = update(module, pipeline_url, headers) 688 | elif action == "update_connection": 689 | status, message, content, url = update_connection(module, connection_url, headers) 690 | elif action == "update_rule": 691 | status, message, content, url = update_rule(module, rule_url, headers) 692 | elif action == "delete": 693 | status, message, content, url = delete(module, pipeline_url, headers, pipeline_id) 694 | elif action == "delete_rule": 695 | status, message, content, url = delete_rule(module, rule_url, headers, rule_id) 696 | elif action == "list": 697 | query = "no" 698 | status, message, content, url = list(module, pipeline_url, headers, pipeline_id, query) 699 | elif action == "query_pipelines": 700 | pipeline_id = query_pipelines(module, pipeline_url, headers, pipeline_name) 701 | query = "yes" 702 | status, message, content, url = list(module, pipeline_url, headers, pipeline_id, query) 703 | elif action == "list_rules": 704 | query = "no" 705 | status, message, content, url = list_rules(module, rule_url, headers, rule_id, query) 706 | elif action == "query_rules": 707 | rule_id = query_rules(module, rule_url, headers, rule_name) 708 | query = "yes" 709 | status, message, content, url = list_rules(module, rule_url, headers, rule_id, query) 710 | 711 | uresp = {} 712 | content = to_text(content, encoding='UTF-8') 713 | 714 | try: 715 | js = json.loads(content) 716 | except ValueError: 717 | js = "" 718 | 719 | uresp['json'] = js 720 | uresp['status'] = status 721 | uresp['msg'] = message 722 | uresp['url'] = url 723 | 724 | module.exit_json(**uresp) 725 | 726 | 727 | if __name__ == '__main__': 728 | main() 729 | -------------------------------------------------------------------------------- /library/graylog_roles.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_roles 14 | short_description: Communicate with the Graylog API to manage roles 15 | description: 16 | - The Graylog roles module manages Graylog roles. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endoint. (i.e. graylog.mydomain.com). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against role API. 50 | required: false 51 | default: list 52 | choices: [ create, update, delete, list ] 53 | type: str 54 | name: 55 | description: 56 | - Role name. 57 | required: false 58 | type: str 59 | description: 60 | description: 61 | - Role description. 62 | required: false 63 | type: str 64 | permissions: 65 | description: 66 | - Permissions list for role. 67 | required: false 68 | type: list 69 | read_only: 70 | description: 71 | - Read only, true or false. 72 | required: false 73 | default: "false" 74 | type: str 75 | ''' 76 | 77 | EXAMPLES = ''' 78 | # List roles 79 | - graylog_roles: 80 | endpoint: "graylog.mydomain.com" 81 | graylog_user: "username" 82 | graylog_password: "password" 83 | 84 | # Create role 85 | - graylog_roles: 86 | action: create 87 | endpoint: "graylog.mydomain.com" 88 | graylog_user: "username" 89 | graylog_password: "password" 90 | name: "analysts" 91 | description: "Analyst role" 92 | permissions: 93 | - "streams:read" 94 | - "dashboards:read" 95 | read_only: "true" 96 | 97 | # Create admin role 98 | - graylog_roles: 99 | action: create 100 | endpoint: "graylog.mydomain.com" 101 | graylog_user: "username" 102 | graylog_password: "password" 103 | name: "admins" 104 | description: "Admin role" 105 | permissions: 106 | - "*" 107 | read_only: "false" 108 | 109 | # Delete role 110 | - graylog_roles: 111 | action: delete 112 | endpoint: "graylog.mydomain.com" 113 | graylog_user: "username" 114 | graylog_password: "password" 115 | name: "admins" 116 | ''' 117 | 118 | RETURN = ''' 119 | json: 120 | description: The JSON response from the Graylog API 121 | returned: always 122 | type: complex 123 | contains: 124 | name: 125 | description: Role name. 126 | returned: success 127 | type: str 128 | sample: 'Administrators' 129 | description: 130 | description: Role description. 131 | returned: success 132 | type: str 133 | sample: 'Administrators group' 134 | permissions: 135 | description: Role permissions (dashboards, streams, collectors, etc). 136 | returned: success 137 | type: list 138 | sample: [ "dashboards:read:4c58eef77ec84145c3a2d9f3", "sidecars:update" ] 139 | read_only: 140 | description: Whether or not the role is a read-only role. 141 | returned: success 142 | type: bool 143 | sample: false 144 | status: 145 | description: The HTTP status code from the request 146 | returned: always 147 | type: int 148 | sample: 200 149 | url: 150 | description: The actual URL used for the request 151 | returned: always 152 | type: str 153 | sample: https://www.ansible.com/ 154 | ''' 155 | 156 | 157 | # import module snippets 158 | import json 159 | import base64 160 | from ansible.module_utils.basic import AnsibleModule 161 | from ansible.module_utils.urls import fetch_url, to_text 162 | 163 | 164 | def create(module, base_url, headers): 165 | 166 | url = base_url 167 | 168 | payload = {} 169 | 170 | for key in ['name', 'description', 'permissions', 'read_only']: 171 | if module.params[key] is not None: 172 | payload[key] = module.params[key] 173 | 174 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 175 | 176 | if info['status'] != 201: 177 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 178 | 179 | try: 180 | content = to_text(response.read(), errors='surrogate_or_strict') 181 | except AttributeError: 182 | content = info.pop('body', '') 183 | 184 | return info['status'], info['msg'], content, url 185 | 186 | 187 | def update(module, base_url, headers): 188 | 189 | url = "/".join([base_url, module.params['name']]) 190 | 191 | payload = {} 192 | 193 | for key in ['name', 'description', 'permissions', 'read_only']: 194 | if module.params[key] is not None: 195 | payload[key] = module.params[key] 196 | 197 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 198 | 199 | if info['status'] != 200: 200 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 201 | 202 | try: 203 | content = to_text(response.read(), errors='surrogate_or_strict') 204 | except AttributeError: 205 | content = info.pop('body', '') 206 | 207 | return info['status'], info['msg'], content, url 208 | 209 | 210 | def delete(module, base_url, headers): 211 | 212 | url = "/".join([base_url, module.params['name']]) 213 | 214 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 215 | 216 | if info['status'] != 204: 217 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 218 | 219 | try: 220 | content = to_text(response.read(), errors='surrogate_or_strict') 221 | except AttributeError: 222 | content = info.pop('body', '') 223 | 224 | return info['status'], info['msg'], content, url 225 | 226 | 227 | def list(module, base_url, headers): 228 | 229 | url = base_url 230 | 231 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 232 | 233 | if info['status'] != 200: 234 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 235 | 236 | try: 237 | content = to_text(response.read(), errors='surrogate_or_strict') 238 | except AttributeError: 239 | content = info.pop('body', '') 240 | 241 | return info['status'], info['msg'], content, url 242 | 243 | 244 | def get_token(module, endpoint, username, password): 245 | 246 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 247 | 248 | url = endpoint + "/api/system/sessions" 249 | 250 | payload = { 251 | 'username': username, 252 | 'password': password, 253 | 'host': endpoint 254 | } 255 | 256 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 257 | 258 | if info['status'] != 200: 259 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 260 | 261 | try: 262 | content = to_text(response.read(), errors='surrogate_or_strict') 263 | session = json.loads(content) 264 | except AttributeError: 265 | content = info.pop('body', '') 266 | 267 | session_string = session['session_id'] + ":session" 268 | session_bytes = session_string.encode('utf-8') 269 | session_token = base64.b64encode(session_bytes) 270 | 271 | return session_token 272 | 273 | 274 | def main(): 275 | module = AnsibleModule( 276 | argument_spec=dict( 277 | endpoint=dict(type='str'), 278 | graylog_user=dict(type='str'), 279 | graylog_password=dict(type='str', no_log=True), 280 | allow_http=dict(type='bool', required=False, default=False), 281 | validate_certs=dict(type='bool', required=False, default=True), 282 | action=dict(type='str', default='list', choices=['create', 'update', 'delete', 'list']), 283 | name=dict(type='str'), 284 | description=dict(type='str'), 285 | permissions=dict(type='list'), 286 | read_only=dict(type='str', default="false") 287 | ) 288 | ) 289 | 290 | endpoint = module.params['endpoint'] 291 | graylog_user = module.params['graylog_user'] 292 | graylog_password = module.params['graylog_password'] 293 | allow_http = module.params['allow_http'] 294 | action = module.params['action'] 295 | 296 | if allow_http == True: 297 | endpoint = "http://" + endpoint 298 | else: 299 | endpoint = "https://" + endpoint 300 | 301 | base_url = endpoint + "/api/roles" 302 | 303 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 304 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 305 | "Authorization": "Basic ' + api_token.decode() + '" }' 306 | 307 | if action == "create": 308 | status, message, content, url = create(module, base_url, headers) 309 | elif action == "update": 310 | status, message, content, url = update(module, base_url, headers) 311 | elif action == "delete": 312 | status, message, content, url = delete(module, base_url, headers) 313 | elif action == "list": 314 | status, message, content, url = list(module, base_url, headers) 315 | 316 | uresp = {} 317 | content = to_text(content, encoding='UTF-8') 318 | 319 | try: 320 | js = json.loads(content) 321 | except ValueError: 322 | js = "" 323 | 324 | uresp['json'] = js 325 | uresp['status'] = status 326 | uresp['msg'] = message 327 | uresp['url'] = url 328 | 329 | module.exit_json(**uresp) 330 | 331 | 332 | if __name__ == '__main__': 333 | main() 334 | -------------------------------------------------------------------------------- /library/graylog_streams.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_streams 14 | short_description: Communicate with the Graylog API to manage streams 15 | description: 16 | - The Graylog streams module manages Graylog streams. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endoint. (i.e. graylog.mydomain.com). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against stream API. 50 | required: false 51 | default: list 52 | choices: [ create, create_rule, start, pause, update, update_rule, delete, delete_rule, list, query_streams ] 53 | type: str 54 | title: 55 | description: 56 | - Stream title. 57 | required: false 58 | type: str 59 | description: 60 | description: 61 | - Stream description. 62 | required: false 63 | type: str 64 | stream_id: 65 | description: 66 | - Stream ID. 67 | required: false 68 | type: str 69 | rule_id: 70 | description: 71 | - Rule ID. 72 | required: false 73 | type: str 74 | index_set_id: 75 | description: 76 | - Index set ID. 77 | required: false 78 | type: str 79 | matching_type: 80 | description: 81 | - Matching type for the stream rules. 82 | required: false 83 | type: str 84 | remove_matches_from_default_stream: 85 | description: 86 | - Remove matches from default stream, true or false. 87 | required: false 88 | default: False 89 | type: bool 90 | stream_name: 91 | description: 92 | - Stream name to use with the query_streams action. 93 | required: false 94 | type: str 95 | field: 96 | description: 97 | - Field name for the stream rule to check. 98 | required: false 99 | type: str 100 | type: 101 | description: 102 | - Rule type for the stream rule, 1-7. 103 | required: false 104 | default: 1 105 | type: int 106 | value: 107 | description: 108 | - Value to check rule against. 109 | required: false 110 | type: str 111 | inverted: 112 | description: 113 | - Invert rule (must not match value). 114 | required: false 115 | default: False 116 | type: bool 117 | rules: 118 | description: 119 | - List of rules associated with a stream. 120 | required: false 121 | type: list 122 | ''' 123 | 124 | EXAMPLES = ''' 125 | # List streams 126 | - graylog_streams: 127 | endpoint: "graylog.mydomain.com" 128 | graylog_user: "username" 129 | graylog_password: "password" 130 | 131 | # Get stream from stream name query_streams 132 | - graylog_streams: 133 | action: query_streams 134 | endpoint: "graylog.mydomain.com" 135 | graylog_user: "username" 136 | graylog_password: "password" 137 | stream_name: "test_stream" 138 | register: stream 139 | 140 | # List single stream by ID 141 | - graylog_streams: 142 | endpoint: "graylog.mydomain.com" 143 | graylog_user: "username" 144 | graylog_password: "password" 145 | stream_id: "{{ stream.json.id }}" 146 | 147 | # Create stream 148 | - graylog_streams: 149 | action: create 150 | endpoint: "graylog.mydomain.com" 151 | graylog_user: "username" 152 | graylog_password: "password" 153 | title: "Client XYZ" 154 | description: "Windows and IIS logs" 155 | matching_type: "AND" 156 | remove_matches_from_default_stream: False 157 | rules: 158 | - '{"field":"message", "type":"6", "value":"test", "inverted":true, "description":"testrule"}' 159 | 160 | # Update stream 161 | - graylog_streams: 162 | action: update 163 | endpoint: "graylog.mydomain.com" 164 | graylog_user: "username" 165 | graylog_password: "password" 166 | stream_id: "{{ stream.json.id }}" 167 | remove_matches_from_default_stream: True 168 | 169 | # Create stream rule 170 | - graylog_streams: 171 | action: create_rule 172 | endpoint: "graylog.mydomain.com" 173 | graylog_user: "username" 174 | graylog_password: "password" 175 | stream_id: "{{ stream.json.id }}" 176 | description: "Windows Security Logs" 177 | field: "winlogbeat_log_name" 178 | type: 1 179 | value: "Security" 180 | inverted: False 181 | 182 | # Start stream 183 | - graylog_streams: 184 | action: start 185 | endpoint: "graylog.mydomain.com" 186 | graylog_user: "username" 187 | graylog_password: "password" 188 | stream_id: "{{ stream.json.id }}" 189 | 190 | # Pause stream 191 | - graylog_streams: 192 | action: pause 193 | endpoint: "graylog.mydomain.com" 194 | graylog_user: "username" 195 | graylog_password: "password" 196 | stream_id: "{{ stream.json.id }}" 197 | 198 | # Update stream rule 199 | - graylog_streams: 200 | action: update_rule 201 | endpoint: "graylog.mydomain.com" 202 | graylog_user: "username" 203 | graylog_password: "password" 204 | stream_id: "{{ stream.json.id }}" 205 | rule_id: "{{ rule.json.id }}" 206 | description: "Windows Security and Application Logs" 207 | 208 | # Delete stream rule 209 | - graylog_streams: 210 | action: delete_rule 211 | endpoint: "graylog.mydomain.com" 212 | graylog_user: "username" 213 | graylog_password: "password" 214 | stream_id: "{{ stream.json.id }}" 215 | rule_id: "{{ rule.json.id }}" 216 | 217 | # Delete stream 218 | - graylog_streams: 219 | action: delete 220 | endpoint: "graylog.mydomain.com" 221 | graylog_user: "username" 222 | graylog_password: "password" 223 | stream_id: "{{ stream.json.id }}" 224 | ''' 225 | 226 | RETURN = ''' 227 | json: 228 | description: The JSON response from the Graylog API 229 | returned: always 230 | type: complex 231 | contains: 232 | title: 233 | description: Stream title. 234 | returned: success 235 | type: str 236 | sample: 'Windows Logs' 237 | alert_conditions: 238 | description: Alert conditions. 239 | returned: success 240 | type: dict 241 | sample: | 242 | [ 243 | { 244 | "created_at": "2018-10-18T18:40:21.582+0000", 245 | "creator_user_id": "admin", 246 | "id": "cc43d4e7-e7b2-4abc-7c44-4b29cadaf364", 247 | "parameters": { 248 | "backlog": 1, 249 | "grace": 0, 250 | "repeat_notifications": true, 251 | "threshold": 0, 252 | "threshold_type": "MORE", 253 | "time": 1 254 | }, 255 | "title": "Failed Logon", 256 | "type": "message_count" 257 | } 258 | ] 259 | alert_receivers: 260 | description: Alert receivers. 261 | returned: success 262 | type: dict 263 | sample: '{ "emails": [], "users": [] }' 264 | content_pack: 265 | description: Content pack. 266 | returned: success 267 | type: str 268 | sample: null 269 | created_at: 270 | description: Stream creation time. 271 | returned: success 272 | type: str 273 | sample: "2018-10-17T15:29:20.735Z" 274 | creator_user_id: 275 | description: Stream creator. 276 | returned: success 277 | type: str 278 | sample: "admin" 279 | description: 280 | description: Stream description. 281 | returned: success 282 | type: str 283 | sample: "Stream for Windows logs" 284 | disabled: 285 | description: Whether or not the stream is enabled. 286 | returned: success 287 | type: bool 288 | sample: false 289 | id: 290 | description: Stream ID. 291 | returned: success 292 | type: str 293 | sample: "5bc7666089675c7f7d7f08d7" 294 | index_set_id: 295 | description: Index set ID associated with the stream. 296 | returned: success 297 | type: str 298 | sample: "4bc7444089575c7f7d7f08d7" 299 | is_default: 300 | description: Whether or not it is the default stream. 301 | returned: success 302 | type: bool 303 | sample: false 304 | matching_type: 305 | description: Stream rule matching type. 306 | returned: success 307 | type: str 308 | sample: "AND" 309 | outputs: 310 | description: Stream outputs. 311 | returned: success 312 | type: dict 313 | sample: [] 314 | remove_matches_from_default_stream: 315 | description: Whether or messages are removed from the default stream. 316 | returned: success 317 | type: bool 318 | sample: false 319 | rules: 320 | description: Rules associated with the stream. 321 | returned: success 322 | type: dict 323 | sample: [] 324 | status: 325 | description: The HTTP status code from the request 326 | returned: always 327 | type: int 328 | sample: 200 329 | url: 330 | description: The actual URL used for the request 331 | returned: always 332 | type: str 333 | sample: https://www.ansible.com/ 334 | ''' 335 | 336 | 337 | # import module snippets 338 | import json 339 | import base64 340 | from ansible.module_utils.basic import AnsibleModule 341 | from ansible.module_utils.urls import fetch_url, to_text 342 | 343 | 344 | def create(module, base_url, headers, index_set_id): 345 | 346 | url = base_url 347 | 348 | payload = {} 349 | 350 | for key in ['title', 'description', 'remove_matches_from_default_stream', 'matching_type', 'rules']: 351 | if module.params[key] is not None and module.params[key] != "": 352 | payload[key] = module.params[key] 353 | 354 | payload['index_set_id'] = index_set_id 355 | 356 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 357 | 358 | if info['status'] != 201: 359 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 360 | 361 | try: 362 | content = to_text(response.read(), errors='surrogate_or_strict') 363 | except AttributeError: 364 | content = info.pop('body', '') 365 | 366 | return info['status'], info['msg'], content, url 367 | 368 | 369 | def create_rule(module, base_url, headers): 370 | 371 | url = "/".join([base_url, module.params['stream_id'], "rules"]) 372 | 373 | payload = {} 374 | 375 | for key in ['field', 'type', 'value', 'inverted', 'description']: 376 | if module.params[key] is not None: 377 | payload[key] = module.params[key] 378 | 379 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 380 | 381 | if info['status'] != 201: 382 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 383 | 384 | try: 385 | content = to_text(response.read(), errors='surrogate_or_strict') 386 | except AttributeError: 387 | content = info.pop('body', '') 388 | 389 | return info['status'], info['msg'], content, url 390 | 391 | 392 | def update(module, base_url, headers, stream_id, title, description, remove_matches_from_default_stream, matching_type, rules, index_set_id): 393 | 394 | url = "/".join([base_url, stream_id]) 395 | 396 | payload = {} 397 | 398 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 399 | 400 | if info['status'] != 200: 401 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 402 | 403 | try: 404 | content = to_text(response.read(), errors='surrogate_or_strict') 405 | payload_current = json.loads(content) 406 | except AttributeError: 407 | content = info.pop('body', '') 408 | 409 | if title is not None: 410 | payload['title'] = title 411 | else: 412 | payload['title'] = payload_current['title'] 413 | if description is not None: 414 | payload['description'] = description 415 | else: 416 | payload['description'] = payload_current['description'] 417 | if remove_matches_from_default_stream is not None: 418 | payload['remove_matches_from_default_stream'] = remove_matches_from_default_stream 419 | else: 420 | payload['remove_matches_from_default_stream'] = payload_current['remove_matches_from_default_stream'] 421 | if matching_type is not None: 422 | payload['matching_type'] = matching_type 423 | else: 424 | payload['matching_type'] = payload_current['matching_type'] 425 | if rules is not None: 426 | payload['rules'] = rules 427 | else: 428 | payload['rules'] = payload_current['rules'] 429 | if index_set_id is not None: 430 | payload['index_set_id'] = index_set_id 431 | else: 432 | payload['index_set_id'] = payload_current['index_set_id'] 433 | 434 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 435 | 436 | if info['status'] != 200: 437 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 438 | 439 | try: 440 | content = to_text(response.read(), errors='surrogate_or_strict') 441 | except AttributeError: 442 | content = info.pop('body', '') 443 | 444 | return info['status'], info['msg'], content, url 445 | 446 | 447 | def update_rule(module, base_url, headers, stream_id, rule_id, field, type, value, inverted, description): 448 | 449 | payload = {} 450 | 451 | url = "/".join([base_url, stream_id, "rules", rule_id]) 452 | 453 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 454 | 455 | if info['status'] != 200: 456 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 457 | 458 | try: 459 | content = to_text(response.read(), errors='surrogate_or_strict') 460 | payload_current = json.loads(content) 461 | except AttributeError: 462 | content = info.pop('body', '') 463 | 464 | if field is not None: 465 | payload['field'] = field 466 | else: 467 | payload['field'] = payload_current['field'] 468 | if type is not None: 469 | payload['type'] = type 470 | else: 471 | payload['type'] = payload_current['type'] 472 | if value is not None: 473 | payload['value'] = value 474 | else: 475 | payload['value'] = payload_current['value'] 476 | if inverted is not None: 477 | payload['inverted'] = inverted 478 | else: 479 | payload['inverted'] = payload_current['inverted'] 480 | if description is not None: 481 | payload['description'] = description 482 | else: 483 | payload['description'] = payload_current['description'] 484 | 485 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 486 | 487 | if info['status'] != 200: 488 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 489 | 490 | try: 491 | content = to_text(response.read(), errors='surrogate_or_strict') 492 | except AttributeError: 493 | content = info.pop('body', '') 494 | 495 | return info['status'], info['msg'], content, url 496 | 497 | 498 | def delete(module, base_url, headers, stream_id): 499 | 500 | url = "/".join([base_url, stream_id]) 501 | 502 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 503 | 504 | if info['status'] != 204: 505 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 506 | 507 | try: 508 | content = to_text(response.read(), errors='surrogate_or_strict') 509 | except AttributeError: 510 | content = info.pop('body', '') 511 | 512 | return info['status'], info['msg'], content, url 513 | 514 | 515 | def delete_rule(module, base_url, headers, stream_id, rule_id): 516 | 517 | url = "/".join([base_url, stream_id, "rules", rule_id]) 518 | 519 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 520 | 521 | if info['status'] != 204: 522 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 523 | 524 | try: 525 | content = to_text(response.read(), errors='surrogate_or_strict') 526 | except AttributeError: 527 | content = info.pop('body', '') 528 | 529 | return info['status'], info['msg'], content, url 530 | 531 | 532 | def start(module, base_url, headers, stream_id): 533 | 534 | url = "/".join([base_url, stream_id, "resume"]) 535 | 536 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST') 537 | 538 | if info['status'] != 200: 539 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 540 | 541 | try: 542 | content = to_text(response.read(), errors='surrogate_or_strict') 543 | except AttributeError: 544 | content = info.pop('body', '') 545 | 546 | return info['status'], info['msg'], content, url 547 | 548 | 549 | def pause(module, base_url, headers, stream_id): 550 | 551 | url = "/".join([base_url, stream_id, "pause"]) 552 | 553 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST') 554 | 555 | if info['status'] != 200: 556 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 557 | 558 | try: 559 | content = to_text(response.read(), errors='surrogate_or_strict') 560 | except AttributeError: 561 | content = info.pop('body', '') 562 | 563 | return info['status'], info['msg'], content, url 564 | 565 | 566 | def list(module, base_url, headers, stream_id): 567 | 568 | if stream_id is not None: 569 | url = "/".join([base_url, stream_id]) 570 | else: 571 | url = base_url 572 | 573 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 574 | 575 | if info['status'] != 200: 576 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 577 | 578 | try: 579 | content = to_text(response.read(), errors='surrogate_or_strict') 580 | except AttributeError: 581 | content = info.pop('body', '') 582 | 583 | return info['status'], info['msg'], content, url 584 | 585 | 586 | def query_streams(module, base_url, headers, stream_name): 587 | 588 | url = base_url 589 | 590 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 591 | 592 | if info['status'] != 200: 593 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 594 | 595 | try: 596 | content = to_text(response.read(), errors='surrogate_or_strict') 597 | streams = json.loads(content) 598 | except AttributeError: 599 | content = info.pop('body', '') 600 | 601 | stream_id = "" 602 | if streams is not None: 603 | 604 | i = 0 605 | while i < len(streams['streams']): 606 | stream = streams['streams'][i] 607 | if stream_name == stream['title']: 608 | stream_id = stream['id'] 609 | break 610 | i += 1 611 | 612 | return stream_id 613 | 614 | 615 | def default_index_set(module, endpoint, base_url, headers): 616 | 617 | url = "https://%s/api/system/indices/index_sets?skip=0&limit=0&stats=false" % (endpoint) 618 | 619 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 620 | 621 | if info['status'] != 200: 622 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 623 | 624 | try: 625 | content = to_text(response.read(), errors='surrogate_or_strict') 626 | indices = json.loads(content) 627 | except AttributeError: 628 | content = info.pop('body', '') 629 | 630 | default_index_set_id = "" 631 | if indices is not None: 632 | default_index_set_id = indices['index_sets'][0]['id'] 633 | 634 | return default_index_set_id 635 | 636 | 637 | def get_token(module, endpoint, username, password): 638 | 639 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 640 | 641 | url = endpoint + "/api/system/sessions" 642 | 643 | payload = { 644 | 'username': username, 645 | 'password': password, 646 | 'host': endpoint 647 | } 648 | 649 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 650 | 651 | if info['status'] != 200: 652 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 653 | 654 | try: 655 | content = to_text(response.read(), errors='surrogate_or_strict') 656 | session = json.loads(content) 657 | except AttributeError: 658 | content = info.pop('body', '') 659 | 660 | session_string = session['session_id'] + ":session" 661 | session_bytes = session_string.encode('utf-8') 662 | session_token = base64.b64encode(session_bytes) 663 | 664 | return session_token 665 | 666 | 667 | def main(): 668 | module = AnsibleModule( 669 | argument_spec=dict( 670 | endpoint=dict(type='str'), 671 | graylog_user=dict(type='str'), 672 | graylog_password=dict(type='str', no_log=True), 673 | allow_http=dict(type='bool', required=False, default=False), 674 | validate_certs=dict(type='bool', required=False, default=True), 675 | action=dict(type='str', required=False, default='list', choices=['create', 'create_rule', 'start', 'pause', 676 | 'update', 'update_rule', 'delete', 'delete_rule', 'list', 'query_streams']), 677 | stream_id=dict(type='str'), 678 | stream_name=dict(type='str'), 679 | rule_id=dict(type='str'), 680 | title=dict(type='str'), 681 | field=dict(type='str'), 682 | type=dict(type='int', default=1), 683 | value=dict(type='str'), 684 | index_set_id=dict(type='str'), 685 | inverted=dict(type='bool', default=False), 686 | description=dict(type='str'), 687 | remove_matches_from_default_stream=dict(type='bool', default=False), 688 | matching_type=dict(type='str'), 689 | rules=dict(type='list') 690 | ) 691 | ) 692 | 693 | endpoint = module.params['endpoint'] 694 | graylog_user = module.params['graylog_user'] 695 | graylog_password = module.params['graylog_password'] 696 | action = module.params['action'] 697 | stream_id = module.params['stream_id'] 698 | stream_name = module.params['stream_name'] 699 | rule_id = module.params['rule_id'] 700 | title = module.params['title'] 701 | field = module.params['field'] 702 | type = module.params['type'] 703 | value = module.params['value'] 704 | index_set_id = module.params['index_set_id'] 705 | inverted = module.params['inverted'] 706 | description = module.params['description'] 707 | remove_matches_from_default_stream = module.params['remove_matches_from_default_stream'] 708 | matching_type = module.params['matching_type'] 709 | rules = module.params['rules'] 710 | allow_http = module.params['allow_http'] 711 | 712 | if allow_http == True: 713 | endpoint = "http://" + endpoint 714 | else: 715 | endpoint = "https://" + endpoint 716 | 717 | base_url = endpoint + "/api/streams" 718 | 719 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 720 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 721 | "Authorization": "Basic ' + api_token.decode() + '" }' 722 | 723 | if action == "create": 724 | if index_set_id is None: 725 | index_set_id = default_index_set(module, endpoint, base_url, headers) 726 | status, message, content, url = create(module, base_url, headers, index_set_id) 727 | elif action == "create_rule": 728 | status, message, content, url = create_rule(module, base_url, headers) 729 | elif action == "update": 730 | status, message, content, url = update(module, base_url, headers, stream_id, title, description, remove_matches_from_default_stream, matching_type, rules, index_set_id) 731 | elif action == "update_rule": 732 | status, message, content, url = update_rule(module, base_url, headers, stream_id, rule_id, field, type, value, inverted, description) 733 | elif action == "delete": 734 | status, message, content, url = delete(module, base_url, headers, stream_id) 735 | elif action == "delete_rule": 736 | status, message, content, url = delete_rule(module, base_url, headers, stream_id, rule_id) 737 | elif action == "start": 738 | status, message, content, url = start(module, base_url, headers, stream_id) 739 | elif action == "pause": 740 | status, message, content, url = pause(module, base_url, headers, stream_id) 741 | elif action == "list": 742 | status, message, content, url = list(module, base_url, headers, stream_id) 743 | elif action == "query_streams": 744 | stream_id = query_streams(module, base_url, headers, stream_name) 745 | status, message, content, url = list(module, base_url, headers, stream_id) 746 | 747 | uresp = {} 748 | content = to_text(content, encoding='UTF-8') 749 | 750 | try: 751 | js = json.loads(content) 752 | except ValueError: 753 | js = "" 754 | 755 | uresp['json'] = js 756 | uresp['status'] = status 757 | uresp['msg'] = message 758 | uresp['url'] = url 759 | 760 | module.exit_json(**uresp) 761 | 762 | 763 | if __name__ == '__main__': 764 | main() 765 | -------------------------------------------------------------------------------- /library/graylog_users.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Copyright: (c) 2019, Whitney Champion 3 | # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) 4 | 5 | from __future__ import (absolute_import, division, print_function) 6 | __metaclass__ = type 7 | 8 | ANSIBLE_METADATA = {'metadata_version': '1.1', 9 | 'status': ['preview'], 10 | 'supported_by': 'community'} 11 | 12 | DOCUMENTATION = ''' 13 | module: graylog_users 14 | short_description: Communicate with the Graylog API to manage users 15 | description: 16 | - The Graylog user module manages Graylog users. 17 | version_added: "2.9" 18 | author: "Whitney Champion (@shortstack)" 19 | options: 20 | endpoint: 21 | description: 22 | - Graylog endpoint. (i.e. graylog.mydomain.com:9000). 23 | required: false 24 | type: str 25 | graylog_user: 26 | description: 27 | - Graylog privileged user username, used to auth with Graylog API. 28 | required: false 29 | type: str 30 | graylog_password: 31 | description: 32 | - Graylog privileged user password, used to auth with Graylog API. 33 | required: false 34 | type: str 35 | allow_http: 36 | description: 37 | - Allow non HTTPS connexion 38 | required: false 39 | default: false 40 | type: bool 41 | validate_certs: 42 | description: 43 | - Allow untrusted certificate 44 | required: false 45 | default: false 46 | type: bool 47 | action: 48 | description: 49 | - Action to take against user API. 50 | required: false 51 | default: list 52 | choices: [ create, update, delete, list ] 53 | type: str 54 | username: 55 | description: 56 | - Username. 57 | required: false 58 | type: str 59 | password: 60 | description: 61 | - Password. 62 | required: false 63 | type: str 64 | full_name: 65 | description: 66 | - Display name. 67 | required: false 68 | type: str 69 | email: 70 | description: 71 | - Email. 72 | required: false 73 | type: str 74 | roles: 75 | description: 76 | - List of role names to add the user to. 77 | required: false 78 | type: list 79 | permissions: 80 | description: 81 | - List of permission names to add the user to. 82 | required: false 83 | type: list 84 | timezone: 85 | description: 86 | - Timezone. 87 | required: false 88 | default: 'UTC' 89 | type: str 90 | ''' 91 | 92 | EXAMPLES = ''' 93 | # List users 94 | - graylog_users: 95 | endpoint: "graylog.mydomain.com" 96 | graylog_user: "username" 97 | graylog_password: "password" 98 | 99 | # Create user 100 | - graylog_users: 101 | action: create 102 | endpoint: "graylog.mydomain.com" 103 | graylog_user: "username" 104 | graylog_password: "password" 105 | username: "whitney" 106 | full_name: "Whitney" 107 | email: "whitney@unicorns.lol" 108 | password: "cookiesaredelicious" 109 | roles: 110 | - "analysts" 111 | permissions: 112 | - "*" 113 | 114 | # Create user with role(s) 115 | - graylog_users: 116 | action: create 117 | endpoint: "graylog.mydomain.com" 118 | graylog_user: "username" 119 | graylog_password: "password" 120 | username: "whitney" 121 | full_name: "Whitney" 122 | email: "whitney@unicorns.lol" 123 | password: "cookiesaredelicious" 124 | roles: 125 | - "analysts" 126 | 127 | # Create multiple users with admin roles 128 | - graylog_users: 129 | action: create 130 | endpoint: "graylog.mydomain.com" 131 | graylog_user: "username" 132 | graylog_password: "password" 133 | username: "{{ item.username }}" 134 | full_name: "{{ item.full_name }}" 135 | email: "{{ item.email }}" 136 | password: "{{ item.password }}" 137 | roles: 138 | - "admins" 139 | with_items: 140 | - { username: "alice", full_name: "Alice", email: "alice@aolcom", password: "ilovebob111" } 141 | - { username: "bob", full_name: "Bob", email: "bob@aolcom", password: "ilovealice111" } 142 | 143 | # Update user's email address 144 | - graylog_users: 145 | action: update 146 | endpoint: "graylog.mydomain.com" 147 | graylog_user: "username" 148 | graylog_password: "password" 149 | username: "whitney" 150 | email: "whitney@ihateunicorns.lol" 151 | 152 | # Delete user 153 | - graylog_users: 154 | action: delete 155 | endpoint: "graylog.mydomain.com" 156 | graylog_user: "username" 157 | graylog_password: "password" 158 | username: "whitney" 159 | ''' 160 | 161 | RETURN = ''' 162 | json: 163 | description: The JSON response from the Graylog API 164 | returned: always 165 | type: complex 166 | contains: 167 | username: 168 | description: Username. 169 | returned: success 170 | type: str 171 | sample: 'john' 172 | id: 173 | description: User ID. 174 | returned: success 175 | type: str 176 | sample: '4bc73d5108e33b4810f3eab0' 177 | email: 178 | description: User email. 179 | returned: success 180 | type: str 181 | sample: 'john@domain.com' 182 | full_name: 183 | description: Full name of the user. 184 | returned: success 185 | type: str 186 | sample: 'John Smith' 187 | last_activity: 188 | description: Last user activity. 189 | returned: success 190 | type: str 191 | sample: '2019-04-21T23:16:53.500+0000' 192 | external: 193 | description: Whether or not the user was created from an external authentication source (such as LDAP). 194 | returned: success 195 | type: bool 196 | sample: true 197 | session_active: 198 | description: Whether or not the user's session is active. 199 | returned: success 200 | type: bool 201 | sample: true 202 | session_timeout_ms: 203 | description: Session automatically ends after this amount of time. 204 | returned: success 205 | type: int 206 | sample: 3600000 207 | startpage: 208 | description: User's start page. 209 | returned: success 210 | type: dict 211 | sample: { "id": "5b05eea33f5e865e57babfae", "type": "dashboard" } 212 | preferences: 213 | description: User's preferences. 214 | returned: success 215 | type: dict 216 | sample: { "enableSmartSearch": true } 217 | timezone: 218 | description: User's timezone. 219 | returned: success 220 | type: str 221 | sample: 'America/Chicago' 222 | permissions: 223 | description: User permissions (dashboards, streams, collectors, etc). 224 | returned: success 225 | type: list 226 | sample: [ "dashboards:read:4c58eef77ec84145c3a2d9f3" ] 227 | roles: 228 | description: User roles. 229 | returned: success 230 | type: list 231 | sample: [ "analyst", "Administrator" ] 232 | read_only: 233 | description: Whether or not the user is a read-only user. 234 | returned: success 235 | type: bool 236 | sample: false 237 | msg: 238 | description: The HTTP message from the request 239 | returned: always 240 | type: str 241 | sample: OK (unknown bytes) 242 | status: 243 | description: The HTTP status code from the request 244 | returned: always 245 | type: int 246 | sample: 200 247 | url: 248 | description: The actual URL used for the request 249 | returned: always 250 | type: str 251 | sample: https://www.ansible.com/ 252 | ''' 253 | 254 | 255 | # import module snippets 256 | import json 257 | import base64 258 | from ansible.module_utils.basic import AnsibleModule 259 | from ansible.module_utils.urls import fetch_url, to_text 260 | 261 | 262 | def create(module, base_url, headers): 263 | 264 | url = base_url 265 | 266 | payload = {} 267 | 268 | for key in ['full_name', 'email', 'username', 'password', 'roles', 'permissions', 'timezone']: 269 | if module.params[key] is not None: 270 | payload[key] = module.params[key] 271 | 272 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 273 | 274 | if info['status'] != 201: 275 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 276 | 277 | try: 278 | content = to_text(response.read(), errors='surrogate_or_strict') 279 | except AttributeError: 280 | content = info.pop('body', '') 281 | 282 | return info['status'], info['msg'], content, url 283 | 284 | 285 | def update(module, base_url, headers): 286 | 287 | url = "/".join([base_url, module.params['username']]) 288 | 289 | payload = {} 290 | 291 | for key in ['full_name', 'email', 'username', 'password', 'roles', 'permissions', 'timezone']: 292 | if module.params[key] is not None: 293 | payload[key] = module.params[key] 294 | 295 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='PUT', data=module.jsonify(payload)) 296 | 297 | if info['status'] != 204: 298 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 299 | 300 | try: 301 | content = to_text(response.read(), errors='surrogate_or_strict') 302 | except AttributeError: 303 | content = info.pop('body', '') 304 | 305 | return info['status'], info['msg'], content, url 306 | 307 | 308 | def delete(module, base_url, headers): 309 | 310 | url = "/".join([base_url, module.params['username']]) 311 | 312 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='DELETE') 313 | 314 | if info['status'] != 204: 315 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 316 | 317 | try: 318 | content = to_text(response.read(), errors='surrogate_or_strict') 319 | except AttributeError: 320 | content = info.pop('body', '') 321 | 322 | return info['status'], info['msg'], content, url 323 | 324 | 325 | def list(module, base_url, headers): 326 | 327 | url = base_url 328 | 329 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='GET') 330 | 331 | if info['status'] != 200: 332 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 333 | 334 | try: 335 | content = to_text(response.read(), errors='surrogate_or_strict') 336 | except AttributeError: 337 | content = info.pop('body', '') 338 | 339 | return info['status'], info['msg'], content, url 340 | 341 | 342 | def get_token(module, endpoint, username, password): 343 | 344 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json" }' 345 | 346 | url = endpoint + "/api/system/sessions" 347 | 348 | payload = {} 349 | payload['username'] = username 350 | payload['password'] = password 351 | payload['host'] = endpoint 352 | 353 | response, info = fetch_url(module=module, url=url, headers=json.loads(headers), method='POST', data=module.jsonify(payload)) 354 | 355 | if info['status'] != 200: 356 | module.fail_json(msg="Fail: %s" % ("Status: " + str(info['msg']) + ", Message: " + str(info['body']))) 357 | 358 | try: 359 | content = to_text(response.read(), errors='surrogate_or_strict') 360 | session = json.loads(content) 361 | except AttributeError: 362 | content = info.pop('body', '') 363 | 364 | session_string = session['session_id'] + ":session" 365 | session_bytes = session_string.encode('utf-8') 366 | session_token = base64.b64encode(session_bytes) 367 | 368 | return session_token 369 | 370 | 371 | def main(): 372 | module = AnsibleModule( 373 | argument_spec=dict( 374 | endpoint=dict(type='str'), 375 | graylog_user=dict(type='str'), 376 | graylog_password=dict(type='str', no_log=True), 377 | allow_http=dict(type='bool', required=False, default=False), 378 | validate_certs=dict(type='bool', required=False, default=True), 379 | action=dict(type='str', required=False, default='list', choices=['create', 'update', 'delete', 'list']), 380 | username=dict(type='str'), 381 | password=dict(type='str', no_log=True), 382 | full_name=dict(type='str'), 383 | email=dict(type='str'), 384 | timezone=dict(type='str', default='UTC'), 385 | roles=dict(type='list'), 386 | permissions=dict(type='list', default=[]) 387 | ) 388 | ) 389 | 390 | endpoint = module.params['endpoint'] 391 | graylog_user = module.params['graylog_user'] 392 | graylog_password = module.params['graylog_password'] 393 | action = module.params['action'] 394 | allow_http = module.params['allow_http'] 395 | 396 | if allow_http == True: 397 | endpoint = "http://" + endpoint 398 | else: 399 | endpoint = "https://" + endpoint 400 | 401 | base_url = endpoint + "/api/users" 402 | 403 | api_token = get_token(module, endpoint, graylog_user, graylog_password) 404 | headers = '{ "Content-Type": "application/json", "X-Requested-By": "Graylog API", "Accept": "application/json", \ 405 | "Authorization": "Basic ' + api_token.decode() + '" }' 406 | 407 | if action == "create": 408 | status, message, content, url = create(module, base_url, headers) 409 | elif action == "update": 410 | status, message, content, url = update(module, base_url, headers) 411 | elif action == "delete": 412 | status, message, content, url = delete(module, base_url, headers) 413 | elif action == "list": 414 | status, message, content, url = list(module, base_url, headers) 415 | 416 | uresp = {} 417 | content = to_text(content, encoding='UTF-8') 418 | 419 | try: 420 | js = json.loads(content) 421 | except ValueError: 422 | js = "" 423 | 424 | uresp['json'] = js 425 | uresp['status'] = status 426 | uresp['msg'] = message 427 | uresp['url'] = url 428 | 429 | module.exit_json(**uresp) 430 | 431 | 432 | if __name__ == '__main__': 433 | main() 434 | -------------------------------------------------------------------------------- /main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Testing Graylog modules 3 | hosts: localhost 4 | tasks: 5 | 6 | - name: Create Graylog role 7 | graylog_roles: 8 | action: create 9 | endpoint: "{{ endpoint }}" 10 | graylog_user: "{{ graylog_user }}" 11 | graylog_password: "{{ graylog_password }}" 12 | name: "ansible_role" 13 | description: "Ansible test role" 14 | permissions: 15 | - "dashboards:read" 16 | read_only: "true" 17 | 18 | - name: Update Graylog role 19 | graylog_roles: 20 | action: update 21 | endpoint: "{{ endpoint }}" 22 | graylog_user: "{{ graylog_user }}" 23 | graylog_password: "{{ graylog_password }}" 24 | name: "ansible_role" 25 | description: "Ansible test role update" 26 | permissions: 27 | - "dashboards:read" 28 | read_only: "true" 29 | 30 | - name: List Graylog roles 31 | graylog_roles: 32 | action: list 33 | endpoint: "{{ endpoint }}" 34 | graylog_user: "{{ graylog_user }}" 35 | graylog_password: "{{ graylog_password }}" 36 | register: graylog_roles 37 | 38 | - name: Print roles 39 | debug: 40 | msg: "{{ graylog_roles }}" 41 | 42 | - name: Create Graylog user 43 | graylog_users: 44 | action: create 45 | endpoint: "{{ endpoint }}" 46 | graylog_user: "{{ graylog_user }}" 47 | graylog_password: "{{ graylog_password }}" 48 | username: "ansible_user" 49 | full_name: "Ansible User" 50 | password: "{{ password }}" 51 | email: "old-email@aol.com" 52 | roles: 53 | - "ansible_role" 54 | 55 | - name: Update Graylog user 56 | graylog_users: 57 | action: update 58 | endpoint: "{{ endpoint }}" 59 | graylog_user: "{{ graylog_user }}" 60 | graylog_password: "{{ graylog_password }}" 61 | username: "ansible_user" 62 | email: "new-email@aol.com" 63 | 64 | - name: List Graylog users 65 | graylog_users: 66 | action: list 67 | endpoint: "{{ endpoint }}" 68 | graylog_user: "{{ graylog_user }}" 69 | graylog_password: "{{ graylog_password }}" 70 | register: graylog_users 71 | 72 | - name: Print users 73 | debug: 74 | msg: "{{ graylog_users }}" 75 | 76 | - name: Remove user 77 | graylog_users: 78 | action: delete 79 | endpoint: "{{ endpoint }}" 80 | graylog_user: "{{ graylog_user }}" 81 | graylog_password: "{{ graylog_password }}" 82 | username: "ansible_user" 83 | 84 | - name: Remove role 85 | graylog_roles: 86 | action: delete 87 | endpoint: "{{ endpoint }}" 88 | graylog_user: "{{ graylog_user }}" 89 | graylog_password: "{{ graylog_password }}" 90 | name: "ansible_role" 91 | 92 | - name: List streams 93 | graylog_streams: 94 | action: list 95 | endpoint: "{{ endpoint }}" 96 | graylog_user: "{{ graylog_user }}" 97 | graylog_password: "{{ graylog_password }}" 98 | 99 | - name: Create stream 100 | graylog_streams: 101 | action: create 102 | endpoint: "{{ endpoint }}" 103 | graylog_user: "{{ graylog_user }}" 104 | graylog_password: "{{ graylog_password }}" 105 | title: "test_stream" 106 | description: "Windows and IIS logs" 107 | matching_type: "AND" 108 | remove_matches_from_default_stream: False 109 | rules: 110 | - {"field":"message","type":1,"value":"test_stream rule","inverted": false,"description":"test_stream rule"} 111 | 112 | - name: Get stream from stream name query 113 | graylog_streams: 114 | action: query_streams 115 | endpoint: "{{ endpoint }}" 116 | graylog_user: "{{ graylog_user }}" 117 | graylog_password: "{{ graylog_password }}" 118 | stream_name: "test_stream" 119 | register: stream 120 | 121 | - name: List single stream by ID 122 | graylog_streams: 123 | action: list 124 | endpoint: "{{ endpoint }}" 125 | graylog_user: "{{ graylog_user }}" 126 | graylog_password: "{{ graylog_password }}" 127 | stream_id: "{{ stream.json.id }}" 128 | 129 | - name: Update stream 130 | graylog_streams: 131 | action: update 132 | endpoint: "{{ endpoint }}" 133 | graylog_user: "{{ graylog_user }}" 134 | graylog_password: "{{ graylog_password }}" 135 | stream_id: "{{ stream.json.id }}" 136 | remove_matches_from_default_stream: True 137 | 138 | - name: Create stream rule 139 | graylog_streams: 140 | action: create_rule 141 | endpoint: "{{ endpoint }}" 142 | graylog_user: "{{ graylog_user }}" 143 | graylog_password: "{{ graylog_password }}" 144 | stream_id: "{{ stream.json.id }}" 145 | description: "Windows Security Logs" 146 | field: "winlogbeat_log_name" 147 | type: "1" 148 | value: "Security" 149 | inverted: False 150 | register: rule 151 | 152 | - name: Update stream rule 153 | graylog_streams: 154 | action: update_rule 155 | endpoint: "{{ endpoint }}" 156 | graylog_user: "{{ graylog_user }}" 157 | graylog_password: "{{ graylog_password }}" 158 | stream_id: "{{ stream.json.id }}" 159 | rule_id: "{{ rule.json.streamrule_id }}" 160 | description: "Windows Security and Application Logs" 161 | 162 | - name: Delete stream rule 163 | graylog_streams: 164 | action: delete_rule 165 | endpoint: "{{ endpoint }}" 166 | graylog_user: "{{ graylog_user }}" 167 | graylog_password: "{{ graylog_password }}" 168 | stream_id: "{{ stream.json.id }}" 169 | rule_id: "{{ rule.json.streamrule_id }}" 170 | 171 | - name: Validate pipeline rule 172 | graylog_pipelines: 173 | action: parse_rule 174 | endpoint: "{{ endpoint }}" 175 | graylog_user: "{{ graylog_user }}" 176 | graylog_password: "{{ graylog_password }}" 177 | source: | 178 | rule "test_rule_domain_threat_intel" 179 | when 180 | has_field("dns_query") 181 | then 182 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 183 | set_fields(dns_query_intel); 184 | end 185 | register: validate_rule 186 | 187 | - name: Create pipeline rule 188 | graylog_pipelines: 189 | action: create_rule 190 | endpoint: "{{ endpoint }}" 191 | graylog_user: "{{ graylog_user }}" 192 | graylog_password: "{{ graylog_password }}" 193 | title: "test_rule" 194 | description: "test" 195 | source: | 196 | rule "test_rule_domain_threat_intel" 197 | when 198 | has_field("dns_query") 199 | then 200 | let dns_query_intel = threat_intel_lookup_domain(to_string($message.dns_query), "dns_query"); 201 | set_fields(dns_query_intel); 202 | end 203 | register: pipeline_rule 204 | when: validate_rule.status == 200 205 | 206 | - name: Create pipeline 207 | graylog_pipelines: 208 | action: create 209 | endpoint: "{{ endpoint }}" 210 | graylog_user: "{{ graylog_user }}" 211 | graylog_password: "{{ graylog_password }}" 212 | title: "test_pipeline" 213 | source: | 214 | pipeline "test_pipeline" 215 | stage 0 match either 216 | end 217 | description: "test_pipeline description" 218 | 219 | - name: Get pipeline from pipeline name query_pipelines 220 | graylog_pipelines: 221 | action: query_pipelines 222 | endpoint: "{{ endpoint }}" 223 | graylog_user: "{{ graylog_user }}" 224 | graylog_password: "{{ graylog_password }}" 225 | pipeline_name: "test_pipeline" 226 | register: pipeline 227 | 228 | - name: Validate pipeline 229 | graylog_pipelines: 230 | action: parse_pipeline 231 | endpoint: "{{ endpoint }}" 232 | graylog_user: "{{ graylog_user }}" 233 | graylog_password: "{{ graylog_password }}" 234 | source: | 235 | pipeline "test_pipeline" 236 | stage 0 match either 237 | rule "test_rule_domain_threat_intel" 238 | end 239 | register: validate_pipeline 240 | 241 | - name: Update pipeline with rule 242 | graylog_pipelines: 243 | action: update 244 | endpoint: "{{ endpoint }}" 245 | graylog_user: "{{ graylog_user }}" 246 | graylog_password: "{{ graylog_password }}" 247 | pipeline_id: "{{ pipeline.json.id }}" 248 | source: | 249 | pipeline "test_pipeline" 250 | stage 0 match either 251 | rule "test_rule_domain_threat_intel" 252 | end 253 | when: validate_pipeline.status == 200 254 | 255 | - name: Create Stream connection to processing pipeline 256 | graylog_pipelines: 257 | action: create_connection 258 | endpoint: "{{ endpoint }}" 259 | graylog_user: "{{ graylog_user }}" 260 | graylog_password: "{{ graylog_password }}" 261 | pipeline_id: "{{ pipeline.json.id }}" 262 | stream_ids: 263 | - "{{ stream.json.id }}" 264 | 265 | - name: Delete pipeline 266 | graylog_pipelines: 267 | action: delete 268 | endpoint: "{{ endpoint }}" 269 | graylog_user: "{{ graylog_user }}" 270 | graylog_password: "{{ graylog_password }}" 271 | pipeline_id: "{{ pipeline.json.id }}" 272 | 273 | - name: Delete pipeline rule 274 | graylog_pipelines: 275 | action: delete_rule 276 | endpoint: "{{ endpoint }}" 277 | graylog_user: "{{ graylog_user }}" 278 | graylog_password: "{{ graylog_password }}" 279 | rule_id: "{{ pipeline_rule.json.id }}" 280 | 281 | - name: Get all index sets 282 | graylog_index_sets: 283 | action: list 284 | endpoint: "{{ endpoint }}" 285 | graylog_user: "{{ graylog_user }}" 286 | graylog_password: "{{ graylog_password }}" 287 | register: index_sets 288 | 289 | - name: Create index set 290 | graylog_index_sets: 291 | action: create 292 | endpoint: "{{ endpoint }}" 293 | graylog_user: "{{ graylog_user }}" 294 | graylog_password: "{{ graylog_password }}" 295 | title: "test_index_set" 296 | index_prefix: "test_index_" 297 | description: "test index set" 298 | 299 | - name: Get index set by name 300 | graylog_index_sets: 301 | action: query_index_sets 302 | endpoint: "{{ endpoint }}" 303 | graylog_user: "{{ graylog_user }}" 304 | graylog_password: "{{ graylog_password }}" 305 | title: "test_index_set" 306 | register: index_set 307 | 308 | - name: Update stream with index set 309 | graylog_streams: 310 | action: update 311 | endpoint: "{{ endpoint }}" 312 | graylog_user: "{{ graylog_user }}" 313 | graylog_password: "{{ graylog_password }}" 314 | stream_id: "{{ stream.json.id }}" 315 | index_set_id: "{{ index_set.json.id }}" 316 | 317 | - name: Delete stream 318 | graylog_streams: 319 | action: delete 320 | endpoint: "{{ endpoint }}" 321 | graylog_user: "{{ graylog_user }}" 322 | graylog_password: "{{ graylog_password }}" 323 | stream_id: "{{ stream.json.id }}" 324 | 325 | - name: Delete new index set 326 | graylog_index_sets: 327 | action: delete 328 | endpoint: "{{ endpoint }}" 329 | graylog_user: "{{ graylog_user }}" 330 | graylog_password: "{{ graylog_password }}" 331 | index_set_id: "{{ index_set.json.id }}" 332 | 333 | - name: Setup Active Directory authentication without SSL and set "Reader" as default role 334 | graylog_ldap: 335 | endpoint: "graylog.mydomain.com" 336 | graylog_user: "username" 337 | graylog_password: "password" 338 | enabled: "true" 339 | active_directory: "true" 340 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 341 | system_password_set: "true" 342 | system_username: "ldapbind@mydomain.com" 343 | system_password: "bindPassw0rd" 344 | search_base: "cn=users,dc=mydomain,dc=com" 345 | search_pattern: "(&(objectClass=user)(sAMAccountName={0}))" 346 | display_name_attribute: "displayName" 347 | group_search_base: "cn=groups,dc=mydomain,dc=com" 348 | group_search_pattern: "(&(objectClass=group)(cn=graylog*))" 349 | group_id_attribute: "cn" 350 | 351 | - name: Remove current LDAP authentication configuration 352 | graylog_ldap: 353 | endpoint: "graylog.mydomain.com" 354 | graylog_user: "username" 355 | graylog_password: "password" 356 | action: "delete" 357 | 358 | - name: Get current LDAP authentication configuration 359 | graylog_ldap: 360 | endpoint: "graylog.mydomain.com" 361 | graylog_user: "username" 362 | graylog_password: "password" 363 | action: "get" 364 | register: currentConfiguration 365 | 366 | - name: Print current LDAP authentication configuration 367 | debug: 368 | msg: "{{ currentConfiguration }}" 369 | 370 | - name: Test LDAP bind 371 | graylog_ldap: 372 | endpoint: "graylog.mydomain.com" 373 | graylog_user: "username" 374 | graylog_password: "password" 375 | action: "test" 376 | active_directory: "true" 377 | ldap_uri: "ldap://domaincontroller.mydomain.com:389" 378 | system_password_set: "true" 379 | system_username: "ldapbind@mydomain.com" 380 | system_password: "bindPassw0rd" 381 | 382 | - name: Get all LDAP groups 383 | graylog_ldap_groups: 384 | endpoint: "graylog.mydomain.com" 385 | graylog_user: "username" 386 | graylog_password: "password" 387 | action: "list" 388 | 389 | - name: Get the LDAP group to Graylog role mapping 390 | graylog_ldap_groups: 391 | endpoint: "graylog.mydomain.com" 392 | graylog_user: "username" 393 | graylog_password: "password" 394 | action: "list_mapping" 395 | 396 | - name: Update the LDAP group to Graylog role mapping 397 | graylog_ldap_groups: 398 | endpoint: "graylog.mydomain.com" 399 | graylog_user: "username" 400 | graylog_password: "password" 401 | action: "update" 402 | group: "{{ item.group }}" 403 | role: "{{ item.role }}" 404 | with_items: 405 | - { group : "ldap-group-admins", role : "Admin" } 406 | - { group : "ldap-group-read", role : "Reader" } 407 | --------------------------------------------------------------------------------